Git Product home page Git Product logo

kibana's People

Contributors

dan70402 avatar lindt avatar rfarley3 avatar wleese avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar

Forkers

braoru catalinvr craig-cogdill wleese dan70402 aya lindt vuokkovuorinnen awake416 astraladi neil-orans shakexingwu lijielife meowtochondria

kibana's Issues

SSL hosts

Hello,

Good job on this. Is it possible for you to implement a way we can use SSL (basic auth) hosts?

Thanks

Usage problems (when installation is not used)

Readme says

"Use without installing: python -m kibana
For any example that follows, replace dotkibana with python -m kibana"

However I'm seeing the following errors if I try to do that:

python -m kibana --help
Traceback (most recent call last):
File "/Users/dubauski/anaconda/lib/python2.7/runpy.py", line 163, in _run_module_as_main
mod_name, _Error)
File "/Users/dubauski/anaconda/lib/python2.7/runpy.py", line 111, in _get_module_details
import(mod_name) # Do not catch exceptions initializing package
File "kibana/init.py", line 1, in
from .dotkibana import DotKibana # noqa
File "kibana/dotkibana.py", line 5, in
from .manager import KibanaManager
File "kibana/manager.py", line 4, in
from elasticsearch import Elasticsearch, RequestError
ImportError: No module named elasticsearch

Thank you for any help with this!

Unable to run the module

Hello
I unable to start the module.
I have installed Python 2.7.11

Do you think that I need to downgrade the python version?

root@ip-10-13-166-157:/tmp/Kibana# python setup.py install
running install
running bdist_egg
running egg_info
creating kibana.egg-info
writing requirements to kibana.egg-info/requires.txt
writing kibana.egg-info/PKG-INFO
writing top-level names to kibana.egg-info/top_level.txt
writing dependency_links to kibana.egg-info/dependency_links.txt
writing entry points to kibana.egg-info/entry_points.txt
writing manifest file 'kibana.egg-info/SOURCES.txt'
reading manifest file 'kibana.egg-info/SOURCES.txt'
writing manifest file 'kibana.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/kibana
copying kibana/__main__.py -> build/lib/kibana
copying kibana/dotkibana.py -> build/lib/kibana
copying kibana/mapping.py -> build/lib/kibana
copying kibana/__init__.py -> build/lib/kibana
copying kibana/manager.py -> build/lib/kibana
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/kibana
copying build/lib/kibana/__main__.py -> build/bdist.linux-x86_64/egg/kibana
copying build/lib/kibana/dotkibana.py -> build/bdist.linux-x86_64/egg/kibana
copying build/lib/kibana/mapping.py -> build/bdist.linux-x86_64/egg/kibana
copying build/lib/kibana/__init__.py -> build/bdist.linux-x86_64/egg/kibana
copying build/lib/kibana/manager.py -> build/bdist.linux-x86_64/egg/kibana
byte-compiling build/bdist.linux-x86_64/egg/kibana/__main__.py to __main__.pyc
byte-compiling build/bdist.linux-x86_64/egg/kibana/dotkibana.py to dotkibana.pyc
byte-compiling build/bdist.linux-x86_64/egg/kibana/mapping.py to mapping.pyc
byte-compiling build/bdist.linux-x86_64/egg/kibana/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/kibana/manager.py to manager.pyc
  File "build/bdist.linux-x86_64/egg/kibana/manager.py", line 107
    print e.error
          ^
SyntaxError: invalid syntax

creating build/bdist.linux-x86_64/egg/EGG-INFO
copying kibana.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying kibana.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying kibana.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying kibana.egg-info/entry_points.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying kibana.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying kibana.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/kibana-0.1-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing kibana-0.1-py2.7.egg
Copying kibana-0.1-py2.7.egg to /usr/local/lib/python2.7/site-packages
Adding kibana 0.1 to easy-install.pth file
Installing dotkibana script to /usr/local/bin

Installed /usr/local/lib/python2.7/site-packages/kibana-0.1-py2.7.egg
Processing dependencies for kibana==0.1
Searching for requests
Reading https://pypi.python.org/simple/requests/
Best match: requests 2.9.1
Downloading https://pypi.python.org/packages/source/r/requests/requests-2.9.1.tar.gz#md5=0b7f480d19012ec52bab78292efd976d
Processing requests-2.9.1.tar.gz
Writing /tmp/easy_install-gfc97i/requests-2.9.1/setup.cfg
Running requests-2.9.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-gfc97i/requests-2.9.1/egg-dist-tmp-KzO04g
creating /usr/local/lib/python2.7/site-packages/requests-2.9.1-py2.7.egg
Extracting requests-2.9.1-py2.7.egg to /usr/local/lib/python2.7/site-packages
Adding requests 2.9.1 to easy-install.pth file

Installed /usr/local/lib/python2.7/site-packages/requests-2.9.1-py2.7.egg
Searching for argparse
Reading https://pypi.python.org/simple/argparse/
Best match: argparse 1.4.0
Downloading https://pypi.python.org/packages/source/a/argparse/argparse-1.4.0.tar.gz#md5=08062d2ceb6596fcbc5a7e725b53746f
Processing argparse-1.4.0.tar.gz
Writing /tmp/easy_install-4_d9kj/argparse-1.4.0/setup.cfg
Running argparse-1.4.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-4_d9kj/argparse-1.4.0/egg-dist-tmp-RQl1dv
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '*.pyo' found anywhere in distribution
warning: no previously-included files matching '*.orig' found anywhere in distribution
warning: no previously-included files matching '*.rej' found anywhere in distribution
no previously-included directories found matching 'doc/_build'
no previously-included directories found matching 'env24'
no previously-included directories found matching 'env25'
no previously-included directories found matching 'env26'
no previously-included directories found matching 'env27'
zip_safe flag not set; analyzing archive contents...
Moving argparse-1.4.0-py2.7.egg to /usr/local/lib/python2.7/site-packages
Adding argparse 1.4.0 to easy-install.pth file

Installed /usr/local/lib/python2.7/site-packages/argparse-1.4.0-py2.7.egg
Searching for elasticsearch
Reading https://pypi.python.org/simple/elasticsearch/
Best match: elasticsearch 2.3.0
Downloading https://pypi.python.org/packages/source/e/elasticsearch/elasticsearch-2.3.0.tar.gz#md5=2550f3b51629cf1ef9636608af92c340
Processing elasticsearch-2.3.0.tar.gz
Writing /tmp/easy_install-eUtR8U/elasticsearch-2.3.0/setup.cfg
Running elasticsearch-2.3.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-eUtR8U/elasticsearch-2.3.0/egg-dist-tmp-HwLbWG
warning: no files found matching 'elasticsearch/connection/esthrift/Rest-remote'
no previously-included directories found matching 'docs/_build'
no previously-included directories found matching 'test_elasticsearch'
warning: no previously-included files matching '__pycache__' found under directory '*'
warning: no previously-included files matching '*.py[co]' found under directory '*'
zip_safe flag not set; analyzing archive contents...
Moving elasticsearch-2.3.0-py2.7.egg to /usr/local/lib/python2.7/site-packages
Adding elasticsearch 2.3.0 to easy-install.pth file

Installed /usr/local/lib/python2.7/site-packages/elasticsearch-2.3.0-py2.7.egg
Searching for urllib3<2.0,>=1.8
Reading https://pypi.python.org/simple/urllib3/
Best match: urllib3 1.14
Downloading https://pypi.python.org/packages/source/u/urllib3/urllib3-1.14.tar.gz#md5=5e1407428ac33b521c71a7ac273b3847
Processing urllib3-1.14.tar.gz
Writing /tmp/easy_install-d_g9UC/urllib3-1.14/setup.cfg
Running urllib3-1.14/setup.py -q bdist_egg --dist-dir /tmp/easy_install-d_g9UC/urllib3-1.14/egg-dist-tmp-tHnS2F
warning: no previously-included files matching '*' found under directory 'docs/_build'
zip_safe flag not set; analyzing archive contents...
Moving urllib3-1.14-py2.7.egg to /usr/local/lib/python2.7/site-packages
Adding urllib3 1.14 to easy-install.pth file

Installed /usr/local/lib/python2.7/site-packages/urllib3-1.14-py2.7.egg
Finished processing dependencies for kibana==0.1

root@ip-10-13-166-157:/tmp/Kibana# python -m kibana --help
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/runpy.py", line 151, in _run_module_as_main
    mod_name, loader, code, fname = _get_module_details(mod_name)
  File "/usr/local/lib/python2.7/runpy.py", line 109, in _get_module_details
    return _get_module_details(pkg_main_name)
  File "/usr/local/lib/python2.7/runpy.py", line 101, in _get_module_details
    loader = get_loader(mod_name)
  File "/usr/local/lib/python2.7/pkgutil.py", line 464, in get_loader
    return find_loader(fullname)
  File "/usr/local/lib/python2.7/pkgutil.py", line 474, in find_loader
    for importer in iter_importers(fullname):
  File "/usr/local/lib/python2.7/pkgutil.py", line 430, in iter_importers
    __import__(pkg)
  File "kibana/__init__.py", line 1, in <module>
    from .dotkibana import DotKibana  # noqa
  File "kibana/dotkibana.py", line 5, in <module>
    from .manager import KibanaManager
  File "kibana/manager.py", line 107
    print e.error
          ^
SyntaxError: invalid syntax
root@ip-10-13-166-157:/tmp/Kibana#

Thank you :)

Debug is enabled by default

Any thoughts about setting the output message level from the CLI? I can put together a quick PR if that helps.

Zero or negative time interval not supported

After refreshing the mappings for my logstash-* index, I get the following error in Kibana: 'Zero or negative time interval not supported'. This because after refreshing the mapping, the timeFieldName has been removed from the index-pattern by this tool:

before:

{
  "_index" : ".kibana",
  "_type" : "index-pattern",
  "_id" : "logstash-*",
  "_version" : 4,
  "found" : true,
  "_source" : {
    "fields" : " << FIELDS REMOVED BECAUSE TOO LONG >>"
    "customFormats" : "{}",
    "timeFieldName" : "@timestamp",
    "title" : "logstash-*"
  }
}

after:

{
  "_index" : ".kibana",
  "_type" : "index-pattern",
  "_id" : "logstash-*",
  "_version" : 4,
  "found" : true,
  "_source" : {
    "fields" : " << FIELDS REMOVED BECAUSE TOO LONG >>"
    "customFormats" : "{}",
    "title" : "logstash-*"
  }
}

If this timeFieldName param is present, it should be retained somehow. Wrote a quick workaround for mapping.py, but this will only work for instances where the timefieldname is @timestamp:

    def field_cache_to_index_pattern(self, field_cache):
        """Return a .kibana index-pattern doc_type"""
        mapping_dict = {}
        mapping_dict['customFormats'] = "{}"
        mapping_dict['title'] = self.index_pattern
        mapping_dict['timeFieldName'] = '@timestamp'
        # now post the data into .kibana
        mapping_dict['fields'] = json.dumps(field_cache, separators=(',', ':'))
        # in order to post, we need to create the post string
        mapping_str = json.dumps(mapping_dict, separators=(',', ':'))
        return mapping_str

Elastic search transport

Folks in manager to connect to elastic search, we should have an option to supply a transport class which help to solve the custom authentication.

I can pick this if you want.

in mapping, type having "float" as value would be also convert to "number"

context

when I generate a new index pattern there is 2 attributes

geoip.longitude and geoip.latitude wich are "float"
kibana want type="number" and format="string"

ref : https://www.elastic.co/guide/en/kibana/current/managing-fields.html#_geographic_point_field_formatters

my poor solution

So, even if I never code in python, I found in mapping.py
https://github.com/rfarley3/Kibana/blob/master/kibana/mapping.py#L257

    if key in self.mappings:
                if (key == 'type' and
                    (val == "long" or
                     val == "integer" or
                     val == "double")):
                    val = "number"

I've change it

 if key in self.mappings:
                if (key == 'type' and
                    (val == "long" or
                     val == "float" or
                     val == "integer" or
                     val == "double")):
                    val = "number"

and it's works fine

(even if format still empty and not "sprint" as request)

Question of beginner :
Why export config, viz, search and board but not index-pattern ?
index-pattern is needed to a correct import.

Mapping Mapping is incomplete, doing update

Hi,

We are using ES for indexing cloudtrail data and it hitting 11 million events in a day. While refreshing the index pattern we are getting an error like "[INF] Mapping Mapping is incomplete, doing update". Could you please let us know how can we fix this issue?

Some saved searches are missed

When exporting dashboards, saved searches that were only referenced as a visualization dependency (panel[id]['_source']['savedSearchId']) were exported.

Verify that json output is always recursively sorted

This may already be done, but verify with some tests.

Goal is to facilitate git revisioning of dashboards. Also include sorting of the json.dumps data within some of the object types.

json.dumps(d, sort_keys=True)

Export do not include document of .kibana index where _type:index-pattern

As proposed in #9 , I've check dotkibana --export all --pkg but entries where _type:index-pattern is not in the exported json.

I see data I expected to found in export with this query
GET /.kibana/_search?q=_type:index-pattern

When I export the kibana config, I can't import them in a new kibana because search need to found the index-pattern

So I need to manually create/define the index-pattern.

My need is to prepare the board in my local vagrant, and then share it via export.
Other developers need to import the config into their local installation.
It must also be able to import new board / visually / search in Kibana production and staging.

Export Filename cannot be choosen

export kibana settings is great. Importing even greater. So great that I want to use your tool in a containerized, automatic way.
But I do not care about the filename. I do not want the timestamp within the export name, just a simple name. Timestamping/diffing is later done by a VC.

Unable to refresh mapping

Hello,

Thank you for solving the install issue. My environment uses elasticsearch 2.2 and kibana 4.4.
I want to use your module to update automatically the "logstash-*' index pattern.

root@ip-10-123-208-126:/tmp/Kibana# python -m kibana --refresh 'logstash-*'
Mimicking Kibana GUI refreshFields
Error, couldn't find subkey doc_type[_type]['mapping'][_type]
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/tmp/Kibana/kibana/__main__.py", line 98, in <module>
    sys.exit(main())
  File "/tmp/Kibana/kibana/__main__.py", line 89, in main
    return handle_mapping(dotk, map_cmd)
  File "/tmp/Kibana/kibana/__main__.py", line 13, in handle_mapping
    return dotk.do_mapping_refresh()
  File "kibana/dotkibana.py", line 36, in do_mapping_refresh
    return self.mapping.do_refresh()
  File "kibana/mapping.py", line 245, in do_refresh
    es_cache = self.get_field_cache('es')
  File "kibana/mapping.py", line 89, in get_field_cache
    self.get_index_mappings(es_mappings[index_name]['mappings'])
TypeError: 'NoneType' object is not iterable
root@ip-10-123-208-126:/tmp/Kibana# python -m kibana --help

Thank you,
Catalin

Similar project

I started the following project a couple of months ago: https://github.com/jpparis-orange/kibtool. Although I searched github before starting my project, I stumbled upon rfarley3/Kibana right now.

As both projects seem to share common goals, would it be a good way to "merge" the efforts?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.