Git Product home page Git Product logo

docker-logstash's Introduction

Logstash Dockerfile

This is a highly configurable logstash (1.4.2) image running Elasticsearch (1.1.1) and Kibana (3.0.1).

How to use this image

To run the image, you have to first decide which services you want to run in your container:

Full ELK stack (default)

$ docker run -d \
  -p 9292:9292 \
  -p 9200:9200 \
  pblittle/docker-logstash

Logstash agent only

$ docker run \
  <your_run_options> \
  pblittle/docker-logstash \
  agent

Logstash config test only

$ docker run \
  <your_run_options> \
  pblittle/docker-logstash \
  configtest

Embedded Kibana web interface only

$ docker run \
  <your_run_options> \
  pblittle/docker-logstash \
  web

Logstash configuration

There are currently two supported ways of including your Logstash config files in your container:

  • Download your hosted config files from the Internet
  • Mount a volume on the host machine containing your config files

Regardless of the download method, your config files must be one of the following two file types:

  • A monolithic config file. The file extension must be .conf.
  • A tarball containing your config files. The file extension must be either .tar, .tar.gz, or .tgz.

Any files in /opt/logstash/conf.d with the .conf extension in the container will get loaded by logstash.

Download your config files from the Internet

To use your own hosted config files, set the LOGSTASH_CONFIG_URL environment variable to your logstash config URL using the -e flag as follows:

$ docker run -d \
  -e LOGSTASH_CONFIG_URL=<your_logstash_config_url> \
  -p 9292:9292 \
  -p 9200:9200 \
  pblittle/docker-logstash

By default, if LOGSTASH_CONFIG_URL isn't defined, an example logstash.conf will be downloaded and used in your container.

$ docker run -d \
  -p 9292:9292 \
  -p 9200:9200 \
  pblittle/docker-logstash

The default logstash.conf only listens on stdin and file inputs. If you wish to configure tcp and/or udp input, use your own logstash configuration files and expose the ports yourself. See logstash documentation for config syntax and more information.

Mount a volume containing your config files

To use config files from the local file system, mount the config directory as a volume using the -v flag. For example:

$ docker run -d \
  -v <your_logstash_config_dir>:/opt/logstash/conf.d \
  -p 9292:9292 \
  -p 9200:9200 \
  pblittle/docker-logstash

Elasticsearch server integration

If you plan on using Elasticsearch, the following three integration methods are supported:

  • A linked container running Elasticsearch
  • An external Elasticsearch server
  • The embedded Elasticsearch server

Linked container running Elasticsearch

If you want to link to a container running Elasticsearch, simply use the --link flag to connect to the container:

$ docker run -d \
  --link <your_es_container_name>:es \
  -p 9292:9292 \
  pblittle/docker-logstash

To have the linked Elasticsearch container's bind_host and port automatically detected, you will need to set the bind_host and port to ES_HOST and ES_PORT respectively in your elasticsearch output config. For example:

output {
  elasticsearch {
    bind_host => "ES_HOST"
    port => "ES_PORT"
    protocol => "http"
  }
}

If you are linking to an Elasticsearch container running on 172.0.4.20:9200, the config above will be transformed into:

output {
  elasticsearch {
    host => "172.0.4.20"
    port => "9200"
    protocol => "http"
  }
}

External Elasticsearch server

If you are using an external Elasticsearch server, simply set the ES_HOST and ES_PORT environment variables in your run command:

$ docker run -d \
  -e ES_HOST=<your_es_service_host> \
  -e ES_PORT=<your_es_service_port> \
  -p 9292:9292 \
  pblittle/docker-logstash

Embedded Elasticsearch server

The embedded Elasticsearch server will be used by default if you don't provide either of the configuration options above.

Please note, the embedded Elasticsearch server was not designed for use in Production.

Optional, build and run the image from source

If you prefer to build from source rather than use the pblittle/docker-logstash trusted build published to the public Docker Registry, execute the following:

$ git clone https://github.com/pblittle/docker-logstash.git
$ cd docker-logstash

If you are using Vagrant, you can build and run the container in a VM by executing:

$ vagrant up
$ vagrant ssh
$ cd /vagrant/1.4

From there, build and run a container using the newly created virtual machine:

$ make

Finally, verify the installation

You can now verify the logstash installation by visiting the sample Kibana dashboard:

http://<your_container_ip>:9292/index.html#/dashboard/file/default.json

Thank you

A huge thank you to the project Contributors and users. I really appreciate the support.

Contributing

  1. Fork it
  2. Checkout the develop branch (git checkout -b develop)
  3. Create your feature branch (git checkout -b my-new-feature)
  4. Commit your changes (git commit -am 'Add some feature')
  5. Push to the branch (git push origin my-new-feature)
  6. Create new Pull Request

License

This application is distributed under the Apache License, Version 2.0.

docker-logstash's People

Contributors

pblittle avatar hamiltont avatar lorenz avatar pavelvanecek avatar aidan- avatar ianks avatar

Watchers

Michael avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.