Git Product home page Git Product logo

docker-filebeat's Introduction

Docker Filebeat image

Filebeat is a lightweight, open source shipper for log file data. As the next-generation Logstash Forwarder, Filebeat tails logs and quickly sends this information to Logstash for further parsing and enrichment or to Elasticsearch for centralized storage and analysis.

Supported tags and respective Dockerfile links

Build and update process

This image is automatically built at every push of this repository and every time that the debian:jessie base image gets updated in order to ensure that bugfixes and security updates are immediately applied.

Run

docker run -v /path/to/filebeat.yml:/filebeat.yml prima/filebeat:6

Or, you can create your own derived image, with the configuration in the image itself.

FROM prima/filebeat
COPY my-config/filebeat.yml /filebeat.yml

Configuration file

You can find an example of the filebeat.yml in the official filebeat repository: https://github.com/elastic/beats/blob/master/filebeat/filebeat.yml

And in the official docs: https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-configuration.html

docker-filebeat's People

Contributors

andreausu avatar leedm777 avatar marcoh00 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

docker-filebeat's Issues

Module not found

When I start your container I get :
modules.go:95: ERR Not loading modules. Module directory not found: /module

Any ideas ?

What is an example use case of this image?

I am hoping to get some clarification on best use case for this image.

When I start the filebeat container, my yaml file says to look at the syslogs and ship them to logstash. In order to get the logs from my other containers, I have them logging to the host syslog, and then I mount the host syslog file to the filebeat container so it can have access to all the logs.

Here's an image to hopefully do a better job of showing what my setup is like ๐Ÿ™ƒ
elk-setup

This works when I first start the filebeat container, but after several hours it stops shipping the logs to logstash. I'm trying to debug the issue but filebeat itself isn't outputting any error messages. It appears that the mounted syslog file is simply no longer getting any logs from the host machine. This is leading me to believe that I may not be using the image correctly.

Could you please provide an example use case? Possibly add it to the README.

filebeat.yml permissions

I'm seeing the following when running this with my own filebeat.yml file:
Exiting: error loading config file: config file ("filebeat.yml") must be owned by the beat user (uid=0) or root

This works fine when I'm running docker for mac, but I get the error on linux.

Error setup

After run:
docker run -v /path/to/filebeat.yml:/filebeat.yml prima/filebeat:1

I got this:

Loading config file error: Failed to read /filebeat.yml: read /filebeat.yml: is a directory. Exiting.

Docker volumed data/registry not working as intended

In my Docker run, I created a Docker volume from the data folder:
-v filebeatData:/data

When restarting, filebeat detects the data/registry file. I get this in the console:

2017/02/27 10:39:59.052887 beat.go:207: INFO filebeat start running. 2017/02/27 10:39:59.054510 registrar.go:85: INFO Registry file set to: /data/registry 2017/02/27 10:39:59.055889 registrar.go:106: INFO Loading registrar data from /data/registry 2017/02/27 10:39:59.057644 registrar.go:123: INFO States Loaded from registrar: 2 2017/02/27 10:39:59.058042 crawler.go:34: INFO Loading Prospectors: 1 2017/02/27 10:39:59.058362 prospector_log.go:57: INFO Prospector with previous states loaded: 0

However, during the restart, filebeat still sends all the logs previously sent to my ELK container:
screenshot 2017-02-28 10 25 08

This is my filebeat.yml:

image

This does not happen when I run filebeat on the host. Filebeat will only send the logs that it has not sent since the restart. Has anyone faced the same issue?

Error loading template /filebeat.template.json

Following your run instruction,

docker@default:~$ docker run -v /<host>/filebeat.yml:/filebeat.yml prima/filebeat
2017/02/20 19:20:11.737857 beat.go:267: INFO Home path: [/] Config path: [/] Data path: [//data] Logs path: [//logs]
2017/02/20 19:20:11.738035 beat.go:177: INFO Setup Beat: filebeat; Version: 5.2.1
2017/02/20 19:20:11.738143 output.go:167: INFO Loading template enabled. Reading template file: /filebeat.template.json
2017/02/20 19:20:11.738197 outputs.go:100: ERR failed to initialize elasticsearch plugin as output: Error loading template /filebeat.template.json: open /filebeat.template.json: no suc
h file or directory
2017/02/20 19:20:11.738227 beat.go:288: CRIT Exiting: error initializing publisher: Error loading template /filebeat.template.json: open /filebeat.template.json: no such file or direct
ory
Exiting: error initializing publisher: Error loading template /filebeat.template.json: open /filebeat.template.json: no such file or directory
docker@default:~$

single.go:140: ERR Connecting error publishing events (retrying): dial tcp ****:5044: i/o timeout

Hello,primait:
I use your docker image prima/filebeat:5.0.1 to work with sebp/elk .Everything is well untill yesterday the filebeat had the error : " single.go:140: ERR Connecting error publishing events (retrying): dial tcp elk server ip : 5044: i/o timeout" .And this error I have traced for long time , noting is effective . Maybe you can give me more information to investigate this error .
Thank you very much ~
0mr_mun xf ei7q 1pu 5

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.