EFK stack
A sample environment running an EFK stack on your local machine.
Includes:
Introduction
As software systems grow and become more and more decoupled, log aggregation is a key aspect to take care of.
The issues to tackle down with logging are:
- Having a centralized overview of all log events
- Normalizing different log types
- Automated processing of log messages
- Supporting several and very different event sources
While Elasticsearch and Kibana are the reference products de facto for log searching and visualization in the open source community, there's no such agreement for log collectors.
The two most-popular data collectors are:
- Logstash, most known for being part of the ELK Stack
- Fluentd, used by communities of users of software such as Docker and GCP
Logging systems using Fluentd as collector are usually referenced as EFK stack.
Aim of this repository is to run an EFK stack on your local machine using docker-compose.
I'm not personally involved with companies supporting Logstash nor Fluentd.
If you need help to choose between Logstash and Fluent, take a look to the reference.
Launching the EFK stack
Requirements
On your machine, make sure you have installed:
Run
docker-compose upPlease note: in this example Fluentd will run on port 8080 instead of the default 24224.
This settings has been changed to show how to configure Fluentd to listen on a different port.
Kibana is exposed on port 5601.
Testing with sample data
If you are running macOS and you want to send sample data to test the EFK stack, you'll need RESTed.
Files are available in the examples folder.
Please note that RESTed is not strictly necessary as any other REST client application will work fine.
If you have curl client installed on your system, you can generate sample data by running:
curl -X POST -d 'json={"action":"login","userId":"5b07fbbb4e6b8"}' http://localhost:8080/myapp.log
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent