Kibana is an excellent tool for visualising and performing analysis on your log data but I've found that the Discover pane isn't that effective when you are trying to pull contextual information about a particular event in a log (what happened before and after a particular error?) or in a number of other situations where commandline utilities like grep have traditionally been useful.
catl is my attempt to address this problem. It's a simple program that executes a search query against an elasticsearch cluster and returns the message field in timestamp order on stdout. This allows you to use utilities like grep, awk and sed to parse your logs for information as if it was a file on your local disk.
$ go build
$ catl -h
usage: catl [<flags>] <query>
Writes a logstash elasticsearch query to stdout as if it were a logfile
Flags:
-h, --help Show context-sensitive help (also try --help-long and --help-man).
-i, --index="logstash-*" Index pattern.
-u, --url="http://localhost:9200"
Logstash server URL.
-m, --message-field="message" Field to be returned
-s, --sort-field="@timestamp" Field to sort the results by
Args:
<query> Elasticseach query string.
You can set any of the flags with environment variables in the form CATL_$FLAG:
export CATL_INDEX=logstash-2018.07.02
export CATL_MESSAGE_FIELD=logmessageReturns ERROR level logs along with the preceding message:
$ catl 'meta.service: "foobar" AND meta.environment: "live" AND type: "java_logback"' | grep 'ERROR' -B 1
2018-07-03 00:00:00,522 [http-nio-8080-exec-7] DEBUG c.b.b.f.c.SillyClass - Doing something...
2018-07-03 00:00:00,522 [http-nio-8080-exec-4] ERROR c.b.b.f.SillyErrorClass - Something foolish happened
Save the output to a file:
$ catl 'meta.service: "foobar" AND meta.environment: "live" AND type: "java_logback" AND loglevel: "ERROR"' > errors.log
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent