Given log files that contain JSON records and are rotated with suffixes using file-rotate
:
- Assemble the contents of log files in chronological order
- Filter records based on
jq
-syntax query - Format remaining records for terminal viewing
This crate is made to work with the file-rotate
crate and slog-json
.
It assumes that slog records are written as JSON, and files rotated with file-rotate
.
Example log record:
{"msg":"Start poll worker","level":"DEBG","ts":"2022-01-31T15:20:20.637","tag":"poller","worker":"http://localhost:8002/","poller":20166}
So it has the keys "msg", "level", "ts", "tag"
. The rest of the keys are key-value pairs from slog (which can obviously also be used in your jq
filter).
We currently use a hard-coded suffix scheme:
let suffix_scheme = TimestampSuffixScheme::default(FileLimit::Age(Duration::weeks(1)));
This implies the default timestamp format which is %Y%m%dT%H%M%S
. (the FileLimit
above does not matter at all).
There is no reason this has to be hard-coded and can be parameterized in future work.
To build, you need to have libjq/jq installed, and set
export JQ_LIB_DIR=/usr/lib/libjq.so
The main work so far has went into the assemble
subcommand
$ assemble-logs assemble -h
assemble-logs-assemble
USAGE:
assemble-logs assemble [FLAGS] [OPTIONS] <LOG_PATH> [JQ]
ARGS:
<LOG_PATH> The path to the main log file
<JQ> JQ query; must return a bool; only used for filtering
FLAGS:
-c, --compact Compact - don't print newline on each key-value
-e, --error-details Print error details (default is without details)
-h, --help Print help information
-n, --no-format No formatting: just print json
-V, --version Print version information
OPTIONS:
-a, --after <AFTER>
Any prefix of a timestamp in the format "%Y%m%dT%H%M%S" The system will already filter
out files that have a lexically older stimestamp. This will also be used to filter
records, so you don't necessarily have to use the `jq` filter for that
--jq-transformation <JQ_TRANSFORMATION>
Only effective with --no-format
Show only logs within a 30 minutes timespan.
assemble-logs assemble logs/all.log '.ts > "2021-09-02T22" and .ts < "2021-09-02T22:30"' | less -r
Filter by (short) log level ("CRIT", "ERRO", "WARN", "INFO", "DEBG", "TRCE"):
'.level == "WARN" or level == "ERRO"'
Show only output with a specific tag
'.tag == "poller"'
Everything actix-related (e.g. all incoming requests):
'.tag | startswith("actix")'
The -a/--after
argument may seem superfluous since we can do the same with a filter on .ts
as seen above.
However, it is difficult to extract the .ts
part of the filter in order to filter out files (as opposed to records).
Much time can be saved by filtering out entire files so that they need not be decompressed and their contents processed.
-a
should thus be used instead of a jq filter when you want to filter out records before the given timestamp.
This will apply the same filter on individual records.
$ assemble-logs assemble logs/all.log -a 2022-01-31T12 | less -r
This outputs formatted records starting with:
Jan 31 12:00:00.151 DEBG [poller] Poll
poller: 8160
workers online: 1
Jan 31 12:00:00.151 DEBG [poller] Debug info
in-flight analysis job submissions: 0
poller: 8160
Future work could include a -b/--before
argument