parse-nl / systemdjournal2gelf Goto Github PK

Export entries from systemds' journal and send them to a graylog server using gelf

License: BSD 2-Clause "Simplified" License

Go 100.00%

systemdjournal2gelf's Introduction

SystemdJournal2Gelf

Export entries from systemd's journal and send them to a Graylog server using gelf. This script is written in Google go to make it easier to compile and distribute to your machines.

Dependencies:

this repo includes https://github.com/DECK36/go-gelf
Google golang

Install / Compile

Compile this package by checking out the repo and run:

go get github.com/parse-nl/SystemdJournal2Gelf

The binary will be compiled in $GOPATH/bin/SystemdJournal2Gelf

Or install the package for:

Archlinux

Running as a service

Copy the included SystemdJournal2Gelf.service to /etc/systemd/system.

Usage:

SystemdJournal2Gelf will connect to the server you specify as first argument and passes all other arguments to journalctl. It prepends these arguments with --output=json

Export only the kernel messages

SystemdJournal2Gelf localhost:11201 _TRANSPORT=kernel

Perform initial import, reading entire journal

SystemdJournal2Gelf localhost:11201 --merge

Monitor the journal

SystemdJournal2Gelf localhost:11201 --follow

Logging additional properties:

Letting this script decode json encoded properties in Messages has been removed, please see this issue that explains how to configure graylog to do that instead

License

Released under the Simplified BSD license, see LICENSE for details.

systemdjournal2gelf's People

Contributors

Stargazers

Watchers

Forkers

travelping nailgun towolf plexinc jeremytrimble sjonhortensius atlsapi buzzy3 silicht gnanderson

systemdjournal2gelf's Issues

remove JSON parsing as graylog now supports it fully

remove json parsing from SystemdJournal2Gelf - let users configure this themselves as a pipeline. Steps to recreate this as a custom pipeline:

Go to /system/pipelines in your graylog install and create a new pipeline
Edit the new pipeline and add a stage:
In the first stage, add a rule to interpret the message as json:

rule "interpret message as JSON"
when
    has_field("message") && starts_with(to_string($message.message), "{\"")
then
    let json = parse_json(to_string($message.message));
    let map = to_map(json);
    set_fields(map);

    rename_field("Message", "message");
    rename_field("FullMessage", "full_message");
end

you can prefix additional stages to cleanup your message eg. when receiving messages from php-fpm:

rule "strip fpm pool prefix"
when
  has_field("message") && starts_with(to_string($message.message), "pool ")
then
  set_field("message", regex_replace("^pool [a-zA-Z_\\[\\d\\]]+: ", to_string($message.message), ""));
end

attach the pipeline to the appropriate stream(s). Make sure the Pipeline Processor is configured after Message Filter Chain in system/configurations

Panic Error on merging

I get the following error on a merge:

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x402d76]

goroutine 1 [running]:
panic(0x602900, 0xc820010130)
        /usr/lib/go/src/runtime/panic.go:481 +0x3e6
main.main()

--help should not panic & other argument-passing interrogations

$ go version
go version go1.13.8 linux/amd64

$ cd ./go/bin/ && ./SystemdJournal2Gelf --help 
panic: usage: SystemdJournal2Gelf SERVER:12201 [JOURNALCTL PARAMETERS]

goroutine 1 [running]:
main.main()
	~/go/src/github.com/parse-nl/SystemdJournal2Gelf/SystemdJournal2Gelf.go:164 +0x64d


$ ./SystemdJournal2Gelf localhost:12201 -- --version
Failed to add match '--version': Invalid argument
# ^^ I would expect it to be passed to journald...


$ ./SystemdJournal2Gelf localhost:12201 --version
panic: could not parse journal output: invalid character 's' looking for beginning of value
# ^^ Why?

This Go-Binary currently doesn't support sending the logs via TCP.
For a more secure Graylog deployment sending logs via TCP with TLS is recommended.
Is there a way to implement the option of using TCP as transport instead of UDP?

Graylog additional header

OVH hosted Graylog instance (https://www.ovh.com/fr/data-platforms/logs/) works on authentication. The TCP payload must include a specific string X-OVH-TOKEN: authentication (could be a header too).

README.md reads This script supports a special syntax to send additional properties; when you log a JSON encoded object in the Message field it Unmarshalls it for you but I don't see how to make a use of this to solve above issue.

Add Info for AUR package

FYI, I created an Arch Linux package for your software:
https://aur.archlinux.org/packages/systemdjournal2gelf/

It has a configuration file to set the Gelf-Endpoint-Url and installs the systemd service from your repo.

Maybe you would like to add it to your readme for your users :)

Thanks for your good-working piece of software :)))

remove messageReplace/process() in favor of graylogs extractors

the message cleanup feature was introduced when graylog didn't support any similar functionality. It provides message extractors now - so this client doesn't necessarily need to keep providing this

issues with reading systemd v237 on Ubuntu Bionic

Trying this for the first time:

1

$ ~/go/bin/SystemdJournal2Gelf 127.0.0.1:12201 --merge
panic: could not parse journal output: json: cannot unmarshal array into Go struct field entryAlias.SYSLOG_FACILITY of type string

goroutine 1 [running]:
main.main()
	/home/arno/go/src/github.com/parse-nl/SystemdJournal2Gelf/SystemdJournal2Gelf.go:207 +0x5b3

2

$ ~/go/bin/SystemdJournal2Gelf 127.0.0.1:12201 _TRANSPORT=kernel
panic: runtime error: slice bounds out of range

goroutine 1 [running]:
main.(*SystemdJournalEntry).isJsonMessage(...)
	/home/arno/go/src/github.com/parse-nl/SystemdJournal2Gelf/SystemdJournal2Gelf.go:128
main.(*SystemdJournalEntry).toGelf(0xc4200e86e0, 0x7f2110535d90)
	/home/arno/go/src/github.com/parse-nl/SystemdJournal2Gelf/SystemdJournal2Gelf.go:57 +0x78c
main.(*SystemdJournalEntry).send(0xc4200e86e0)
	/home/arno/go/src/github.com/parse-nl/SystemdJournal2Gelf/SystemdJournal2Gelf.go:115 +0x2f
main.(*pendingEntry).Push(0xc42009e160, 0xc4200a4000, 0x7b, 0x571c27d944a13, 0xc4200a60d0, 0xb, 0xc4200c2000, 0x20, 0xc4200a6008, 0x6, ...)
	/home/arno/go/src/github.com/parse-nl/SystemdJournal2Gelf/SystemdJournal2Gelf.go:140 +0xc9
main.main()
	/home/arno/go/src/github.com/parse-nl/SystemdJournal2Gelf/SystemdJournal2Gelf.go:210 +0x396

It looks like that only the --follow works well:

$ ~/go/bin/SystemdJournal2Gelf 127.0.0.1:12201 --follow
$ echo 'test message to journald' | systemd-cat

Probably people are configuring journald write to files and feed them to filebeats, since filebeats isn't reading the systemd journald directly yet elastic/beats#7955

json parser fails when journalctl outputs number array on unprintable inputs

message can be an array of numbers [ 116, 104, 105, 115, 32, 105, 115, 32, 97, 32, 98, 105, 110, 97, 114, 121, 32, 118, 97, 108, 117, 101, 32, 7 ], - we could use a custom json parser to handle this

Error from Scanner: bufio.Scanner: token too long: Issue when merging

Recieved another issue when merging:

Error from Scanner: bufio.Scanner: token too long