kelseyhightower / journal-2-logentries Goto Github PK

Ship systemd journal entries to logentries.com

Go 100.00%

journal-2-logentries's Introduction

journal-2-logentries

Ship systemd journal entires to logentries.com over SSL.

Usage

sudo LOGENTRIES_TOKEN=<token> journal-2-logentries

sudo docker run -d -e 'LOGENTRIES_TOKEN=<token>' -v /run/journald.sock:/run/journald.sock \
quay.io/kelseyhightower/journal-2-logentries

Configuration

All configuration is done through env vars.

LOGENTRIES_JOURNAL_SOCKET - The systemd-journal-gatewayd socket. /run/journald.sock
LOGENTRIES_URL - The log entry url. api.logentries.com:20000
LOGENTRIES_TOKEN - The logentries.com TCP token -- See https://logentries.com/doc/input-token

Note: Make sure that systemd-journal-gatewayd is actually listening on /run/journald.sock. This is not done by default on CoreOS -- See example cloud-config

Building

GO_ENABLED=0 GOOS=linux go build -a -tags netgo -ldflags '-w' .

Docker

docker build -t quay.io/<username>/journal-2-logentries .
docker push quay.io/<username>/journal-2-logentries

Fleet integration

etcdctl set /logentries.com/token <token>

Edit journal-2-logentries.service

[Unit]
Description=Forward Systemd Journal to logentries.com

[Service]
TimeoutStartSec=0
ExecStartPre=-/usr/bin/docker kill journal-2-logentries
ExecStartPre=-/usr/bin/docker rm journal-2-logentries
ExecStartPre=/usr/bin/docker pull quay.io/kelseyhightower/journal-2-logentries
ExecStart=/usr/bin/bash -c \
"/usr/bin/docker run --name journal-2-logentries \
-v /run/journald.sock:/run/journald.sock \
-e LOGENTRIES_TOKEN=`etcdctl get /logentries.com/token` \
quay.io/kelseyhightower/journal-2-logentries"

[X-Fleet]
Global=true

fleetctl start journal-2-logentries.service

journal-2-logentries's People

Contributors

Stargazers

Watchers

Forkers

epipho rosskukulinski wiredthing akirill0v amos6224 watermarkchurch lukaf bronislav stephenhynes7 pebble octoblu arthur-c f-wong ibuystuff ukbg coreos linearregression sirspudd

journal-2-logentries's Issues

Allow for filtering by service

It would be nice not to have to ship the whole journal. Service whitelist or blacklist filtering could increase the utility of this library.

Add loggly adapter

I'd like to take a crack at adding a loggly adapter.

licence?

I'd like to reuse part of your code to forward journald to AWS CloudWatch Logs. How is the project licensed?

journald.sock: Connection Refused

Regardless if I run this through fleet or docker directly I get the following error:

Get http://journal/entries?follow: dial unix /run/journald.sock: connection refused

Thoughts?

Is there a way to avoid resending all events after restarting the container?

On container restart all event in journal are resend to logentries. Maybe this could be prevented by using the range header.

wish: document how it fails

It would be helpful if the documentation what happened if LogEntries can't be reached over the network for some reason.

Are the logs queued in memory in this case until it's available? If so, is there an upperbound on the queue size, or would the queue size grow until memory is exhausted and the system crashes?

Add docs on configuring and enabling systemd-journal-gatewayd

systemd-journal-gatewayd is not started by default on CoreOS and the default config has it listening on 19531 instead of a unix domain socket.

wish: document log format set over

Normally syslog-to-LogEntries integration doesn't include metadata, notably the "unit" name is omitted.

It would be helpful of a few sample lines of logging were provided as they would be sent to LogEntries, and notes included if the format is configurable at all.

Thanks.

It seems that the embedded certificates in https://github.com/kelseyhightower/journal-2-logentries/blob/master/logentries/certs.go expired yesterday, and our running journal-2-logentries containers are crashing with the error x509: certificate signed by unknown authority.

I'm forking the repo and disabling SSL certificate validation for now, but this should probably be updated in the code too. I've contacted Logentries about this as well, as the page referenced in certs.go have not been updated and still contains expired certificates. I can make a pull request with some changes here once I receive something - I'm creating a ticket for visibility right now, or if someone else has a more imminent fix for it.