cloudfoundry / blackbox Goto Github PK

Blackbox assumes that all your logs are ".log". In some cases that's not true. Could you add a PR to pass a logprefix as a variable from the yaml.

example yaml:
syslog:
address: 10.244.9.5
transport: tcp
blackbox:
log_prefix: ap_log

We had a bosh deployment failing due to the blackbox process crashing with error

2018/11/14 19:13:27 Seeked /var/vcap/sys/log/kube-apiserver/audit-2018-11-09T19-23-44.754.log - &{Offset:0 Whence:2}
2018/11/14 19:13:27 lines flushed; exiting tailer
2018/11/14 19:13:27 FATAL -- failed to create Watcher
goroutine 4089 [running]:
runtime/debug.Stack(0xc420012ba0, 0x21, 0x0)
        /var/vcap/data/packages/golang/d2abe6ed3ac3b89eb235e73ed653ec3a635e3d9f/src/runtime/debug/stack.go:24 +0xa7
github.com/hpcloud/tail/util.Fatal(0x6498cf, 0x18, 0x0, 0x0, 0x0)
        /var/vcap/packages/blackbox/src/github.com/hpcloud/tail/util/util.go:22 +0xc7
github.com/hpcloud/tail/watch.(*InotifyTracker).run(0xc421c26600)
        /var/vcap/packages/blackbox/src/github.com/hpcloud/tail/watch/inotify_tracker.go:231 +0x4c8
created by github.com/hpcloud/tail/watch.glob..func1
        /var/vcap/packages/blackbox/src/github.com/hpcloud/tail/watch/inotify_tracker.go:54 +0x1dd

We found that this was due to a large number of files matching the tail pattern which exceeded the number of file handles available. We now have a solution for our release where we are not creating such a large number of files.

Just wanted to ask if the behaviour above is expected?

test

Current behaviour

All log files found below the configured source_dir: folder are passed to the syslog server.
The directory name below source_dir: is passed as the app-name in the syslog protocol.

The name of the log files is not passed to the syslog server!

Issue

Very often there are several different log files, with different semantics and sometimes even different format. Since the log file name is lost, it is hard the filter specific log files or use different log parsers.

Possible solution

One possible solution could be to pass the relative folder name, including the file name in the syslog app-name field.

For example:

Blackbox configuration:

syslog:
...
    source_dir: /var/vcap/sys/log/

Log files:

/var/vcap/sys/log/director/access.log
/var/vcap/sys/log/director/director.stdout.log
/var/vcap/sys/log/director/director.error.log
...

Passed app-name values

director/access.log
director/director.stdout.log
director/director.error.log
...

If the current behaviour should not change by default, the new behaviour could be configurable.

Should we create a PR?