Also .lumberjack only seem to keep the last file write not all of them.

$logger foobar
$cat .lumberjack
{"/var/log/messages":{"source":"/var/log/messages","offset":11919,"inode":541247,"device":2049}}

No more /tmp/foo

from logstash-forwarder.

I'm seeing the same behavior where the .lumberjack file is replaced after each event and only contains data of the last written file.

However, it seems that it would want to keep the state of each file for restarts. That is not the intended behavior is it?

from logstash-forwarder.

It should keep the history of all files. It's a bug if it does what you described. I'll work on a fix ;)

from logstash-forwarder.

It appears to be a bug for me:

cat /etc/fedora-release
Fedora release 17 (Beefy Miracle)

uname -a
Linux im1 3.3.7-1.fc17.x86_64 #1 SMP Mon May 21 22:32:19 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

cat /opt/lumberjack/etc/lumberjack.cfg
{
"network": {
"servers": [ "localhost:12345" ],
"ssl certificate": "/opt/logstash/ssl/lumberjack.crt",
"ssl key": "/opt/logstash/ssl/lumberjack.key",
"ssl ca": "/opt/logstash/ssl/lumberjack.crt",
"timeout": 15
},
"files": [
{
"paths": ["/var/log/messages", "/var/log/*.log"],
"fields": {"syslog": "true"}
},
{
"paths": [ "-" ],
"fields": {"stdin": "true"}
},
{
"paths": ["/var/log/jboss/application.log"],
"fields": {"log4j": "true"}
}
]
}

cat .lumberjack
{"/var/log/messages":{"source":"/var/log/messages","offset":138896,"inode":263575,"device":64512}}

stat .lumberjack
File: `.lumberjack'
Size: 99 Blocks: 8 IO Block: 4096 regular file
Device: fc00h/64512d Inode: 794640 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2013-08-31 17:05:45.871000383 +0000
Modify: 2013-08-31 17:04:32.602000130 +0000
Change: 2013-08-31 17:04:32.602000130 +0000
Birth: -

logger test123

{"/var/log/messages":{"source":"/var/log/messages","offset":138930,"inode":263575,"device":64512}}

stat .lumberjack
File: `.lumberjack'
Size: 99 Blocks: 8 IO Block: 4096 regular file
Device: fc00h/64512d Inode: 789006 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2013-08-31 17:06:24.529999965 +0000
Modify: 2013-08-31 17:06:15.103000109 +0000
Change: 2013-08-31 17:06:15.103000109 +0000
Birth: -

echo "some message" >> /var/log/jboss/application.log

cat .lumberjack
{"/var/log/jboss/application.log":{"source":"/var/log/jboss/application.log","offset":2906212,"inode":262501,"device":64512}}

stat .lumberjack
File: `.lumberjack'
Size: 126 Blocks: 8 IO Block: 4096 regular file
Device: fc00h/64512d Inode: 794640 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2013-08-31 17:07:03.817999915 +0000
Modify: 2013-08-31 17:07:02.600000083 +0000
Change: 2013-08-31 17:07:02.600000083 +0000
Birth: -

from logstash-forwarder.

I think there are two bugs here.
#1, that lumberjack is not tracking "all" the files.
#2, even with a single the, it does not start with the offset is at. Does mot seem to read .lumberjack on start. See below, the stats and also the offset jumps by two log entries.

#3 Also may a third buy, it does not create an .lumberjack on start for a "new" file.

#4 Lastly it took two entries before lumber send any messages, and the first one was lost.

rm .lumberjack
start lumberjack (/opt/lumberjack/bin/lumberjack -config /etc/lumberjack/local.conf)
2013/09/09 22:08:47 publisher init
2013/09/09 22:08:47 {
"network": {
"servers": [ "localhost:8514" ],
"ssl ca": "/etc/logstash/ssl/lumberjackCA.pem"
},
"files": [
{
"paths": [ "/tmp/local.log" ],
"fields": { "type": "test" }
}
]
}

2013/09/09 22:08:47.242122 Setting trusted CA from file: /etc/logstash/ssl/lumberjackCA.pem
2013/09/09 22:08:47.242612 Connecting to localhost:8514
2013/09/09 22:08:47.305040 Connected to localhost:8514

~ # cat .lumberjack
cat: .lumberjack: No such file or directory

~ # echo '{ "somekey":"somevalue", "@message":"tester1"}' >> /tmp/local.log

log:
2013/09/09 22:10:37.243931 Launching harvester on new file: /tmp/local.log
2013/09/09 22:10:37.243963 Starting harvester: /tmp/local.log

~ # cat .lumberjack (MAYBE BUG #3)
cat: .lumberjack: No such file or directory

~ # cat /tmp/local.log
{ "somekey":"somevalue", "@message":"tester1"}

~ # echo '{ "somekey":"somevalue", "@message":"tester2"}' >> /tmp/local.log

log:
2013/09/09 22:12:47.248647 Registrar received 1 events
2013/09/09 22:12:47.248677 Saving registrar state.

~ # cat .lumberjack
{"/tmp/local.log":{"source":"/tmp/local.log","offset":94,"inode":7248,"device":51713}}

~ # cat /tmp/local.log
{ "somekey":"somevalue", "@message":"tester1"}
{ "somekey":"somevalue", "@message":"tester2"}

stop lumber jack ^C
~ # stat .lumberjack
File: `.lumberjack'
Size: 87 Blocks: 8 IO Block: 4096 regular file
Device: ca01h/51713d Inode: 15291 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2013-09-09 22:12:47.240921412 +0000
Modify: 2013-09-09 22:12:47.244921413 +0000
Change: 2013-09-09 22:12:47.244921413 +0000

~ # echo '{ "somekey":"somevalue", "@message":"tester3"}' >> /tmp/local.log

~ # cat /tmp/local.log
{ "somekey":"somevalue", "@message":"tester1"}
{ "somekey":"somevalue", "@message":"tester2"}
{ "somekey":"somevalue", "@message":"tester3"}

start lumberjack (~ # /opt/lumberjack/bin/lumberjack -config /etc/lumberjack/local.conf)
2013/09/09 22:15:30 publisher init
2013/09/09 22:15:30 {
"network": {
"servers": [ "localhost:8514" ],
"ssl ca": "/etc/logstash/ssl/lumberjackCA.pem"
},
"files": [
{
"paths": [ "/tmp/local.log" ],
"fields": { "type": "test" }
}
]
}

2013/09/09 22:15:30.793070 Loading registrar data
2013/09/09 22:15:30.793224 Launching harvester on new file: /tmp/local.log
2013/09/09 22:15:30.793305 Setting trusted CA from file: /etc/logstash/ssl/lumberjackCA.pem
2013/09/09 22:15:30.793808 Connecting to localhost:8514
2013/09/09 22:15:30.793877 Starting harvester: /tmp/local.log
2013/09/09 22:15:30.856955 Connected to localhost:8514

~ # stat .lumberjack
File: `.lumberjack'
Size: 87 Blocks: 8 IO Block: 4096 regular file
Device: ca01h/51713d Inode: 15291 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2013-09-09 22:12:47.240921412 +0000
Modify: 2013-09-09 22:12:47.244921413 +0000
Change: 2013-09-09 22:12:47.244921413 +0000

~ # cat .lumberjack
{"/tmp/local.log":{"source":"/tmp/local.log","offset":94,"inode":7248,"device":51713}}

~ # echo '{ "somekey":"somevalue", "@message":"tester4"}' >> /tmp/local.log
2013/09/09 22:18:25.800894 Registrar received 1 events
2013/09/09 22:18:25.800925 Saving registrar state.

cat .lumberjack

{"/tmp/local.log":{"source":"/tmp/local.log","offset":188,"inode":7248,"device":51713}}

~ # cat /tmp/local.log
{ "somekey":"somevalue", "@message":"tester1"}
{ "somekey":"somevalue", "@message":"tester2"}
{ "somekey":"somevalue", "@message":"tester3"}
{ "somekey":"somevalue", "@message":"tester4"}

from logstash-forwarder.

It does not seem to down the f h.Offset > 0 path, dont see the log message "Starting harvester at position.... "

func (h *Harvester) Harvest(output chan *FileEvent) {
if h.Offset > 0 {
log.Printf("Starting harvester at position %d: %s\n", h.Offset, h.Path)
} else {
log.Printf("Starting harvester: %s\n", h.Path)
}

from logstash-forwarder.

The pull request #84 only addressed the restart issue. The issue that dusty documented was not addressed.

That issue is that on each register .lumberjack.new is created (with just that one file registry) its then renamed to .lumberjack over writing the other registered watched file.

register.go:34

if len(state) > 0 {
  write(state)
  os.Rename(".lumberjack.new", ".lumberjack")
}

from logstash-forwarder.

Is this still an issue?

from logstash-forwarder.

I think this was resolved with the last release.

from logstash-forwarder.