Comments (29)
zstyblik
commented on August 17, 2024
1
We've tested it and it seems to be working. Thanks.
from dockbeat.
marminthibaut
commented on August 17, 2024
Thanks for reporting, i'm on it!
from dockbeat.
marminthibaut
commented on August 17, 2024
Problem identified: this is when docker API is not reachable, we get a nil object.
To reproduce: launch dockerbeat watching a docker daemon and then stop the docker daemon. It crashed dockerbeat.
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x4625bc]
goroutine 99 [running]:
github.com/ingensi/dockerbeat/beat.(*EventGenerator).getContainerEvent(0xc82009a8c8, 0xc820286900, 0x0, 0xc820285e60)
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/beat/EventGenerator.go:19 +0xa5c
github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats.func2(0xc8203c8e40, 0xc82009a880, 0xc820286900)
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:163 +0x8c
created by github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:172 +0x1f8
goroutine 1 [select]:
github.com/ingensi/dockerbeat/beat.(*Dockerbeat).Run(0xc82009a880, 0xc82001c600, 0x0, 0x0)
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:90 +0x368
github.com/elastic/beats/libbeat/beat.(*Beat).Run(0xc82001c600)
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/beat/beat.go:178 +0x31c
github.com/elastic/beats/libbeat/beat.Run(0x95fe10, 0xa, 0x956db0, 0xb, 0x7fd540a0cc28, 0xc82009a880, 0x2)
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/beat/beat.go:89 +0x1a1
main.main()
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/main.go:14 +0xa0
goroutine 17 [syscall, locked to thread]:
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1696 +0x1
goroutine 5 [syscall]:
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
/usr/local/go/src/os/signal/signal_unix.go:28 +0x37
goroutine 18 [select]:
github.com/elastic/beats/libbeat/publisher.(*messageWorker).run(0xc82012a160)
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/publisher/worker.go:59 +0x285
created by github.com/elastic/beats/libbeat/publisher.(*messageWorker).init
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/publisher/worker.go:53 +0xdf
goroutine 19 [select]:
github.com/elastic/beats/libbeat/publisher.(*bulkWorker).run(0xc820138240)
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/publisher/bulk.go:50 +0x503
created by github.com/elastic/beats/libbeat/publisher.newBulkWorker
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/publisher/bulk.go:38 +0x1c0
goroutine 20 [select]:
github.com/elastic/beats/libbeat/publisher.(*messageWorker).run(0xc8201380c0)
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/publisher/worker.go:59 +0x285
created by github.com/elastic/beats/libbeat/publisher.(*messageWorker).init
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/publisher/worker.go:53 +0xdf
goroutine 21 [select]:
github.com/elastic/beats/libbeat/publisher.(*messageWorker).run(0xc8201264b0)
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/publisher/worker.go:59 +0x285
created by github.com/elastic/beats/libbeat/publisher.(*messageWorker).init
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/publisher/worker.go:53 +0xdf
goroutine 34 [select, locked to thread]:
runtime.gopark(0xa37508, 0xc8201e6728, 0x952e48, 0x6, 0x18, 0x2)
/usr/local/go/src/runtime/proc.go:185 +0x163
runtime.selectgoImpl(0xc8201e6728, 0x0, 0x18)
/usr/local/go/src/runtime/select.go:392 +0xa64
runtime.selectgo(0xc8201e6728)
/usr/local/go/src/runtime/select.go:212 +0x12
runtime.ensureSigM.func1()
/usr/local/go/src/runtime/signal1_unix.go:227 +0x353
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1696 +0x1
goroutine 35 [chan receive]:
github.com/elastic/beats/libbeat/service.HandleSignals.func1(0xc8201de000, 0xc8201c4320, 0xc8201cc560)
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/service/service.go:29 +0x3a
created by github.com/elastic/beats/libbeat/service.HandleSignals
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/service/service.go:32 +0x1f8
goroutine 101 [runnable]:
github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc8201b1300, 0x40, 0xc8203c8e40, 0xc8203c8e00, 0xc8203c8ea0, 0xffffffffffffffff, 0xc820283a78, 0xc8203c8fc0)
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:798 +0xe3
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:804 +0x3a7
goroutine 79 [select]:
github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc8201b1240, 0x40, 0xc8203c8d20, 0xc8203c8d00, 0xc8203c8d80, 0xffffffffffffffff, 0xc820030fd8, 0xc8202e9b00)
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:798 +0xe3
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:804 +0x3a7
goroutine 96 [semacquire]:
sync.runtime_Syncsemacquire(0xc820100a00)
/usr/local/go/src/runtime/sema.go:237 +0x201
sync.(*Cond).Wait(0xc8201009f0)
/usr/local/go/src/sync/cond.go:62 +0x9b
io.(*pipe).read(0xc8201009c0, 0xc820366800, 0x200, 0x200, 0x0, 0x0, 0x0)
/usr/local/go/src/io/pipe.go:52 +0x2d2
io.(*PipeReader).Read(0xc820030fd8, 0xc820366800, 0x200, 0x200, 0x0, 0x0, 0x0)
/usr/local/go/src/io/pipe.go:134 +0x50
encoding/json.(*Decoder).refill(0xc8203ce1a0, 0x0, 0x0)
/usr/local/go/src/encoding/json/stream.go:152 +0x287
encoding/json.(*Decoder).readValue(0xc8203ce1a0, 0x1, 0x0, 0x0)
/usr/local/go/src/encoding/json/stream.go:128 +0x41b
encoding/json.(*Decoder).Decode(0xc8203ce1a0, 0x7c5c60, 0xc820352b00, 0x0, 0x0)
/usr/local/go/src/encoding/json/stream.go:57 +0x159
github.com/fsouza/go-dockerclient.(*Client).Stats(0xc820122240, 0xc8201b1240, 0x40, 0xc8203c8d20, 0xc8203a6200, 0xc8203c8d80, 0xffffffffffffffff, 0x0, 0x0)
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:808 +0x497
github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats.func1(0xc8203c8de0, 0xc82009a880, 0xc8201b1240, 0x40, 0xc8203c8d20, 0x0, 0xc8203c8d80, 0xffffffffffffffff)
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:155 +0x70
created by github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:157 +0x1bf
goroutine 97 [chan receive]:
github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats.func2(0xc8203c8d20, 0xc82009a880, 0xc820286870)
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:160 +0x4f
created by github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:172 +0x1f8
goroutine 78 [IO wait]:
net.runtime_pollWait(0x7fd540a0d578, 0x72, 0xc82000e160)
/usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc8201b73a0, 0x72, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc8201b73a0, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc8201b7340, 0xc8203b7000, 0x1000, 0x1000, 0x0, 0x7fd540a08028, 0xc82000e160)
/usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc82012e780, 0xc8203b7000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:172 +0xe4
bufio.(*Reader).fill(0xc820311ce0)
/usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).ReadSlice(0xc820311ce0, 0xc81fff6e0a, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:328 +0x21a
bufio.(*Reader).ReadLine(0xc820311ce0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:357 +0x53
net/textproto.(*Reader).readLineSlice(0xc82004fa30, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:55 +0x81
net/textproto.(*Reader).ReadLine(0xc82004fa30, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:36 +0x40
net/http.ReadResponse(0xc820311ce0, 0xc8201bab60, 0xc82012e780, 0x0, 0x0)
/usr/local/go/src/net/http/response.go:128 +0xf6
github.com/fsouza/go-dockerclient.(*Client).stream(0xc820122240, 0x94c048, 0x3, 0xc820311c20, 0x5f, 0x10100, 0x0, 0x0, 0x0, 0x7fd53e7b2528, ...)
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/client.go:489 +0x85f
github.com/fsouza/go-dockerclient.(*Client).Stats.func2(0xc820122240, 0xc8201b1240, 0x40, 0xc8203c8d20, 0xc8203c8d00, 0xc8203c8d80, 0xffffffffffffffff, 0xc820030fe0, 0xc8202e9aa0)
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:773 +0x26d
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/home/tmarmin/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:792 +0x2e5
from dockbeat.
marminthibaut
commented on August 17, 2024
@pcallewaert all should be ok now. Can you try and give us a feedback?
Thankx
from dockbeat.
pcallewaert
commented on August 17, 2024
It seems the problem is not resolved (tested with latest develop docker container)
dockerbeat-fail.zip
docker info
Containers: 35
Images: 360
Server Version: 1.9.1
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 430
Dirperm1 Supported: true
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.19.0-39-generic
Operating System: Ubuntu 14.04.3 LTS
CPUs: 8
Total Memory: 31.24 GiB
Name: dep-tolling-app00
ID: Q3HS:EQI7:4ZVR:MTJN:ERA6:Z5ZB:SRAG:7NKJ:E6GL:76JA:WW6I:75LB
WARNING: No swap limit support
from dockbeat.
marminthibaut
commented on August 17, 2024
I did not succeed in reproducing this bug.. After 3 hours of dockerbeat running on my personal host (some containers and images) -> no pb. I should try to on a similar environment than yours.
@erwanncloarec any idea?
from dockbeat.
gpolaert
commented on August 17, 2024
@marminthibaut what is your go version ? I'll build a new environment ; try to reproduce the issue ...
@pcallewaert How events/s have you? Just to be sure, what is your dockerbeat id image.
from dockbeat.
erwanncloarec
commented on August 17, 2024
Same as you, running for more than 2 hours with docker image, no problem
@pcallewaert do you use the default config?
from dockbeat.
erwanncloarec
commented on August 17, 2024
The way I see, the error is from the error comes from getContainerEvent at that line of code:
"@timestamp": common.Time(stats.Read),
Obviously it is not the common.Time that is doing the null pointer, so it is stats that is null.
The stats is comming from the chan populated by the docker client library, so we have to invastigate in what case we have a null in the stats.
from dockbeat.
pcallewaert
commented on August 17, 2024
@erwanncloarec just using this command:
docker run -d -v /var/run/docker.sock:/var/run/docker.sock --link elasticsearch-001:elasticsearch ingensi/dockerbeat:develop
@gpolaert imageid is d13cb6762b06c57e9a5d39bb80b2efe23bb4d2124d524de4c3ccbcf78a996040
We do use https://github.com/garethr/garethr-docker for container deployment. We're not sure, but we think it can be related when we deploy/update containers in a puppet run. However, at that point, the docker daemon is never restarted (this i'm pretty sure) so the docker api should be available at all time.
It's now running for 19 hours without problems, but we are not really deploying new stuff on it now.
from dockbeat.
marminthibaut
commented on August 17, 2024
Can you try to reproduce this behaviour with the beta2 version to ensure that we are working on the same version?
Docker image: ingensi/dockerbeat:1.0.0-beta2
Tag on github: https://github.com/Ingensi/dockerbeat/tree/v1.0.0-beta2
Thanks
from dockbeat.
pcallewaert
commented on August 17, 2024
Damn, after 43 hours running on beta 2, it crashed also.. :/
docker run -d --name dockerbeat -v /var/run/docker.sock:/var/run/docker.sock --link elasticsearch-001:elasticsearch ingensi/dockerbeat:1.0.0-beta2
dockerbeat-fail.zip
from dockbeat.
buehler
commented on August 17, 2024
Hey there
I may experience the same error. After some time the following log appears and the container dies:
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/client.go:489 +0x85f
github.com/fsouza/go-dockerclient.(*Client).Stats.func2(0xc82001b830, 0xc820011480, 0x40, 0xc820208e40, 0xc820208e00, 0xc820208ea0, 0xffffffffffffffff, 0xc820035140, 0xc82039ede0)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:773 +0x26d
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:792 +0x2e5
goroutine 325 [select]:
github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc820011480, 0x40, 0xc820208e40, 0xc820208e00, 0xc820208ea0, 0xffffffffffffffff, 0xc820035138, 0xc82039ee40)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:798 +0xe3
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:804 +0x3a7
goroutine 326 [IO wait]:
net.runtime_pollWait(0x7f8d371cc358, 0x72, 0xc82001c090)
/usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc820106a70, 0x72, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc820106a70, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc820106a10, 0xc820560000, 0x1000, 0x1000, 0x0, 0x7f8d371c5028, 0xc82001c090)
/usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc8201de100, 0xc820560000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:172 +0xe4
bufio.(*Reader).fill(0xc8205174a0)
/usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).ReadSlice(0xc8205174a0, 0xc81fff1e0a, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:328 +0x21a
bufio.(*Reader).ReadLine(0xc8205174a0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:357 +0x53
net/textproto.(*Reader).readLineSlice(0xc8202ada30, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:55 +0x81
net/textproto.(*Reader).ReadLine(0xc8202ada30, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:36 +0x40
net/http.ReadResponse(0xc8205174a0, 0xc8200aa7e0, 0xc8201de100, 0x0, 0x0)
/usr/local/go/src/net/http/response.go:128 +0xf6
github.com/fsouza/go-dockerclient.(*Client).stream(0xc82001b830, 0x94c1a8, 0x3, 0xc8205173e0, 0x5f, 0x10100, 0x0, 0x0, 0x0, 0x7f8d371cb4c8, ...)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/client.go:489 +0x85f
github.com/fsouza/go-dockerclient.(*Client).Stats.func2(0xc82001b830, 0xc820011d00, 0x40, 0xc820209ce0, 0xc820209d00, 0xc820209d40, 0xffffffffffffffff, 0xc820035150, 0xc82039eea0)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:773 +0x26d
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:792 +0x2e5
goroutine 327 [select]:
github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc820011d00, 0x40, 0xc820209ce0, 0xc820209c00, 0xc820209d40, 0xffffffffffffffff, 0xc820035148, 0xc82039ef00)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:798 +0xe3
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:804 +0x3a7
goroutine 328 [IO wait]:
net.runtime_pollWait(0x7f8d371ca4b0, 0x72, 0xc82001c090)
/usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc820018ca0, 0x72, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc820018ca0, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc820018c40, 0xc820562000, 0x1000, 0x1000, 0x0, 0x7f8d371c5028, 0xc82001c090)
/usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc820035190, 0xc820562000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:172 +0xe4
bufio.(*Reader).fill(0xc82039f620)
/usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).ReadSlice(0xc82039f620, 0xc81ffff20a, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:328 +0x21a
bufio.(*Reader).ReadLine(0xc82039f620, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:357 +0x53
net/textproto.(*Reader).readLineSlice(0xc820201a30, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:55 +0x81
net/textproto.(*Reader).ReadLine(0xc820201a30, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:36 +0x40
net/http.ReadResponse(0xc82039f620, 0xc8201c07e0, 0xc820035190, 0x0, 0x0)
/usr/local/go/src/net/http/response.go:128 +0xf6
github.com/fsouza/go-dockerclient.(*Client).stream(0xc82001b830, 0x94c1a8, 0x3, 0xc82039f560, 0x5f, 0x10100, 0x0, 0x0, 0x0, 0x7f8d371cb4c8, ...)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/client.go:489 +0x85f
github.com/fsouza/go-dockerclient.(*Client).Stats.func2(0xc82001b830, 0xc820011d80, 0x40, 0xc820209e00, 0xc820209e00, 0xc820209e60, 0xffffffffffffffff, 0xc820035160, 0xc82039ef60)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:773 +0x26d
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:792 +0x2e5
goroutine 329 [select]:
github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc820011d80, 0x40, 0xc820209e00, 0xc820209e00, 0xc820209e60, 0xffffffffffffffff, 0xc820035158, 0xc82039efc0)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:798 +0xe3
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:804 +0x3a7
goroutine 330 [IO wait]:
net.runtime_pollWait(0x7f8d371cab70, 0x72, 0xc82001c090)
/usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc8201aaa00, 0x72, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc8201aaa00, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc8201aa9a0, 0xc8201f4000, 0x1000, 0x1000, 0x0, 0x7f8d371c5028, 0xc82001c090)
/usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc8201340b0, 0xc8201f4000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:172 +0xe4
bufio.(*Reader).fill(0xc82018b4a0)
/usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).ReadSlice(0xc82018b4a0, 0xc81ffd810a, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:328 +0x21a
bufio.(*Reader).ReadLine(0xc82018b4a0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:357 +0x53
net/textproto.(*Reader).readLineSlice(0xc8202a9a30, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:55 +0x81
net/textproto.(*Reader).ReadLine(0xc8202a9a30, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:36 +0x40
net/http.ReadResponse(0xc82018b4a0, 0xc82029a620, 0xc8201340b0, 0x0, 0x0)
/usr/local/go/src/net/http/response.go:128 +0xf6
github.com/fsouza/go-dockerclient.(*Client).stream(0xc82001b830, 0x94c1a8, 0x3, 0xc82018b3e0, 0x5f, 0x10100, 0x0, 0x0, 0x0, 0x7f8d371cb4c8, ...)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/client.go:489 +0x85f
github.com/fsouza/go-dockerclient.(*Client).Stats.func2(0xc82001b830, 0xc820011e00, 0x40, 0xc820209f20, 0xc820209f00, 0xc820209f80, 0xffffffffffffffff, 0xc820035170, 0xc82039f020)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:773 +0x26d
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:792 +0x2e5
goroutine 331 [select]:
github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc820011e00, 0x40, 0xc820209f20, 0xc820209f00, 0xc820209f80, 0xffffffffffffffff, 0xc820035168, 0xc82039f080)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:798 +0xe3
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:804 +0x3a7
goroutine 219 [IO wait]:
net.runtime_pollWait(0x7f8d371ca030, 0x72, 0xc82001c090)
/usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc8201aa760, 0x72, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc8201aa760, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc8201aa700, 0xc8201ae000, 0x1000, 0x1000, 0x0, 0x7f8d371c5028, 0xc82001c090)
/usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc8201340a0, 0xc8201ae000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:172 +0xe4
bufio.(*Reader).fill(0xc82018b1a0)
/usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).ReadSlice(0xc82018b1a0, 0xc81ffd810a, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:328 +0x21a
bufio.(*Reader).ReadLine(0xc82018b1a0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:357 +0x53
net/textproto.(*Reader).readLineSlice(0xc8204eba30, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:55 +0x81
net/textproto.(*Reader).ReadLine(0xc8204eba30, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:36 +0x40
net/http.ReadResponse(0xc82018b1a0, 0xc82029a460, 0xc8201340a0, 0x0, 0x0)
/usr/local/go/src/net/http/response.go:128 +0xf6
github.com/fsouza/go-dockerclient.(*Client).stream(0xc82001b830, 0x94c1a8, 0x3, 0xc82018b0e0, 0x5f, 0x10100, 0x0, 0x0, 0x0, 0x7f8d371cb4c8, ...)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/client.go:489 +0x85f
github.com/fsouza/go-dockerclient.(*Client).Stats.func2(0xc82001b830, 0xc820011b40, 0x40, 0xc820209860, 0xc820209800, 0xc8202098c0, 0xffffffffffffffff, 0xc820134080, 0xc82018ade0)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:773 +0x26d
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:792 +0x2e5
goroutine 220 [select]:
github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc820011b40, 0x40, 0xc820209860, 0xc820209800, 0xc8202098c0, 0xffffffffffffffff, 0xc820134078, 0xc82018ae40)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:798 +0xe3
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:804 +0x3a7
goroutine 221 [IO wait]:
net.runtime_pollWait(0x7f8d371cbf98, 0x72, 0xc82001c090)
/usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc8201aa8b0, 0x72, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc8201aa8b0, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc8201aa850, 0xc8201f2000, 0x1000, 0x1000, 0x0, 0x7f8d371c5028, 0xc82001c090)
/usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc8201340a8, 0xc8201f2000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:172 +0xe4
bufio.(*Reader).fill(0xc82018b320)
/usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).ReadSlice(0xc82018b320, 0xc81ffd810a, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:328 +0x21a
bufio.(*Reader).ReadLine(0xc82018b320, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:357 +0x53
net/textproto.(*Reader).readLineSlice(0xc820205a30, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:55 +0x81
net/textproto.(*Reader).ReadLine(0xc820205a30, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:36 +0x40
net/http.ReadResponse(0xc82018b320, 0xc82029a540, 0xc8201340a8, 0x0, 0x0)
/usr/local/go/src/net/http/response.go:128 +0xf6
github.com/fsouza/go-dockerclient.(*Client).stream(0xc82001b830, 0x94c1a8, 0x3, 0xc82018b260, 0x5f, 0x10100, 0x0, 0x0, 0x0, 0x7f8d371cb4c8, ...)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/client.go:489 +0x85f
github.com/fsouza/go-dockerclient.(*Client).Stats.func2(0xc82001b830, 0xc820011c00, 0x40, 0xc820209980, 0xc820209b00, 0xc820209b00, 0xffffffffffffffff, 0xc820134090, 0xc82018aea0)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:773 +0x26d
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:792 +0x2e5
goroutine 222 [select]:
github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc820011c00, 0x40, 0xc820209980, 0xc820209900, 0xc820209b00, 0xffffffffffffffff, 0xc820134088, 0xc82018af00)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:798 +0xe3
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:804 +0x3a7
goroutine 141 [IO wait]:
net.runtime_pollWait(0x7f8d371cc118, 0x72, 0xc82001c090)
/usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc820246610, 0x72, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc820246610, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc8202465b0, 0xc8201b1000, 0x1000, 0x1000, 0x0, 0x7f8d371c5028, 0xc82001c090)
/usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc820498080, 0xc8201b1000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:172 +0xe4
bufio.(*Reader).fill(0xc82000f200)
/usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).ReadSlice(0xc82000f200, 0xc81ffd800a, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:328 +0x21a
bufio.(*Reader).ReadLine(0xc82000f200, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:357 +0x53
net/textproto.(*Reader).readLineSlice(0xc82055fa30, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:55 +0x81
net/textproto.(*Reader).ReadLine(0xc82055fa30, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:36 +0x40
net/http.ReadResponse(0xc82000f200, 0xc8201ee380, 0xc820498080, 0x0, 0x0)
/usr/local/go/src/net/http/response.go:128 +0xf6
github.com/fsouza/go-dockerclient.(*Client).stream(0xc82001b830, 0x94c1a8, 0x3, 0xc82000f140, 0x5f, 0x10100, 0x0, 0x0, 0x0, 0x7f8d371cb4c8, ...)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/client.go:489 +0x85f
github.com/fsouza/go-dockerclient.(*Client).Stats.func2(0xc82001b830, 0xc820011e80, 0x40, 0xc82039e060, 0xc82039e200, 0xc82039e240, 0xffffffffffffffff, 0xc820498050, 0xc82000ed80)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:773 +0x26d
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:792 +0x2e5
goroutine 142 [select]:
github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc820011e80, 0x40, 0xc82039e060, 0xc82039e000, 0xc82039e240, 0xffffffffffffffff, 0xc820498048, 0xc82000ede0)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:798 +0xe3
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:804 +0x3a7
goroutine 143 [IO wait]:
net.runtime_pollWait(0x7f8d371cc418, 0x72, 0xc82001c090)
/usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc820132b50, 0x72, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc820132b50, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc820132af0, 0xc820151000, 0x1000, 0x1000, 0x0, 0x7f8d371c5028, 0xc82001c090)
/usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc820398058, 0xc820151000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:172 +0xe4
bufio.(*Reader).fill(0xc820248cc0)
/usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).ReadSlice(0xc820248cc0, 0xc81fff250a, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:328 +0x21a
bufio.(*Reader).ReadLine(0xc820248cc0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:357 +0x53
net/textproto.(*Reader).readLineSlice(0xc82055ba30, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:55 +0x81
net/textproto.(*Reader).ReadLine(0xc82055ba30, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:36 +0x40
net/http.ReadResponse(0xc820248cc0, 0xc8205a8620, 0xc820398058, 0x0, 0x0)
/usr/local/go/src/net/http/response.go:128 +0xf6
github.com/fsouza/go-dockerclient.(*Client).stream(0xc82001b830, 0x94c1a8, 0x3, 0xc820248c00, 0x5f, 0x10100, 0x0, 0x0, 0x0, 0x7f8d371cb4c8, ...)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/client.go:489 +0x85f
github.com/fsouza/go-dockerclient.(*Client).Stats.func2(0xc82001b830, 0xc820011f80, 0x40, 0xc82039e300, 0xc82039e300, 0xc82039e3c0, 0xffffffffffffffff, 0xc820498060, 0xc82000ee40)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:773 +0x26d
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:792 +0x2e5
goroutine 144 [select]:
github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc820011f80, 0x40, 0xc82039e300, 0xc82039e300, 0xc82039e3c0, 0xffffffffffffffff, 0xc820498058, 0xc82000eea0)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:798 +0xe3
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:804 +0x3a7
goroutine 145 [IO wait]:
net.runtime_pollWait(0x7f8d371ca1b0, 0x72, 0xc82001c090)
/usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc820018ed0, 0x72, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc820018ed0, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc820018e70, 0xc82043e000, 0x1000, 0x1000, 0x0, 0x7f8d371c5028, 0xc82001c090)
/usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc820035198, 0xc82043e000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:172 +0xe4
bufio.(*Reader).fill(0xc82039f7a0)
/usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).ReadSlice(0xc82039f7a0, 0xc81ffff20a, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:328 +0x21a
bufio.(*Reader).ReadLine(0xc82039f7a0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:357 +0x53
net/textproto.(*Reader).readLineSlice(0xc820203a30, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:55 +0x81
net/textproto.(*Reader).ReadLine(0xc820203a30, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:36 +0x40
net/http.ReadResponse(0xc82039f7a0, 0xc8201c08c0, 0xc820035198, 0x0, 0x0)
/usr/local/go/src/net/http/response.go:128 +0xf6
github.com/fsouza/go-dockerclient.(*Client).stream(0xc82001b830, 0x94c1a8, 0x3, 0xc82039f6e0, 0x5f, 0x10100, 0x0, 0x0, 0x0, 0x7f8d371cb4c8, ...)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/client.go:489 +0x85f
github.com/fsouza/go-dockerclient.(*Client).Stats.func2(0xc82001b830, 0xc82039a000, 0x40, 0xc82039e540, 0xc82039e600, 0xc82039e660, 0xffffffffffffffff, 0xc820498070, 0xc82000ef00)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:773 +0x26d
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:792 +0x2e5
goroutine 354 [select]:
github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc82039a000, 0x40, 0xc82039e540, 0xc82039e500, 0xc82039e660, 0xffffffffffffffff, 0xc820498068, 0xc82000ef60)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:798 +0xe3
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:804 +0x3a7
Error opening syslog: Unix syslog delivery error
from dockbeat.
marminthibaut
commented on August 17, 2024
Thank you for reporting them this bug. I'll look at this in few days.
from dockbeat.
zstyblik
commented on August 17, 2024
Hello,
I've seen couple panics too. Originally, I thought they were caused by broken pipe connection, but looking at panic output again, I don't think so. Also, I'm unable to reproduce this again with breaking connection now, despite the fact I had like three panics in a row before.
2016/03/03 13:49:40.452946 sync.go:95: ERR Failed to publish events caused by: write tcp 172.17.0.1:45890->10.0.3.1:5504: write: broken pipe
2016/03/03 13:49:41.454546 sync.go:95: ERR Failed to publish events caused by: write tcp 172.17.0.1:45894->10.0.3.1:5504: write: broken pipe
2016/03/03 13:49:43.455960 sync.go:95: ERR Failed to publish events caused by: write tcp 172.17.0.1:45898->10.0.3.1:5504: write: broken pipe
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x4630d4]
goroutine 57 [running]:
github.com/ingensi/dockerbeat/beat.(*EventGenerator).getContainerEvent(0xc82000cc48, 0xc8202d8000, 0x0, 0x0)
/go/src/github.com/ingensi/dockerbeat/beat/EventGenerator.go:19 +0xa84
github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats.func2(0xc820292780, 0xc820292840, 0xc82000cc00, 0xc8202d8000)
/go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:167 +0xf4
created by github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats
/go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:179 +0x210
goroutine 1 [chan receive]:
github.com/elastic/beats/libbeat/beat.Run(0x9efa80, 0xa, 0x9e6000, 0xb, 0x7f29d3002d68, 0xc82000cc00, 0x0, 0x0)
/go/src/github.com/elastic/beats/libbeat/beat/beat.go:137 +0xc2
main.main()
/go/src/github.com/Ingensi/dockerbeat/main.go:14 +0xa0
goroutine 17 [syscall, locked to thread]:
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1
goroutine 5 [syscall]:
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
/usr/local/go/src/os/signal/signal_unix.go:28 +0x37
goroutine 18 [select]:
github.com/elastic/beats/libbeat/outputs/mode.(*backoff).Wait(0xc820122880, 0x9)
/go/src/github.com/elastic/beats/libbeat/outputs/mode/backoff.go:36 +0x1fa
github.com/elastic/beats/libbeat/outputs/mode.(*SingleConnectionMode).publish(0xc820104d20, 0x0, 0x0, 0xc8202d7e00, 0xc82037bae0, 0x0, 0x0)
/go/src/github.com/elastic/beats/libbeat/outputs/mode/single.go:143 +0x1a9
github.com/elastic/beats/libbeat/outputs/mode.(*SingleConnectionMode).PublishEvents(0xc820104d20, 0x0, 0x0, 0xc82003db00, 0xc8202d0000, 0x5, 0x800, 0x0, 0x0)
/go/src/github.com/elastic/beats/libbeat/outputs/mode/single.go:86 +0x75
github.com/elastic/beats/libbeat/outputs/logstash.(*logstash).BulkPublish(0xc820122780, 0x0, 0x0, 0x0, 0xc8202d0000, 0x5, 0x800, 0x0, 0x0)
/go/src/github.com/elastic/beats/libbeat/outputs/logstash/logstash.go:137 +0x39d
github.com/elastic/beats/libbeat/publisher.(*outputWorker).sendBulk(0xc82028e050, 0xc82037bd88, 0xc8202d0000, 0x5, 0x800)
/go/src/github.com/elastic/beats/libbeat/publisher/output.go:106 +0x190
github.com/elastic/beats/libbeat/publisher.(*outputWorker).onBulk(0xc82028e050, 0xc82037bd88, 0xc8202d0000, 0x5, 0x800)
/go/src/github.com/elastic/beats/libbeat/publisher/output.go:82 +0x39d
github.com/elastic/beats/libbeat/publisher.(*outputWorker).onMessage(0xc82028e050, 0xffffffffffff0000, 0x0, 0x0, 0x0, 0xc8202d0000, 0x5, 0x800)
/go/src/github.com/elastic/beats/libbeat/publisher/output.go:65 +0x79
github.com/elastic/beats/libbeat/publisher.(*messageWorker).run(0xc82028e050)
/go/src/github.com/elastic/beats/libbeat/publisher/worker.go:69 +0x424
created by github.com/elastic/beats/libbeat/publisher.(*messageWorker).init
/go/src/github.com/elastic/beats/libbeat/publisher/worker.go:55 +0x11e
goroutine 7 [select]:
github.com/ingensi/dockerbeat/beat.(*Dockerbeat).Run(0xc82000cc00, 0xc820080f30, 0x0, 0x0)
/go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:90 +0x368
github.com/elastic/beats/libbeat/beat.(*Beat).Run(0xc820080f30, 0x0, 0x0)
/go/src/github.com/elastic/beats/libbeat/beat/beat.go:271 +0x361
github.com/elastic/beats/libbeat/beat.(*Beat).Start(0xc820080f30, 0x0, 0x0)
/go/src/github.com/elastic/beats/libbeat/beat/beat.go:174 +0x126
github.com/elastic/beats/libbeat/beat.Run.func1(0xc820080f30)
/go/src/github.com/elastic/beats/libbeat/beat/beat.go:121 +0x3c
created by github.com/elastic/beats/libbeat/beat.Run
/go/src/github.com/elastic/beats/libbeat/beat/beat.go:133 +0x8d
goroutine 19 [select]:
github.com/elastic/beats/libbeat/publisher.(*bulkWorker).run(0xc8202d4000)
/go/src/github.com/elastic/beats/libbeat/publisher/bulk.go:58 +0x371
created by github.com/elastic/beats/libbeat/publisher.newBulkWorker
/go/src/github.com/elastic/beats/libbeat/publisher/bulk.go:42 +0x222
goroutine 20 [select, locked to thread]:
runtime.gopark(0xae4280, 0xc820024f28, 0x9e1d90, 0x6, 0x42dd18, 0x2)
/usr/local/go/src/runtime/proc.go:185 +0x163
runtime.selectgoImpl(0xc820024f28, 0x0, 0x18)
/usr/local/go/src/runtime/select.go:392 +0xa64
runtime.selectgo(0xc820024f28)
/usr/local/go/src/runtime/select.go:212 +0x12
runtime.ensureSigM.func1()
/usr/local/go/src/runtime/signal1_unix.go:227 +0x353
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1
goroutine 21 [chan receive]:
github.com/elastic/beats/libbeat/service.HandleSignals.func1(0xc8202930e0, 0xc82028ad00, 0xc82028acf0)
/go/src/github.com/elastic/beats/libbeat/service/service.go:29 +0x3a
created by github.com/elastic/beats/libbeat/service.HandleSignals
/go/src/github.com/elastic/beats/libbeat/service/service.go:32 +0x1f8
goroutine 24 [select]:
github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc82031a780, 0x40, 0xc8202928a0, 0xc820292800, 0xc820292900, 0xffffffffffffffff, 0xc82002c000, 0xc820686060)
/go/src/github.com/fsouza/go-dockerclient/container.go:807 +0xe3
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/go/src/github.com/fsouza/go-dockerclient/container.go:813 +0x3a7
goroutine 23 [IO wait]:
net.runtime_pollWait(0x7f29d3004340, 0x72, 0xc8200100a0)
/usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc8202d4140, 0x72, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc8202d4140, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc8202d40e0, 0xc8200a8000, 0x1000, 0x1000, 0x0, 0x7f29d2ffe028, 0xc8200100a0)
/usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc82002c028, 0xc8200a8000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:172 +0xe4
bufio.(*Reader).fill(0xc8206861e0)
/usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).ReadSlice(0xc8206861e0, 0xc81fffbf0a, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:328 +0x21a
bufio.(*Reader).ReadLine(0xc8206861e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:357 +0x53
net/textproto.(*Reader).readLineSlice(0xc82037da30, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:55 +0x81
net/textproto.(*Reader).ReadLine(0xc82037da30, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:36 +0x40
net/http.ReadResponse(0xc8206861e0, 0xc8200aa000, 0xc82002c028, 0x0, 0x0)
/usr/local/go/src/net/http/response.go:128 +0xf6
github.com/fsouza/go-dockerclient.(*Client).stream(0xc8202d81b0, 0x9dad80, 0x3, 0xc8206860c0, 0x5f, 0x10100, 0x0, 0x0, 0x0, 0x7f29d0176340, ...)
/go/src/github.com/fsouza/go-dockerclient/client.go:490 +0x85f
github.com/fsouza/go-dockerclient.(*Client).Stats.func2(0xc8202d81b0, 0xc82031a780, 0x40, 0xc8202928a0, 0xc820292900, 0xc820292900, 0xffffffffffffffff, 0xc82002c020, 0xc820686000)
/go/src/github.com/fsouza/go-dockerclient/container.go:782 +0x26d
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/go/src/github.com/fsouza/go-dockerclient/container.go:801 +0x2e5
goroutine 54 [IO wait]:
net.runtime_pollWait(0x7f29d30044c0, 0x72, 0xc8200100a0)
/usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc8200601b0, 0x72, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc8200601b0, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc820060150, 0xc820015000, 0x1000, 0x1000, 0x0, 0x7f29d2ffe028, 0xc8200100a0)
/usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc82030a020, 0xc820015000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:172 +0xe4
net/http.noteEOFReader.Read(0x7f29d0172100, 0xc82030a020, 0xc8200da058, 0xc820015000, 0x1000, 0x1000, 0x634ca9, 0x0, 0x0)
/usr/local/go/src/net/http/transport.go:1370 +0x67
net/http.(*noteEOFReader).Read(0xc82028c120, 0xc820015000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
<autogenerated>:126 +0xd0
bufio.(*Reader).fill(0xc8202924e0)
/usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc8202924e0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc8200da000)
/usr/local/go/src/net/http/transport.go:876 +0xf7
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:685 +0xc78
goroutine 55 [select]:
net/http.(*persistConn).writeLoop(0xc8200da000)
/usr/local/go/src/net/http/transport.go:1009 +0x40c
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:686 +0xc9d
goroutine 58 [semacquire]:
sync.runtime_Syncsemacquire(0xc8202b6040)
/usr/local/go/src/runtime/sema.go:237 +0x201
sync.(*Cond).Wait(0xc8202b6030)
/usr/local/go/src/sync/cond.go:62 +0x9b
io.(*pipe).read(0xc8202b6000, 0xc820380000, 0x200, 0x200, 0x0, 0x0, 0x0)
/usr/local/go/src/io/pipe.go:52 +0x2d2
io.(*PipeReader).Read(0xc82002c000, 0xc820380000, 0x200, 0x200, 0x0, 0x0, 0x0)
/usr/local/go/src/io/pipe.go:134 +0x50
encoding/json.(*Decoder).refill(0xc8203841a0, 0x0, 0x0)
/usr/local/go/src/encoding/json/stream.go:152 +0x287
encoding/json.(*Decoder).readValue(0xc8203841a0, 0x1, 0x0, 0x0)
/usr/local/go/src/encoding/json/stream.go:128 +0x41b
encoding/json.(*Decoder).Decode(0xc8203841a0, 0x8197a0, 0xc8202b0000, 0x0, 0x0)
/usr/local/go/src/encoding/json/stream.go:57 +0x159
github.com/fsouza/go-dockerclient.(*Client).Stats(0xc8202d81b0, 0xc82031a780, 0x40, 0xc8202928a0, 0x180000, 0xc820292900, 0xffffffffffffffff, 0x0, 0x0)
/go/src/github.com/fsouza/go-dockerclient/container.go:817 +0x497
github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats.func1(0xc820292960, 0xc82000cc00, 0xc82031a780, 0x40, 0xc8202928a0, 0x0, 0xc820292900, 0xffffffffffffffff)
/go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:157 +0x70
created by github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats
/go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:159 +0x1cd
goroutine 59 [chan receive]:
github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats.func2(0xc8202928a0, 0xc820292960, 0xc82000cc00, 0xc8202d8090)
/go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:162 +0x52
created by github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats
/go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:179 +0x210
stybla@zstyblik-lenovo-w541:~/work/zooplus/stash/zdeneks/docker-dockerbeat$ docker run -it -v /var/run/docker.sock:/var/run/docker.sock -e DOCKERBEAT_MODE="logstash" -e LOGSTASH_HOSTS="['10.0.3.1:5504']" repo.zooplus.de/dockerbeat:20160303.090501
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x4630d4]
goroutine 57 [running]:
github.com/ingensi/dockerbeat/beat.(*EventGenerator).getContainerEvent(0xc82000cc48, 0xc8202e63f0, 0x0, 0xc82029a600)
/go/src/github.com/ingensi/dockerbeat/beat/EventGenerator.go:19 +0xa84
github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats.func2(0xc820016ea0, 0xc820016f60, 0xc82000cc00, 0xc8202e63f0)
/go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:167 +0xf4
created by github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats
/go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:179 +0x210
goroutine 1 [chan receive]:
github.com/elastic/beats/libbeat/beat.Run(0x9efa80, 0xa, 0x9e6000, 0xb, 0x7fc441c68c80, 0xc82000cc00, 0x0, 0x0)
/go/src/github.com/elastic/beats/libbeat/beat/beat.go:137 +0xc2
main.main()
/go/src/github.com/Ingensi/dockerbeat/main.go:14 +0xa0
goroutine 17 [syscall, locked to thread]:
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1
goroutine 5 [syscall]:
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
/usr/local/go/src/os/signal/signal_unix.go:28 +0x37
goroutine 7 [select]:
github.com/ingensi/dockerbeat/beat.(*Dockerbeat).Run(0xc82000cc00, 0xc820080f30, 0x0, 0x0)
/go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:90 +0x368
github.com/elastic/beats/libbeat/beat.(*Beat).Run(0xc820080f30, 0x0, 0x0)
/go/src/github.com/elastic/beats/libbeat/beat/beat.go:271 +0x361
github.com/elastic/beats/libbeat/beat.(*Beat).Start(0xc820080f30, 0x0, 0x0)
/go/src/github.com/elastic/beats/libbeat/beat/beat.go:174 +0x126
github.com/elastic/beats/libbeat/beat.Run.func1(0xc820080f30)
/go/src/github.com/elastic/beats/libbeat/beat/beat.go:121 +0x3c
created by github.com/elastic/beats/libbeat/beat.Run
/go/src/github.com/elastic/beats/libbeat/beat/beat.go:133 +0x8d
goroutine 18 [select]:
github.com/elastic/beats/libbeat/publisher.(*messageWorker).run(0xc820298000)
/go/src/github.com/elastic/beats/libbeat/publisher/worker.go:61 +0x449
created by github.com/elastic/beats/libbeat/publisher.(*messageWorker).init
/go/src/github.com/elastic/beats/libbeat/publisher/worker.go:55 +0x11e
goroutine 19 [select]:
github.com/elastic/beats/libbeat/publisher.(*bulkWorker).run(0xc8202d8000)
/go/src/github.com/elastic/beats/libbeat/publisher/bulk.go:58 +0x371
created by github.com/elastic/beats/libbeat/publisher.newBulkWorker
/go/src/github.com/elastic/beats/libbeat/publisher/bulk.go:42 +0x222
goroutine 20 [select, locked to thread]:
runtime.gopark(0xae4280, 0xc820025728, 0x9e1d90, 0x6, 0x42dd18, 0x2)
/usr/local/go/src/runtime/proc.go:185 +0x163
runtime.selectgoImpl(0xc820025728, 0x0, 0x18)
/usr/local/go/src/runtime/select.go:392 +0xa64
runtime.selectgo(0xc820025728)
/usr/local/go/src/runtime/select.go:212 +0x12
runtime.ensureSigM.func1()
/usr/local/go/src/runtime/signal1_unix.go:227 +0x353
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1
goroutine 21 [chan receive]:
github.com/elastic/beats/libbeat/service.HandleSignals.func1(0xc82029b0e0, 0xc820296cf0, 0xc820296ce0)
/go/src/github.com/elastic/beats/libbeat/service/service.go:29 +0x3a
created by github.com/elastic/beats/libbeat/service.HandleSignals
/go/src/github.com/elastic/beats/libbeat/service/service.go:32 +0x1f8
goroutine 12 [IO wait]:
net.runtime_pollWait(0x7fc441c6a258, 0x72, 0xc8200100a0)
/usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc820106370, 0x72, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc820106370, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc820106310, 0xc82032c000, 0x1000, 0x1000, 0x0, 0x7fc441c64028, 0xc8200100a0)
/usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc820306798, 0xc82032c000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:172 +0xe4
bufio.(*Reader).fill(0xc820507aa0)
/usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).ReadSlice(0xc820507aa0, 0xc81fffbf0a, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:328 +0x21a
bufio.(*Reader).ReadLine(0xc820507aa0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:357 +0x53
net/textproto.(*Reader).readLineSlice(0xc82037ba30, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:55 +0x81
net/textproto.(*Reader).ReadLine(0xc82037ba30, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/textproto/reader.go:36 +0x40
net/http.ReadResponse(0xc820507aa0, 0xc82010c2a0, 0xc820306798, 0x0, 0x0)
/usr/local/go/src/net/http/response.go:128 +0xf6
github.com/fsouza/go-dockerclient.(*Client).stream(0xc8202e61b0, 0x9dad80, 0x3, 0xc8205079e0, 0x5f, 0x10100, 0x0, 0x0, 0x0, 0x7fc4405df340, ...)
/go/src/github.com/fsouza/go-dockerclient/client.go:490 +0x85f
github.com/fsouza/go-dockerclient.(*Client).Stats.func2(0xc8202e61b0, 0xc82000b1c0, 0x40, 0xc820016d80, 0xc820016d00, 0xc820016de0, 0xffffffffffffffff, 0xc820306790, 0xc8205078c0)
/go/src/github.com/fsouza/go-dockerclient/container.go:782 +0x26d
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/go/src/github.com/fsouza/go-dockerclient/container.go:801 +0x2e5
goroutine 55 [chan receive]:
github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats.func2(0xc820016d80, 0xc820016e40, 0xc82000cc00, 0xc8202e6360)
/go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:162 +0x52
created by github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats
/go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:179 +0x210
goroutine 10 [IO wait]:
net.runtime_pollWait(0x7fc441c6a318, 0x72, 0xc8200100a0)
/usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc8200601b0, 0x72, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc8200601b0, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc820060150, 0xc820015000, 0x1000, 0x1000, 0x0, 0x7fc441c64028, 0xc8200100a0)
/usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc820306020, 0xc820015000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:172 +0xe4
net/http.noteEOFReader.Read(0x7fc4405db100, 0xc820306020, 0xc82031c108, 0xc820015000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/http/transport.go:1370 +0x67
net/http.(*noteEOFReader).Read(0xc820304220, 0xc820015000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
<autogenerated>:126 +0xd0
bufio.(*Reader).fill(0xc8203184e0)
/usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc8203184e0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc82031c0b0)
/usr/local/go/src/net/http/transport.go:876 +0xf7
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:685 +0xc78
goroutine 11 [select]:
net/http.(*persistConn).writeLoop(0xc82031c0b0)
/usr/local/go/src/net/http/transport.go:1009 +0x40c
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:686 +0xc9d
goroutine 54 [semacquire]:
sync.runtime_Syncsemacquire(0xc82010a1c0)
/usr/local/go/src/runtime/sema.go:237 +0x201
sync.(*Cond).Wait(0xc82010a1b0)
/usr/local/go/src/sync/cond.go:62 +0x9b
io.(*pipe).read(0xc82010a180, 0xc82005e200, 0x200, 0x200, 0x0, 0x0, 0x0)
/usr/local/go/src/io/pipe.go:52 +0x2d2
io.(*PipeReader).Read(0xc820306788, 0xc82005e200, 0x200, 0x200, 0x0, 0x0, 0x0)
/usr/local/go/src/io/pipe.go:134 +0x50
encoding/json.(*Decoder).refill(0xc82005c1a0, 0x0, 0x0)
/usr/local/go/src/encoding/json/stream.go:152 +0x287
encoding/json.(*Decoder).readValue(0xc82005c1a0, 0x1, 0x0, 0x0)
/usr/local/go/src/encoding/json/stream.go:128 +0x41b
encoding/json.(*Decoder).Decode(0xc82005c1a0, 0x8197a0, 0xc82032a000, 0x0, 0x0)
/usr/local/go/src/encoding/json/stream.go:57 +0x159
github.com/fsouza/go-dockerclient.(*Client).Stats(0xc8202e61b0, 0xc82000b1c0, 0x40, 0xc820016d80, 0x4c6b00, 0xc820016de0, 0xffffffffffffffff, 0x0, 0x0)
/go/src/github.com/fsouza/go-dockerclient/container.go:817 +0x497
github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats.func1(0xc820016e40, 0xc82000cc00, 0xc82000b1c0, 0x40, 0xc820016d80, 0x0, 0xc820016de0, 0xffffffffffffffff)
/go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:157 +0x70
created by github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats
/go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:159 +0x1cd
goroutine 13 [select]:
github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc82000b1c0, 0x40, 0xc820016d80, 0xc820016d00, 0xc820016de0, 0xffffffffffffffff, 0xc820306788, 0xc820507920)
/go/src/github.com/fsouza/go-dockerclient/container.go:807 +0xe3
created by github.com/fsouza/go-dockerclient.(*Client).Stats
/go/src/github.com/fsouza/go-dockerclient/container.go:813 +0x3a7
from dockbeat.
ShibaOnTheMoon
commented on August 17, 2024
I'm currently trying to reproduce the issue, but I can't find a way to do it. Any ideas so I can start working on debugging this issue ?
from dockbeat.
buehler
commented on August 17, 2024
We saw the issue, when we used it in the docker environment.
We ran the beat in a dockerbox, on our docker-environment. Currently there are ~30 containers running. After a hour or two, the beat crashes.
Maybe because there are too many events? I don't know the source of libbeat so that could be an issue.
Cheers
from dockbeat.
ShibaOnTheMoon
commented on August 17, 2024
I was watching the kibana monitoring when something hit me : the crashs seems to happen when a container is stopped and/or removed.
Maybe the crash occurs if we remove a container after container listing but before getting statistics. We're currently investigating on this.
from dockbeat.
erwanncloarec
commented on August 17, 2024
Thanks to @HipsterWhale we can now reproduce the issue:
for i in `seq 1 10000`;
do
docker run --rm centos bash
done
With that script, dockerbeat will eventually stop.
It's clear that it is stopping because a container stops!
from dockbeat.
buehler
commented on August 17, 2024
Whoa 😯
Well then it could be a solution to collect all data first and then deliver them? Or how would you deliver partial data?
from dockbeat.
ShibaOnTheMoon
commented on August 17, 2024
I've made a pull request with the fix for this bug, it will be released in a beta soon.
from dockbeat.
buehler
commented on August 17, 2024
@HipsterWhale
Thanks! Looking forward to use it again and test it :-)
from dockbeat.
marminthibaut
commented on August 17, 2024
Thanks @HipsterWhale ! Awesome work
I just reopen this issue until @buehler @pcallewaert and @zstyblik report if it's ok (or not).
from dockbeat.
marminthibaut
commented on August 17, 2024
@HipsterWhale can you explain why this bug occurred?
from dockbeat.
ShibaOnTheMoon
commented on August 17, 2024
The crash occurred because the dockerclient return the stats to nil with an error to nil too if the container have been removed in the interval between the listing and the statistics retrieving. I've just added the case to handle this error with a message to the log to say that something abnormal happened.
from dockbeat.
marminthibaut
commented on August 17, 2024
👍
from dockbeat.
buehler
commented on August 17, 2024
I ran the new image from the develop branch. By now, it seems to be working 👍
Neat!
:)
from dockbeat.
marminthibaut
commented on August 17, 2024
Thanks!
from dockbeat.
zstyblik
commented on August 17, 2024
Sorry, I didn't have time to give this a test, but it's good to hear it's working.
from dockbeat.
Related Issues (20)
- Existing ElasticSearch cluster HOT 9
- dockbeat doesnot writes anything HOT 5
- Label parsing in Elasticsearch
- dockbeat.hostname not showing up HOT 1
- Dockbeat logging
- Kibana Dashboard HOT 1
- To make labels exploitable in Kibana HOT 8
- Example configuration shipper
- docker oom-killer / possible memory leak ? HOT 7
- metricbeat docker plugin HOT 6
- Tags on dockerhub? HOT 3
- Dockbeat fails with npipe:// socket on Windows; exits with status 1
- Dockbeat cannot collect memory stats on Windows
- Run dockbeat in a docker network and listen only data of containers in that network
- Question: cpu.totalUsage units HOT 2
- Timestamp is 0001-01-01T00:00:00.000Z for containers which aren't up HOT 1
- Problem with containerLabels entry failing date format in ElasticSearch
- Plans to update for ES6? HOT 2
- if docker running on remote server how can i configure dockbeat
- issue with publish data to elasticsearch HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dockbeat.