Make cAdvisor work on systemd systems about cadvisor HOT 16 CLOSED

--docker_only=true would filter cgroups to only report docker and root containers.

from cadvisor.

👍

from cadvisor.

some cgroups are under /system.slice/docker-*. lmctfy stats doesn't detect that. It should use device or cpu cgroup to detect. For libcontainer, one way to track stats is to use nsinit pid and find paths using /proc/pid-nsinit/cgroups

$ ps -efww | grep cadvisor
root 3490 3362 0 21:26 ? 00:00:00 /bin/sh -c /usr/bin/cadvisor
root 3514 3490 0 21:26 ? 00:00:02 /usr/bin/cadvisor
$ cat /proc/3490/cgroup
10:perf_event:/
9:blkio:/system.slice/docker-d293e6edbf522628caf322c1d603ddbfbd891c7014e7a747e1bb4bfa122d3b1f.scope
8:net_cls:/
7:freezer:/
6:devices:/system.slice/docker-d293e6edbf522628caf322c1d603ddbfbd891c7014e7a747e1bb4bfa122d3b1f.scope
5:memory:/system.slice/docker-d293e6edbf522628caf322c1d603ddbfbd891c7014e7a747e1bb4bfa122d3b1f.scope
4:cpu,cpuacct:/system.slice/docker-d293e6edbf522628caf322c1d603ddbfbd891c7014e7a747e1bb4bfa122d3b1f.scope
3:cpuset:/
2:name=systemd:/system.slice/docker-d293e6edbf522628caf322c1d603ddbfbd891c7014e7a747e1bb4bfa122d3b1f.scope

from cadvisor.

Does this affect /, /docker, /docker/, or all?

from cadvisor.

Perhaps lmctfy should have configurable job tracking hierarchy and we
should use /systemd hierarchy on systemd machines? Using device or cpu
hierarchy to detect seems somewhat arbitrary?

On Tue, Jun 17, 2014 at 2:39 PM, Victor Marmol [email protected]
wrote:

Does this affect /, /docker, /docker/, or all?

—
Reply to this email directly or view it on GitHub
#59 (comment).

from cadvisor.

+1

On Tue, Jun 17, 2014 at 2:39 PM, Victor Marmol [email protected]
wrote:

Does this affect /, /docker, /docker/, or all?

—
Reply to this email directly or view it on GitHub
#59 (comment).

from cadvisor.

That may be tricky given freezer support and systemd not using freezer.

from cadvisor.

Why can't we change libcontainer as Rohit suggested and not change lmctfy?

On Tue, Jun 17, 2014 at 2:42 PM, Victor Marmol [email protected]
wrote:

That may be tricky given freezer support and systemd not using freezer.

—
Reply to this email directly or view it on GitHub
#59 (comment).

from cadvisor.

Changing libcontainer makes sense but at the moment lmctfy gives us more
detailed statistics so there is benefit for handling this in lmctfy.

On Tue, Jun 17, 2014 at 2:45 PM, Vish Kannan [email protected]
wrote:

Why can't we change libcontainer as Rohit suggested and not change lmctfy?

On Tue, Jun 17, 2014 at 2:42 PM, Victor Marmol [email protected]
wrote:

That may be tricky given freezer support and systemd not using freezer.

—
Reply to this email directly or view it on GitHub
#59 (comment).

—
Reply to this email directly or view it on GitHub
#59 (comment).

from cadvisor.

We will need to change both :)

from cadvisor.

/ is fine. Everything else is broken.

libcontainer seems like a simpler change, but I am going to try and change both lmctfy and libcontainer.

One problem with systemd systems is that there are many containers (some of them just within a named systemd hierarchy). We might want to filter some out.

I'd be fine if anyone wants to take over either of the changes :)

from cadvisor.

/ is handled by lmctfy, everything else is handled by libcontainer. So the
libcontainer change should be enough to fix this.

WDYT of ignoring non-Docker and non-root containers in systemd systems? We
should be able to degrade gracefully if that's the case anyways.

On Tue, Jun 17, 2014 at 4:07 PM, Rohit Jnagal [email protected]
wrote:

/ is fine. Everything else is broken.

libcontainer seems like a simpler change, but I am going to try and change
both lmctfy and libcontainer.

One problem with systemd systems is that there are many containers (some
of them just within a named systemd hierarchy). We might want to filter
some out.

I'd be fine if anyone wants to take over either of the changes :)

—
Reply to this email directly or view it on GitHub
#59 (comment).

from cadvisor.

This is done thanks to the raw driver. We've tested in CoreOS and RHEL. If you find any issues on other systems please file issues :)

from cadvisor.

@vmarmol Was a patch applied to cadvisor to ignore non-docker and non-root containers in systemd systems? I'm using the latest tag of google/cadvisor available on docker hub with CentOS Linux release 7.2.1511 and still seeing systemd containers appearing in the stats from cadvisor.

from cadvisor.

cAdvisor does not ignore any containers as of now.

On Thu, Jan 28, 2016 at 1:23 AM, Kamba Abudu [email protected]
wrote:

@vmarmol https://github.com/vmarmol Was a patch applied to cadvisor to
ignore non-docker and non-root containers in systemd systems? I'm using the
latest tag of google/cadvisor available on docker hub with CentOS Linux
release 7.2.1511 and still seeing systemd containers appearing in the stats
from cadvisor.

—
Reply to this email directly or view it on GitHub
#59 (comment).

from cadvisor.

--docker_only=true would filter cgroups to only report docker and root containers.

This is really use to reduce the output from stats/container when used with heapster legacy kubelet source.

from cadvisor.