kinvolk-archives / cgnet Goto Github PK

License: Apache License 2.0

Makefile 6.85% Go 58.15% C 35.00%

cgnet's Introduction

cgnet

cgnet uses eBPF to gather network statistics from cgroups.

$ make

# to test the cli tool
$ ./cgnet top <path to cgroup>

# to run the exporter outside of k8s
$ ./cgnet export --kubeconfig <path to kubeconfig>

# to run the exporter as a DaemonSet in the cluster
$ kubectl apply -f manifests/deploy/all-in-one.yaml

cgnet top

Uses the BPF program to monitor the cgroup specified on the command line.

cgnet export

Interacts with the Kubernetes API to retrieve a list of running pods and installs the BPF program for their cgroups. It then exports the data for Prometheus.

We provide the following to deploy and test the exporter:

Dockerfile - builds a simple minimal container running cgnet export
manifests/deploy - has Kubernetes manifests to run the container with the exporter
manifests/example - has configuration files to set up a cluster for testing the exporter

Dependencies

This repository experiments with Daniel Mack's eBPF hooks for cgroups.

Linux v4.10-rc
CONFIG_CGROUP_BPF=y

Vendoring

We use dep.

cgnet's People

Contributors

Stargazers

Watchers

Forkers

robertgzr linus5 ukinau chenk008 sattishv beatrizmr sirnexus bretagne-peiqi triplekill huanwei pranay01 yashbhutwala keyolk jepsenwan readerx hexfusion anantjakhmola bliklabs

cgnet's Issues

Proper CLI

We should have a nice cli, more similar to top; maybe not taking up the entire terminal buffer, but updating rather than logging.

CI build and push Docker image on hub

You could use CircleCI with a deployment directive (example: scope).

And then in the README, you can explain how to use it in Kubernetes without recompiling anything.

cannot load github.com/cespare/xxhash/v2

I make cgent but meet this error

...
go build \
     -ldflags "-X github.com/kinvolk/cgnet/cmd.version=967821a" \
     -o cgnet .
build github.com/kinvolk/cgnet: cannot load github.com/cespare/xxhash/v2: open /usr/lib/golang/src/vendor/github.com/cespare/xxhash/v2: no such file or directory
...

The go version I used is go version go1.13.4 linux/amd64

Prometheus exporter port

A user may have many exporters and Prometheus components on the same machine, so to make that easier each has a unique port number. (...)
Feel free to grab the next free port number when developing your exporter, preferably before publicly announcing it. If you’re not ready to release yet, putting your username and WIP is fine.
src

https://github.com/prometheus/prometheus/wiki/Default-port-allocations

Get pod cgroup path

see #13 (comment)

bpf: count incoming/outgoing packages

and sum up the sent/received bytes.

Install bpf program for every pod cgroup

ref: https://github.com/kubernetes/community/blob/master/contributors/design-proposals/pod-resource-management.md

kubepods is top-level cgroup (cgroup-root config?)
QoS class cgroups:
- besteffort
- burstable
- guaranteed
pod cgroup: pod<uid>
bpf module/map per pod

Document usage in README

the cmdline tool will be documented via the --help flag
the exporter part will need some explanation on setup and usage (bc multiple deployment options of Prometheus)

Example deployment and tests

How can we do tests?

Test bpf programs using BPF_PROG_TEST_RUN
We will provide k8s manifests for an example deployment of cgnet as a DaemonSet

Additional statistics

A list of possible metrics to collect:

bandwith
errors
dropped packages?

Detect the kubelet cgroup-root setting

We default to /sys/fs/cgroup/systemd/kubepods for now.

It looks like querying of the kubelet config is part of
this PR for dynamic kubelet configuration.

There seems to be a way without that:

A way to query/monitor the config in-use on a given node. Today this is possible via the configz endpoint, but this is just for debugging, not production use.
(src)

What happens when a pod disappears

do we have to do anything to "clean up" after the bpf program?
keep the metrics?

Subscribe to pod events via k8s API

new pod
delete pod
do we need update pod?

Expose metrics to Prometheus

total number of tracked pods
per pod metrics:
- incoming/outgoing packets/bytes