traas-stack / holoinsight-agent Goto Github PK

Describe This Problem

K8s changes the Container Runtime on a Node from Docker Engine to containerd.
See https://kubernetes.io/docs/tasks/administer-cluster/migrating-from-dockershim/change-runtime-containerd/

Proposal

Supports containerd as container layer.

Additional Context

Describe This Problem

Now, the log monitor feature can compute metrics from logs in Agent side.
Some users to this feature to compute their business error count.
But the result only contains error count, not log texts that lead to this error count.

Proposal

Supports keeping some log texts when a certain condition(for example, when wrong) is met.

Additional Context

Describe This Problem

When there is a network isolation between HoloInsight-Agent pod and target pod, network plugins have to work with nsenter now.
But nsenter solution only works for runc runtime class not rund.

If we use docker Standard AP when interacting with the container, we can get better compatibility.
Of course, performance issues must also be properly considered.
It is recommended to perform nsenter and docker exec performance tests on runc/rund respectively.

Proposal

Additional Context

Describe This Problem

Currently there is no arm64v8 (and others arch) agent image, which make it inconvenient to run in my M1 mac environment although Apple supports 'Rosetta 2' to help amd64 binary run in arm64.

Proposal

Supports building arm64v8 agent image

Additional Context

Describe this problem

Container helper is compile with glibc. When running the helper bin in alpine.

To Reproduce

Expected behavior

Additional Information

• https://stackoverflow.com/questions/34729748/installed-go-binary-not-found-in-path-on-alpine-linux-docker
• https://www.dyxmq.cn/uncategorized/installed-go-binary-not-found-in-path-on-alpine-linux-docker.html

Add DCGM-export metrics collection

Describe This Problem

When running in sidecar/VM mode, it is almost always necessary to enable the collection of system indicators.

Proposal

Enable system metrics collector by default

Additional Context

Describe this problem

For unknown reasons, there may be some execs that remain and do not exit.
I think we need to have some means to prevent arbitrary execs from hanging permanently.

Steps to reproduce

This problem is not easy to reproduce.
It is very likely that holoinsight-agent exited abnormally while tar was reading data from STDIN, causing STDIN to be unable to read EOF for a long time, so it hung.

Expected behavior

No exec remains.

Additional Information

There are several ideas:

• Always use timeout to wrap all commands, and KILL when timeout occurs. But be aware that timeout can generate zombie processes in some scenarios. See #99
• All exec commands carry special environment variable tags to indicate that they are calls from holoinsight-agent. Periodically clean up processes that need to be marked. See utils.go

No response

Describe this problem

pkg/pipeline/sys/sys.go:169
SysPipeline.emitLoop

func (p *SysPipeline) emitLoop() {
  ...
  p.emitOnce(alignTs)
  ...
}

Here alignTs is the start time of current time window when execute.
But the data event time should be changed to (alignTs - windowSize).

To Reproduce

Expected behavior

Additional Information

Describe This Problem

During that reboot or deploy time window there will be data loss, about 1~2 minutes.
This can lead to a poor user experience, as the data on the page will drop significantly, especially when the user's cluster size is relatively small.

Proposal

Record the data location of consumption, and start consumption from the last position after restarting.
This is useful for log monitoring.

Additional Context

Refactor Request

Add license header and related scripts

Description of the proposed solution

Add license header and related scripts

Additional context

Describe This Problem

The main branch is important. We need to make sure that the codes of the main branch are always available to compile and run.
But currently there is no any protection for the main branch.

Proposal

Configure github 'branch protection rule' for the main branch.

Check items before merging:

1. Compile successfully
2. All UT passed
3. Codes are formatted
4. All Golang files have license header

Additional Context

Describe This Problem

Proposal

Additional Context

Describe This Problem

There are many metadatas in the pod labels. If we can support ref pod label as metric data tag. Then we can create more powerful queries with more dimensions.

Proposal

Add ability to support ref pod label as metric data tag.

Additional Context

Describe this problem

I have deployed a holoinsight-agent daemonset in my k8s cluster.
First, I configure the daemonset with apikey1 of tenant1.
And then I configure it with apikey2 of tenant2 and redeploy it.
In server side database, the metadata of these agents in 'gaea_agent' table still have tenant = 'tenant1'.

To Reproduce

See description above

Expected behavior

I think the correct case should be tenant = 'tenant2'.

Additional Information

Describe This Problem

Add deploy files for k8s.
For example deployment yaml or helm.

Proposal

Add deploy files for k8s.
For example deployment yaml or helm.

Additional Context