lunar / snyk_exporter Goto Github PK

It is too easy to forget to update the help snippet in the readme when making changes.

Can we automate this, maybe in the release process?

Describe the bug
Upgrading the exporter from v1.2.0->v1.3.1 is causing it to crash multiple times a day. The exporter is running on kubernetes and crashes when it's liveness probe tries to access /healthz and gets the following error - net/http: request canceled (Client.Timeout exceeded while awaiting headers).

To Reproduce
Steps to reproduce the behavior:

1. Publish the exporter as a k8s pod, with /healthz configured as the liveness probe
2. Wait for a few hours until the pod crashes.

Expected behavior
The container health check should either return a healthy response or produce a descriptive error of the cause for crashing.

Application (please complete the following information):

• Distribution: Docker image
• Version 1.3.1

Log output (if applicable):
This log record is printed just before the pod crashes, but it's probably the sigterm received from k8s causing it -

time="2019-07-15T07:24:58Z" level=error msg="Snyk exporter exited due to error: received os signal 'terminated'" source="main.go:119"

Additional context
Exporter is running fine on v1.2.0. I will try to bisect the issue to a specific release and perhaps narrow it down to a specific feature by gradually incrementing the exporter version and update when I have more info.

It should be expected to see request timeouts once in a while and the application should just ignore these as another scrape will occur in the near future anyway.

Here is an example of the log line just before an exit.

time="2019-02-23T10:34:32Z" level=error msg="Snyk exporter exited due to error: Get https://snyk.io/api/v1/org/UUID/projects: net/http: request canceled (Client.Timeout exceeded while awaiting headers)" source="main.go:83"

The Reporting API under "Get List of Issues" https://snyk.docs.apiary.io/#reference/reporting-api/issues/get-list-of-issues includes a filter "isFixed". This would allow you to calculate the number of fixed vulnerabilities needed for displaying burn down rate.

I highly recommend adding this in. Please let me know if you have any questions regarding this feature request.

Describe the bug
We see that the exporter exits regularly due to some intermediate errors or issues at the Snyk APIs. This is generally an odd behaviour and should instead just be logged, as the exporter is not failing in its operation.

Here is a common error we see due to connection closing:

Get https://snyk.io/api/v1/org/UUID/projects: read tcp IP:36810-\u003eIP:443: read: connection reset by peer

To Reproduce

Add a test case for above error to TestPoll.

{
	name: "tcp connection reset by peer",
	collectorErr: &url.Error{
		Op:  "GET",
		URL: "/url",
		Err: &net.OpError{
			Op:  "read",
			Net: "tcp",
			Addr: &net.IPAddr{
				IP: net.IPv4zero,
			},
			Err: &net.OpError{
				Op:  "read",
				Err: syscall.ECONNRESET,
			},
		},
	},
	output: nil,
},

Expected behavior
The error should just be logged as above but the application should keep on running.

Application (please complete the following information):

• Distribution: Docker image
• Version 1.4.0

Log output (if applicable):

{"level":"error","msg":"Snyk exporter exited due to error: organization NAME (UUID): Get https://snyk.io/api/v1/org/UUID/projects: read tcp IP:36810-\u003eIP:443: read: connection reset by peer","source":"main.go:121","time":"2019-10-04T10:08:23Z"}

Currently, the request to get issues does not filter ignored issues:

postData := issuesPostData{
		Filters: issueFilters{
			Severities: []string{
				"high", "medium", "low",
			},
		},
	}

See the API documentation for more details. I'm working on a fix PR.

Wrapping the binary in a docker container would make the usability of the project better. A simple example of a docker run with the arguments to deploy and run this container.

If you specify an organization through the flags, that name will be used for the metrics. Here you need to specify the organizations original name, opposed to the display name.

We tag metrics with the original name in these cases.

If you run the exporter without the organization flag, we export all organizations related to the token. Here we lavel metrics with the display name instead which can lead to confusing behaviour.

snyk_vulnerabilities_total{organization="my-org",project="my-app",severity="low",type="Sandbox (chroot) Escape"} 2.0
snyk_vulnerabilities_total{organization="my-org",project="My App",severity="low",type="Sandbox (chroot) Escape"} 2.0

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

• Update actions/setup-go action to v4
• Update goreleaser/goreleaser-action action to v4

Edited/Blocked

These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.

• Update external go packages (github.com/prometheus/common, gopkg.in/alecthomas/kingpin.v2)

Other Branches

These updates are pending. To force PRs open, click the checkbox below.

• Update actions/checkout action to v3.5.3

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• Update module github.com/prometheus/client_golang to v1.11.1 [SECURITY]
• Update alpine Docker tag to v3.18.3
• Update golang Docker tag to v1.21.0
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

Some might argue that it is not a good practice to read sensitive information from an env var. What do you think on reading the API key from a config file?

Hi,

After ignoring a vulnerability the exporter adds another gauge with the ignore: true label, but doesn't remove the old metric with ignore: false.

It requires server restart to get only the updated metric.

Adding metadata like the team defined as owner of the repository could be interesting, what do you think?

We are currently using the new reporting Snyk has pushed to their customers (https://apidocs.snyk.io/experimental?version=2022-11-15%7Eexperimental#overview). Is there plans on adding this for Snyk Exporter?

For easy kubernetes monitoring, we need to provide readyness and liveness probes. The /healthz should always return true, I would like to have /ready that returns true only if the initial scrap completed. So the pod will get traffic only when it finish scarping, and expose the correct metrics

How can I publish this on kubernetes cluster? Is there a chart that I can use?

Hi. I just skimmed the code, so apologies if I missed something, but it looks like you might want to add in pagination support against the Snyk API, particularly around issues.

See page and perPage in https://snyk.docs.apiary.io/#reference/reporting-api/issues/get-list-of-issues

Thank you for the project!

We should setup a machine user in GitHub to provide the token for releases on Travis CI instead of using my personal API token.

Snyk API exposes 2 properties that can help devs understand if the issue is actionable - isUpgradeable and isPatchable. I think this can be nice to expose them also on the exporter.