Git Product home page Git Product logo

github-actions-exporter's Introduction

Change of organization

The project was migrated to another organization because it was no longer maintained. In the coming weeks, changes will be made on various points (code and container image) and also an update of all the libraries. If there are feature requests, you can make them in the issues. For a better understanding of the requested features, having a detailed description or example will be greatly appreciated.

github-actions-exporter

github-actions-exporter for prometheus

Release pipeline Go Report Card

Container image : https://github.com/Labbs/github-actions-exporter/pkgs/container/github-actions-exporter

Helm Chart :

 helm chart repo: https://labbs.github.io/helm-charts/
 chart: github-actions-exporter
 version: "0.1.4"

Information

If you want to monitor a public repository, you must put the public_repo option in the repo scope of your github token or Github App Authentication.

Authentication

Authentication can either via a Github Token or the Github App Authentication 3 parameters. When installing via the Helm Chart the authentication is provided via a secret.

Options

Name Flag Env vars Default Description
Github Token github_token, gt GITHUB_TOKEN - Personnel Access Token
Github App Id app_id, gai GITHUB_APP_ID Github App Authentication App Id
Github App Installation Id app_installation_id, gii GITHUB_APP_INSTALLATION_ID - Github App Authentication Installation Id
Github App Private Key app_private_key, gpk GITHUB_APP_PRIVATE_KEY - Github App Authentication Private Key
Github Refresh github_refresh, gr GITHUB_REFRESH 30 Refresh time Github Actions status in sec
Github Organizations github_orgas, go GITHUB_ORGAS - List all organizations you want get informations. Format <orga1>,<orga2>,<orga3> (like test1,test2)
Github Repos github_repos, grs GITHUB_REPOS - [Optional] List all repositories you want get informations. Format <orga>/<repo>,<orga>/<repo2>,<orga>/<repo3> (like test/test). Defaults to all repositories owned by the organizations.
Exporter port port, p PORT 9999 Exporter port
Github Api URL github_api_url, url GITHUB_API_URL api.github.com Github API URL (primarily for Github Enterprise usage)
Github Enterprise Name enterprise_name ENTERPRISE_NAME "" Enterprise name. Needed for enterprise endpoints (/enterprises/{ENTERPRISE_NAME}/*). Currently used to get Enterprise level tunners status
Fields to export export_fields EXPORT_FIELDS repo,id,node_id,head_branch,head_sha,run_number,workflow_id,workflow,event,status A comma separated list of fields for workflow metrics that should be exported

Exported stats

github_workflow_run_status

Gauge type

Result possibility

ID Description
0 Failure
1 Success
2 Skipped
3 In Progress
4 Queued

Fields

Name Description
event Event type like push/pull_request/...
head_branch Branch name
head_sha Commit ID
node_id Node ID (github actions) (mandatory ??)
repo Repository like <org>/<repo>
run_number Build id for the repo (incremental id => 1/2/3/4/...)
workflow_id Workflow ID
workflow Workflow Name
status Workflow status (completed/in_progress)

github_workflow_run_duration_ms

Gauge type

Result possibility

Gauge Description
milliseconds Number of milliseconds that a specific workflow run took time to complete.

Fields

Name Description
event Event type like push/pull_request/...
head_branch Branch name
head_sha Commit ID
node_id Node ID (github actions) (mandatory ??)
repo Repository like <org>/<repo>
run_number Build id for the repo (incremental id => 1/2/3/4/...)
workflow_id Workflow ID
workflow Workflow Name
status Workflow status (completed/in_progress)

github_job

⚠️ This is a duplicate of the github_workflow_run_status metric that will soon be deprecated, do not use anymore.

github_runner_status

Gauge type (If you have self hosted runner)

Result possibility

ID Description
0 Offline
1 Online

Fields

Name Description
id Runner id (incremental id)
name Runner name
os Operating system (linux/macos/windows)
repo Repository like <org>/<repo>
status Runner status (online/offline)
busy Runner busy or not (true/false)

github_runner_organization_status

Gauge type (If you have self hosted runner for an organization)

Result possibility

ID Description
0 Offline
1 Online

Fields

Name Description
id Runner id (incremental id)
name Runner name
os Operating system (linux/macos/windows)
orga Organization name
status Runner status (online/offline)
busy Runner busy or not (true/false)

github_runner_enterprise_status

Gauge type (If you have self hosted runner for an enterprise)

Result possibility

ID Description
0 Offline
1 Online

Fields

Name Description
id Runner id (incremental id)
name Runner name
os Operating system (linux/macos/windows)

github_workflow_usage_seconds

Gauge type (If you have private repositories that use GitHub-hosted runners)

Result possibility

Gauge Description
seconds Number of billable seconds used by a specific workflow during the current billing cycle.

Fields

Name Description
id Workflow id (incremental id)
node_id Node ID (github actions)
name workflow name
os Operating system (linux/macos/windows)
repo Repository like <org>/<repo>
status Workflow status

Example:

# HELP github_workflow_usage Number of billable seconds used by a specific workflow during the current billing cycle. Any job re-runs are also included in the usage. Only apply to workflows in private repositories that use GitHub-hosted runners.
# TYPE github_workflow_usage gauge
github_workflow_usage_seconds{id="2862037",name="Create Release",node_id="MDg6V29ya2Zsb3cyODYyMDM3",repo="xxx/xxx",state="active",os="UBUNTU"} 706.609

Setting up authentication with GitHub API

There are two ways for github-actions-exporter to authenticate with the GitHub API (only 1 can be configured at a time however):

  1. Using a GitHub App (not supported when you use Github Enterprise )
  2. Using a Personal Access Token

Functionality wise, there isn't much of a difference between the 2 authentication methods. The primarily benefit of authenticating via a GitHub App is an increased API quota.

If you are deploying the solution for a GitHub Enterprise Server environment you are able to configure your rate limiting settings making the main benefit irrelevant. If you're deploying the solution for a GitHub Enterprise Cloud or regular GitHub environment and you run into rate limiting issues, consider deploying the solution using the GitHub App authentication method instead.

Deploying using GitHub App Authentication

You can create a GitHub App for either your account or any organization. If you want to create a GitHub App for your account, open the following link to the creation page, enter any unique name in the "GitHub App name" field, and hit the "Create GitHub App" button at the bottom of the page.

If you want to create a GitHub App for your organization, replace the :org part of the following URL with your organization name before opening it. Then enter any unique name in the "GitHub App name" field, and hit the "Create GitHub App" button at the bottom of the page to create a GitHub App.

Github Token configuration

Scopes needed configuration for the Github token

repo
  - repo:status
  - repo_deployment
  - public_repo

admin:org
  - write:org
  - read:org

Authentication Errors

Invalid Github Token

if token is invalid then 401 Bad credentials will be returned on github API error and displayed in an error message.

Invalid Github App configuration

if the app id or app installation id value is incorrect then messages like the following are displayed:

could not refresh installation id 12345678's token: request &{Method:POST URL:https://api.github.com/app/installations/12345678/access_tokens

if the github_app_private_key is incorrect then errors like the following are displayed.

 Error: Client creation failed.authentication failed: could not parse private key: Invalid Key: Key must be PEM encoded PKCS1 or PKCS8 private ke

Secret actions-exporter

In the kubernetes deployment authentication is passed via a kubernetes secret:

kind: Secret
apiVersion: v1
metadata:
  name: actions-exporter
  namespace: github-actions-exporter
type: Opaque
data:
  github_token: AAAAAA
#  github_app_id: BBBBBB
#  github_app_installation_id: CCCCCCCCC
#  github_app_private_key: DDDDDDD

Or more probably using an external secret manager. Here is an example of using External Secrets with the EKS Secret Manager to define the authentication in a secret:

apiVersion: 'kubernetes-client.io/v1'
kind: ExternalSecret
metadata:
  name: actions-exporter
  namespace: github-actions-exporter
spec:
  backendType: secretsManager
  data:
 #   - key: MySecretManagerKey
 #     name: github_token
 #     property: github_token
    - key: MySecretManagerKey
      name: github_app_id
      property: github_app_id
    - key: MySecretManagerKey
      name: github_app_installation_id
      property: github_app_installation_id
  # separate plaintext aws secret needed for ssh key
    - key: MySecretManagerKeyPrivateKey
      name: github_app_private_key

github-actions-exporter's People

Contributors

dennybaa avatar derektbrown avatar goobaroo avatar jstewart612 avatar mla-spendesk avatar mom0tomo avatar moutonjeremy avatar neillturner avatar paologallinaharbur avatar r-savcenko avatar rsvoboda avatar tarasovych avatar vdemonchy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

github-actions-exporter's Issues

Missing github metrics

I recently run this exporter but do not see any metrics for github actions (please find the below ). Is it expected output?

HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles.

TYPE go_gc_duration_seconds summary

go_gc_duration_seconds{quantile="0"} 1.64e-05
go_gc_duration_seconds{quantile="0.25"} 2.23e-05
go_gc_duration_seconds{quantile="0.5"} 3.07e-05
go_gc_duration_seconds{quantile="0.75"} 4.15e-05
go_gc_duration_seconds{quantile="1"} 0.0001462
go_gc_duration_seconds_sum 0.1766211
go_gc_duration_seconds_count 4872

HELP go_goroutines Number of goroutines that currently exist.

TYPE go_goroutines gauge

go_goroutines 14

HELP go_info Information about the Go environment.

TYPE go_info gauge

go_info{version="go1.16.7"} 1

HELP go_memstats_alloc_bytes Number of bytes allocated and still in use.

TYPE go_memstats_alloc_bytes gauge

go_memstats_alloc_bytes 3.349072e+06

HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed.

TYPE go_memstats_alloc_bytes_total counter

go_memstats_alloc_bytes_total 1.2189910928e+10

HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table.

TYPE go_memstats_buck_hash_sys_bytes gauge

go_memstats_buck_hash_sys_bytes 1.464648e+06

HELP go_memstats_frees_total Total number of frees.

TYPE go_memstats_frees_total counter

go_memstats_frees_total 1.64109421e+08

HELP go_memstats_gc_cpu_fraction The fraction of this program's available CPU time used by the GC since the program started.

TYPE go_memstats_gc_cpu_fraction gauge

go_memstats_gc_cpu_fraction 0.007794144909621236

HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata.

TYPE go_memstats_gc_sys_bytes gauge

go_memstats_gc_sys_bytes 4.777744e+06

HELP go_memstats_heap_alloc_bytes Number of heap bytes allocated and still in use.

TYPE go_memstats_heap_alloc_bytes gauge

go_memstats_heap_alloc_bytes 3.349072e+06

HELP go_memstats_heap_idle_bytes Number of heap bytes waiting to be used.

TYPE go_memstats_heap_idle_bytes gauge

go_memstats_heap_idle_bytes 6.1956096e+07

HELP go_memstats_heap_inuse_bytes Number of heap bytes that are in use.

TYPE go_memstats_heap_inuse_bytes gauge

go_memstats_heap_inuse_bytes 4.46464e+06

HELP go_memstats_heap_objects Number of allocated objects.

TYPE go_memstats_heap_objects gauge

go_memstats_heap_objects 23378

HELP go_memstats_heap_released_bytes Number of heap bytes released to OS.

TYPE go_memstats_heap_released_bytes gauge

go_memstats_heap_released_bytes 5.9817984e+07

HELP go_memstats_heap_sys_bytes Number of heap bytes obtained from system.

TYPE go_memstats_heap_sys_bytes gauge

go_memstats_heap_sys_bytes 6.6420736e+07

HELP go_memstats_last_gc_time_seconds Number of seconds since 1970 of last garbage collection.

TYPE go_memstats_last_gc_time_seconds gauge

go_memstats_last_gc_time_seconds 1.6558920931608884e+09

HELP go_memstats_lookups_total Total number of pointer lookups.

TYPE go_memstats_lookups_total counter

go_memstats_lookups_total 0

HELP go_memstats_mallocs_total Total number of mallocs.

TYPE go_memstats_mallocs_total counter

go_memstats_mallocs_total 1.64132799e+08

HELP go_memstats_mcache_inuse_bytes Number of bytes in use by mcache structures.

TYPE go_memstats_mcache_inuse_bytes gauge

go_memstats_mcache_inuse_bytes 19200

HELP go_memstats_mcache_sys_bytes Number of bytes used for mcache structures obtained from system.

TYPE go_memstats_mcache_sys_bytes gauge

go_memstats_mcache_sys_bytes 32768

HELP go_memstats_mspan_inuse_bytes Number of bytes in use by mspan structures.

TYPE go_memstats_mspan_inuse_bytes gauge

go_memstats_mspan_inuse_bytes 172856

HELP go_memstats_mspan_sys_bytes Number of bytes used for mspan structures obtained from system.

TYPE go_memstats_mspan_sys_bytes gauge

go_memstats_mspan_sys_bytes 212992

HELP go_memstats_next_gc_bytes Number of heap bytes when next garbage collection will take place.

TYPE go_memstats_next_gc_bytes gauge

go_memstats_next_gc_bytes 4.194304e+06

HELP go_memstats_other_sys_bytes Number of bytes used for other system allocations.

TYPE go_memstats_other_sys_bytes gauge

go_memstats_other_sys_bytes 1.983408e+06

HELP go_memstats_stack_inuse_bytes Number of bytes in use by the stack allocator.

TYPE go_memstats_stack_inuse_bytes gauge

go_memstats_stack_inuse_bytes 688128

HELP go_memstats_stack_sys_bytes Number of bytes obtained from system for stack allocator.

TYPE go_memstats_stack_sys_bytes gauge

go_memstats_stack_sys_bytes 688128

HELP go_memstats_sys_bytes Number of bytes obtained from system.

TYPE go_memstats_sys_bytes gauge

go_memstats_sys_bytes 7.5580424e+07

HELP go_threads Number of OS threads created.

TYPE go_threads gauge

go_threads 12

HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.

TYPE process_cpu_seconds_total counter

process_cpu_seconds_total 56.66

HELP process_max_fds Maximum number of open file descriptors.

TYPE process_max_fds gauge

process_max_fds 1.048576e+06

HELP process_open_fds Number of open file descriptors.

TYPE process_open_fds gauge

process_open_fds 10

HELP process_resident_memory_bytes Resident memory size in bytes.

TYPE process_resident_memory_bytes gauge

process_resident_memory_bytes 1.8620416e+07

HELP process_start_time_seconds Start time of the process since unix epoch in seconds.

TYPE process_start_time_seconds gauge

process_start_time_seconds 1.65589204671e+09

HELP process_virtual_memory_bytes Virtual memory size in bytes.

TYPE process_virtual_memory_bytes gauge

process_virtual_memory_bytes 7.34769152e+08

HELP process_virtual_memory_max_bytes Maximum amount of virtual memory available in bytes.

TYPE process_virtual_memory_max_bytes gauge

process_virtual_memory_max_bytes 1.8446744073709552e+19

HELP promhttp_metric_handler_requests_in_flight Current number of scrapes being served.

TYPE promhttp_metric_handler_requests_in_flight gauge

promhttp_metric_handler_requests_in_flight 1

HELP promhttp_metric_handler_requests_total Total number of scrapes by HTTP status code.

TYPE promhttp_metric_handler_requests_total counter

promhttp_metric_handler_requests_total{code="200"} 0
promhttp_metric_handler_requests_total{code="500"} 0
promhttp_metric_handler_requests_total{code="503"} 0

Examples of use?

Does anyone have an example Grafana dashboard(s) they can share or a set of Prometheus alerting rules based on this exporter?

Problem with repetitive data

Hello,

I'm using the collector in a big environment with almost 500 repositories, and we have a lot of actions. I've created a grafana

dashboard to show successfull and failed workflow runs. I have the exporter running every 15 minutes, and the ServiceMonitor gets the data also every 15min.
I notice that the data is always the same, but every new scrape adds the data from the last 15min. So its cumulative.

Its this how it supose to be, or am I implementing the wrong way?

Authentication issue

ISSUE

If a wrong token is passed currently no warning message is shown and the exporter will be carry on running as if everything was working correctly.

Expected Behaviour

The exporter should log the issue and exit

Constant crashes when a lot of gh actions

With too many (20+) gh actions in place, exporter started to crash with the following error:

2022/08/08 12:03:47 authenticating with Github Token
2022/08/08 12:03:48 exporter listening on 0.0.0.0:9999
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x99451b]

goroutine 45 [running]:
github-actions-exporter/pkg/metrics.getFieldValue(0xc00002600d, 0x12, 0xc0001c8520, 0xc000253200, 0xc000253210, 0xc000253220, 0xc000253230, 0xc0001c8528, 0xc000253240, 0xc000253250, ...)
	/app/pkg/metrics/get_workflow_runs_from_github.go:32 +0x27b
github-actions-exporter/pkg/metrics.getRelevantFields(0xc00002600d, 0x12, 0xc0006b3180, 0x5, 0xb7c9e0, 0xc000234a40)
	/app/pkg/metrics/get_workflow_runs_from_github.go:46 +0x18e
github-actions-exporter/pkg/metrics.getWorkflowRunsFromGithub()
	/app/pkg/metrics/get_workflow_runs_from_github.go:72 +0xea
created by github-actions-exporter/pkg/metrics.InitMetrics
	/app/pkg/metrics/metrics.go:64 +0x52f

Runner status reporting is broken

The github runner status reporting has the following issues:

  • Data overlaps and becomes useless due to the above
  • Is not organized well for use in prometheus alerting

An example /metrics snippet from our actual environment:

github_runner_organization_status{id="720",name="github-actions-runners-tkkjp-qqscv",organization="foo",os="linux",status="offline"} 0
github_runner_organization_status{id="720",name="github-actions-runners-tkkjp-qqscv",organization="foo",os="linux",status="online"} 1

Right now, there are, effectively, two separate metrics for reporting the status of each runner. One for offline, and one for online. The client does not remove the previous metric entry when the state changes, so you end up reporting both states simultaneously, and the offline metric makes no sense because foo{status="offline"}==0 should mean that "it is not offline", which is not the case right now. And you cannot tell which metric is current.

Recommended Fix Version 1

Since there appears to be only two states, the easiest way to do this would be to instead implement a simple "runner is online" metric:

runner_online{name="{{name}}",...} = (r.status == "online")

I'm not a github expert; this won't work in and of itself if there is more than online and offline to the runner statuses.

Recommended Fix Version 2

You could report a metric for each possible runner status, which would be a bit more flexible if github implements more than "online" and "offline". And always make the value 1 for "this state is true" or 0 for "this state is false":

runner_status{name="{{name}}",...,status="offline"} = (r.status == "offline")
runner_status{name="{{name}}",...,status="online") = (r.status == "online")

Grafana Dashboards with GitAction runner metrics

I have a self-hosted Runner configured on top of EKS, and I want to use the Github Runner metrics to monitor how well the data is being served from the queue.

I'm trying to configure Graphana with that metric, but do you have any examples of dashboards based on histograms or timestamps?

It would be very helpful if you could provide one :)

Irregularities in github_workflow_run_status

Hi!

According to the documentation the expected values from github_workflow_run_status is either 0 (failed), 1 (success), 2 (skipped) or 3 (in_progress). When we look at the metrics we only see values of 0 or 1 even though the runners have been observed with other states. The runners show the state long enough to expect the metric to change the value to 3.

Running $ gh run list --json name,status,conclusion reveals the problem. Seems the conclusion-field is only updated by GitHub Actions when the status-field is set to "completed".

  {
    "conclusion": "",
    "name": "Build repo",
    "status": "queued"
  },
  {
    "conclusion": "",
    "name": "Build repo",
    "status": "in_progress"
  },
  {
    "conclusion": "",
    "name": "Build repo",
    "status": "pending"
  },
  {
    "conclusion": "success",
    "name": "Build repo",
    "status": "completed"
  }

Looking at the code it seems the conclusion-field is used to set the value of the metric which explains why we only observe values of 0 or 1. Also 0 is used as a "catch all other status-variants" it seems?
I think it would be better to split this into separate metrics per status instead. That way you'll get a total number for metrics with the state "in_progress" for instance. github_workflow_run_in_progress with a counter of number of runners with that state so it's possible to sum+rate the metric.

I noticed it's a long time since the last release and any activity in general on the project, so I'm also wondering if this project is considered abandoned or just lacking feedback from users in general?

Missing Metrics

I am only seeing the metric for:
github_runner_organization_status

I have set up my env as following:

- name: GITHUB_ORGAS
  value: AAA
- name: GITHUB_TOKEN
  value: BBB
- name: PORT
  value: "8080"

My PAT is set up with the authorization listed in the documentation.

I am only using the hosted runners now. Wondering why I am not seeing my workflow statuses?

Am I just plain doing something wrong?

GHE enterprise runner support

Hi,
need your suggestion on below topic.
is this exporter support to get matrix from enterprise default runner in GHE .
we have installed it and its not showing anything on pod log . or if we are missing something.
could you pls suggest.

k logs -f action-export-github-actions-exporter-7ccfcd957-v8xzr
2022/08/15 17:13:32 authenticating with Github Token
2022/08/15 17:13:32 exporter listening on 0.0.0.0:9999

Org level write permissions

I see this app needs write permissions at the org level.

repo
  - repo:status
  - repo_deployment
  - public_repo

admin:org
  - write:org
  - read:org

Can I get details as to why this is needed?

Get all repository workflows

This is really great, although I would rather not have to list the ~70 repositories we have.

Would it be possible to make it so when GITHUB_REPOS is empty, that means all repos for the organisation?

I saw #7, but wasn't sure what the answer meant.

bug: workflow_id is missing

  • github_job{event="push",head_branch="master",head_sha="0144e8d5a573c5ea9a5b1d92f597e3318310fb6c",id="497499385",instance="localhost:9183",job="github_actions",node_id="MDExOldvcmtmbG93UnVuNDk3NDk5Mzg1",repo="unicode-org/cldr",run_number="1120",status="completed"} | 1
  • github_job{event="push",head_branch="master",head_sha="0144e8d5a573c5ea9a5b1d92f597e3318310fb6c",id="497499386",instance="localhost:9183",job="github_actions",node_id="MDExOldvcmtmbG93UnVuNDk3NDk5Mzg2",repo="unicode-org/cldr",run_number="660",status="completed"} | 1

I just realized that the workflow_id is missing. So as you see above, I get two duplicate entries.

Ideally the workflow's name would be given, not just the numeric id (which isn't very helpful)

Need date and time of workflow build run information

Currently the exporter supports workflows status like success and failures (owner, repo, workflow, event, workflow name, status, branch, workflow id number, run number, run number id.).

But this exporter doesn’t support when the workflow is triggered like date and time. It is helpful to add workflow trigger date and time feature on this exporter. Could you pls help on this.

Runtime error after the exporter successfully starts and fetches workflows

We have the github actions exporter running inside ECS, using the latest image from ghcr.io/labbs/github-actions-exporter:latest. It worked well until last night but now it started to fail. The task starts well, it authenticates with github and fetches multiple workflows from the repositories, but then after 1 min, it fails with the error below. We think it might have been broken by something externally as it worked fine until now

   2024-03-14T10:15:38.552+02:00  2024/03/14 08:15:38 exporter listening on 0.0.0.0:9999
   2024-03-14T10:18:16.265+02:00  panic: runtime error: invalid memory address or nil pointer dereference
   2024-03-14T10:18:16.265+02:00  [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x936780]
   2024-03-14T10:18:16.265+02:00  goroutine 355 [running]:
   2024-03-14T10:18:16.265+02:00  github.com/spendesk/github-actions-exporter/pkg/metrics.getFieldValue({_, _}, {0xc002b58550, 0xc002c04f60, 0xc002c04f70, 0x0, 0xc002c04f80, 0xc002b58558, 0xc002b585e0, 0xc002c04f90, ...}, ...)
   2024-03-14T10:18:16.265+02:00  /app/pkg/metrics/get_workflow_runs_from_github.go:25 +0x420
   2024-03-14T10:18:16.265+02:00  github.com/spendesk/github-actions-exporter/pkg/metrics.getRelevantFields({0xc000339ec0, 0x26}, 0xc002803560)
   2024-03-14T10:18:16.265+02:00  /app/pkg/metrics/get_workflow_runs_from_github.go:57 +0x177
   2024-03-14T10:18:16.265+02:00  github.com/spendesk/github-actions-exporter/pkg/metrics.getWorkflowRunsFromGithub()
   2024-03-14T10:18:16.265+02:00  /app/pkg/metrics/get_workflow_runs_from_github.go:125 +0x2ac
   2024-03-14T10:18:16.265+02:00  created by github.com/spendesk/github-actions-exporter/pkg/metrics.InitMetrics
   2024-03-14T10:18:16.265+02:00  /app/pkg/metrics/metrics.go:67 +0x471

Example of the values.yml file

Hello,
Can someone please supply an example for the values.yaml file for this chart?
I'm struggling with the GH authentication when trying to set the flags and secrets in place.

I've tried to set the flags for app id, installation id and private key when installing the chart but looks like the chart is not creating the "actions-exporter" secrets.

Please advise.

need cumulative counters

SO this is what I wanted to query: rate(count(github_workflow_run_status)[30m])

but it doesn't work, maybe because count() is an aggregate function(?). Probably better would be if there's an inbuilt cumulative counter for workflows of different status:

github_workflow_run_status_total{state="success"} - incremented on success
github_workflow_run_status_total{state="started"} - incremented when job is started (or if an otherwise-unknown job is seen)
github_workflow_run_status_total{state="error"}- incremented on err
github_workflow_run_status_total{state="queued"} - incremented when job is queued, not decremented when it leaves the queue
github_workflow_run_status_total{state="in_progress"} - incremented when job is in progress, NOT decremented when it completes

If jobs go from error/success back to started, then let them start incrementing again.

Exporter is not using exponential backoff/maxRetries on GitHub API-Calls

As the github api exerts a rate limit, and the exporter does not respect that which can cause delays in metric delivery.

Expected Behavior

The Exporter should respect a configurable ratelimit to not exhaust all requests at once.

Current Behavior

On an recurring error, such as wrongly configured creds (wrong permissions) the exporter just depletes all of its requests in a matter of seconds.
For what ever reason this infinite loop does not stop when it hits an error
https://github.com/Spendesk/github-actions-exporter/blob/ec79f05a21c132a0c27f546c4955a159a3721baa/pkg/metrics/get_runners_organization_from_github.go#L29-L46

Possible Solution

Add a ratelimiting RoundTripper to the github client, like this one https://gist.github.com/zdebra/10f0e284c4672e99f0cb767298f20c11
This would also eleviate the need for the caching layer

Steps to Reproduce

  1. Create a GH token with wrong permissons (or anything that reliably produces an error with the gh client)
  2. See it flood the GH API

Context (Environment)

Just me being stoopid and not configureing a proper token, but this could have happend on any error with the github client.
Lets say you alert based on metrics provided my this exporter, any error that would cause the rate limit to be reached would cause alerts.

Collecting metrics for only latest workflow runs

I find that I cannot setup alerting based on the status of the latest pipeline runs. Is it possible to setup tracking for run numbers so that only the metrics for runs since the last Prometheus scrape are exposed? Persisting a pointer or using a key-value store may be helpful in this regard.

skip “runners” if no perms

i don't have any hosted runners, but wanted to keep an eye on the workflow status.

Using this patch locally. Allows use with a find grained token.

diff --git a/pkg/metrics/metrics.go b/pkg/metrics/metrics.go
index c9206d8..e375193 100644
--- a/pkg/metrics/metrics.go
+++ b/pkg/metrics/metrics.go
@@ -59,10 +59,10 @@ func InitMetrics() {
        }

        go getBillableFromGithub()
-       go getRunnersFromGithub()
-       go getRunnersOrganizationFromGithub()
+       //go getRunnersFromGithub()
+       //go getRunnersOrganizationFromGithub()
        go getWorkflowRunsFromGithub()
-       go getRunnersEnterpriseFromGithub()
+       //go getRunnersEnterpriseFromGithub()
 }

 // NewClient creates a Github Client

Helm Chart Repo not found

The Helm chart repo mentioned in the readme should be updated to

https://labbs.github.io/github-actions-exporter/

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.