grafana / github-datasource Goto Github PK

View Code? Open in Web Editor NEW

228.0 133.0 54.0 96.21 MB

Grafana data source plugin using the Github API to retrieve and visualize Github data.

Home Page: https://grafana.com/grafana/plugins/grafana-github-datasource/

License: Apache License 2.0

JavaScript 1.61% Go 67.45% TypeScript 28.82% Shell 1.27% Dockerfile 0.85%

grafana github

github-datasource's Introduction

Grafana GitHub datasource

The GitHub datasource allows GitHub API data to be visually represented in Grafana dashboards.

GitHub API V4 (GraphQL)

This datasource uses the githubv4 package, which is under active development.

Features

Backend

Frontend

Visualize queries
Template variables
Annotations

Caching

Caching on this plugin is always enabled.

Configuration

Options:

Setting	Required
Access token	true
Default Organization	false
Default Repository	true
GitHub Enterprise URL	false

To create a new Access Token, navigate to Personal Access Tokens and press Generate new token.

Provisioning

It’s possible to configure data sources using config files with Grafana’s provisioning system.

With the prom-operator

promop:
  grafana:
    additionalDataSources:
      - name: GitHub Repo Insights
        type: grafana-github-datasource
        jsonData:
          owner: ''
          repository: ''
        secureJsonData:
          accessToken: '<github api token>'

Annotations

Annotations overlay events on a graph.

With annotations, you can display:

Commits
Issues
Pull Requests
Releases
Tags

on a graph.

All annotations require that you select a field to display on the annotation, and a field that represents the time that the event occurred.

Variables

Variables allow you to substitute values in a panel with pre-defined values.

You can reference them inside queries, allowing users to configure parameters such as Query or Repository.

Macros

You can use the following macros in your queries

Macro Name	Syntax	Description	Example
multiVar	`$__multiVar(prefix,$var)`	Expands a multi value variable into github query string	`$__multiVar(label,$labels)` will expand into `label:first-label label:second-label`
		When using all in multi variable, use * as custom all value
day	`$__toDay(diff)`	Returns the day according to UTC time, a difference in days can be added	`created:$__toDay(-7)` on 2022-01-17 will expand into `created:2022-01-10`

Access Token Permissions

For all repositories:

public_repo
repo:status
repo_deployment
read:packages
read:user
user:email

For Github projects:

read:org
read:project

An extra setting is required for private repositories

repo (Full control of private repositories)

Sample Dashboard

For documentation on importing dashboards, check out the documentation on grafana.com

The sample dashboard can be obtained from either of two places.

From the Grafana dashboards page located here
From this repository

If loading it from this repository, open Grafana and click "Import Dashboard".

Copy the JSON in ./src/dashboards/dashboard.json, and paste it into the "Import via panel json" box.

Frequently Asked Questions

I am using GitHub OAuth on Grafana. Can my users make requests with their individual GitHub accounts instead of a shared access_token?

No. This requires changes in Grafana first. See this issue in the Grafana project.

Why does it sometimes take up to 5 minutes for my new pull request / new issue / new commit to show up?

We have aggressive caching enabled due to GitHub's rate limiting policies. When selecting a time range like "Last hour", a combination of the queries for each panel and the time range is cached temporarily.

Why are there two selection options for Pull Requests and Issue times when creating annotations?

There are two times that affect an annotation:

The time range of the dashboard or panel
The time that should be used to display the event on the graph

The first selection is used to filter the events that display on the graph. For example, if you select "closed at", only events that were "closed" in your dashboard's time range will be displayed on the graph.

The second selection is used to determine where on the graph the event should be displayed.

Typically these will be the same, however there are some cases where you may want them to be different.

github-datasource's People

Contributors

Stargazers

Watchers

github-datasource's Issues

Use QueryTypeMux instead of map with QueryTypeFunc

https://pkg.go.dev/github.com/grafana/[email protected]/backend/datasource?tab=doc#QueryTypeMux

Allow users to query Projects

Add ability to see all pull requests regardless of time

This has the possibility of maxing out a a user's allowance on their access token if used without care.

Solution:

Add a "None" option for "PullRequestTimeField"
When querying pull requests, if "None" is selected then do not add it to the query.

Add Tags QueryEditor

Scoped variables are not being used

This means repeated panels are not functioning.

Causes #50

Github v1.0.0

v1.0.0 Release

Filters which are nullable or "possibly undefined" should have a toggle that enables or disables them

Problem: When changing inputs from populated to empty, the query fails to complete.

Reproducing:

Step 1: Search for issues with a valid assignee.

Step 2: Removing the content from the "assignee" field should result in an error:

Could not find an assignee with the login ''

This is caused by the behavior of the omitempty flag on the IssueFilters type.

For more information on omitempty: https://golang.org/pkg/encoding/json/

Solution: Filters which are nullable or "possibly undefined" should have a toggle that enables or disables them.

rxjs upgrade & migrate from `toPromise()`

rxjs has deprecated toPromise() and we use it in 2 or 3 different places in datasource.ts

Query type dropdown label is "Pull_Requests" and not "Pull Request"

We probably have the regex replace flipped.

Add acceptance tests (golden files for dataframes)

Support querying milestones

Is there any way to create a line chart of number of PRs / Commits over time?

I wanted to do something like the annotations example but plot commits / pr over time

Implement HealthCheck

Listing repositories doesn't find all of them

This is a follow-up to #52, I tested creating a dashboard variable for repositories of my team within the organization (using regex /^myteam-.*/), and I noticed that the resulting variable doesn't contain all of our repos.

I think it's simply because our organization has more than 1.5k repos currently, and I presume that the current approach limits the amount of repos it can return.

Let me know if you need some extra info and I will try to collect it for you 🙂

Add PR ID, URL, and Repository to Pull Requests results

When querying pull requests, I would like to present them in a table and use "data link" to make the PR title a link that on click opens the PR on Github in a new browser tab. In general adding url property would probably be easiest to use, but if you could at least expose the PR id then I would be able to re-construct the full URL myself 🙂

Let me know if this makes sense 🙂

Finish implementing default org and repository

Filtering issues by "closed_at" returns no results

Because for some reason they're all showing up as 1754-08-30 16:53:05

test

testing notifications

Milestones should allow filtering by state (OPEN, CLOSED)

Announcement Blog Post

empty datetimes show as "1754-08-30 16:53:05" (empty time.Time)

Times should respond with a *time.Time instead of time.Time

HealthCheck should return a message as well as a status.

Probably just needs to say "OK".

https://github.com/grafana/github-datasource/blob/master/pkg/plugin/datasource.go#L36

`plugin/server.go` and `plugin/datasource.go` are redundant.

We can probably combine the query handlers in the plugin package with the datasource interface and make it easier to add new resource / query types.

Update README about support for template variables

As the README says:

We have aggressive caching enabled due to GitHub's rate limiting policies. When selecting a time range like "Last hour", a combination of the queries for each panel and the time range is cached temporarily.

I am trying to use a template variable in the "Query" field and it's not working

Template variables are currently not supported outside of the "Owner / Organization" and "Repository" fields.

I see it's done (#14, #29), but I feel like the FAQ on the README is outdated. I just tested and it works fine having a "milestone" variable addressed on Query field, as the prints show:

If you guys think this is a good idea, I can open a PR. We could just remove this FAQ item and add extra info about the support in the "variables" section.

Add contribution guide

Add QueryEditor

The QueryEditor allows users to create queries for displaying GitHub data.

It should:

Support variables in all query fields

Add support for GitHub Enterprise

The underlying GitHub library includes support for GitHub Enterprise (https://github.com/shurcooL/githubv4#authentication). Adding support should consist of adding an option to configure the GitHub Enterprise URL and creating the Enterprise Client when this is set.

Ability to search multiple repositories for PR query

Hi @kminehart,

You mentioned in #50 (comment) that it might be possible to query multiple repositories per PR query, do you know if this is something that is impossible to do, or there is still some chance?

If not, could this perhaps be done on the datasource side?

The reason I'm asking is that we are a huge org with many teams and tons of repositories, and it actually is more interesting to draw some charts on per-team level, i.e. how many PRs are currently active, how many have been closed over time, etc., and even within a team we just want to have an overview of which PRs still require a review or merge, we aren't as interested in per-repo granularity.

Using dashboard variables and repeating a table kinda works, but the dashboard becomes quite large, if we have 20+ repos and only 2 repos have active PRs, and so the entire dashboard consists of empty tables... Instead I would really like to just see one table with all the PRs (where repo is just a column), and also have a single stat with number of open PRs per entire team.

What do you think? 🙂

Add support for annotations

Support querying pull requests

Support repeating repository by variable

Hi there!

I would like to have a simple table showing open PRs in all repos of my team within the organization.

At first I was hoping I could just specify a matching rule for repo name (all our repos start with myteam-* prefix), either specifying myteam-* as a repo name, or leaving repo name blank and using something like myteam in:name.

That didn't seem to work, so I tried using "Repeat" as an alternative, I would create a dashboard variable repo with values repo1,repo2, use $repo in the query and set the table to repeat on repo variable. This almost seems to work, if I look into "Query inspector" I can see that in the request repository: "{repo1,repo2}", and I think that curly brackets are perhaps to blame here.

What do you think, could this be supported, or am I missing something?

When trying to setup datasource getting a cannot read property 'SecretFormField' of undefined error

TypeError: Cannot read property 'SecretFormField' of undefined
at t.render (ConfigEditor.tsx:67)
at Ka (react-dom.production.min.js:181)
at Xa (react-dom.production.min.js:180)
at Ou (react-dom.production.min.js:261)
at vs (react-dom.production.min.js:230)
at ps (react-dom.production.min.js:229)
at os (react-dom.production.min.js:223)
at react-dom.production.min.js:121
at e.unstable_runWithPriority (scheduler.production.min.js:18)
at Hi (react-dom.production.min.js:120)

I've installed the plugin and when I try to add github as a datasource I get the above error in the chrome console and the following page. Are there any other steps to be performed before the datasource can be configured other than installing the plugin?

My setup is:
Grafana running in a docker container
Version: 6.7.2

Support querying packages

Screenshots are not showing up on grafana.com

Documentation for 1.0.0 release

Include "Approved" flag in Pull Requests

Add ability to use Search endpoint for Issues

[Feature Request]: Graphing data over time

Would love the ability to have running totals of open/closed issues over time to be able to graph this kind of data. I tried multiple ways using transforms of the table data, but nothing I did seem to work out the way I anticipated. Tried group by transform with several different calculation options and only came out to a single counted item.