Comments (9)
No, I don't believe you'll be able to accomplish any meaningful historical data yet with Grafana transformations, even with the improvements recently released in 7.2. This isn't really Grafana's fault, but really just because the data isn't there.
For me I would want to be able to see a graph of issue counts over time, grouped by closed status. That way I can see both the increase in issues as well as the decrease as things get fixed/answered.
This is totally doable and I'll look at implementing something like this over the next few days. There's a lot that has to go into this to do it properly, in my opinion, so I want to make sure I get it right the first time so we can do the same thing for pull requests / commits / etc.
I have several repos to monitor, and having the ability to have all of them transposed on each other would be a bonus
This is something the datasource could likely handle and has already been requested more than once. I'm sure I can make something there work. :)
from github-datasource.
This is totally doable and I'll look at implementing something like this over the next few days. There's a lot that has to go into this to do it properly, in my opinion, so I want to make sure I get it right the first time so we can do the same thing for pull requests / commits / etc.
Totally understandable, and most importantly thank you for this great plugin. It will simplify my world and I can get rid of the middlemen I'm having to use now with github-to-es and then graphing es data.
from github-datasource.
Unfortunately that's because the GitHub data that's being parsed isn't a timeseries. It has a time, like closed_at or created_at, but no value.
I'm not sure that's accurate is it? The table data I see does show date/time values in those fields, albeit not all of them. The closed_at seems to be incorrect but created_at seems accurate.
from github-datasource.
The current data you're seeing in those tables can't be plotted on a graph. While there is a time for the X axis, there is no value to plot on the Y axis, which is the real meat of this issue. At every graph interval we need to have a value to plot.
Out of curiosity, what would you expect to see when graphing issues? Just so I can get an idea of what aggregations / groupings will be needed :)
from github-datasource.
Yeah, I thought this would be possible with transformations, but the combining of values with group by is what's not getting the expected results. Using Group By with count calculation on a non value field and group by on the closed field does give me value data, but not over time, because once you group by that created_at time it flattens everything back out, so not a running total of those and the value is always 1 (as expected). I'm not a grafana guru so take all this with a grain of salt, but maybe there's a way to overlap transformations to get a running total over time?
For me I would want to be able to see a graph of issue counts over time, grouped by closed status. That way I can see both the increase in issues as well as the decrease as things get fixed/answered. Kind of a performance metric for myself to make sure I'm not lagging behind. I have several repos to monitor, and having the ability to have all of them transposed on each other would be a bonus, but not necessary as I've gotten a variable to work as a repo selector and can quickly swap between them.
from github-datasource.
I've been sort of messing around with this over the last couple of hours. I'll put my thoughts so far here since there's not really a better place.
It definitely provides some interesting challenges. The biggest hurdle is really a combination of the limit of 5,000 requests per hour per access token and the maximum page size of 100
For example, if we want to count the total number of commits per user in the grafana/grafana project, we have to query all commits in the project, or:
- 26,000 commits, or
- 260 pages, or
- at least 5.2% of your hourly allowance, and
- 260 sequential HTTP requests. In order for the next request to be made, data from the previous one has to be used (the pagination cursor).
Grafana's codebase isn't a perfect example, I don't know how many projects that have that many commits, but I don't think it's that high of a number to hit.
I figured I could just let users do a normal query, which is typically only 1 or 2 pages, run an extra query to find out how many values came before the time range, and use that number as the starting point. That will likely work in a lot of cases, but it's not perfect... Aggregations that have to actually group or bucket the data won't be so easily calculated and would require traversing through all of the values.
In your example, you want to know the number of closed vs the number of opened, which is doable, as you would have two separate queries with a single static query. If, however, someone wanted to do an aggregation like a "total number of commits grouped by author", I don't see that as being realistic. No one is going to happily wait for 25+ HTTP requests, much less 250.
from github-datasource.
This feature will require to store the results of the GitHub API requests to some persistent storage identified by time of the request, e.g. into the plugin's cache. I am not sure what are the limits of how many data can be in the cache at the moment as I haven't studied the code yet but I am considering to help with implementation of this feature. I'd just like to first ask whether anyone has a closer idea how this feature could be implemented?
from github-datasource.
Grafana Data Sources are really not supposed to store data, and are just supposed to act as a translation layer which is what makes this issue so difficult. In order for this to work, there has to be a way use the data that is there to create a tiemseries. There aren't a whole lot of opportunities for that.
If you're using Grafana Enterprise this is the use-case that influenced us to add the recorded queries feature, which basically allows you to use a grafana data source as a prometheus exporter.
from github-datasource.
Check this thread out
https://community.grafana.com/t/problem-using-group-by-with-unix-datetime-value/86685/8
And this one
There is talk of implementing grouping by pieces of the datetime: day, hour etc
Thanks
from github-datasource.
Related Issues (20)
- Add GitHub followers endpoint
- Migrate @grafana/toolkit to @grafana/plugin-tools
- FR: Add support for GH Discussions
- Improve e2e tests
- Feature Request: Add support for fine-grained GitHub tokens HOT 1
- Feature Request: Add support for code scanning alerts
- Include state enum for Vulnerability Alerts HOT 1
- Monitoring GitHub workflow via Grafana HOT 1
- Feature Request: Add support for GitHub workflow and Actions too HOT 1
- [Grafana Cloud] v1.4.2: The content of this plugin does not match its signature HOT 1
- x509: certificate signed by unknown authority HOT 3
- Feature Request: Support for querying a single issue
- Docs Fixes: Developer Guide Steps Are Not Reproducible
- plugin automatically removed HOT 2
- Add usage tracking
- Create schema model HOT 1
- Retrieve Reviewers and assignee for PRs and issues HOT 2
- Feature Request: Support for Querying multiple Repositories HOT 3
- Project queries are potentially very costly and become unusable due to rate limits HOT 1
- Cannot view private repos HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from github-datasource.