stryker-mutator / stryker-dashboard Goto Github PK
View Code? Open in Web Editor NEWSee your mutation testing reports from anywhere π
Home Page: https://dashboard.stryker-mutator.io/
License: Apache License 2.0
See your mutation testing reports from anywhere π
Home Page: https://dashboard.stryker-mutator.io/
License: Apache License 2.0
Is your feature request related to a problem? Please describe.
My current setup has ~1600 mutants across 9 files. To reduce mutation testing time (takes 12 hours total), I parallelize these in my GitHub workflow (slowest takes 6 hours). Some of these files are in the same folder eg. "App/App_functions.js" and "App/App_helpers.js". During mutation testing the natural module to make for both (in dashboard) is "App", but since they end at different times, one overwrites the other.
Describe the solution you'd like
Specifying a module for dashboard reporter should not overwrite existing files in that module. Maybe include an additional config parameter like "overwriteModule: Boolean" so that the user can decide whether or not that is what they want.
Describe alternatives you've considered
I created different modules for each file, in total I had 9 unique modules for my 9 files - this is not ideal.
Additional context
When creating new modules, the old modules remain in the dashboard and there seems to be no way to clear/remove them so they end up interfering with the mutation score. It would be great if a feature for this was provided as well. Maybe add another config variable in the dashboard reporter that removes specific module names. Eg. "existingModules.names: string[]" // contains module names to consider
"existingModules.actions: {'overwrite', 'clear', 'remove'}" // what to do with these modules
We're requesting too many privileges. We shouldn't need "commit" access.
See also:
https://twitter.com/tfidry/status/1210651591918997508
https://dashboard.stryker-mutator.io/ certificate has expired today : Expired Thursday, May 3, 2018 at 16:35:17 Central European Time (translate from french)
Does Stryker support private repositories? I can't seem to grant access to those when using the app?
When trying to post to the API with an invalid report, it will show the stacktrace, we should probably fix that.
Maybe a combination of both could be done:
This would allow the list to be much more friendly when like me you have 152 repositories but only one enabled
When I access to the dashboard logged in (e.g. https://dashboard.stryker-mutator.io/repos/theofidry
) and click on the stryker icon top left, I get redirected to the following page:
And then it takes a solid 1-2 seconds before redirecting me to https://dashboard.stryker-mutator.io/repos/theofidry
because I'm logged in
We should not have to do this manually.
When logging into the app with GitHub the Stryker dashboard requests access to read and write to all public and private repositories of the user. This doesn't seem necessary.
As an example, Coveralls, which is a somewhat similar type of app, requests the following permissions.
This is a link to the documentation on scopes:
https://developer.github.com/apps/building-oauth-apps/understanding-scopes-for-oauth-apps/#available-scopes
The scope "repo:status" can be used to give the app access to the commit statuses. I think this will allow you to show the mutation score for each commit without needing full access to the code of user's repositories.
I don't think we should expect all mutation testing tools to provide the ability to track mutations. So with that in mind, maybe it would be worth to allow the ID to be optional and simply not show it when not available.
We need to create unit tests for the front end
We should improve logging in the azure function with context.log
and context.log.error
(not console.log
or something else)
I noticed some time ago that the badge-api
service is not working (including the badge for this repository):
https://badge-api.stryker-mutator.io/github.com/stryker-mutator/stryker-dashboard/master produces 503: The service is unavailable.
:
I was hoping to add the xmldom project from the xmldom org (https://github.com/xmldom) but for some reason the dashboard does not seem to see xmldom as one of my orgs. I am one of 3 shared owners of this org.
Thanks in advance!
https://dashboard.stryker-mutator.io/repos/brodybits - long scrollable list since I have over 400 forks
I think it would be nice for the UX to be more easily searchable and to keep enabled repos at the top.
I do keep a number of personal GitHub orgs to manage some of my work. I should probably clean up my forks and keep most of them in a personal GitHub org.
This issue depends on stryker-mutator/mutation-testing-elements#2434
Realtime reporting locally is doable, but it would also be nice to have this feature available in the Stryker dashboard.
The implementation is potentially similar to the implementations found in the different Strykers, except that the dashboard should have an API endpoint available for pushing mutants to.
The flow will be like this:
Lets start with some explanation on how end users can use the dashboard to add a badge to their repo.
We would like to have a compare page. Comparable (pun unintended) to githubs compare feature.
The comparison would show the diff between 2 mutation testing reporters. Mutants that did not change (and the tests didn't change) could be shown as ignored
(or not shown entirely). We can reuse internals from StrykerJS's "incremental mode" feature.
URL design could also be inspired by Github's design:
I.e /compare/github.com/stryker-mutator/stryker-js/master...feat/octocat
compares master
to feat/octocat
.
I.e /compare/github.com/stryker-mutator/stryker-js/fix/bar...feat/foo
compares fix/bar
to feat/foo
.
Rather return a http 200 + an 'unknown score' badge.
TODO:
We should implement this in the different reporters behind a flag IMHO. Maybe people only want the mutation score badge and not the full report.
When the repositorySlug does not exist such as by not prepending github.com/
to it (stryker-mutator/stryker4s#221) the dashboard reporter returns a 200 with a body of null
(string). This is not very useful. If the dashboard were to return a 404, possibly with a message that the repositorySlug is not found, it would be clearer what went wrong
Question
Does https://dashboard.stryker-mutator.io/ support private org repos? I can see my public repos but no way to authorise access to my private repos held under my enterprise?
Stryker environment
+-- @stryker-mutator/[email protected]
+-- @stryker-mutator/[email protected]
+-- [email protected]
+-- [email protected]
+-- [email protected]
+-- [email protected]
+-- [email protected]
Additional context
N/A
We should be able to use deploy the StrykerBadgeAPI using a pipeline
We should create a few ui tests for the front end
Time for the next step: hosting mutation testing reports. We will use mutation-testing-elements for the json schema as well as the html report itself.
Suggestion for the new api:
HTTP POST https://dashboard.stryker-mutator.io/api/reports
{
"apiKey": "61213121e3wdasdASD",
"branch": "master",
"mutationScore": 80,
"repositorySlug": "github.com/stryker-mutator/stryker",
"report": { "...": "..." }
}
Hello !
I am wondering if there are plans to support backends other than Azure Storage ?
We are considering using stryker-dashboard to store the result of mutation testing for our Typescript & Kotlin projects (report generated with pitest).
We would love to self-host a dashboard instance but we do not use Azure, we use AWS.
Cheers and thanks for a great product :) !
Maybe on the following homepage:
Some curated repositories could be displayed. I don't know if private/public repositories are displayed, but the repositories are public you could maybe list them on the homepage to show-case the dashboard.
Since one may not necessarily be featured on the homepage, I would have a setting allowing to disallow to be displayed on the homepage though (on a repository or organisation basis)
When the version string does contain a "." the resulting report url does not work.
Example:
https://dashboard.stryker-mutator.io/reports/github.com/richardwerkman/stryker-net/my.test
It could be that the . causes the path to look like a file.
Please provide an option to delete a module from the Stryker dashboard, either via UI or at least via an API call.
Background:
I used stryker to analyse my projects (see Testably.Abstractions. Later on I removed the "Testably.Abstractions.Extensions" project, but the latest analysis is still present in the combined analysis for the main
branch and also changed the total score for the Badge.
Note: the latest branch-specific analysis doesn't contain this project.
I already tried removing the repository and creating a new API key, but to no avail.
Originally posted here infection/infection#1646
The main idea is to replace unpkg
with more reliable CDN
@nicojs suggest the following infection/infection#1646 (comment)
Yeah unpkg isn't the most reliable CDN out there. Might be time to switch over to a locally served one, or include it in the JS bundle.
Feel free to open an issue at the dashboard GH repo
Is your feature request related to a problem? Please describe. I just started using Infection and I noted that the generated HTML log includes a call to a remote resource:
<script defer src="https://www.unpkg.com/mutation-testing-elements"></script>
Unfortunately, this URL is not available 100% of the time and I'm getting constant timeout issues that restrict me from seeing the HTML logs.
Describe the solution you'd like I would like to be able to work with Infection without having to depend on a remotely hosted resource that could fail at any time.
Describe alternatives you've considered I believe that the
mutation-testing-elements
js should be included in local, otherwise host it in a more reliable place (GitHub pages?)
GitHub has something called 'check-runs' they allow adding annotations to a 'check'. Checks are for example what GitHub Actions create. These annotations show up at a line of code. A good example is the eslint-action which lints code and displays error messages directly in GitHub. More info about checks can be found here or here. Annotations can be added to a new check created by the action, or an existing one that is currently running like the CI action. It looks a little like this:
We could add something similar to the Stryker Dashboard by adding a GitHub reporter. This reporter could annotate survived mutants at their position in the source code on GitHub so people can view their mutation testing results in GitHub. This would also work very well in combination with a diff-runner (not supported yet by Stryker4s). A score could also be reported at the end
Some things that need discussion:
I want to link to dashboard of the default branch of my project.
I was hoping to use HEAD
(https://badge-api.stryker-mutator.io/github.com/regseb/cronnor/HEAD or https://dashboard.stryker-mutator.io/reports/github.com/regseb/cronnor/HEAD), but it doesn't work. Is it possible to support HEAD
?
I could use master
, but if my default changes: I would have to change all links.
For example, this is me switching between tabs 2 times:
mutant
to test
view.#bla123
, does a request as well)Only do the GET request once π.
potentially something in the Router:
https://badge.stryker-mutator.io/github.com/my/repo/5.0
goβ¬https://img.shields.io/endpoint?url=https%3A%2F%2Fbadge-api.stryker-mutator.io%2Fgithub.com%2Fmy%2Frepo%2F5
Notice the branch name has been truncated from 5.0
-> 5
. Seems something is interpreting the path component as a number and simplifying it from a float to an int. Nasty.
It would be nice to be able to change the badge style, similar to what https://shields.io does.
This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.
Warning
These dependencies are deprecated:
Datasource | Name | Replacement PR? |
---|---|---|
npm | azure-storage |
These updates are awaiting their schedule. Click on a checkbox to get an update now.
These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.
docker/Dockerfile
node 20.16.0-alpine
.github/workflows/ci.yml
actions/checkout v4
actions/setup-node v4
.github/workflows/deploy.yml
actions/checkout v4
actions/setup-node v4
docker/login-action v3
docker/login-action v3
docker/setup-buildx-action v3
docker/metadata-action v5
docker/build-push-action v6
actions/checkout v4
Azure/login v2
azure/webapps-deploy v3
actions/checkout v4
Azure/login v2
Azure/functions-action v1.5.2
.github/workflows/e2e.yml
actions/checkout v4
actions/setup-node v4
docker/login-action v3
docker/setup-buildx-action v3
docker/metadata-action v5
docker/build-push-action v6
actions/checkout v4
azure/login v2
Azure/functions-action v1.5.2
azure/webapps-deploy v3
actions/setup-node v4
actions/upload-artifact v4
.github/workflows/mutation-testing.yml
actions/checkout v4
actions/setup-node v4
actions/checkout v4
actions/setup-node v4
package.json
@actions/core 1.10.1
@eslint/js 9.9.0
@stryker-mutator/core 8.5.0
@stryker-mutator/mocha-runner 8.5.0
@types/chai 4.3.17
@types/chai-as-promised 7.1.8
@types/mocha 10.0.7
@types/node 20.14.15
@types/sinon 17.0.3
azurite 3.31.0
c8 10.1.2
chai 5.1.1
chai-as-promised 8.0.0
concurrently 8.2.2
lerna 8.1.8
mocha 10.7.3
prettier 3.3.3
prettier-plugin-tailwindcss 0.6.6
rimraf 6.0.1
sinon 18.0.0
typescript 5.5.4
typescript-eslint 8.1.0
packages/badge-api/package.json
@azure/functions 4.5.0
source-map-support 0.5.21
azure-functions-core-tools 4.0.5907
packages/common/package.json
mutation-testing-report-schema 3.1.1
packages/data-access/package.json
azure-storage 2.10.7
mutation-testing-metrics 3.2.0
mutation-testing-report-schema 3.1.1
packages/e2e/package.json
azure-storage 2.10.7
jsonwebtoken 9.0.2
mutation-testing-report-schema 3.1.1
@playwright/test 1.46.0
@types/jsonwebtoken 9.0.6
packages/stryker-elements/package.json
@storybook/addon-essentials 8.2.9
@storybook/addon-links 8.2.9
@storybook/blocks 8.2.9
@storybook/manager-api 8.2.9
@storybook/test 8.2.9
@storybook/theming 8.2.9
@storybook/web-components 8.2.9
@storybook/web-components-vite 8.2.9
autoprefixer 10.4.20
postcss 8.4.41
storybook 8.2.9
tailwindcss 3.4.10
vite 5.4.0
vite-plugin-dts 4.0.2
lit 3.2.0
packages/website-backend/package.json
@nestjs/common 10.4.1
@nestjs/config 3.2.3
@nestjs/core 10.4.1
@nestjs/platform-express 10.4.1
@nestjs/serve-static 4.0.2
ajv 8.17.1
ajv-formats 3.0.1
compression 1.7.4
express 4.19.2
express-jwt 8.4.1
helmet 7.1.0
passport 0.7.0
passport-github2 0.1.12
@nestjs/schematics 10.1.3
@nestjs/testing 10.4.1
@types/compression 1.7.5
@types/express 4.17.21
@types/jsonwebtoken 9.0.6
@types/passport 1.0.16
@types/passport-github2 1.2.9
@types/supertest 6.0.2
mutation-testing-report-schema 3.1.1
supertest 7.0.0
packages/website-contract/package.json
mutation-testing-report-schema 3.1.1
packages/website-frontend/package.json
@angular-devkit/build-angular 18.1.4
@angular/animations 18.1.4
@angular/cli 18.1.4
@angular/common 18.1.4
@angular/compiler 18.1.4
@angular/compiler-cli 18.1.4
@angular/core 18.1.4
@angular/forms 18.1.4
@angular/platform-browser 18.1.4
@angular/platform-browser-dynamic 18.1.4
@angular/router 18.1.4
@github/clipboard-copy-element 1.3.0
@primer/octicons 19.11.0
@stryker-mutator/karma-runner 8.5.0
@types/jasmine 5.1.4
bootstrap 5.3.3
jasmine-core 5.2.0
karma 6.4.4
karma-chrome-launcher 3.2.0
karma-coverage 2.2.1
karma-jasmine 5.1.0
karma-jasmine-html-reporter 2.1.0
mutation-testing-elements 3.2.0
mutation-testing-metrics 3.2.0
mutation-testing-report-schema 3.1.1
rxjs 7.8.1
zone.js 0.14.10
packages/badge-api/extensions.csproj
Microsoft.Azure.WebJobs.Script.ExtensionsMetadataGenerator 1.2.3
.nvmrc
node 20.16.0
I think it would still not be super-obvious to newcomers that the "Mutation score" badge would link to the nice, valuable HTML report on the Stryker dashboard.
Hi, I've been using stryker a lot and I've been adding the badges to every readme, it's great and encourages people to write good tests in there PRs!
I have a suggestion/feature request, for the codecov badge I don't need to set a key on travis-ci for any open source projects
Would it be possible to add a feature like this to the stryker dashboard?
Thanks! π
Plus more examples of each.
Why are some working and some not? I've set up each one the same way with its own key as I've activated it from the Stryker dashboard.
When disabling a repository, a previously uploaded report and badge are still available
When disabling a repository, the report and mutation score information are removed and no longer available
They are still available
Summary
Thanks for the awesome tool!
My repo is a mono repo, I added Stryker to my api
server folder at https://github.com/Hongbo-Miao/hongbomiao.com/blob/main/api/stryker.conf.js
Also, I added Stryker to my GitHub Actions pipeline, and it runs successfully.
https://github.com/Hongbo-Miao/hongbomiao.com/runs/2116804199?check_suite_focus=true
with log
...
22:31:43 (1715) INFO DashboardReporterClient PUT report to https://dashboard.stryker-mutator.io/api/reports/github.com/Hongbo-Miao/hongbomiao.com/PR-1972 (~976888 bytes)
22:31:43 (1715) INFO HtmlReporter Your report can be found at: file:///home/runner/work/hongbomiao.com/hongbomiao.com/api/reports/mutation/html/index.html
22:31:45 (1715) INFO DashboardReporter Report available at: https://dashboard.stryker-mutator.io/reports/github.com/Hongbo-Miao/hongbomiao.com/PR-1972
22:31:45 (1715) INFO MutationTestExecutor Done in 3 minutes 52 seconds.
You can see my pull request Stryker report
However, my badge is not working,
Also, my dashboard is also not working.
https://dashboard.stryker-mutator.io/reports/github.com/Hongbo-Miao/hongbomiao.com/main
Does this issue is related with mono repo?
(I am also planning to add Stryker to my web
and mobile
app in this mono repo in future.)
Stryker config
/**
* @type {import('@stryker-mutator/api/core').StrykerOptions}
*/
module.exports = {
packageManager: 'npm',
reporters: ['html', 'clear-text', 'progress', 'dashboard'],
testRunner: 'jest',
coverageAnalysis: 'perTest',
};
Test runner Jest config
module.exports = {
testEnvironment: 'node',
testPathIgnorePatterns: ['<rootDir>/build'],
};
Stryker environment
"@stryker-mutator/core": "4.5.1",
"@stryker-mutator/jest-runner": "4.5.1",
"jest": "26.6.3",
Test runner environment
npm test
api/jest.config.js
Your Environment
software | version(s) |
---|---|
node | 14.16.0 |
npm | 7.6.3 |
Operating System |
Add stryker.log
N/A. The test did run successfully in the GitHub Actions build which can be found above.
Currently, it is not possible to deploy a pr concurrently (only one allowed). We could fix that by using subdomains in combination with Azurite or something along the lines of that.
No direct link from https://dashboard.stryker-mutator.io
Starting from https://stryker-mutator.io I had to follow one link to the handbook on GitHub and then find dashboard.md.
I would really favor some kind of a direct link from https://dashboard.stryker-mutator.io.
I want to rename the module but can't seem to replace or remove the old one: https://dashboard.stryker-mutator.io/reports/github.com/riezebosch/CommandLiners/master
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.