OpenSSF Scorecard Monitor

Simplify OpenSSF Scorecard tracking in your organization with automated markdown and JSON reports, plus optional GitHub issue alerts.

🔮 About

If you're feeling overwhelmed by an avalanche of repository scorecards in your organization, you can breathe easy: Automation is here to make your life easier! It will streamline the process of keeping track of them all by providing a comprehensive report in Markdown and a local database in JSON with all the scores. Furthermore, to stay on top of any changes in the scores, you can choose to get notifications through Github Issues.

✅ Requirements

Please ensure that any repository you wish to track with Scorecard Monitor has already been analyzed by OpenSSF Scorecard at least once. This can be accomplished using the official GitHub Action or the Scorecard CLI.

It's also possible that some repositories in your organization are already being automatically tracked by OpenSSF in this CSV file via weekly cronjob. One caveat: Automatically tracked projects do not include certain checks in their analysis (CI-Tests,Contributors,Dependency-Update-Tool,Webhooks).

If you're not sure whether a specific project is already using Scorecard, you can always spot-check with the following URL pattern: https://securityscorecards.dev/viewer/?uri=github.com/<ORG_NAME>/<REPO_NAME> (substitute <ORG_NAME> and <REPO_NAME> as appropriate). The Scorecard API is also able to fetch scores for a given repository.

📺 Tutorial

This section is coming soon. If you would like to contribute to the documentation, please feel free to open a pull request for review.

❤️ Awesome Features

Easy to use with great customization
Easy to patch the scoring as the reports includes a direct link to StepSecurity
Easy way to visualize the scorecard results with The Scorecard Visualizer or deps.dev
Cutting-edge feature that effortlessly compares OpenSSF scorecards between previous and current commits with The Scorecard Visualizer Comparator
Discovery mode: list all the repos in one or many organizations that are already being tracked with OpenSSF Scorecard
Reporting in Markdown with essential information (hash, date, score) and comparative against the prior score
Self-hosted: The reporting data is stored in JSON format (including previous records) in the repo itself
Generate an issue (assignation, labels..) with the last changes in the scores, including links to the full report
Automatically create a pull request for repositories that have branch protection enabled
Easy to exclude/include new repositories in the scope from any GitHub organization
Extend the markdown template with you own content by using tags
Easy to modify the files and ensure the integrity with JSON Schemas
The report data is exported as an output and can be used in the pipeline
Great test coverage (in progress)

🎉 Demo

Sample Report

Sample report

Sample Issue

Sample issue

Used By

Nodejs: The Node.js Ecosystem Security Working Group is using this pipeline to generate a report with scores for all the repositories in the Node.js org.
One Beyond: The Maintainers are using this pipeline to generate a scoring report inside a specific document, in order to generate a web version of it
NodeSecure: The Maintainers are using this pipeline to generate a scoring report and notification issues.
More users

📡 Usage

Standalone with auto-discovery version

With the following workflow, you will get the most out of this action:

Trigger manually or by Cron job every Sunday
It will scan the org(s) in scope looking for repositories that are available in the OpenSSF Scorecard
It will store the database and the scope files in the repo
It will generate an issue if there are changes in the score

name: "OpenSSF Scoring"
on: 
  # Scheduled trigger
  schedule:
    # Run every Sunday at 00:00
    - cron: "0 0 * * 0"
  # Manual trigger
  workflow_dispatch:

permissions:
  # Write access in order to update the local files with the reports
  contents: write
  # Write access only required if creating PRs (see Advanced Tips below)
  pull-requests: none 
  # Write access in order to create issues
  issues: write
  packages: none

jobs:
  security-scoring:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: OpenSSF Scorecard Monitor
        uses: UlisesGascon/[email protected]
        with:
          scope: reporting/scope.json
          database: reporting/database.json
          report: reporting/openssf-scorecard-report.md
          auto-commit: true
          auto-push: true
          generate-issue: true
          # The token is needed to create issues, discovery mode and pushing changes in files
          github-token: ${{ secrets.GITHUB_TOKEN }}
          discovery-enabled: true
          # As an example nodejs Org and Myself
          discovery-orgs: 'UlisesGascon,nodejs'

Options

scope: Defines the path to the file where the scope is defined
database: Defines the path to the JSON file usage to store the scores and compare
report: Defines the path where the markdown report will be added/updated
auto-commit: Commits the changes in the database and report files
auto-push: Pushes the code changes to the branch
generate-issue: Creates an issue with the scores that had been updated
issue-title: Defines the issue title
issue-assignees: List of assignees for the issue
issue-labels: List of labels for the issue
github-token: The token usage to create the issue and push the code
max-request-in-parallel: Defines the total HTTP Request that can be done in parallel
discovery-enabled: Defined if the discovery is enabled
discovery-orgs: List of organizations to be includes in the discovery, example: discovery-orgs: owasp,nodejs. The OpenSSF Scorecard API is case sensitive, please use the same organization name as in the GitHub url, like: https://github.com/NodeSecure is NodeSecure and not nodesecure. See example
report-tags-enabled: Defines if the markdown report must be created/updated around tags by default is disabled. This is useful if the report is going to be include in a file that has other content on it, like docusaurus docs site or similar
report-start-tag Defines the start tag, default 
report-end-tag: Defines the closing tag, default 
render-badge: Defines if the OpenSSF badge must be rendered in the reportor to only show the score
report-tool: Defines the reporting review tool in place: scorecard-visualizer Example or deps.dev Example, by default scorecard-visualizer

Outputs

scores: Score data in JSON format

name: "OpenSSF Scoring"
on: 
  # ...

permissions:
  # ...

jobs:
  security-scoring:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: OpenSSF Scorecard Monitor
        uses: UlisesGascon/[email protected]
        id: openssf-scorecard-monitor
        with:
          # ....
      - name: Print the scores
        run: |
          echo '${{ steps.openssf-scorecard-monitor.outputs.scores }}'

🚀 Advanced Tips

Avoid committing directly to the branch and instead generate a PR

If you have implemented the recommended branch protection rules from the OpenSSF Scorecard, committing and pushing directly to the main branch will be impossible. An easy alternative is to extend the pipeline to automatically generate a PR for you:

name: "OpenSSF Scoring"
on: 
  # ...

permissions:
  contents: write
  pull-requests: write
  issues: write
  packages: none

jobs:
  security-scoring:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: OpenSSF Scorecard Monitor
        uses: UlisesGascon/[email protected]
        id: openssf-scorecard-monitor
        with:
          auto-commit: false
          auto-push: false
          generate-issue: true
          # ....
      - name: Print the scores
        run: |
          echo '${{ steps.openssf-scorecard-monitor.outputs.scores }}'
      - name: Create Pull Request
        uses: peter-evans/create-pull-request@38e0b6e68b4c852a5500a94740f0e535e0d7ba54 # v4.2.4
        with:
            token: ${{ secrets.GITHUB_TOKEN }}
            commit-message: OpenSSF Scorecard Report Updated
            title: OpenSSF Scorecard Report Updated
            body: OpenSSF Scorecard Report Updated
            base: main
            assignees: ${{ github.actor }}
            branch: openssf-scorecard-report-updated
            delete-branch: true

Embed Report version

If you want to mix the report in markdown format with other content, then you can use report-tags-enabled=true then report file will use the tags to add/update the report summary without affecting what is before or after the tagged section.

This is very useful for static websites, here is an example using docusaurus.

Custom tags

By default we use  and , but this can be customize by adding your custom tags as report-start-tag and report-end-tag

Increase HTTP request in parallel

You can control the amount of parallel requests performed against the OpenSSF Scorecard Api by defining any numerical value in max-request-in-parallel, like max-request-in-parallel=15.

By default the value is 10, higher values might not be a good use of the API and you can hit some limits, please check with OpenSSF if you want to rise the limits safely.

Exclude repos

In some scenarios we want to enable the auto-discovery mode but we want to ignore certain repos, the best way to achieve that is by editing the scope.json file and add any report that you want to ignore in the excluded section for that specific organization.

🍿 Other

Scoping Structure

Just for reference, the scope will be stored this way:

File: reporting/scope.json

{
    "github.com": {
      "included": {
        "UlisesGascon":[
          "tor-detect-middleware", 
          "check-my-headers", 
          "express-simple-pagination"
        ]
      },
      "excluded": {
        "UlisesGascon": [
          "demo-stuff"
        ]
      }
    }

}

Database structure

Just for reference, the database will store the current value and previous values with the date:

{
  "github.com": {
    "UlisesGascon": {
      "check-my-headers": {
        "previous": [ {
          "score": 6.7,
          "date": "2022-08-21"
        }],
        "current": {
          "score": 4.4,
          "date": "2022-11-28"
        }
      }
    }
  }
}

Add auto-scoping

Overview

As a user, I want to be able to configure auto-scoping feature as true and include several github organizations as auto-scope-orgs (e.g. ['nodejs', 'OWASP']) in the Github action, so that the Github Action will list all the public repos available for the org, update the allowed scope with repos that were not previously included or in the exclusion list, save the changes in the file, commit the changes, and continue with the regular process.

Technical Requirements

New input auto-scope to enable/disable auto-scoping feature
New input auto-scope-orgs in the Github action that allow to include several github organizations, like ['nodejs', 'OWASP']
Add new properties to the scope file to divide the content in allowed or excluded repositories {allowed: {org: [repo]}, excluded: {org: [repo]}}
Collect the Org repos and compare against the included / excluded list
Check if the api has indexed the repo or ignore it

Main logic

Check that the Github token has been added as input or throw an error
Generate a list from all the repos (public only) available for the org using the github api and the token
Update the allowed scope with the repos that were not previously included or present in the exclusion list
Save the changes in the scope file
Commit the changes
Continue with the regular process (ensure that the regular process is using the latest version of the scope ).

Context

See 📺

ulisesgascon / openssf-scorecard-monitor Goto Github PK