ossf / allstar Goto Github PK
View Code? Open in Web Editor NEWGitHub App to set and enforce security policies
License: Apache License 2.0
GitHub App to set and enforce security policies
License: Apache License 2.0
Branch Protection is not enabled on free-tier private repos: https://docs.github.com/en/github/administering-a-repository/defining-the-mergeability-of-pull-requests/about-protected-branches
GitHub responds with:
GET https://api.github.com/repos/org/repo/branches/master/protection: 403 Upgrade to GitHub Pro or make this repository public to enable this feature. []
Currently we error out of the whole installation/org.
Change to fail BP policy, notify text should include the message from GitHub "Upgrade to GitHub Pro or make this repository public to enable this feature."
The readme (https://github.com/ossf/allstar) links to a sample config repository whose policy files contain "disableRepoOverride: true". It's not clear from the documentation whether this overrides, combines with, or is overridden by the global "disableRepoOverride: true" in the "allstar.yaml" (which also has it set in that sample repo).
After discussing with scorecard folks, we will try to use checker.Runner https://github.com/ossf/scorecard/blob/main/checker/check_runner.go as the interface, update: https://github.com/ossf/allstar/blob/main/pkg/policies/binary/binary.go#L117-L137 to use that.
Include Go requirements, how to clone, build, test. Install linter and run lint.
Also include how to setup a test app in GitHub and run locally.
The in-memory cache is great for simple workflows. But for large repositories it usually causes OOM.
A sample repository that was causing OOM was github.com/ApolloAuto/apollo
related issue ossf/scorecard#589
allstar/pkg/ghclients/ghclients.go
Line 83 in 70d3e36
This might not be applicable for allstar but wanted to point out the issues it could run into.
Look at : https://github.com/ossf/scorecard/blob/main/.golangci.yml
See if there are any more linters we want to enable over default. Only include additions to config file necessary over default settings.
Hello Team,
I am curious to understand the frequency in which allstar scans the repositories to identify security policy violations?
I believe allstar doesn't use webhooks, so how will it know if a push or any action that happens in the GitHub repository?
Thanks in advance
We get 502 and 504 occasionally from GitHub. These should be retried.
Dart and Flutter Orgs branch protection requires 1 codereview but we allow some bot accounts to bypass the requirement. This is causing the allstar application to detect that the repositories as out of compliance:
Allstar has detected that this repository’s Branch Protection security policy is out of compliance. Status:
PR Approvals not configured for branch master
Could this be related to ossf/scorecard#1614?
Hello @jeffmendoza,
Is there a possibility that I can look for a specific branch which using allstar?
Assume i have two different branch ,
Main and dev
Main branch as Security.md file and dev doesnt have.
Does allstar detect this and raise an issue?
Please help me understand
Thanks in advance
There are various ways to opt repos in and out, including these:
optConfig:
optOutStrategy: true
optOutRepos:
- ...
optInRepos:
- ...
optOutPrivateRepos: true
optOutPublicRepos: false
...and including a per-repo file and an org-wide .allstar repo file.
It is highly unclear from the documentation what will win if there is a conflict (additionally it seems there's an undocumented feature that changes which will win, as mentioned briefly in opt-out.md, which makes this even more unclear). (edit: found the documentation for this feature)
For example, does optOutPrivateRepos
win vs an explicit optInRepos
? What if optOutStrategy
is true, but optOutPrivateRepos
and optOutPublicRepos
are also true? What if a repo opts out locally but is listed explicitly in the global .allstar file?
https://github.com/ossf/allstar#policies links to golang documentation as part of its Yaml documentation. It's quite confusing. I was unable to determine at a glance what form the .yaml files should take and what options were available.
Accept an optional yaml list of status check names for the branch protection policy to enforce repos have enabled an organization's expected checks. The yaml list should accept objects containing at least the fields name
and required
(to enforce whether a particular status check should be marked as required).
I'm happy to take on this pull request myself if the maintainers approve of the issue.
It would be great if AllStar supported creating JIRA tickets instead of just GitHub issues, as many projects use JIRA instead.
I might find time to contribute this next year, if no-one else jumps in before then.
Line 13 in 1c2befa
The scorecard version v3 has more reliable features.
Would be great if allstar could verify whether certain GitHub Actions exist and a minimum version.
Often GitHub Actions contain a set of security screenings to block the PR.
I administer an org with many, many repositories with varying levels of security (and varying levels of importance). I would love to enable allstar globally, but I am concerned that it will file hundreds of issues on various repos that may be secret, archived, unimportant, or some combination of the above, as well as filing issues that are known to be bogus (for example some of our repos are intentionally for storing binary files so the whole "don't store binary files" thing won't work in those repos).
What I would like is a way to get a preview of what allstar is going to complain about ahead of me doing something with public visibility.
Allstar always creates it's issue(s) with a gray label called allstar
. We use an organization-wide label associated with security / risk that we'd like it to use. I've read through the code and config files and I don't see how to configure allstar to use a custom label, but it seems like something that would be possible based on the way the config works currently.
Thanks for the great tool!
When creating an issue for scorecard, include the details from:
https://github.com/ossf/scorecard/blob/main/checker/check_result.go#L45
in the issue. Only include DetailWarn and above, only include first ~10 lines and note if some lines were cut.
To allow flexibility, we should allow org-level config in the .github
repo and allstar
sub-directory. This should be searched only if the .allstar
repo does not exist in the org. Note, we must check for repo existance of .allstar
, and not just try to get the particular config file (ex: .allstar/foo.yaml
), as some config files will not exist on purpose. If the .allstar
repo exists at all, then .github
should not be used at all.
This will address #27
Proposal is to look like:
(No change to title)
This issue was automatically created by Allstar
Security policy violation
Security policy Binary Artifacts failed: binaries present in source code
Rule Description
Binary Artifacts are an increased security risk in your repository. Binary artifacts cannot be reviewed, allowing the introduction of possibly obsolete or maliciously subverted executables. For more information see Scorecards Documentation.
Remediation Steps
To remediate, remove the generated executable artifacts from the repository.
Artifacts found
Additional Information
This policy is drawn from Security Scorecards, which is a tool that scores security best practices for a project. You may wish to run a Scorecards scan directly on this repository for more details.
Staging deploy of Allstar for testing, see https://github.com/ossf-tests/.allstar for config. (This is the custom footer)
This issue will auto resolve when the policy is in compliance. (This is current Allstar footer)
Issue created by Allstar. See https://github.com/ossf/allstar/ for more information. For questions specific to the repository, please contact the owner or maintainer.
Currently looks like: https://github.com/ossf-tests/scorecard-check-binary-artifacts-e2e-4-binaries/issues/4
If an organization has an organization level SECURITY.md
file defined as outline in About default community health files, I would expect that this would satisfy the SECURITY.md policy check.
Instead, an issue was created with a link to create this policy, but this link is actually just to our policy itself, which is defined here.
Hello Team,
I am new to allstar and tried to use it in my GitHub repository. I have few questions
Is it specific only to the GitHub repo? Does it have a docker image or CLI to use it in local/ other git remote repositories
I have configured .allstar folder wiht the specified 4 policy yaml file. I dont see any issue or any process running to indicate a check is running and i need to wait for the result. Where can i see the result?
Does allstar use scorecard internally?
Thanks in advance.
POST https://api.github.com/repos/org/repo/issues: 410 Issues are disabled for this repo
If Allstar is configured to create an issue, but issues are disabled, fail silently, I guess, and continue.
Right now the opt in/out config must list all repos to opt in or out. It would be good to be able to opt in/out all public or private repos.
It would be nice to be able to define the repository types to include for the security issues. For example, we might want a SECURITY.md for all public and internal repos, but not care about whether private repos have it or not.
example issue:
stackables/cloudevents-router-gcp#3
Security Policy SECURITY.md is out of compliance, status:
SECURITY.md not found.
Go to https://github.com/stackables/cloudevents-router-gcp/security/policy to enable.
github docs:
https://docs.github.com/en/code-security/getting-started/adding-a-security-policy-to-your-repository
Show SECURITY.md can be under root, docs, or .github
Hello Team,
I have followed your documentation and enabled allstart github app as below.
Do not see any issues being created automatically, Does it support GitHub enterprise with allstar as internal repository type?
Is there anything else to configure to get the issues created in respective repositories?
Thanks
"Allstar has detected that this repository’s Binary Artifacts security policy is out of compliance. Status: Scorecard Check Binary Artifacts failed: binaries present in source code" is what the message says today.
This is not super helpful. It would be nice if the message was more specific about exactly which files it was referring to.
Improved readability.
https://github.com/ossf/allstar/blob/main/pkg/config/config.go#L40
Forking from: #103 (comment)
My thoughts:
it is also a goal for me to have Allstar's default settings represent the highest level of security best practices that we (OpenSSF) can recommend. ...
That said, I also acknowledge that my point of view is from a large-org perspective, and while the org admins do have transient permissions on all repos, they can't be expected to be responsible for all repos, and a direct admin should be required. I see how this would not be the case for a small org. I'd like to bring this up at the next "Best Practices" WG meeting to discuss further.
Since issues for an org can all be sent to one issue database, it would be nice if the issues themselves linked to the offending repositories, so that the admin didn't have to copy-paste the name into a URL to go to the settings of that repo to fix the issue.
Currently the bot posts an update to each issue every few days. Since we have around a hundred issues open (primarily because of #108) this causes a huge amount of bugspam to anyone following the repository.
In pondering this I realised that technically the bot is violating our contribution guidelines, which say that a comment shouldn't be added if it doesn't add any new information. In general, if a bug is open, it is taken that it is still valid, and nothing need be said if nothing has changed.
It would be significantly clearer if the documentation only included actual implemented features rather than being aspirational.
For example, "fix" is listed as a possible action, separate from the "Proposed, but not yet implemented actions", but later is revealed to do nothing yet. It would be better to just not list it until it does something.
Hi,
We got a recent issue opened using the allstar-app:
"Allstar has detected that this repository’s Outside Collaborators security policy is out of compliance. Status:
Did not find any owners of this repository
This policy requires all repositories to have an organization member or team assigned as an administrator. Either there are no administrators, or all administrators are outside collaborators."
I guess due to this change: 01e4892
However, the repo has a team with admin access, and a member of the org is on this team. Perhaps the change didn't take into account the scenario of an admin team as opposed to a member being an admin directly on the team?
Users who want to opt out of or disable Allstar need to know the strategy configuration (opt-in vs opt-out vs. repo-level), which involves looking in several places to find certain files or settings. Is it possible to add a feature that tells users the configuration in each issue they receive? For example, a line in the issue footer could say:
Allstar is enabled on your repository
Allstar is enabled on your organization in the opt-in strategy
Allstar is enabled on your organization in the opt-out strategy with/without repository override
.
Should have multiple checks enabled (3-5 to start). Default config will be to run all enabled checks, but can override in config to only run certain ones. Should have a "success value" to decide what score (0-10) is passing.
Hello @jeffmendoza,
I am trying to create an own instance of allstar.
{"severity":"CRITICAL","error":"open variable gcpsecretmanager://projects/allstar-ossf/secrets/allstar-private-key?decoder=bytes: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.","time":"2021-08-23T15:16:18+05:30","message":"Could not load app secret, shutting down"}
https://developer.github.com/changes/19/#--the-451-status-code-is-now-supported
repo cannot be accessed: GET https://api.github.com/repos/...: 451 Repository access blocked []
Looks like repos can be listed, apps installed, but unavailable due to DMCA takedown request. Allstar should catch this abort this repo, and continue enforcing on other repos in the installation.
Currently we are aborting the whole installation/org when hitting this.
https://github.com/ossf/allstar/blob/main/pkg/enforce/enforce.go#L107
I think that it's crucial to include some actionable information when creating an issue of any kind. This is a follow up from this issue: envoyproxy/java-control-plane#173.
I think that a well created ticket should at least include the following information:
Let's look at the message in the linked issue:
Security Policy Outside Collaborators is out of compliance, status:
Found 1 outside collaborators with admin access.
In my opinion it does not inform why it's an issue and how to resolve it. A better message would be:
Security Policy "Outside Collaborators" is out of compliance.
This policy requires all users with admin access to be members of the organisation. That way if an account is compromised it can be quickly denied access to all resources of the organisation. To fix this you should:
1. Remove the user from the repository based access by going to settings -> permissions -> ...
2. Add the user to the organisation by going to -> ...
If you don't see the settings tab you probably don't have administrative access. Reach out to the administrators of the organisation to fix this issue.
This issue will auto resolve when the policy is in compliance.
If you have any questions - please reach out to XYZ.
If you have any questions or you'd like to agree/disagree please comment below. Also, I'm creating this issue in good faith, I really think better issue contents will improve this project.
Best regards,
slonka
Eat your own dog food. Run scorecard on allstar.
it seems that requireAppproval
has a typo (should it be requireApproval
?)
allstar/pkg/policies/branch/branch.go
Line 79 in 84b402d
Similar to the SECURITY.md
community health check, for organizations that are interested in having dependabot automated updates enabled on their repositories, it'd be great to have a check that alerts when a .github/dependabot.yml
config is not present within the repo.
Ideally, it'd be cool to have this baked into https://github.com/organizations/<org-name>/settings/security_analysis
.
cc: @jhutchings1
create a repository named .allstar in your organization
Many GitHub apps are based on Probot and default to the .github repository for organization config. The default setting of this app requires splitting config into yet another repository.
At a minimum it would be nice if this app allowed specifying the org config location.
It would be nice to mirror https://probot.github.io/docs/best-practices/#configuration.
It's not immediately clear to me whether I need to give special rights to any particular user for allstar to be able to see the .allstar repo if it's private, or file issues in private repos. The documentation seems to be silent about this. I'm guessing that apps get magic access to everything somehow?
It would be great if the enforceBranches
configuration for branch_protection.yaml
could be set to apply to any repository which matches a pattern.
Example of usage:
In our repositories across our organization we perform our releases from branches which match *.*.x
, we would like to enforce our branch protections on multiple repositories for those branches, but would love to have this requirement centralized in our .allstar
repository.
You could just include it in the yaml as the branch name:
enforceBranches:
"*-releasable":
- "*.*.x"
For our usage we would have:
enforceBranches:
"*":
- "*.*.x"
For the flutter org, we have lots of repos but (with a few small exceptions) we use flutter/flutter/issues for all of them. Where will allstar try to submit issues if I enable that?
When we enable allstar on the flutter org, it filed about 37 issues complaining that branch protection wasn't enabled on various repos.
All of these were archived, read-only repos. These don't need branch protection, since they have the ultimate protection of being entirely unmodifiable.
I've created a sample repository to see how Allstar works. It was 9PM JST, and an issue is created in 6AM in the next day.
I couldn't find documents about when Allstar app is triggered. Is it once a day?
Can we configure when it's triggered, or manually trigger?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.