mvines / ci-gate Goto Github PK

Probably need to define a way to whitelist a build artifact too, perhaps it it contains "public" in it somewhere

I ported over all of our webhooks to use ci-gate, but it seems that master is no longer building. It'd be nice to also trigger master from this so we don't have to have two webhooks.

Wondering if you're open to doing this by default, or if we should have an environment variable to turn this on.

Buildkite logs are normally only accessible to repo members, but this makes it much harder for 3rd party contributors from debugging their PR issues without help.

Certain PRs from 3rd parties require more scrutiny, such as a buildkite pipeline definition. Build a mechanism to notify mantainers of such changes by a comment in the PR and/or a label that must be manually removed from the PR (in addition to manually adding the CI label) to grant CI.

https://buildkite.com/docs/builds/managing-log-output#collapsing-output

Would you be open to adding a unit testing framework for this repo? Given the nature of this logic, my recommendation would be going with Jest for it's powerful mocking abilities. A unit testing framework would allow for more contributions, as we could make them with confidence.

Eventually we'll do uplifts #6 and automerging #1 too, so consider how that will fit into the README as well while shuffling content around

ref: 0ff5e4e

Adding this label to a PR should cause the PR to get merged automatically once CI is green and the PR is approved by a reviewer

The current setup makes several assumptions about how your buildkite pipelines are setup. Some of these assumptions are unnecessary and should be refactored/removed, while others just need to be documented.

Likely simply need to indicate to the user that they need help from the project maintainer

Once PRs are merged into the master branch (or any other branch), the user should be able to set a label on the PR to have the system attempt to cherry pick it into another branch (based on the label name).

Buildkite now offers a beta feature to mark pipelines as beta, which is a great start towards making this project obsolete: see
buildkite/feedback#137 (comment)

For buildkite public pipelines, ci-gate does not need to serve its own version of the buildkite log and thus can avoid overriding the buldkite github commit status URL. Unless the buildkite API provides a way to check if a pipeline is public, we’d likely just use another environment variable to whitelist pipelines as public.

eg: solana-labs/solana#641

Rather that using the CI_USER_WHITELIST environment variable to determine who gets automatic CI service, can this be determined by looking at who has write access to the destination repository?

We're seeing a bunch of CI failures after switching to CI-Gate with the following error:

error: cannot lock ref 'refs/remotes/origin/pull/XXX/head': unable to resolve reference 'refs/remotes/origin/pull/XXX/head'

The only difference in the build log appears that we're no longer using the commit sha, and are instead using the PR head reference.

Happy to try to work on this, just want to see if this would be an acceptable change to merge upstream.

With the GitHub Checks API, it is possible to add a neutral/failing commit status that includes a button to trigger an arbitrary action.

For example, this might look something like:

Clicking "details" will take you to the checks tab of the PR, which might show something like:

Clicking the button will trigger a check_run.requested_action event. Then this app can respond to the action_requested webhook and trigger a CI run.

I think this is preferable than the label-based approach for a few reasons:

• Adding/removing labels creates noise in the PR
• Because checks apply to a specific commit, it avoids the current potential race conditions where a user pushes new commits right before a label is added
• Having an explicit status check and associated action button is more intuitive than a convention regarding adding/removing labels

If this app used probot, the code might look something like this:

context.github.checks.create(
  context.repo({
    name: "CI Gate",
    head_branch,
    head_sha,
    status: "completed",
    conclusion: "action_required",
    completed_at: new Date(),
    output: {
      title: "CI run not yet authorized",
      summary:
        "Click the above <kbd>Trigger CI run</kbd> button to trigger a CI run."
    },
    actions: [
      {
        label: "Trigger CI run",
        description: "Trigger a CI run",
        identifier: "trigger_ci"
      }
    ]
  })
);

If that sounds good to you, I could probably help implement this.

It would be nice if we could specify regular expression matching for items in BUILDKITE_PIPELINE_PUBLIC_LOG_WHITELIST. Specific use case, I want to whitelist all plugins in this organization: http://github.com/fusionjs/

Happy to help with this assuming I find some time :)

Sometimes people like to push PRs to show their work but it's known that the PR is not ready to be merged. In such a case, it would be nice to have a label (noci?) to prevent testing a PR that is known to not work