isabella232 / github-pr-resource-1 Goto Github PK

Github PR resource

❗ This repo is a fork of Telia OSS's resource for internal use ❗

A Concourse resource for pull requests on Github. Written in Go and based on the Github V4 (GraphQL) API. Inspired by the original, with some important differences:

• Github V4: check only requires 1 API call per 100th open pull request. (See #costs for more information).
• Fetch/merge: get will always merge a specific commit from the Pull request into the latest base.
• Metadata: get and put provides information about which commit (SHA) was used from both the PR and base.
• Webhooks: Does not implement any caching thanks to GraphQL, which means it works well with webhooks.

Make sure to check out #migrating to learn more.

Source Configuration

Notes:

• If v3_endpoint is set, v4_endpoint must also be set (and the other way around).
• Look at the Concourse Resources documentation for webhook token configuration.
• When using required_review_approvals, you may also want to enable GitHub's branch protection rules to dismiss stale pull request approvals when new commits are pushed.

Behaviour

check

Produces new versions for all commits (after the last version) ordered by the committed date. A version is represented as follows:

• pr: The pull request number
• commit: The commit SHA
• updated: Timestamp of when the pull request was last updated at the time of the check

If several commits are pushed to a given PR at the same time, the PR with the latest updated at will be the newest version.

search

The GraphQL search for pull requests uses the Search endpoint and follows the following pattern:

is:pr is:open repo:%s/%s updated:>%s sort:updated

Which means that we want to search for only OPEN PULL REQUESTS that have been UPDATED since the latest updated timestamp of the last check. To test this query, you can simply use the search box in the navigation of github.com.

Then, we use the PullRequestTimelineItemsConnection to fetch all commits / events on the PRs timeline since the latest updated timestamp of the last check. This allows us to iterate over the pull requests and filter them as is covered in the next section.

filters

There are many ways by which this resource filters pull requests and each filter is a function in the filter.go file within the pullrequest package. This makes it very easy to test the functionality of each filter. There are two types:

• negative filters which when return true, the pull request should be excluded (skipped) from versions
• positive filters which when return true, the pull request should be included from versions

Current negative filters:

• pullrequest.SkipCI which will exclude PRs containing [skip ci|ci skip] in the PR Title / Message
• pullrequest.BaseBranch which will exclude PRs where the base branch (e.g. master) does not match the source configuration
• pullrequest.Fork which will exclude PRs from forks when disable_forks is configured true

Current positive filters:

• pullrequest.Created which will include PRs with Created == Updated OR Created > HeadRef.Commited | Authored | Pushed
• pullrequest.BaseRefChanged which will include PRs where a BaseRefChanged occurred
• pullrequest.BaseRefForcePushed which will include PRs where a BaseRefForcePushed occurred
• pullrequest.HeadRefForcePushed which will include PRs where a HeadRefForcePushed occurred
• pullrequest.Reopened which will include PRs where a BaseRefChanged occurred
• pullrequest.BuildCI which will include PRs with a new comment containing [build ci|ci build]
• pullrequest.NewCommits which will include PRs with a new commit since the last updated timestamp of the last check

Note on webhooks:

This resource does not implement any caching, so it should work well with webhooks (should be subscribed to push and pull_request events). One thing to keep in mind however, is that pull requests that are opened from a fork and commits to said fork will not generate notifications over the webhook. So if you have a repository with little traffic and expect pull requests from forks, you'll need to discover those versions with check_every: 1m for instance. check in this resource is not a costly operation, so normally you should not have to worry about the rate limit.

get

Clones the base (e.g. master branch) at the latest commit, and merges the pull request at the specified commit into master. This ensures that we are both testing and setting status on the exact commit that was requested in input. Because the base of the PR is not locked to a specific commit in versions emitted from check, a fresh get will always use the latest commit in master and report the SHA of said commit in the metadata. Both the requested version and the metadata emitted by get are available to your tasks as JSON:

• .git/resource/version.json
• .git/resource/metadata.json
• .git/resource/changed_files (if enabled by list_changed_files)

The information in metadata.json is also available as individual files in the .git/resource directory, e.g. the base_sha is available as .git/resource/base_sha. For a complete list of available (individual) metadata files, please check the code here.

When specifying skip_download the pull request volume mounted to subsequent tasks will be empty, which is a problem when you set e.g. the pending status before running the actual tests. The workaround for this is to use an alias for the put (see telia-oss/github-pr-resource#32 for more details).

git-crypt encrypted repositories will automatically be decrypted when the git_crypt_key is set in the source configuration.

put: update-status <-- Use an alias for the pull-request resource
resource: pull-request
params:
    path: pull-request
    status: pending
get_params: {skip_download: true}

Note that, should you retrigger a build in the hopes of testing the last commit to a PR against a newer version of the base, Concourse will reuse the volume (i.e. not trigger a new get) if it still exists, which can produce unexpected results (#5). As such, re-testing a PR against a newer version of the base is best done by pushing an empty commit to the PR.

put

Note that comment, comment_file and target_url will all expand environment variables, so in the examples above $ATC_DEFAULT_URL will be replaced by the public URL of the Concourse ATCs. See https://concourse-ci.org/implementing-resource-types.html#resource-metadata for more details about metadata that is available via environment variables.

Example

resource_types:
- name: pull-request
  type: docker-image
  source:
    repository: teliaoss/github-pr-resource

resources:
- name: pull-request
  type: pull-request
  check_every: 24h
  webhook_token: ((webhook-token))
  source:
    repository: itsdalmo/test-repository
    access_token: ((github-access-token))

jobs:
- name: test
  plan:
  - get: pull-request
    trigger: true
    version: every
  - put: pull-request
    params:
      path: pull-request
      status: pending
  - task: unit-test
    config:
      platform: linux
      image_resource:
        type: docker-image
        source: {repository: alpine/git, tag: "latest"}
      inputs:
        - name: pull-request
      run:
        path: /bin/sh
        args:
          - -xce
          - |
            cd pull-request
            git log --graph --all --color --pretty=format:"%x1b[31m%h%x09%x1b[32m%d%x1b[0m%x20%s" > log.txt
            cat log.txt
    on_failure:
      put: pull-request
      params:
        path: pull-request
        status: failure
  - put: pull-request
    params:
      path: pull-request
      status: success

Costs

The Github API(s) have a rate limit of 5000 requests per hour (per user). This resource will incur the following costs:

• check: Minimum 1, max 1 per 100th open pull request.
• in: Fixed cost of 1. Fetches the pull request at the given commit.
• out: Minimum 1, max 3 (1 for each of status, comment and comment_file).

E.g., typical use for a repository with 125 open pull requests will incur the following costs for every commit:

• check: 2 (paginate 125 PR's with 100 per page)
• in: 1 (fetch the pull request at the given commit ref)
• out: 1 (set status on the commit)

With a rate limit of 5000 per hour, it could handle 1250 commits between all of the 125 open pull requests in the span of that hour.

Migrating

If you are coming from jtarchie/github-pullrequest-resource, its important to know that this resource is inspired by but not a drop-in replacement for the original. Here are some important differences:

New parameters:

• source:
  • v4_endpoint (see description above)
• put:
  • comment (see description above)

Parameters that have been renamed:

• source:
  • repo -> repository
  • ci_skip -> disable_ci_skip (the logic has been inverted and its true by default)
  • api_endpoint -> v3_endpoint
  • base -> base_branch
  • base_url -> target_url
  • require_review_approval -> required_review_approvals (bool to int)
• get:
  • git.depth -> git_depth
• put:
  • comment -> comment_file (because we added comment)

Parameters that are no longer needed:

• src:
  • uri: We fetch the URI directly from the Github API instead.
  • private_key: We clone over HTTPS using the access token for authentication.
  • username: Same as above
  • password: Same as above
  • only_mergeable: We are opinionated and simply fail to get if it does not merge.
• get:
  • fetch_merge: We are opinionated and always do a fetch_merge.

Parameters that did not make it:

• src:
  • authorship_restriction
  • label
  • git_config: You can now get the pr/author info from .git/resource/metadata.json instead
• get:
  • git.* (with the exception of git_depth, see above)
• put:
  • merge.*
  • label

Note that if you are migrating from the original resource on a Concourse version prior to v5.0.0, you might see an error failed to unmarshal request: json: unknown field "ref". The solution is to rename the resource so that the history is wiped. See #64 for details.