Git Product home page Git Product logo

github-codebuild-integration's Introduction

github-codebuild-integration (gci)

GitHub release MIT License

gci is a CI dispatching/status handling tool to integrate AWS CodeBuild with GitHub Push/Pull-Request webhook events, created with love of Serverless Architecture.

Overview

Yay, Serverless!

Overview

Commits:

Commits

Pull Request:

Pull Request

Features

  • Invoking a pre-configured AWS CodeBuild project by hooking Push or Pull Reqeust webhook events.
  • Setting GitHub's CI status based on status/result of builds on AWS CodeBuild.

AWS account / gci / GitHub repository

gci allows you to provision multiple installations in one AWS account as follows:

Resources Relation
AWS account : gci installations 1 : n
gci installation : GitHub repository 1 : 1
GitHub repository : AWS CodeBuild project 1 : 1 (will be extended to 1 : n in the future)

As mentioned above, gci can be installed as many as you want to integrate with your GitHub repositories. If you want to build 3 repositories, you may provision 3 of gci installation for instance.

Background

GitHub has a feature to show each commit's status like 'success', 'failure', 'pending' on their Commit/PR pages, and based on that status, we can protect any branches from CI failed branch to be merged.

GitHub accepts status creation via their APIs and many third-party CI services implement functionalities to integrate with that APIs to show their job status on GitHub.

On the other hand, AWS CodeBuild doesn't have such a feature to save its build project status to GitHub for now. gci is a missing piece of AWS CodeBuild to make things better. I heard that AWS CodeBuild now supports for building by GitHub push/PR and for updating GitHub status natively. We'll find another motivation to keep developing gci ๐Ÿ˜›

Requirements

Prerequisites

gci requires the following to be installed on your AWS account.

Required Tools

We use a Makefile to manage gci's lifecycle.

  • GNU Make (if you are using macOS, brew install make is handy)

The Makefile depends on the following tools:

  • Node.js v14.1.0 or later
  • Yarn 1.22.0 or later
  • AWS-CLI 1.11.132 or later
  • curl

Required Accounts & Resources

  • AWS Account
  • AdministratorAccess to your AWS Account (to use AWS CloudFormation in the installation command)
  • GitHub Account

And the listed resources below are created in the process of installation, which means they are required as available AWS services in a region where you want to run gci.

  • Amazon S3
  • Amazon SNS
  • Amazon CloudWatch Events
  • AWS Lambda
  • AWS CodeBuild
  • AWS IAMv
  • AWS CloudFormation

Installation

NOTE: Make sure you already have:
- AWS credentials for AWS CLI access.
- at least one AWS CodeBuild project created.

AWS Account-wide Resource

Create an S3 Bucket to store gci's artifacts to proceed.

$ aws s3api create-bucket \
    --bucket {YOUR_S3_BUCKET_NAME} \
    --create-bucket-configuration LocationConstraint=$AWS_DEFAULT_REGION

NOTE: Create an S3 bucket for each AWS region if you use gci in multiple AWS regions.

Per-Project Resources

Clone this repository

$ git clone https://github.com/toricls/github-codebuild-integration.git
$ cd $(pwd)/github-codebuild-integration

Create GitHub Personal Access Token

Open New personal access token page and create one for a gci's installation.

Input token description like codebuild-YOUR_REPO_NAME and enable admin:repo_hook and repo:status as scopes, then click the Generate token button.

Copy the personal access token value and proceed to the next section, "Configure parameters".

Configure parameters

Copy example configuration file and edit it to configure paramters for your GitHub repository.

$ export YOUR_PROJECT_NAME=xxxxxxxxxxxxxxxxx
$ cp env/example.env env/$YOUR_PROJECT_NAME.env
$ editor env/$YOUR_PROJECT_NAME.env

Next table describes about all available parameters of gci.

Required Parameter Name What is this Example
yes S3_SAM_ARTIFACTS_BUCKET_NAME An S3 bucket to store AWS SAM's artifacts. Set the name of the S3 bucket you created on previous step. your.sam.artifacts.bucket
yes GITHUB_REPOSITORY_URL A repository URL you wanted build. Use https style path and make sure trailing '.git' is removed. https://github.com/your-org/your-repo
yes GITHUB_PERSONAL_ACCESS_TOKEN Used for updating GitHub PR's status and Webhook configuration. Minimum scope are admin:repo_hook and repo:status. You can create and obtain a new token via settings page. your-github-personal-access-token
yes GITHUB_TARGET_RESOURCE Choose one event to decide when your CodeBuild project runs. Available value is pr or push. push
optional GITHUB_IGNORE_BRANCH_REGEX Regex string to specify branch name to ignore commit events. This parameter will be enabled only the GITHUB_TARGET_RESOURCE value is push. wip.*
yes AWS_DEFAULT_REGION The region where you want to provision this tool via CloudFormation. us-east-1
yes CODEBUILD_PROJECT_NAME The AWS CodeBuild project name you've already configured for your GitHub repository. your-codebuild-project-name
yes CODEBUILD_PROJECT_REGION The region where you've created a CodeBuild project. You can specify a different region from the region of CloudFormation. us-east-1
optional BUILD_SKIPPED_BY Build invocation will be skipped if the head commit message includes the value of this parameter. This parameter will be used only the GITHUB_TARGET_RESOURCE value is push. "skip ci"

Deploy

Package all artifacts and deploy to your AWS account. You can use this command to update your existing gci installation.

$ make deploy ENV_FILE_PATH=env/$YOUR_PROJECT_NAME.env

Uninstall

You can delete most of generated resources by executing:

$ make destroy ENV_FILE_PATH=env/YOUR-PROJECT-NAME.env

NOTE: CloudFormation doesn't delete CloudWatch's Log Groups. You may want to remove it manually on the AWS Management Console or via the AWS CLI. Also you may want to remove the S3 Bucket(s) you created.

FAQ

Installation & Uninstallation

Q. My IAM role is too weak to install your tool, I guess.

A. Ask your administrator and show him/her the following:

Q: I want to remove all resources of gci from my AWS account.

A: Read the Uninstall section above :X

Changing Configurations

Q: I changed my repository name after gci install.

A: Change the value of GITHUB_REPOSITORY_URL in your env file, then deploy again.

Q: I want to stop CI invoking for a bit.

A: Open GitHubWebhookHandler function (the function name on the Lambda Management Console may something like YOUR-PROJECT-NAME-GitHubWebhookHandler-XXXXXXXXXXX), then set true to the environment variable DO_NOT_RUN.
Don't forget to back that value to false after your quick work.

Feature Request

Q: I need more than one AWS CodeBuild project for my GitHub repository.

A: I totally agree with you! It will be supported in the future. I think the feature will be implemented with a mapping configuration for 'branch name reg-expressions' and 'AWS CodeBuild projects'. But PRs are always welcome ๐Ÿ˜ƒ

Q. Can you change the icon which shown at PR page's CI status?

A. GitHub shows the avatar of the user who owns the personal access token you provided. You can change the icon by using something like Machine users to create a personal access token for gci.

Contribution

  1. Fork (https://github.com/toricls/github-codebuild-integration/fork)
  2. Create a feature branch
  3. Commit your changes
  4. Rebase your local changes against the master branch
  5. Create a new Pull Request

Licence

MIT

Author

toricls

github-codebuild-integration's People

Contributors

dependabot[bot] avatar toricls avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

github-codebuild-integration's Issues

Slack notification

I want to know about build status changes without opening GitHub. Slack notification would be very handy ๐Ÿ˜ƒ

Expose builds status via Views

To show build status and its history even if viewer doesn't have any role for AWS Management Console.

Expected use-case is for open source projects or something like that.


split from #3 to make it simple

Update to documentation

I think we are going to need and env variable set before running make
export ENV_FILE_PATH=$(pwd)/env/$YOUR_PROJECT_NAME.env

I was not able to run it on my mac until I set this one, which I figured out looking into the makefile.

Deploy Error: 'As of October 2018, GitHub Services can no longer be added.'

CloudWatch Logs

2019-01-04T06:00:36.687Z	0d594680-0fe6-11e9-9a88-eb8893158f5c	Response body:
 {"Status":"FAILED","Reason":"See the details in CloudWatch Log Stream: 2019/01/04/[$LATEST]13b0b45ca11744a189d233bbac77****","PhysicalResourceId":"2019/01/04/[$LATEST]13b0b45ca11744a189d233bbac77****","StackId":"arn:aws:cloudformation:ap-northeast-1:52829650****:stack/codebuild-sample/e0495320-0fe5-11e9-8940-0ec45775****","RequestId":"ccaf9c0a-3feb-4f91-afae-a260c332256b","LogicalResourceId":"GitHubWebhook","Data":{"Error":"{\"message\":\"As of October 2018, GitHub Services can no longer be added. Please see the blog post for details: https://developer.github.com/changes/2018-04-25-github-services-deprecation\\nYou can use the \\\"Replacing GitHub Services\\\" guide to help you update your services to webhooks: https://developer.github.com/v3/guides/replacing-github-services\\n\",\"documentation_url\":\"https://developer.github.com/v3/repos/hooks/#create-a-hook\"}"}}

Use Amazon CloudWatch Events instead of AWS Step Functions

We're using AWS Step Functions to wait for build jobs on AWS CodeBuild to complete. Since github-codebuild-integration has built from the context of Serverless Architecture, it should be replaced with Amazon CloudWatch Events to treat build job as an "event" not a status.

To achieve that:

  • Remove AWS Step Functions stuffs from the code and the CloudFormation template
  • Invoke build-result-notifier Lambda function by CloudWatch Events which is associated with CodeBuild project's status changing

Enhanced configuration

The current way to use .env file for configuration is too poor to express more complicated or advanced parameters. e.g. arrays, sructured/typed values, maps...

We should consider to use something like YAML, TOML or JSON to represent gci's configurations.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.