Git Product home page Git Product logo

aws-codebuild-samples's Introduction

AWS CodeBuild Samples

Utilities and samples for building on CodeBuild

Sample App: Simple Calculator Service

CodeBuild badge

Simple Node.js Express-based web service that demonstrates continuous integration with AWS CodeBuild, AWS CodeCommit, and GitHub, as well as continuous deployment with AWS CodeDeploy/CodePipeline. This application was written for demo purposes only, and is definitely not production ready.

CI Tooling (Buildspecs)

The buildspecs folder contains the following buildspec files for use with AWS CodeBuild:

  • build.yml: Basic npm-based build with unit tests and code coverage report.
  • shrinkwrap.yml: Upgrade npm dependencies and push an updated shrinkwrap file to source code repository.
  • sonarqube.yml: Run static code analysis against a SonarQube endpoint, with the endpoint and token stored in SSM Parameter Store.
  • build-test-suite-1.yml and build-test-suite-2.yml: Parallelize the unit tests into two separate projects.
  • environment.yml: Build a Docker image with cached npm dependencies
  • build-with-image-cache.yml: Use the cached npm dependencies from a custom Docker image.

CI Tooling (Glue)

The ci_tools folder contains the following tools for use with AWS Lambda and Amazon CloudWatch Events to hook together the end-to-end CI process:

  • trigger_codebuild.js: Lambda function to start a CodeBuild build.
  • slack_notifications.js: Lambda function to post CodeBuild build notifications into a Slack channel.
  • email_notifications.js: Lambda function to send CodeBuild build notification emails via Amazon SES.
  • codecommit_pr_notifications: Lambda function to start a CodeBuild build for CodeCommit pull request notifications, and comment on a CodeCommit pull request for CodeCommit build notifications.
  • cwe-rule-configuration/branch_ci.json: CloudWatch Events rule pattern to start a CodeBuild build for every push to the master branch of a CodeCommit repository.
  • cwe-rule-configuration/slack_event_pattern.json: CloudWatch Events rule pattern to notify Slack for failed CodeBuild builds.
  • cwe-rule-configuration/nightly_build_input.json: CloudWatch Events target input to start a CodeBuild build with a specific buildspec override and project name.

CD Tooling

Contains an appspec.yml file and deploy_scripts folder for deploying the service with AWS CodeDeploy.

CloudFormation Templates

Create a CodeCommit repository called 'aws-codebuild-samples' and push this sample code into the repo. Then spin up all of the above easily with CloudFormation.

Continuous Deployment

Set up continuous deployment with a CodePipeline pipeline:

aws cloudformation deploy --stack-name aws-codebuild-samples --template-file cloudformation/continuous-deployment.yml --capabilities CAPABILITY_NAMED_IAM

aws cloudformation describe-stacks --stack-name aws-codebuild-samples --query 'Stacks[0].Outputs[?OutputKey==`PipelineConsoleUrl`].OutputValue' --output text

Wait for the pipeline to finish deploying, then access the Test and Prod stack applications:

aws cloudformation describe-stacks --stack-name aws-codebuild-samples-test-stack --query 'Stacks[0].Outputs[?OutputKey==`Url`].OutputValue' --output text

aws cloudformation describe-stacks --stack-name aws-codebuild-samples-prod-stack --query 'Stacks[0].Outputs[?OutputKey==`Url`].OutputValue' --output text

Continuous Integration: Nightly Checks

Choose an email address for receiving email notifications. Then, verify the email address in SES before setting up the CloudFormation stack.

mkdir build

S3_BUCKET=$(aws cloudformation describe-stacks --stack-name aws-codebuild-samples --query 'Stacks[0].Outputs[?OutputKey==`ArtifactsBucket`].OutputValue' --output text)

aws cloudformation package --template-file cloudformation/continuous-integration-nightly-checks.yml --s3-bucket $S3_BUCKET --force-upload --output-template-file build/continuous-integration-nightly-checks.yml

aws cloudformation deploy --stack-name aws-codebuild-samples-nightly-checks --template-file build/continuous-integration-nightly-checks.yml --capabilities CAPABILITY_NAMED_IAM --parameter-overrides NotificationEmailAddress="[email protected]"

Continuous Integration: Branch Checks

Configure the webhook in Slack:

  1. Navigate to https://.slack.com/apps
  2. Search for and select "Incoming Webhooks".
  3. Click "Add Configuration".
  4. Choose a channel and click "Add Slash Command Integration".
  5. Copy the Webhook URL from the integration settings.
  6. Store the token in Parameter Store: aws ssm put-parameter --name codebuild-samples-slack-webhook --type SecureString --value <webhook URL> (Note: you will need to set cli_follow_urlparam = false in your AWS CLI config file first)

Then spin up the stack in CloudFormation:

mkdir build

S3_BUCKET=$(aws cloudformation describe-stacks --stack-name aws-codebuild-samples --query 'Stacks[0].Outputs[?OutputKey==`ArtifactsBucket`].OutputValue' --output text)

aws cloudformation package --template-file cloudformation/continuous-integration-branch-checks.yml --s3-bucket $S3_BUCKET --force-upload --output-template-file build/continuous-integration-branch-checks.yml

aws cloudformation deploy --stack-name aws-codebuild-samples-branch-checks --template-file build/continuous-integration-branch-checks.yml --capabilities CAPABILITY_NAMED_IAM

Continuous Integration: Pull Request Checks

Spin up the stack in CloudFormation:

mkdir build

S3_BUCKET=$(aws cloudformation describe-stacks --stack-name aws-codebuild-samples --query 'Stacks[0].Outputs[?OutputKey==`ArtifactsBucket`].OutputValue' --output text)

aws cloudformation package --template-file cloudformation/continuous-integration-pull-request-checks.yml --s3-bucket $S3_BUCKET --force-upload --output-template-file build/continuous-integration-pull-request-checks.yml

aws cloudformation deploy --stack-name aws-codebuild-samples-pull-request-checks --template-file build/continuous-integration-pull-request-checks.yml --capabilities CAPABILITY_NAMED_IAM

License

This library is licensed under the Apache 2.0 License.

aws-codebuild-samples's People

Contributors

abiydv avatar chriscoombs avatar clareliguori avatar dependabot[bot] avatar hardyeats avatar jpeddicord avatar subinataws avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aws-codebuild-samples's Issues

Caching won't work with node_modules.

CodeBuild keeps its cache locally and then symlinks it to our target location as evidenced by this line:

[Container] 2019/02/27 02:24:38 Symlinking: /codebuild/output/src718408252/src/github.com/dashmug/api-tests/node_modules => /codebuild/local-cache/custom/dba27c31aad935787bb275c3e5e4e957708f15386de599eff1db476022cd7e4c

This behaviour, however, breaks Node's module resolution causing our Node apps to fail.

internal/modules/cjs/loader.js:596
throw err;
^
Error: Cannot find module 'source-map-support'
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:594:15)
at Function.Module._load (internal/modules/cjs/loader.js:520:25)
at Module.require (internal/modules/cjs/loader.js:650:17)
at require (internal/modules/cjs/helpers.js:20:18)

This is a widely-known issue.
npm/npm#9479
nodejs/node-v0.x-archive#25090

TLDR: node's require() doesn't work with a symlinked node_modules directory if the source directory is not named node_modules.

Because of this bug in node, having the following in buildspec.yml will fail:

cache:
  paths:
    - 'node_modules/**/*'

How to use your code?

Hi
Thank you for our code
I need your help with the loading of your code to aws code commit.
I ran the first command and it failed because the aws code commit has no code. so I need to load the your code to it but 'm not sure which code files I need to push to the aws code commit.
Please help guide me through this. I'm quite new to git too. I do know how to create git and am able to connect to the AWS CodeComiit... but I'm not sure what source code files I need to push to the repo?
Thanks
John Nguyen

Lerna command in AWS Codebuild

    Error: Cannot find module '.'
    Require stack:
    - /codebuild/output/src309915592/src/node_modules/.bin/lerna
        at Function.Module._resolveFilename (internal/modules/cjs/loader.js:831:15)
        at Function.Module._load (internal/modules/cjs/loader.js:687:27)
        at Module.require (internal/modules/cjs/loader.js:903:19)
        at require (internal/modules/cjs/helpers.js:74:18)
        at Object.<anonymous> (/codebuild/output/src309915592/src/node_modules/.bin/lerna:11:3)
        at Module._compile (internal/modules/cjs/loader.js:1015:30)
        at Object.Module._extensions..js (internal/modules/cjs/loader.js:1035:10)
        at Module.load (internal/modules/cjs/loader.js:879:32)
        at Function.Module._load (internal/modules/cjs/loader.js:724:14)
        at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12) {
      code: 'MODULE_NOT_FOUND',
      requireStack: [ '/codebuild/output/src309915592/src/node_modules/.bin/lerna' ]
    }

I get this error on running the test command:

lerna run --stream test --parallel Also, tried with

lerna run --stream test --parallel --scope tenant-registration-service In the services, test script looks like,

"test": "serverless-bundle test --verbose --coverage", I installed Lerna globally in the pipeline. How do I debug this? What module is it not able to find? It looks like it already has lerna installed which it should.

npm run test command runs just fine on the local system.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.