Git Product home page Git Product logo

invoicer-chapter2's Introduction

Securing DevOps's invoicer

A simple REST API that manages invoices.

This is the code for Chapter 2 of Securing DevOps. It only contains the code and scripts relevant to the basic setup of the invoicer application and infrastructure.

The master branch is kept at https://securing-devops.com/invoicer but if you are interested in chapter-specific versions of the invoicer.

Get your own copy

To try out the code in this repository, first create a fork in your own github account. Now before you do anything, edit the file in .circleci/config.yml and replace the working_directory parameter with your own namespace.

For example:

    working_directory: /go/src/github.com/Securing-DevOps/invoicer-chapter2

would become:

    working_directory: /go/src/github.com/jvehent/invoicer-chapter2

Then, sign up for circleci and build the project.The build will initially fail because it lacks the Docker credentials to push the invoicer's image to dockerhub.

Head over to hub.docker.com, create an account, then create a repository to store the docker containers into, as explained in chapter 2.

Head back to CircleCi and in the "Project Setting", add the following "Environment Variables" with the appropriate values.

  • DOCKER_USER
  • DOCKER_PASS

These variable will allow CircleCI to upload the docker container of the invoicer into your own account. Once added, trigger a rebuild of the CircleCI job and it should succeed and upload the container without issue.

You can then pull and run the container from your repository as follows:

$ docker run -it <MyGitHubUser>/invoicer

Build the AWS infrastructure

The script create_ebs_env.sh creates a complete AWS infrastructure ready to host the invoicer. The script first creates a database, then an elastic beanstalk environment, and finally deploys the docker container of the invoicer. All you need to get started is creating an AWS account, then create a local profile (see chapter 2 for details) and run the following:

$ export AWS_PROFILE=cloudservices-aws-dev

$ export AWS_REGION=eu-west-1

$ ./create_ebs_env.sh

The script will outputs resources name and credentials for the new environment.

Creating EBS application ulfr-invoicer-201709231530
default vpc is vpc-c02b81a5
DB security group is sg-fc20478f
RDS Postgres database is being created. username=invoicer; password='i0l2jSQ5WkK_441c8dXwlYods9'
..................dbhost=ulfr-invoicer-201709231530.czvvrkdqhklf.us-east-1.rds.amazonaws.com
ElasticBeanTalk application created
API environment e-qp2pyjhhma is being created
.....
API security group sg-732d4a00 authorized to connect to database security group sg-fc20478f
make_bucket: ulfr-invoicer-201709231530
upload: ./app-version.json to s3://ulfr-invoicer-201709231530/app-version.json
waiting for environment....................
Environment is being deployed. Public endpoint is http://ulfr-invoicer-201709231530-invoicer-api.egmh2pupxy.us-east-1.elasticbeanstalk.com

Build the AWS infrastructure

The script create_ebs_env.sh creates a complete AWS infrastructure ready to host the invoicer. The script first creates a database, then an elastic beanstalk environment, and finally deploys the docker container of invoicer-chapter2. All you need to get started is creating an AWS account, then create a local profile (see chapter 2 for details) and run the following:

$ export AWS_PROFILE=cloudservices-aws-dev

$ export AWS_REGION=eu-west-1

$ ./create_ebs_env.sh

The script will outputs resources name and credentials for the new environment.

Creating EBS application ulfr-invoicer-201709231530
default vpc is vpc-c02b81a5
DB security group is sg-fc20478f
RDS Postgres database is being created. username=invoicer; password='i0l2jSQ5WkK_441c8dXwlYods9'
..................dbhost=ulfr-invoicer-201709231530.czvvrkdqhklf.us-east-1.rds.amazonaws.com
ElasticBeanTalk application created
API environment e-qp2pyjhhma is being created
.....
API security group sg-732d4a00 authorized to connect to database security group sg-fc20478f
make_bucket: ulfr-invoicer-201709231530
upload: ./app-version.json to s3://ulfr-invoicer-201709231530/app-version.json
waiting for environment....................
Environment is being deployed. Public endpoint is http://ulfr-invoicer-201709231530-invoicer-api.egmh2pupxy.us-east-1.elasticbeanstalk.com

Manual build

Build a statically linked invoicer binary. Requires Go 1.6.

$ mkdir bin
$ go build --ldflags '-extldflags "-static"' -o bin/invoicer .

Then build the container.

$ docker build -t securingdevops/invoicer .

Database Configuration

The invoicer will automatically create a local sqlite database by default, but if you want to run with postgres, follow these instructions.

Create a postgres database named invoicer and grant user invoicer full access to it.

CREATE DATABASE invoicer;
CREATE ROLE invoicer;
ALTER ROLE invoicer WITH NOSUPERUSER INHERIT NOCREATEROLE NOCREATEDB LOGIN PASSWORD 'invoicer';
ALTER DATABASE invoicer OWNER TO invoicer;

When running PG and Docker on the same box, configure pg_hba to allow the local docker network to connect.

...
# trust local docker hosts
host    all             all             172.17.0.0/16           trust

Run

$ docker run -it \
    -e INVOICER_USE_POSTGRES="yes" \
    -e INVOICER_POSTGRES_USER="invoicer" \
    -e INVOICER_POSTGRES_PASSWORD="invoicer" \
    -e INVOICER_POSTGRES_HOST="172.17.0.1" \
    -e INVOICER_POSTGRES_DB="invoicer" \
    -e INVOICER_POSTGRES_SSLMODE="disable" \
    securingdevops/invoicer-chapter2

Use

Create an invoice

$ curl -X POST \
--data '{"is_paid": false, "amount": 1664, "due_date": "2016-05-07T23:00:00Z", "charges": [ { "type":"blood work", "amount": 1664, "description": "blood work" } ] }' \
http://172.17.0.2:8080/invoice

Retrieve an invoice

$ curl http://172.17.0.2:8080/invoice/1
{"ID":1,"CreatedAt":"2016-05-21T15:33:21.855874Z","UpdatedAt":"2016-05-21T15:33:21.855874Z","DeletedAt":null,"is_paid":false,"amount":1664,"payment_date":"0001-01-01T00:00:00Z","due_date":"2016-05-07T23:00:00Z","charges":[{"ID":1,"CreatedAt":"2016-05-21T15:33:21.8637Z","UpdatedAt":"2016-05-21T15:33:21.8637Z","DeletedAt":null,"invoice_id":1,"type":"blood
work","amount":1664,"description":"blood work"}]}

invoicer-chapter2's People

Contributors

jvehent avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

invoicer-chapter2's Issues

Invoicer compiling issue on mac arm64

I installed the arm based version of go on my mac, when trying to compile the source code for invoicer 2.4 i get the following error.

$ go install --ldflags '-extldflags "-static"' -v github.com/Securing-Devops/invoicer-chapter2@latest
go: finding module for package go.mozilla.org/mozlog
go: finding module for package github.com/gorilla/mux
go: finding module for package github.com/jinzhu/gorm/dialects/sqlite
go: finding module for package github.com/jinzhu/gorm
go: finding module for package github.com/wader/gormstore
go: finding module for package github.com/jinzhu/gorm/dialects/postgres
go: found github.com/gorilla/mux in github.com/gorilla/mux v1.8.0
go: found github.com/jinzhu/gorm in github.com/jinzhu/gorm v1.9.16
go: found github.com/jinzhu/gorm/dialects/postgres in github.com/jinzhu/gorm v1.9.16
go: found github.com/jinzhu/gorm/dialects/sqlite in github.com/jinzhu/gorm v1.9.16
go: found github.com/wader/gormstore in github.com/wader/gormstore v0.0.0-20220402164052-db5a9afbfd30
go: found go.mozilla.org/mozlog in go.mozilla.org/mozlog v0.0.0-20170222151521-4bb13139d403
# github.com/Securing-Devops/invoicer-chapter2
/usr/local/go/pkg/tool/darwin_arm64/link: running clang failed: exit status 1
ld: library not found for -lcrt0.o
clang: error: linker command failed with exit code 1 (use -v to see invocation)

clang version

$ clang --version                                                                                    
Apple clang version 13.1.6 (clang-1316.0.21.2)
Target: x86_64-apple-darwin21.4.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

Please let me know if you need anything else to reproduce, any guidance on how to fix error would be appreciated.

Cannot load package

I am following the instructions in chapter 2 and in the readme.md file to setup my environment. I have registered for circleCI and docker. I have imported my forked invoicer app and setup a docker repo and added the DOCKER_USER and DOCKER_PASS to the project settings. I have updated the working directory in my config.yml to: /go/src/github.com/jasonduquain/invoicer-chapter2. When I trigger a rebuild I get the following error in the 'testing application' section of circleCI:

	/usr/local/go/src/github.com/JasonDuquain/invoicer-chapter2 (from $GOROOT)
	/go/src/github.com/JasonDuquain/invoicer-chapter2 (from $GOPATH)
Exited with code 1````

I have searched around and maybe I need to run some go commands in order to get a bulidable source?:

https://github.com/hu17889/go_spider/issues/26

I tried intalling go and doing the equivelent commands for this app but I get errors when running them and I am not even sure if this needs to be done.

Can you assist at all with this? I am running Windows 10 and if you need any output from the circleCI failure let me know. I should also mention I am a security guy with node/express/mongo/react experience but none with Go so I may be missing something that is obvious and if so please forgive my ignorance. 

cannot load package

I am following the instructions in chapter 2 and in the readme.md file to setup my environment. I have registered for circleCI and docker. I have imported my forked invoicer app and setup a docker repo and added the DOCKER_USER and DOCKER_PASS to the project settings. I have updated the working directory in my config.yml to: /go/src/github.com/jasonduquain/invoicer-chapter2. When I trigger a rebuild I get the following error in the 'testing application' section of circleCI:

can't load package: package github.com/JasonDuquain/invoicer-chapter2: cannot find package "github.com/JasonDuquain/invoicer-chapter2" in any of:
/usr/local/go/src/github.com/JasonDuquain/invoicer-chapter2 (from $GOROOT)
/go/src/github.com/JasonDuquain/invoicer-chapter2 (from $GOPATH)
Exited with code 1

I have searched around and maybe I need to run some go commands in order to get a bulidable source?:

hu17889/go_spider#26

I tried intalling go and doing the equivelent commands for this app but I get errors when running them and I am not even sure if this needs to be done.

Can you assist at all with this? I am running Windows 10 and if you need any output from the circleCI failure let me know. I should also mention I am a security guy with node/express/mongo/react experience but none with Go so I may be missing something that is obvious and if so please forgive my ignorance.

main.go test error

I'm still learning Go, so I don't know if this is relevant, but, just FYI, when I built invoicer-chapter2 with Go 1.13 on CircleCI, I got the error below ... Changing "%d" to "%s" fixed it, which may make sense because mux.Vars is a map[string]string ...

#!/bin/bash -eo pipefail
go test \
github.com/${CIRCLE_PROJECT_USERNAME}/${CIRCLE_PROJECT_REPONAME}

# github.com/******/invoicer-chapter2
./main.go:122:3: httpError format %d has arg vars["id"] of wrong type string

Exited with code exit status 2

InvalidParameterValue when updating environment

When calling update-environment (using your create_ebs_env.sh script), I get an error that says:

An error occurred (InvalidParameterValue) when calling the DescribeEnvironmentResources operation: No Environment found for EnvironmentId = 'e-************'.
configuration failed

I've tried this both manually and using your script and I have the same problem. Strangely enough, if I run aws elasticbeanstalk describe-environments the environment does come up, so I don't know why it can't find it. Why is this happening?

Help Please - Invoicer Chapter 2 (Step before AWS infrastructure)

Hi,

Please pardon my ignorance. I bought the securing devops book and i am following the steps in there.

The rebuild of the circleCI job succeeded but i do not know how to go about next step below:

You can then pull and run the container from your repository as follows:

$ docker run -it /invoicer

where do i put in the above command? I have tried to download Docker CLI but it isnt working.

Please advise.

Regards,

Ade

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.