Git Product home page Git Product logo

guides-configuration's Introduction

Guides Configuration

This repo contains Packer templates used by HashiCorp Terraform modules in the hashicorp-modules GitHub Org.

Building HashiStack images locally (outside of the CI pipeline)

This is a workflow that's designed to allow you to trigger local builds of enterprise or OSS Packer images. This functionality is currently under development.

This is particularly useful for customers using Azure, as it's not possible to share machine images.

Prerequisites

An example using Packer to build images on Azure

After authenticating (see above) with Azure, perform the following steps.

  • Authenticate with Azure using the Azure setup instructions.

  • Create a file like the below with your credentials and source it before running the next step. You can skip this step if you want. The azure-local-env.sh will take care of it for you, asking you to input each variable that is not already set in your environment.

    vi env.sh
    
    #!/bin/bash
    # env.sh
    # Exporting variables in both cases just in case, no pun intended
    export ARM_SUBSCRIPTION_ID="aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"
    export ARM_CLIENT_ID="bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb"
    export ARM_CLIENT_SECRET="cccccccc-cccc-cccc-cccc-cccccccccccc"
    export ARM_TENANT_ID="dddddddd-dddd-dddd-dddd-dddddddddddd"
    export subscription_id="aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"
    export client_id="bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb"
    export client_secret="cccccccc-cccc-cccc-cccc-cccccccccccc"
    
    source env.sh
    
  • With the root of this repo as your working directory, run the following before each packer build:

    # Source azure-local-env.sh before each packer build to regenerate URLs,
    # as the enterprise download URLs expire after 10 minutes.
    #
    $ source azure-local-env.sh # aws-local-env.sh for AWS (AWS local build untested)
    $ cd hashistack
    $ packer build hashistack-azure.json
    

Consul

Contains Consul specific installation scripts, configuration files. Also has Packer templates specific to Consul usage.

Example AWS Consul build command:

source aws-local-env.sh
AWS_REGION="us-west-1" packer build consul-aws.json

Example Azure Consul build command:

source azure-local-env.sh
AZURE_RESOURCE_GROUP="PackerImages" AZURE_LOCATION="West US" PACKER_ENVIRONMENT="dev" CONSUL_VERSION="0.9.2" packer build consul-azure.json

Vault

Contains Vault specific installation scripts, configuration files. Also has Packer templates specific to Vault usage.

Example AWS Vault (including Consul) build command:

source aws-local-env.sh
AWS_REGION="us-west-1" packer build vault-aws.json

Nomad

Contains Nomad specific installation scripts, configuration files. Also has Packer templates specific to Nomad usage.

Example AWS Nomad (including Consul) build command:

source aws-local-env.sh
AWS_REGION="us-west-1" packer build nomad-aws.json

HashiStack

Contains provider specific templates that installs HashiCorp software on a single node (Consul, Nomad, Vault, consul-template and envconsul).

Example AWS HashiStack build command:

source aws-local-env.sh
AWS_REGION="us-west-1" packer build hashistack-aws.json

Example Azure HashiStack build command:

source azure-local-env.sh
VCS_NAME="local" PACKER_ENVIRONMENT="production" CONSUL_VERSION="1.2.0" VAULT_VERSION="0.10.3" NOMAD_VERSION="0.8.4" packer build hashistack-azure.json

Example GCP HashiStack build command:

source gcp-local-env.sh
VCS_NAME="local" PACKER_ENVIRONMENT="production" CONSUL_VERSION="1.2.0" VAULT_VERSION="0.10.3" NOMAD_VERSION="0.8.4" packer build hashistack-gcp.json

Continuous Integration

Product versions for the builds are set on the versions.sh file. This file should be sourced at the start of the CI Build process. The ci-functions.sh file should be sourced at the start of the CI Build process and introduces three functions:

  • prepare() will download the ${PACKER_VERSION}
  • validate() will run packer validate on the packer templates, for a set of arguments (like consul nomad vault hashistack)
  • build() will build and deploy the images

The following script can be used to parallelize the image build process:

build consul &
build vault &
build nomad &
build hashistack &

for job in `jobs -p`; do
  echo $job
  wait $job || let "FAIL+=1"
done

echo $FAIL

if [ "$FAIL" == "0" ]; then
  echo -e "\033[32m\033[1m[BUILD SUCCESFUL]\033[0m"
else
  echo -e "\033[31m\033[1m[BUILD ERROR]\033[0m"
fi

Some notes on the CI pipeline

Image Version Table:

Release Consul Vault Nomad
0.1.0 1.2.0 0.10.3 0.8.4
0.1.1 1.2.0-ent 0.10.3-ent 0.8.4-ent
0.1.2 1.2.1 0.10.3 0.8.4
0.1.3 1.2.1-ent 0.10.3-ent 0.8.4-ent

guides-configuration's People

Contributors

andrew-klaas avatar bensojona avatar calebalbers avatar greenbrian avatar jharley avatar marsupermammal avatar ncorrare avatar norhe avatar rberlind avatar roooms avatar scarolan avatar tdsacilowski avatar thehob avatar vvitayau avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

guides-configuration's Issues

I *think* the nomad systemd files are slightly wrong

I am a systemd novice, but reading other systemd .target and .service files, and some docs, I think the files here are slightly wrong: https://github.com/hashicorp/guides-configuration/tree/master/nomad/init/systemd

If one looks in https://github.com/hashicorp/guides-configuration/tree/master/consul/init/systemd there is this pattern:

consul-online.service is WantedBy consul-online.target

So, I understand that as "once consul-online-service succeeds, we will have arrived at consul-online.target

However, looking at https://github.com/hashicorp/guides-configuration/blob/master/nomad/init/systemd/

I see in nomad-vault.service this line: Requires=consul-online.target vault-token-ready.target

However, in vault-token-ready.service I see:

WantedBy=vault-ready.target multi-user.target

There is no vault-ready.target in https://github.com/hashicorp/guides-configuration/tree/master/vault/init/systemd, so I am assuming the line in vault-token-ready.service should be WantedBy=vault-token-ready.target multi-user.target

Am I understanding these files and systemd correctly?

Use #!/bin/bash instead of #!/usr/bin/env bash in install-vault.sh and install-consul.sh

I suggest using #!/bin/bash, as #!/usr/bin/env bash does not pass the environment variables in. I modified the script and trying to run aws s3 cp s3://bucketstuff... but the AWS environment variables were not being passed in through environment_vars. Only worked once I moved install-vault.sh and install-consul.sh over to #!/bin/bash.

I'll submit a PR soon. Trying to meet a deadline at the moment so can't right now.

@bensojona I promise I'll reply to your email soon lol.

"Spot Price" line causes an error

This line:

  "spot_price_auto_product": "Linux/UNIX (Amazon VPC)",

Caused an error at a partner who was trying to do packer builds. Apparently this config also required the spot_price be set to auto as well. I'm not sure what in their settings triggered that. Removing any spot price references fixed the issue.

Vault Enterprise currently requires public URL in this guide

I'd rather not expose the URL; obviously HashiCorp would not want the enterprise image to be publicly exposed.

For the aws version, can you adjust the script so that it copies the image from an S3 bucket? Basic flow:

  1. User supplies S3 bucket name (perhaps via environment variable, or variables file)
  2. awscli is included in the packer build on the ami image
  3. in the bootup script, aws s3 cp s3://{bucketnameandurletc} /tmp (instead of curl)

Default download URL for Nomad is broken

It's fetching this default URL from somewhere, but this doesn't exist:

NOMAD_URL=https://releases.hashicorp.com/nomad//nomad__linux_amd64.zip

Setting NOMAD_ENT_URL to a valid download URL seems to fix the issue:

export NOMAD_ENT_URL=https://releases.hashicorp.com/nomad/0.8.3/nomad_0.8.3_linux_amd64.zip

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.