Git Product home page Git Product logo

circleci-public / terraform-orb Goto Github PK

View Code? Open in Web Editor NEW
10.0 11.0 44.0 170 KB

Deploy your infrastructure via a CircleCI pipeline using the Terraform orb. Integrate Infrastructure-as-Code (IaC) to help provision and manage any cloud, infrastructure, or service of your choice.

Home Page: https://circleci.com/orbs/registry/orb/circleci/terraform

License: MIT License

HCL 8.08% Shell 91.92%
terraform circleci-orbs circleci-orb

terraform-orb's Introduction

Terraform Orb for CircleCI CircleCI Build Status CircleCI Orb Version GitHub License CircleCI Community

Run Terraform in your CircleCI Pipelines with the Terraform Orb. Integrate Infrastructure-as-Code (IaC) to help provision and manage any cloud, infrastructure, or service of your choice.

View the usage examples on the Orb Registry.


Resources

CircleCI Orb Registry Page - The official registry page of this orb for all versions, executors, commands, and jobs described.

CircleCI Orb Docs - Docs for using, creating, and publishing CircleCI Orbs.


How to Contribute

We welcome issues to and pull requests against this repository!

How to Publish An Update

  1. Merge pull requests with desired changes to the main branch.
  2. Find the current version of the orb.
    • You can run circleci orb info circleci/terraform | grep "Latest" to see the current version.
  3. Create a new Release on GitHub.
    • Click "Choose a tag" and create a new semantically versioned tag. (ex: v1.0.0)
      • We will have an opportunity to change this before we publish if needed after the next step.
  4. Click "+ Auto-generate release notes".
    • This will create a summary of all of the merged pull requests since the previous release.
    • If you have used Conventional Commit Messages it will be easy to determine what types of changes were made, allowing you to ensure the correct version tag is being published.
  5. Now ensure the version tag selected is semantically accurate based on the changes included.
  6. Click "Publish Release".
    • This will push a new tag and trigger your publishing pipeline on CircleCI.

terraform-orb's People

Contributors

alexjurkiewicz avatar andrebask avatar bendrucker avatar dmechas avatar dsayling avatar jaryt avatar kyletryon avatar kynefuk avatar punkdata avatar samcook avatar sapslaj avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

terraform-orb's Issues

Terraform workspace is expected to exist

Orb Version
1.0.1

Describe the bug
The terraform/plan and terraform/apply (and probably other) jobs require the Terraform workspace to already exist prior to usage.

To Reproduce
Use the terraform/plan or terraform/apply job with a completely empty state and specify a workspace. The call to terraform workspace select will fail.

Expected behavior
The orb creates the workspace before trying to use it and doesn't throw an error.

Additional context
The current workaround I'm using is:

    - terraform/init:
        path: XYZ
        backend_config: *backend_config_prd
    - run:
        name: Create Terraform workspace if it doesn't already exist
        command: terraform workspace list XYZ/ | egrep -q '\sWORKSPACE$' || terraform workspace new WORKSPACE XYZ/
    - terraform/apply:
        path: XYZ
        backend_config: *backend_config_prd
        workspace: WORKSPACE

Preferably I should be able to remove both the terraform/init (as apply and plan do that anyway) and the run blocks.

Path test for tfvar files should accept paths outside of the module_root

Orb Version
3.0.0 and 3.1.0

Describe the bug
In 2.1.0 you can pass in a absolute path to a var-file. which was honoured by

if [[ -f "$file" ]]; then
however in the move to 3.0.0 this check was removed

To Reproduce

  1. create repo with a tfvar file that is outside of the module path, or directory that is set as current working directory
  2. set the command for terraform/plan similar to
- terraform/plan:
    var_file: ${HOME}/common.tfvars

Expected behavior

the terraform/plan command will run because the script passes the test to confirm that the file exists. And not fail with the following error

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
var file '${HOME}/common.tfvars' wasn't found

Exited with code exit status 1

NOTE

This is a duplicate of #56 as the 3.1.0 release did not fix the problem

Support Terraform 0.15.0

Is your feature request related to a problem? Please describe.
Some users would like to start using Terraform 0.15.0, which is not supported by the latest release of the Orb

Describe the solution you'd like
The two PRs being worked on should provide users with support for the latest version and features
#33
#35

"validate" job supports terraform init without accessing the backend.

Is your feature request related to a problem? Please describe.
I simply want to check the format and validate my Terraform source with your orb, and I faced a little trouble.
The validate command run with terraform init -backend=false ..., but the validate job cannot be passed '-backend' option. So if we set to run "terraform validate" without accessing the backend, we must use the validate command in a job and set it to a workflow (refer to below source code).

https://github.com/shirakiya/torii/blob/12f7ed36463859fa37fe4e29bb93a3302b776aff/.circleci/config.yml#L145-L148

I want to use your validate job in my workflow directly.

Describe the solution you'd like
It would be nice to be able to pass the parameter backend to validate job.

Include `terraform taint` command

Is your feature request related to a problem? Please describe.
The terraform taint command is not included in the Terraform orb commands so this functionality is not accessible via the orb.

Describe the solution you'd like
Add taint to the list of available commands in the Terraform orb.

Additional context
Starting from Terraform v1.0 the taint command is deprecated however:

  • The alternative (create a plan with the "replace" option: terraform apply -replace=...) is not available in versions prior to Terraform v1.0
  • The current Terraform orb apply command doesn't seem to allow options to be passed
  • The orb install command defaults to Terraform v0.13.5
  • The default executor uses the 'hashicorp/terraform:0.14.10' image

How to read and use azure blob as remote backend. Example needed please

Hi I am trying to use this orb for to create azure infrastructure with terraform. But struggling to find out how to implement azure blob as remote backend .

Please help with circleci.yml with azure as remote backend

what are variable that I need to need to add in context for this azure remote backend to work ?

Terraform Airbyte Registry No compatible versions of provider registry.terraform.io/airbytehq/airbyte were found.

Orb Version
circleci/[email protected]

When trying to terraform init airbyte provider I got:

Error: Incompatible provider version

│ No compatible versions of provider registry.terraform.io/airbytehq/airbyte were found.

Describe the bug

Circleci is unable to find airbyte provider for terraform

To Reproduce

Steps to reproduce the behavior:

  1. Add airbyte provider to your mainfile https://registry.terraform.io/providers/airbytehq/airbyte/latest/docs
  2. create a config.yml with terraform init command
  3. push the config.yml
  4. Circleci will Error: Incompatible provider version

│ No compatible versions of provider registry.terraform.io/airbytehq/airbyte were found.

Expected behavior

get provider installed

The registry is found when you terraform init

Add any other context about the problem here.

Goocle Cloud Credentials on plan command does not work

Hi all!

The command plan does not recognize google credential variable.
A set this variable in a context, but auth does not work

orbs:
  terraform: circleci/[email protected]
jobs:
  single-job-lifecycle:
    executor: terraform/default
    steps:
      - checkout
      - attach_workspace:
          at: .
      - run:
          command: >-
            rm -rf ~/.gitconfig
          name: clean gitconfg
      - run:
          command: >-
            git config --global url.https://oauth2:[email protected] https://github.com
          name: Github Auth webeta
      - persist_to_workspace:
          root: .
          paths:
            - ""
      - terraform/validate:
          path: .
      - terraform/init:
          backend: true
          path: .
      - terraform/plan:
          path: .
          var: GOOGLE_CREDENTIALS=$GOOGLE_CREDENTIALS
workflows:
  single-job-lifecycle:
    jobs:
      - single-job-lifecycle:
          context: gcp-insane-wizard

any one know how to do this?

terraform/install Fails with /bin/sh: shasum: not found

Orb Version

circleci/[email protected]

Describe the bug

terraform/install fails with error /bin/sh: shasum: not found.

To Reproduce

With the following file as contents of

version: 2.1

orbs:
  terraform: circleci/[email protected]

jobs:
  deploy-infrastructure:
    executor: terraform/default
    steps:
      - checkout
      - terraform/install:
          arch: amd64
          os: linux
          # note that the version must be in quotes
          terraform_version: "0.14.11"
        
workflows:
  version: 2
  deploy-infrastructure:
    jobs:
      - deploy-infrastructure

Expected behavior

Expected behavior is that the terraform file would be installed at the specified version.

** Notes**

We have done the following to resolve, but

apk add perl # to install the sha1sum binary
ln -s /usr/bin/sha1sum /usr/bin/shasum # to link the sha1sum to shasum

Which also results in failure (error: shasum: applet not found) at which point we felt it would be worthwhile to file a bug report as it seems that the hashicorp company's image may not support shasum.

I realize the larger issue here may be that there is no (or little) value in using the hashicorp Docker image (as provided by terraform/default) with terraform/install.

Path test for tfvar files should accept paths outside of the module_root

Orb Version
3.0.0

Describe the bug
In 2.1.0 you can pass in a absolute path to a var-file. which was honoured by

if [[ -f "$file" ]]; then
however in the move to 3.0.0 this check was removed

To Reproduce

  1. create repo with a tfvar file that is outside of the module path, or directory that is set as current working directory
  2. set the command for terraform/plan similar to
- terraform/plan:
    var_file: ${HOME}/common.tfvars

Expected behavior

the terraform/plan command will run because the script passes the test to confirm that the file exists. And not fail with the following error

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
var file '${HOME}/common.tfvars' wasn't found

Exited with code exit status 1

Add Terraform parallelism option on apply

Is your feature request related to a problem? Please describe:

In some projects with specific Terraform version i need to be able to set terraform parallelism parameter.

Describe the solution you'd like:

Just need the parallelism parameter to be available in command/job definition.

# Simple use case example: 

- terraform/apply:
    attach-workspace: true
    backend_config_file: backend.hcl
    path: ./terraform
    plan: ./plan.out
    parallelism: 1

Would be happy to submit a PR

Command terraform/install requires sudo, which is missing

Orb Version

1.2.0

Describe the bug

When using the terraform/install command to install a specific version of Terraform, the command fails executing sudo mv terraform /usr/local/bin with the following error: /bin/sh: sudo: not found.

To Reproduce

  1. Use the circleci/[email protected] orb in your pipeline
  2. Setup a job running with executor terraform/default
  3. Add the terraform/install command to the job
  4. Schedule a pipeline run
  5. Observer your terraform job failing at the terraform/install step

Expected behavior

EITHER The terraform/default execution environment is configured to support the execution of the terraform/install command.

OR The orb documentation mentions the additional steps required to prepare the execution environment for the execution of the terraform/install command.

Additional context

version: 2.1

orbs:
  terraform: circleci/[email protected]

jobs:
  # Terraform
  deploy_infrastructure:
    executor: terraform/default
    working_directory: ~/my-repository/infrastructure
    steps:
      - attach_workspace:
          name: Attach workplace from previous job(s)
          at: ~/my-repository
      - terraform/install:
          terraform_version: 0.13.5

publishing failing pls suggest

Hello i get this error. pls help
Preparing to publish your orb.
No Orb Publishing Token detected.
Please set the ORB_PARAM_ORB_PUB_TOKEN environment variable.
Aborting deployment.
Exited with code exit status 1
how to assign token? is that public cloud credential

Terraform init is missing the -upgrade flag

Is your feature request related to a problem? Please describe.
Whenever providers in a terraform project change (e.g. version upgrades), we need to run 'terraform init -upgrade'. However the terraform init doesn't pass this -upgrade flag in by default.

Describe the solution you'd like
By default, we should be passing in the -upgrade flag to terraform init. This way, any provider changes naturally get upgraded as versions need to change.

Describe alternatives you've considered
An alternative is you could have a separate parameter for users to pass in any flags they want, bypassing the need for feature requests for additional parameters.

Additional context
Add any other context about the feature request here.

Cannot run my circleci workflow with latest (or older) orb versions

Orb Version
2.0.0
1.1.0

Describe the bug
Everything was working fine I believe 24 hours ago. Today is another day, and the job validate started failing at the init stage as it uses the module_path as an argument and not as the global option -chdir. Some people have noticed that too and two PRs could be fixing this issue I believe.
Because I can't control the TF version to be used with the executor used for the various orb jobs, I am just at the mercy of the terraform orb and terraform:light docker image combo. If the combination is a failing one, so are my workflows.

To Reproduce

  1. Create a workflow
  2. Add the orb job validate to that workflow
  3. run it

Expected behavior
I would expect it to work or give me a chance to set the tag of the docker image used as the executor so that when using an older orb version I can tell it to use the according TF version.

validate does not include `workspace`

Is your feature request related to a problem? Please describe.
when using terraform/validate, there's no way to pass terraform.workspace value.

Describe the solution you'd like
a parameter on the validate job that accepts the terraform workspace. We don't use the default workspace, so our modules aren't compatible with this validate task.

Pass backend_config to the init command on apply job

Orb Version
1.0.

Describe the bug
Trying to use the terraform/apply with a custom backend_config like so:

workflows:
  branch_push:
    jobs:
      - terraform/apply:
          checkout: true
          path: ./terraform/envs/generic
          backend_config: 'bucket=some-bucket,key=development/some-appi/terraform.tfstate'
          context: API_BUILD

But this throws:
Screenshot 2020-10-14 at 23 42 03

To Reproduce

  1. Add a terraform/apply job to a workflow with backend_config
  2. Start workflow
  3. Error throws

Expected behavior

I expect the terraform to apply presuming that the backend_config would be passed to the init command within the apply job.

Additional context

Would it work to pass the backend_config parameter through here.

Circle CI context environment variables not being interpolated in 3.0.0

Orb Version
3.0.0

Describe the bug

It appears that starting in 3.0.0 of the orb Circle CI context variables are no longer being interpolated in both backend-config and var parameters

To Reproduce

  1. Create a new Circle CI context with the following:

    • TF_RESOURCE_GROUP_NAME = <CUSTOM_VALUE>
    • TF_STORAGE_ACCOUNT_NAME = <CUSTOM_VALUE>
    • TF_CONTAINER_NAME = <CUSTOM_VALUE>
    • TF_KEY_NAME = <CUSTOM_VALUE>
  2. Reference your context variables in your job config:

     - terraform/init:
         backend: true
         backend_config:
           "resource_group_name=${TF_RESOURCE_GROUP_NAME}, \
           storage_account_name=${TF_STORAGE_ACCOUNT_NAME}, \
           container_name=${TF_CONTAINER_NAME}, \
           key=${TF_KEY_NAME}"
  1. Execute your CircleCI workflow/job

Expected behavior

Circle CI context values should be interpolated before being executed in scripts.

Actual behavior

In our case we are using the Azure Storage account as backend remote state and are getting an error: Failed to get existing workspaces: Error retrieving keys for Storage Account "${TF_STORAGE_ACCOUNT_NAME}"...

Where the environment variables are not getting interpolated before script is executed

Additional context

I believe this all stems from the refactor of when the scripts were moved out from being inline to referenced shell scripts. You are creating TF_ variables for your script execution but the input of those variables is not interpolated and the environment variable notations are being sent in verbatim.

var_file path is not absolute

Is your feature request related to a problem? Please describe.
When trying to use a dynamic value with a plan or apply job, it is very difficult to pass the dynamically created value as a var or a var_file.

Describe the solution you'd like
The var_file attribute should accept an absolute path instead of expecting a var file from within the module path.

Describe alternatives you've considered
I've tried passing a variable directly, with the var attribute, but it does not seem that the plan/apply reads from environment variables set in a pre-step step.

Additional context
Here's my current setup, please let me know if there's a better way to do this:

The pre-step command, grabbing a variable created in another job:

commands:
  set-image-tag-terraform-var:
    steps:
      - attach_workspace:
          at: /tmp/deploy/workspace
      - run: |
          cat /tmp/deploy/workspace/new-env-vars >> $BASH_ENV
          source $BASH_ENV
          echo "image_tag=\"$VERSION\"" >> /tmp/vars.tfvars

and the plan job:

- terraform/plan:
    pre-steps:
      - set-image-tag-terraform-var
    name: terraform-plan-staging
    path: terraform-cluster-service
    checkout: true
    context: aws
    persist-workspace: true
    workspace: staging
    var_file: ../../../tmp/vars.tfvars
    requires:
      - terraform-validate-staging
      - pipeline/build-deploy

I have tried unsuccessfully with the pre-step command, grabbing a variable created in another job and creating a new variable:

commands:
  set-image-tag-terraform-var:
    steps:
      - attach_workspace:
          at: /tmp/deploy/workspace
      - run: |
          cat /tmp/deploy/workspace/new-env-vars >> $BASH_ENV
          source $BASH_ENV
          echo $VERSION

and the plan job:

- terraform/plan:
    pre-steps:
      - set-image-tag-terraform-var
    name: terraform-plan-staging
    path: terraform-cluster-service
    checkout: true
    context: aws
    persist-workspace: true
    workspace: staging
    var: image_tag=${VERSION}
    requires:
      - terraform-validate-staging
      - pipeline/build-deploy

but it seems $BASH_ENV is not loaded as $VERSION is empty.

Typo in Flag names `-var` and `-var-file`

Orb Version
1.0.1

Describe the bug
There is a typo in the flags -vars and -vars-file. They should be -var and -var-file. This causes the steps terraform/plan, terraform/destroy, and terraform/apply to fail when the var_file or vars arguments are supplied. (One exception: It looks like terraform/plan already has the correct spelling of var-file.)

Sample output (it correctly initializes, and then fails the plan step):

Initializing the backend...

Initializing provider plugins...
- Using previously-installed hashicorp/aws v2.64.0

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Usage: terraform plan [options] [DIR]

  Generates an execution plan for Terraform.

  This execution plan can be reviewed prior to running apply to get a
  sense for what Terraform will do. Optionally, the plan can be saved to
  a Terraform plan file, and apply can take this plan file to execute
  this plan exactly.

Options:

  -compact-warnings   If Terraform produces any warnings that are not
                      accompanied by errors, show them in a more compact form
                      that includes only the summary messages.

  -destroy            If set, a plan will be generated to destroy all resources
                      managed by the given configuration and state.

  -detailed-exitcode  Return detailed exit codes when the command exits. This
                      will change the meaning of exit codes to:
                      0 - Succeeded, diff is empty (no changes)
                      1 - Errored
                      2 - Succeeded, there is a diff

  -input=true         Ask for input for variables if not directly set.

  -lock=true          Lock the state file when locking is supported.

  -lock-timeout=0s    Duration to retry a state lock.

  -no-color           If specified, output won't contain any color.

  -out=path           Write a plan file to the given path. This can be used as
                      input to the "apply" command.

  -parallelism=n      Limit the number of concurrent operations. Defaults to 10.

  -refresh=true       Update state prior to checking for differences.

  -state=statefile    Path to a Terraform state file to use to look
                      up Terraform-managed resources. By default it will
                      use the state "terraform.tfstate" if it exists.

  -target=resource    Resource to target. Operation will be limited to this
                      resource and its dependencies. This flag can be used
                      multiple times.

  -var 'foo=bar'      Set a variable in the Terraform configuration. This
                      flag can be set multiple times.

  -var-file=foo       Set variables in the Terraform configuration from
                      a file. If "terraform.tfvars" or any ".auto.tfvars"
                      files are present, they will be automatically loaded.

Exited with code exit status 1

To Reproduce
Add a terraform/plan terraform/destroy or terraform/apply step with the vars or var_file arguments.
ie.

- terraform/plan:
          path: .
          var_file: $VAR_FILE
          vars: version=$VERSION_NUMBER

Expected behavior
Run the command and not crash.

Additional context
Lines that need to be changed:
https://github.com/CircleCI-Public/terraform-orb/blob/master/src/commands/apply.yml#L68
https://github.com/CircleCI-Public/terraform-orb/blob/master/src/commands/apply.yml#L76
https://github.com/CircleCI-Public/terraform-orb/blob/master/src/commands/destroy.yml#L69
https://github.com/CircleCI-Public/terraform-orb/blob/master/src/commands/destroy.yml#L77
https://github.com/CircleCI-Public/terraform-orb/blob/master/src/commands/plan.yml#L70

I don't have permissions to create the PR myself, despite the README saying it welcomes PRs 😓

Thanks! :)

terraform init doesn't work with 1.0.3

Error message: Too many command line arguments. Did you mean to use -chdir?
Usage:

- terraform/install:
    terraform_version: '1.0.3'
- terraform/init:
    path: environment/<<parameters.environment>>/stateful

Screenshot 2021-07-29 at 13 15 13

Can't use List(String) varible type in parameter var

Orb Version
3.0.1

Describe the bug
Using a List(String) type is not permit with the parameter var since it use a , to separate value.
i.e. foo=[bar,foo] is separated as -var foo=[bar -var foo]
Giving the error: -var option foo] is not correctly specified.

To Reproduce

- terraform/apply:
    var: 'foo=[bar,foo]'

Expected behavior

foo=[bar,foo] is separated as -var foo=[bar,foo]

Additional context

In the code a tr is use to replace , by a \n

for var in $(echo "${TF_PARAM_VAR}" | tr ',' '\n'); do
    PLAN_ARGS="$PLAN_ARGS -var $(eval echo "$var")"
done

terraform/validate doesn't support import blocks, but terraform v1.7.5 on my machine does

Orb Version
3.2.1

Describe the bug
Import blocks available from Terraform v1.5.0 and later are valid when terraform validate is ran locally, but the latest orb doesn't recognise it.

To Reproduce

  1. create a terraform configuration with a simple resource and an import {} block that imports that resource that already exists remotely
  2. run terraform validate locally on your configuration
  3. make CircleCI run the terraform/validate job from the circleci/[email protected] orb on your configuration

Expected behavior

Locally:

$ terraform validate
Success! The configuration is valid.

In CircleCI job output:

Success! The configuration is valid.

Actual behavior

Locally:

$ terraform validate
Success! The configuration is valid.

In CircleCI job output:

Error: Unsupported block type
│ 
│   on main.tf line 112:
│  112: import {
│ 
│ Blocks of type "import" are not expected here.
image

Additional context

$ terraform -v
Terraform v1.7.5
on darwin_arm64
+ provider registry.terraform.io/cyrilgdn/postgresql v1.19.0
+ provider registry.terraform.io/hashicorp/google v5.21.0
+ provider registry.terraform.io/hashicorp/google-beta v5.21.0
+ provider registry.terraform.io/hashicorp/kubernetes v2.27.0
+ provider registry.terraform.io/hashicorp/random v3.6.0

Example configuration:

resource "google_project_service" "container" {
  project = "my-project-in-google"
  service = "container.googleapis.com"
}
import {
  id = "my-project-in-google/container.googleapis.com"
  to = google_project_service.container
}

init does not have a way to set the `workspace`

Is your feature request related to a problem? Please describe.
Sometimes it can be interesting to set the workspace for init, e.g. if some resources in the state need a provider which is not in the module anymore, and won't be downloaded without the correct state set (thats a behavior we don't see often, but we had to correct that in our pipelines).

Describe the solution you'd like
Something like #78

Describe alternatives you've considered
Setting the TF_WORKSPACE variable, but its scope is the full job and it does not create automatically the workspace like other commands of this orbs do.

Additional context
I can open the PR inspired on the the PR 78 if that's ok with the maintainers.

Add the ability to specify the lock timeout on terraform plan and apply

Is your feature request related to a problem? Please describe.
I have my terraform state files on Google. I have run into situations where some one is running a terraform apply on their desk and CircleCi is running a build. This causes a build failure because CircleCi can not get the terraform remote lock. The default amount of wait time to get the lock is 0s.

Describe the solution you'd like
I would like to be able to add a -lock-timeout=sometimeout or pass a lock timeout to the parameter to terraform commands/jobs plan and apply. This way I would be able to have terraform wait for a specified amount of time to get the lock. This would avoid build failures in most cases.

Describe alternatives you've considered
I have looked through the code and since this has to be a command line parameter I see no way to pass it to the orb to add the parameter to terraform command

Additional context
With some guidance I will be willing to make a pull request for this.

terraform/plan does not work with Terraform Cloud remote backend

Orb Version
circleci/[email protected]

Describe the bug
When using the terraform orb with a Terraform Cloud based remote backend, terraform/plan fails with this error:

Error: Saving a generated plan is currently not supported

The "remote" backend does not support saving the generated execution plan locally at this time.

This happens because the orb calls terraform plan with -out=plan.out which is not supported for Terraform Cloud

To Reproduce
Use a the terraform/plan command in a CircleCI job with a Terraform Cloud based backend.

Example config.yml:

version: 2.1

orbs:
  terraform: circleci/[email protected]

commands:
  terraform_setup:
    description: Configures terraform cloud credentials
    steps:
      - run:
          name: Create Terraform credentials file
          command: >-
            echo "credentials \"app.terraform.io\" {token =
            \"$TERRAFORM_TOKEN\"}" > $HOME/.terraformrc

jobs:
  terraform_plan:
    executor:
      name: terraform/default
      tag: 1.0.5
    steps:
      - checkout
      - terraform_setup
      - terraform/init:
          path: terraform
      - terraform/plan:
          path: terraform

workflows:
  Terraform_Plan:
    jobs:
      - terraform_plan

Expected behavior
Terraform plan should run

Additional context
The terraform orb should detect that your remote backend doesn't support saving local plans and only attempt to do so when supported. Instead the output can link to the plan in Terraform Cloud. At the very least, provide a parameter option to disable file output.

Orb parameters don't work with CircleCI defined env vars

Orb Version
3.2.0

Describe the bug
When using the orb, I can't find a way to use an env variable defined in the CircleCI contexts inside an orb command. In my example I have the terraform modules in different paths based on environment, something like env/qa and env/prod and I define the qa and prod part in an ENV env variable in the contexts, e.g. env/${ENV}.

However I configure this path I get the error

Path does not exist: ./env/${ENV}/

I tried using an absolute path as well, no luck.

To Reproduce

Use an environment variable in the path parameter in orb commands.

Expected behavior

I expect the orb to see and use the environment variables I set in CircleCI contexts or project settings.

BASH_ENV variables not passed through to orb

Orb Version
3.1

Describe the bug

In our circleCI build, I am attempting to push an env variable with an image tag into BASH_ENV, but the value gets lost in between the steps of the job. For example, I'm trying to do something like this: https://circleci.com/docs/env-vars#example-configuration-of-environment-variables. However, the only env variables available within the orb's steps are variables set by circleci itself.

I'm not really positive this is a bug report, but it didn't feel like a feature request either. I believe I'm likely misunderstanding something about the config that needs to change, but I'm hoping if nothing else I might get a bit of a nudge in the right direction :)

An example job config:

version: '2.1'
orbs:
    terraform: circleci/[email protected]
jobs:
  deploy-infrastructure:
    docker:
      - image: hashicorp/terraform:1.1.2
    steps:
      - checkout
      - run:
          name: Set terraform tag
          command: |
            echo "export TERRAFORM_TAG=${CIRCLE_SHA1}" >> $BASH_ENV
            source $BASH_ENV
            echo "Deploying image tag '${TERRAFORM_TAG}' via terraform."
      - terraform/fmt:
          path: ./deployment/terraform
      - terraform/validate:
          path: ./deployment/terraform
      - terraform/init:
          path: ./deployment/terraform
      - terraform/plan:
          path: ./deployment/terraform
          var: "image_tag=${TERRAFORM_TAG}"
      - terraform/apply:
          path: ./deployment/terraform
          var: "image_tag=${TERRAFORM_TAG}"

in the above example, the value of TERRAFORM_TAG is lost after the run step, and ends up empty.

As a workaround, I have configured circleci to manually run the commands like this:

version: '2.1'
orbs:
    terraform: circleci/[email protected]
jobs:
  deploy-infrastructure:
    docker:
      - image: hashicorp/terraform:1.1.2
    steps:
      - checkout
      - run:
          name: Set terraform tag
          command: |
            echo "export TERRAFORM_TAG=${CIRCLE_SHA1}" >> $BASH_ENV
            source $BASH_ENV
      - terraform/fmt:
          path: ./deployment/terraform
      - terraform/validate:
          path: ./deployment/terraform
      - run:
          name: Deploy via terraform
          command: |
            source $BASH_ENV
            echo "Deploying image tag '${TERRAFORM_TAG}' via terraform."
            terraform -chdir=./deployment/terraform init -input=false -backend=true
            terraform -chdir=./deployment/terraform plan -input=false -out=plan.out -var="image_tag=${TERRAFORM_TAG}"
            terraform -chdir=./deployment/terraform apply -input=false -auto-approve -var="image_tag=${TERRAFORM_TAG}"

To Reproduce

  1. Set up a configuration like the above example, where an environment variable is saved to BASH_ENV
  2. Attempt to access that value inside a followup step within the same job
  3. Confirm that the BASH_ENV variable is not available as expected

Expected behavior

That BASH_ENV would be sourced into the terraform orb and variables would also be accessible within the orb's command contexts as well.

Additional context

Verify download checksum

Is your feature request related to a problem? Please describe.
Downloads should be verified against at least their checksums (currently sha256) before execution.

Describe the solution you'd like
Compare the downloaded file to the checksum. If verification fails, retry some number of times, then fail the job.

Describe alternatives you've considered
chtf, which validates the checksums:

https://github.com/Yleisradio/chtf/blob/c34f6265b47dda812cc1936871ed79d250933ed1/chtf/__chtf_terraform-install.sh#L202-L212

tfenv, which validates checksum and GPG signature:

https://github.com/tfutils/tfenv/blob/6d16fa586a991b94b1dcb6dd7e4597b24b8b2a1e/libexec/tfenv-install#L148-L160

terraform-exec : tfinstall, which also doesn't validate the download.

Additional context
Currently, we're using a Docker image for Debian Buster and installing terraform. Addressing this issue would be a step toward allowing us to migrate to this Orb, allowing for a simpler CircleCI customer config.

Terraform init with S3 backend is failing with terraform version 0.15+ and orb 3.0.0

Orb Version
The version number of the orb
3.0.0

Describe the bug

When terraform/init job is ran with backend_config, we get an error: Error refreshing state: InvalidBucketName: The specified bucket is not valid.
To Reproduce
Run a terraform init using aws s3 as backend_config

Expected behavior
Terraform to init properly with an s3 backend

Additional context
If we use terraform version 0.14.x with orb 2.x.x we do not get this error, only with version 0.15.x + and orb 3.0.0

Ability to override executor (or image)

Is your feature request related to a problem? Please describe.
The provided executor always uses the latest hashicorp/terraform:light image, which can force unexpected upgrades and non-deterministic results between runs. For example CI/CD may work one week on Terraform 0.13 but the job would no longer run the following week due to syntax changes in Terraform 0.14. To override this currently you need to define your own jobs to wrap the Orb's commands which produces a lot of boilerplate code.

Describe the solution you'd like
The ability to pass your own executor to the jobs.

Describe alternatives you've considered
The terraform/default executor does have an image parameter that could be made accessible from the jobs, like the Node Orb does. I don't believe you can use a "null" default for a parameter though, so it results in duplicating the default versions across many places in the Orb, which would be harder to maintain. I'd be inclined to remove the executor parameter altogether to make it clearer that it's not usable by the Orb.

Additional context
I'll be happy to submit a PR if you can advice which solution you'd prefer.

circleci local execute --job terraform/{fmt,validate,plan,etc} errors with --storage-opt is supported only for overlay over xfs with 'pquota' mount option

Orb Version
circleci/[email protected]

Describe the bug
Attempting to use the Terraform orb with the circleci-cli utility and trigger a job with circleci local execute --job terraform/fmt results in a failed run and the error "Error: error starting container hashicorp/terraform:0.14.11: Error response from daemon: --storage-opt is supported only for overlay over xfs with 'pquota' mount option"

To Reproduce

  1. Start with a generic, valid, Terraform repo, the circleci-cli utility 0.1.15410+ba6fe81 (homebrew), and Docker Desktop for Mac v3.5.2
  2. Have a basic terraform-orb config like https://gist.github.com/law/507574557a3fe5af31a5d74aa486c11b
  3. Try to execute any of the Orb sub-commands like circleci local execute --job terraform/fmt
  4. Observe the error

Expected behavior
terraform fmt executes against the Terraform repo

terraform/destroy

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
I was not able to use terraform/destroy because it does not automatically initialize the backend (like the other commands do, example terrafom/apply)

Describe the solution you'd like
A clear and concise description of what you want to happen.
I would like to call in the config.yaml terraform/destroy and do not receive the backend initialization error

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
I can destroy from my console instead of within circleci

Additional context
Add any other context about the feature request here.
Perhaps I am not running the command correctly but I was not able to find documentation. Please help! Thanks

Option to automatically use the latest terraform_version

Is your feature request related to a problem? Please describe.
In the current implementation when a new TF version came out you have to update your CircleCI pipeline to accommodate these changes.

Describe the solution you'd like
New boolean variable in the install.yml. Name like "terraform_latest" default value: false.
If the value set to true then the << parameters.terraform_version >> is overwritten

if [[ << parameters.terraform_latest >> == "dev" ]] then https://checkpoint-api.hashicorp.com/v1/check/terraform wget https://releases.hashicorp.com/terraform/<< parameters.terraform_version >>/terraform_<< parameters.terraform_version >>_linux_amd64.zip else wget https://releases.hashicorp.com/terraform//{{ version }}/terraform_{{ version }}_linux_amd64.zip unzip terraform_{{ version }}_linux_amd64.zip fi

Additional context
I'll be happy to submit a PR if you like these changes

(1) Can't install, (2) Can't validate

Orb Version
circleci/[email protected]

I have the following code in my circle ci file

terraformation:
  executor: terraform/default
  steps:
    - attach_workspace:
        at: ~/project
    - aws-cli/setup:
        aws_access_key_id: AWS_ACCESS_KEY_ID
        aws_secret_access_key: AWS_SECRET_ACCESS_KEY
        region: ap-southeast-1
    - terraform/install:
        terraform_version: 1.6.3
    - run:
        name: TFVars
        command: |
          rm -rf terraform.tfvars
          touch terraform.tfvars
          //.. bunch of echoes that to terraform.tfvars like echo 'something = "something"' >> terraform.tfvars
          cat terraform.tfvars
    - terraform/init
    - terraform/validate
    - terraform/apply

I'm getting 2 errors here,

Here's the first one:
image

The 2nd one is when I remove the terraform/install step, I will get the following
Screen Shot 2023-11-11 at 6 27 07 PM

I have tested these variables locally and when I run terraform validate, I don't get any errors.

The variables.tf file:

variable "project_name" {
  type = string
}

variable "stage" {
  type = string

  validation {
    condition     = contains(["development", "staging", "production"], var.stage)
    error_message = "Allowed values for stage are \"development\", \"staging\", and \"production\""
  }
}

variable "aws_region" {
  type = string
}

variable "hosted_zone_id" {
  type = string
}

variable "hosted_zone_name" {
  type = string
}

variable "images_server_subdomain" {
  type    = string
  default = ""
}

variable "cache_policy_id" {
  type = string
}

variable "origin_request_policy_id" {
  type = string
}

To Reproduce

Expected behavior

Additional context

`terraform/install` with `latest` fails

Orb Version
3.1.0

Describe the bug
When using the terraform/install step with terraform_version: latest, shasum checking fails because it's using the wrong variable for the name of the file.

To Reproduce
Run a job with a step like this:

      - terraform/install:
          terraform_version: latest

Expected behavior
terraform/install should work with latest

Additional context
Logs from usage:

Using Terraform version '1.2.9'
2022-09-17 01:06:11 URL:https://releases.hashicorp.com/terraform/1.2.9/terraform_1.2.9_linux_amd64.zip [19906116/19906116] -> "terraform_1.2.9_linux_amd64.zip" [1]
2022-09-17 01:06:11 URL:https://releases.hashicorp.com/terraform/1.2.9/terraform_1.2.9_SHA256SUMS [1378/1378] -> "terraform_1.2.9_SHA256SUMS" [1]
cat: terraform_latest_SHA256SUMS: No such file or directory

terraform fmt is working only the given directory (or current directory). It should also process subdirectories. (-recursive)

Orb Version
[email protected]

Using [email protected] and looks like terraform fmt is working only the given directory (or current directory). It should also process subdirectories. (-recursive)
Reference: https://www.terraform.io/docs/commands/fmt.html

Current terraform fmt is working only for the given directory. It's not formatting the subdirectories.
teraform fmt -recursive (This command formats subdirectories as well)
Could please check and fix this.

Terraform documentation:
https://www.terraform.io/docs/commands/fmt.html

Thanks,
Vinusha

Support `terraform test`

Is your feature request related to a problem? Please describe.
I have written unit tests for a Terraform module (see https://developer.hashicorp.com/terraform/language/tests) and want to run them in CircleCI.

Describe the solution you'd like
A terraform/test command and terraform/test job.

Describe alternatives you've considered
I am currently using my own job definition to achieve this.

jobs:
  terraform-test:
    description: 'Execute Terraform tests.'
    executor:
      name: 'terraform/default'
      resource_class: '<< parameters.resource_class >>'
      tag: '<< parameters.tag >>'
    parameters:
      checkout:
        description: 'Perform checkout as first step in job.'
        type: 'boolean'
        default: false
      path:
        description: 'Path to the Terraform module.'
        type: 'string'
        default: '.'
      resource_class:
        description: 'Specify the resource class for Docker executor.'
        type: 'string'
        default: 'medium'
      tag:
        description: 'Specify the Terraform Docker image tag for the executor.'
        type: 'string'
        default: '1.0.0'
      test_directory:
        description: 'Set the Terraform test directory.'
        type: 'string'
        default: 'tests'
      timeout:
        description: 'Configure a custom timeout limit'
        type: 'string'
        default: '10m'
    steps:
      - when:
          condition: '<< parameters.checkout >>'
          steps:
            - 'checkout'
      - run:
          command: 'terraform -chdir=<< parameters.path >> init -input=false -test-directory=<< parameters.test_directory >>'
          name: 'terraform init'
      - run:
          command: 'terraform -chdir=<< parameters.path >> test -test-directory=<< parameters.test_directory >>'
          name: 'terraform test'
          no_output_timeout: '<< parameters.timeout >>'

Ability to override execution plan name

Is your feature request related to a problem? Please describe.
In the current implementation the execution plan name is fixed to plan.out. If you CircleCI workflow looks like this [Plan A, Plan B, Plan C] -> HOLD -> [Apply A, Apply B, Apply C] then the parallelly running terraform plan job will be overwrite each others execution plans because they are using the same name.

Describe the solution you'd like
It would be great if this could be moved to a variable and the default value of plan.out. It requires small changes in then plan.yml. However in the apply.yml if you specify non default execution plan name them you have add the name of the plan to execute.

Additional context
I'll be happy to submit a PR if you can advice how would you like to implement this in the apply.yml

How to pass the Terraform version I need?

Is your feature request related to a problem? Please describe.
I want to use a specific version of Terraform, looks like the executor allows an image parameter to set the Terraform Docker image version but I am not able to change this.

Describe the solution you'd like
I would like to define it in my config.yml so I can upgrade the Terraform version in a controlled way.

Describe alternatives you've considered
N/A

Additional context
N/A

git lfs failed when using the terraform orb due to no image or something?

3.1.0

Many repos use "git lfs" for large files. We also use AWS tools for other things in our deploy job which would like to do terraform(We are trialing it with a wire into CI of the next 4 weeks but that didn't work out well so we can't start using it). The orb has an executor and not an image which is weird. Is there another way(a terraform image with AWS tools) we can use.

git lfs install failed so not all files are there in the checkout(and this failed the build as well) ->

#!/bin/sh -eo pipefail
sudo apt-get update && sudo apt-get install git-lfs; git lfs install
/bin/sh: sudo: not found
git: 'lfs' is not a git command. See 'git --help'.

The most similar command is
log

Exited with code exit status 1
CircleCI received exit code 1

Ability to persist plan stdout to file

Is your feature request related to a problem? Please describe.
We're running the pipelines in CircleCI and would like to get the plan output as test we can post as a GitHub comment for reviewers to check

Describe the solution you'd like
When running the plan job, the output (stdout) will be saved to file that then can be persisted and used in another job to be pushed to GitHub as a comment.

Optionally adding that functionality to other jobs like verify can be useful to catch errors

Describe alternatives you've considered
At the moment we're using in-house orb that does exactly that but we prefer to reduce the amount of orbs we maintain to a minimum and have this functionality available in this orb

Not the greatest implementation but working example

(terraform plan -no-color -compact-warnings -out=plan.out \
     -lock=<< parameters.lock-state >>  << parameters.terraform-extra-args >> | tee plan-stdout.txt) 3>&1 1>&2 2>&3 | tee error.txt

Additional context
Add any other context about the feature request here.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.