Git Product home page Git Product logo

cli's Introduction

Overmind

Overmind CLI

Discord Server

🎥 Watch a demo | 📖 Docs | 🚀 Sign up | 💻 Playground | 🙌 Follow us

What is Overmind?

Overmind is a powerful tool for real-time impact analysis on Terraform changes. Overmind can identify the blast radius and uncover potential risks with overmind terrafrom plan before they harm your infrastructure, allowing anyone to make changes with confidence. We also track the impacts of the changes you make with overmind teraform apply, so that you can be sure that your changes haven't had any unexpected downstream impact.

Quick Start

Install the Overmind CLI using brew:

brew install overmindtech/overmind/overmind-cli

Run a terraform plan:

overmind terraform plan

Running 'overmind terraform plan' and viewing in the app

Install on other platforms

Prerequisites

  • Terraform environment set up
  • Access to all required credentials
  • Ability to install and run the Overmind CLI

Installation

MacOS

To install on Mac with homebrew use:

brew install overmindtech/overmind/overmind-cli

Windows

Install using winget:

winget install Overmind.OvermindCLI

Or manually download the latest release, extract overmind.exe, and add to your PATH

Ubuntu / Debian

Set up the repository automatically:

curl -1sLf \
  'https://dl.cloudsmith.io/public/overmind/tools/setup.deb.sh' \
  | sudo -E bash

Or set it up manually

# NOTE: For Debian Stretch, Ubuntu 16.04 and later
keyring_location=/usr/share/keyrings/overmind-tools-archive-keyring.gpg
# NOTE: For Debian Jessie, Ubuntu 15.10 and earlier
keyring_location=/etc/apt/trusted.gpg.d/overmind-tools.gpg

# Capture the codename
codename=$(lsb_release -cs)

apt-get install -y debian-keyring  # debian only
apt-get install -y debian-archive-keyring  # debian only

apt-get install -y apt-transport-https
curl -1sLf 'https://dl.cloudsmith.io/public/overmind/tools/gpg.BC5CDEFB4E37A1B3.key' |  gpg --dearmor >> ${keyring_location}
curl -1sLf 'https://dl.cloudsmith.io/public/overmind/tools/config.deb.txt?distro=ubuntu&$codename=xenial&component=main' > /etc/apt/sources.list.d/overmind-tools.list
chmod 0644 /etc/apt/sources.list.d/overmind-tools.list
chmod 0644 /usr/share/keyrings/overmind-tools-archive-keyring.gpg
apt-get update

Then install the CLI:

apt-get install overmind-cli

RHEL

Set up the repository automatically:

curl -1sLf \
  'https://dl.cloudsmith.io/public/overmind/tools/setup.rpm.sh' \
  | sudo -E bash

Or set it up manually

yum install yum-utils pygpgme
rpm --import 'https://dl.cloudsmith.io/public/overmind/tools/gpg.BC5CDEFB4E37A1B3.key'
curl -1sLf 'https://dl.cloudsmith.io/public/overmind/tools/config.rpm.txt?distro=amzn&codename=2023' > /tmp/overmind-tools.repo
yum-config-manager --add-repo '/tmp/overmind-tools.repo'
yum -q makecache -y --disablerepo='*' --enablerepo='overmind-tools'

Then install the CLI:

sudo yum install overmind-cli

Alpine

Set up the repository automatically:

sudo apk add --no-cache bash
curl -1sLf \
  'https://dl.cloudsmith.io/public/overmind/tools/setup.alpine.sh' \
  | sudo -E bash

Or set it up manually

curl -1sLf 'https://dl.cloudsmith.io/public/overmind/tools/rsa.7B6E65C2058FDB78.key' > /etc/apk/keys/[email protected]
curl -1sLf 'https://dl.cloudsmith.io/public/overmind/tools/config.alpine.txt?distro=alpine&codename=v3.8' >> /etc/apk/repositories
apk update

Then install the CLI:

apk add overmind-cli

Arch

Packages for Arch are available on the releases page for manual download and installation.

Additionally a community maintained package can be found in the aur.

Why Use Overmind?

  • ☁️ Cloud Complexity: Terraform tells you what it's going to change, but not whether this change will break everything. Teams need to understand dependencies to properly understand impact.
  • 👨‍🏫 Onboarding & Productivity: Due to the reliance on "tribal knowledge", expert staff are stuck doing approvals rather than productive work and newer staff take longer to become productive.
  • 📋 Change Management Process: IaC and automation mean that changes spend substantially more time in review and approval steps than the change itself actually takes.
  • 🔥 Downtime: Outages are not caused by simple cause-and-effect relationships. More often than not, downtime is a result of dependencies people didn't know existed.

How We Solve It?


🔍 Blast Radius: Overmind maps out all potential dependencies and interactions within your infra in realtime. Supports over 120 AWS resources and all Kubernetes.
🚨 Risks: Discover specific risks that would be invisible otherwise. Risks are delivered directly to the pull request. Make deployment decisions within minutes not hours.

Advanced Use

Passing Arguments

Overmind's overmind terraform plan and overmind terraform apply commands mostly just wrap the terraform that you already have installed, adding all of Overmind's features on top. This means that no matter how you're using Terraform today, this will still work with Overmind. For example if you're using a more complex command like:

terraform plan -var-file=production.tfvars -parallelism=20 -auto-approve

Then you would add overmind to the beginning, and your arguments after a double-dash e.g.

overmind terraform plan -- -var-file=production.tfvars -parallelism=20 -auto-approve

Join the Community

Additional Resources

Reporting Bugs

License

See the LICENSE file for licensing information.

Overmind is made with ❤️ in 🇺🇸🇬🇧🇦🇹🇫🇷🇷🇴

cli's People

Contributors

davids-ovm avatar dylanratcliffe avatar renovate[bot] avatar getinnocuous avatar jameslaneovermind avatar dependabot[bot] avatar tphoney avatar lerentis avatar

Stargazers

Antonio Dourado avatar Liam Baker avatar Christian Bagley avatar Javier Juarez avatar Gonçalo Amaral avatar hirak0 avatar avatar Can Evgin avatar Brice avatar Lens0021 / Leslie avatar Danny Jang avatar Yuto Takano avatar ywsrock avatar ik5 avatar Aurélien Perrier avatar Thomas Labarussias avatar Cory Plastek avatar loisg avatar DavidPham avatar myoung-su,shin avatar avatar Melwin Kieffer avatar Thomas Martin avatar Garry Smith avatar Anthony Camilo avatar Leslie-Alexandre DENIS avatar Rebecca / Riobhca avatar Lukas M avatar avatar Robert Joost avatar Jochen Seeber avatar David Schmitt avatar Wownepiece avatar imcom avatar Loïc Roux avatar Aniket Patwardhan avatar Erik avatar Joseph D. avatar Tobias avatar avatar Maxence Depire avatar Greg Batiz avatar Chad Mayfield avatar Liam avatar avatar Anthony Putignano avatar avatar Christopher Bain avatar Omari avatar Florent Lerat avatar Benjamin Marte avatar David Danielsson avatar Rafael Simples avatar Ajmal avatar Luis Meza avatar avatar Bex avatar Denis Belyatsky avatar Tertius Stander avatar Hieu Tran avatar Jan-Arve Nygård avatar David CM avatar Tsiry Sandratraina avatar avatar mrexojo ☁️ avatar Carlos Guzmán Velasco avatar Noah Halstead avatar Nathaniel Varona avatar James Corteciano avatar Ryan Raub avatar Sébastien Bonami avatar Zachary Loeber avatar Andy Bold avatar avatar avatar Prabhu Jayakumar avatar Pavlos-Petros Tournaris avatar Florian Dhaussy avatar avatar avatar George Pchelkin avatar Kirill Morozov avatar Volodymyr Rozdolsky avatar Bruno Ferreira avatar swann croiset avatar Viacheslav Vasilyev avatar avatar Edson C. avatar Frank avatar Dmitri Korobtsov avatar Kobzar Slava avatar Anton Patsev avatar Karim Shakirov avatar Igor Kurochkin avatar Jérôme Gamez avatar Oleksandr B. avatar кірілл avatar avatar avatar Leonardo Poggiani avatar

Watchers

Hieu Tran avatar avatar avatar

Forkers

antverpp lerentis tphoney

cli's Issues

Support getting the scope from the attributes of the matched resources

Currently the scope for a query is taken literally. However we've known for a while this wasn't going to fly. We will need to be able to pull the scope at least from the attributes of the matched resources, and maybe from other places too (like provider config).

We need to remember that we will always need to support the following static scopes:

  • global
  • *

I'm thinking of maybe creating a special namespace like values (this matches what is in the plan). So if we wanted the scope to be the metadata.namespace attribute we'd use values.metadata.namespace

For kubernetes though, the scope of the resources includes the "name" of the cluster, which if course doesn't really exist. I think the solution here is to also allow the cli to map from the outputs of the plan to the scopes too. In this case we would have a well-known output such as overmind_kubernetes_cluster_name, which would be interpolated in the scope, along with the namespace. We would also need to allow the user to name the cluster arbitrarily, which would be achieved through a parameter in helm and therefore used by the source.

Then when evaluating the plan, if all of the required variables for the scope are present, then we set it, and if not then we just fall back to *. This means that people can get value immediately, but also improve their results with a little work

Interpolation format

I'm thinking we should just use the same as terraform here. e.g.

${outputs.overmind_kubernetes_cluster_name}.${values.metadata.namespace}

Move all spinners to be fixed width

The current use of emojis means that things move around as the emoji aren't fixed width when we go from a loading state to a completed state. We should instead changed to spinners that are fixed width so things stay consistent

Improve `get-change` markdown output to show planned changes' attribute differences

In order to avoid double commenting and demonstrating the new abilities around planned changes, change get-change --format markdown to also add a terraform plan-like diff output showing the planned changes.

  • Use the ItemDiffs in ChangeProperties.plannedChanges to render the before/after diff of attributes for all planned changes.
  • Add risks from the change
  • Redo the example in overmindtech/terraform-example#5 with the new output.

Note that the comment's formatting should follow the Change page's design to help easy switching between the web UI and CLI.

This depends on #109 and #111

Update the comment to show risks & changes

Now that change have a new shape, we can improve the comment that we post.

Add Terraform plan information

Since we are capturing this, we should post this as the expected changes, this should remove the need for the Terraform plan in a separate comment.

Add risks

Now that we have risks, we need to be able to format these in the comment too

Submit-plan gives 404 but still works

In this plan run I get a 404 error when running the submit-plan action, however the plan has been created somewhat successfully: https://app.overmind.tech/changes/4f60c09d-8493-42c0-85d6-b07dbed65439

++ ./overmindtech/ovm-cli submit-plan --title 'Increase node group size (EXAMPLE ONLY)' --description 'This change was automatically created from a PR' --ticket-link https://github.com/overmindtech/deploy/pull/350 --plan-json ./tfplan.json
time="2023-07-17T11:03:49Z" level=error msg="Error reading config file" err="Config File \"config\" Not Found in \"[]\""
time="2023-07-17T11:03:49Z" level=error msg="couldn't parse `log` config, defaulting to `info`" err="not a valid logrus Level: \"\"" fields.level=
time="2023-07-17T11:03:49Z" level=info msg="set log level from config" fields.level=info
time="2023-07-17T11:03:49Z" level=info msg="otlptracehttp client configured itself: &{traces {localhost:4318 false <nil> map[] 0 10000000000 /v1/traces <nil>} {{localhost:4318 false <nil> map[] 0 10000000000 /v1/traces <nil>} {true 5000000000 30000000000 60000000000} 0  [] <nil>} 0xcab500 0xc00060eff0 0xc0000462a0 {0 {0 0}}}"
time="2023-07-17T11:03:49Z" level=info msg="resolving items from terraform plan" gateway_url="https://api.prod.overmind.tech/api/gateway"
time="2023-07-17T11:03:49Z" level=warning msg="skipping unmapped resource" gateway_url="https://api.prod.overmind.tech/api/gateway" terraform-address=auth0_client.api_keys
time="2023-07-17T11:03:50Z" level=info msg="created a new change" change=4f60c09d-8493-42c0-85d6-b07dbed65439 gateway_url="https://api.prod.overmind.tech/api/gateway"
time="2023-07-17T11:03:50Z" level=info msg="identifying items" change=4f60c09d-8493-42c0-85d6-b07dbed65439 gateway_url="https://api.prod.overmind.tech/api/gateway" item_count=1
time="2023-07-17T11:03:50Z" level=error msg="Failed to connect to overmind API" change=4f60c09d-8493-42c0-85d6-b07dbed65439 error="failed to WebSocket dial: expected handshake response status code 101 but got 404" gateway_url="https://api.prod.overmind.tech/api/gateway"
time="2023-07-17T11:03:50Z" level=error msg="Error flushing tracer provider" error="traces export: Post \"https://localhost:4318/v1/traces\": dial tcp [::1]:4318: connect: connection refused"
+ echo change-url=

Rename project and binary

Background

We want the new CLI to be as easy to use as possible. To this end we want user to basically be able to run overmind terraform plan in place of where they would previously run terraform plan. In order to do this though we need to reorganise the CLI

Tasks

  • Rename this project from ovm-cli to cli
  • Rename the binary that is produced to overmind (pretty sure this will happen automatically)
  • See if we can separate the existing commands in the help text. We want the "happy path" of using the overmind terraform command to be super obvious. We might be able to separate this somehow, or maybe we will need to nest them under a subcommand. (Cobra claims "Grouping help for subcommands" as a feature)
  • Update the Github Actions to use the new binary and different commands if we changed them (Done in the rename-cli branch)
  • Merge the rename-cli branch in the Github action
  • Find all the places this repo is used in the website and replace (though github will give us a redirect for some time)
  • Now that we are using a hierarchy, review all of the arguments and move them to where they are actually required. Ideally the terraform command wouldn't accept any arguments that are Overmind-specific for example
  • Update the Readme with an interim description

Subcommand vs Group

Is is possible to group commands which gives you output like this:

$ overmind help
Calculate the blast radius of your changes, track risks, and make changes with confidence

Usage:
  overmind [command]

Infrastructure as Code Commands

Utility Commands
  create-bookmark        Creates a bookmark from JSON.
  end-change             Finishes the specified change. Call this just after you finished the change. This will store a snapshot of the current system state for later reference.
  get-affected-bookmarks Calculates the bookmarks that would be overlapping with a snapshot.
  get-bookmark           Displays the contents of a bookmark.
  get-change             Displays the contents of a change.
  get-snapshot           Displays the contents of a snapshot.
  list-changes           Displays the contents of a change.
  list-invites           List all invites
  manual-change          Creates a new Change from a given query
  request                Runs a request against the overmind API
  start-change           Starts the specified change. Call this just before you're about to start the change. This will store a snapshot of the current system state for later reference.
  submit-plan            Creates a new Change from a given terraform plan file

Additional Commands:
  completion             Generate the autocompletion script for the specified shell
  create-invite          Create a new invite
  help                   Help about any command
  revoke-invites         Revoke an existing invite

Flags:
      --api-key string             The API key to use for authentication, also read from OVM_API_KEY environment variable
      --api-key-url string         The overmind API Keys endpoint (defaults to --url)
      --auth0-client-id string     OAuth Client ID to use when connecting with auth (default "j3LylZtIosVPZtouKI8WuVHmE6Lluva1")
      --auth0-domain string        Auth0 domain to connect to (default "om-prod.eu.auth0.com")
      --gateway-url string         The overmind Gateway endpoint (defaults to /api/gateway on --url)
  -h, --help                       help for overmind
      --honeycomb-api-key string   If specified, configures opentelemetry libraries to submit traces to honeycomb. This requires --otel to be set.
      --json-log                   Set to true to emit logs as json for easier parsing.
      --log string                 Set the log level. Valid values: panic, fatal, error, warn, info, debug, trace (default "info")
      --otel                       If specified, configures opentelemetry and - optionally, see --sentry-dsn - sentry using their default environment configs.
      --run-mode string            Set the run mode for this service, 'release', 'debug' or 'test'. Defaults to 'release'. (default "release")
      --sentry-dsn string          If specified, configures sentry libraries to capture errors. This requires --otel to be set.
      --stdout-trace-dump          Dump all otel traces to stdout for debugging. This requires --otel to be set.
      --url string                 The overmind API endpoint (default "https://api.prod.overmind.tech")
  -v, --version                    version for overmind

Use "overmind [command] --help" for more information about a command.

The problem is that I think this is way too much text for a default help command. There are a number of things here that users will likely never use like creating invites, I am thinking maybe we should group these based on the service in the API so that they are more similar using both groups and subcommands. This would mean it would look like this:

$ overmind help
Calculate the blast radius of your changes, track risks, and make changes with confidence

Usage:
  overmind [command]

Infrastructure as Code Commands
  terrafrorm             Run Terraform with Overmind as a wrapper calculating the blast radius and risks

Overmind API
  changes                Create, update and delete changes in Overmind
  bookmarks              Interact with the bookarks that were created in the Explore view
  snapshots              Create, view and delete snapshots if your infrastructure
  invites                Manage invitations to your Overmind team

Additional Commands:
  completion             Generate the autocompletion script for the specified shell
  help                   Help about any command

Flags:
      --api-key string             The API key to use for authentication, also read from OVM_API_KEY environment variable
      --api-key-url string         The overmind API Keys endpoint (defaults to --url)
      --auth0-client-id string     OAuth Client ID to use when connecting with auth (default "j3LylZtIosVPZtouKI8WuVHmE6Lluva1")
      --auth0-domain string        Auth0 domain to connect to (default "om-prod.eu.auth0.com")
      --gateway-url string         The overmind Gateway endpoint (defaults to /api/gateway on --url)
  -h, --help                       help for overmind
      --honeycomb-api-key string   If specified, configures opentelemetry libraries to submit traces to honeycomb. This requires --otel to be set.
      --json-log                   Set to true to emit logs as json for easier parsing.
      --log string                 Set the log level. Valid values: panic, fatal, error, warn, info, debug, trace (default "info")
      --otel                       If specified, configures opentelemetry and - optionally, see --sentry-dsn - sentry using their default environment configs.
      --run-mode string            Set the run mode for this service, 'release', 'debug' or 'test'. Defaults to 'release'. (default "release")
      --sentry-dsn string          If specified, configures sentry libraries to capture errors. This requires --otel to be set.
      --stdout-trace-dump          Dump all otel traces to stdout for debugging. This requires --otel to be set.
      --url string                 The overmind API endpoint (default "https://api.prod.overmind.tech")
  -v, --version                    version for overmind

Use "overmind [command] --help" for more information about a command.

I think this would be a bit better for a few reasons:

  • It would be obvious where new functionality should go for actions that are direct API interactions since they can simply sit in the same subcommands share names with the API actions that they call. This would help developers building this CLI as there are less decisions to make
  • Users that are familiar with the API could easily find where in the CLI they need to poke to get the API to do something and vice-versa

Arguments

I want to get rid of all the arguments on the root command, to that end I have audited the ones that we have and what they are used for:

  • --url Move this to all of the API endpoints, consider what to do about Terraform though....
    • All auth clients use this
    • Lots of things use this for logging
    • Used in ensureToken() which is used in basically everything
    • A couple of things use the directly for determing the URL of the API endpoint:
      • changes manual-change
      • changes submit-plan
      • request
      • TODO: Why only these three? Wouldn't all endpoints need to construct a similar URL
  • --gateway-url REMOVE, only use base URL
    • As an override for url in"
      • changes manual-change
      • changes submit-plan
      • request
  • --api-key
    • Used in ensureToken() which is everywhere
  • --api-key-url REMOVE, only use base URL
    • Most of the auth clients
    • Logs
  • --auth0-client-id Can probably remove this
    • ensureToken()
  • --auth0-domain Can probably remove this
    • ensureToken()
  • --otel REMOVE
    • Nowhere
  • --honeycomb-api-key Move to env var, only configure tracing if set. Could also set this to hidden so that it doesn't show up in the help output
    • Root cmd pre-run
  • --sentry-dsn REMOVE
    • Nowhere
  • --run-mode REMOVE
    • Nowhere
  • --json-log REMOVE
    • Root cmd pre-run
  • --stdout-trace-dump REMOVE
    • Nowhere

Also add blast radius for resources that have drifted

Terraform is capable of detecting drift like so:

Terraform detected the following changes made outside of Terraform since the
last "terraform apply" which may have affected this plan:

  # module.eks.aws_eks_cluster.this[0] has changed
  ~ resource "aws_eks_cluster" "this" {
        id                        = "dogfood"
        name                      = "dogfood"
      ~ platform_version          = "eks.2" -> "eks.3"
        tags                      = {}
        # (10 unchanged attributes hidden)

        # (4 unchanged blocks hidden)
    }


Unless you have made equivalent changes to your configuration, or ignored the
relevant attributes using ignore_changes, the following plan may include
actions to undo or respond to these changes.

When it does, the data is stored in resource_drift rather than resource_changes. Since wither of these things could represent a change that has impact, we should add these to the blast radius too.

This would mean merging the two, probably here: https://github.com/overmindtech/ovm-cli/blob/b4cdf352357c3a93eac933ef72435a2171b060c8/cmd/submitplan.go#L76

cli: issue with ending the change

possibly because after the S3 test change no items were available for snapshotting?

david in debian in cli on  dev [$!?] via 🐹 via 💠 default on ☁️  sso-david (eu-west-2) took 6s 
❯ terraform plan
module.bucket.data.aws_region.current: Reading...
module.bucket.data.aws_caller_identity.current: Reading...
module.bucket.data.aws_partition.current: Reading...
module.bucket.data.aws_region.current: Read complete after 0s [id=eu-west-2]
module.bucket.data.aws_partition.current: Read complete after 0s [id=aws]
module.bucket.aws_s3_bucket.this[0]: Refreshing state... [id=cli-testtest20240502144618415800000001]
module.bucket.data.aws_caller_identity.current: Read complete after 0s [id=261928782070]
module.bucket.aws_s3_bucket_public_access_block.this[0]: Refreshing state... [id=cli-testtest20240502144618415800000001]
module.bucket.aws_s3_bucket_ownership_controls.this[0]: Refreshing state... [id=cli-testtest20240502144618415800000001]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following
symbols:
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # module.bucket.aws_s3_bucket.this[0] must be replaced
-/+ resource "aws_s3_bucket" "this" {
      + acceleration_status         = (known after apply)
      + acl                         = (known after apply)
      ~ arn                         = "arn:aws:s3:::cli-testtest20240502144618415800000001" -> (known after apply)
      ~ bucket                      = "cli-testtest20240502144618415800000001" -> (known after apply)
      ~ bucket_domain_name          = "cli-testtest20240502144618415800000001.s3.amazonaws.com" -> (known after apply)
      ~ bucket_prefix               = "cli-testtest" -> "cli-testdemo" # forces replacement
      ~ bucket_regional_domain_name = "cli-testtest20240502144618415800000001.s3.eu-west-2.amazonaws.com" -> (known after apply)
      ~ hosted_zone_id              = "Z3GKZC51ZF0DB4" -> (known after apply)
      ~ id                          = "cli-testtest20240502144618415800000001" -> (known after apply)
      + policy                      = (known after apply)
      ~ region                      = "eu-west-2" -> (known after apply)
      ~ request_payer               = "BucketOwner" -> (known after apply)
      - tags                        = {} -> null
      ~ tags_all                    = {} -> (known after apply)
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)
        # (2 unchanged attributes hidden)

      - grant {
          - id          = "e55964492974ba2b8f56bade88cfb30ca2e092bdf672d45d26c46b4a90fb3434" -> null
          - permissions = [
              - "FULL_CONTROL",
            ] -> null
          - type        = "CanonicalUser" -> null
        }

      - server_side_encryption_configuration {
          - rule {
              - bucket_key_enabled = false -> null

              - apply_server_side_encryption_by_default {
                  - sse_algorithm = "AES256" -> null
                }
            }
        }

      - versioning {
          - enabled    = false -> null
          - mfa_delete = false -> null
        }
    }

  # module.bucket.aws_s3_bucket_ownership_controls.this[0] must be replaced
-/+ resource "aws_s3_bucket_ownership_controls" "this" {
      ~ bucket = "cli-testtest20240502144618415800000001" # forces replacement -> (known after apply) # forces replacement
      ~ id     = "cli-testtest20240502144618415800000001" -> (known after apply)

        # (1 unchanged block hidden)
    }

  # module.bucket.aws_s3_bucket_public_access_block.this[0] must be replaced
-/+ resource "aws_s3_bucket_public_access_block" "this" {
      ~ bucket                  = "cli-testtest20240502144618415800000001" # forces replacement -> (known after apply) # forces replacement
      ~ id                      = "cli-testtest20240502144618415800000001" -> (known after apply)
        # (4 unchanged attributes hidden)
    }

Plan: 3 to add, 0 to change, 3 to destroy.

────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform
apply" now.

david in debian in cli on  dev [$!?] via 🐹 via 💠 default on ☁️  sso-david (eu-west-2) took 8s 
❯ overmind terraform plan
✅ Connected to Overmind
✅ Authentication successful, token stored locally (/home/david/.overmind/token.json)
✅ Configuring AWS Access (config stored)

✅ AWS Source: running
✅ stdlib Source: running
█ Planning Changes

  Running terraform plan -out overmind.plan                                     


⏳ Processing Planned Changes
module.bucket.data.aws_caller_identity.current: Reading...
module.bucket.data.aws_partition.current: Reading...
module.bucket.data.aws_region.current: Reading...
module.bucket.data.aws_partition.current: Read complete after 0s [id=aws]
module.bucket.data.aws_region.current: Read complete after 0s [id=eu-west-2]
module.bucket.aws_s3_bucket.this[0]: Refreshing state... [id=cli-testtest20240502144618415800000001]
module.bucket.data.aws_caller_identity.current: Read complete after 0s [id=261928782070]
module.bucket.aws_s3_bucket_public_access_block.this[0]: Refreshing state... [id=cli-testtest20240502144618415800000001]
module.bucket.aws_s3_bucket_ownership_controls.this[0]: Refreshing state... [id=cli-testtest20240502144618415800000001]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following
symbols:
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # module.bucket.aws_s3_bucket.this[0] must be replaced
-/+ resource "aws_s3_bucket" "this" {
      + acceleration_status         = (known after apply)
      + acl                         = (known after apply)
      ~ arn                         = "arn:aws:s3:::cli-testtest20240502144618415800000001" -> (known after apply)
      ~ bucket                      = "cli-testtest20240502144618415800000001" -> (known after apply)
      ~ bucket_domain_name          = "cli-testtest20240502144618415800000001.s3.amazonaws.com" -> (known after apply)
      ~ bucket_prefix               = "cli-testtest" -> "cli-testdemo" # forces replacement
      ~ bucket_regional_domain_name = "cli-testtest20240502144618415800000001.s3.eu-west-2.amazonaws.com" -> (known after apply)
      ~ hosted_zone_id              = "Z3GKZC51ZF0DB4" -> (known after apply)
      ~ id                          = "cli-testtest20240502144618415800000001" -> (known after apply)
      + policy                      = (known after apply)
      ~ region                      = "eu-west-2" -> (known after apply)
      ~ request_payer               = "BucketOwner" -> (known after apply)
      - tags                        = {} -> null
      ~ tags_all                    = {} -> (known after apply)
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)
        # (2 unchanged attributes hidden)

      - grant {
          - id          = "e55964492974ba2b8f56bade88cfb30ca2e092bdf672d45d26c46b4a90fb3434" -> null
          - permissions = [
              - "FULL_CONTROL",
            ] -> null
          - type        = "CanonicalUser" -> null
        }

      - server_side_encryption_configuration {
          - rule {
              - bucket_key_enabled = false -> null

              - apply_server_side_encryption_by_default {
                  - sse_algorithm = "AES256" -> null
                }
            }
        }

      - versioning {
          - enabled    = false -> null
          - mfa_delete = false -> null
        }
    }

  # module.bucket.aws_s3_bucket_ownership_controls.this[0] must be replaced
-/+ resource "aws_s3_bucket_ownership_controls" "this" {
      ~ bucket = "cli-testtest20240502144618415800000001" # forces replacement -> (known after apply) # forces replacement
      ~ id     = "cli-testtest20240502144618415800000001" -> (known after apply)

        # (1 unchanged block hidden)
    }

  # module.bucket.aws_s3_bucket_public_access_block.this[0] must be replaced
-/+ resource "aws_s3_bucket_public_access_block" "this" {
      ~ bucket                  = "cli-testtest20240502144618415800000001" # forces replacement -> (known after apply) # forces replacement
      ~ id                      = "cli-testtest20240502144618415800000001" -> (known after apply)
        # (4 unchanged attributes hidden)
    }

Plan: 3 to add, 0 to change, 3 to destroy.

────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Saved the plan to: overmind.plan

To perform exactly these actions, run the following command to apply:
    terraform apply "overmind.plan"


✅ Connected to Overmind
✅ Authentication successful, token stored locally (/home/david/.overmind/token.json)
✅ Configuring AWS Access (config stored)

✅ AWS Source: running
✅ stdlib Source: running
✅ Planning Changes
✅ Processing Planned Changes

  Processing plan from terraform plan -out overmind.plan                        


Calculating Blast Radius - finished: 1 item

Check the blast radius graph and risks at:
https://app.overmind.tech/changes/24d3171f-194d-4ebd-a669-88f8bd995e98/blast-radius



david in debian in cli on  dev [$!?] via 🐹 via 💠 default on ☁️  sso-david (eu-west-2) took 2m40s 
❯ overmind terraform apply
✅ Connected to Overmind
✅ Using stored token
✅ Configuring AWS Access (config stored)

✅ AWS Source: running
✅ stdlib Source: running

     Applying Changes                                                           
                                                                                
  Applying changes with terraform apply overmind.plan                           


Starting Change - pending
Ending Change - pending

module.bucket.aws_s3_bucket_ownership_controls.this[0]: Destroying... [id=cli-testtest20240502144618415800000001]
module.bucket.aws_s3_bucket_ownership_controls.this[0]: Destruction complete after 1s
module.bucket.aws_s3_bucket_public_access_block.this[0]: Destroying... [id=cli-testtest20240502144618415800000001]
module.bucket.aws_s3_bucket_public_access_block.this[0]: Destruction complete after 0s
module.bucket.aws_s3_bucket.this[0]: Destroying... [id=cli-testtest20240502144618415800000001]
module.bucket.aws_s3_bucket.this[0]: Destruction complete after 1s
module.bucket.aws_s3_bucket.this[0]: Creating...
module.bucket.aws_s3_bucket.this[0]: Creation complete after 1s [id=cli-testdemo20240503144813634000000001]
module.bucket.aws_s3_bucket_public_access_block.this[0]: Creating...
module.bucket.aws_s3_bucket_public_access_block.this[0]: Creation complete after 0s [id=cli-testdemo20240503144813634000000001]
module.bucket.aws_s3_bucket_ownership_controls.this[0]: Creating...
module.bucket.aws_s3_bucket_ownership_controls.this[0]: Creation complete after 1s [id=cli-testdemo20240503144813634000000001]

Apply complete! Resources: 3 added, 0 changed, 3 destroyed.

✅ Connected to Overmind
✅ Using stored token
✅ Configuring AWS Access (config stored)

✅ AWS Source: running
✅ stdlib Source: running

     Applying Changes                                                           
                                                                                
  Applying changes with terraform apply overmind.plan                           


Starting Change - pending
Ending Change - pending








✅ Connected to Overmind
✅ Using stored token
✅ Configuring AWS Access (config stored)

✅ AWS Source: running
✅ stdlib Source: running

     Applying Changes                                                           
                                                                                
  Applying changes with terraform apply overmind.plan                           


Starting Change - pending
Ending Change - pending



                                                                                
  │ Fatal Error: failed to process end change: internal: error processing:       
  │ internal: all queries failed                                                 


david in debian in cli on  dev [$!?] via 🐹 via 💠 default on ☁️  sso-david (eu-west-2) took 14s 
❯

Add more validation to AWS before spawning source

We should do a bit more validation than we currently are on the AWS credentials before we pass them on to the source and potentially get a cryptic error. A good example if this is me running the following command:

❯ go run main.go terraform plan
✅ Connected to Overmind
✅ Authentication successful, token stored locally (/Users/dylanratcliffe/.overmind/token.json)
▓ Configuring AWS Access (config stored)
⏳ Planning Changes
⏳ Processing Planned Changes
ERRO Error retrieving account information          error="operation error STS: GetCallerIdentity, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, request canceled, context deadline exceeded" region=eu-west-1 strategy=defaults


✅ Connected to Overmind
✅ Authentication successful, token stored locally (/Users/dylanratcliffe/.overmind/token.json)
⛔️ Error while configuring AWS Access: failed to initialise sources: failed to initialize AWS source engine: error getting caller identity for region eu-west-1: operati
⏳ Planning Changes
⏳ Processing Planned Changes
Error: failed to initialise sources: failed to initialize AWS source engine: error getting caller identity for region eu-west-1: operation error STS: GetCallerIdentity,

I think what went wrong is that it's trying to use inbuilt EC2 auth like as if it's running on an EC2 instance, no clue why especially sine it can't actually access the IMDS endpoint. I'm thinking what we should do is before starting up the AWS source we should do some pre-flight checks on the config to make sure it makes sense i.e.

  • Note down the auth method we are using
  • Run STS: GetCallerIdentity to make sure we can authenticate
  • Maybe use this to try a few regions? We could try to authenticate against all regions to make sure it works and then configure the working regions? Though to be fair terraform will likely only be running against whatever is the main region so maybe that's not necessary

If any of these pre-flight checks fail we should print a super nice error that includes:

  • What AWS auth strategy we used
  • Which region we used
  • What the error was

In fairness the user would have got the same error if they ran terraform plan e.g.

❯ terraform plan

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: No valid credential sources found
│
│   with provider["registry.terraform.io/hashicorp/aws"],
│   on main.tf line 14, in provider "aws":
│   14: provider "aws" {}
│
│ Please see https://registry.terraform.io/providers/hashicorp/aws
│ for more information about providing credentials.
│
│ Error: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, exceeded maximum number of attempts, 3, request send
│ failed, Get "http://169.254.169.254/latest/meta-data/iam/security-credentials/": dial tcp 169.254.169.254:80: connect: host is down
│
╵

However I think we should strive to be at least as good as the Terraform errors, and we should be extra clear about what we are doing in case there are differences between the way we get configured and the way Terraform does (i.e. the user has a hardcoded region that we don't read, we would want this to be nice and obvious)

Failed to identify change errors

During some CI runs like this one we are getting errors when looking for the change:

+ '[' pull_request = pull_request ']'
++ jq -r .pull_request.title
+ title='Fix GRPC service names'
+ description='This change was automatically created from a PR'
+ ticket_link=https://github.com/overmindtech/deploy/pull/485
+ ./overmindtech/ovm-cli submit-plan --title 'Fix GRPC service names' --description 'This change was automatically created from a PR' --ticket-link https://github.com/overmindtech/deploy/pull/485 --log debug ./tfplan.json
level=debug msg="using provided token for authentication"
level=debug msg="Connecting to overmind apikeys API" api-key-url="https://api.prod.overmind.tech/"
level=debug msg="successfully authenticated"
level=info msg="Reading 1 plan file"
level=debug msg="Skipping unmapped resource" file=./tfplan.json terraform-address=data.auth0_client.api_keys
level=debug msg="Skipping unmapped resource" file=./tfplan.json terraform-address=data.auth0_client.frontend
level=debug msg="Skipping unmapped resource" file=./tfplan.json terraform-address=auth0_client.api_keys
level=debug msg="Skipping unmapped resource" file=./tfplan.json terraform-address=auth0_client.frontend
level=debug msg="Found provider mappings" file=./tfplan.json provider-config-key=kubernetes terraform-address=kubernetes_deployment.api_server
level=debug msg="Mapped terraform to query" method=GET query=api-server scope=dogfood.default type=Deployment
level=debug msg="Found provider mappings" file=./tfplan.json provider-config-key=kubernetes terraform-address=kubernetes_ingress_v1.api_lb
level=debug msg="Mapped terraform to query" method=GET query=api-lb scope=dogfood.default type=Ingress
level=debug msg="Skipping unmapped resource" file=./tfplan.json terraform-address=vercel_project_environment_variable.auth0_client_secret
level=info msg="Plan (./tfplan.json) contained 7 changing resources: 2 supported 5 unsupported"
level=info msg="  ✓ kubernetes_deployment (1)"
level=info msg="  ✓ kubernetes_ingress_v1 (1)"
level=info msg="  ✗ auth0_client (4)"
level=info msg="  ✗ vercel_project_environment_variable (1)"
level=debug msg="Connecting to overmind changes API" changes-url="https://api.prod.overmind.tech/"
level=debug msg="Connecting to overmind changes API" changes-url="https://api.prod.overmind.tech/"
level=info msg="Created a new change" change=9d06ac65-25f2-4d90-816b-874387040e37
level=info msg="Finding expected changes in Overmind"
level=debug msg="Query status update" change=9d06ac65-25f2-4d90-816b-874387040e37 query=682042a3-d183-4176-8aa6-e8edaa4a0f6a status=STARTED
level=debug msg="Query status update" change=9d06ac65-25f2-4d90-816b-874387040e37 query=783650f2-83ea-492b-abab-9f6f6ddf65f1 status=STARTED
level=debug msg="New item" change=9d06ac65-25f2-4d90-816b-874387040e37 item=dogfood.default.Ingress.api-lb
level=debug msg="New item" change=9d06ac65-25f2-4d90-816b-874387040e37 item=dogfood.default.Deployment.api-server
level=debug msg="Still waiting for responders" change=9d06ac65-25f2-4d90-816b-8743[870](https://github.com/overmindtech/deploy/actions/runs/6248877436/job/16964370207#step:12:872)40e37 post_processing_complete=false queriesSent=true responders=2 summary="complete:2  responders:2"
level=debug msg="Query status update" change=9d06ac65-25f2-4d90-816b-874387040e37 query=783650f2-83ea-492b-abab-9f6f6ddf65f1 status=FINISHED
level=debug msg="Query status update" change=9d06ac65-25f2-4d90-816b-874387040e37 query=682042a3-d183-4176-8aa6-e8edaa4a0f6a status=FINISHED
level=info msg="All queries complete" allDone=true change=9d06ac65-25f2-4d90-816b-874387040e37 post_processing_complete=true queriesSent=true responders=2 summary="complete:2  responders:2"
level=info msg="  kubernetes_ingress_v1"
level=info msg="    ✓ api_lb (found)"
level=info msg="  kubernetes_deployment"
level=info msg="    ✓ api_server (found)"
level=info msg="Updating changing items on the change record" change=9d06ac65-25f2-4d90-816b-[874](https://github.com/overmindtech/deploy/actions/runs/6248877436/job/16964370207#step:12:876)387040e37 received_items=2
level=info msg="Status update" change=9d06ac65-25f2-4d90-816b-874387040e37 fields.msg="state:STATE_DISCOVERING"
level=info msg="Status update" change=9d06ac65-25f2-4d90-816b-874387040e37 fields.msg="state:STATE_DISCOVERING  numItems:5  numEdges:5"
level=info msg="Status update" change=9d06ac65-25f2-4d90-816b-874387040e37 fields.msg="state:STATE_FINDING_APPS  numItems:9  numEdges:9"
level=info msg="Status update" change=9d06ac65-25f2-4d90-816b-874387040e37 fields.msg="state:STATE_DONE  numItems:9  numEdges:9"
level=info msg="Change ready" change=9d06ac65-25f2-4d90-816b-874387040e37 change-url="https://app.overmind.tech/changes/9d06ac65-25f2-4d90-816b-874387040e37/blast-radius"
level=debug msg="Websocket closing" change=9d06ac65-25f2-4d90-816b-874387040e37 code=StatusNormalClosure reason=
+ echo ticket-link=https://github.com/overmindtech/deploy/pull/485
+ echo https://github.com/overmindtech/deploy/pull/485
+ echo change-url=https://app.overmind.tech/changes/9d06ac65-25f2-4d90-816b-874387040e37/blast-radius
+ ./overmindtech/ovm-cli get-change --change https://app.overmind.tech/changes/9d06ac65-25f2-4d90-816b-874387040e37/blast-radius --format markdown --log debug
level=debug msg="using provided token for authentication"
level=debug msg="Connecting to overmind apikeys API" api-key-url="https://api.prod.overmind.tech/"
level=debug msg="successfully authenticated"
level=error msg="failed to identify change" error="invalid --change value 'https://app.overmind.tech/changes/9d06ac65-25f2-4d90-816b-874387040e37/blast-radius', couldn't parse UUID: invalid UUID length: 7"
Error: Process completed with exit code 1.

As far as I can tell nothing actually went wrong so we need to work out why this is failing

Allow the CLI to update changes rather than create new ones

When we are running the github action (or via some other method) it should be possible to update the blast radius of a change each time the CLI is run, rather than always creating a new one. This means that if someone creates a PR, then adds more commits, there will only be a single change created.

This may also require changes in the github action depending on the implementation

Not generating changes for some reason

With a plan that affects the following resources:

  • kubernetes_persistent_volume_claim_v1
  • kubernetes_service
  • kubernetes_deployment

It's not generating a correct blast radius:

time="2023-08-07T15:37:00Z" level=info msg="resolving items from terraform plan"
time="2023-08-07T15:37:00Z" level=warning msg="skipping unmapped resource" terraform-address=auth0_client.api_keys
time="2023-08-07T15:37:00Z" level=warning msg="skipping unmapped resource" terraform-address=auth0_client.frontend
time="2023-08-07T15:37:00Z" level=warning msg="skipping unmapped resource" terraform-address=helm_release.nats
time="2023-08-07T15:37:00Z" level=warning msg="skipping unmapped resource" terraform-address=kubectl_manifest.srcman_manager
time="2023-08-07T15:37:00Z" level=warning msg="skipping resource without values" terraform-address=kubernetes_deployment.nats_box terraform-query-field="metadata[0].name"
time="2023-08-07T15:37:00Z" level=warning msg="skipping resource without values" terraform-address=kubernetes_deployment.nats_deployment terraform-query-field="metadata[0].name"
time="2023-08-07T15:37:00Z" level=warning msg="skipping resource without values" terraform-address=kubernetes_persistent_volume_claim_v1.nats_config terraform-query-field="metadata[0].name"
time="2023-08-07T15:37:00Z" level=warning msg="skipping resource without values" terraform-address=kubernetes_service.nats terraform-query-field="metadata[0].name"
time="2023-08-07T15:37:01Z" level=info msg="created a new change" change=1d6f8336-814c-4255-9865-ed77f9e89286
time="2023-08-07T15:37:01Z" level=info msg="identifying items" change=1d6f8336-814c-4255-9865-ed77f9e89286 item_count=1
time="2023-08-07T15:37:02Z" level=info msg="still waiting for responders" change=1d6f8336-814c-4255-9865-ed77f9e89286 post_processing_complete=false queriesSent=true responders=0 summary=
time="2023-08-07T15:37:03Z" level=info msg="all responders and queries done" allDone=true change=1d6f8336-814c-4255-9865-ed77f9e89286 post_processing_complete=true queriesSent=true responders=0 summary=
time="2023-08-07T15:37:03Z" level=info msg="updating change record with no items" change=1d6f8336-814c-4255-9865-ed77f9e89286 received_items=0
time="2023-08-07T15:37:04Z" level=info msg="status update" change=1d6f8336-814c-4255-9865-ed77f9e89286 fields.msg="state:STATE_DISCOVERING"
time="2023-08-07T15:37:05Z" level=info msg="status update" change=1d6f8336-814c-4255-9865-ed77f9e89286 fields.msg="state:STATE_FINDING_APPS"
time="2023-08-07T15:37:05Z" level=info msg="status update" change=1d6f8336-814c-4255-9865-ed77f9e89286 fields.msg="state:STATE_DONE"

Create packages

Now that we have people using this manually, we really should be creating packages for package managers so they can update it easily.

Rationalise CLI logging

The logging in the CLI could be a lot better. The things that the users care about are:

  • Did anything fail mapping. What resources in this plan are unsupported and there won't be shown
  • Did any of the queries fail to find anything?

If we end up going with spinners, we'll need to also have a way to detect if we're in a CI ai that we still end up with a output. Potentially mattn/go-isatty could be an option here.

This is probably the most basic example I've found so I'm going to try that: https://github.com/gosuri/uiprogress

I can also use https://github.com/ttacon/chalk for colour and text style

What to convey

Submit plan

  1. Mapping from plan: Here we'll want to convey how many resources are changing, and how many we were able to map successfully. If there were any unmapped resources, they should be called out so the user knows why they won't be shown in the Overmind blast radius
  2. Finding mapped items: If we aren't able to find the mapped items, we need to be telling people that
  3. Calculating blast radius: Ideally we'd give a progress bar here

Start change

Progress bar, instructions that we have taken a snapshot and you can now proceed with the change, link to the change.

End change

Progress bar, link to the completed change, maybe some stats about what happened.

Add more detail to "waiting for risks" logs 

level=info msg="waiting for risk calculation" status=STATUS_INPROGRESS
level=info msg="waiting for risk calculation" status=STATUS_INPROGRESS

We now have more info here so there is no reason why we can't show this ins the logs in CI

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Pending Approval

These branches will be created by Renovate only once you click their checkbox below.

  • Update actions/cache action to v4
  • Update hashicorp/setup-terraform action to v3
  • 🔐 Create all pending approval PRs at once 🔐

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

  • Update pnpm to v9.7.1
  • Lock file maintenance

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

github-actions
.github/actions/go_init/action.yml
  • actions/setup-go v5
  • actions/checkout v4
  • actions/checkout v4
.github/workflows/e2e.yaml
  • actions/cache v3
  • hashicorp/setup-terraform v2
  • aws-actions/configure-aws-credentials v4
  • actions/checkout v4
  • actions/cache v3
  • hashicorp/setup-terraform v2
  • aws-actions/configure-aws-credentials v4
  • actions/checkout v4
  • actions/upload-artifact v4
  • actions/upload-artifact v4
  • actions/setup-node v4
  • pnpm/action-setup v3
  • actions/cache v4
  • lost-pixel/lost-pixel v3.18.2
.github/workflows/release.yml
  • actions/checkout v4
  • goreleaser/goreleaser-action v6
  • dawidd6/action-homebrew-bump-formula v3
.github/workflows/test_release.yml
  • actions/checkout v4
  • goreleaser/goreleaser-action v6
.github/workflows/tests.yml
  • actions/checkout v4
  • actions/checkout v4
  • golangci/golangci-lint-action v6
gomod
go.mod
  • go 1.22.5
  • connectrpc.com/connect v1.16.2
  • github.com/aws/aws-sdk-go-v2 v1.30.3
  • github.com/aws/aws-sdk-go-v2/config v1.27.27
  • github.com/aws/aws-sdk-go-v2/credentials v1.17.27
  • github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.11
  • github.com/charmbracelet/bubbles v0.18.0
  • github.com/charmbracelet/bubbletea v0.26.6
  • github.com/charmbracelet/glamour v0.8.0
  • github.com/charmbracelet/huh v0.5.2
  • github.com/charmbracelet/lipgloss v0.12.1
  • github.com/getsentry/sentry-go v0.28.1
  • github.com/go-jose/go-jose/v4 v4.0.4
  • github.com/google/uuid v1.6.0
  • github.com/hashicorp/hcl/v2 v2.21.0
  • github.com/hashicorp/terraform-config-inspect v0.0.0-20240801114854-6714b46f5fe4@6714b46f5fe4
  • github.com/hexops/gotextdiff v1.0.3
  • github.com/jedib0t/go-pretty/v6 v6.5.9
  • github.com/mattn/go-isatty v0.0.20
  • github.com/mitchellh/go-homedir v1.1.0
  • github.com/muesli/reflow v0.3.0
  • github.com/muesli/termenv v0.15.3-0.20240618155329-98d742f6907a@98d742f6907a
  • github.com/overmindtech/aws-source v0.0.0-20240814120642-8f003757bf74@8f003757bf74
  • github.com/overmindtech/sdp-go v0.89.0
  • github.com/overmindtech/stdlib-source v0.0.0-20240814201637-a1fece628b57@a1fece628b57
  • github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c@5ac0b6a4141c
  • github.com/sirupsen/logrus v1.9.3
  • github.com/spf13/cobra v1.8.1
  • github.com/spf13/viper v1.19.0
  • github.com/stretchr/testify v1.9.0
  • github.com/ttacon/chalk v0.0.0-20160626202418-22c06c80ed31@22c06c80ed31
  • github.com/uptrace/opentelemetry-go-extra/otellogrus v0.3.1
  • github.com/xiam/dig v0.0.0-20191116195832-893b5fb5093b@893b5fb5093b
  • github.com/zclconf/go-cty v1.15.0
  • go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.53.0
  • go.opentelemetry.io/otel v1.28.0
  • go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.28.0
  • go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.28.0
  • go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.28.0
  • go.opentelemetry.io/otel/sdk v1.28.0
  • go.opentelemetry.io/otel/trace v1.28.0
  • golang.org/x/net v0.28.0
  • golang.org/x/oauth2 v0.22.0
  • google.golang.org/protobuf v1.34.2
  • gopkg.in/yaml.v3 v3.0.1
npm
package.json
  • @types/node ^20.14.9
  • lost-pixel ^3.17.0
  • pnpm 9.7.0
terraform
main.tf
  • aws >= 4.56
  • hashicorp/terraform >= 1.5.0
  • terraform-aws-modules/s3-bucket/aws ~> 4.0
tfutils/testdata/config_from_provider/test.tf
tfutils/testdata/providers.tf
  • aws ~> 5.60
  • hashicorp/terraform >= 1.2.0
tfutils/testdata/subfolder/more_providers.tf
  • Check this box to trigger a request for Renovate to run again on this repository

Fail with a nice error when pointed at a statefile

Turns out if you run terraform show -json you get a JSON representation of the statefile. This is close enough to the plan file that Overmind doesn't complain, it just doesn't find any changes. We should check to see if the JSON looks like a statefile, raise a good error.

This is what a statefile looks like: https://github.com/hashicorp/terraform/blob/d9ccdfb7d1191d58868c618e6f0e9a799fd949b5/internal/command/jsonstate/state.go#L34-L39

As part of this change, improve the README to have the exact commands you need to make sure that terraform show is showing a planfile and not a statefile.

The error should also specify what they need to do to fix it

--app option doesn't work against dogfood

When I run the CLI with --app against the dogfood environment it doesn't seem to work:

❯ go run main.go terraform plan --app https://df.overmind-demo.com
go: downloading github.com/charmbracelet/bubbletea v0.26.1
go: downloading github.com/aws/aws-sdk-go-v2/service/dynamodb v1.32.0
go: downloading github.com/aws/aws-sdk-go-v2/service/ec2 v1.160.0
go: downloading github.com/aws/aws-sdk-go-v2/service/rds v1.78.0
✅ Connected to Overmind
▒ Ensuring Token
⏳ Configuring AWS Access
⏳ Planning Changes
⏳ Processing Planned Changes




✅ Connected to Overmind
⛔️ Ensuring Token Error: error getting device code: oauth2: cannot fetch token: 400 Bad Request
Response: {"error":"invalid_request","error_description":"Service not found: https://api.df.overmind-demo.com"}
⏳ Configuring AWS Access
⏳ Planning Changes
⏳ Processing Planned Changes
Error: error getting device code: oauth2: cannot fetch token: 400 Bad Request
Response: {"error":"invalid_request","error_description":"Service not found: https://api.df.overmind-demo.com"}

Collapse auth step once done

Once we have passed the auth step it should collapse down to be a single line so that the scroll-back output is more useful

Create teams CLI

Create a cli for the new teams APIs so people can generate invites easily using the CLI

Implement device oauth flow for CLI

To reduce interruptions to users the above changes should happen on a new Auth0 app, so that everything works with the device flow when downloading a new version of the CLI.

Parse the planned changes from the terraform plan into UpdatePlannedChanges

With UpdatePlannedChanges implemented in api-server, the submit-plan command can now start collecting the planned changes from the terraform plan.

  • Have submit-plan use UpdatePlannedChanges instead of UpdateChangingItems
  • Change the changingItems calculations to also show Before/After attributes
  • Pass MappingQuerys to api-server instead of evaluating locally

With this in place, the planned changes will show up in the blast-radius graph and unmapped items will be displayed in the sidebar.

Mapping problem for modules with count

In this plan we are deleting the resource module.infra.aws_route53_record.frontend_on_vercel[0]. However when we look at the submit-plan output we see some issues:

level=info msg="Reading 1 plan file"
level=warning msg="Skipping resource without query field" app="https://app.overmind.tech/" file=./tfplan.json terraform-address="module.infra.aws_route53_record.frontend_a[0]" terraform-query-field=arn
level=warning msg="Skipping resource without query field" app="https://app.overmind.tech/" file=./tfplan.json terraform-address="module.infra.aws_route53_record.frontend_aaaa[0]" terraform-query-field=arn
level=warning msg="Skipping resource without query field" app="https://app.overmind.tech/" file=./tfplan.json terraform-address="module.infra.aws_route53_record.frontend_on_vercel[0]" terraform-query-field=arn
level=info msg="Plan (./tfplan.json) contained 7 changing resources: 3 supported 4 unsupported"
level=info msg="  ✓ kubernetes_deployment (2)"
level=info msg="  ✓ kubernetes_secret (1)"
level=info msg="  ✗ aws_guardduty_detector_feature (1)"
level=info msg="  ✗ auth0_client (1)"
level=info msg="  ✗ github_actions_environment_secret (1)"
level=info msg="  ✗ kubectl_manifest (1)"

It's fine that we can't find the query field (well maybe not fine, we might want to change the mapping, but it's working as designed) however what isn't fine is the fact that the aws_route53_record resources don't end up in the change at all.

I'm assuming that the logic is bailing out in such a way that it's not actually loading the unmapped ItemDiff, which it should be.

We need to create a test that replicates this, then fix the logic

CLI should just exit on error rather than printing steps that didn't happen

If I get an error, there is a surprisingly large amount of output for an early error. For example if I force an error with a bad value for --app I see the following output:

❯ go run main.go terraform plan --app https://foo.bar.com
ERRO could not fetch instance-data                 error="Get \"https://foo.bar.com/api/public/instance-data\": dial tcp: lookup foo.bar.com: no such host"














                                                                                                                                                           ▓ Connecting to Overmind
⏳ Ensuring Token
⏳ Configuring AWS Access
⏳ Planning Changes
⏳ Processing Planned Changes
Error: failed to get instance data from app: could not fetch instance-data: Get "https://foo.bar.com/api/public/instance-data": dial tcp: lookup foo.bar.com: no such hos



  │ Fatal Error: failed to get instance data from app: could not fetch instance-
  │ data: Get "https://foo.bar.com/api/public/instance-data": dial tcp: lookup
  │ foo.bar.com: no such host

There are a number of issues here:

  • There is a lot of empty space for some reason
  • The "Connecting to Overmind" spinner is in a really weird spot, that isn't a bad copy-paste it actually looks like that
  • We still show all the other steps e.g. ⏳ Ensuring Token despite the fact that we never execute them
  • The 2nd last error (the same one as in the first line is truncated, but not the one that is wrapped) is truncated, it ends with "no such hos"
  • The last error is extremely hard to read on my screen
Screenshot 2024-05-07 at 9 20 48 PM

Another similar error example is this one:

❯ go run main.go terraform plan --app https://df.overmind-demo.com
✅ Connected to Overmind
▓ Ensuring Token
⏳ Configuring AWS Access
⏳ Planning Changes
⏳ Processing Planned Changes




✅ Connected to Overmind
⛔️ Ensuring Token Error: error getting device code: oauth2: cannot fetch token: 400 Bad Request
Response: {"error":"invalid_request","error_description":"Service not found: https://api.df.overmind-demo.com"}
⏳ Configuring AWS Access
⏳ Planning Changes
⏳ Processing Planned Changes
Error: error getting device code: oauth2: cannot fetch token: 400 Bad Request
Response: {"error":"invalid_request","error_description":"Service not found: https://api.df.overmind-demo.com"}

Image:

Screenshot 2024-05-07 at 9 24 58 PM

Thie above one is better, but we still see steps executed that shouldn't be, dead whitespace, and the error being printed twice

Create default change title for ovm-cli

The changes created by the CLI by default don't have a title. We should make sure there is a default. this could be:

  • The name of the latest commit?
  • The name of the parent directory and the timestamp?
  • Some random ID?
  • Something from the plan itself?

cli: word-wrap all bubbletea View() output to terminal size

This is from #270:

The 2nd last error (the same one as in the first line is truncated, but not the one that is wrapped) is truncated, it ends with "no such hos"

it happens because

cli/cmd/terraform_plan.go

Line 382 in 8601c72

bits = append(bits, deletedLineStyle.Render(fmt.Sprintf("Error: %v", m.fatalError)))
doesn't do line wrapping.

Since we don't do this here or anywhere else, the issue probably applies to everything and requires a holistic solution. For now, the markdownToString version on cmdModel suffices

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.