Git Product home page Git Product logo

aws-nuke's Introduction

aws-nuke

license release Go Report Card Maintainability

Overview

Remove all resources from an AWS account.

aws-nuke is stable, but it is likely that not all AWS resources are covered by it. Be encouraged to add missing resources and create a Pull Request or to create an Issue.

What's New in Version 3

Version 3 is a rewrite of this tool using libnuke with a focus on improving a number of the outstanding things that I couldn't get done with the original project without separating out the core code into a library. See Goals below for more.

This is not a comprehensive list, but here are some of the highlights:

  • New Feature: Signed Darwin Binaries for MacOS
  • New Feature: Published Homebrew Tap (ekristen/tap/aws-nuke@3)
  • New Feature: Global Filters
  • New Feature: Run Against All Enabled Regions
  • New Feature: Explain Account and Explain Config Commands
  • Upcoming Feature: Filter Groups (in progress)
  • Breaking Change: root command no longer triggers the run, must use subcommand run (alias: nuke)
  • Breaking Change: CloudFormation Stacks now support a hold and wait for parent deletion process
  • Breaking Change: Nested CloudFormation Stacks are now eligible for deletion and no longer omitted
  • Completely rewrote the core of the tool as a dedicated library libnuke
    • This library has over 95% test coverage which makes iteration and new features easier to implement.
  • Semantic Releases with notifications on issues / pull requests
  • Context is passed throughout the entire library now, including the listing function and the removal function
    • This is in preparation for supporting AWS SDK Go v2
  • New Resources
  • Broke away from rebuy-de/aws-nuke project as a fork for reasons outlined in the history section

Goals

  • Easier maintainability and bug fixing, see go report and code climate badges above
  • Adding additional tests around the core library
  • Adding more tests around specific resource types
  • Adding additional resources and tooling to make adding resources easier
  • Adding documentation for adding resources and using the tool
  • Consider adding DAG for dependencies between resource types and individual resources
  • Support for AWS SDK Go v2

Documentation

All documentation is in the docs/ directory and is built using Material for Mkdocs.

It is hosted at https://ekristen.github.io/aws-nuke/.

History of this Fork

Important: this is a full fork of the original tool written by the folks over at rebuy-de. This fork became necessary after attempting to make contributions and respond to issues to learn that the current maintainers only have time to work on the project about once a month and while receptive to bringing in other people to help maintain, made it clear it would take time. Considering the feedback cycle was already weeks on initial communications, I had to make the hard decision to fork and maintain it.

libnuke

I also needed a version of this tool for Azure and GCP, and initially I just copied and altered the code I needed for Azure, but I didn't want to have to maintain multiple copies of the same code, so I decided to create libnuke to abstract all the code that was common between the two tools and write proper unit tests for it.

Attribution, License, and Copyright

The rewrite of this tool to use libnuke would not have been posssible without the hard work that came before me on the original tool by the team and contributors over at rebuy-de and their original work on rebuy-de/aws-nuke.

This tool is licensed under the MIT license. See the LICENSE file for more information. The bulk of this tool was rewritten to use libnuke which was in part originally sourced from rebuy-de/aws-nuke.

Contribute

You can contribute to aws-nuke by forking this repository, making your changes and creating a Pull Request against this repository. If you are unsure how to solve a problem or have other questions about a contributions, please create a GitHub issue.

aws-nuke's People

Contributors

bethge avatar bjoernhaeuser avatar dependabot[bot] avatar der-eismann avatar ekristen avatar ga-paul-t avatar guillermo-menjivar avatar hligit avatar hmalphettes avatar hv202x1 avatar jami avatar kurtmc avatar mikeschouw avatar mrkenkeller avatar mrprimate avatar nelsonjchen avatar npellegrin avatar optplx avatar rbroemeling avatar renovate[bot] avatar richardneililagan avatar sambattalio avatar sas-pemcne avatar sstoops avatar stephanlindauer avatar steved avatar svenwltr avatar swhite-oreilly avatar tomvachon avatar tylersouthwick avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

aws-nuke's Issues

Feature Idea: Remove Plans

If you have a lot of resources in a lot of regions, it can take a long time to process everything, it might be nice to allow for something like terraform where you can output a "plan" and review it, then run the command a second time with it as an input and then the tool doesn't have to query resources a second time, it can just immediately move to removal state.

Filter for all resource types

We're tagging all of our resources we want to keep from nuking with a specific tag.

Is it possible to do something like this in the config:

presets:
  keep-tag:
    filters:
      *:
        - property: 'tag:nuke.keep'
          value: 'true'

Direct docker build image binary does not work

Unrelated, but mentioning, in case I did something wrong, or something wrong with the build process

I did try to build the docker image locally, and run the binary from the container, but something seemed to be not right with the resulting binary:

in aws-nuke on ๎‚  fix-aws-parition-detection via ๐Ÿณ desktop-linux ๐Ÿน
โฏ history | grep build
  522  rg build
  524  docker build -t aws-nuke:fix-aws-parition-detection .
  525  docker run -w /config -v $(pwd):/config --entrypont sh aws-nuke:fix-aws-parition-detection
/config $ /usr/local/bin/aws-nuke -c /config/forked-nuke.yaml  --force --quiet --assume-role-arn arn:aws:iam::983055175492:role/shared-services-nuke-role
sh: /usr/local/bin/aws-nuke: not found
/config $ ls /usr/local/bin/aws-nuke
/usr/local/bin/aws-nuke
/config $ /usr/local/bin/aws-nuke
sh: /usr/local/bin/aws-nuke: not found
/config $ file /usr/local/bin/aws-nuke
sh: file: not found
/config $ /usr/local/bin/aws-nuke
sh: /usr/local/bin/aws-nuke: not found
/config $ ls -lrtha /usr/local/bin/aws-nuke
-rwxr-xr-x    1 root     root      213.8M Apr 17 07:38 /usr/local/bin/aws-nuke
/config $ chmod +x /usr/local/bin/aws-nuke
chmod: /usr/local/bin/aws-nuke: Operation not permitted
/config $ cd /usr/local/bin/
/usr/local/bin $ ls
aws-nuke
/usr/local/bin $ ls -lrtha
total 214M
drwxr-xr-x    1 root     root        4.0K Jan 26 17:53 ..
-rwxr-xr-x    1 root     root      213.8M Apr 17 07:38 aws-nuke
drwxr-xr-x    1 root     root        4.0K Apr 17 07:38 .
/usr/local/bin $ ./aws-nuke
sh: ./aws-nuke: not found

Originally posted by @stv-io in #142 (comment)

Issues when AWS_DEFAULT_REGION is set as an env var

Hi, I've been trying this fork after discovering it from rebuy-de/aws-nuke#1187 (comment)

I had CI pipelines which I adapted to use this image (keeping in mind the breaking changes). One thing which caused me a bit of pain, was that with the following env vars set AWS_REGION and AWS_DEFAULT_REGION (to eu-west-1 in my case) the cli was exiting with a misleading and confusing error:

time="2024-04-16T14:28:47Z" level=error msg="the custom region 'eu-west-1' must be specified in the configuration 'endpoints'"
time="2024-04-16T14:28:47Z" level=fatal msg="the custom region 'eu-west-1' must be specified in the configuration 'endpoints'"

I have just gotten around to figuring this out, let me know if I can provide additional context.

I plan to come back to this issue if I find anything out

[idea] use aws resource groups tagging api to retrieve tags for bulk of resources

I'm very happy to see this issue already exists, as I was disappointed to see that there are quite a lot of resource types where tags are not fetched. By my very rough count (grep'ing for tag in the resources/ folder) only 127 out of 471 resource types get tags, which is both better (total number) and worse (percentage) than the original rebuy.de version's 117 out of 428.

I would like to suggest that there may be a less time-consuming way to get tags for resource types that don't currently support this. The AWS Resource Groups Tagging API supports many, but not all, AWS services and it's not too hard to get tags for all resources using that API and it would then be possible to merge them into the properties for any resource.

If there's any interest in this, I'd be glad to contribute at least some initial code.

Originally posted by @dupuyarc in #118 (comment)

[idea] IaC quick start for scheduled deletion

A common use case for aws-nuke is to run it on a schedule on sandbox / development account. AWS Even provides prescriptive guidance using eventBridge and CodeDeploy to run aws-nuke across multiple regions or in a hub-and-spoke setup.

Would it make sense to have a CDK Construct to deploy aws-nuke this way?

Fix: S3 Bucket Empty Blocks

Recently discovered that the S3Bucket delete ends up blocking if there are a lot of objects and versions in the bucket. It would be good to refactor this to run as a goroutine in the background while other resources cleanup and the loops occur.

Tags are not respected?

We tag our resources that we want to keep from nuking with key do-not-nuke and value true, then we add this to filter section - and for majority of resources it works. But for some resources it's ignored. For example we tagged a Sagemaker domain:

Screenshot 2024-03-07 at 4 29 54 PM

And using filter similar to this one:

accounts:
  '123456789':
    filters:
      SageMakerDomain:
      - property: "tag:do-not-nuke"
        value: "true"

Similar setup works for majority of other resources, yet here we're getting output

us-west-2 - SageMakerDomain - d-lkdfjldfadsf - [DomainID: "d-lkdfjldfadsf"] - would remove

is it a bug, or some omission on our side? How do we make it respect the tags?

Review Resources for Tag Support

SageMakerUserProfiles
ECSCluster
ServiceDiscoveryNamespace
SQSQueue
CognitoUserPoolDomain
CognitoUserPoolClient
CognitoUserPool
CognitoIdentityProvider
ElasticacheSubnetGroup
ElasticacheCacheCluster

Originally posted by @YuriGal in #110 (comment)

Filter properties are not respected

Part of my filter is set like this

      EC2InternetGatewayAttachment:
      - property: "vpc:OwnerID"
        value: "123456789"
      EC2RouteTable:
      - property: "OwnerID"
        value: "123456789"
      EC2DHCPOption:
      - property: "OwnerID"
        type: "regex"
        value: "(123456789|987654321)"

The account has resources with Owner ID that match the filter. The original nuke had those filtered out. The fork still goes


us-west-2 - EC2InternetGatewayAttachment - igw-03be1339519e46fa9 -> vpc-0debf6fec89321668 - [DefaultVPC: "false"] - would remove
us-west-2 - EC2DHCPOption - dopt-0c578847d991d4df5 - [DefaultVPC: "false"] - would remove
us-west-2 - EC2RouteTable - rtb-0d7b4aa83ad8d4a37 - [DefaultVPC: "false", vpcID: "vpc-0debf6fec89321668"] - would remove

Would it be possible to have those filtered out? Not sure if there're other resources with the same issue, but these 3 definetely have it.

Merge Upstream PRs

Thanks for your work on this fork and maintaining aws-nuke! I noticed that there are several PRs in the original project repo (like this one) that might be a good idea to merge (manually) into this fork. Any plans to do this?

[Idea] Delete EC2 Instance `Name` Tags On Termination

When an EC2 instance is terminated, the instance stays in the terminated state for about an hour, then it disappears from the AWS account. The problem with this is that once aws-nuke finishes by terminating all EC2 instances in the AWS account, the EC2 instances are still leftover in the account. So if subsequent infrastructure migrations create EC2 instances with names (using the Name tag), then the migrations could run into issues because there are now multiple EC2 instances with the same name. For example, a running instance and a terminated instance both with the Name tag value of "webserver". So it might be a good idea if aws-nuke deletes the tags of all EC2 instances before terminating them.

Errors unrelated to nuking cause exit status 1.

During run the nuke logs errors about problems I have no control over, for example

time="2024-02-22T16:06:58Z" level=error msg="Listing FMSNotificationChannel failed:\n    AccessDeniedException: Operation GetNotificationChannel is only available to AWS Firewall Manager Administrators.\n    \tstatus code: 400, request id: db521f67-5dc4-40b4-b0b5-7cf7501841d0" error="AccessDeniedException: Operation GetNotificationChannel is only available to AWS Firewall Manager Administrators.\n\tstatus code: 400, request id: db521f67-5dc4-40b4-b0b5-7cf7501841d0"

time="2024-02-22T16:06:59Z" level=error msg="Listing ECRPublicRepository failed:\n    UnsupportedCommandException: DescribeRepositories command is only supported in us-east-1." error="UnsupportedCommandException: DescribeRepositories command is only supported in us-east-1."

time="2024-02-22T16:06:59Z" level=error msg="Listing FMSPolicy failed:\n    AccessDeniedException: Operation ListPolicies is only available to AWS Firewall Manager Administrators.\n    \tstatus code: 400, request id: 1765d4b5-1813-4e76-b8df-7ce3db7ee609" error="AccessDeniedException: Operation ListPolicies is only available to AWS Firewall Manager Administrators.\n\tstatus code: 400, request id: 1765d4b5-1813-4e76-b8df-7ce3db7ee609"

time="2024-02-22T16:07:00Z" level=fatal msg="*resources.SNSSubscription does not support custom properties"

If I remember correctly original nuke also detected these errors, but allowed execution to continue, with nuking the resources it could nuke. I believe the fork exits with the status code 1, which fails the entire run.

Is it possible to ignore errors such as mentioned above?

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

dockerfile
Dockerfile
  • docker/dockerfile 1.7-labs
  • alpine 3.19.1
  • ghcr.io/acorn-io/images-mirror/golang 1.21
github-actions
.github/workflows/aws-sdk-mocks.yml
  • actions/checkout v4
  • actions/setup-go v5
  • tibdex/github-app-token v2
.github/workflows/commit-lint.yaml
  • actions/checkout v4
  • wagoid/commitlint-github-action v6
.github/workflows/docs.yaml
  • actions/checkout v4
  • actions/configure-pages v5
  • actions/setup-python v5
  • actions/cache v4
  • actions/upload-pages-artifact v3
  • actions/deploy-pages v4
.github/workflows/golangci-lint.yml
  • actions/checkout v4
  • actions/setup-go v5
  • golangci/golangci-lint-action v6
.github/workflows/goreleaser.yml
  • actions/checkout v4
  • actions/checkout v4
  • actions/setup-go v5
  • docker/setup-qemu-action v3
  • docker/setup-buildx-action v3
  • docker/login-action v3
  • sigstore/cosign-installer v3
  • 1password/load-secrets-action v2
  • goreleaser/goreleaser-action v5
  • actions/upload-artifact v4
.github/workflows/semantic-lint.yml
  • amannn/action-semantic-pull-request v5
.github/workflows/semantic.yml
  • actions/checkout v4
  • actions/setup-node v4
.github/workflows/tests.yml
  • actions/checkout v4
  • actions/setup-go v5
gomod
go.mod
  • go 1.21.6
  • github.com/aws/aws-sdk-go v1.52.4
  • github.com/ekristen/libnuke v0.12.0
  • github.com/fatih/color v1.16.0
  • github.com/golang/mock v1.6.0
  • github.com/google/uuid v1.6.0
  • github.com/gotidy/ptr v1.4.0
  • github.com/pkg/errors v0.9.1
  • github.com/sirupsen/logrus v1.9.3
  • github.com/stretchr/testify v1.9.0
  • github.com/urfave/cli/v2 v2.27.2
  • golang.org/x/text v0.15.0
  • gopkg.in/yaml.v3 v3.0.1
regex
.github/workflows/aws-sdk-mocks.yml
.github/workflows/golangci-lint.yml
.github/workflows/goreleaser.yml
.github/workflows/tests.yml

  • Check this box to trigger a request for Renovate to run again on this repository

Need to support --max-wait-retries option

Could you please add --max-wait-retries option that the original nuke had? it tells it how many times it can retry for waiting resources. e.g.

aws-nuke nuke --config nuke-config.yml  --max-wait-retries 10 

will retries 10 times and if not done by then - exits with status 255 and message "Error: Max wait retries of 10 exceeded"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.