It would be very useful if you can provide possibility delete any resource by ARN

When leveraging the entire all.yml services list and using 0.4.1 on macos 10.15.3 against AWS GovCloud regions, us-gov-west-1 and us-gov-east-1 where resource counts are higher than 800 per service awsweeper hangs and often needs to have the config file only include a subset or each service individually.

Had to break out ebs, eip, and security groups out to individual executions. Also, it appears as though vpc peering and public ip associations make it difficult to easily delete vpc's.

Hey,
There are few parts in the code structure that can be improved.
For instance, wipe functionality can not be imported and get executed by another project because it's highly coupled to CLI args and printing the output directly to stdout.

By decoupling wiping functionality and keep command/* only to convert cli args to input params to existing functionalities the project can be used in both way, either from cli or by code.

Another thing that I think can be improved is logging.
I can see currently the project is using all log, logrus and cli.UI to output messages and logs.
This can be unified as well.

Let me know if you agree and need help to improve this. :)
Would be happy to help if I can.

Regards

Requesting addition of load balancer (ALB) and network load balancer (NLB) support within the AWS EC2 service as only elastic load balancers are supported.

A common scenario in AWS deployments is to have a production account and a test account. To ensure that the deployment is only run against the test account, it would be really helpful if the awsweeper could check the account id passed in on the command line or environment variable and then verify that the credentials its using are for that account. If they don't match, then fail before executing.

If this feature sounds worthwhile, I'd be happy to submit a PR for it.

Great tool, by far the best out the several I've used.

I'd like to add some more resources, starting with aws_athena_named_query but possibly more in the future.

Is this guide still relevant, #21? It seems like you're working on a way to auto-generate the code. Would it be better to wait?

Thanks,

When executing awsweeper 0.4.1 on macos 10.15.3 against either us-gov-west-1 or us-gov-east-1, AWS GovCloud regions, the s3 buckets appear in --dry-run mode but are not deleted when the parameter is removed. Command was invoked with full administrator privileges with no policies prohibiting bucket deletion.

Hi - I am trying to get the project and try it out, but this error, please advise, thank you.

`github.com/aws/aws-sdk-go/service/ec2/ec2iface
github.com/cloudetc/awsweeper/resource
github.com/cloudetc/awsweeper/command

github.com/cloudetc/awsweeper/command

go/src/github.com/cloudetc/awsweeper/command/wrapped_main.go:137:27: cannot use conf (type *"github.com/hashicorp/terraform/terraform".ResourceConfig) as type *"github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/terraform".ResourceConfig in argument to p.Validate
go/src/github.com/cloudetc/awsweeper/command/wrapped_main.go:146:23: cannot use conf (type *"github.com/hashicorp/terraform/terraform".ResourceConfig) as type *"github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/terraform".ResourceConfig in argument to p.Configure
go/src/github.com/cloudetc/awsweeper/command/wrapped_main.go:151:9: cannot use &p (type *"github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/terraform".ResourceProvider) as type *"github.com/hashicorp/terraform/terraform".ResourceProvider in return argument:
*"github.com/hashicorp/terraform/terraform".ResourceProvider is pointer to interface, not interface
`

This is what it currently looks like:

  {
    "Type": "aws_iam_role",
    "ID": "AWSServiceRoleForAutoScaling",
    "Tags": null,
    "Created": null,
    "Attrs": null
  },

- type: aws_iam_role
  id: AWSServiceRoleForAutoScaling
  tags: {}
  created: null
  attrs: {}

We have some corporate governance stuff applied over my company's 100+ AWS accounts. We developers use Terraform, the corporate governance resources are created using a variety of other tools.

I'm trying to clean-up all IAM policies that are created by developers, and not managed by these governance tools. But it would appear that regex support is not working correctly against aws_iam_policies → id field.

I've tried:

aws_iam_policy:
  - id: NOT(CloudTrail_CloudWatchLogs_Policy)
  - id: NOT(DatapipeOpsAccessPolicy)
  - id: NOT(Kentik)
  - id: NOT(turbot)

…as well as…

aws_iam_policy:
  - id: NOT(.*CloudTrail_CloudWatchLogs_Policy.*)
  - id: NOT(.*DatapipeOpsAccessPolicy.*)
  - id: NOT(.*Kentik.*)
  - id: NOT(.*turbot.*)

Am I applying the values correctly?

The following explanation hopefully makes it easy contributing the resources you need to this project. As an example, we go through adding the resource aws_vpc (which is already supported).

1) Add Terraform type ID of the new resource type

Fine the terraform name of the AWS resource type to support (here is a list of all existing resource types) and add it to resource/supported.go.

	Vpc                 TerraformResourceType = "aws_vpc"

2) Add function to list resources

Add here a function that lists all resources of the new resource type. For this you need to find the method of the AWS go API to list all VPCs (you find it by looking in the go doc of the AWS API)

func (a *AWS) vpcs() (interface{}, error) {
	output, err := a.DescribeVpcs(&ec2.DescribeVpcsInput{})
	if err != nil {
		return nil, err
	}
	return output.Vpcs, nil
}

Supported resources need to be added to resource/supported.go.

3) Add delete ID

The field name of the ID used by Terraform to delete the resource. For this we actually have to look in the delete method of the AWS provider for the VPC

var (
	deleteIDs = map[TerraformResourceType]string{
                ...
		Vpc:                 "VpcId",
                ...
        }
)

4) Add custom filter method

In most cases can be left as filterGeneric. Some resources require custom filter methods.

5) Integration tests

Solid testing is necessary, since we don't want this tool to delete resources unexpectedly :-) Integration tests for each resource have its own file; the ones for aws_vpc are in test/vpc_test.go.

The test for each resource is structured as follows:

1. Create two different resources of aws_vpc (one with ID/tag foo, another with ID/tag bar)
2. Create a yaml filter for AWSweeper to delete resource with ID/tag foo
3. Check if resource with ID/tag foo is gone
4. Check if resource with ID/tag bar hasn't been deleted

Requesting addition of AWS Kinesis Data Streams as time permits.

Requesting addition of AWS Relational Database (RDS) as time permits. Initially focusing on the deletion of the database instances themselves.

Hi @jckuester, I just have a specific scenario
I want to run the awsweeper in AWS Lambda.
Things are quite good if I just deploy each Lambda with awsweeper in different region
And awsweeper is working as expected in each Lambda of regions

But for cost reason, setup 4 NAT GATEWAY for Lambda use in each region is quite expensive as awsweeper need to download something when running the CMD in Lambda

So I just try to setup 1 Lambda in a region basically running 4 CMD cleanup for save cost purpose with following:

./bin/awsweeper --region ap-northeast-1 --dry-run ./config/aws_resource.yml
./bin/awsweeper --region ap-northeast-2 --dry-run ./config/aws_resource.yml
./bin/awsweeper --region ap-southeast-1 --dry-run ./config/aws_resource.yml
./bin/awsweeper --region ap-southeast-2 --dry-run ./config/aws_resource.yml

But I got the issue in CloudWatch Loggroup which implies CMD not running correctly.

• downloaded and installed provider name=aws version=2.59.0
--
  | 2020-06-03T20:00:07.051+08:00 | • configured provider name=aws version=2.59.0
  | 2020-06-03T20:00:07.095+08:00 | • SHOWING RESOURCES THAT WOULD BE DELETED (DRY RUN)
  | 2020-06-03T20:00:09.369+08:00 | panic: can't use ElementIterator on null value
  | 2020-06-03T20:00:09.369+08:00 | goroutine 1 [running]:
  | 2020-06-03T20:00:09.369+08:00 | github.com/zclconf/go-cty/cty.Value.ElementIterator(0x6378ba0, 0xc000506410, 0x0, 0x0, 0x10, 0x2)
  | 2020-06-03T20:00:09.369+08:00 | /home/jan/go/pkg/mod/github.com/zclconf/[email protected]/cty/value_ops.go:1038 +0x101
  | 2020-06-03T20:00:09.369+08:00 | github.com/zclconf/go-cty/cty.Value.AsValueMap(0x6378ba0, 0xc000506410, 0x0, 0x0, 0x1)
  | 2020-06-03T20:00:09.369+08:00 | /home/jan/go/pkg/mod/github.com/zclconf/[email protected]/cty/value_ops.go:1243 +0xe5
  | 2020-06-03T20:00:09.369+08:00 | github.com/cloudetc/awsweeper/pkg/resource.GetTags(0xc0008f3770, 0xb, 0xb, 0x0)
  | 2020-06-03T20:00:09.369+08:00 | /home/jan/git/github.com/cloudetc/awsweeper/pkg/resource/select.go:54 +0xe7
  | 2020-06-03T20:00:09.369+08:00 | github.com/cloudetc/awsweeper/pkg/resource.Filter.Apply(0xc0004633e0, 0xc000ba2ea0, 0x3, 0x4, 0xc000ba2ea0, 0x3, 0x4)
  | 2020-06-03T20:00:09.369+08:00 | /home/jan/git/github.com/cloudetc/awsweeper/pkg/resource/select.go:15 +0x10a
  | 2020-06-03T20:00:09.369+08:00 | github.com/cloudetc/awsweeper/pkg/resource.List(0xc000528170, 0xc000bc0000, 0xc00068e400, 0xc000440b20, 0x5ab8ee6, 0x6, 0xc00032a390, 0x0, 0x0)
  | 2020-06-03T20:00:09.369+08:00 | /home/jan/git/github.com/cloudetc/awsweeper/pkg/resource/list.go:56 +0x61f
  | 2020-06-03T20:00:09.369+08:00 | main.mainExitCode(0x0)
  | 2020-06-03T20:00:09.369+08:00 | /home/jan/git/github.com/cloudetc/awsweeper/main.go:157 +0xd77
  | 2020-06-03T20:00:09.369+08:00 | main.main()
  | 2020-06-03T20:00:09.369+08:00 | /home/jan/git/github.com/cloudetc/awsweeper/main.go:30 +0x22
  | 2020-06-03T20:00:09.373+08:00 | [ERROR] CalledProcessError: Command '['./bin/awsweeper', '--region', 'ap-northeast-1', '--dry-run', './config/aws_resource.yml']' returned non-zero exit status 2. Traceback (most recent call last):   File "/var/task/src/awsweeper.py", line 17, in lambda_handler     output2 = subprocess.check_output(CMD2, universal_newlines=True)   File "/var/lang/lib/python3.8/subprocess.py", line 411, in check_output     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,   File "/var/lang/lib/python3.8/subprocess.py", line 512, in run     raise CalledProcessError(retcode, process.args,
  | 2020-06-03T20:00:09.374+08:00 | END RequestId: ae1b42fc-50be-4e77-8b27-ceb92a1664a1

The above 4 CMD is running good in my local laptop.
So just want to know if something happen with the code that is not compatible for executing CMD with --region ap-northeast-1 and this specific region is not what Lambda deployed in.

i.e. I run ./bin/awsweeper --region ap-northeast-1 --dry-run ./config/aws_resource.yml in one Lambda and the Lambda is deployed in ap-northeast-2

The Lambda role is quite loose as I give Lambda most power to running anything, so I think this error is not related to permission issue.

Hello,

I'm running awsweeper (macOS Mojave 10.14.1) with a config.yml with inside :

aws_ami:

i got an error :

---

panic: interface conversion: interface {} is *string, not *time.Time

goroutine 1 [running]:
github.com/cloudetc/awsweeper/resource.DeletableResources(0xc000618958, 0x7, 0x32e4280, 0xc0005bf100, 0xc0005bf100, 0x0, 0x0, 0xc00071ae80, 0xc000a5d400)
	/home/janck/git/github.com/cloudetc/awsweeper/resource/resource.go:37 +0x71a
github.com/cloudetc/awsweeper/command.(*Wipe).Run(0xc000a892c0, 0xc00028c2d0, 0x1, 0x1, 0xc000a89230)
	/home/janck/git/github.com/cloudetc/awsweeper/command/wipe.go:65 +0x1ea
github.com/mitchellh/cli.(*CLI).Run(0xc00028e8c0, 0xc000a89200, 0x3d13023, 0x4)
	/home/janck/go/pkg/mod/github.com/mitchellh/[email protected]/cli.go:255 +0x207
github.com/cloudetc/awsweeper/command.WrappedMain(0xc00003a0b8)
	/home/janck/git/github.com/cloudetc/awsweeper/command/wrapped_main.go:92 +0x753
main.main()
	/home/janck/git/github.com/cloudetc/awsweeper/main.go:14 +0x22

other supported resources works fine.

I've this config:

aws_cloudformation_stack:
  - id: awsweeper-del-me

running it I get these logs:

awsweeper --region eu-west-1 --parallel 1 --debug --force config.yaml

   • downloaded and installed provider                  name=aws version=2.59.0
   • configured provider                                name=aws version=2.59.0
   • SHOWING RESOURCES THAT WOULD BE DELETED (DRY RUN)
<snipping>
      • failed to import resource; trying to read resource without import error=Throttling: Rate exceeded
        status code: 400, request id: 18399ea7-5742-409a-9e62-14cf0a14cb51 id=arn:aws:cloudformation:eu-west-1:<redacted-aws-account-id>:stack/awsweeper-del-me/80789cf0-b9ec-11ea-8cc3-02aed67d8a72 type=aws_cloudformation_stack
Error: failed to read current state of resource: Throttling: Rate exceeded
        status code: 400, request id: dca88124-cc18-4375-8aaf-39bcd0bbd6dd
Error: failed to read current state of resource: Throttling: Rate exceeded
        status code: 400, request id: cc2e00e1-1554-4bd3-9ca9-f235fd738e37
Error: failed to read current state of resource: Throttling: Rate exceeded
        status code: 400, request id: f78f05c8-a354-4465-92d1-5e1099bae9b5
<snipping>
      • failed to get tags        error=state is nil: <nil> id=arn:aws:cloudformation:eu-west-1:<redacted-aws-account-id>:stack/awsweeper-del-me/80789cf0-b9ec-11ea-8cc3-02aed67d8a72 type=aws_cloudformation_stack
<snipping>
        ---
        Type: aws_cloudformation_stack
        Found: 1

                Id:             arn:aws:cloudformation:eu-west-1:<redacted-aws-account-id>:stack/awsweeper-del-me/80789cf0-b9ec-11ea-8cc3-02aed67d8a72
                Created:        2020-06-29 09:40:07.952 +0000 UTC

        ---

   • TOTAL NUMBER OF RESOURCES THAT WOULD BE DELETED: 1
   • USER WILL NOT BE ASKED FOR CONFIRMATION (FORCE MODE)
   • STARTING TO DELETE RESOURCES
      • start distributing resources to workers for this run
      • unable to delete resource                          error=resource state is nil; need to call update first resource_id=arn:aws:cloudformation:eu-west-1:<redacted-aws-account-id>:stack/awsweeper-del-me/80789cf0-b9ec-11ea-8cc3-02aed67d8a72 type=aws_cloudformation_stack
   • TOTAL NUMBER OF DELETED RESOURCES: 0

and I can still see the cfn stack in the aws console :(

I've tried it with version 0.7 and it seems to work:

 awsweeper --region eu-west-1 --debug --force config.yaml        
   • downloaded and installed provider                  name=aws version=2.43.0
   • configured provider                                name=aws version=2.43.0
   • using region: eu-west-1  
   • SHOWING RESOURCES THAT WOULD BE DELETED (DRY RUN)

        ---
        Type: aws_cloudformation_stack
        Found: 1

                Id:             arn:aws:cloudformation:eu-west-1:<redacted>/:stack/awsweeper-del-me/80789cf0-b9ec-11ea-8cc3-02aed67d8a72
        Created:        2020-06-29 09:40:07.952 +0000 UTC

        ---

   • TOTAL NUMBER OF RESOURCES THAT WOULD BE DELETED: 1
   • USER WILL NOT BE ASKED FOR CONFIRMATION (FORCE MODE)
   • STARTING TO DELETE RESOURCES
      • start distributing resources to workers for this run
      • worker starts deleting resource                    id=arn:aws:cloudformation:eu-west-1:<redacted>:stack/awsweeper-del-me/80789cf0-b9ec-11ea-8cc3-02aed67d8a72 type=aws_cloudformation_stack worker_id=5
      ⨯ aws_cloudformation_stack                           id=arn:aws:cloudformation:eu-west-1:<redacted>:stack/awsweeper-del-me/80789cf0-b9ec-11ea-8cc3-02aed67d8a72
   • TOTAL NUMBER OF DELETED RESOURCES: 1

Requesting addition of AWS GuardDuty as time permits.

The fear I have with deleting by creation time is that I have old resources that are still being used. This is meaningful for things like CloudWatch Log Groups and IAM Access Keys.

Some of these are easy (e.g., IAM Access Keys have a last-used property) and some are less so (e.g., CloudWatch Log Groups last use time would be defined by the latest event in each Log Stream).

The addition of this functionality would make this tool incredibly useful to me.

Requesting addition of AWS CloudWatch Logs, Alarms, and Events as time permits.

Requesting addition of AWS SNS Topics and Subscriptions as time permits.

Requesting addition of AWS CloudTrail trails as time permits.

FYI I submitted a pr to get awsweeper into Homebrew: Homebrew/homebrew-core#33984

Requesting addition of AWS Certificate Manager (ACM) as time permits.

Requesting addition of AWS Lambda functions as time permits.

Hi,

since you're rely on terraform resources, can you add aws_eks_cluster support?

Thanks!

Hi There,

Awesome project, it would be fantastic if Creation Date filter would exist for all type of objects.

Cheers,
smarunich

When executing awsweeper 0.4.1 on macos 10.15.3 against either us-gov-west-1 or us-gov-east-1, AWS GovCloud regions, the route53 hosted zones appear in --dry-run mode but are not deleted when the parameter is removed. Command was invoked with full administrator privileges with no policies prohibiting hosted zone deletion.

It would be neat if the tool supported deleting aws_lambda_functions.

Suggestion to not use aws sdk describe-* API to enumerate instances, instead use output of terraform data lookups. This makes more sense we use terraform to delete resources so why not offload discovery too.

`
data "aws_instances" "ec2" {
instance_tags {
Name = "*"
}
instance_state_names = ["running", "stopped"]
}

output "ec2" {
value = "${data.aws_instances.ec2.ids}"
}
`

Hi! I'm using awsweeper v0.3.2 (binary release off github) with the following config file:

aws_ami:
aws_autoscaling_group:
aws_cloudformation_stack:
aws_ebs_snapshot:
aws_ebs_volume:
aws_efs_file_system:
aws_eip:
aws_elb:
aws_iam_group:
aws_iam_instance_profile:
aws_iam_policy:
aws_iam_role:
    - id: OrganizationAccountAccessRole
aws_iam_user:
aws_instance:
aws_internet_gateway:
aws_key_pair:
#aws_kms_alias:
#aws_kms_key:
aws_launch_configuration:
aws_nat_gateway:
aws_network_acl:
aws_network_interface:
aws_route53_zone:
aws_route_table:
aws_s3_bucket:
aws_security_group:
aws_subnet:
aws_vpc:
aws_vpc_endpoint:

The role with id OrganizationAccountAccessRole is deleted anyway. --dry-run shows OrganizationAccountAccessRole as the id of the resource, so it's not like the problem is it's secretly using an ARN as the id or anything.

As far I understand, we can filter resources based on a specific tag's value. For example, the following is going to select all ecs clusters based on a tag named environment whose value is not play

aws_ecs_cluster:
  - tags:
      environment: NOT(play)

Is there any way we can filter resources, if a specific tag is not present altogether. So based on previous example, I want to delete all ecs_clusters where the tag environment is not present.

Thanks.

Using UPX you can shrink the size of the binaries from ~100mb to ~25mb. The settings I used were upx -9 <BINARY>.

$ upx -9 awsweeper
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2018
UPX git-716d20  Markus Oberhumer, Laszlo Molnar & John Reiser   Aug 27th 2018

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
 118215644 ->  25976848   21.97%   macho/amd64   awsweeper

Packed 1 file.

my config:

aws_cloudformation_stack:
  - id: NOT(mystackname1|mystackname2)
    created:
      after: 2020-01-01

my command: ./awsweeper --profile my-sandbox --region eu-west-1 --dry-run --output string ./config.yml

gives an empty result (so nothing to be deleted)

INFO[0000] using region: eu-west-1                      
INFO[0001] This is a test run, nothing will be deleted! 

If I check the json by removing the date filter I will see an object like this one:

[
  {
    "Type": "aws_cloudformation_stack",
    "ID": "arn:aws:cloudformation:eu-west-1:*****:stack/*****/****-****-****-***-****",
    "Tags": {
      "***": "****"
    },
    "Created": null,
    "Attrs": null
  },
... <redacted>

You can see here that the date field is null

Update 1: To be noted that i'm using version v0.4.1 because of #81
Update 2: I've cloned the project and compiled it, same result

Hi,

some reason I'm not able to install on my Mac os is there any help or guide,

Thanks you in advance.

$ curl -sSfL https://raw.githubusercontent.com/cloudetc/awsweeper/master/install.sh | sh -s v0.5.0
cloudetc/awsweeper info checking GitHub for tag 'v0.5.0'
cloudetc/awsweeper info found version: 0.5.0 for v0.5.0/darwin/amd64
localuser@xxxxxxxxx ~/allstate_Devops/awsweeper (master)
$ awsweeper --help
bash: awsweeper: command not found

I have a number of users defined in my aws credentials file that I would like to run awsweeper against, but in different regions. I have observed the following behaviour (all done using the --dry-run option)

1. When my credentials do not contain a region (which is what I want), I get the following response:

err: Not a valid region:

That is what I expected. However, when supplying the --region override for any region in which I have resources (along with the relevant --profile), it returns no resources. Running a describe on the resource with the same --profile and --region using the aws cli, I see the resources I expect.

2. When my credentials do contain a region, everything works fine.

3. When my credentials do contain a region, and I try to override it with the --region flag, it still just runs against the region defined in the credentials. So the override does not seem to work.

The behaviour I expected was that if I had credentials defined with or without a region, using the --region override would cause awsweeper to honour this override.

Didn't investigate much, but I figured it's useful to leave this stacktrace:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x26db73b]

goroutine 177 [running]:
github.com/cloudetc/awsweeper/vendor/github.com/terraform-providers/terraform-provider-aws/aws.resourceAwsKmsAliasRead(0xc420134000, 0x300f2a0, 0xc4200cca00, 0xc420134000, 0x0)
	/home/janck/go/src/github.com/cloudetc/awsweeper/vendor/github.com/terraform-providers/terraform-provider-aws/aws/resource_aws_kms_alias.go:122 +0x35b
github.com/cloudetc/awsweeper/vendor/github.com/hashicorp/terraform/helper/schema.(*Resource).Refresh(0xc4203d4540, 0xc42056a000, 0x300f2a0, 0xc4200cca00, 0xc42024d408, 0x1, 0x3286980)
	/home/janck/go/src/github.com/cloudetc/awsweeper/vendor/github.com/hashicorp/terraform/helper/schema/resource.go:354 +0x167
github.com/cloudetc/awsweeper/vendor/github.com/hashicorp/terraform/helper/schema.(*Provider).Refresh(0xc4204e6460, 0xc42070c2d0, 0xc42056a000, 0xa, 0xc4205aa088, 0x0)
	/home/janck/go/src/github.com/cloudetc/awsweeper/vendor/github.com/hashicorp/terraform/helper/schema/provider.go:308 +0x9a
github.com/cloudetc/awsweeper/command.(*Wipe).wipe.func1(0xc42063d800, 0xc420920480, 0xc42070c2d0, 0xc4206d9660, 0xc4204cec50)
	/home/janck/go/src/github.com/cloudetc/awsweeper/command/wipe.go:126 +0x26c
created by github.com/cloudetc/awsweeper/command.(*Wipe).wipe
	/home/janck/go/src/github.com/cloudetc/awsweeper/command/wipe.go:100 +0x210

We have some corporate governance stuff applied over my company's 100+ AWS accounts. We developers use Terraform, the corporate governance resources are created using CloudFormation.

I'm trying to nuke all security groups created by developers, while not touching the ones generated by our overlords.

aws_security_group:
  - tags
      "aws:cloudformation:logical-id": NOT(.*)

tags is supposed to be a map, and since the key has colons in it, I believe I'm supposed to wrap it in quotation marks so the parser doesn't freak out. But when I do this (0.5.0 release), I get the following error message:

failed to unmarshal config: config.yml error=yaml: line 67: mapping values are not allowed in this context

Is there a schema I can look at, are the docs incorrect, or is is because of my weird YAML key? This issue seems a little different from #45, but maybe I'm wrong and they're related?

It would be great if I could set a config for all supported resource types. An example config might be:

"*":
  - tags:
    environment: test

or

all:
  - tags:
    environment: test

Requesting addition of AWS Directory Service and AWS Workspaces as time permits.

Hi,
Using awsls I can retreive some IAM Roles, for example :

• AWSCodePipelineServiceRole-eu-west-1-code-pipe
• AWSReservedSSO_AWSManager_43EDDe333zzerqszee23
• AWSServiceRoleForSSO
• codebuild-as-code-build-service-role
• MyCustomAccountAccessRole

My goal, delete all IAM Roles except :

• those who start with "AWSReserved*"
• those who start with "AWSServiceRole*"
• the dedicated "MyCustomAccountAccessRole"

So awsweeper should delete :

• AWSCodePipelineServiceRole-eu-west-1-code-pipe
• codebuild-as-code-build-service-role

I try to build a filter.yml file to do this... but without success. How can I build a multi filtering on one attribute like "id" here.
Any help please ?

Thanks.

Weired stuff

For me, the tags filter is not working... I have multiple resources that have the tag environment: test. However, when I run the following config:

aws_ami:
  - tags:
      environment: test
aws_autoscaling_group:
  - tags:
      environment: test
aws_cloudformation_stack:
  - tags:
      environment: test
aws_ebs_snapshot:
  - tags:
      environment: test
aws_ebs_volume:
  - tags:
      environment: test
aws_efs_file_system:
  - tags:
      environment: test
aws_eip:
  - tags:
      environment: test
aws_elb:
  - tags:
      environment: test
aws_iam_group:
  - tags:
      environment: test
aws_iam_instance_profile:
  - tags:
      environment: test
aws_iam_policy:
  - tags:
      environment: test
aws_iam_role:
  - tags:
      environment: test
aws_iam_user:
  - tags:
      environment: test
aws_instance:
  - tags:
      environment: test
aws_internet_gateway:
  - tags:
      environment: test
aws_key_pair:
  - tags:
      environment: test
aws_kms_alias:
  - tags:
      environment: test
aws_kms_key:
  - tags:
      environment: test
aws_launch_configuration:
  - tags:
      environment: test
aws_nat_gateway:
  - tags:
      environment: test
aws_network_acl:
  - tags:
      environment: test
aws_network_interface:
  - tags:
      environment: test
aws_route53_zone:
  - tags:
      environment: test
aws_route_table:
  - tags:
      environment: test
aws_s3_bucket:
  - tags:
      environment: test
aws_security_group:
  - tags:
      environment: test
aws_subnet:
  - tags:
      environment: test
aws_vpc:
  - tags:
      environment: test
aws_vpc_endpoint:
  - tags:
      environment: test

To me it looks like awsweeper doesn't support filtering by tags for all the AWS resources that officially support tags (e.g. aws_iam_role) in Terraform. Is that possible?

Hello folks,

I just discovered awseeper and I'm wondering about the usage of --region option.

With no AWS related environment variables, I have something strange:

$ env | grep AWS
$ awsweeper --region eu-west-3 --dry-run --output yaml awsweeper.yml  | cksum
INFO[0001] This is a test run, nothing will be deleted! 
3240463751 18140
awsweeper --region eu-west-1 --dry-run --output yaml awsweeper.yml  | cksum
INFO[0001] This is a test run, nothing will be deleted! 
3240463751 18140

Both call list me the same resources, but if I set AWS_DEFAULT_REGION:

export AWS_DEFAULT_REGION=eu-west-3
awsweeper --dry-run --output yaml awsweeper.yml  | cksum          
INFO[0001] This is a test run, nothing will be deleted! 
3240463751 18140
export AWS_DEFAULT_REGION=eu-west-1
awsweeper --dry-run --output yaml awsweeper.yml  | cksum
INFO[0001] This is a test run, nothing will be deleted! 
3032715009 2173

My awsweeper.yml is just a list of resources without any filters, like:

head awsweeper.yml 
---
aws_instance:
aws_autoscaling_group:
aws_ebs_snapshot:
aws_ebs_volume:

Thanks for feedbacks!

Hi @jckuester , just let you know there may be a bug happened for aws_iam_user of #101
I confirmed all the iam users in my aws account have already tagged the Owner and the awsweeper policy with following:

aws_iam_user:
  - tags:
      NOT(Owner): .*

The result after applying this policy shows up it will filler out entire iam users which actually shouldn't be filler out.

For the other resources applying the same #101 pattern, they look good for now.

Feel free to let me know if anything miss-config from my side.

delete aws_ecs_snapshot failed with the following error:

error=failed to import resource: resource aws_ebs_snapshot doesn't support import

Hi @jckuester , recently I got the following error while cleanup ecs resource. the weird thing is current region doesn't have any ecs cluster, but it still happens each time.

	Type: aws_ecs_cluster
	Found: 1

		Id:		arn:aws:ecs:ap-southeast-2:xxxxxxxxxxxx:cluster/default

	---

   • TOTAL NUMBER OF RESOURCES THAT WOULD BE DELETED: 1
   • USER WILL NOT BE ASKED FOR CONFIRMATION (FORCE MODE)
   • STARTING TO DELETE RESOURCES
panic: can't use ElementIterator on null value

goroutine 194 [running]:
github.com/zclconf/go-cty/cty.Value.ElementIterator(0x5f98800, 0xc00048b450, 0x0, 0x0, 0x7, 0x0)
	/home/jan/go/pkg/mod/github.com/zclconf/[email protected]/cty/value_ops.go:1038 +0x11a
github.com/zclconf/go-cty/cty.Value.AsValueMap(0x5f98800, 0xc00048b450, 0x0, 0x0, 0x1)
	/home/jan/go/pkg/mod/github.com/zclconf/[email protected]/cty/value_ops.go:1243 +0xfe
github.com/jckuester/terradozer/pkg/provider.enableForceDestroyAttributes(0x5f98800, 0xc00048b450, 0x0, 0x0, 0x203000, 0x203000, 0x203000, 0xf4a6622f9a8800fb)
	/home/jan/go/pkg/mod/github.com/jckuester/[email protected]/pkg/provider/provider.go:236 +0xf7
github.com/jckuester/terradozer/pkg/provider.TerraformProvider.DestroyResource.func1(0x4c0da0)
	/home/jan/go/pkg/mod/github.com/jckuester/[email protected]/pkg/provider/provider.go:200 +0x86
github.com/hashicorp/terraform/helper/resource.Retry.func1(0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/jan/go/pkg/mod/github.com/hashicorp/[email protected]/helper/resource/wait.go:22 +0x5e
github.com/hashicorp/terraform/helper/resource.(*StateChangeConf).WaitForState.func1(0xc00926e720, 0xc0001d4540, 0xc000850600, 0xc007e240c0, 0xc0082242d0, 0xc0082242c8)
	/home/jan/go/pkg/mod/github.com/hashicorp/[email protected]/helper/resource/state.go:103 +0x298
created by github.com/hashicorp/terraform/helper/resource.(*StateChangeConf).WaitForState
	/home/jan/go/pkg/mod/github.com/hashicorp/[email protected]/helper/resource/state.go:80 +0x1bf

The awsweeper policy is following with latest version v.0.10.1

aws_ecs_cluster:
  - tags:
      NOT(Owner): .*

Could you help to take a look?

It would be nice if I could specify all available regions to be swept. An example might be:

$ awsweep --region all config.yml

I would like to filter by a relative date instead of an absolute one. An example might be:

aws_instance:
  - created:
      before: 5d

This would delete all resources that are older than 5 days at the time of running awsweeper. We could use the standard Go terms like 5d, 2h, 2m etc.

I'm using MacOS and I'm testing the tool with all.yml, I think there is a bug with the option: aws_kms_alias. If I comment this option #aws_kms_alias works well.

Regards!

---
Type: aws_kms_alias
Found: 14
    Id: alias/aws/acm
    Id: alias/aws/connect
    Id: alias/aws/dynamodb
    Id: alias/aws/ebs
    Id: alias/aws/elasticfilesystem
    Id: alias/aws/es
    Id: alias/aws/glue
    Id: alias/aws/kinesisvideo
    Id: alias/aws/rds
    Id: alias/aws/lambda
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x26db73b]
goroutine 619 [running]:
github.com/cloudetc/awsweeper/vendor/github.com/terraform-providers/terraform-provider-aws/aws.resourceAwsKmsAliasRead(0xc420298700, 0x300f2a0, 0xc4200b4a00, 0xc420298700, 0x0)
    /home/janck/go/src/github.com/cloudetc/awsweeper/vendor/github.com/terraform-providers/terraform-provider-aws/aws/resource_aws_kms_alias.go:122 +0x35b
github.com/cloudetc/awsweeper/vendor/github.com/hashicorp/terraform/helper/schema.(*Resource).Refresh(0xc42038ff10, 0xc420502640, 0x300f2a0, 0xc4200b4a00, 0xc420462390, 0x1, 0x3286980)
    /home/janck/go/src/github.com/cloudetc/awsweeper/vendor/github.com/hashicorp/terraform/helper/schema/resource.go:354 +0x167
github.com/cloudetc/awsweeper/vendor/github.com/hashicorp/terraform/helper/schema.(*Provider).Refresh(0xc42044b5e0, 0xc420502460, 0xc420502640, 0xa, 0xc420734ce8, 0x0)
    /home/janck/go/src/github.com/cloudetc/awsweeper/vendor/github.com/hashicorp/terraform/helper/schema/provider.go:308 +0x9a
github.com/cloudetc/awsweeper/command.(*Wipe).wipe.func1(0xc4205e34a0, 0xc42052acf0, 0xc420502460, 0xc4208876e0, 0xc420510cc0)
    /home/janck/go/src/github.com/cloudetc/awsweeper/command/wipe.go:126 +0x26c
created by github.com/cloudetc/awsweeper/command.(*Wipe).wipe
    /home/janck/go/src/github.com/cloudetc/awsweeper/command/wipe.go:100 +0x210

Was testing this running in an ec2 instance with admin iam role and it hangs:

./awsweeper --region us-east-1 --dry-run dev.yml 
INFO[0000] using region: us-east-1

Haven't had time to dig into why it would be permanently hanging here. Initial permissions tested were:

"ec2:Delete*",
"ec2:Describe*",
"ec2:Disassociate*",
"ec2:TerminateInstances"

Changed that to an admin role and still hangs. Any ideas?