Running 'awless list buckets' shows ~1-2 buckets out of a total of ~50 in our account. There doesn't seem to be a pattern to this.

We want initial support for local policies, to be improved later.

Consider preloading the global AWS policies, and try to minimize requests.

I work with many AWS accounts everyday and exporting my AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY every time I is not plausible. Adding --profile NAME support would make this tool usable.

A CLI tool (interactive and/or line-based) that can store parameters into a database, including the keys to connect to an AWS account.

Implement awless configure to configure the CLI.

We should prepare standard templates to add EC2 instances to an infrastructure. We should take in account definitions from #3 to help generate the instances easily.

Hi,
Encountered with such error whenever I'd like to retrieve my instance info.

awless show i-xxxxxxxxxxxx

[info] cannot resolve resource - running full sync
Error: BucketRegionError: incorrect region, the bucket is not in 'ap-southeast-1' region
status code: 301, request id:

Each role will map to a policy, but for awless, we use a simple name tag to dedicate a particular role to a given group of instances.

Maybe it will be a product of
role_type x role_perimeter

Syncing infra failed when a route targets to a NetworkInterface and Instance.

ex:

[info] cannot resolve resource - running full sync
Error: extract values: multiple non-empty target type in route {
  DestinationCidrBlock: "0.0.0.0/0",
  InstanceId: "i-8fxxxxxx",
  InstanceOwnerId: "xxxxxxxxxx",
  NetworkInterfaceId: "eni-a5xxxxxx",
  Origin: "CreateRoute",
  State: "active"
}

When there are many users, it might be interesting to know the ones that are unused in order to delete them.
Perhaps add also list instances/vpcs/groups/roles/... --unused

1. With awless diff :

• an instance created in the cloud is shown in red (the local repo has one missing instance)
• an instance deleted in the cloud is shown in green (the local repo has an extra instance)

2. After awless sync
awless show revisions -n 1 displays :

• the instance that has been created in green
• the instance that has been deleted in red

We have to choose one between 1. and 2.
For my part, I prefer 2.

awless should integrate with configuration management software. For instance, we could easily start an ansible "central" server and automatically add newly created instances to the ansible hosts.

We should not timeout when opening BoltDB if another Awless process is hanging open.

For instance in a console make awless hang on a command:

$ awless create instance

Then in another console window run:

$ awless list instances

You will get a timeout error on opening the awless db.

There are no writes on the db in both case so we should be able to execute both commands concurrently.

Would be great to be able to override the region set in the config with a command line flag. In addition to specific regions, the ability to list resources in all regions would be very useful. Maybe add a new column to the output in that case with the region name.

Great project !
Would love to have a fish shell completion. Wishlist

Start from documentation already available in wiki and README.md.

After brew update, just to be sure, brew install awless returns No available formula with the term "awless".

MacOS latest, homebrew latest...

Tried on two separate development Macs.

We want to export a current infrastructure to a git repository. git will enable to automatically generate diffs for the infrastructure and get back in time.

Probably will make use of Git to Go

For example:
awless list instances --id i-0413908eb542d8102

to see all the attributes of that instance include its EC2 tags.

As mentioned in #36, it would be nice to be able to change config parameters (for example, the region, or the aws profile) only for one command.

We could do that with, for example (git syntax)

awless -c "aws.profile=prod" -c "region=us-west-2" list instances 

We are not responsible of what happen to the users' cloud and the AWS costs through awless.

awless completion zsh is executed when launching a new terminal session.

We should have a flag that disables the collection of statistics, and explain this in the README.

And start the library of templates in awless-templates. This repository is already open.

Users should be able to use their own key to login via ssh to a running instance. Auto lookup could be used.

awless ssh foomatty@i-fr483984
ssh login to instance i-fr483984 using key /home/foomatty/.awless/keys/foomatty

If there are too many resources, they may be sent by bunches, which is currently not managed in awless.

Earlier:

awless config os linux

(in which linux default to Amazon Linux). We could also expect to define here a preferred instance type.

awless create instance ram=4gb
-> “instance_foo”

awless rename instance_foo gogs

awless ssh gogs

To do this, it means:

1. That we have notions of "defaults" which are predefined by the user; For instance, the preferred AMI. There should a "pass" in awless that inserts default values when they are missing.
2. There should be a pass which automatically selects the instance type based on constraints (for instance, the cheapest instance with a given amount of memory)
3. There should an "alias pass" which replaces instance names with their IDs. In the end, the engine which applies the commands in stacks (and which takes place after aliasing) expects only IDs.
4. There should be a ssh command which logins into the instance. The ssh keys are taken from Amazon output and stored in awless safe.

hi,

my .aws/credentials file describes many different accounts. Currently I use aws --profile but don't see such option in awless - am I missing something ?

awless should have a way to estimate the cost of a template, using the price list API

We should prepare the core definitions that will be used later for the generation of configuration elements.

I'm trying to use awless from a machine that has valid (temporary) credentials on the standard 169.254.169.254 endpoint and it complains:

command hook failed: Your AWS credentials seem undefined!
AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY need to be exported in your CLI environment

My expectation from all other AWS tools and SDKs is that it would respect explicitly provided credentials if provided, but would fall back on the instance metadata credentials if those weren't provided.

We can strip the executables using: -ldflags "-s -w" when building awless.

Additionally, some people use upx to further shrink them.
I tested (without goupx which is supposedly not needed) but the (much) reduced awless binary does not run anymore (it quits with code 127).

Support to list and update queues and topics would be a big help!

Once configured, awless should be able to connect to an AWS infrastructure and automatically grab the configuration of most common elements, with EC2 and S3 to start with.

awless create instance ami=ami-0b33d91d subnet=subnet-1b6f2240

verified operations => create instance image=ami-9398d3e0 type=t2.nano count=1 subnet=subnet-1b6f2240 ami=ami-0b33d91d

Like many people, I manage a number of AWS accounts. Is there a way to have awless use a profile, as the official AWS CLI does?

For example, given I have a typical setup, development in one account, production in another, with AWS CLI I have this.

~/.aws/credentials

[default]
aws_access_key_id = obfuscated
aws_secret_access_key = obfuscated

[DEV]
aws_access_key_id = obfuscated
aws_secret_access_key = obfuscated

[PROD]
aws_access_key_id = obfuscated
aws_secret_access_key = obfuscated

aws s3 ls --profile DEV

I can easily work around exporting the keys to the appropriate env variables, so this isn't a huge deal but it would make things work way better.

Thanks!

~bdunbar

We should integrate awless with serverless frameworks, especially serverless

We want to implement a stack of unrealized commands.

• Each command has a given number of arguments.
• Results of previously stacked commands are named res1, res2, etc.
• Results could be a single value or an array
• Results can be referenced as arguments of further commands
• Stacks as a whole can be shared as templates where some values may be missing: They are the arguments of the template
• We should use templates as a way to tackle the awless scenarios

awless syncs automatically cloud resources (across services) locally as RDF graphs.

At the moment the auto-sync run as such:

• pre and post awless run command
• pre and post awless create, delete, ... (template one-liners) commands
• pre awless show command

Note that a manual sync (per service if chosen) can be done via awless sync.

Although the sync fetches in parallel, we might want to ignore some services and therefore not automatically resolve their resources.

Examples:

• You might want to work and administrate solely your infra, access and queuing services while ignoring huge amount of resources on your storage service.
• You might not have rights to access every cloud service, hence triggering some errors while fetching resources.

In the config we will flush by default:

infra.autosync = true
storage.autosync = true
access.autosync = true
...

Disabling an auto-sync is therefore now just awless config set storage.autosync false

When we launch for the first time awless without any AWS credentials (env variables nor ~/.aws/credentials file), awless freezes and after few seconds displays:

2017/01/02 15:28:27 Your AWS credentials seem undefined: NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors

We could rather propose to create the ~/.aws/credentials file.

Hi. When having a large number of resources, I find awless is for me hard to use. I'd like some ability to filter results. Typically I use tags to identify resources, and would want to filter for resources who have a tag that matches some kind of value.

Over on Hacker News, @fxaguessy wrote:

The hash functions are totally unrevertable, so it is impossible to come back to the original identifiers.

I replied:

You don't need to break SHA256 to de-anonymize these values.

awless collects account number hashes. AWS account numbers are 12 decimal digits long, meaning there's a total of 10^12 unique values. Values are anonymized before submission using a single round of SHA256, so in ~2^40 hash operations, anyone with your database of hashes can invert every single account number.

For comparison, the bitcoin blockchain presently has a hash rate of ~2^61 SHA256 hashes per second. (Edit: I incorrectly stated 2^41 based on a hash rate of 3 TH/s, when it's actually 3 million TH/s.)

"SHA256 with a global salt" is not appropriate for password storage because it is trivial to brute force. It is equally inappropriate for the anonymization of user identifiers, and for exactly the same reason.

However the statistics reporting works going forward (#38), awless should not continue using this or any other similar anonymization scheme.

With an awless fresh install, if you have an invalid region (for example, with a Ctrl-C during the process). Then, you can not launch any command:

invalid region ''

We need to make the initialization of the environment after the launch of the CLI, in order to have access to commands such as awless version, awless config set ...