Add cross-account download functionality about cloudsplaining HOT 9 CLOSED

I like the idea of having a standard "download org metadata" process. What about something like this?

• Consider doing aws organizations list-accounts and saving the metadata as a yml file
• Require that trust policies have been created in target accounts with the same target role name
• Have Cloudsplaining assumerole into all target accounts matching account alias regex matches and run the scan but in multi account mode

Idk, that's one way to do it. I do feel like that requires too much tweaking and it would be slow, whereas we could basically do the same by kicking off simultaneous Lambda executions and requiring one target role ARN parameter.

from cloudsplaining.

From what I've seen, most companies have these kinds of cross account roles named the same in each account

I've worked in shops that had 8 fat AWS accounts, 100, or here where we have thousands. Keeping all of those scenarios in mind - I do think that Lambdas is the most scalable way to execute a ton of scans in any number of accounts within a short period of time

from cloudsplaining.

Oh and for using this hypothetical Lambda, the results of the download command could be funneled into the scan command, save as json, push to S3 and purge that file from tmp

from cloudsplaining.

Okay so I've pushed a version to https://github.com/fruechel/cloudsplaining/tree/feature/add-org-download. It's a little different to what you described so I'd be interested in how we can make it work.

One thing I'd like to keep in mind is that I'd prefer if we could decouple the deployment method from the functionality. I'm not certain that I'd deploy this through Lambda in production. However, I do think that you could easily build it in such a way that we could use StepFunctions that spawn a Lambda for each of the jobs:

1. Lambda to pull down account list
2. Fan out to one function per account to pull down data
3. Process each file individually through scan
4. Push results into S3
5. Collect the list of resulting reports

As a side note, I'm experimenting with a few use cases that might perform different functionality which is why I'm so keen on the decoupling. Right now, I've just moved my existing code into this branch and reworked it to fit the way the code is written already.

Let me know what you think. Does it make sense to treat the case of having the org-download feature and the Lambda deployment option as two separate features?

from cloudsplaining.

We've also had a need to run Cloudsplaining for multiple accounts so I wrapped it in this script https://github.com/yegorski/multi-aws-account-cloudsplaining

I used a bit (ugly) HTML to connect the reports together in a custom index.html.

from cloudsplaining.

@yegorski that's great! I'd love to collaborate on improvements, if you are interested.

Internally, we have an SNS topic that you can pummel with account metadata to AssumeRole into target accounts and save the results to an S3 bucket. It runs as a Lambda function, and the SNS topic triggers Lambda runs - so we actually scan thousands of accounts in about 10 minutes :) I have been moving it over to AWS SAM but it's been on my backlog for a bit. Would you be interested in collaborating on this?

from cloudsplaining.

@yegorski and @fruechel - in the interest of multiaccount Lambda support as I mentioned above, we could enable that via a multi-account download+scan command.

from cloudsplaining.

@yegorski and @fruechel - I started some work on this in #172. It contains my initial work on this and it seems straightforward, but I have not tested it yet; I intend to this week.

Let me know if you can take a peek at it, I'd love to hear what you think.

from cloudsplaining.

I'd love to collaborate! Due to my schedule the best I can do is async collaboration. Looking at #172 now. How about we continue the conversation there?

from cloudsplaining.