assume_role function found in opinel/utils/credentials.py does not support DurationSeconds parameter, unlike the init_sts_session parameter. This creates some issues when using Scout2 to scan large AWS accounts, as the default duration for the credentials using assume_role is only 1 hour, which may not be sufficient for scanning of a large AWS account. Suggest that assume_role should, like init_sts_session, take in a parameter for duration with a large default value.

Currently, If you specify an mfa-code on the Scout2 command line, it will be ignored without any warnings or errors.

In Scout2, __main.py__ calls read_creds from opinel/utils/credentials.py

In opinel/utils/credentials.py, the read_creds function only does an init_sts_session with the mfa-code if the force_init bool is set to True, but that never happens if your ~/.aws/credentials file doesn't contain a SessionToken already. If the session token doesn't exist, it should probably set force_init to True.

Hi,

I'm using Scout2 from an AWS instance to scan both the local AWS account and other AWS accounts through the STS assumeRole API call. When setting the environment variables to access a remote AWS account it is still scanning the local account because of the search order in the read_creds() function in utils.py.

The current search order is: csv_file, instance_metadata and then environment variables. A more logical search order would be: csv_file, environment variables, instance_metadata.

Mike

The file opinel/opinel/utils/credentials.py line 77, 78 and 79 is not looking for the correct variable to load the token code and serial number to be passed to the assumeRole Function.

This result in ScoutSuite failing to scan AWS environments when assume role and MFA are required.

The current code looks like this:
if 'mfa_serial' in credentials and 'mfa_code' in credentials: sts_args['TokenCode'] = credentials['mfa_code'] sts_args['SerialNumber'] = credentials['mfa_serial']

The correct code and solution for this problem(which I tested and found to be working properly)is to replace the code above with the following:
if 'TokenCode' in credentials and 'TokenCode' in credentials: sts_args['TokenCode'] = credentials['TokenCode'] sts_args['SerialNumber'] = credentials['TokenCode']

Reported in nccgroup/Scout2#136

Ref: https://github.com/nccgroup/Scout2/issues/237

Spin off of nccgroup/Scout2#177

If no expiration information is available and no profile exists to renew the session, read_creds() fails currently

Currently opinel looks for the custom configuration variable aws_mfa_serial in the ~/.aws/credentials file to get the arn of the MFA device.

For instance, my current ~/.aws/credentials file might look like this:

[development]
aws_access_key_id=foo
aws_secret_access_key=bar
aws_mfa_serial=arn:aws:iam::XXXXXXXXXXXX:mfa/john.doe

However, the AWS/boto3 configuration variables have changed a bit since opinel was first introduced. When assuming roles, boto3 now looks for the arn of the MFA device via the mfa_serial variable in the `~/.aws/config file.

It would be nice if opinel used the mfa_serial variable in the ~/.aws/config file, too, both for assuming roles and other standard operations.

So a typical configuration might look something like this…

In ~/.aws/credentials:

[development]
aws_access_key_id=foo
aws_secret_access_key=bar

In ~/.aws/config:

[profile development]
mfa_serial=arn:aws:iam::XXXXXXXXXXXX:mfa/john.doe
region=us-east-1
output=json


[profile crossaccount]
source_profile=development
role_arn=arn:aws:iam::XXXXXXXXXXXX:role/whatever
mfa_serial=arn:aws:iam::XXXXXXXXXXXX:mfa/john.doe
region=us-east-1
output=json

To avoid hitting expired session errors, automatically prompt for an MFA code and re-initiate an STS session when a session token has expired

Many of the scripts that rely on opinel are currently failing unless they are run from the directory where the scripts reside. Otherwise, you get something along the lines of…

Traceback (most recent call last):
  File "/Users/xxxxxxx/Code/AWS-recipes/Python/awsrecipes_init_sts_session.py", line 40, in <module>
    sys.exit(main())
  File "/Users/xxxxxxx/Code/AWS-recipes/Python/awsrecipes_init_sts_session.py", line 31, in main
    if not check_requirements(os.path.realpath(__file__)):
  File "/usr/local/lib/python2.7/site-packages/opinel/utils/globals.py", line 35, in check_requirements
    with open(requirements_file, 'rt') as f:
IOError: [Errno 2] No such file or directory: 'requirements.txt'

I think the reason is that opinel purports to look for the requirements file in two places:

• "next to the script"
• "in data/requirements"

Instead, opinel appears to be looking in:

• Current working directory (Incorrect)
• "in data/requirements" (Correct)

My Python skills are rudimentary (at best), but I believe that line #34 in opinel/utils/globals.py, which currently ends with:

...else 'requirements.txt'

...should end with:

...else os.path.join(script_dir, 'requirements.txt')

I get a ValueError: ('Unknown string format:', '2018-12-04 00:07:02 +0000 UTC') exception in Opinel expiration = dateutil.parser.parse(credentials['Expiration']).

The problem is the string having UTC at the end, if I modify the source to remove this from the string it seems to parse correctly.

While using opinel with scout2 or ScoutSuite with crossaccount MFA access, STS assume role failing.
After investigation I found that the root cause is incorrect naming inside assume_role function credentials argument TokenCode and SerialNumber:
- if 'mfa_serial' in credentials and 'mfa_code' in credentials:
- sts_args['TokenCode'] = credentials['mfa_code']
- sts_args['SerialNumber'] = credentials['mfa_serial']
+ if 'SerialNumber' in credentials and 'TokenCode' in credentials:
+ sts_args['TokenCode'] = credentials['TokenCode']
+ sts_args['SerialNumber'] = credentials['SerialNumber']

I was trying to use Scout2 and was wondering why some of the newer regions weren't listed. It appears that boto-endpoints.json isn't quite up to date, and I had to update the list using http://docs.aws.amazon.com/general/latest/gr/rande.html

Just thought I'd give a heads up.

As reported in nccgroup/Scout2#180

It would be great if the aws_config_dir variable could be overridden by an environmental variable. The awscli library supports reading from the AWS_CONFIG_FILE environmental variable for setups where the config file may not live in ~/.aws/config.

Similar issue applies for the temp file.

opinel 1.0x seems to have introduced a (roughly) 75-second delay to any anything making calls to the AWS SDK, whether running one of the AWS-recipes Python scripts or running Scout2.

For instance, when running aws_recipes_assume_role.py, there's a 75-second delay before results are returned and the credentials are cached in the ~/.aws/cli/cache directory.

When running aws_recipes_init_sts_session.py, there's a 75-second delay before it prompts for the MFA code.

When running aws_recipes_create_ip_ranges.py it says "Fetching public IP information for the 'xxxxxxx' environment..." and then pauses for 75 seconds before it starts iterating through the various regions and displaying "...in us-east-1: EC2 instances", etc.

After that initial 75-second delay, everything runs as expected. (For instance, there's not an additional 75-second delay for each step of the Scout2 report generation process, only at the very beginning.)

And running the scripts or Scout2 with the --debug option returns no additional information. The scripts are succeeding; they just take 75 seconds longer than expected.

Hi,

The logic to get STS tokens if you're running in ECS is not quite the same as the logic that boto3 appears to use. In boto3, we don't notice the IAM creds used by ECS tasks to leverage the EC2 role arn, unless the ECS Task has no Role defined ( which we also don't do anymore, but we once did, and found this to be the case maybe 2ish years back ).

The condition by which ECS credentials are attempted to be gathered is dependent on EC2 credential data not being found.

We run with EC2 IAM Profiles/Roles, and ECS Task IAM Profiles/Roles. Scout2 works well for us, until we deployed it as a ECS task.

this code here

As a user, I would like for scout2 to be able to assume the role assigned to an ECS Task, while also being able to have a distinct role assigned to the underlying EC2 instance in my ECS Cluster.

cc/ @preston-bejabeng

Walking through the workflow, it seems somewhat troublesome.

status quo workflow

Here's what is in what cred file, when. From a clean slate, start by aws configure.

Now credentials contains

[default]
aws_access_key_id = AKIA123121123123123123
aws_secret_access_key = 12312312312312312312312312312313123

Now, aws_iam_enable_mfa.py, go through the prompts. credentials is unchanged (still has the AKIA.... key) and credentials.no-mfa has these contents:

[default]
aws_access_key_id = AKIA123121123123123123
aws_secret_access_key = 12312312312312312312312312312313123
aws_mfa_serial = arn:aws:iam::987654321:mfa/UserName

Now, we need to aws_recipes_init_sts_session.py. After that, credentials.no-mfa is unchanged, and credentials contains:

[default]
aws_access_key_id = ASIA8768768768768
aws_secret_access_key = 876876876876876876876876
aws_session_token = 8768768768768.......................876876876876

Now we can use awscli commands and it will use credentials. But, consider the case where some organizations only require MFA for certain API actions. The original credentials have been sequestered away in credentials.no-mfa, and to perform actions that they would ordinarily be allowed to do, they have to either re-MFA or else fetch them from credentials.no-mfa somehow.

But, why do any of this?

We could just keep all the bookkeeping in the same credentials file, under a new default-MFA profile. Under that model, the flow would go like this:

From a clean slate, aws configure, then aws_iam_enable_mfa.py. Now credentials is:

[default]
aws_access_key_id = AKIA123121123123123123
aws_secret_access_key = 12312312312312312312312312312313123
aws_mfa_serial = arn:aws:iam::987654321:mfa/UserName

Then after aws_recipes_init_sts_session.py:

[default]
aws_access_key_id = AKIA123121123123123123
aws_secret_access_key = 12312312312312312312312312312313123
aws_mfa_serial = arn:aws:iam::987654321:mfa/UserName
[default-MFA]
aws_access_key_id = ASIA8768768768768
aws_secret_access_key = 876876876876876876876876
aws_session_token = 8768768768768.......................876876876876

So then the user can use awscli 's --profile option to pick whether they want to use their MFA/ASIA... creds or their non-MFA/AKIA... creds.

I just verified that aws configure seems to pass through lines that it doesn't know about. I put a line

some_garbage = osidjfsoidjfoij

and it survived a re-run of aws configure, as did the aws_mfa_serial line belonging to the default profile.

Alternatively, the final state of the credentials file could be similar but with profiles default-NO-MFA and default (which contains the MFA creds). That's probably more likely to be useful, since people who are setting this up are probably going to want to prefer their MFA creds but be able to fall back to non-MFA creds.

If you think this is reasonable, I can take a stab at it.

I have really only been paying attention to the MFA-related things in AWSUtils, so if there are other reasons to use credentials.no-mfa let me know and I'll see if I can think of a workaround.

@l01cd3v thanks for looking

As reported in nccgroup/Scout2#145

Similar in spirit to #25, I would like to be able to pass in role_arn and external_id as environment variables.

The use case is calling scout2 from another process. It is a bit hacky to rewrite configs on the fly (which is how we have been doing it).

I plan to write the code and submit a PR myself if this is ok - are there any steps I need to take first? Or shall I just go ahead and submit the PR?