I get the following issue when tyring to deploy the create-benchmark-rules.yaml cloudformation templtae:

Template validation error: Template may not exceed 460800 bytes in size.

Every time that we try to run this, it fails with the following error:-

The following resource(s) failed to create: [FunctionForVpcPeeringRouteTablesRule, FunctionForVpcFlowLogRule, FunctionForEvaluateCloudTrailBucketRule, SnsTopicForCloudWatchEvents, ConsoleLoginFailureCloudWatchMetric, FunctionForRoleForMfaOnUsersRule, FunctionForEvaluateConfigInAllRegionsRule, FunctionForEvaluateCloudTrailLogIntegrityRule, ConfigRuleForRequiredTags, UnauthorizedAttemptsCloudWatchFilter, ConfigRuleForIamPasswordPolicy, ConfigRuleForEncryptedVolumes, ConfigRuleForUnrestrictedPorts, IAMRootActivityCloudWatchMetric, BillingChangesCloudWatchFilter, FunctionForEvaluateRootAccountRule]. . Rollback requested by user.

We have tried running as root and as an admin.
Error persists.

We are running this from EU-WEST-1

Please advise.

3.8, 3.9, 3.13 Fail checks because of metric /alarm / subscription missing.

Create-benchmark-rules.yaml fails when deploying into a region that does not have Glacier (like Singapore)

Error:

S3BucketForGovernance "This region does not support transition to GLACIER"

Propose to fix using the following(would love to know a smarter way):

StorageClass:
!FindInMap [RegionMap, !Ref "AWS::Region", storage]

With a mapping:

Mappings:
RegionMap:
us-east-1:
storage: "GLACIER"
us-east-2:
storage: "GLACIER"
us-west-1:
storage: "GLACIER"
ca-central-1:
storage: "GLACIER"
us-west-2:
storage: "GLACIER"
eu-west-1:
storage: "GLACIER"
sa-east-1:
storage: "GLACIER"
ap-southeast-1:
storage: "STANDARD_IA"
ap-southeast-2:
storage: "GLACIER"
ap-northeast-1:
storage: "GLACIER"

If I create an IAM user called "root-account-mfa-device", then assign a virtual MFA to it. Now even my root user is configured a hardware MFA, the script will report "Root account not using hardware MFA". I believe this is becuase using string "mfa/root-account-mfa-device" to check hardware MFA for root user.

From what I can see after reviewing the CloudFormation template, it doesn't define a way of alerting on noncompliant aws-config rules. May I ask how you do this?

Hi All,

Does the create-benchmark-rules.yaml file have Lambda functions to delete any AWS resources/users/secrets if they are found to be incompatible with either the Level 1 or Level 2 CIS Security Benchmark rules? Please let me know about this when you get a chance.

We want to make sure that they won't be deleted and if there is an option to just report on what's not compatible with the mapped rules per the CIS framework.

thanks,
Anand.

The string search for "arn:aws:logs" is too restrictive and fails in gov-cloud. Consider using a regex or adding a second conditional for "arn:aws-us-gov:iam:"

Rule 1.21: "Ensure IAM instance roles are used for AWS resource access from instances, application code is not audited" is marked as scored in the script but the CIS PDF v 1.1.0 lists it as Not scored

https://github.com/awslabs/aws-security-benchmark/blob/1bcaa25e54d8d69558222fc120680c72d3c4804a/architecture/create-benchmark-rules.yaml#L1995

Same issue as cpollard had in dec.

I uploaded it to an S3 bucket and received the template size error message.

What am I doing wrong?

It is currently possible to see the json in the CW console.
Dumping the json in S3 will allow flexibility on the reporting mechanism.
Jon

After detecting an IAM user not compliant, if you delete it, the execution of the benchmark will fail. The root cause is that the cred report is cached for 4h. But multiple functions in the benchmark go through the cred report user list assuming that all the users exist.

I notice that check 1.24 searches only locally scoped policies. I agree that this seems to satisfy the intent of the requirement, which states:

Ensure IAM policies that allow full ":" administrative privileges are not created

(emphasis on the not created).

However, the audit step doesn't say anything about local scope, and if one didn't include local scope, this requirement would not be achievable as the admin managed policy cannot be deleted. At at minimum, it does seem like the admin policy shouldn't be attached for the requirement to be satisfied. This is currently skipped in the audit.

What are your thoughts?

1.18 Ensure IAM Master and IAM Manager roles are active does not seem to be checking anything. It is just returning a result of True.

I have metric filters and alarms for 3.6 and 3.7 but the report still fails them. My Metrics and filters are launched using the create-benchmark-rules.yaml

If you try to generate a report for a different (i.e. non-default) AWS profile using the the --profile command line option, like this…:

python ./aws-cis-foundation-benchmark-checklist.py --profile test

…you end up with a hybrid, multi-profile, Frankenstein's monster of a report.

Using the example above, the EC2-related sections of the resulting report would be based (correctly) on data from the [test] profile.

But no matter which profile you specify on the command line, the IAM- and S3-related sections of the report are always based (incorrectly) on data from the [default] profile.

Possible Explanation

I've tried digging into the code, and I think the problem is that the low-level service clients for IAM and S3 are created at the very beginning of the script, like so:

IAM_CLIENT = boto3.client('iam')
S3_CLIENT = boto3.client('s3')

But, since the script hasn't even gotten around parsing the command-line arguments yet, those two service clients are created using (and thus use) the [default] profile.

Then, much later on, the command-line arguments are parsed and the script tells boto3 to use the profile we specified on the command line as the new default:

    if not profile_name == "":
        try:
            boto3.setup_default_session(profile_name=profile_name)

…and then the EC2 service client is created using that new (correct) default profile:

    try:
        client = boto3.client('ec2')

If I manually specify the correct profile to use before the IAM and S3 service clients are created, like so:

boto3.setup_default_session(profile_name='test')
IAM_CLIENT = boto3.client('iam')
S3_CLIENT = boto3.client('s3')

…then the resulting report correctly reflects data from the [test] profile throughout.

I'm not a programmer, so that explanation could be absolute rubbish, but I've tested and retested and that's the only hypothesis I've been able to come up with that explains the behavior I'm seeing.

While trying to launch:

https://github.com/awslabs/aws-security-benchmark/blob/master/aws_cis_foundation_framework/aws-cis-foundation-benchmark-checklist-lambdarole.json

I get the following error:

Template validation error: Invalid template property or properties [Version, Statement]

Team,

I review the Cloudformation template code but unable to found code for audit appropriate subscribers to each SNS topic. Can you please let me know how the CloudWatch Rule DetectNetworkChangeEvents will audit undesired subscription.

3.15 | Ensure appropriate subscribers to each SNS topic.

Thanks,
Indrajeet Mourya

A link to the CIS Benchmarks document would be a better approach rather than including the PDF in this repository.

(https://d0.awsstatic.com/whitepapers/compliance/AWS_CIS_Foundations_Benchmark.pdf or https://www.cisecurity.org/benchmark/amazon_web_services/)

An error occurred (SignatureDoesNotMatch) when calling the ListBuckets operation: The request signature we calculated does not match the signature you provided. Check your key and signing method.

I wanted to make you aware of that. Maybe it makes sense to extend the Lambda for this purpose.

So it can be saved in a local HTML file instead of sending it to S3.

Anyone else having issues with html format output?

1: KeyError
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 2019, in lambda_handler
htmlReport = json2html(controls)
File "/var/task/lambda_function.py", line 1849, in json2html
if str(controlResult[n+1][x+1]['Result']) == "False":
KeyError: 1

While trying to deploy this cloudformation stack:

https://github.com/awslabs/aws-security-benchmark/blob/master/architecture/create-benchmark-rules.yaml

I get this error:

Template validation error: Template format error: Unresolved resource dependencies [FunctionForEvaluateKeyRotationRule] in the Resources block of the template

Hi there

We think we have everything correctly configured but the script always stops with the error detailed below. Can you help??

START RequestId: 89291a84-36f1-11e7-a7f4-1f479e181a6e Version: $LATEST
iterator should return strings, not bytes (did you open the file in text mode?): Error
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 2244, in lambda_handler
cred_report = get_cred_report()
File "/var/task/lambda_function.py", line 1943, in get_cred_report
for row in reader:
File "/var/lang/lib/python3.6/csv.py", line 111, in next
self.fieldnames

I deleted a CloudTrail that was marked as noncompliant.

I've re-evaluated several times over the last day. Last successful invocation is 2 days ago for the one deleted, but the others are being updated.

I have many service accounts that do not have console passwords. But their API key were created at the same time as the account. Can the report script be updated to exclude those accounts without console passwords?

"ControlId": "1.23",
"Description": "Do not setup access keys during initial user setup for all IAM users that have a console password",

There are some rules which are optional for certain projects/use-cases and may not be required to be deployed. Adding a condition for each config rule would allow users to de-select if that rule is not needed(true as default option). My team had a discussion on the same and would like to know if anyone has any thoughts on this feature/ change?
We are planning to implement this and would like to contribute a PR if the above makes sense.

I am opening this issue to understand if we still need this rule InstancesMustUseIamRoles in CIS Profile.
I know AWS changed the way how we attach the roles to the EC2 instances. Earlier we could not assign an IAM instance Role to EC2 instances after they were launched and were in running state and back then keeping this rule InstancesMustUseIamRoles helped avoiding the storage of AWS Credentials in the Server.

InstancesMustUseIamRoles Rule checks for EC2 instances which doesn't have any IAM Roles attached to it and marks the Instance as non-compliant if found without a role. But Many of the instances don't need the Roles and if needed can be attached later.
Any one here having any thoughts on this ?? Do we still need to keep this rule or disable this ?
Or Am I missing something ?

The script is failing with an error, seems like Osaka-local(ap-northeast-3) is not available to all the AWS Accounts.

Traceback (most recent call last):
File "aws_cis_foundation_framework.py", line 2403, in
lambda_handler("test", "test")
File "aws_cis_foundation_framework.py", line 2298, in lambda_handler
control2.append(control_2_5_ensure_config_all_regions(region_list))
File "aws_cis_foundation_framework.py", line 958, in control_2_5_ensure_config_all_regions
response = configClient.describe_configuration_recorder_status()
File "/Users/monparth/.virtualenvs/myproject/lib/python2.7/site-packages/botocore/client.py", line 314, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/Users/monparth/.virtualenvs/myproject/lib/python2.7/site-packages/botocore/client.py", line 599, in _make_api_call
operation_model, request_dict)
File "/Users/monparth/.virtualenvs/myproject/lib/python2.7/site-packages/botocore/endpoint.py", line 143, in make_request
return self._send_request(request_dict, operation_model)
File "/Users/monparth/.virtualenvs/myproject/lib/python2.7/site-packages/botocore/endpoint.py", line 172, in _send_request
success_response, exception):
File "/Users/monparth/.virtualenvs/myproject/lib/python2.7/site-packages/botocore/endpoint.py", line 265, in _needs_retry
caught_exception=caught_exception, request_dict=request_dict)
File "/Users/monparth/.virtualenvs/myproject/lib/python2.7/site-packages/botocore/hooks.py", line 227, in emit
return self._emit(event_name, kwargs)
File "/Users/monparth/.virtualenvs/myproject/lib/python2.7/site-packages/botocore/hooks.py", line 210, in _emit
response = handler(**kwargs)
File "/Users/monparth/.virtualenvs/myproject/lib/python2.7/site-packages/botocore/retryhandler.py", line 183, in call
if self._checker(attempts, response, caught_exception):
File "/Users/monparth/.virtualenvs/myproject/lib/python2.7/site-packages/botocore/retryhandler.py", line 251, in call
caught_exception)
File "/Users/monparth/.virtualenvs/myproject/lib/python2.7/site-packages/botocore/retryhandler.py", line 277, in _should_retry
return self._checker(attempt_number, response, caught_exception)
File "/Users/monparth/.virtualenvs/myproject/lib/python2.7/site-packages/botocore/retryhandler.py", line 317, in call
caught_exception)
File "/Users/monparth/.virtualenvs/myproject/lib/python2.7/site-packages/botocore/retryhandler.py", line 223, in call
attempt_number, caught_exception)
File "/Users/monparth/.virtualenvs/myproject/lib/python2.7/site-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
raise caught_exception
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://config.ap-northeast-3.amazonaws.com/"

"fixes #31"
I think by adding this check it will skip service users that don't need/have a password and therefore not need MFA.

# Only check MFA on User with passwords
 try:
     profile = iam.get_login_profile(UserName=user_name)
 except:
     print('No login profile exists for {}. This user will be skipped.'.format(user_name))
     return 'NOT_APPLICABLE'

 if len(mfa['MFADevices']) > 0:
     return 'COMPLIANT'
else:
     return 'NON_COMPLIANT'

Thanks for all the hard work to contribute this to the community! Unfortunately, it seems like the project may be abandoned. There haven't been any commits since 097ddf7 in April 2018 and there are a bunch of outstanding issues and PRs. I've just submitted a couple more. Do the original authors care to comment on the status of the project? If it is abandoned, I'll consider forking it and maintaining the fork going forward.

The links present on "CIS_Amazon_Web_Services_Foundations_Benchmark_v1.1.0.pdf"
is pointing to https://community.cisecurity.org/ but the current address is: https://www.cisecurity.org/communities

Unfortunately the CIS site is not doing the redirect to the right page

I'm getting an error while the benchmark function runs.

Could not connect to the endpoint URL: "https://config.ap-northeast-3.amazonaws.com/": EndpointConnectionError

Didn't know ap-northeast-3 region existed. From the CLI, did a ec2 describe-regions and low and behold, there it is:

{
"Regions": [
{
"Endpoint": "ec2.ap-south-1.amazonaws.com",
"RegionName": "ap-south-1"
},
{
"Endpoint": "ec2.eu-west-3.amazonaws.com",
"RegionName": "eu-west-3"
},
{
"Endpoint": "ec2.eu-west-2.amazonaws.com",
"RegionName": "eu-west-2"
},
{
"Endpoint": "ec2.eu-west-1.amazonaws.com",
"RegionName": "eu-west-1"
},
{
"Endpoint": "ec2.ap-northeast-3.amazonaws.com",
"RegionName": "ap-northeast-3"
},
{
"Endpoint": "ec2.ap-northeast-2.amazonaws.com",
"RegionName": "ap-northeast-2"
},
{
"Endpoint": "ec2.ap-northeast-1.amazonaws.com",
"RegionName": "ap-northeast-1"
},
{
"Endpoint": "ec2.sa-east-1.amazonaws.com",
"RegionName": "sa-east-1"
},
{
"Endpoint": "ec2.ca-central-1.amazonaws.com",
"RegionName": "ca-central-1"
},
{
"Endpoint": "ec2.ap-southeast-1.amazonaws.com",
"RegionName": "ap-southeast-1"
},
{
"Endpoint": "ec2.ap-southeast-2.amazonaws.com",
"RegionName": "ap-southeast-2"
},
{
"Endpoint": "ec2.eu-central-1.amazonaws.com",
"RegionName": "eu-central-1"
},
{
"Endpoint": "ec2.us-east-1.amazonaws.com",
"RegionName": "us-east-1"
},
{
"Endpoint": "ec2.us-east-2.amazonaws.com",
"RegionName": "us-east-2"
},
{
"Endpoint": "ec2.us-west-1.amazonaws.com",
"RegionName": "us-west-1"
},
{
"Endpoint": "ec2.us-west-2.amazonaws.com",
"RegionName": "us-west-2"
}
]
}

Hi,
I get the following error when trying to launch the CF Template for the aws-cis-foundation-benchmark-checklist-lambdarole.json file.

Any ideas on why the json file doesn't work?

Template validation error: Invalid template property or properties [Version, Statement]

Thanks,
Anand.

Hi,
only after adding "TerminateInstances" to eventName the rule worked.
has amazon changed the event name ?
{
"detail-type": [
"AWS API Call via CloudTrail"
],
"detail": {
"eventSource": [
"ec2.amazonaws.com"
],
"eventName": [
"TerminateInstance",
"TerminateInstances"
]
}
}

When running this script on an EC2 instance, I keep getting this error:

# python aws-cis-foundation-benchmark-checklist.py Traceback (most recent call last): File "aws-cis-foundation-benchmark-checklist.py", line 2403, in <module> lambda_handler("test", "test") File "aws-cis-foundation-benchmark-checklist.py", line 2260, in lambda_handler cred_report = get_cred_report() File "aws-cis-foundation-benchmark-checklist.py", line 1935, in get_cred_report while IAM_CLIENT.generate_credential_report()['State'] != "COMPLETE": File "/usr/local/lib/python2.7/site-packages/botocore/client.py", line 314, in _api_call return self._make_api_call(operation_name, kwargs) File "/usr/local/lib/python2.7/site-packages/botocore/client.py", line 612, in _make_api_call raise error_class(parsed_response, operation_name) botocore.exceptions.ClientError: An error occurred (Throttling) when calling the GenerateCredentialReport operation (reached max retries: 4): Rate exceeded

Any idea on how to resolve this?

Hey Guys,

first: wow, great work on all of this. Im already a big fan of your work regarding automation of AWS compliance.

Is there maybe a typo when it comes to evaluating root account MFA CIS 1.14? Is seems like you are evaluating an account to have an MFA for root by default and when you find one you evaluate to "false".

result = False
(...)
if "mfa/root-account-mfa-device" in str(pagedResult):
        #failReason = "Root account not using hardware MFA"
        result = False

I can submit a pull request if you like, im just quite new to github and still need to figure out how to do that :)

best regards,
Markus

I think the EvaluateRootAccount Lambda can't work like this.

https://github.com/awslabs/aws-security-benchmark/blob/e223c20539c6bcc992aa15daafca0fb386b09e80/architecture/create-benchmark-rules.yaml#L265-L269

list_virtual_mfa_devices() lists all virtual MFA devices. The code needs to check for those assigned to the root account in order to avoid false-positives.

The Rule 1.24 Lambda function is returning a false positive for policies containing "*" anywhere in the action.

The policy should be looking for an exact match, but it is matching if the field contains * even if it is something like "ec2:*".

2. 1 is a check for Ensure CloudTrail is enabled in all regions

If that check fails with No enabled multi region trails found

Should'nt the other checks in this category (2.2, 2.3, 2.4, 2.6, 2.7) be marked as fail too? If there are no CloudTrails defined, then the other checks should not pass, as they are checking for things like Ensure CloudTrail logs are encrypted at rest using KMS CMKs etc.

Is that accurate?

Hello,

The create-benchmark-rules.yaml template creates some checks as Metrics and some as Events. The ones created as Events do not seem to satisfy the CIS Benchmark requirements, at least as they are coded in the aws-cis-foundation-benchmark-checklist.py script.

So, my first question is, is having an Event instead of a Metric/Alarm for certain checks (3.8 and 3.9 are examples) actually violations of the Benchmark? If they are violations, is it possible to construct those as Metric/Alarm combinations instead of Events? If they are not violations, is there a way to modify the script to check for Events for these particular rules?

1.4 Ensure access keys are rotated every 90 days or less (Scored)

def control_1_4_rotated_keys(credreport):
# Look for unused credentails
for i in range(len(credreport)):
if credreport[i]['access_key_1_active'] == "true":
try:
delta = datetime.strptime(now, frm) - datetime.strptime(credreport[i]['access_key_1_last_rotated'], frm)
# Verify keys have rotated in the last 90 days
if delta.days > 90:
result = False
failReason = "Key rotation >90 days or not used since rotation"
offenders.append(str(credreport[i]['arn']) + ":unrotated key1")
except:
pass
try:
last_used_datetime = datetime.strptime(credreport[i]['access_key_1_last_used_date'], frm)
last_rotated_datetime = datetime.strptime(credreport[i]['access_key_1_last_rotated'], frm)
# Verify keys have been used since rotation.
if last_used_datetime < last_rotated_datetime:
result = False
failReason = "Key rotation >90 days or not used since rotation"
offenders.append(str(credreport[i]['arn']) + ":unused key1")
except:
pass
if credreport[i]['access_key_2_active'] == "true":
try:
delta = datetime.strptime(now, frm) - datetime.strptime(credreport[i]['access_key_2_last_rotated'], frm)
# Verify keys have rotated in the last 90 days
if delta.days > 90:
result = False
failReason = "Key rotation >90 days or not used since rotation"
offenders.append(str(credreport[i]['arn']) + ":unrotated key2")
except:
pass
try:
last_used_datetime = datetime.strptime(credreport[i]['access_key_2_last_used_date'], frm)
last_rotated_datetime = datetime.strptime(credreport[i]['access_key_2_last_rotated'], frm)
# Verify keys have been used since rotation.
if last_used_datetime < last_rotated_datetime:
result = False
failReason = "Key rotation >90 days or not used since rotation"
offenders.append(str(credreport[i]['arn']) + ":unused key2")
except:
pass
return {'Result': result, 'failReason': failReason, 'Offenders': offenders, 'ScoredControl': scored, 'Description': description, 'ControlId': control}

CIS 1.4 control is Ensure access keys are rotated every 90 days or less. I am wondering why are we comparing last used data with last rotated date.

We switched some of our IAM users from TOTP-based MFA to U2F-based MFA.

Now when we run this tool, we get the following errors for those IAM users:

ControlId    1.2
Description  Ensure multi-factor authentication (MFA) is enabled for all IAM users that have a console password
failReason   No MFA on users with password.

I've noticed that the evaluatios of 2.1 and 2.4 return a false positive.

They both use the same lambda function EvaluateCloudTrail. When checking to see if CloudTrail is enabled in all regions, this line seems to evaluate to true even when trail['IsMultiRegionTrail'] returns False.

is_multi_region = is_multi_region or trail['IsMultiRegionTrail']

https://github.com/awslabs/aws-security-benchmark/blob/097ddf7461745f684dab0ca00aa608c2047dbd80/architecture/create-benchmark-rules.yaml#L732

CIS 1.24 Ensure IAM policies that allow full "*:*" administrative privileges are not created

This expression should be 'Statement[?Effect == \'Allow\' && Action == \'*\' && Resource == \'*\']'. Currently it matches for example action ec2:Describe* on resource *.

What do you think?

CIS 3.2 Ensure a log metric filter and alarm exist for Management Console sign-in without MFA

This log metric filter is also matching on console login failures, specifically those where the username was invalid.

ex: "errorMessage": "No username found in supplied account"
I am submitting a PR to address that will exclude the response type of failure as this should only be a concern for successful logins.

Recently, I was involved in reviewing data platform in the context of security for large enterprises. I wish I known something like this repo -- definitely would like to use it in new future.

I've got few questions and would appreciate your prompt response --

In regards to the architecture, please correct me if I am wrong. Your code will create VPC. Would it create one of those compliant vpc architecture where you will have management & prod vpc. As you must be aware, all the quick start guide (PCI DSS, NIST) aws published followed this ..

Would your code also create the subnet topology - public private etc?

Would I be able to do the below that we recommend to client, if yes, then what would be the instructions -

1. Create proper vpc architecture
2. Apply all the CIS rules
3. Turn on monitoring and logs
4. Create/turn on alerts
5. Auto alerts and fix (if any rules are broken, it will be fixed and email will sent

Finally, if what I am saying is too comprehensive and not supported at all, any chance you would support/add these features soon?

Please advise.