mapbox / deprecated-patrol-rules-aws Goto Github PK

The splitOnComma function (see https://github.com/mapbox/crowsnest-public/blob/master/rules/assumeRole.js#L37-L39) will be re-used in multiple rules. It should be split into its own utils module with its own tests in order to avoid duplication.

Fill out the readme

• how to implement a function
• pointer to streambot and how it is used
• pointer to cfn-config
• how to build template node build.js

Convert blacklisted IAM resources rule to lambda

cc. @ianshward @zmully

Needed for streambot env.

Rules that check IAM for policies, roles, etc. should not only handle new events, but should check all existing policies on a schedule. This would catch events missed during a CW outage and resources created before IAM was actively monitored. This addition will also catch the "I'm going to fix it" ("oh sorry I forgot to fix it") type of process errors where the first alert on the event is immediately closed out, but then fix is never done.

Instead of a parameter being blacklistedResourcesArns in a rule config, the parameter could be Arns and then we have lambda-cfn auto prefix all parameters with the value of config.name. This helps avoid running into parameter name collisions.

cc @alulsh

right now it just says "Blacklisted role AdminRole assumed" but should say "Blacklisted role AdminRole assumed by jdoe"

SNS based rules will allow easy integration of non-AWS event streams (e.g. Zapier) while maintaining a consistent Crowsnest develop/build/deploy experience. I think the easiest way to integrate this will be to create a new empty rule class snsRule.

Requirements for an SNS rule:

• Preexisting SNS topic ARN (passed in as a template parameter)
• lambda permission resource for SNS to invoke it
• SNS subscription to the lambda (created by crowsnest-rules.js)

Since the SNS message is free form, I don't see any way to mimic the existing eventRule format, so the snsRule would act as simply as a flag to the rule builder and creation scripts. The message validation and parsing would be handled by the rule.fn.

I'm hesitant to have Crowsnest create and maintain the SNS topic, but I could see how it would be advantageous to have Crowsnest create and manage all AWS pieces of this security pipeline. The difficulty I think will be how rules are associated to topics. I can see scenario where all events from an external service end up in one topic, and we have multiple rules subscribed to the topic. @alulsh @ianshward any strong opinions as to whether or not SNS topics should be managed by the Crowsnest template?

Rough todo:

• snsRule: spec and format
• lambdaCfn.lambdaPermission: handle SNS invoke permissions
• lambdaCfn.role: requires SNS permissions?
• crowsnest-rule.js: subscribe lambdas to topics

For example here the statement may either be an array, or not. This causes TypeError: Object #<Object> has no method 'forEach'. Solution is to handle the case when not an array.

This also at least affects the disallowed resources rule https://github.com/mapbox/patrol-rules-aws/blob/master/rules/disallowedResources.js

Add a catchall SNS topic and subscription to which lambda functions can report

Trusted Adviser is able to report whether an AWS Access Key has been exposed / publicly leaked. The announcement is here

This could be added to the trusted adviser rule, or, as its own rule, depending on whichever makes sense based on what the API looks like.

Couldn't resist that title. We need a meta rule which checks everything about crowsnest is plugged together and working:

• There's a subscription to the alarm topic
• That subscription is the email address from the stack
• The subscription is confirmed
• CloudTrail is running
• Optionally send out a heart beat to third party (ie., Librato)

Looks like requestParameters will end up being "requestParameters":null when "errorCode":"AccessDenied" so we just need a case to handle this. Probably makes sense to alarm on this case as well.

cc @alulsh

A third class of rules, scheduled rules, will be useful for checks on a service that does not generate usable push events, as well as for periodic stateful checks.

Because there is no way to schedule a lambda via the API, either every scheduled rule will need manual intervention, or Crowsnest can create and maintain single timekeeper lambda which pushes messages to Crowsnest managed SNS topics. Rule lambdas would then be subscribed to the appropriate timekeeper SNS topic. The only manual setup would be when first creating a Crowsnest stack and setting the schedule on the timekeeper function. Since CWE schedules have a 5 min precision, the timekeeper could simply be set to run every 5 mins, and generate appropriate events. From looking at existing scheduled lambdas, the CWE scheduler appears to be fairly timely, so generating every 5, 15, 30 and hourly and daily queues should be reasonably robust.

Doh! This is possible via the API, though possibly undocumented. Create the CWE using ScheduleExpression instead of an event, then everything else is the same as an eventRule.

I think it would be good to only create unique schedules, since there is no point in creating an every 5 min CWE for every single scheduled rule. Have to noodle on that.

Examples of such rules:

• All users in Github organization/team have MFA enabled?
• Github SSH key lifetime/last rotation

Rough todo:

• master timekeeper lamdba (just another rule?)
• crowsnest-rule.js: check master timekeeper lambda is scheduled, create CWE */5 schedule, subscribe SNS topics to rule lambdas.
• lambda-cfn.js: create SNS topics
• scheduledRule spec and format

Convert cloudfront rule to lambda.

Here and in other rules, we ignore errors. This was added in order to ignore 403's since it means the API call was not successful and therefore never executed. However, there may be other error status codes that should not be ignored.

We could put an error handler function in lambda-cfn and rules could implement it, so that if we learn of new error codes that should be ignored, we can add them there.

CloudTrail now has Trusted Advisor checks https://aws.amazon.com/about-aws/whats-new/2016/11/trusted-advisor-enables-automated-actions-and-custom-notifications/

It looks like we could switch the Trusted Advisor rules to use CloudTrail events instead of API calls to Trusted Advisor.

We may want a reusable way to template messages, so that they look good in PagerDuty. Thinking longish list of items that may have been found in violation of a rule, like a list of security groups and the instances they're attached to... We could balance this with an option to link off to a Sumologic search.

Figure out whether to use a custom lambda function, or direct API calls, to create the CloudWatch Events Rules.

2016-02-23T19:00:29.385Z b4a30993-da5f-11e5-8fd2-374b32524e79 TypeError: Cannot read property 'roleArn' of null at LambdaContext.module.exports.fn (/var/task/rules/assumeRole.js:32:54) at Response.dynamodb.getItem.on.res.error.retryable (/var/task/node_modules/streambot/index.js:57:15) at Request. (/var/runtime/node_modules/aws-sdk/lib/request.js:354:18) at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20) at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10) at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:596:14) at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:21:10) at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12) at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10 at Request. (/var/runtime/node_modules/aws-sdk/lib/request.js:37:9)

Pretty sure this was because I used a non existant role name

I'm not sure if this a recent update to TA, but the service limit check needs to be refreshed before requesting the output of the check, else the rule will trigger until someone goes into the console and manually refreshes the check.

Because the refresh can take several minutes this be get interesting.

Build and compile steps need some tests

The idea is to get an alarm before an SSL certificate in IAM expires. For some certificates I would want an alarm a month ahead of time, and others I would want a year ahead of time. The year use-case is for certificates that are pinned in mobile applications, and require updating and getting into new versions of mobile software so that clients update over the course of a year.

aws iam list-server-certificates

This will list certificates in IAM, along with an expiration time. Therefore, this rule will not be event-based. It will be schedule based, and could run once a day.

One question is how to support custom expiration times per-certificate, since the certificate list is going to vary per each person's / company's patrol installation. Maybe it could be a single parameter, in a format like:

certificate-name;1209600,other-certificate-name;31536000

where 1209600 is at how many seconds before expiration you want to be notified.

Right now only an Errors and NoInvocations alarm are defined for all lambdas. It will be easy to let a rule define their own alarms by specifying some properties the rule's in module.exports.config.

blacklisted_resources.js --> blacklistedResources.js, etc. etc.

@alulsh @zmully per talk today, we'll:

• Rename crowsnest-public to crowsnest-rules-aws, which will just contain the AWS related rules and their tests.
• Create a new repo called crowsnest, which will contain:
  • a cloudformation directory and template
  • package.json
  • documentation on what crowsnest is, how to use and extend it, and link to:
    • other public repos. containing rules
    • lambda-cfn and the rules spec.
  • the cloudformation template will include rules from crowsnest-rules-aws and maybe crowsnest-rules-github
• One more repo which we have and will use as our own internal version of this crowsnest repo and which will stay private, which is where we'll handle any necessary private issue tracking, and manage our own cloudformation template with the rules - coming from public repositories - which we want to activate and run on our service.
• lambda-cfn will contain docs on how to use lambda-cfn, how to write a rule

On the blacklisted resources rule:

2016-02-22T05:23:58.912Z 79a4516b-d924-11e5-8af8-d74844b07e29 TypeError: Cannot read property 'policyDocument' of null at LambdaContext.module.exports.fn (/var/task/rules/blacklisted_resources.js:41:48) at Response.dynamodb.getItem.on.res.error.retryable (/var/task/node_modules/streambot/index.js:57:15) at Request. (/var/runtime/node_modules/aws-sdk/lib/request.js:354:18) at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20) at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10) at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:596:14) at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:21:10) at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12) at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10 at Request. (/var/runtime/node_modules/aws-sdk/lib/request.js:37:9)

SNS message subject is limited to 100 characters. To get around this for now, i've done a7a97c5

We may not be able to put back the detail in the subject, but the tests should be improved to check that the correct resources were matched in the message body.

Currently hardcoded to 60sec and 128MB lambda. These should be configurable within the rule config.

Why is this 'undefined' maybe someone who was assuming a role assumed a role.

Convert blacklisted IAM actions rule to lambda. I'm thinking to change up the behavior, so that it is like:

• Must pass explicit list of blacklisted resources, so, not iam:*, but, iam:foo, iam:bar, iam:etc.
• If a policy is created which has iam:*, then since iam:foo was passed in as a parameter, this will match iam:*
• As for actions on particular resources, this should be dealt with in #13

The CloudFront rule (https://github.com/mapbox/crowsnest-public/blob/master/rules/cloudfrontModifyDelete.js) should output the entire event JSON, not just what action was taken on what protected distribution.

It's a industry-wide practice to send an email to a user when a login is made from a unknown device.

A PoC would consist of a lambda function that parses console.login events and matches the user-agent against a database.

Additional parameters for notifying a slack channel or super-admins group could be useful too.

cc: @ianshward @zmully

As @k-mahoney suggest, It would be great to update the .eslintrc file to

module.exports = {
  'env': {
    'node': true,
    'es6': true
  },
  'rules': {
    'quotes': [2, 'single', {'avoidEscape': true, 'allowTemplateLiterals': true}],
    'indent': ['error', 2],
    'no-multi-spaces': [2],
    'no-unused-vars': [1],
    'no-mixed-spaces-and-tabs': [2],
    'no-underscore-dangle': [2],
    'no-loop-func': [2],
    'handle-callback-err': [2],
    'space-unary-ops': [2],
    'space-in-parens': ['error', 'never'],
    'space-before-function-paren': ['error', 'never'],
    'no-trailing-spaces': [2],
    'no-tabs': [2],
    'new-cap': [2],
    'block-spacing': ['error', 'always'],
    'no-multiple-empty-lines': [2],
    'semi': ['error', 'always']
  }
};

There's a WIP of this here https://github.com/mapbox/crowsnest-public/tree/lambda-perms the problem is that the event source mappings still don't show up on listing page on each lambda function. Need to play around w/ the value of the SourceArn, which seems to be the main diff.

http://docs.aws.amazon.com/sns/latest/api/API_Publish.html this perm, to just the particular topic resource defined by the stack.

Patrol should alarm on bucket creation and deletion in AWS S3.

/cc @arunasank @aarthykc

Notify user and a sec group, if my account signs in far away from my last login (e.g. different country, city, 500KM away from home). This could be a sign that an account has been compromised. Potential false positives for users using a VPN, but IMO this would be a small number of users.

This ticket share some ideas: #79

The crowsnest-rules.js needs a --delete option to remove the ruleset if the main Crowsnest stack is deleted.

Provide a generic interface for state storage:
Possible backends:

• DynamoDB
• SimpleDB
• S3 files

Possible state usage patterns:

• compare and insert: if new state != oldstate create new record with new state
• compare and update: if new state != oldstate, upsert
• state ring buffer: only keep X recent states
• state change ring buffer: only keep X recent state changes