amazon-archives / aws-waf-sample Goto Github PK

I have a question around this solution in terms of separation of endpoints, s3 buckets and WAF rules.

Example:
If I have 2 separate domain names and 2 separate clouldfront instance that share the same WAF Name/ rules and s3 bucket, does that mean if someone is hitting api.dev.abc.com hard and it triggers the lambda function to block the IP, would this also affect all domains sharing the same WAF rules and s3 log bucket? Assuming I didn't want this behaviour would I have to create an s3 access log bucket for each domain and separate WAF rules for each or just a separate WAF rule for each domain and attach the different WAF name/rule to each cloudfront instance?

api.dev.abc.com --> cloudfront (WAF named WAF1) --> s3 (bucket name: cloudfront-access-logs) --> lambda
 
api.qa.abc.com --> cloudfront (WAF named WAF1) --> s3 (bucket name: cloudfront-access-logs) --> lambda

I noticed that if I configure the same bucket name, that inside the bucket it creates separate folders based on the domain name (which is good), but is the lambda function using teh domain name to determine which endpoint to disable? How does it know which WAF name to update?

Following code is creating problem in waf-reactive-blacklist:
"Code": {
"S3Bucket": {"Fn::Join": [".", [{ "Ref" : "AWS::Region" },"heitorc"]]},
"S3Key": "waf-reactive-blacklist/parser.zip"
},
The above code in cloudformation results in BucketNotFound error while I tested with python api and was able to fetch file.

import boto3
s3 = boto3.resource('s3')
obj = s3.Object('heitorc','waf-reactive-blacklist/parser.zip')
obj.get()['Body'].read().decode('utf-8')

So the bucket name formed is somehow wrong in cloudformation and needs to be updated correctly.

172.16.0.0/16 should be 172.16.0.0/12 to cover the full RFC1918 space

*edit, after trying to edit /16 to /12 in WAF it throws a "Not a valid CIDR format." error so there is a bug in WAF that appears not to allow masks from /9-/15 and /0-/7. The only way to cover 172.16.0.0/12 is to use individual /16s for 172.16-31.0.0. I created a patch version with this work around and after looking I did notice the "IP match condition" states that "AWS WAF supports /8 or any range from /16 to /32 CIDR blocks for IPv4" so I guess this is expected behavior even so the work around is actually the way.

Hi,

Is it possible to allow a block of ip addresses access even if they go over the maximum request amount? Kind of like a whitelist?

Shouldn't this be blocking the following querystring:

?test=onclick="alert(document.cookie)"

Using this ruleset it allows that through the WAF. It only blocks if you add in <script> tags.

When importing the waf-reputation-lists template in cloud formation, it fails with the following error:

The runtime parameter of nodejs is no longer supported for creating or updating AWS Lambda functions. We recommend you use the new runtime (nodejs4.3) while creating or updating functions.

It seems the specified runtime version on the lambda function is deprecated (nodejs instead of nodejs4.3)

It'd be great now that WAF and CF support IPv6 to add support for Spamhaus DROPv6.

As far as I can tell, the rules seem to be based most closely on the RC1 version of the OWASP 2017 Top 10:
https://www.owasp.org/images/3/3c/OWASP_Top_10_-_2017_Release_Candidate1_English.pdf

Are there any plans to update to the final version:
https://www.owasp.org/images/7/72/OWASP_Top_10-2017_%28en%29.pdf.pdf

Or is there a way to accomplish that?

In the file
https://github.com/aws-samples/aws-waf-sample/blob/master/waf-owasp-top-10/owasp_10_base.yml
lines 259 and 257.
Why is "example-session-id" used as the string to match inside the cookie? I am not aware of an attack that uses this string in the cookie. Also, if we are meant to put our own string there shouldn't this be a parameter that we set up? or perhaps this is for something I am unfamiliar with or I am miss-interpreting this rule condition.

Hi all,
When i launch your Cloudformation template in my account it failed and it is Roll backed,i get below error.It is not recognize my existing S3 bucket name and new S3 bucket name also.
Error: Error occurred while GetObject. S3 Error Code: NoSuchBucket. S3 Error Message: The specified bucket does not exist

I am getting following error -->

The runtime parameter of nodejs is no longer supported for creating or updating AWS Lambda functions. We recommend you use the new runtime (nodejs4.3) while creating or updating functions.

I think,
update to index.js file , changing code in line number 255 to 258 is required
For support of Node.js runtime v4.3

The WAF config resulting from the CF template can't support the number of CIDR ranges sourced from the supplied 3 URLs (which currently stands at ~12k). The template creates two IPSets. Each IPSet can contain up to 1000 CIDR ranges, therefore the WAF config can support only 2k ranges.

What's more, WAF itself is unable to support the number of CIDR ranges brought back by the Lambda script. (WAF max is 10k)

• One IPSet can contain up to 1000 CIDR ranges.
• Due to the nature of the match, only one IPSet condition can be used per Rule.
• One Web ACL can contain up to 10 Rules.
• Therefore the maximum number of CIDR ranges that can be supported by WAF is 10k.
• Currently, the number of CIDR ranges sourced from the URLs provided is ~12k.

I can't see a way around this unless:
a) AWS changes the limits for WAF, or
b) AWS changes the way that WAF rules work.
c) We ignore one or more of the source URLs and therefore don't block all 'known' bad IPs.

Source:
http://docs.aws.amazon.com/waf/latest/developerguide/limits.html
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-waf-rule.html#cfn-waf-rule-predicates

Template format error: Unrecognized resource types: [AWS::WAFRegional::WebACL, AWS::WAFRegional::SizeConstraintSet, AWS::WAFRegional::Rule, AWS::WAFRegional::ByteMatchSet, AWS::WAFRegional::XssMatchSet, AWS::WAFRegional::SqlInjectionMatchSet]

Not sure what I am doing wrong. Using the playbook as is.

LambdaWAFBlacklistingFunction | Error occurred while GetObject. S3 Error Code: NoSuchBucket. S3 Error Message: The specified bucket does not exist

Please help
"LambdaWAFBlacklistingFunction": { "Type": "AWS::Lambda::Function", "DependsOn": "LambdaRole", "Properties": { "Description": { "Fn::Join": [":", [{ "Ref": "RequestThreshold" }, { "Ref": "WAFBlockPeriod" }, { "Ref": "WAFQuarantinePeriod" }]] }, "Handler": "parser.lambda_handler", "Role": { "Fn::GetAtt": ["LambdaRole", "Arn"] }, "Code": { "S3Bucket": "mybucketthatcontains-the-file", "S3Key": "parser.zip" },

parser.py:363

If the CloudFormation stack is created with a hyphen in the name, then parser.py errors out on Line 364 at runtime due to the fact that the parse of the function name via 'context.invoked_function_arn' on line 363 incorrectly includes the function name. (the 'split' command does not parse the string correctly due to the extra hyphens)

To replicate:
Run CloudFormation with stackname = 'my-waf-ratelimit'

Error in logs:
An error occurred (ValidationError) when calling the DescribeStacks operation: Stack with id my-waf-ratelimit-LambdaWAFBlacklistingFunction does not exist: ClientError Traceback (most recent call last): File "/var/task/parser.py", line 443, in lambda_handler raise e ClientError: An error occurred (ValidationError) when calling the DescribeStacks operation: Stack with id my-waf-ratelimit-LambdaWAFBlacklistingFunction

Hi
I use waf in my site

and when I trying to submit post contain <style=" "> I got 403 forbidden.

is that because of waf? and is there any way to unblock that word?

Hello, I did install waf-block-bad-behaving, but it occured that blocked IP were never removed from Auto Block Set. After checking parser.py, I couldn't find in the code where they were removed from this set, and I did fix this issue by replacing:

if total_diff_min > (BLACKLIST_BLOCK_PERIOD):
print "[merge_current_blocked_requesters] \t\tExpired BLOCK %s rule"%k
outstanding_requesters['block'][k] = v

With :

if total_diff_min > (BLACKLIST_BLOCK_PERIOD):
print "[merge_current_blocked_requesters] \t\tExpired BLOCK %s rule"%k
else:
print "[merge_current_blocked_requesters] \t\tKeeping data of BLOCK %s rule"%k
outstanding_requesters['block'][k] = v

Regards

How do I use this set of conditions with app elb and an app that doesn't use cloudfront but uses a third-party CDN instead.

Hello,

How can I set create this rule in Region EU (Ireland) by default its creating both rule in global Region

thanks

def update_waf_ip_set(outstanding_requesters, ip_set_id, ip_set_already_blocked):
print "[update_waf_ip_set] Start"

counter = 0
try:
    if ip_set_id == None:
        print "[update_waf_ip_set] Ignore process when ip_set_id is None"
        return

    updates_list = []
    waf = boto3.client('waf')

    #--------------------------------------------------------------------------------------------------------------
    print "[update_waf_ip_set] \tTruncate [if necessary] list to respect WAF limit"
    #--------------------------------------------------------------------------------------------------------------
    top_outstanding_requesters = {}
    for key, value in sorted(outstanding_requesters.items(), key=lambda kv: kv[1]['max_req_per_min'], reverse=True):
        if counter < LIMIT_IP_ADDRESS_RANGES_PER_IP_MATCH_CONDITION:
            if not is_already_blocked(key, ip_set_already_blocked):
                top_outstanding_requesters[key] = value
                counter += 1
        else:
            break

    #--------------------------------------------------------------------------------------------------------------
    print "[update_waf_ip_set] \tRemove IPs that are not in current outstanding requesters list"
    #--------------------------------------------------------------------------------------------------------------
    response = waf_get_ip_set(ip_set_id)
    if response != None:
        for k in response['IPSet']['IPSetDescriptors']:
            ip_value = k['Value'].split('/')[0]
            if ip_value not in top_outstanding_requesters.keys():
                updates_list.append({
                    'Action': 'DELETE',
                    'IPSetDescriptor': {
                        'Type': 'IPV4',
                        'Value': k['Value']
                    }
                })
            else:
                # Dont block an already blocked IP
                top_outstanding_requesters.pop(ip_value, None)

    #--------------------------------------------------------------------------------------------------------------
    print "[update_waf_ip_set] \tBlock remaining outstanding requesters"
    #--------------------------------------------------------------------------------------------------------------
    for k in top_outstanding_requesters.keys():
        updates_list.append({
            'Action': 'INSERT',
            'IPSetDescriptor': {
                'Type': 'IPV4',
                'Value': "%s/32"%k
            }
        })

    #--------------------------------------------------------------------------------------------------------------
    print "[update_waf_ip_set] \tCommit changes in WAF IP set"
    #--------------------------------------------------------------------------------------------------------------
    response = waf_update_ip_set(ip_set_id, updates_list)

except Exception, e:
    print "[update_waf_ip_set] Error to update waf ip set"
    print e

print "[update_waf_ip_set] End"
return counter

This code appears to take the list of IPS that we determined that we needed to block / keep blocked and make a new list of IPs that are not already blocked.

               if not is_already_blocked(key, ip_set_already_blocked):
                top_outstanding_requesters[key] = value
                counter += 1

Using that list it appears as if we delete from the ip set anything that is not in this new list, which would include any ips that should still be blocked.

                if ip_value not in top_outstanding_requesters.keys():
                updates_list.append({
                    'Action': 'DELETE',
                    'IPSetDescriptor': {
                        'Type': 'IPV4',
                        'Value': k['Value']
                    }
                })

Wouldn't everything in the top list not be in the ip set and since we only add to it if its not already blocked. We then immediately remove everything thats not in the top list from the ip set so we are really clearing the ip set every time.

Am I reading this right?

While the cloud formation is created the following error occurred and the status changed to Rollback.

I have one bucket in N.Virginia region and pointed that for cloudfront access-logs. It has logs too.

"Error occurred while GetObject. S3 Error Code: PermanentRedirect. S3 Error Message: The bucket is in this region: null. Please use this region to retry the request"

This is the error i am getting.

Is it possible to get the actual IP that is blocked/unblocked and/or the actual injection where it pertains?

Hi,

The following request is not blocked by SQLi rule, even SQLMap was not intercepted :

Host: host.com
Connection: close
Content-Length: 71
Accept: application/json, text/plain, */*
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36
Sec-Fetch-Mode: cors
Content-Type: application/json
Sec-Fetch-Site: same-site
Accept-Encoding: gzip, deflate
Accept-Language: fr-FR,fr;q=0.9,en-US;q=0.8,en;q=0.7
Cookie: AWSALB=AWSLAB

{"email":"COUCOU' OR 1=1; --","captcha":null,"password":"coucou"}

We think that it might be due to the fact that the body is embedded in a json.

Thank you in advance,
Sincerely Yours,
Cou Cou

Hello,

I would want to use your samples with regional ALB's but all CloudFormation templates configure WAF in the CloudFront scope.

Is it possible to do this with your templates ?

Is it possible to immediate block IP when receive attacks (DDOS)? Apparently S3 has a lag to record a log list.