Signature file and web_ids interplay about web_ids HOT 17 CLOSED

Do you mean against which keys to match one request message?

Since all messages have the same format, idea is to match all request messages against all keys, and if any of them match we print a log out.

This allows us to have bigger signature range while keeping signature database small.

I.e. Instead of providing signatures for all combinations of while request messages, we provide smaller signatures for specific keys. These can be then combined programmatically if needed, which is easier and faster then writing the combinations manually in signatures file.

from web_ids.

My idea was to have two main signatures ipSignature and requestSignature. The ipSignature is an array of ips.
The requestSignature can have any of the following keys: method, status, uri. and body. At least one of those keys must be provided, but others are optional and get treated as AND conditions. In code, we then have two functions to match a request.

1. match against array of ipSignature
2. match against array of requestSignature, checking if a key is present and trying to match request against it

A possible signature file would then look like:

{
  "ipSignature" : [ "^10\\.0\\.0\\.10$", "^192\\.168\\.0\\.255$" ],

  "requestSignature" : [
       {
          "uri" : ".*\\.php"
        } ,
        {
           "method" : "GET",
           "body" : ".+"
        },
        {
           "status" : "[45]\\d\\d"
        }
   ]
}

from web_ids.

The idea is to use regular expressions to match request message against signature database.
Request message is split into parts, and each part is matched against a set of keys stored as an array in JSON files.

Each key has a regular expression used to match the request message and this can directly be used in web_ids javascript.
This makes matching easier and more flexible. At the moment, plan is to simply log any request which matches against any of the keys. If we need to be more specific, additional field severity or similar could be added to any of the signature keys.

from web_ids.

The problem by separating the keys might be, that some matches are not as precise as they should be.
example: if something is in the request_body, and the request_uri is /, or method is GET, this would not make much sense,
but having something in the request_body for other requests is perfectly fine.

(see this discussion about request_body in a GET request https://stackoverflow.com/questions/978061/http-get-with-request-body)

from web_ids.

I see what you mean, but I don't think it makes a difference in our case. We're not trying to interpret the request per se, instead we should try to detect any suspicious behavior.

i.e. wouldn't GET request with a body be suspicious?

from web_ids.

yes exactly. a GET request with a body might be suspicious, but if we only check for GET and body separately, it would be perfectly fine

from web_ids.

The logic could for such checks can be implemented in checker functions. Instead of passing specific part of request message like only the URI, we could instead pass the whole message object.

Then part of the URI check could then be existence of request body when method is GET etc.

from web_ids.

yes exactly. a GET request with a body might be suspicious, but if we only check for GET and body separately, it would be perfectly fine

Thats why there is a request method key in the bodySignatures. So in the checkBody method only if everything in a request matches with a signature we log it as suspicious.

Instead of passing specific part of request message like only the URI, we could instead pass the whole message object.

Yes, it makes a lot more sense, otherwise the rest of the keys in a signature would be redundant.

from web_ids.

but then, how do we know which key to match?
maybe it would be better to just distinguish between ip matches and request matches.

In request matches, all of the fields like uri, status, body are optional, but at least one must be provided.
Both ip and request keys would then be arrays in json files.

In this way we only need two functions and dont have to worry about what field should now be matched and so on

from web_ids.

In my opinion it should go something like this (in this instance for body signature) for a message m:

for s in BODY_SIGNATURES:
  if m.requestMethod matches s.requestMethod and m.requestUri matches s.requestUri and m.body matches s.body:
    log as a malicious body activity

EDIT: flipped s and m to make it clearer

from web_ids.

Now I get what you mean. I approve of this idea!

from web_ids.

That does seem even cleaner, implementation should still be mostly the same (go through requests and if any of them match, log the request).

I like it 👍

from web_ids.

If there is no opposition from @Matthiastxt, I will use this style for the upcoming signatures.

from web_ids.

fine for me (y)

from web_ids.

Alright then. I will close this issue.

from web_ids.

Oof. Did I close it too soon?

from web_ids.

sorry. wanted to try the fixes ... tag in a pull request :D
i will create a new issue for the implementation of the discussed signature syntax

I will close this issue now since the label fits for question but not implementation :)

from web_ids.