Patch for issue 35. about pulledpork HOT 20 CLOSED

I think that adding an include option adds undue complexity... This is how I 
see it, and maybe I'm way off base here ;-).  Adding a category 
(filename).rules to the ignore list is the same as commenting out in 
snort.conf.. whereas not adding it is the same as including the rules file in 
the snort.conf.  Adding an include option would allow for users to add 
contradicting entries etc etc.

As to the second option, (the etc foo) that's certainly a doable item and one 
that will be added in the future.. as such I will leave this open as a low 
priority enhancement request.

JJC

I'd like to add a +1 vote for an rule file include feature. I tried making the 
jump from oinkmaster to PP this past weekend and the default include-all / 
exclude-some filter methodology for rule management really didn't work for me. 
I like to selectively enable filters in order to keep false-positives low, 
alert volume manageable, and not create duplicate alerts for activities that I 
know other security policy enforcement devices will handle. Enumerating a 
comprehensive exclude list crossed my threshold for pain, and I ultimately 
threw the oinkmaster setup back in place. I would not want to have to repeat 
that enumeration process each time a new rules file was deployed, nor would I 
want to deal with alerts generated before I had the opportunity to adjust the 
exclude list.

Tony,

Can you provide an example use-case for me, I'm not exactly sure what you were 
trying to do with PP that oinkmaster did in a better way.

If you only ever want certain categories enabled you can simply do the 
following right now:

1. Adjust the order of state change operations to drop,disable,enable (or 
similar, just so that disable is before enable). 

2. Disable all rules using a pcre:.  

3. Selectively enable the categories that you want. 

 This would be 1 modification line for the state order, one line for the pcre in the disablesid.conf and then a line per category that you want to enable in the enablsid.conf.

Unfortunately this is *not* equivalent because this will turn on rules that are 
disabled by default.  There are quite a lot of these as I am sure you are aware.

Or am I missing something (I'd be delighted to find I am deluded, again ;).

My reading of the code was that once you had used a pcre to disable all rules 
then there was nothing to distinguish a rule that was shipped disabled and one 
that was disabled by PP.

From  memory PP disables a rule by putting a hash at the start so there is no 
means of distinguishing it from a rule that was commented out to start with.

BTW I have now been using my patched PP in production for about a month and I 
am very pleased with PP :) 

Ok, I see where you are going here...

Yes, use case is exactly what Russell pointed out. To use PP as-is I would 
either need to use Russell's patches (and I hate dealing with local 
modifications through upgrades), or write some sort of wrapper that keeps track 
of which rules are disabled by default and re-disables them after enabling a 
category through PP.

[deleted comment]

I am looking at adding an "include" option.. this would be disabled by default 
and in the "optional" section of the master config.. the obvious risks here are 
that new rules categories would have to be manually added, or someone new to 
snort uncomments the line and screws themselves etc etc....

I like the way things currently work (I think that using an include file adds 
unnecessary complexity and circumvents pulledpork's rule-modification logic), 
although to achieve the behavior Russell and I are looking for (disable 
everything, then only enable what I want)... It seems like the easy fix would 
be to differentiate between rules that are disabled upstream vs. rules that are 
disabled in the pulledpork config. Perhaps instead of just adding a simple '#' 
in front of the rules in the "disabled" pass it could be changed to '# disabled 
upstream #' and '# disabled by pulledpork #'? I've been debating creating a 
patch to do this, but I've been rather busy and was hoping there'd be something 
available from the main distribution that would accomplish this. If that's not 
possible I'll work on cobbling together a patch. (comment edited - my 
previously-mentioned workaround doesn't work, so I'll create a patch... I still 
prefer my idea over "include" option because if I understand the intent of that 
method correctly it would circumvent the current enable/disable functionality 
for rules files that were "included" - which I don't like)

I am not fussed how it is done.  It is actually trivial to make PP leave 
disabled rules disabled -- single character patch iirc.  The enable code is 
something like s/^#+//  remove the '+' and this will remove the # inserted by 
the disable  but not ones in the original rule.   This has the disadvantage 
that any original rule with more that a single # will never get enabled (I'm 
not sure if this is much of a problem in practice but certainly could lead to 
unexpected behavior -- something to be avoided).

To do this properly you need a re-enable command as distinct from enable which 
just removes what ever pattern disable puts in while enable behaves the same.

Which ever method is used putting it in some optional part so it does not 
confuse users is a good idea.  It is only professionals who fiddle with 
rulesets to this extent and so keeping it out of ordinary users hair is all to 
the good.

Another thought would be to have a command line flag the set expert mode that 
exposed this functionality...

I can modify the internal data structure so that PP knows what the "default" 
shipped method was, but now I'm confused again.. explain again what the 
ultimate desired result is here..  I just re-read all of the postings and I can 
now not re-wrap my head around the issue lol

ok, just re-read the original issue #35.. gonna merge these.. but is this what 
you want to do.. as an example use-case

enable only certain rules categories but only in their default state?..

so maybe in disablesid have it disable everything.. and in enablesid you will 
enable by category.. but have a flag that sets the rules (in that category) 
back to their ORIGINAL state?

The risk here is a potential to increase runtime dramatically, as the rulestate 
itself needs to be checked and recorded within the data structure.. as of now 
the data is simply read and stuck into a data structure.. no actual testing / 
checking of it.

I think that this will do what you need... 

I am now reading the default state of the rules into a new area of the data 
structure.  if the -R (return state) flag is set.. then the categories 
specified in the enablesid.conf will return to their original state.. ALL other 
functions stay the same.. so if you specify a category in the disablesid.conf 
it will disable all of those rules regardless of -R flag state, unless of 
course you process the enablesid.conf last.. make sense, I hope so!

The runtime issue is one of the reasons I like working at the file level -- up 
front.  You only have to process the files you want.

I have just thought of another reason why the disable all then enable 
categories won't work.

What Tony and I are trying to do is have just certain rule categories enabled 
but we don't want the rules that were shipped disabled.

We also (well I do and i'm 99% Tony will too ;) want to be able to disable 
individual rules that are giving FPs with in the category.  This is impossible 
using the disable/enable strategy.

EXample:

I have a sensor on our dmz inside the firewall that runs just some spyware 
rules, p2p rules and local rules looking for machines contacting bot 
controllers.  From my sensor config:

rule_cateories => 
['emerging-p2p','p2p','spyware-put','emerging-malware','emerging-user_agents'],
rule_files => ['botnet.rules','local.rules']

The rule categories line specify which categories I want PP to handle.

For these rule sets I want the rules shipped disabled to stay disabled -- and I 
also disable some other rules:

[rful011@secpupprd01 ~]$ wc -l ~sensors/Sensors/masters/dmzi/pp/disabled.conf
49 /home/sensors/Sensors/masters/dmzi/pp/disabled.conf

so I think that by far the simplest way of doing this is to handle the rule 
files up front.

Russell

I think that what I just did will work for you.. here is the example...

you disable everything using the disablesid.conf range or pcre etc... 
you enable only your categories that you want on that sensor..

rule_cateories => 
['emerging-p2p','p2p','spyware-put','emerging-malware','emerging-user_agents'],
rule_files => ['botnet.rules','local.rules']

when you run PP, you change the order of the functions to disable,enable,drop 
(just as long as disable is before enable) and you add the new -R flag....  
This will disable EVERYTHING but set the rules to the state that they were in 
the ORIGINAL tarball.

eh, the rules specified in the enablesid.conf that is

Committed SVN rev 202

Ummm... how do we then disable rules within those categories?

Russell

I'm going to play with this and see if it meets my needs... I'm still too early 
in my deployment to have false-positives to tune out yet, but I'm thinking (to 
Russell's point) that it might be possible to use the "ignore" option to filter 
out the categories we don't want at all, then enable/disable from what's left 
and add the -R option to disable the rules that came disabled from upstream... 
hopefully it works that way, I'll let you know if that doesn't.