Stream5 Mass Alerts and Threshold Suppression Failing

[Kevin O'Grady]

Hey everyone,
Lately I've been hammered with Stream5 alerts in Squil after switching back to the Snort IDS engine (due to ongoing issues with stale PIDs).

To address these alerts I've added the entries into the thresholds.conf file and restarted the services. The problem is that no matter how I add these entries in they're never getting suppressed.

I've tried (this rule is for "stream5: TCP Small Segment Threshold Exceeded"):

-suppress gen_id 129, sig_id 12

then:

-suppress gen_id 129, sig_id 12, track by_src 192.168.1.1/24

then:

-suppress gen_id 129, sig_id 12, track by_src 192.168.1.1/24
-suppress gen_id 129, sig_id 12, track by_dst 192.168.1.1/24

Each time I've got to go in and stop the services, manually clear out all of the alerts in the Sguil DB (since there's as many as 100k within a minute or two of restarting the services), then start the services back up, rinse and repeat.

I'd really like to figure out what caused the Stream5 messages to blow up and if it requires some config changes in the snort.conf file from me but I'd also like to get this addressed so I can use Sguil again and have fought this for two days.

Attachd my Sostat-redacted for review (keep in mind that I just cleared all of the Sguil events just a minute or two before getting this output) and as always any help is greatly appreciated!

Thanks,
Kevin

[Wes]

Kevin,

Are you modifying this on the sensor or the server? Did you run a rule update on the server? If so, try running sudo salt "*" state.highstate to immediately update the sensor configuration.

Salt auto-replicates this information to the sensors every 15 minutes--is there any chance you edited it on the sensor and the configuration got overwritten?

Thanks,
Wes

[Kevin O'Grady]

Hi Wes,

I'm modifying this is on the server itself (it's a standalone setup). I did run rule-update with no change.

[Wes]

On Tuesday, April 19, 2016 at 3:00:07 PM UTC-4, Kevin O'Grady wrote:
> Hi Wes,
>
> I'm modifying this is on the server itself (it's a standalone setup). I did run rule-update with no change.

Ultimately, it may be better to try tuning the preprocessor in /etc/nsm/hostname-interface/snort.conf.

https://www.snort.org/faq/readme-stream5

Thanks,
Wes

[Shane Castle]

Just to be clear about this: you are modifying /etc/nsm/<sensor>/threshold.conf,
right? Not "thresholds.conf", as you wrote in your original message?

And, have you read the info in /usr/share/doc/snort/README.stream5.gz? Way back
in the day, before there was a threshold.conf, modifying (or turning off) the
stream5 preprocessor was the only way to throttle these alerts. (Yes, it's a
slog. Snort ain't all that easy.)

On 19.04.2016 21:00, Kevin O'Grady wrote:
> Hi Wes,
>
> I'm modifying this is on the server itself (it's a standalone setup). I did run rule-update with no change.
>

--
Mit besten Grüßen
Shane Castle

[Kevin O'Grady]

Hi Wes,

What are some things that I should look into tuning with the preprocessor in /etc/nsm/hostname-interface/snort.conf?

Hi Shane,

That's correct. I've read (in this group) that you want to tune it versus disabling it since there is the threshold.conf file now. Haven't read that particular document yet but will look into it.

Thanks,
Kevin

[Shane Castle]

Take a look at this thread from a while ago: http://seclists.org/snort/2012/q1/515

Also, can you just please double check that you don't have a "thresholds.conf"
file in the config directory for your misbehaving sensors?

[Kevin O'Grady]

Hi Shane,
I double-checked that there's not a thresholds.conf file and there is not. I've got other suppressed rules in there from the past that are working fine.

I'll work on the settings from the link you provided and report back with the outcome.

Thanks!
Kevin

[Kevin O'Grady]

Hi Shane,

Worked through the link you provided and still getting "stream5:TCP small segment threshold exceeded", "ssh:protocol mismatch", and "http_inspect" alerts like crazy.

Especially odd with the "ssh:protocol mismatch" alerts since I followed to-
In "preprocessor ssh", remove "enable_protomismatch"

Running a rule-update didn't change it either. Any other ideas?

Thanks!
Kevin

[Doug Burks]

Hi Kevin,

What are the timestamps on the alerts? Is it possible that you're
seeing a backlog of alerts from before your config changes?

> --
> Follow Security Onion on Twitter!
> https://twitter.com/securityonion
> ---
> You received this message because you are subscribed to the Google Groups "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at https://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/d/optout.



--
Doug Burks

[Kevin O'Grady]

Hi Doug,

The timestamps are all current once I kick the services back on after clearing out the Sguil DB so I don't believe it's backlogged.

I'm following Richard Bejtlich's blog post (http://taosecurity.blogspot.com/2013/02/recovering-from-suricata-gone-wild.html) to address all of the Sguil alerts.

Thanks!
Kevin

[Shane Castle]

I've been doing a little thinking about this. There are several things that may
need looking at.

One: could this be an indication of a network device that is misbehaving?
Determining this will need knowledge of the network architecture and layout. Is
there a commonality or point of intersection in the packets alerted on? Are just
one or two interfaces generating the errors? Granted, determining the path that
a packet took before it got to the monitored interfaces may be difficult.

Two: You have five snort.conf files (and threshold.conf files) to modify for any
changes. Have you been careful to make the same changes in all of them?

Three: read the README file for stream5 (in /usr/share/doc/snort) and note the
parameters for the small_segments option for stream5_tcp. What are yours set to?
Again, look at a few of the alerts for information about the packets that are
causing the alerts and compare that to the small_segments option.

Finally, I have to say that I have never had alerts from the stream5
preprocessor that were anything but FP. While a nice idea, it has always just
been a PITA for me to deal with. (This is just IMHO of course.)

Good luck, and when you find the cause, or get them all suppressed if not,
please let us know what worked for you.

[Kevin O'Grady]

Hi Shane,
Not sure what the root cause is since there haven't been any network hiccups according to other monitoring tools in place and I've played with the small_fragments without change (currently set to 0).

Modifications to the threshold.conf were made on eth1 and then the file was copied to the additional interfaces to avoid editing mishaps.

Despite my issues with Suricata and stale PIDs, I've switched back from Snort to Suricata, ran a rule update, and now things are back to normal. No idea why Snort was giving such a headache but it's resolved now. If I have anymore issues I'm going to bite the bullet and rebuild the box from scratch.

It's not the best approach to resolve problems like these as I really did want to get to the root cause for myself and anyone else reading this but time is limited.

Thanks everyone for the help and being an awesome community!
Kevin