Lack of alerts from Snort (and dropped packets)
Kartikeya Puri
I am trying to monitor edge links that we have with peak traffic of 3.5G and 2G. The standalone box that I have has 256G memory, with 40 cores and about 12T storage. I have just got this up and running (redacted sostat attached), and I am seeing heavy packet drops (expected, but not so much). In addition to the drop, strangely I am seeing that snort is only picking up one alert (GPL ICMP_INFO PING *NIX). Even a curl call to testmyids.org (which goes through the monitored part) is not creating any alert. Stopping all sensor processes and running snort in fast alert mode, I see the same behavior i.e. only GPL ICMP_INFO PING *NIX is captured as alert.
I would love to believe that we have a peaceful network, however I don't think that is the case. So that has lead me to believe that something is wrong with my snort config. I would be really grateful for any and all help that I can get in troubleshooting the issue. Also, how many snort and Bro instances should I ideally be running?
One of the task for me is to tune the number of active rules. In absence of any noisy rules, I am wondering how would I go about tuning these.
Cheers,
Kartik
Wes
It looks like your average load is about 2x the number of available cores on average. You may want to consider increasing the number of available cores and performing tuning as described here:
https://github.com/Security-Onion-Solutions/security-onion/wiki/PF_RING
You may also want to look into the provided guidance for the increasing number of ELSA buffers (included in the sostat output)--likely a result of the overworked box.
You could also check /var/log/nsm/hostname-interface/snortu-*.log for more clues, in addition to /var/log/nsm/hostname-interface/barnyard2.log and snort-agent.log
Thanks,
Wes
Kartikeya Puri
Increasing the number of cores or buying a similarly speced box is on the cards, but it will take some time. Hence I need to look at reducing the workload in alternative ways such as:
* have a BPF drop ssl traffic, as it is actually not being analyzed by the edge sensor (at least the payload is not analyzed)
* reducing the number of snort rules
* I may turnoff full packet capture if it helps
Until I get an additional box to share the load, this box will have to overwork. Are there other suggestions about how can I go about reducing the load?
Now, about snort alerting issue. I configured snort with registered rule set (with an oink code) and did a rule-update. I have started seeing some additional alerts (while no. of active rules just went to 30k :-[ ).
I am going through the log files to scour for hints.
Cheers,
Kartik
--
Follow Security Onion on Twitter!
https://twitter.com/securityonion
---
You received this message because you are subscribed to a topic in the Google Groups "security-onion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/security-onion/JaK8KKKvxHA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to security-onio...@googlegroups.com.
To post to this group, send email to securit...@googlegroups.com.
Visit this group at https://groups.google.com/group/security-onion.
For more options, visit https://groups.google.com/d/optout.
Wes
The points you mentioned:
* have a BPF drop ssl traffic, as it is actually not being analyzed by the edge sensor (at least the payload is not analyzed)
* reducing the number of snort rules
* I may turnoff full packet capture if it helps
Thanks,
Wes
Kartikeya Puri
> >
> > For more options, visit https://groups.google.com/d/optout.
>
> Kartik,
>
> The points you mentioned:
>
>
> * have a BPF drop ssl traffic, as it is actually not being analyzed by the edge sensor (at least the payload is not analyzed)
>
> * reducing the number of snort rules
>
> * I may turnoff full packet capture if it helps
>
>
> ...will also help with your overall performance. You will definitely want to tune your rules for your environment -- try to get them around 10k, if possible, and it could help a lot.
>
> Thanks,
> Wes
Thanks Wes,
I did the following:
* turned off monitoring for one of the interfaces (so I am now monitoring only a single link with 3Gbps capacity).
* reduced the number of processes for Snort and bro to 10 each
* configured autoconf for noisy rules
This has helped me a lot in terms of dropped packets and overall performance. Load avg. now hovers between 14-20 (I am keeping a close watch on it). Loads of unused memory is available for the box.
Now I will start ruleset tweaking to see if I can disable some of the non-essential rules.
Cheers,
Kartik