edge case - sensor cleanup fails when too many pcaps

[Tom Wood]

Hi,

Ran into an issue last night with one of our sensors in a very busy
enviroment. It's possible for the current daily log directory to contain
more pcaps than will fit in the ARG buffer when doing ls snort.*.

When this happens the clean script won't be able to delete old pcaps
until midnight when the previous day is deleted, this also causes
nestsniff-ng to fail.

I've added a potential fix for this here:
https://github.com/Security-Onion-Solutions/securityonion-nsmnow-admin-scripts/compare/master...tom564:patch-1

I appreciate that this is probably an edge case as this is on a sensor
with ~15TB storage and around the same growth rate. I have previously
increased the pcap size to 300mb to try and reduce the amount of files
and this helped.

Did some rough performance testing and replacing ls with find seems more
efficient.

root@XXXXXX:~# time find
/nsm/sensor_data/gcc-022-eth2/dailylogs/2018-04-07/ -type f -name
'snort.*' | awk -F "." '{print $NF,$0}' | sort -k1 -t " " | head -1 |
cut -f2- -d ' '

/nsm/sensor_data/XXXX-eth2/dailylogs/2018-04-07/snort.log.1523059208


real    0m0.173s

user    0m0.144s

sys     0m0.052s

 root@XXXXX:~# time ls
/nsm/sensor_data/XXXX-eth2/dailylogs/2018-04-07/snort.log.* | head -1

/nsm/sensor_data/XXXXX-eth2/dailylogs/2018-04-07/snort.log.1523059208

 real    0m0.758s

user    0m0.644s

sys     0m0.108s

[Doug Burks]

Hi Tom,

Thanks for the potential fix!

I've created Issue 1234 for this:
https://github.com/Security-Onion-Solutions/security-onion/issues/1234

--
Doug Burks