sensor-clean help?

[Corbin Fletcher]

Hello SO,

I am having a difficult time understanding how to properly purge files
every six hours. This is where the growth activity occurs:
/nsm/sensor_data/onion-eth1/dailylogs/$date/**

/etc/cron.d/sensor-clean is not cleaning this directory?

I have researched this and I have made very little progress. I do not
have any experience with writing scripts. I am manually running rm * on
the directory.

Additionally, manually running sudo /usr/local/sbin/nsm --sensor �clean
does not clean the sensor.

Could someone please advise on how to purge these log files every six
hours?

The server hard drive is not sufficient for the task and if fills in
about six hours (71G). Upgrading the hard drive is not an option, now.

//Corbin

[Doug Burks]

Hi Corbin,

From our Hardware page (http://code.google.com/p/security-onion/wiki/Hardware):

"You need LOTS of storage as Security Onion does full packet capture
and it can fill a disk quickly. We have an hourly cronjob that purges
old pcaps once the disk reaches 90% capacity. However, if you have a
small disk and/or are monitoring a large amount of traffic, you may
fill that last 10% before the next hourly purge. Additionally, the
purge scripts are designed with the idea that you want to keep at
LEAST 1 day's worth of full packet capture on disk, so they won't
delete any pcaps with today's date on them. For example, suppose you
are monitoring a 50 Mb/s link, here are some quick calculations:
50Mb/s = 6.25 MB/s = 375 MB/minute = 22,500 MB/hour = 540,000 MB/day.
So you're going to need about 540GB for one day's worth of pcaps
(multiply this by the number of days you want to keep on disk for
investigative/forensic purposes). Note that this is just pcaps (other
logs will take up additional storage), so you may want to round up to
the next terabyte to ensure sufficient storage. The more disk space
you have, the more log retention you'll have for doing investigations
after the fact. Disk is cheap, get all you can!"

If all you have is 71GB and you can't upgrade it, your best bet may be
to simply disable the full packet capture using something like the
following:

# stop the running daemonlogger process
sudo nsm_sensor_ps-stop --only-daemonlogger

# permanently disable the daemonlogger binary
sudo chmod 0 /usr/bin/daemonlogger

However, I would really recommend against that as it severely limits
your investigative capability. If at all possible, please upgrade
your hard drive. Hardware is cheap, incidents are not!

Hope that helps!

Thanks,
Doug

On Mon, Jul 2, 2012 at 11:45 AM, Corbin Fletcher <cor...@freeway.com> wrote:
> Hello SO,
>
> I am having a difficult time understanding how to properly purge files every
> six hours. This is where the growth activity occurs:
> /nsm/sensor_data/onion-eth1/dailylogs/$date/**
>
> /etc/cron.d/sensor-clean is not cleaning this directory?
>
> I have researched this and I have made very little progress. I do not have
> any experience with writing scripts. I am manually running rm * on the
> directory.
>

> Additionally, manually running sudo /usr/local/sbin/nsm --sensor –clean does

> not clean the sensor.
>
> Could someone please advise on how to purge these log files every six hours?
>
> The server hard drive is not sufficient for the task and if fills in about
> six hours (71G). Upgrading the hard drive is not an option, now.
>
> //Corbin

--
Doug Burks
http://securityonion.blogspot.com