Enable/Disable Full Packet Capture

[riema...@gmail.com]

Hello,

I've searched the Wiki and this mailing list but can't find instructions for manually enabling full packet capture. I'd like to be able to turn it on and back off as needed without re-running SOSETUP, since we've already done a decent bit of configuration. Is this possible?

Thanks,

de

[Doug Burks]

Hi de,

Have you looked at the following?
https://github.com/Security-Onion-Solutions/security-onion/wiki/DisablingProcesses

> --
> You received this message because you are subscribed to the Google Groups "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at http://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/d/optout.



--
Doug Burks
Need Security Onion Training or Commercial Support?
http://securityonionsolutions.com

[jumbo jim]

Is it possible to wipe all packet capture from disk and start again?

[Doug Burks]

Yes, pcaps are stored in
/nsm/sensor_data/HOSTNAME-INTERFACE/dailylogs/YYYY-MM-DD/.

On Wed, Sep 2, 2015 at 1:37 AM, jumbo jim <jumbo...@gmail.com> wrote:
> Is it possible to wipe all packet capture from disk and start again?
>

[jumbo jim]

Once the pcap files in that location are deleted, are there any other remnants of packet content anywhere in indexes or anywhere on disk? I would imaginine there may be some IP headers indexed in ELSA etc, but just want to confirm once they're deleted, all packet contents are gone. Of course, remaining disk space would also need to be zeroed out.

[riema...@gmail.com]

Hi Doug,

Thanks for your reply. I did see the ability to disable, and actually followed those instructions to disable Snorby after seeing that it was to be excluded from future updates.

I guess the question should have been how to enable packet capture after already running setup, so that I can enable/disable as needed. I've got a week's worth of effort invested in configuring and tuning SO, so I would rather not lose that by re-running setup. Is there a guide to manually enabling netsniff-ng, and what other components would need to be enabled with it?

Thanks,
de

[Doug Burks]

On Wed, Sep 2, 2015 at 9:06 AM, jumbo jim <jumbo...@gmail.com> wrote:
> Once the pcap files in that location are deleted, are there any other remnants of packet content anywhere in indexes or anywhere on disk? I would imaginine there may be some IP headers indexed in ELSA etc, but just want to confirm once they're deleted, all packet contents are gone. Of course, remaining disk space would also need to be zeroed out.

There is going to be different packet information recorded in
different places depending on which services you have enabled.

[Doug Burks]

You can use the instructions in
https://github.com/Security-Onion-Solutions/security-onion/wiki/DisablingProcesses
to set the PCAP_ENABLED and PCAP_AGENT_ENABLED settings in
sensor.conf.

[jumbo jim]

Thanks Doug

Is the pcap service and pcap dailylogs the only service/files that have any reference to packet contents? I need to ensure I don't inadvertently store packet contents. And, if i do with the pcap service, I need to ensure I can safely wipe all pcaps when needed, which is why I am asking if deleting dailylogs will that suffice?

[jumbo jim]

Sorry, to clarify -

What services in SO will store and/or index packet *contents* ? Is it only the pcap service?

I have some sensitive information in HTTP POSTs. I therefore place a BPF filter on those packets - do not capture.

ELSA still stores and indexes the TCP headers (which I am ok with as it gives me some meta info). When attempted to pivot to capME, the result fails as the pcaps don't exist. This is ok.

I just want to clarify there is no deep packet analysis stored or indexed anywhere, other than /nsm/sensor_data/HOSTNAME-INTERFACE/dailylogs/YYYY-MM-DD/ ?

Thanks

[Kevin Branch]

Well, in addition to netsniff-ng doing full packet capture, the payloads of packets that trigger Snort/Suricata events are encoded into the event records added to the securityonion_db(sguil) and/or snorby databases.  In both databases the table is named "data" and I presume the data table records could be selectively or globally deleted without wrecking the associated events' integrity -- test to be sure.  Snorby is going away anyway so it would be best to just disable it.  If you need to not store any packet payloads in Sguil at all, there may be a barnyard2.conf option for the sguil output plugin to suppress that, or some sguild setting you can specify to not store the payloads.  Otherwise I presume will have to hack the code.  

Kevin

[jumbo jim]

Thanks Kevin,

So, for example, I deleted everything from -

/nsm/sensor_data/hostname/dailylogs/*

I then went and viewed a PADS Asset entry in Squert and drilled down to "Generate Transcript". Low and behold it pulled up the pcap.

Is this an example you are referring to ? This is pulling from the squil db?

I guess I have some more research to do. I would like to find a definitive way to wipe packet contents when needed.

Thanks

[Kevin Branch]

"Generate Transcript" pulls from the netsniff-ng snort.log files in the relevant dailylogs directory.

To wipe ALL of your dailylogs pcap files, I'd run this on your sensor

rm -f /nsm/sensor_data/*-*/dailylogs/*/snort.log.*

Also, any time you generate a transcript in Sguil or Squert, it gets cached as a .raw file, so to wipe out all those cached transcript extraction files, you could do:

rm -f /nsm/server_data/securityonion/archive/*/*-*/*.raw

You'd also need to delete all the records from the MySQL securityonion_db.data, and possibly optimize/compact the table to reclaim the space occupied by those records.

I don't personally know anywhere else pcap data would be lying around.

Kevin

Thanks

[jumbo jim]

Thanks Kevin. This is very helpful.

Also - if I have BPF filter in place to drop everything for a specific port, ELSA still appears to log the packet headers under "destination ports". CapMe cannot pull up the packet however (which is good).

If I have that BPF filter in place, can other sensors still potentially store and index that packet content?

Is there a diagram or chart describing the path of logged packets, and how they may be treated throughout SO depending on services installed?

[Kevin Branch]

If ELSA is returning BRO_CONN records related to traffic that does not match the BPF you want to limit things to. then either the BPF is wrong or not properly referenced, or the returned records date back to before your BPF was changed and Bro was restarted.  Check this closely:

https://github.com/Security-Onion-Solutions/security-onion/wiki/BPF

If you use BPFs correctly, the excluded traffic should never be stored in a pcap file, trip Suricata/NIDS alerts, generate a PRADS/SANCP event or Argus or Bro event.  Nor will any of that traffic end of in a "data" table in Sguil or Snorby.  

BPFs define what network traffic your various SO subsystems will operate upon.  If you have multiple SO sensors, the BPF you want will need to be in place on all of them.  As I understand it, with an SO server / SO sensor setup, the BPF configuration set up on the server is propagated out to all the SO sensors.  I don't have any experience with that however, as I only user standalone SO systems at this time.

Kevin

[jumbo jim]

Thanks Kevin for confirming the BPFs should not trip anything else further down the chain.

Re: ELSA - the BPF is only referencing HTTP POST traffic in one direction, but what is returned by the application is fine to pcap. It does partially break the capMe view as expected.