Validating BPF filters on Snort and Daemonlogger

[PhilipW]

Hey Doug,

I forgot to post this question in the barnyard/ssh tunnel thread, but thought it wasn't exactly related and might warrant a new topic.

In attempting to ignore all unnecessary traffic to reduce dropped packets and spare disk space, I have applied a common BPF filter to Snort and Daemonlogger, using:

# UTC specific options
if [ "$SENSOR_UTC" == "Y" ]
then
SNORT_OPTIONS="-U ${SNORT_OPTIONS}"
SNORT_BPF="/etc/nsm/bpf.filter"
DAEMONLOGGER_BPF="/etc/nsm/bpf.filter"
BARNYARD2_OPTIONS="-U ${BARNYARD2_OPTIONS}"
DATE_OPTIONS="-u"
fi

and then:

[ -z "$SKIP_SNORT_ALERT" ] && process_start "snort" "-u $SENSOR_USER -g $SENSOR_GROUP
-c $SNORT_CONFIG --daq afpacket -i $SENSOR_INTERFACE_SHORT -F $SNORT_BPF -l $SENSOR_LOG_DIR $SNORT_OPT
IONS" "$PROCESS_PID_DIR/$SENSOR/snortu.pid" "$PROCESS_LOG_DIR/$SENSOR/snortu.log" "snort (alert data)"

and

[ -z "$SKIP_DAEMONLOGGER" ] && process_start "daemonlogger" "-u $SENSOR_USER -g $SENSOR_GROUP -i $SENSOR_INTERFACE_SHORT -f $DAEMONLOGGER_BPF -l $SENSOR_LOG_DIR/dailylogs/$TODAY -n snort.log -s 13
4217728" "$PROCESS_PID_DIR/$SENSOR/daemonlogger.pid" "$PROCESS_LOG_DIR/$SENSOR/daemonlogger.log" "daem
onlogger (full packet data)"

For reference, I'm trying to ignore all traffic to and from the specified IPs, so the filter file looks like:

!(host firewall.ip.to.ignore) &&

!(host firewall.ip.to.ignore) &&

!(host firewall.ip.to.ignore) &&

!(host firewall.ip.to.ignore)

The seen traffic still seems high compared to pre-BPF numbers and I was wondering if you knew of a way to verify that the filters are working properly?

Thanks for any insight.

- Philip

[Liam Randall]

Philip,

Prototype them using tcpdump on the sensor:

tcpdump -i eth1 -c 10 -n ether src 00-C0-ED-1B-A5-00

watch out for your edge cases (see previous list posts on vlan tag)

Thanks,

Liam

[Doug Burks]

Also reference these articles from Richard Bejtlich on using "tcpdump
-d" for troubleshooting BPFs:
http://taosecurity.blogspot.com/2004/09/understanding-tcpdumps-d-option-have.html
http://taosecurity.blogspot.com/2004/12/understanding-tcpdumps-d-option-part-2.html
http://taosecurity.blogspot.com/2008/12/bpf-for-ip-or-vlan-traffic.html

Hope that helps!

Thanks,
Doug

--
Doug Burks
Security Onion | http://securityonion.blogspot.com
President, Greater Augusta ISSA | http://augusta.issa.org
SANS Augusta 6/11 - 6/16 | http://www.sans.org/augusta-2012-cs/

[Castle, Shane]

Yep I've done this almost exactly that way, and yes trying them out with tcpdump is the way to go. Also, mining the data using the sguil query tools will tell you if the packets you don't want are actually not there ;).

--
Shane Castle
Data Security Mgr, Boulder County IT
CISSP GSEC GCIH

[PhilipW]

Hi Liam,

Thanks for the reply.

Would putting a BPF filter on my PC IP, then executing 'curl http://testmyids.com' and looking for the alert, before and after restarting the nsm scripts, also be a valid test?

If that test is valid and the script syntax is correct, is it safe to assume that the filters are working and the disk filling is at the correct rate, or might there be other processes logging large amounts of data?

Thanks.

- Philip

[Scott Runnels]

Hi Philip, 

Are you needing a way to check if your capture filter works and to test it by generating traffic?

I like to use scapy and tshark. I use tshark much like Liam suggested to test my capture filter and I use scapy to generate traffic with the significant information for that capture filter.  

For example, 

I want to check that I have the correct format to exclude the ip address 172.16.1.31 as source or destination.  I'm reasonably sure it's a capture filter like this:  "not (host 172.16.1.31)" or even "! (host 172.16.1.31)" but I'm not 100% and unfortunately the machine that's usually there is currently turned off so I can't test it. 

Like Liam suggested, I'm going to work with a secondary tool to prototype my BPF filter.  I like tshark because it makes your house smell like cookies every time you use it(trust me on this, I'm a doctor*).

From my sensor I'm going to issue the command:  

sudo tshark -n -f "! (host 172.16.1.31) && not icmp" -i eth1 -R ip.host==172.16.1.31 -T fields -e ip.src -e -e tcp.srcport -e ip.dst -e tcp.dstport -e tcp.flags

That will capture with the BPF indicated by -f (I've included not icmp because I don't want to see the ICMP unreachable kickback) using the interface indicated by the "-i" and using two pieces of special sauce.  

The -R flag is our display flag.  I only want to see packets if one of the hosts is 172.16.1.31 this way I don't have to try to grep it out later.

The -T and -e flags indicate that I'm going to extract just those fields from the pcap.  So, with this in place if anything headed for 172.16.1.31 crosses the line, it's going to pop up in a very succinct and attractive list showing the source ip, the source port, the destination ip  and destination port and then the tcp flags.  

But like I said, that machine is down.  Clearly, I need to be flogging my PFY with regularity for I am inconvenienced.  Enter scapy!  SecurityOnion includes scapy, because Doug knows you need also I suspect he has access to a time machine.  From either another machine that has scapy installed or even just another session on the same security onion box:

sudo scapy

Once it loads up, enter this in line by line, hitting enter with aplomb after every line:

ip = IP()

tcp = TCP()

ip.src = "172.16.1.31"

ip.dst = "172.16.1.55"

tcp.sport = 1234

tcp.dport = 9876

payload = "Would you say I have a plethora of pinatas?"

send(ip/tcp/payload)

Now check back to the capture being created with tshark.  If it's empty, the BPF filter is working.  Well, it's either working or the sniffing NIC is placed somewhere it wouldn't see that traffic!  If you're cautious, just rerun the tshark command editing the "-f" entry to remove the "not" and then in your scapy buffer, just hit the upkey and you should be back to "send(ip/tcp/payload)" and hit enter.  This time you should see it.

v/r

Scott

* I am not a doctor, but you can still trust me, I'm a dentist **

** Nah.

--
Scott Runnels

[Doug Burks]

Wow, not much I can add to Scott's masterpiece other than "needs more cowbell".

Doug

--

[PhilipW]

Hey guys,

Thanks for the replies and tools to look at.

Initially my main concern was not only the traffic being processed by Snort, but the pcaps from daemonlogger filling the HDD.

My other response seems to have come in a little late, but I figured the curl test was a very rudimentary way to test Snort, but I wasn't sure how to verify what exactly daemonlogger was writing to the drive.

With the script additions I made and past experience with the BPF filters reducing traffic much more, the high traffic I was seeing compelled me to think I had mis-scripted something and only gotten half the solution in place.

I guess I wanted a way to search the pcaps independent of Snort/Sguil. Actually, in thinking about it, I guess the pcaps could be opened with Wireshark or something...

Ultimately, the answer was much more simple than all of this. In my absence, the enterprise team had installed multiple backup appliances that were replicating data in real time and they had not been accounted for in the BPF filters. In the end, I went from 360 Mbps all day down to 20 Mbps.

Thanks again for the insight and pointers on tracking this down.

- Philip

[Liam Randall]

All hail El_Guapo!*

Most epic post to this list.  EVER.

*you better change your irc handle to this :)