Feedback needed: VLAN tagged traffic and Snort/Suricata

[Doug Burks]

Summary:

I was doing some testing and noticed that PF_RING would report the
following Bucket Len (AKA Snap Length):

Snort - 1514
Suricata - 1516
Bro - 8192

Bro's default setting of 8192 is high enough to handle normal Ethernet
traffic whether it has VLAN tags or not. For Snort/Suricata, the
default settings are fine for normal Ethernet traffic with no VLAN
tags, but may cause inconsistent results if the traffic has VLAN tags.

I increased Snort's snaplen to 1518 in snort.conf, restarted Snort,
and it fired IDS alerts consistently.

For Suricata, I don't see a snaplen option in the pfring section of
suricata.yaml, so I increased the MTU of my sniffing interface as
follows:
sudo ifconfig eth1 mtu 1502

I then restarted Suricata, verified that PF_RING showed it with a
Bucket Len of 1518, and it fired IDS alerts consistently.

For more background information, please see:
https://groups.google.com/d/topic/security-onion/1sDHn0AwDXc/discussion

Questions (feedback needed!):

1. Are you currently monitoring traffic that contains VLAN tags?

2. Have you ever manually modified the MTU of your monitoring
(sniffing) interface?

3. Have you ever manually modified the snaplen of Snort/Suricata?

4. How should we improve this? A few options:

New installations only - update Setup to ask if you're going to
monitor VLAN tagged traffic and if so configure appropriately

OR

New installations only - update Setup to automatically set MTU/snaplen
to 1518 or higher for ALL new installations?

OR

Existing installations - update the NSM scripts to automatically
update all default MTU/snaplen settings to 1518?

OR

other options?

Thanks in advance for your feedback!


--
Doug Burks

[Kevin Branch]

1. One of my SO installs does directly sniff a VLAN trunk as eth1, and uses Suricata.  I haven't upgraded that system to 14.04 yet but will need to take this issue into consideration when I do.
   
2. I've never needed to tweak the MTU of a sniffing interface.
3. I've not tweaked the snaplen of Snort or Suricata, but my standard procedure is to change Bro's snaplen from 8192 to 1600 to conserve PF_RING memory by adding "redef snaplen = 1600;" to my local.bro file.
4. I suggest for both upgrades and first time installs, that the MTU be automatically adjusted to achieve a Suricata snaplen of 1518, but only if it is not already something higher.  Some folks may be monitoring jumbo frame links.

Kevin

--
Doug Burks

--
Follow Security Onion on Twitter!
https://twitter.com/securityonion
---
You received this message because you are subscribed to the Google Groups "security-onion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
To post to this group, send email to securit...@googlegroups.com.
Visit this group at https://groups.google.com/group/security-onion.
For more options, visit https://groups.google.com/d/optout.

[Doug Burks]

Hi Kevin,

Thanks for the feedback!

Based on your response to #1, did I give the impression that this only
applies to 14.04 and not 12.04? My testing seems to indicate it
applies to both. You might want to check the PF_RING Bucket Len
setting for your Suricata workers as follows:
grep -A20 "Suricata" /proc/net/pf_ring/*eth* |grep "Bucket Len"

--
Doug Burks

[Kevin Branch]

Somehow I did get that impression -- my mistake.  Thanks for pointing that out.  Indeed, Suricata on my SO 12.04 system does show a PF_RING bucket len of 1516.  The system has been generating Suricata alerts, but perhaps I'll start seeing new categories of alerts coming through from there, since I've now applied your recommended change.

Thanks,

Kevin

[Doug Burks]

On Tue, Feb 16, 2016 at 5:12 PM, Kevin Branch
<ke...@branchnetconsulting.com> wrote:
> Somehow I did get that impression -- my mistake. Thanks for pointing that
> out. Indeed, Suricata on my SO 12.04 system does show a PF_RING bucket len
> of 1516. The system has been generating Suricata alerts, but perhaps I'll
> start seeing new categories of alerts coming through from there, since I've
> now applied your recommended change.

Please let me know if you see any difference as I'm trying to gauge
how much of an impact this has. We definitely want to maximize
detection for as many different organizations and networks as
possible.

[Wes]

I'm not currently monitoring VLAN tagged traffic, but I plan to. I agree with what Kevin has suggested:

4.I suggest for both upgrades and first time installs, that the MTU be automatically adjusted to achieve a Suricata snaplen of 1518, but only if it is not already something higher. Some folks may be monitoring jumbo frame links.

Thanks,
Wes

[Brian Kellogg]

Is it possible just to default to the jumbo frame size to catch all possibilities without any adverse affects? Perhaps have three options in the setup; standard (default), VLAN tagged, and Jumbo?

1. No
2. Yes
3. No

[Doug Burks]

I'd be concerned that defaulting to jumbo would waste resources for
most folks. How many folks are monitoring jumbo frames?

Adding standard/VLAN/Jumbo options to Setup is possible, but I have a
feeling some folks are monitoring VLAN tagged traffic without actually
realizing it, so I wonder if we should just default to VLAN tagged
(not that much bigger than "standard" so shouldn't be wasting too many
resources) and then have an option for Jumbo.

Thoughts?

[Victor Julien]

On 18-02-16 12:52, Doug Burks wrote:
> I'd be concerned that defaulting to jumbo would waste resources for
> most folks. How many folks are monitoring jumbo frames?
>
> Adding standard/VLAN/Jumbo options to Setup is possible, but I have a
> feeling some folks are monitoring VLAN tagged traffic without actually
> realizing it, so I wonder if we should just default to VLAN tagged
> (not that much bigger than "standard" so shouldn't be wasting too many
> resources) and then have an option for Jumbo.

I can't speak for the other tools, but for Suricata defaulting to a high
packet size is going to waste resources.

One thing that might be helpful is our lua scripting for stats. Using a
simple script we could probably detect this case fairly reliably.

Some logic like: if more than 10% of the packets is invalid
(decoder.invalid) and most/all packets have a vlan tag (decoder.vlan vs
decoder.pkts) then it's highly likely that the capture length is too small.

I created an example script for inspecting stats here:
https://github.com/inliniac/surilua/blob/master/stats.lua

Cheers,
Victor

>
> Thoughts?
>
> On Wed, Feb 17, 2016 at 9:21 PM, Brian Kellogg <thef...@gmail.com> wrote:
>> Is it possible just to default to the jumbo frame size to catch all possibilities without any adverse affects? Perhaps have three options in the setup; standard (default), VLAN tagged, and Jumbo?
>>
>> 1. No
>> 2. Yes
>> 3. No
>>
>> --
>> Follow Security Onion on Twitter!
>> https://twitter.com/securityonion
>> ---
>> You received this message because you are subscribed to the Google Groups "security-onion" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
>> To post to this group, send email to securit...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/security-onion.
>> For more options, visit https://groups.google.com/d/optout.
>
>
>


--

---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------

[Brian Kellogg]

Thanks Victor for the script. I'm excited to get my installation to version 3 so I can start working with the Lua script engine more.

I agree with Doug that maybe two options are best with the default snaplen size being set to the size of VLAN tagged frames.

Jumbo frames are becoming more prevalent on the LAN with the use of iSCSI and other protocols that benefit from it. Had a SANS instructor not to long ago talk about an attacker hiding and moving laterally in an iSCSI storage network.

[Doug Burks]

Thanks for the feedback, everybody!

Kevin, have you noticed any difference since you made the change on Feb 16?

> --
> Follow Security Onion on Twitter!
> https://twitter.com/securityonion
> ---
> You received this message because you are subscribed to the Google Groups "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at https://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/d/optout.



--

Doug Burks

[Kevin Branch]

Doug,

I don't see a compelling increase in the raw volume of alerts before and after I raised the MTU on the interface receiving tagged packets.  This is the sensor of one of my smallest clients and they don't wide a terribly broad range of alerts anyway.  

I thought I'd also compare the count of unique signatures firing per day in the week before and after the MTU change, but I'm having trouble getting at the data I need to do that.  The data volume is over 40K which seems to be more than ELSA will allow me to get from the web interface, which would have allowed me to get a CSV I could easily manipulate.  Switching over to cli.sh, I'm just not having any luck selecting just the fields I want and joining them into CSV records, even with the help of jq.  Has anyone collected up some good jq-fu examples for parsing the output of sh.cli?  I haven't gotten much farther than piping to "jq ." and then piping that along to grep,etc.   

I'd be happy to assemble a Wiki page about using sh.cli and parsing/selecting its output, if we can amass some good examples as a group.

Kevin

[Doug Burks]

I've created the following page:
https://github.com/Security-Onion-Solutions/security-onion/wiki/VLAN-Traffic

Please feel free to add or modify as necessary.