monitor ethernet interface dropping all packets after running sosetup

[Austin Murphy]

Hello,

Thanks in advance for any assistance in helping me understand my problem here.

On a fresh install of Security Onion, before I run sosetup I can see traffic on my monitor interface with tcpdump, but then after I run sosetup and let it configure my network interfaces, I no longer see any of the traffic that I just saw. It looks like now my monitor interface is dropping all traffic at the interface. I'm a little puzzled why.

eth2 is my

ajm@nsm:~$ ifconfig eth2
eth2 Link encap:Ethernet HWaddr c8:3a:35:de:f0:8e
UP BROADCAST RUNNING NOARP PROMISC MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:74818 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Interrupt:21 Base address:0x800

ajm@nsm:~$

ajm@nsm:~$ sudo tcpdump -i eth2
tcpdump: WARNING: eth2: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel
24146 packets dropped by interface
ajm@nsm:~$

ajm@nsm:~$ cat /etc/network/interfaces
# This configuration was created by the Security Onion setup script. The original network
# interface configuration file was backed up to /etc/networking/interfaces.bak.

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# loopback network interface
auto lo
iface lo inet loopback

# Management network interface
auto eth1
iface eth1 inet dhcp

auto eth0
iface eth0 inet manual
up ip link set $IFACE promisc on arp off up
down ip link set $IFACE promisc off down
post-up ethtool -G $IFACE rx ; for i in rx tx sg tso ufo gso gro lro; do ethtool -K $IFACE $i off; done
post-up echo 1 > /proc/sys/net/ipv6/conf/$IFACE/disable_ipv6

auto eth2
iface eth2 inet manual
up ip link set $IFACE promisc on arp off up
down ip link set $IFACE promisc off down
post-up ethtool -G $IFACE rx ; for i in rx tx sg tso ufo gso gro lro; do ethtool -K $IFACE $i off; done
post-up echo 1 > /proc/sys/net/ipv6/conf/$IFACE/disable_ipv6

I'm not exactly sure how to troubleshoot this.

Anybody have any thoughts?

Thanks,
Austin

[Doug Burks]

Hi Austin,

What kind of NIC is eth2?

What is it connected to? Tap or span port?

Please run the following command:

sudo sostat-redacted

sostat-redacted will automatically redact any IPv4/IPv6/MAC addresses,
but there may be additional sensitive info that you still need to
redact manually.

Attach the output to your email in plain text format (.txt) OR use a
service like http://pastebin.com.

> --
> You received this message because you are subscribed to the Google Groups "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at http://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/d/optout.



--
Doug Burks

[Austin Murphy]

Thanks for the response Doug.

I've attached the output to sosetup-redacted.

eth2 is a TP-Link - http://www.amazon.com/gp/product/B0034CSUZ8/ref=oh_details_o03_s00_i00?ie=UTF8&psc=1

if you're looking at the output from sosetup, eth0 is my management interface and it's working fine.

eth1 and eth2 are my monitor interfaces. I have them connected to each side of a network throwing star tap http://www.amazon.com/Throwing-Star-LAN-Tap-device/dp/B00BK4VMCK

I was planning on bridging them and then running sosetup against the bridged interface to get bi-directional traffic on a single bridged interface.

When I re-enable the network-manager and remove the lines that sosetup puts into /etc/network/interfaces, then I can see traffic on both eth1 and eth2 with tcpdump.

[Doug Burks]

Your TP-Link NIC may be reacting badly to some of the ethtool settings
in /etc/network/interfaces:
https://code.google.com/p/security-onion/wiki/NetworkConfiguration

What kind of NICs are eth0 and eth1? Is eth1 a TP-Link as well?

If you just purchased this equipment and can return it, you might want
to consider replacing your two monitoring NICs with a single Intel
NIC, and replacing the Throwing Star with either a Dualcomm tap or a
Microtik switch:
https://code.google.com/p/security-onion/wiki/Hardware#Packets

These should "just work" with no bridging required and will save you
time and effort.

If you really have to get your current NICs working, then here are
some things to try.

What is the output of the following?
ethtool -k eth2
ethtool -g eth2

You could try manually running the ethtool commands in
/etc/network/interfaces (prefixed with sudo) to see if you get any
errors.

Also, you should consider increasing your RAM, as 2GB is below our
minimum recommendation:
https://code.google.com/p/security-onion/wiki/Hardware#RAM

[Austin Murphy]

Ok so this might be a hardware issue then.

ajm@nsm:~$ ethtool -k eth1
Offload parameters for eth1:
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: off

ajm@nsm:~$ ethtool -g eth1
Ring parameters for eth1:

Cannot get device ring settings: Operation not supported
ajm@nsm:~$ sudo ethtool -G eth1 rx ; for i in rx tx sg tso ufo gso gro lro; do ethtool -K eth1 $i off; done
ethtool: bad command line argument(s)
For more information run ethtool -h
Cannot set device rx csum settings: Operation not permitted
Cannot set device tx csum settings: Operation not permitted
Cannot set device scatter-gather settings: Operation not permitted
Cannot set device tcp segmentation offload settings: Operation not permitted
Cannot set device udp large send offload settings: Operation not permitted
Cannot set device generic segmentation offload settings: Operation not permitted
Cannot set device GRO settings: Operation not permitted
Cannot set device flag settings: Operation not permitted
ajm@nsm:~$

[Doug Burks]

Replies inline.

On Sun, May 4, 2014 at 10:46 AM, Austin Murphy <austinj...@gmail.com> wrote:
> Ok so this might be a hardware issue then.
>
> ajm@nsm:~$ ethtool -k eth1
> Offload parameters for eth1:
> rx-checksumming: off
> tx-checksumming: off
> scatter-gather: off
> tcp-segmentation-offload: off
> udp-fragmentation-offload: off
> generic-segmentation-offload: off
> generic-receive-offload: off
> large-receive-offload: off
> rx-vlan-offload: on
> tx-vlan-offload: on
> ntuple-filters: off
> receive-hashing: off

That looks fine.

> ajm@nsm:~$ ethtool -g eth1
> Ring parameters for eth1:
>
> Cannot get device ring settings: Operation not supported

That could be an issue.

> ajm@nsm:~$ sudo ethtool -G eth1 rx ; for i in rx tx sg tso ufo gso gro lro; do ethtool -K eth1 $i off; done
> ethtool: bad command line argument(s)
> For more information run ethtool -h
> Cannot set device rx csum settings: Operation not permitted
> Cannot set device tx csum settings: Operation not permitted
> Cannot set device scatter-gather settings: Operation not permitted
> Cannot set device tcp segmentation offload settings: Operation not permitted
> Cannot set device udp large send offload settings: Operation not permitted
> Cannot set device generic segmentation offload settings: Operation not permitted
> Cannot set device GRO settings: Operation not permitted
> Cannot set device flag settings: Operation not permitted

I think the problem here is that each invocation of "ethtool -K" is
running with sudo. Try this instead :

for i in rx tx sg tso ufo gso gro lro; do sudo ethtool -K eth1 $i off; done

[Austin Murphy]

Operation not supported.

ajm@nsm:~$ for i in rx tx sg tso ufo gso gro lro; do sudo ethtool -K eth1 $i off; done
Cannot set device udp large send offload settings: Operation not supported
ajm@nsm:~$

[Doug Burks]

For clarity, you could unroll the for-loop like this:

sudo ethtool -K eth1 rx off
sudo ethtool -K eth1 tx off
sudo ethtool -K eth1 sg off
sudo ethtool -K eth1 tso off
sudo ethtool -K eth1 ufo off
sudo ethtool -K eth1 gso off
sudo ethtool -K eth1 gro off
sudo ethtool -K eth1 lro off

Any parameters which cause errors you could try removing from
/etc/network/interfaces. You may also need to remove the "ethtool -G"
setting as well. Then reboot and see if that allows the NIC to see
traffic.

Depending on what changes you make to /etc/network/interfaces, that
could cause some side effects:
http://blog.securityonion.net/2011/10/when-is-full-packet-capture-not-full.html

Your best bet may be to replace the TP-Link NICs with Intel NIC(s),
which work flawlessly for most folks.

[Austin Murphy]

So it's ufo that's causing the errors. I took that out of /etc/network/interfaces so that it looks like this:

ajm@nsm:~$ cat /etc/network/interfaces
# This configuration was created by the Security Onion setup script. The original network
# interface configuration file was backed up to /etc/networking/interfaces.bak.

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# loopback network interface
auto lo
iface lo inet loopback

# Management network interface
auto eth0
iface eth0 inet dhcp

auto eth1
iface eth1 inet manual

up ip link set $IFACE promisc on arp off up
down ip link set $IFACE promisc off down

post-up for i in rx tx sg tso gso gro lro; do ethtool -K $IFACE $i off; done
post-up echo 1 > /proc/sys/net/ipv6/conf/$IFACE/disable_ipv6

auto eth2
iface eth2 inet manual
up ip link set $IFACE promisc on arp off up
down ip link set $IFACE promisc off down

post-up for i in rx tx sg tso gso gro lro; do ethtool -K $IFACE $i off; done
post-up echo 1 > /proc/sys/net/ipv6/conf/$IFACE/disable_ipv6

ajm@nsm:~$

Still no luck after a reboot.

I'm going to order some new hardware today on amazon but for the next few days while I tinker I'd like to see if I can get this running so that i can at least start profiling the traffic on my network and find out how much ram I need to run this box. This SO is just for my home lab for educational purposes, so i don't need much out of it.