Sniffing mixed vlan traffic

[Tim Sailer]

Hi all,
  I'm trying to move to the new Ubuntu LTS version that's going to be up in April, but I have an issue. With the new 3.x kernels, libpcap support is broken regarding sniffing vlans. It works perfectly in 2.4 and 2.6 kernels.

Since I saw that netsniff-ng didn't use libpcap, I figured it would work. I'm sniffing traffic outside a firewall that is a mix of multiple vlans and non-vlan-tagged traffic. So, I installed the package, fired it up and... no vlan traffic. I'm sure I'm missing something, but I've googled my fingers off today, and I'm not too proud to ask for help... So, HELP? Can someone please show me what I'm missing? I've tried with no filters, with filters with or without vlans, and I just get results on non vlan frames.

Tim

[Daniel Borkmann]

What filter do you use? Have you tried the bpfc man page (man bpfc) from a
recent netsniff-ng git tree?

Cheers,

Daniel

> Tim
>

[Tim Sailer]

I tried with no filter (in theory, all packets, right?), then  'vlan and dst port 80' converted to bpf with tcpdump -dd. That gives absolutely no results. 'dst port 80' gives me all the untagged packets.

Tim
 

[Daniel Borkmann]

On 02/21/2014 01:15 AM, Tim Sailer wrote:
...

>> What filter do you use? Have you tried the bpfc man page (man bpfc) from a
>> recent netsniff-ng git tree?
>

> I tried with no filter (in theory, all packets, right?), then 'vlan and
> dst port 80' converted to bpf with tcpdump -dd. That gives absolutely no
> results. 'dst port 80' gives me all the untagged packets.

So, without any filter, you should actually get all packets ... vlan tags
might be stripped though due to hw offloading. Do you see packets w/o any
filtering?

[Tim Sailer]

I do see packets, but... I can run tcpdump in the broken mode wherne I can use the filter 'dst port 80 and not vlan' and it will give me every packet that has a vlan tag... (real nice, uhu? ). Comparing those packets with with the netsniff dump without any filter gives me no matching packets.

[Daniel Borkmann]

On 02/21/2014 05:15 PM, Tim Sailer wrote:
> I do see packets, but... I can run tcpdump in the broken mode wherne I can
> use the filter 'dst port 80 and not vlan' and it will give me every packet
> that has a vlan tag... (real nice, uhu? ). Comparing those packets with
> with the netsniff dump without any filter gives me no matching packets.

Sorry for the delay. If I recall and understand you correctly, I think this
seems to be expected behaviour with hardware accelerated vlan. Try:

ethtool -k <dev>
Offload parameters for <dev>:
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: off
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: off <--
tx-vlan-offload: off <--
ntuple-filters: off
receive-hashing: off

If that is on, which could be in your case, then the NIC automatically
strips VLAN headers and just passes meta information onwards inside the
skb (tci). libpcap still does the workaround internally to rebuild the
vlan header once the packet is in user space. netsniff-ng doesn't touch
packet content and therefore leaves the vlan header stripped. When using
high level filters such as "vlan and dst port 80" both, tcpdump/netsniff-ng
will use the internal libpcap expression compiler to transfer that into
BPF. The BPF will be loaded into the kernel and will do filtering before
the packet enters the RX_RING buffer.

tcpdump version 4.2.1
libpcap version 1.2.1

tcpdump -i<dev> -d vlan and dst port 80
(000) ldh [12]
(001) jeq #0x8100 jt 3 jf 2 <-- ETH_P_8021Q
(002) jeq #0x9100 jt 3 jf 22 <-- ETH_P_QINQ1
(003) ldh [16]
(004) jeq #0x86dd jt 5 jf 11 <-- ETH_P_IPV6
(005) ldb [24]
(006) jeq #0x84 jt 9 jf 7
(007) jeq #0x6 jt 9 jf 8
(008) jeq #0x11 jt 9 jf 22
(009) ldh [60]
(010) jeq #0x50 jt 21 jf 22
(011) jeq #0x800 jt 12 jf 22 <-- ETH_P_IP
(012) ldb [27]
(013) jeq #0x84 jt 16 jf 14
(014) jeq #0x6 jt 16 jf 15
(015) jeq #0x11 jt 16 jf 22
(016) ldh [24]
(017) jset #0x1fff jt 22 jf 18
(018) ldxb 4*([18]&0xf)
(019) ldh [x + 20]
(020) jeq #0x50 jt 21 jf 22
(021) ret #65535
(022) ret #0

So as you can see, after loading the ethertype and probing for 0x8100 and
0x9100 we just drop the packet. This filter is being created regardless if
the driver has {rx,tx}-vlan-offload set to on or off. Thus it is a bug in
the libpcap compiler.

Instead, try using the following BPF filter:

(Any VLAN packet)

ld vlanp
jeq #0, drop
ret #-1
drop: ret #0

(Packet with VLAN ID 10)

ld vlanp
jeq #0, drop
ld vlant
jneq #10, drop
ret #-1
drop: ret #0

You can put that code into a file e.g. 'foo', then run 'bpfc foo > bar' and
pass that onwards to netsniff-ng like 'netsniff-ng -f bar ...'.

Further instruction set information you can find either in 'man bpfc' or in
the kernel documentation itself [1].

Hope that helps.

Cheers,

Daniel

[1] http://lingrok.org/xref/linux-net-next/Documentation/networking/filter.txt

[Tim Sailer]

On Tuesday, February 25, 2014 5:19:06 AM UTC-8, Daniel Borkmann wrote:

Instead, try using the following BPF filter:

(Any VLAN packet)

ld vlanp
jeq #0, drop
ret #-1
drop: ret #0

I'm just getting back to this madness... I try to compile that and get an error:

Syntax error at line 1: vlanp! syntax error, unexpected label, expecting '[' or 'M' or '#'!

Is there any way to get both van and 'not vlan' packets this way?

Tim

[Daniel Borkmann]

On 03/11/2014 12:19 AM, Tim Sailer wrote:
> On Tuesday, February 25, 2014 5:19:06 AM UTC-8, Daniel Borkmann wrote:

...

>> Instead, try using the following BPF filter:
>>
>> (Any VLAN packet)
>>
>> ld vlanp
>> jeq #0, drop
>> ret #-1
>> drop: ret #0
>>
>>
> I'm just getting back to this madness... I try to compile that and get an
> error:
>
> Syntax error at line 1: vlanp! syntax error, unexpected label, expecting
> '[' or 'M' or '#'!

$ cat foo

ld vlanp
jeq #0, drop
ret #-1
drop: ret #0

$ bpfc foo
{ 0x20, 0, 0, 0xfffff030 },
{ 0x15, 1, 0, 0x00000000 },
{ 0x6, 0, 0, 0xffffffff },
{ 0x6, 0, 0, 0x00000000 },
$ bpfc -v
bpfc 0.5.8-rc3+ (Ziggomatic), Git id: v0.5.8-rc3-7-ge6033f4

So what version are you using?

> Is there any way to get both van and 'not vlan' packets this way?

Of course, instead of jumping to drop label you could jump to
a different label instead, and continue processing.

> Tim

[Tim Sailer]

The machine I jumped on has 0.5.7

Tim