hsflowd export vlan's data

Michał Markiewicz

unread,

Mar 6, 2017, 7:37:20 AM3/6/17

to Host-sFlow

Hello,

I'm trying to configure hsflowd in order to export as well information about VLANs, because the interfaces eth3, 1 , 1.3738 and 1.2577 are virtual. Any suggestions what should I add to the following configuration ?

sflow {
     agent = eth3
     polling = 30
     sampling.1G = 1024
     sampling.10G = 1024
     sampling.40G = 1024
     collector { ip=192.168.200.1 udpport=6343 }
     pcap { dev = eth3 }
     pcap { dev = eth1 }
     pcap { dev = eth1.3738 }
     pcap { dev = eth1.2577 }
}

I have found something in sflow documentation, maybe this could be a solution

/* Extended Flow Data

   Extended data types provide supplimentary information about the
   sampled packet. All applicable extended flow records should be
   included with each flow sample. */

/* Extended Switch Data */
/* opaque = flow_data; enterprise = 0; format = 1001 */
/* Note: For untagged ingress ports, use the assigned vlan and priority
         of the port for the src_vlan and src_priority values.
         For untagged egress ports, use the values for dst_vlan and
         dst_priority that would have been placed in the 802.Q tag
         had the egress port been a tagged member of the VLAN instead
         of an untagged member. */

struct extended_switch {
   unsigned int src_vlan;     /* The 802.1Q VLAN id of incoming frame */
   unsigned int src_priority; /* The 802.1p priority of incoming frame */
   unsigned int dst_vlan;     /* The 802.1Q VLAN id of outgoing frame */
   unsigned int dst_priority; /* The 802.1p priority of outgoing frame */
}

Regards,
Michal

Neil McKee

unread,

Mar 6, 2017, 2:05:57 PM3/6/17

to Michał Markiewicz, Host-sFlow

When you tap "eth1" I would expect you to get the packets for all eth1.* vlans as well, but with 802.1Q headers included. Does that not happen? What are the OS/libpcap versions?

If there is an 802.1Q header present then feeding the output into sflowtool will print lines like "decodedVLAN 3788".

You are right about the extended_switch structure, but it shouldn't be necessary for an end-host like this. It's only necessary when the vlan is implicit, or when routing from one vlan to another, or when the 802.1Q header is missing or has been stripped out by hardware.

Neil

--
You received this message because you are subscribed to the Google Groups "Host-sFlow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to host-sflow+unsubscribe@googlegroups.com.
To post to this group, send email to host-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/host-sflow/2d718863-cb37-4d5d-ae11-b237d4a81ddd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Michał Markiewicz

unread,

Mar 8, 2017, 3:59:37 AM3/8/17

to Host-sFlow, michalmar...@gmail.com

Hello, Thank you for your answer.
Please take a look at attached screenshot (1.png) from wireshark when we can see information about VLAN. However we also notice some strange values of ifIndex (attachment 2.png), do you know what those (1073741823) are meaning ?
Furthermore, is that possible to export sFlow with linux quagga router connected to few ISP which are configured as VLANs?

Regards,

Neil McKee

unread,

Mar 8, 2017, 1:55:24 PM3/8/17

to Michał Markiewicz, Host-sFlow

I don't see the attachment(?)

1073741823 == 0x3FFFFFFF means "internal" as defined here:

http://sflow.org/sflow_version_5.txt

So it's being used for samples that appear to be going to/from the host OS. In your case that probably means to/from the routing engine in the Linux protocol stack, so ingress and egress traffic is being marked as NIC->internal and internal->NIC respectively. The main question is whether you see the 802.1Q headers in both these cases. It depends when they are added/removed in the pipeline.

There should be no problem using this to instrument a Linux+Quagga router, although there are some hsflowd improvements we could make to have it more closely resemble sFlow from a network router, such as probing the forwarding tables using netlink and filling in the extended_router stucture too (prefix, nexthop etc.).

This might be relevant too for extra background:

http://blog.sflow.com/2016/07/internet-router-using-merchant-silicon.html

------
Neil McKee
InMon Corp.
http://www.inmon.com

To view this discussion on the web visit https://groups.google.com/d/msgid/host-sflow/708d028b-e190-456d-855d-b8b5601c93c1%40googlegroups.com.

Paweł Staszewski

unread,

Mar 12, 2017, 8:49:28 AM3/12/17

to Host-sFlow

I have the same question about support for outgoing interface in Linux when acting as a router.

If I install hsflowd and use NFLOG then for exporting sflow I use
iptables -I FORWARD -m statistic --mode random --probability 0.0025 -j NFLOG --nflog-group 5

So if my host is a router and have for example 4 vlans
Each of this vlan have same mac address because vlans are created on only one interface

Then I have in physical system
eth0 - ifindex 1
vlan2 - ifindex 2
vlan3 - ifindex 3
vlan4 - ifindex 4

But sflow eport will show:
in interfaces as separate : 1,2,3,4
out interface as: 1073741823

So it is impossilbe to monitor linux machine with NFLOG and HSFLOWD because there will be wrong report about outgoing traffic.

Currently im using
https://sourceforge.net/projects/ipt-netflow/
For netflow export - this is giving me good interface indexes

It will be really good to have also sflow exporter that is capable of exporting traffic from Linux machines acting as a routers.

Thanks
Paweł

Neil McKee

unread,

Mar 12, 2017, 10:32:45 PM3/12/17

to Paweł Staszewski, Host-sFlow

NFLOG allows you to tap into the Linux stack in a number of different places. From what you say it sounds like tapping at the "-I FORWARD" step only gives you ingress packets. So one option would be to tap at both "-I INPUT" and "-I OUTPUT" instead...

However it may turn out to be cleaner if you turn off the nflog{} module in hsflowd.conf and use the pcap{} module instead. That means you get in+out packets down at the NIC level, irrespective of what iptables and anything else may be doing above. hsflowd will map them to in/out ifIndex numbers using the MAC addresses that appear there. If the kernel is new enough (>=3.19) then hsflowd will use a tiny BPF program running in the kernel so only the samples have to come to use-space.

In hsflowd.conf: "pcap { speed=1- }" will apply packet-sampling to any NIC with a non-zero ifSpeed.

(Note that a regular CentOS 7 OS may only have a 3.08 kernel, but I've had good luck with the "elrepo kernel-ml" approach to installing a 4.x kernel.)

Please let me know what you find.

Neil

--

You received this message because you are subscribed to the Google Groups "Host-sFlow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to host-sflow+unsubscribe@googlegroups.com.
To post to this group, send email to host-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/host-sflow/4095b8a0-966e-4f24-b312-ea504696411c%40googlegroups.com.

Paweł Staszewski

unread,

Mar 13, 2017, 6:12:56 AM3/13/17

to Host-sFlow, pstas...@itcare.pl

Hi

Thank You for reply.

Putting NFLOG in INPUT or OUTPUT will give me only packets that are directed to the host itself.
https://upload.wikimedia.org/wikipedia/commons/3/37/Netfilter-packet-flow.svg

This data is useless if host that need to collect data is mostly forwarding them from interfaces X,Y,Z to interfaces B,C,D
So the FORWARD path is the best choice for monitoring FORWARD path in Linux.
Sure I can change this to RAW table and PREROUTIG

FORWARD path in Linux have also information about from what interface packet comes and what interface has been selected for outgoing traffic

That is why ipt_netflow module is working in Forward path.

About pcap - this is bad idea to run anything in pcap mode with 5-6Mpps - iptables NFLOG will handle that volume of traffic but not pcap.

Neil McKee

unread,

Mar 13, 2017, 2:49:14 PM3/13/17

to Paweł Staszewski, Host-sFlow

Oh, I understand now. We just need to tell hsflowd that it is on a bridge/router and not just an end-host. In other scenarios it picks that up (such as when tapping a hypervisor vswitch or running on Cumulus Linux), but in mod_nflog we will need to add some extra logic, or maybe a config parameter. For now I can give you a workaround. Just change line 162 of mod_nflog.c from:

sp->nflog.ds_options

to:

sp->nflog.ds_options | HSP_SAMPLEOPT_BRIDGE

and recompile again with: "make FEATURES=NFLOG"

(or perhaps "make FEATURES=NFLOG PCAP DOCKER KVM TCP SYSTEMD DBUS" if you want it to do other stuff).

We are currently leaning towards doing this with a config parameter such as "nflog { forwarding=on ... }", but please weigh in if you have suggestions.

Regarding pcap performance, ordinarily I would agree with you but actually the kernel-based BPF sampling is a game-changer, at least for sFlow-generation anyway. Here's a article about what we did in hsflowd:

http://blog.sflow.com/2016/05/berkeley-packet-filter-bpf.html

but it's still better to use NFLOG -I FORWARD for this scenario because that gives you in+out with the same packet-sample and so the sFlow feed will be that much easier to interpret: just one ingress-sampling datasource per interface.

Neil

To view this discussion on the web visit https://groups.google.com/d/msgid/host-sflow/1c86463b-c120-49cb-b9e5-ee20a6a20a32%40googlegroups.com.

Neil McKee

unread,

Mar 13, 2017, 3:33:37 PM3/13/17

to Paweł Staszewski, Host-sFlow

Actually, for the temporary workaround you probably want this for line 162 in nflog.c:

sp->nflog.ds_options | HSP_SAMPLEOPT_BRIDGE | HSP_SAMPLEOPT_IF_POLLER,

so that hsflowd will also send counter-samples for all the external ports.

Paweł Staszewski

unread,

Mar 13, 2017, 3:48:54 PM3/13/17

to Host-sFlow, pstas...@itcare.pl

Thanks

I will test it and will let You know the results.

Thanks
Paweł

Paweł Staszewski

unread,

Mar 13, 2017, 9:33:50 PM3/13/17

to Host-sFlow, pstas...@itcare.pl

Ok it if working now :)
have input and output interfaces and some vlan ids also... weird

Neil McKee

unread,

Mar 13, 2017, 10:48:52 PM3/13/17

to Paweł Staszewski, Host-sFlow

Do you mean that sometimes you get the VLAN and sometimes you don't?

Currently, hsflowd learns interface->vlan from /proc/net/vlan/config, but I don't know how well that ties in with the in+out ifIndex numbers that nflog gets here:

https://github.com/sflow/host-sflow/blob/master/src/Linux/readInterfaces.c#L30-L54

This logging step might shed light:

https://github.com/sflow/host-sflow/blob/master/src/Linux/readInterfaces.c#L30-L54

Which you can see if you run hsflowd like this:

sudo service hsflowd stop

sudo hsflowd -ddd 2>&1 | grep NFLOG

I expect we need to make netlink calls to probe the routing table before we will get the full picture.

Neil

To view this discussion on the web visit https://groups.google.com/d/msgid/host-sflow/cf2b38dd-0a8d-4854-a129-f5fefeb27ef3%40googlegroups.com.

Paweł Staszewski

unread,

Mar 14, 2017, 1:28:52 PM3/14/17

to Host-sFlow, pstas...@itcare.pl

Sorry my mistake - there is no information about vlan tags.
this is only data that is missing in sflow

I have src/dst host, protocol, port, packets, flows, bytes, in ifindex , out ifindex - so all is good.

For sure there is no information about nexthops from routing table , networks , masks - but this is normal i think because hsflowd is not looking into linux route table

For now to obtain information about networks/masks/asn (for BGP stats) im using pmacct with bgp client
So hsflow is exporting data from linux router - this data is received by pmacct that have bgp client session with BGP router (this router is also sflow exporter) - and then i have all information needed for traffic engeenering :)

But sure it will be good if there will be possible to export sflow data including information from route table - less to do on client/collector side.

The best way is to have hsflow that is exporting also ASN,METRIC,PREFERENCES,COMMUNITIES etc :)
But this will need to have some api for quagga / bird - for quagga there is FPM that is providing information from BGP - also via netlink so maybee this is the way :)

If You there will be any tests needed I can help :)

Thanks
Paweł

Paweł Staszewski

unread,

Mar 14, 2017, 1:39:24 PM3/14/17

to Host-sFlow, pstas...@itcare.pl

Also about:
https://github.com/sflow/host-sflow/blob/master/src/Linux/readInterfaces.c#L30-L54

How this is linked to ifindex ?
Cause /proc/net/vlan/config have no ifindex

Thanks
Paweł

Neil McKee

unread,

Mar 14, 2017, 2:35:33 PM3/14/17

to Paweł Staszewski, Host-sFlow

The ifIndex for each interface is looked up here:

https://github.com/sflow/host-sflow/blob/master/src/Linux/readInterfaces.c#L630-L640

Then readVLANs() reads /proc/net/vlan/config, looks up the interface by name and fills in the VLAN:

https://github.com/sflow/host-sflow/blob/master/src/Linux/readInterfaces.c#L44

Theoretically we could infer the VLAN from the nexthop, but I'm curious to know what you see for this:

sudo service hsflowd stop

sudo hsflowd -ddd 2>&1 | grep NFLOG

Is NFLOG giving you the VLAN interface for NFULA_IFINDEX_OUTDEV? If so, we have the full picture and we just need to use it.

Peering with Quagga/Bird will continue to make sense -- even if we read netlink it won't give us the AS-Paths.

To view this discussion on the web visit https://groups.google.com/d/msgid/host-sflow/3f666e8e-3924-4492-9c28-8a83b294cc41%40googlegroups.com.

Paweł Staszewski

unread,

Mar 14, 2017, 3:21:15 PM3/14/17

to Host-sFlow, pstas...@itcare.pl

Here is the output:
got NFLOG msg: bytes_read=1408 nlmsg_len=1016 nlmsg_type=1024 OK=true
NFLOG prefix: in: 22 (phys=0) out: 107 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 35 (phys=0) out: 128 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 113 (phys=0) out: 128 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=3068 nlmsg_len=3068 nlmsg_type=1024 OK=true
NFLOG prefix: in: 134 (phys=0) out: 18 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=2924 nlmsg_len=1404 nlmsg_type=1024 OK=true
NFLOG prefix: in: 76 (phys=0) out: 129 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 97 (phys=0) out: 22 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 114 (phys=0) out: 134 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 146 (phys=0) out: 21 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 134 (phys=0) out: 36 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 118 (phys=0) out: 129 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 89 (phys=0) out: 15 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 111 (phys=0) out: 18 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 113 (phys=0) out: 128 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 113 (phys=0) out: 134 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=3648 nlmsg_len=1512 nlmsg_type=1024 OK=true
NFLOG prefix: in: 134 (phys=0) out: 113 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 129 (phys=0) out: 28 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 26 (phys=0) out: 15 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 128 (phys=0) out: 113 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 129 (phys=0) out: 113 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 111 (phys=0) out: 128 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 15 (phys=0) out: 88 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=3652 nlmsg_len=148 nlmsg_type=1024 OK=true
NFLOG prefix: in: 35 (phys=0) out: 128 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 128 (phys=0) out: 116 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 88 (phys=0) out: 128 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 108 (phys=0) out: 15 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 60 (phys=0) out: 21 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=3340 nlmsg_len=148 nlmsg_type=1024 OK=true
NFLOG prefix: in: 21 (phys=0) out: 18 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 97 (phys=0) out: 129 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 113 (phys=0) out: 128 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=2840 nlmsg_len=1564 nlmsg_type=1024 OK=true
NFLOG prefix: in: 128 (phys=0) out: 113 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 27 (phys=0) out: 15 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 22 (phys=0) out: 79 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 79 (phys=0) out: 15 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 129 (phys=0) out: 101 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 97 (phys=0) out: 129 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 41 (phys=0) out: 134 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=2072 nlmsg_len=1600 nlmsg_type=1024 OK=true
NFLOG prefix: in: 128 (phys=0) out: 97 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 27 (phys=0) out: 128 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 26 (phys=0) out: 15 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 33 (phys=0) out: 134 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=3236 nlmsg_len=3068 nlmsg_type=1024 OK=true
NFLOG prefix: in: 18 (phys=0) out: 66 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 112 (phys=0) out: 134 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=3344 nlmsg_len=1568 nlmsg_type=1024 OK=true
NFLOG prefix: in: 64 (phys=0) out: 134 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 21 (phys=0) out: 128 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 71 (phys=0) out: 15 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=1740 nlmsg_len=1560 nlmsg_type=1024 OK=true
NFLOG prefix: in: 88 (phys=0) out: 128 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 28 (phys=0) out: 129 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=13132 nlmsg_len=11828 nlmsg_type=1024 OK=true
NFLOG prefix: in: 134 (phys=0) out: 64 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 111 (phys=0) out: 15 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 129 (phys=0) out: 45 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 22 (phys=0) out: 27 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 15 (phys=0) out: 113 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 15 (phys=0) out: 66 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 113 (phys=0) out: 15 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=15972 nlmsg_len=14748 nlmsg_type=1024 OK=true
NFLOG prefix: in: 15 (phys=0) out: 92 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 15 (phys=0) out: 52 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 68 (phys=0) out: 15 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 134 (phys=0) out: 113 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=3348 nlmsg_len=2856 nlmsg_type=1024 OK=true
NFLOG prefix: in: 129 (phys=0) out: 113 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 58 (phys=0) out: 134 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 111 (phys=0) out: 15 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 26 (phys=0) out: 22 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=2644 nlmsg_len=1576 nlmsg_type=1024 OK=true
NFLOG prefix: in: 31 (phys=0) out: 128 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 22 (phys=0) out: 113 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=3444 nlmsg_len=1608 nlmsg_type=1024 OK=true
NFLOG prefix: in: 134 (phys=0) out: 123 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 134 (phys=0) out: 97 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 128 (phys=0) out: 113 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=2048 nlmsg_len=456 nlmsg_type=1024 OK=true
NFLOG prefix: in: 68 (phys=0) out: 134 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 15 (phys=0) out: 49 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 115 (phys=0) out: 15 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 41 (phys=0) out: 128 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=4396 nlmsg_len=4228 nlmsg_type=1024 OK=true
NFLOG prefix: in: 18 (phys=0) out: 113 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 140 (phys=0) out: 15 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=31888 nlmsg_len=23508 nlmsg_type=1024 OK=true
NFLOG prefix: in: 128 (phys=0) out: 138 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 27 (phys=0) out: 128 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 129 (phys=0) out: 52 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 33 (phys=0) out: 134 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 128 (phys=0) out: 113 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 128 (phys=0) out: 98 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 129 (phys=0) out: 111 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 128 (phys=0) out: 35 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 27 (phys=0) out: 22 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 133 (phys=0) out: 22 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 15 (phys=0) out: 98 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 92 (phys=0) out: 22 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 129 (phys=0) out: 79 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=3540 nlmsg_len=1560 nlmsg_type=1024 OK=true
NFLOG prefix: in: 36 (phys=0) out: 22 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 15 (phys=0) out: 66 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 18 (phys=0) out: 113 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 134 (phys=0) out: 21 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=3216 nlmsg_len=1608 nlmsg_type=1024 OK=true
NFLOG prefix: in: 133 (phys=0) out: 128 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 79 (phys=0) out: 128 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=3408 nlmsg_len=572 nlmsg_type=1024 OK=true
NFLOG prefix: in: 15 (phys=0) out: 46 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 122 (phys=0) out: 134 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 113 (phys=0) out: 128 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 18 (phys=0) out: 31 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 28 (phys=0) out: 129 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 15 (phys=0) out: 64 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=3056 nlmsg_len=3056 nlmsg_type=1024 OK=true
NFLOG prefix: in: 129 (phys=0) out: 66 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=6472 nlmsg_len=4384 nlmsg_type=1024 OK=true
NFLOG prefix: in: 128 (phys=0) out: 68 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 134 (phys=0) out: 126 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 15 (phys=0) out: 79 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 41 (phys=0) out: 128 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=7172 nlmsg_len=5588 nlmsg_type=1024 OK=true
NFLOG prefix: in: 129 (phys=0) out: 101 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 68 (phys=0) out: 134 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=3068 nlmsg_len=3068 nlmsg_type=1024 OK=true
NFLOG prefix: in: 128 (phys=0) out: 111 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=1508 nlmsg_len=1508 nlmsg_type=1024 OK=true
NFLOG prefix: in: 129 (phys=0) out: 113 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=5916 nlmsg_len=4180 nlmsg_type=1024 OK=true
NFLOG prefix: in: 128 (phys=0) out: 113 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 27 (phys=0) out: 18 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 129 (phys=0) out: 42 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=3052 nlmsg_len=3052 nlmsg_type=1024 OK=true
NFLOG prefix: in: 129 (phys=0) out: 84 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=2156 nlmsg_len=1564 nlmsg_type=1024 OK=true
NFLOG prefix: in: 129 (phys=0) out: 113 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 137 (phys=0) out: 134 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 76 (phys=0) out: 15 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 128 (phys=0) out: 110 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=7732 nlmsg_len=4468 nlmsg_type=1024 OK=true
NFLOG prefix: in: 134 (phys=0) out: 108 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 22 (phys=0) out: 45 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 18 (phys=0) out: 113 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=3700 nlmsg_len=156 nlmsg_type=1024 OK=true
NFLOG prefix: in: 102 (phys=0) out: 129 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 92 (phys=0) out: 128 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 41 (phys=0) out: 128 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 18 (phys=0) out: 123 (phys=0) seq: 0 seq_global: 0 mark: 0
NFLOG prefix: in: 129 (phys=0) out: 72 (phys=0) seq: 0 seq_global: 0 mark: 0
got NFLOG msg: bytes_read=3676 nlmsg_len=1560 nlmsg_type=1024 OK=true

Reply all

Reply to author

Forward