Running SO in distributed VMWare environment

[Timur]

Hi all!

I use L3 Encapsulated Remote Mirroring on dvSwitch. This is to be able to collect traffic from all VMs connected to dvSwitch even if they run on different hosts.

Please help me to figure out how to set SecurityOnion so that it could properly handle such packets. Now I set an IP addr to sniffing interface but it appears as destination host in Snort output.

Config:

auto eth1
iface eth1 inet static
address ХХ.ХХ.ХХ.ХХ
netmask 255.255.255.248
# up ip link set $IFACE promisc on arp off up
# down ip link set $IFACE promisc off down
post-up ethtool -G $IFACE rx 4096; for i in rx tx sg tso ufo gso gro lro; do ethtool -K $IFACE $i off; done
post-up echo 1 > /proc/sys/net/ipv6/conf/$IFACE/disable_ipv6

[Wes]

Timur,

Try having a look here to ensure you have the interface configured correctly:

http://www.makethenmakeinstall.com/2013/02/the-security-onion-nsm-in-an-esxi-vm/#more-195

https://github.com/Security-Onion-Solutions/security-onion/wiki/NetworkConfiguration

If there does not seem to be any issue with the interface configuration, I would try looking at the device you are mirroring traffic from to ensure its configuration is correct.

Thanks,
Wes

[Timur]

пятница, 1 июля 2016 г., 1:41:38 UTC+3 пользователь Wes написал:

Hi Wes!
Thanks for this. I have followed the article, but it appeared that L3 Encapsulation option in VMWare uses GRE to encapsulate mirrored traffic.

Here is what tcpdump shows on eth1 (192.168.255.194 is IP of eth1):

07:16:00.490370 IP 192.168.165.130 > 192.168.255.194: GREv0, key=0x2000000, length 68: IP 192.168.145.12.80 > 192.168.255.74.63228: Flags [.], ack 23656, win 64860, length 0
07:16:00.491173 IP 192.168.165.130 > 192.168.255.194: GREv0, key=0x0, length 1442: IP 192.168.255.74.63228 > 192.168.145.12.80: Flags [.], seq 23656:25036, ack 1, win 64860, length 1380
07:16:00.492403 IP 192.168.165.130 > 192.168.255.194: GREv0, key=0x0, length 1442: IP 192.168.255.74.63228 > 192.168.145.12.80: Flags [.], seq 25036:26416, ack 1, win 64860, length 1380
07:16:00.492481 IP 192.168.165.130 > 192.168.255.194: GREv0, key=0x2000000, length 68: IP 192.168.145.12.80 > 192.168.255.74.63228: Flags [.], ack 26416, win 64860, length 0

However what I see in Sguil is:

Count:4 Event#3.44801 2016-07-01 07:01:05
GPL DNS zone transfer UDP
192.168.165.130 -> 192.168.255.194
IPVer=4 hlen=5 tos=0 dlen=153 ID=13063 flags=0 offset=0 ttl=63 chksum=36146
Protocol: 47 ------------------------------------------------------------------------
Count:1 Event#3.49250 2016-07-01 07:44:04
GPL NETBIOS SMB IPC$ unicode share access
192.168.165.131 -> 192.168.255.194
IPVer=4 hlen=5 tos=0 dlen=168 ID=65269 flags=0 offset=0 ttl=63 chksum=49459
Protocol: 47 ------------------------------------------------------------------------
Count:1 Event#3.64667 2016-07-01 09:48:09
ET CINS Active Threat Intelligence Poor Reputation IP TCP group 87
192.168.165.130 -> 192.168.255.194
IPVer=4 hlen=5 tos=0 dlen=84 ID=31175 flags=0 offset=0 ttl=63 chksum=18103
Protocol: 47 ------------------------------------------------------------------------
Count:1 Event#3.70823 2016-07-01 10:54:41
ET CINS Active Threat Intelligence Poor Reputation IP TCP group 86
192.168.165.130 -> 192.168.255.194
IPVer=4 hlen=5 tos=0 dlen=84 ID=19287 flags=0 offset=0 ttl=63 chksum=29991
Protocol: 47

Looks like Snort doesn't decode GRE packets. But it shouldn't be the case as GRE is supported by default. Any other idea?

[Wes]

Try having a look at the following:

https://groups.google.com/d/msg/security-onion/CcZNJEm94zo/3k4MjxayAQAJ

https://groups.google.com/d/msg/security-onion/wdcPVlZ8WzU/9bCgady2AgAJ

https://groups.google.com/d/msg/security-onion/WRZnlFSndFs/5OA1LdXOBQAJ

Thanks,
Wes

[Timur]

пятница, 1 июля 2016 г., 17:37:41 UTC+3 пользователь Wes написал:

Thanks Wes!
I figured out that Snort properly decodes GRE, but somehow in Sguil I do see GRE-encapsulated data (protocol 47).

$ sudo u2spewfoo /nsm/sensor_data/servername-eth1/snort-1/snort.unified2.1471248777 > ~/temp.txt
$ head ~/temp.txt -n 20
(Event)
sensor id: 0 event id: 1 event second: 1471248826 event microsecond: 797182
sig id: 2012887 gen id: 1 revision: 2 classification: 19
priority: 1 ip source: 192.168.255.74 ip destination: 192.168.145.12
src port: 62836 dest port: 80 protocol: 6 impact_flag: 0 blocked: 0

Packet
sensor id: 0 event id: 1 event second: 1471248826
packet second: 1471248826 packet microsecond: 797182
linktype: 1 packet_length: 1476
[ 0] 00 50 56 8D 31 03 00 10 DB FF 10 00 08 00 45 00 .PV.1.........E.
[ 16] 05 B6 B7 30 00 00 3F 2F 03 EC 0A DC A5 82 0A DC ...0..?/........
[ 32] FF C2 20 00 65 58 00 00 00 00 00 50 56 8D 3D 17 .. .eX.....PV.=.
[ 48] 00 10 DB FF 10 00 08 00 45 00 05 8C C0 BF 40 00 ........E.....@.
[ 64] C7 06 47 9D 0A DC FF 4A 0A DC 91 0C F5 74 00 50 ..G....J.....t.P
[ 80] 13 24 86 FC F6 E3 60 70 50 10 FF FF C6 20 00 00 .$....`pP.... ..

Sguil export:

Count:1 Event#3.1728423 2016-08-15 08:38:08
GPL NETBIOS SMB-DS IPC$ unicode share access
a.b.c.d -> e.f.g.h
IPVer=4 hlen=5 tos=0 dlen=168 ID=2283 flags=0 offset=0 ttl=63 chksum=46910
Protocol: 47 Payload:
20 00 65 58 00 00 00 00 00 50 56 8D 13 08 00 10 .eX.....PV.....
DB FF 10 00 08 00 45 00 00 7E 5B 60 40 00 7E 06 ......E..~[`@.~.

Any ideas what is causing it?

[Wes]

Timur,

I'm sorry, but I can't effectively determine if this is normal behavior or not, as I don't normally use GRE. I'll take a look around and see if I can find anything that may be of assistance.

How are you "exporting" this from Sguil?

Thanks,
Wes

[Timur]

Wes,

I do "Reports-> Export Events". PSA how it looks in interface.
Could Barnyard2 or snort_agent be the case?

[Timur]

среда, 17 августа 2016 г., 16:26:24 UTC+3 пользователь Timur написал:

> Wes,
>
> I do "Reports-> Export Events". PSA how it looks in interface.
> Could Barnyard2 or snort_agent be the case?

Digging further I found that barnyard2 is sending a proper sig but with local ip addresses instead of real ones (see below 192.168.255.194 - ip of eth1).

What were the configure options when building barnyard2? Were there "--enable-gre"?

Connected to localhost
Sending sguild (sock15e6b40) RegisterAgent snort servername-eth1-1 servername-eth1
Listening on port 8101 for barnyard connections.
Sending sguild (sock15e6b40) PING
Sensor Data Rcvd: AgentInfo servername-eth1-1 snort servername-eth1 3 2408890
Sensor Data Rcvd: PONG
PONG received
barnyard connected: sock1602b20 127.0.0.1 40435
Sending sguild (sock15e6b40) SnortStats {3 0.000 4.830 0.260 0.882 684 228.940 7.190 9.026 1327 {2016-08-26 12:30:00}}
Sending sguild (sock15e6b40) BYEventRcvd sock1602b20 0 3 2408891 servername-eth1-1 6044 6044 {2016-08-26 12:33:09} 1 2012887 2 {ET POLICY Http Client Body contains pass= in cleartext} {2016-08-26 12:33:09} 1 policy-violation 182232458 192.168.165.138 182255554 192.168.255.194 47 4 5 0 1462 40190 0 0 63 7702 {} {} {} {} {} {} {} {} {} {} {} {} {} {} {} {} {}

[Wes]

Try having a look here:

https://groups.google.com/d/msg/security-onion/CcZNJEm94zo/vYV6SiraAwAJ

Thanks,
Wes

[Timur]

пятница, 26 августа 2016 г., 17:41:18 UTC+3 пользователь Wes написал:

Compiling Barnyard2 with '--enable-gre' did the trick. Now Sguil shows real IPs.

[armiofone]

Timur,

Can you share how you recompiled with the --enable-gre option?

[Timur]

вторник, 13 сентября 2016 г., 0:50:09 UTC+3 пользователь armiofone написал:

Hi armiofone,

following this:
https://github.com/firnsy/barnyard2/blob/master/doc/INSTALL

taken source here: https://github.com/firnsy/barnyard2/releases

# cd barnyard2_src
# ./autogen.sh
# ./configure --enable-gre --with-tcl=/usr/lib/tcl8.6/ --prefix=/root/barnyard2/install/
# make
# make install

[armiofone]

Trying that on the latest version of SO and isn't finding the tcl library in /usr/lib/tcl8.6.