Re: [security-onion] Bro not collecting useful data

[Doug Burks]

Hi Joe,

Are you sure the interface is seeing traffic?  Can you see traffic using tcpdump on the interface?

When you ran Setup, did you let it configure /etc/network/interfaces or did you configure it manually?

Any clues in weird.log?

If you need further help, please send the output of the following (redacting sensitive info as necessary):

sudo sostat

Thanks,

Doug

On Monday, January 28, 2013, Joseph Crain wrote:

I am using SO 12.04.

in /nsm/bro/logs/current I have these bro logs:
communication.log  loaded_scripts.log  notice_policy.log  packet_filter.log  reporter.log  stderr.log  stdout.log  weird.log

Can someone help me figure out why I not getting any useful logs such conn.log, http.log, dns.log, etc...?

Thanks
-Joe

--
You received this message because you are subscribed to the Google Groups "security-onion" group.
To post to this group, send email to securit...@googlegroups.com.
To unsubscribe from this group, send email to security-onio...@googlegroups.com.
Visit this group at http://groups.google.com/group/security-onion?hl=en-US.
For more options, visit https://groups.google.com/groups/opt_out.

--
Doug Burks
http://securityonion.blogspot.com

[Doug Burks]

What kind of traffic are you monitoring? Standard tap/span or are you
perhaps monitoring something like GRE?
Doug

On Mon, Jan 28, 2013 at 7:40 PM, Joseph Crain <josep...@gmail.com> wrote:
> Hi Doug,
>
> I am seeing traffic on the interface, tcpdump sees the traffic. Snort seems fine and we have been using sguil/squert successfully on this box for about a month. I did use setup to configure /etc/network/interfaces.
>
> weird.log looks like protocol anomalies and doesn't mean much to me:
>
> #separator \x09
> #set_separator ,
> #empty_field (empty)
> #unset_field -
> #path weird
> #open 2013-01-29-00-18-10
> #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer
> #types time string addr port addr port string string bool string
> 1359418690.533523 - - - - - unknown_packet_type - F bro
> 1359418690.558112 - - - - - truncated_IP - F bro
> 1359418690.558555 - - - - - bad_IP_checksum - F bro
> 1359418788.702422 - - - - - unknown_protocol_2 - F bro
> 1359418808.956959 - - - - - routing0_hdr - F bro
> 1359418808.957906 - - - - - unknown_routing_type_192 - F bro
> 1359418876.234651 - - - - - unknown_routing_type_116 - F bro
> 1359418876.235465 - - - - - unknown_routing_type_252 - F bro
> 1359419011.049020 - - - - - unknown_routing_type_97 - F bro
> 1359419020.552559 - - - - - unknown_routing_type_3 - F bro
> 1359419088.110369 - - - - - unknown_routing_type_197 - F bro
> 1359419290.538741 - - - - - truncated_IP - F bro
> 1359419290.541518 - - - - - unknown_packet_type - F bro
> 1359419331.417585 - - - - - unknown_routing_type_42 - F bro
> 1359419331.418028 - - - - - unknown_routing_type_84 - F bro
> 1359419344.821757 - - - - - unknown_routing_type_78 - F bro
> 1359419413.708034 - - - - - unknown_protocol_2 - F bro
> 1359419450.642839 - - - - - unknown_routing_type_74 - F bro
> 1359419450.644805 - - - - - unknown_routing_type_129 - F bro
> 1359419455.742163 - - - - - unknown_routing_type_26 - F bro
> 1359419455.742241 - - - - - unknown_routing_type_247 - F bro
>
> Here is my sostat minus the snort/snorby top alerts:
>
> =========================================================================
> Service Status
> =========================================================================
> Status: securityonion
> * sguil server[ OK ]
> Status: HIDS
> * ossec_agent (sguil)[ OK ]
> Status: Bro
> Name Type Host Status Pid Peers Started
> bro standalone localhost running 9628 0 29 Jan 00:18:09
> Status: unraveler-eth1
> * netsniff-ng (full packet data)[ OK ]
> * pcap_agent (sguil)[ OK ]
> * snort_agent-1 (sguil)[ OK ]
> * snort-1 (alert data)[ OK ]
> * barnyard2-1 (spooler, unified2 format)[ OK ]
> * prads (sessions/assets)[ OK ]
> * sancp_agent (sguil)[ OK ]
> * pads_agent (sguil)[ OK ]
> * argus[ OK ]
> * http_agent (sguil)[ OK ]
>
> =========================================================================
> Interface Status
> =========================================================================
> eth0 Link encap:Ethernet HWaddr 00:25:90:a1:ad:06
> inet addr:10.0.1.200 Bcast:10.0.255.255 Mask:255.255.0.0
> inet6 addr: fe80::225:90ff:fea1:ad06/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:217900 errors:0 dropped:0 overruns:0 frame:0
> TX packets:124700 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:26971646 (26.9 MB) TX bytes:40763018 (40.7 MB)
> Memory:fbc20000-fbc40000
>
> eth1 Link encap:Ethernet HWaddr 00:25:90:a1:ad:07
> inet6 addr: fe80::225:90ff:fea1:ad07/64 Scope:Link
> UP BROADCAST RUNNING NOARP PROMISC MULTICAST MTU:1500 Metric:1
> RX packets:71416596 errors:0 dropped:20 overruns:0 frame:0
> TX packets:3 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:63041499983 (63.0 GB) TX bytes:210 (210.0 B)
> Memory:fbc00000-fbc20000
> lo Link encap:Local Loopback
> inet addr:127.0.0.1 Mask:255.0.0.0
> inet6 addr: ::1/128 Scope:Host
> UP LOOPBACK RUNNING MTU:16436 Metric:1
> RX packets:474564 errors:0 dropped:0 overruns:0 frame:0
> TX packets:474564 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:107059934 (107.0 MB) TX bytes:107059934 (107.0 MB)
>
>
> =========================================================================
> Disk Usage
> =========================================================================
> Filesystem Size Used Avail Use% Mounted on
> /dev/sda1 454G 21G 410G 5% /
> udev 16G 4.0K 16G 1% /dev
> tmpfs 6.3G 388K 6.3G 1% /run
> none 5.0M 0 5.0M 0% /run/lock
> none 16G 0 16G 0% /run/shm
> /dev/md0 3.6T 2.7T 814G 77% /nsm/sensor_data
>
> =========================================================================
> Network Sockets
> =========================================================================
> COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
> sshd 1258 root 3u IPv4 8457 0t0 TCP *:22 (LISTEN)
> sshd 1258 root 4u IPv6 8459 0t0 TCP *:22 (LISTEN)
> mysqld 1432 mysql 10u IPv4 194 0t0 TCP 127.0.0.1:3306 (LISTEN)
> mysqld 1432 mysql 88u IPv4 222603 0t0 TCP 127.0.0.1:3306->127.0.0.1:43417 (ESTABLISHED)
> mysqld 1432 mysql 92u IPv4 216971 0t0 TCP 127.0.0.1:3306->127.0.0.1:43429 (ESTABLISHED)
> mysqld 1432 mysql 94u IPv4 220845 0t0 TCP 127.0.0.1:3306->127.0.0.1:43434 (ESTABLISHED)
> mysqld 1432 mysql 499u IPv4 274609 0t0 TCP 127.0.0.1:3306->127.0.0.1:44047 (ESTABLISHED)
> mysqld 1432 mysql 501u IPv4 275510 0t0 TCP 127.0.0.1:3306->127.0.0.1:44036 (ESTABLISHED)
> /usr/sbin 1693 root 4u IPv4 10436 0t0 TCP *:443 (LISTEN)
> /usr/sbin 1693 root 5u IPv4 10439 0t0 TCP *:9876 (LISTEN)
> /usr/sbin 1693 root 6u IPv4 10441 0t0 TCP *:3154 (LISTEN)
> /usr/sbin 1693 root 7u IPv4 10445 0t0 TCP *:444 (LISTEN)
> /usr/sbin 1727 www-data 4u IPv4 10436 0t0 TCP *:443 (LISTEN)
> /usr/sbin 1727 www-data 5u IPv4 10439 0t0 TCP *:9876 (LISTEN)
> /usr/sbin 1727 www-data 6u IPv4 10441 0t0 TCP *:3154 (LISTEN)
> /usr/sbin 1727 www-data 7u IPv4 10445 0t0 TCP *:444 (LISTEN)
> /usr/sbin 1728 www-data 4u IPv4 10436 0t0 TCP *:443 (LISTEN)
> /usr/sbin 1728 www-data 5u IPv4 10439 0t0 TCP *:9876 (LISTEN)
> /usr/sbin 1728 www-data 6u IPv4 10441 0t0 TCP *:3154 (LISTEN)
> /usr/sbin 1728 www-data 7u IPv4 10445 0t0 TCP *:444 (LISTEN)
> /usr/sbin 1729 www-data 4u IPv4 10436 0t0 TCP *:443 (LISTEN)
> /usr/sbin 1729 www-data 5u IPv4 10439 0t0 TCP *:9876 (LISTEN)
> /usr/sbin 1729 www-data 6u IPv4 10441 0t0 TCP *:3154 (LISTEN)
> /usr/sbin 1729 www-data 7u IPv4 10445 0t0 TCP *:444 (LISTEN)
> /usr/sbin 1731 www-data 4u IPv4 10436 0t0 TCP *:443 (LISTEN)
> /usr/sbin 1731 www-data 5u IPv4 10439 0t0 TCP *:9876 (LISTEN)
> /usr/sbin 1731 www-data 6u IPv4 10441 0t0 TCP *:3154 (LISTEN)
> /usr/sbin 1731 www-data 7u IPv4 10445 0t0 TCP *:444 (LISTEN)
> /usr/sbin 1731 www-data 33u IPv4 220763 0t0 TCP 127.0.0.1:43417->127.0.0.1:3306 (ESTABLISHED)
> tclsh 2841 root 13u IPv4 18676 0t0 TCP *:7734 (LISTEN)
> tclsh 2841 root 14u IPv4 18677 0t0 TCP *:7736 (LISTEN)
> tclsh 2841 root 15u IPv4 274579 0t0 TCP 127.0.0.1:7736->127.0.0.1:56745 (ESTABLISHED)
> tclsh 2841 root 16u IPv4 274594 0t0 TCP 127.0.0.1:7736->127.0.0.1:56746 (ESTABLISHED)
> tclsh 2841 root 17u IPv4 274600 0t0 TCP 127.0.0.1:7736->127.0.0.1:56747 (ESTABLISHED)
> tclsh 2841 root 18u IPv4 274608 0t0 TCP 127.0.0.1:7736->127.0.0.1:56748 (ESTABLISHED)
> tclsh 2841 root 19u IPv4 273676 0t0 TCP 127.0.0.1:7736->127.0.0.1:56751 (ESTABLISHED)
> tclsh 2841 root 20u IPv4 273726 0t0 TCP 127.0.0.1:7736->127.0.0.1:56752 (ESTABLISHED)
> tclsh 2841 root 25u IPv4 221159 0t0 TCP 127.0.0.1:7734->127.0.0.1:45487 (ESTABLISHED)
> tclsh 2841 root 26u IPv4 226293 0t0 TCP 127.0.0.1:7734->127.0.0.1:45553 (ESTABLISHED)
> ntpd 3862 ntp 16u IPv4 19839 0t0 UDP *:123
> ntpd 3862 ntp 17u IPv6 19840 0t0 UDP *:123
> ntpd 3862 ntp 18u IPv4 19846 0t0 UDP 127.0.0.1:123
> ntpd 3862 ntp 19u IPv4 19847 0t0 UDP 10.0.1.200:123
> ntpd 3862 ntp 20u IPv6 19848 0t0 UDP [fe80::225:90ff:fea1:ad06]:123
> ntpd 3862 ntp 21u IPv6 19849 0t0 UDP [fe80::225:90ff:fea1:ad07]:123
> ntpd 3862 ntp 22u IPv6 19850 0t0 UDP [::1]:123
> tclsh 9575 root 3u IPv4 261941 0t0 TCP 127.0.0.1:56745->127.0.0.1:7736 (ESTABLISHED)
> bro 9628 root 4u IPv4 272807 0t0 UDP 10.0.1.200:34989->10.0.0.8:53
> bro 9637 root 0u IPv4 271052 0t0 TCP *:47760 (LISTEN)
> bro 9637 root 1u IPv6 271053 0t0 TCP *:47760 (LISTEN)
> bro 9637 root 4u IPv4 272807 0t0 UDP 10.0.1.200:34989->10.0.0.8:53
> tclsh 9690 root 3u IPv4 277634 0t0 TCP 127.0.0.1:56746->127.0.0.1:7736 (ESTABLISHED)
> tclsh 9710 root 3u IPv4 277640 0t0 TCP 127.0.0.1:56747->127.0.0.1:7736 (ESTABLISHED)
> tclsh 9710 root 4u IPv4 277641 0t0 TCP 127.0.0.1:8001 (LISTEN)
> tclsh 9710 root 6u IPv4 277708 0t0 TCP 127.0.0.1:8001->127.0.0.1:41187 (ESTABLISHED)
> barnyard2 9762 root 3u IPv4 271112 0t0 TCP 127.0.0.1:41187->127.0.0.1:8001 (ESTABLISHED)
> barnyard2 9762 root 4u IPv4 271115 0t0 TCP 127.0.0.1:44047->127.0.0.1:3306 (ESTABLISHED)
> tclsh 9794 root 3u IPv4 269939 0t0 TCP 127.0.0.1:56748->127.0.0.1:7736 (ESTABLISHED)
> tclsh 9812 root 3u IPv4 269954 0t0 TCP 127.0.0.1:56751->127.0.0.1:7736 (ESTABLISHED)
> tclsh 9850 root 3u IPv4 276757 0t0 TCP 127.0.0.1:56752->127.0.0.1:7736 (ESTABLISHED)
> sshd 19759 ita 8u IPv6 204156 0t0 TCP [::1]:6011 (LISTEN)
> sshd 19759 ita 9u IPv4 204157 0t0 TCP 127.0.0.1:6011 (LISTEN)
> syslog-ng 27498 root 10u IPv4 213618 0t0 TCP *:514 (LISTEN)
> syslog-ng 27498 root 11u IPv4 213619 0t0 UDP *:514
> /usr/sbin 27873 www-data 4u IPv4 10436 0t0 TCP *:443 (LISTEN)
> /usr/sbin 27873 www-data 5u IPv4 10439 0t0 TCP *:9876 (LISTEN)
> /usr/sbin 27873 www-data 6u IPv4 10441 0t0 TCP *:3154 (LISTEN)
> /usr/sbin 27873 www-data 7u IPv4 10445 0t0 TCP *:444 (LISTEN)
> /usr/sbin 27873 www-data 29u IPv4 267212 0t0 TCP 127.0.0.1:34817->127.0.0.1:9306 (ESTABLISHED)
> /usr/sbin 27873 www-data 32u IPv4 267211 0t0 TCP 127.0.0.1:44036->127.0.0.1:3306 (ESTABLISHED)
> /usr/sbin 27874 www-data 4u IPv4 10436 0t0 TCP *:443 (LISTEN)
> /usr/sbin 27874 www-data 5u IPv4 10439 0t0 TCP *:9876 (LISTEN)
> /usr/sbin 27874 www-data 6u IPv4 10441 0t0 TCP *:3154 (LISTEN)
> /usr/sbin 27874 www-data 7u IPv4 10445 0t0 TCP *:444 (LISTEN)
> /usr/sbin 27875 www-data 4u IPv4 10436 0t0 TCP *:443 (LISTEN)
> /usr/sbin 27875 www-data 5u IPv4 10439 0t0 TCP *:9876 (LISTEN)
> /usr/sbin 27875 www-data 6u IPv4 10441 0t0 TCP *:3154 (LISTEN)
> /usr/sbin 27875 www-data 7u IPv4 10445 0t0 TCP *:444 (LISTEN)
> /usr/sbin 27881 www-data 4u IPv4 10436 0t0 TCP *:443 (LISTEN)
> /usr/sbin 27881 www-data 5u IPv4 10439 0t0 TCP *:9876 (LISTEN)
> /usr/sbin 27881 www-data 6u IPv4 10441 0t0 TCP *:3154 (LISTEN)
> /usr/sbin 27881 www-data 7u IPv4 10445 0t0 TCP *:444 (LISTEN)
> /usr/sbin 27882 www-data 4u IPv4 10436 0t0 TCP *:443 (LISTEN)
> /usr/sbin 27882 www-data 5u IPv4 10439 0t0 TCP *:9876 (LISTEN)
> /usr/sbin 27882 www-data 6u IPv4 10441 0t0 TCP *:3154 (LISTEN)
> /usr/sbin 27882 www-data 7u IPv4 10445 0t0 TCP *:444 (LISTEN)
> /usr/sbin 27882 www-data 31u IPv4 222753 0t0 TCP 127.0.0.1:34210->127.0.0.1:9306 (CLOSE_WAIT)
> /usr/sbin 27882 www-data 33u IPv4 220829 0t0 TCP 127.0.0.1:43429->127.0.0.1:3306 (ESTABLISHED)
> /usr/sbin 27883 www-data 4u IPv4 10436 0t0 TCP *:443 (LISTEN)
> /usr/sbin 27883 www-data 5u IPv4 10439 0t0 TCP *:9876 (LISTEN)
> /usr/sbin 27883 www-data 6u IPv4 10441 0t0 TCP *:3154 (LISTEN)
> /usr/sbin 27883 www-data 7u IPv4 10445 0t0 TCP *:444 (LISTEN)
> /usr/sbin 27883 www-data 31u IPv4 222772 0t0 TCP 127.0.0.1:34215->127.0.0.1:9306 (CLOSE_WAIT)
> /usr/sbin 27883 www-data 33u IPv4 222770 0t0 TCP 127.0.0.1:43434->127.0.0.1:3306 (ESTABLISHED)
> searchd 28658 root 6u IPv4 221516 0t0 TCP *:9306 (LISTEN)
> searchd 28658 root 7u IPv4 221517 0t0 TCP *:3307 (LISTEN)
> searchd 28658 root 70u IPv4 271004 0t0 TCP 127.0.0.1:9306->127.0.0.1:34817 (ESTABLISHED)
> sshd 29127 ita 8u IPv6 230386 0t0 TCP [::1]:6010 (LISTEN)
> sshd 29127 ita 9u IPv4 230387 0t0 TCP 127.0.0.1:6010 (LISTEN)
> sshd 29127 ita 11u IPv4 222902 0t0 TCP 127.0.0.1:6010->127.0.0.1:47620 (ESTABLISHED)
> wish 29230 ita 3u IPv4 230394 0t0 TCP 127.0.0.1:47620->127.0.0.1:6010 (ESTABLISHED)
> wish 29230 ita 4u IPv4 227938 0t0 TCP 127.0.0.1:45487->127.0.0.1:7734 (ESTABLISHED)
> sshd 30948 ita 8u IPv6 234585 0t0 TCP [::1]:6012 (LISTEN)
> sshd 30948 ita 9u IPv4 234586 0t0 TCP 127.0.0.1:6012 (LISTEN)
> sshd 30948 ita 11u IPv4 234599 0t0 TCP 127.0.0.1:6012->127.0.0.1:38252 (ESTABLISHED)
> wish 31049 ita 3u IPv4 221827 0t0 TCP 127.0.0.1:38252->127.0.0.1:6012 (ESTABLISHED)
> wish 31049 ita 4u IPv4 233669 0t0 TCP 127.0.0.1:45553->127.0.0.1:7734 (ESTABLISHED)
>
>
> =========================================================================
> IDS Rules Update
> =========================================================================
>
> =========================================================================
> CPU Usage
> =========================================================================
> top - 00:19:13 up 6:05, 3 users, load average: 0.58, 0.39, 0.52
> Tasks: 192 total, 1 running, 190 sleeping, 0 stopped, 1 zombie
> Cpu(s): 4.7%us, 1.1%sy, 0.2%ni, 93.9%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st
> Mem: 32917324k total, 32601604k used, 315720k free, 443756k buffers
> Swap: 16742396k total, 0k used, 16742396k free, 28541156k cached
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 9628 root 20 0 843m 84m 68m S 18 0.3 0:08.87 bro
> 9637 root 25 5 266m 81m 64m S 12 0.3 0:05.84 bro
> 9735 root 20 0 527m 203m 10m S 12 0.6 0:13.09 snort
> 9670 root 20 0 267m 254m 239m S 2 0.8 0:00.65 netsniff-ng
> 9778 sguil 20 0 26256 7580 3748 S 2 0.0 0:00.75 prads
> 9831 sguil 20 0 111m 9612 1160 S 2 0.0 0:01.19 argus
> 1 root 20 0 24460 2368 1356 S 0 0.0 0:02.09 init
> 2 root 20 0 0 0 0 S 0 0.0 0:00.00 kthreadd
> 3 root 20 0 0 0 0 S 0 0.0 0:00.97 ksoftirqd/0
> 4 root 20 0 0 0 0 S 0 0.0 0:05.96 kworker/0:0
> 6 root RT 0 0 0 0 S 0 0.0 0:00.34 migration/0
> 7 root RT 0 0 0 0 S 0 0.0 0:00.06 watchdog/0
> 8 root RT 0 0 0 0 S 0 0.0 0:00.32 migration/1
> 9 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/1:0
> 10 root 20 0 0 0 0 S 0 0.0 0:00.22 ksoftirqd/1
> 12 root RT 0 0 0 0 S 0 0.0 0:00.05 watchdog/1
> 13 root RT 0 0 0 0 S 0 0.0 0:00.32 migration/2
> 14 root 20 0 0 0 0 S 0 0.0 0:03.41 kworker/2:0
> 15 root 20 0 0 0 0 S 0 0.0 0:00.20 ksoftirqd/2
> 16 root RT 0 0 0 0 S 0 0.0 0:00.05 watchdog/2
> 17 root RT 0 0 0 0 S 0 0.0 0:00.32 migration/3
> 18 root 20 0 0 0 0 S 0 0.0 0:01.35 kworker/3:0
> 19 root 20 0 0 0 0 S 0 0.0 0:00.17 ksoftirqd/3
> 20 root RT 0 0 0 0 S 0 0.0 0:00.05 watchdog/3
> 21 root RT 0 0 0 0 S 0 0.0 0:00.32 migration/4
> 22 root 20 0 0 0 0 S 0 0.0 0:04.67 kworker/4:0
> 23 root 20 0 0 0 0 S 0 0.0 0:00.17 ksoftirqd/4
> 24 root RT 0 0 0 0 S 0 0.0 0:00.04 watchdog/4
> 25 root RT 0 0 0 0 S 0 0.0 0:00.32 migration/5
> 26 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/5:0
> 27 root 20 0 0 0 0 S 0 0.0 0:00.16 ksoftirqd/5
> 28 root RT 0 0 0 0 S 0 0.0 0:00.05 watchdog/5
> 29 root RT 0 0 0 0 S 0 0.0 0:00.34 migration/6
> 30 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/6:0
> 31 root 20 0 0 0 0 S 0 0.0 0:00.05 ksoftirqd/6
> 32 root RT 0 0 0 0 S 0 0.0 0:00.04 watchdog/6
> 33 root RT 0 0 0 0 S 0 0.0 0:00.35 migration/7
> 34 root 20 0 0 0 0 S 0 0.0 0:02.30 kworker/7:0
> 35 root 20 0 0 0 0 S 0 0.0 0:00.05 ksoftirqd/7
> 36 root RT 0 0 0 0 S 0 0.0 0:00.04 watchdog/7
> 37 root RT 0 0 0 0 S 0 0.0 0:00.35 migration/8
> 38 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/8:0
> 39 root 20 0 0 0 0 S 0 0.0 0:00.05 ksoftirqd/8
> 40 root RT 0 0 0 0 S 0 0.0 0:00.04 watchdog/8
> 41 root RT 0 0 0 0 S 0 0.0 0:00.35 migration/9
> 42 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/9:0
> 43 root 20 0 0 0 0 S 0 0.0 0:00.04 ksoftirqd/9
> 44 root RT 0 0 0 0 S 0 0.0 0:00.04 watchdog/9
> 45 root RT 0 0 0 0 S 0 0.0 0:00.35 migration/10
> 46 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/10:0
> 47 root 20 0 0 0 0 S 0 0.0 0:00.04 ksoftirqd/10
> 48 root RT 0 0 0 0 S 0 0.0 0:00.06 watchdog/10
> 49 root RT 0 0 0 0 S 0 0.0 0:00.35 migration/11
> 50 root 20 0 0 0 0 S 0 0.0 0:02.30 kworker/11:0
> 51 root 20 0 0 0 0 S 0 0.0 0:00.05 ksoftirqd/11
> 52 root RT 0 0 0 0 S 0 0.0 0:00.05 watchdog/11
> 53 root 0 -20 0 0 0 S 0 0.0 0:00.00 cpuset
> 54 root 0 -20 0 0 0 S 0 0.0 0:00.00 khelper
> 55 root 20 0 0 0 0 S 0 0.0 0:00.00 kdevtmpfs
> 56 root 0 -20 0 0 0 S 0 0.0 0:00.00 netns
> 58 root 20 0 0 0 0 S 0 0.0 0:00.03 sync_supers
> 59 root 20 0 0 0 0 S 0 0.0 0:00.00 bdi-default
> 60 root 0 -20 0 0 0 S 0 0.0 0:00.00 kintegrityd
> 61 root 0 -20 0 0 0 S 0 0.0 0:00.00 kblockd
> 62 root 0 -20 0 0 0 S 0 0.0 0:00.00 ata_sff
> 63 root 20 0 0 0 0 S 0 0.0 0:00.00 khubd
> 64 root 0 -20 0 0 0 S 0 0.0 0:00.00 md
> 65 root 20 0 0 0 0 S 0 0.0 0:00.83 kworker/2:1
> 67 root 20 0 0 0 0 S 0 0.0 0:00.00 khungtaskd
> 68 root 20 0 0 0 0 S 0 0.0 0:06.38 kswapd0
> 69 root 25 5 0 0 0 S 0 0.0 0:00.00 ksmd
> 70 root 39 19 0 0 0 S 0 0.0 0:00.00 khugepaged
> 71 root 20 0 0 0 0 S 0 0.0 0:00.00 fsnotify_mark
> 72 root 20 0 0 0 0 S 0 0.0 0:00.00 ecryptfs-kthrea
> 73 root 0 -20 0 0 0 S 0 0.0 0:00.00 crypto
> 81 root 0 -20 0 0 0 S 0 0.0 0:00.00 kthrotld
> 82 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_0
> 83 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_1
> 84 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_2
> 85 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_3
> 86 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_4
> 87 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_5
> 89 root 20 0 0 0 0 S 0 0.0 0:00.23 kworker/u:3
> 91 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/u:5
> 112 root 0 -20 0 0 0 S 0 0.0 0:00.00 devfreq_wq
> 251 root 20 0 0 0 0 S 0 0.0 0:05.79 kworker/0:2
> 252 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_6
> 291 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_7
> 307 root 0 -20 0 0 0 S 0 0.0 0:00.00 scsi_wq_7
> 314 root 20 0 0 0 0 S 0 0.0 0:04.38 kworker/1:1
> 315 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/7:1
> 319 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/11:1
> 320 root 20 0 0 0 0 S 0 0.0 0:02.24 kworker/10:1
> 321 root 20 0 0 0 0 S 0 0.0 0:03.69 kworker/3:1
> 357 root 20 0 0 0 0 S 0 0.0 0:02.34 kworker/8:1
> 358 root 20 0 0 0 0 S 0 0.0 0:05.40 kworker/5:1
> 359 root 20 0 0 0 0 S 0 0.0 0:00.96 kworker/4:1
> 360 root 20 0 0 0 0 S 0 0.0 0:02.52 kworker/6:1
> 373 root 20 0 0 0 0 S 0 0.0 0:06.47 jbd2/sda1-8
> 374 root 0 -20 0 0 0 S 0 0.0 0:00.00 ext4-dio-unwrit
> 465 root 20 0 17232 636 452 S 0 0.0 0:00.08 upstart-udev-br
> 467 root 20 0 21852 1636 804 S 0 0.0 0:00.04 udevd
> 693 root 0 -20 0 0 0 S 0 0.0 0:00.00 edac-poller
> 733 root 20 0 21848 1200 364 S 0 0.0 0:00.00 udevd
> 734 root 20 0 21848 1156 320 S 0 0.0 0:00.00 udevd
> 736 root 20 0 0 0 0 S 0 0.0 0:02.59 kworker/9:2
> 805 root 20 0 0 0 0 S 0 0.0 0:00.98 jbd2/md0-8
> 808 root 0 -20 0 0 0 S 0 0.0 0:00.00 ext4-dio-unwrit
> 1005 messageb 20 0 23916 984 684 S 0 0.0 0:00.02 dbus-daemon
> 1258 root 20 0 49956 2876 2268 S 0 0.0 0:00.02 sshd
> 1325 root 20 0 15188 400 196 S 0 0.0 0:00.00 upstart-socket-
> 1376 root 20 0 15784 964 800 S 0 0.0 0:00.00 getty
> 1383 root 20 0 15784 956 800 S 0 0.0 0:00.00 getty
> 1390 root 20 0 15784 960 800 S 0 0.0 0:00.00 getty
> 1392 root 20 0 15784 960 800 S 0 0.0 0:00.00 getty
> 1398 root 20 0 15784 960 800 S 0 0.0 0:00.00 getty
> 1409 root 20 0 4328 692 560 S 0 0.0 0:00.00 acpid
> 1413 daemon 20 0 16908 380 220 S 0 0.0 0:00.00 atd
> 1415 root 20 0 19112 1020 780 S 0 0.0 0:00.12 cron
> 1418 root 20 0 15980 732 544 S 0 0.0 0:07.70 irqbalance
> 1432 mysql 20 0 3552m 221m 8292 S 0 0.7 3:53.40 mysqld
> 1444 whoopsie 20 0 195m 5052 3720 S 0 0.0 0:00.03 whoopsie
> 1499 root 20 0 12804 532 348 S 0 0.0 0:00.00 ossec-execd
> 1503 ossec 20 0 14508 2384 800 S 0 0.0 0:02.85 ossec-analysisd
> 1507 root 20 0 4528 548 416 S 0 0.0 0:00.00 ossec-logcollec
> 1527 root 20 0 198m 36m 3688 S 0 0.1 0:57.86 perl
> 1537 root 20 0 5452 1792 640 S 0 0.0 0:16.97 ossec-syscheckd
> 1541 ossec 20 0 13068 836 572 S 0 0.0 0:00.00 ossec-monitord
> 1628 root 20 0 13240 664 496 S 0 0.0 0:00.00 mdadm
> 1693 root 20 0 176m 12m 6596 S 0 0.0 0:00.55 /usr/sbin/apach
> 1707 root 20 0 215m 1936 1684 S 0 0.0 0:00.00 PassengerWatchd
> 1713 root 20 0 288m 2284 1996 S 0 0.0 0:00.13 PassengerHelper
> 1715 root 20 0 108m 8196 2160 S 0 0.0 0:00.04 ruby1.9.1
> 1719 nobody 20 0 165m 4668 3644 S 0 0.0 0:00.04 PassengerLoggin
> 1727 www-data 20 0 366m 96m 6624 S 0 0.3 0:01.75 /usr/sbin/apach
> 1728 www-data 20 0 176m 7636 1320 S 0 0.0 0:00.00 /usr/sbin/apach
> 1729 www-data 20 0 366m 95m 5704 S 0 0.3 0:01.68 /usr/sbin/apach
> 1743 root 20 0 15784 956 800 S 0 0.0 0:00.00 getty
> 1745 root 20 0 0 0 0 S 0 0.0 0:01.46 flush-8:0
> 1746 root 20 0 0 0 0 S 0 0.0 0:10.60 flush-9:0
> 2841 root 20 0 274m 158m 3920 S 0 0.5 1:17.14 tclsh
> 2866 root 20 0 118m 4312 1008 S 0 0.0 0:00.89 tclsh
> 2867 root 20 0 119m 4644 1212 S 0 0.0 0:00.00 tclsh
> 3862 ntp 20 0 37696 2256 1624 S 0 0.0 0:00.85 ntpd
> 3887 www-data 20 0 424m 92m 3812 S 0 0.3 0:25.50 ruby
> 3910 root 20 0 4340 608 516 S 0 0.0 0:00.00 tail
> 8449 root 20 0 7196 680 584 S 0 0.0 0:00.00 tail
> 9223 root 20 0 4400 616 512 S 0 0.0 0:00.00 sh
> 9226 root 20 0 4400 320 216 S 0 0.0 0:00.00 sh
> 9231 root 20 0 4308 352 276 S 0 0.0 0:00.00 sleep
> 9575 root 20 0 40840 5232 2600 S 0 0.0 0:00.00 tclsh
> 9576 root 20 0 7192 616 520 S 0 0.0 0:00.00 tail
> 9619 root 20 0 12332 1524 1284 S 0 0.0 0:00.00 bash
> 9690 root 20 0 36244 5308 3068 S 0 0.0 0:00.04 tclsh
> 9710 root 20 0 35880 4996 3072 S 0 0.0 0:00.04 tclsh
> 9712 root 20 0 7196 616 520 S 0 0.0 0:00.00 tail
> 9762 root 20 0 155m 57m 1848 S 0 0.2 0:19.36 barnyard2
> 9794 root 20 0 35852 4968 3064 S 0 0.0 0:00.08 tclsh
> 9796 root 20 0 7180 360 280 S 0 0.0 0:00.00 cat
> 9812 root 20 0 36992 6224 3104 S 0 0.0 0:00.24 tclsh
> 9850 root 20 0 35896 4932 3044 S 0 0.0 0:00.01 tclsh
> 9852 root 20 0 7196 684 584 S 0 0.0 0:00.00 tail
> 10107 root 20 0 12316 1476 1252 S 0 0.0 0:00.00 sostat
> 10293 root 20 0 17336 1308 916 R 0 0.0 0:00.00 top
> 19004 root 20 0 77492 3592 2768 S 0 0.0 0:00.04 sshd
> 19759 ita 20 0 79980 4364 1192 S 0 0.0 0:01.19 sshd
> 19760 ita 20 0 26504 7740 1692 S 0 0.0 0:00.38 bash
> 19939 root 20 0 43296 1872 1380 S 0 0.0 0:00.02 sudo
> 19972 root 20 0 44932 1428 1092 S 0 0.0 0:00.00 su
> 19973 root 20 0 21032 2268 1692 S 0 0.0 0:00.53 bash
> 27497 root 20 0 26780 440 200 S 0 0.0 0:00.00 syslog-ng
> 27498 root 20 0 70368 4016 2872 S 0 0.0 0:01.10 syslog-ng
> 27499 root 20 0 4400 612 512 S 0 0.0 0:00.00 sh
> 27501 root 20 0 201m 36m 3764 S 0 0.1 0:03.19 perl
> 27873 www-data 20 0 370m 97m 6132 S 0 0.3 0:02.00 /usr/sbin/apach
> 27874 www-data 20 0 366m 95m 5484 S 0 0.3 0:01.69 /usr/sbin/apach
> 27875 www-data 20 0 366m 95m 5484 S 0 0.3 0:01.67 /usr/sbin/apach
> 27882 www-data 20 0 372m 99m 5912 S 0 0.3 0:01.96 /usr/sbin/apach
> 27883 www-data 20 0 369m 96m 5852 S 0 0.3 0:01.91 /usr/sbin/apach
> 28657 root 20 0 102m 5456 204 S 0 0.0 0:00.00 searchd
> 28658 root 20 0 373m 24m 6124 S 0 0.1 0:06.87 searchd
> 28948 root 20 0 77492 3584 2768 S 0 0.0 0:00.04 sshd
> 29127 ita 20 0 77968 2252 1148 S 0 0.0 0:06.96 sshd
> 29128 ita 20 0 26504 7740 1692 S 0 0.0 0:00.37 bash
> 29230 ita 20 0 112m 42m 6328 S 0 0.1 0:17.66 wish
> 30014 ita 20 0 0 0 0 Z 0 0.0 0:01.16 wireshark <defunct>
> 30811 root 20 0 77492 3584 2768 S 0 0.0 0:00.02 sshd
> 30948 ita 20 0 77648 1996 1156 S 0 0.0 0:05.03 sshd
> 30949 ita 20 0 26508 7768 1712 S 0 0.0 0:00.37 bash
> 31049 ita 20 0 111m 41m 6320 S 0 0.1 0:14.26 wish
>
> =========================================================================
> Log Archive
> =========================================================================
> /nsm/sensor_data/lost+found/dailylogs/
> 4.0M .
> 4.0K ./.ssh
> 12K ./.state
>
> /nsm/sensor_data/sohost-eth1/dailylogs/
> 2.6T .
> 41G ./2012-12-15
> 32G ./2012-12-16
> 61G ./2012-12-17
> 64G ./2012-12-18
> 70G ./2012-12-19
> 94G ./2012-12-20
> 96G ./2012-12-21
> 20G ./2012-12-22
> 14G ./2012-12-23
> 19G ./2012-12-24
> 11G ./2012-12-25
> 48G ./2012-12-26
> 53G ./2012-12-27
> 66G ./2012-12-28
> 14G ./2012-12-29
> 10G ./2012-12-30
> 15G ./2012-12-31
> 6.0G ./2013-01-01
> 55G ./2013-01-02
> 59G ./2013-01-03
> 52G ./2013-01-04
> 17G ./2013-01-05
> 13G ./2013-01-06
> 50G ./2013-01-07
> 58G ./2013-01-08
> 74G ./2013-01-09
> 60G ./2013-01-10
> 64G ./2013-01-11
> 19G ./2013-01-12
> 13G ./2013-01-13
> 49G ./2013-01-14
> 59G ./2013-01-15
> 63G ./2013-01-16
> 55G ./2013-01-17
> 42G ./2013-01-18
> 23G ./2013-01-19
> 8.0G ./2013-01-20
> 152G ./2013-01-21
> 617G ./2013-01-22
> 60G ./2013-01-23
> 66G ./2013-01-24
> 73G ./2013-01-25
> 33G ./2013-01-26
> 19G ./2013-01-27
> 76G ./2013-01-28
> 1.2G ./2013-01-29
>
> /nsm/bro/logs/
> 35M .
> 468K ./2012-12-15
> 388K ./2012-12-16
> 580K ./2012-12-17
> 428K ./2012-12-18
> 564K ./2012-12-19
> 576K ./2012-12-20
> 728K ./2012-12-21
> 388K ./2012-12-22
> 528K ./2012-12-23
> 524K ./2012-12-24
> 388K ./2012-12-25
> 784K ./2012-12-26
> 608K ./2012-12-27
> 752K ./2012-12-28
> 452K ./2012-12-29
> 420K ./2012-12-30
> 424K ./2012-12-31
> 456K ./2013-01-01
> 600K ./2013-01-02
> 848K ./2013-01-03
> 684K ./2013-01-04
> 420K ./2013-01-05
> 420K ./2013-01-06
> 568K ./2013-01-07
> 508K ./2013-01-08
> 720K ./2013-01-09
> 528K ./2013-01-10
> 536K ./2013-01-11
> 492K ./2013-01-12
> 452K ./2013-01-13
> 656K ./2013-01-14
> 604K ./2013-01-15
> 492K ./2013-01-16
> 524K ./2013-01-17
> 584K ./2013-01-18
> 388K ./2013-01-19
> 476K ./2013-01-20
> 424K ./2013-01-21
> 600K ./2013-01-22
> 620K ./2013-01-23
> 696K ./2013-01-24
> 468K ./2013-01-25
> 564K ./2013-01-26
> 420K ./2013-01-27
> 1.3M ./2013-01-28
> 24K ./2013-01-29
> 9.6M ./stats
> =========================================================================
> IDS Engine (snort) packet drops
> =========================================================================
> /nsm/sensor_data/unraveler-eth1/snort-1.stats last reported pkt_drop_percent as 0.142
>
> =========================================================================
> pf_ring stats
> =========================================================================
> Appl. Name : <unknown>
> Tot Packets : 56624
> Tot Pkt Lost : 0
> TX: Send Errors : 0
> Reflect: Fwd Errors: 0
> Appl. Name : snort-cluster-77-socket-0
> Tot Packets : 49139
> Tot Pkt Lost : 0
> TX: Send Errors : 0
> Reflect: Fwd Errors: 0
>
> =========================================================================
> Sguil Uncategorized Events
> =========================================================================
> COUNT(*)
> 76103

> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.

> To post to this group, send email to securit...@googlegroups.com.

[Doug Burks]

I just noticed that your eth1 has an IPv6 address, which it shouldn't
have if you used Setup to configure /etc/network/interfaces. Please
send a copy of this file.

Thanks,
Doug

On Mon, Jan 28, 2013 at 7:54 PM, Joseph Crain <josep...@gmail.com> wrote:
> Span port on our WAN switch

[Doug Burks]

You should add the following to the end of the eth1 section:
post-up echo 1 > /proc/sys/net/ipv6/conf/$IFACE/disable_ipv6

That wouldn't cause your original problem, though. Can you send some
raw tcpdump output (appropriately sanitized)?

Thanks,
Doug

On Mon, Jan 28, 2013 at 8:19 PM, Joseph Crain <josep...@gmail.com> wrote:
> Hmmm...It is possible that the IP was changed manually after the setup.
>
> /etc/network/interfaces:
>
> # This configuration was created by the Security Onion setup script. The original network
> # interface configuration file was backed up to /etc/networking/interfaces.bak.
>
> # This file describes the network interfaces available on your system
> # and how to activate them. For more information, see interfaces(5).
>
> # loopback network interface
> auto lo
> iface lo inet loopback
>
> # Management network interface
> auto eth0
> iface eth0 inet static
> address 10.0.1.200
> gateway 10.0.0.2
> netmask 255.255.0.0
> dns-nameservers 10.0.0.8 10.0.0.12
> dns-domain ######.com
> post-up for i in rx tx sg tso ufo gso gro lro; do ethtool -K $IFACE $i off; done
>
> auto eth1
> iface eth1 inet manual
> up ifconfig $IFACE -arp up
> up ip link set $IFACE promisc on
> down ip link set $IFACE promisc off
> down ifconfig $IFACE down
> post-up ethtool -G $IFACE rx 4096; for i in rx tx sg tso ufo gso gro lro; do ethtool -K $IFACE $i off; done

[Liam Randall]

>> Span port on our WAN switch

For you first SO sensor deployment you would probably get the most benefit by placing the sensor on the inside of your firewall. 

Liam

On Wed, Jan 30, 2013 at 11:37 AM, Joseph Crain <josep...@gmail.com> wrote:

Hi Doug,

I disabled IPV6 as suggested.

As a side note, this is the first time I have attempted to sanitize tcpdump output.  Can you comment on my method?

tcpdump -i eth1 -w eth1.pcap
tcprewrite -s ##### -i eth1.pcap -o eth1-sanitized.pcap
tcpdump -r eth1-sanitized.pcap

The following is a snippet of the re-dump output:

16:13:04.571228 IP 22.167.128.167.137 > 22.167.255.242.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
16:13:04.572901 IP 26.115.223.246.16390 > 22.167.129.18.443: Flags [.], seq 155952:157320, ack 1, win 8198, options [nop,nop,TS val 697920656 ecr 554428156], length 1368
16:13:04.573225 IP 22.167.129.18.443 > 26.115.223.246.16390: Flags [.], ack 157320, win 336, options [nop,nop,TS val 554428206 ecr 697920457], length 0
16:13:04.573753 IP 26.115.223.246.16390 > 22.167.129.18.443: Flags [.], seq 157320:158688, ack 1, win 8198, options [nop,nop,TS val 697920656 ecr 554428156], length 1368
16:13:04.574428 IP 115.155.203.33.59983 > 22.167.129.18.443: Flags [S], seq 3051944805, win 65535, options [mss 1460,nop,wscale 1,nop,nop,TS val 1143143416 ecr 0,sackOK,eol], length 0
16:13:04.574471 IP 115.155.203.33.59981 > 22.167.129.18.443: Flags [P.], seq 1:176, ack 1, win 33304, options [nop,nop,TS val 1143143416 ecr 554551814], length 175
16:13:04.574744 IP 22.167.129.18.443 > 115.155.203.33.59983: Flags [S.], seq 1763485194, ack 3051944806, win 8192, options [mss 1460,nop,wscale 8,sackOK,TS val 554551825 ecr 1143143416], length 0
16:13:04.575005 IP 22.167.129.18.443 > 115.155.203.33.59981: Flags [P.], seq 1:139, ack 176, win 260, options [nop,nop,TS val 554551825 ecr 1143143416], length 138
16:13:04.575052 IP 115.155.203.33.59980 > 22.167.129.18.443: Flags [P.], seq 1:176, ack 1, win 33304, options [nop,nop,TS val 1143143416 ecr 554551808], length 175
16:13:04.575526 IP 22.167.129.18.443 > 115.155.203.33.59980: Flags [P.], seq 1:139, ack 176, win 260, options [nop,nop,TS val 554551825 ecr 1143143416], length 138
16:13:04.583793 IP 22.161.250.49.58920 > 86.181.141.105.80: Flags [P.], seq 2186:3240, ack 11724, win 16145, length 1054
16:13:04.603075 IP 22.159.245.1.50077 > 110.171.133.46.443: Flags [S], seq 2994912978, win 8192, options [mss 1460,nop,nop,sackOK], length 0
16:13:04.607833 IP 115.155.203.33.59982 > 22.167.129.18.443: Flags [.], ack 1, win 33304, options [nop,nop,TS val 1143143453 ecr 554551820], length 0
16:13:04.619848 IP 119.99.233.55.80 > 22.159.245.13.52497: Flags [P.], seq 83565:84628, ack 1, win 14, length 1063
16:13:04.625478 IP 22.167.129.206.22 > 22.227.129.91.46319: Flags [P.], seq 2036:2360, ack 421, win 278, length 324
16:13:04.631144 IP 115.155.203.33.59982 > 22.167.129.18.443: Flags [P.], seq 1:176, ack 1, win 33304, options [nop,nop,TS val 1143143474 ecr 554551820], length 175
16:13:04.631349 IP 115.155.203.33.59984 > 22.167.129.18.443: Flags [S], seq 1217500928, win 65535, options [mss 1460,nop,wscale 1,nop,nop,TS val 1143143474 ecr 0,sackOK,eol], length 0
16:13:04.631566 IP 22.167.129.18.443 > 115.155.203.33.59984: Flags [S.], seq 3179792392, ack 1217500929, win 8192, options [mss 1460,nop,wscale 8,sackOK,TS val 554551831 ecr 1143143474], length 0
16:13:04.635687 IP 22.167.129.18.443 > 115.155.203.33.59982: Flags [P.], seq 1:139, ack 176, win 260, options [nop,nop,TS val 554551831 ecr 1143143474], length 138
16:13:04.636319 IP 22.161.223.250.59150 > 22.167.129.206.22: Flags [P.], seq 1:53, ack 180, win 33172, length 52

[Doug Burks]

Have you modified your Bro configuration at all?

Are you using any BPFs?

Have you tried restarting Bro?
sudo broctl restart

Have you tried restarting the box?

Thanks,
Doug

On Wed, Jan 30, 2013 at 11:37 AM, Joseph Crain <josep...@gmail.com> wrote:
> Hi Doug,
>

[Doug Burks]

I'm curious about the following:

0.000000 Reporter::WARNING Template value remaining in
BPFConf filename: /etc/nsm/{{hostname}}-{{interface}}/bpf-bro.conf
/opt/bro/share/bro/securityonion/./bpfconf.bro, line 99
0.000000 Reporter::INFO BPFConf filename set:
/etc/nsm/unraveler-eth1/bpf-bro.conf
/opt/bro/share/bro/securityonion/./bpfconf.bro, line 103
0.000000 Reporter::INFO BPFConf filename set:
/etc/nsm/unraveler-eth1/bpf-bro.conf
/opt/bro/share/bro/securityonion/./bpfconf.bro, line 103

1359685450.766081 - ip or not ip T T

I don't see this on my boxes with the default BPF (no BPF).

What is the output of the following?

ls -alh /etc/nsm/unraveler-eth1/bpf-bro.conf

cat /etc/nsm/unraveler-eth1/bpf-bro.conf

Thanks,
Doug

On Thu, Jan 31, 2013 at 9:31 PM, Joseph Crain <josep...@gmail.com> wrote:
> I can confirm. No changes were made to the bro config prior to troubleshooting. We have not changed the default BPF.
>
> I have made changes in:
> /etc/nsm/pulledpork/disablesid.conf
> /etc/nsm/rules/local.rules
> /etc/nsm/securityonion/autocat.conf
>
> I have also made changes to the mysql database securityonion_db to add additional classification statuses.
>
> If we re-run sosetup, will we loose any of these customizations?
>
> Here is some additional info:
>
> -Running bro -i eth1 manually produces the expected log files in the current folder
>
> -Running broctl nodes shows:
>
> bro - addr=127.0.0.1 aux_scripts= brobase= count=1 env_vars= ether= host=localhost interface=eth1 lb_interfaces= lb_method= lb_procs= name=bro test_mykey= type=standalone zone_id=
>
> -Running broctl diag shows:
>
> [bro]
> No gdb installed.
>
> ==== reporter.log

> #set_separator ,
> #empty_field (empty)
> #unset_field -

> #path reporter
> #open 2013-02-01-02-24-10
> #fields ts level message location
> #types time enum string string
> 0.000000 Reporter::WARNING Template value remaining in BPFConf filename: /etc/nsm/{{hostname}}-{{interface}}/bpf-bro.conf /opt/bro/share/bro/securityonion/./bpfconf.bro, line 99
> 0.000000 Reporter::INFO BPFConf filename set: /etc/nsm/unraveler-eth1/bpf-bro.conf /opt/bro/share/bro/securityonion/./bpfconf.bro, line 103
> 0.000000 Reporter::INFO BPFConf filename set: /etc/nsm/unraveler-eth1/bpf-bro.conf /opt/bro/share/bro/securityonion/./bpfconf.bro, line 103
>
> ==== stderr.log
> listening on eth1, capture length 8192 bytes
>
>
> ==== stdout.log
> unlimited
> unlimited
> unlimited
>
> ==== .cmdline
> -i eth1 -U .status -p broctl -p broctl-live -p standalone -p local -p bro local.bro broctl broctl/standalone broctl/auto
>
> ==== .env_vars
> PATH=/opt/bro/bin:/opt/bro/share/broctl/scripts:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bro/bin
> BROPATH=/nsm/bro/spool/installed-scripts-do-not-touch/site::/nsm/bro/spool/installed-scripts-do-not-touch/auto:/opt/bro/share/bro:/opt/bro/share/bro/policy:/opt/bro/share/bro/site
> CLUSTER_NODE=
>
> ==== .status
> RUNNING [net_run]
>
> ==== No prof.log
>
> ==== packet_filter.log

> #separator \x09
> #set_separator ,
> #empty_field (empty)
> #unset_field -

> #path packet_filter
> #open 2013-02-01-02-24-10
> #fields ts node filter init success
> #types time string string bool bool
> 1359685450.766081 - ip or not ip T T
>
> ==== loaded_scripts.log

> #separator \x09
> #set_separator ,
> #empty_field (empty)
> #unset_field -

> #path loaded_scripts
> #open 2013-02-01-02-24-10
> #fields name
> #types string
> /opt/bro/share/bro/base/init-bare.bro
> /opt/bro/share/bro/base/const.bif.bro
> /opt/bro/share/bro/base/types.bif.bro
> /opt/bro/share/bro/base/strings.bif.bro
> /opt/bro/share/bro/base/bro.bif.bro
> /opt/bro/share/bro/base/reporter.bif.bro
> /opt/bro/share/bro/base/event.bif.bro
> /opt/bro/share/bro/base/frameworks/logging/__load__.bro
> /opt/bro/share/bro/base/frameworks/logging/./main.bro
> /opt/bro/share/bro/base/logging.bif.bro
> /opt/bro/share/bro/base/frameworks/logging/./postprocessors/__load__.bro
> /opt/bro/share/bro/base/frameworks/logging/./postprocessors/./scp.bro
> /opt/bro/share/bro/base/frameworks/logging/./postprocessors/./sftp.bro
> /opt/bro/share/bro/base/frameworks/logging/./writers/ascii.bro
> /opt/bro/share/bro/base/frameworks/logging/./writers/dataseries.bro
> /opt/bro/share/bro/base/frameworks/logging/./writers/elasticsearch.bro
> /opt/bro/share/bro/base/frameworks/logging/./writers/none.bro
> /opt/bro/share/bro/base/frameworks/input/__load__.bro
> /opt/bro/share/bro/base/frameworks/input/./main.bro
> /opt/bro/share/bro/base/input.bif.bro
> /opt/bro/share/bro/base/frameworks/input/./readers/ascii.bro
> /opt/bro/share/bro/base/frameworks/input/./readers/raw.bro
> /opt/bro/share/bro/base/frameworks/input/./readers/benchmark.bro
> /opt/bro/share/bro/base/init-default.bro
> /opt/bro/share/bro/base/utils/site.bro
> /opt/bro/share/bro/base/utils/./patterns.bro
> /opt/bro/share/bro/base/utils/addrs.bro
> /opt/bro/share/bro/base/utils/conn-ids.bro
> /opt/bro/share/bro/base/utils/directions-and-hosts.bro
> /opt/bro/share/bro/base/utils/files.bro
> /opt/bro/share/bro/base/utils/numbers.bro
> /opt/bro/share/bro/base/utils/paths.bro
> /opt/bro/share/bro/base/utils/strings.bro
> /opt/bro/share/bro/base/utils/thresholds.bro
> /opt/bro/share/bro/base/frameworks/notice/__load__.bro
> /opt/bro/share/bro/base/frameworks/notice/./main.bro
> /opt/bro/share/bro/base/frameworks/notice/./weird.bro
> /opt/bro/share/bro/base/frameworks/notice/./actions/drop.bro
> /opt/bro/share/bro/base/frameworks/notice/./actions/email_admin.bro
> /opt/bro/share/bro/base/frameworks/notice/./actions/page.bro
> /opt/bro/share/bro/base/frameworks/notice/./actions/add-geodata.bro
> /opt/bro/share/bro/base/frameworks/notice/./extend-email/hostnames.bro
> /opt/bro/share/bro/base/frameworks/cluster/__load__.bro
> /opt/bro/share/bro/base/frameworks/cluster/./main.bro
> /opt/bro/share/bro/base/frameworks/control/__load__.bro
> /opt/bro/share/bro/base/frameworks/control/./main.bro
> /opt/bro/share/bro/base/frameworks/notice/./actions/pp-alarms.bro
> /opt/bro/share/bro/base/frameworks/dpd/__load__.bro
> /opt/bro/share/bro/base/frameworks/dpd/./main.bro
> /opt/bro/share/bro/base/frameworks/signatures/__load__.bro
> /opt/bro/share/bro/base/frameworks/signatures/./main.bro
> /opt/bro/share/bro/base/frameworks/packet-filter/__load__.bro
> /opt/bro/share/bro/base/frameworks/packet-filter/./main.bro
> /opt/bro/share/bro/base/frameworks/packet-filter/./netstats.bro
> /opt/bro/share/bro/base/frameworks/software/__load__.bro
> /opt/bro/share/bro/base/frameworks/software/./main.bro
> /opt/bro/share/bro/base/frameworks/communication/__load__.bro
> /opt/bro/share/bro/base/frameworks/communication/./main.bro
> /opt/bro/share/bro/base/frameworks/metrics/__load__.bro
> /opt/bro/share/bro/base/frameworks/metrics/./main.bro
> /opt/bro/share/bro/base/frameworks/metrics/./non-cluster.bro
> /opt/bro/share/bro/base/frameworks/intel/__load__.bro
> /opt/bro/share/bro/base/frameworks/intel/./main.bro
> /opt/bro/share/bro/base/frameworks/reporter/__load__.bro
> /opt/bro/share/bro/base/frameworks/reporter/./main.bro
> /opt/bro/share/bro/base/frameworks/tunnels/__load__.bro
> /opt/bro/share/bro/base/frameworks/tunnels/./main.bro
> /opt/bro/share/bro/base/protocols/conn/__load__.bro
> /opt/bro/share/bro/base/protocols/conn/./main.bro
> /opt/bro/share/bro/base/protocols/conn/./contents.bro
> /opt/bro/share/bro/base/protocols/conn/./inactivity.bro
> /opt/bro/share/bro/base/protocols/dns/__load__.bro
> /opt/bro/share/bro/base/protocols/dns/./consts.bro
> /opt/bro/share/bro/base/protocols/dns/./main.bro
> /opt/bro/share/bro/base/protocols/ftp/__load__.bro
> /opt/bro/share/bro/base/protocols/ftp/./utils-commands.bro
> /opt/bro/share/bro/base/protocols/ftp/./main.bro
> /opt/bro/share/bro/base/protocols/ftp/./file-extract.bro
> /opt/bro/share/bro/base/protocols/http/__load__.bro
> /opt/bro/share/bro/base/protocols/http/./main.bro
> /opt/bro/share/bro/base/protocols/http/./utils.bro
> /opt/bro/share/bro/base/protocols/http/./file-ident.bro
> /opt/bro/share/bro/base/protocols/http/./file-hash.bro
> /opt/bro/share/bro/base/protocols/http/./file-extract.bro
> /opt/bro/share/bro/base/protocols/irc/__load__.bro
> /opt/bro/share/bro/base/protocols/irc/./main.bro
> /opt/bro/share/bro/base/protocols/irc/./dcc-send.bro
> /opt/bro/share/bro/base/protocols/smtp/__load__.bro
> /opt/bro/share/bro/base/protocols/smtp/./main.bro
> /opt/bro/share/bro/base/protocols/smtp/./entities.bro
> /opt/bro/share/bro/base/protocols/smtp/./entities-excerpt.bro
> /opt/bro/share/bro/base/protocols/socks/__load__.bro
> /opt/bro/share/bro/base/protocols/socks/./consts.bro
> /opt/bro/share/bro/base/protocols/socks/./main.bro
> /opt/bro/share/bro/base/protocols/ssh/__load__.bro
> /opt/bro/share/bro/base/protocols/ssh/./main.bro
> /opt/bro/share/bro/base/protocols/ssl/__load__.bro
> /opt/bro/share/bro/base/protocols/ssl/./consts.bro
> /opt/bro/share/bro/base/protocols/ssl/./main.bro
> /opt/bro/share/bro/base/protocols/ssl/./mozilla-ca-list.bro
> /opt/bro/share/bro/base/protocols/syslog/__load__.bro
> /opt/bro/share/bro/base/protocols/syslog/./consts.bro
> /opt/bro/share/bro/base/protocols/syslog/./main.bro
> /nsm/bro/spool/installed-scripts-do-not-touch/site/local.bro
> /opt/bro/share/bro/policy/misc/loaded-scripts.bro
> /opt/bro/share/bro/policy/tuning/defaults/__load__.bro
> /opt/bro/share/bro/policy/tuning/defaults/./packet-fragments.bro
> /opt/bro/share/bro/policy/tuning/defaults/./warnings.bro
> /opt/bro/share/bro/policy/frameworks/software/vulnerable.bro
> /opt/bro/share/bro/policy/frameworks/software/version-changes.bro
> /opt/bro/share/bro/policy/protocols/ftp/software.bro
> /opt/bro/share/bro/policy/protocols/smtp/software.bro
> /opt/bro/share/bro/policy/protocols/ssh/software.bro
> /opt/bro/share/bro/policy/protocols/http/software.bro
> /opt/bro/share/bro/policy/protocols/dns/detect-external-names.bro
> /opt/bro/share/bro/policy/protocols/ftp/detect.bro
> /opt/bro/share/bro/policy/protocols/conn/known-hosts.bro
> /opt/bro/share/bro/policy/protocols/conn/known-services.bro
> /opt/bro/share/bro/policy/protocols/ssl/known-certs.bro
> /opt/bro/share/bro/policy/protocols/ssl/cert-hash.bro
> /opt/bro/share/bro/policy/protocols/ssl/validate-certs.bro
> /opt/bro/share/bro/policy/protocols/ssh/geo-data.bro
> /opt/bro/share/bro/policy/protocols/ssh/detect-bruteforcing.bro
> /opt/bro/share/bro/policy/protocols/ssh/interesting-hostnames.bro
> /opt/bro/share/bro/policy/protocols/http/detect-MHR.bro
> /opt/bro/share/bro/policy/protocols/http/detect-sqli.bro
> /opt/bro/share/bro/securityonion/__load__.bro
> /opt/bro/share/bro/securityonion/./hostname.bro
> /opt/bro/share/bro/securityonion/./interface.bro
> /opt/bro/share/bro/securityonion/./bpfconf.bro
> /opt/bro/share/bro/securityonion/./add-interface-to-logs.bro
> /opt/bro/share/bro/securityonion/./load-non-default-scripts.bro
> /opt/bro/share/bro/policy/misc/capture-loss.bro
> /opt/bro/share/bro/securityonion/./conn-add-country.bro
> /opt/bro/share/bro/securityonion/./config-bro.bro
> /opt/bro/share/bro/policy/misc/stats.bro
> /opt/bro/share/bro/broctl/__load__.bro
> /opt/bro/share/bro/broctl/./main.bro
> /opt/bro/share/bro/policy/frameworks/control/controllee.bro
> /opt/bro/share/bro/policy/frameworks/communication/listen.bro
> /opt/bro/share/bro/broctl/standalone.bro
> /nsm/bro/spool/installed-scripts-do-not-touch/auto/standalone-layout.bro
> /opt/bro/share/bro/policy/misc/trim-trace-file.bro
> /opt/bro/share/bro/broctl/auto.bro
> /nsm/bro/spool/installed-scripts-do-not-touch/auto/local-networks.bro
> /nsm/bro/spool/installed-scripts-do-not-touch/auto/broctl-config.bro
>
>
>
> On Thursday, January 31, 2013 4:46:00 PM UTC-8, Mike McLaughlin wrote:

>> It's SPAN on the LAN switch <-> Router port. (I work with Joe)
>>
>> And as far as I know we haven't changed Bro's configuration at all or use any BPFs. The box has been restarted. I'll let Joe verify that though as he has done more configuration than me on this new deployment.
>>
>> Mike

[Seth Hall]

On Feb 3, 2013, at 7:14 AM, Doug Burks <doug....@gmail.com> wrote:

> I'm curious about the following:
>
> 0.000000 Reporter::WARNING Template value remaining in
> BPFConf filename: /etc/nsm/{{hostname}}-{{interface}}/bpf-bro.conf
> /opt/bro/share/bro/securityonion/./bpfconf.bro, line 99

I was curious about that too, but then I noticed that he's running standalone. Doug, are you running standalone mode on the boxes where you aren't seeing that? It's possible the script has a bug in standalone mode.

.Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/

[Doug Burks]

I *think* my personal sensor at home is running standalone and it was
one of the ones I checked. I'll verify next time I'm able.

Thanks,
Doug

[Doug Burks]

Looks correct.

Please try the following:
- edit /opt/bro/share/bro/securityonion/__load__.bro and remove the
following line:
@load ./bpfconf
- sudo broctl install
- sudo broctl restart

Make any difference?

Thanks,
Doug

On Mon, Feb 4, 2013 at 5:38 PM, Mike McLaughlin <obr...@gmail.com> wrote:
> @unraveler:~$ ls -alh /etc/nsm/unraveler-eth1/bpf-bro.conf
> lrwxrwxrwx 1 root root 8 Dec 15 01:08 /etc/nsm/unraveler-eth1/bpf-bro.conf -> bpf.conf
> @unraveler:~$ cat /etc/nsm/unraveler-eth1/bpf-bro.conf
> @unraveler:~$
>
> For reference, /etc/nsm/unraveler-eth1/bpf.conf then links to /etc/nsm/rules/bpf.conf, which is again blank.
>
> Mike

[Doug Burks]

Are you perhaps using multiple PF_RING workers?
https://groups.google.com/d/topic/security-onion/WHc8rgNj9LI/discussion

Doug

On Tue, Feb 5, 2013 at 2:54 PM, Joseph Crain <josep...@gmail.com> wrote:
> Hi Doug,
>
> I removed the line from __load__.bro then ran broctl install and restart. Bro failed to start with the error:
> error in /opt/bro/share/bro/securityonion/./config-bro.bro, line 6: unknown identifier BPFConf::InvalidFilter, at or near "BPFConf::InvalidFilter"
>
> I then commented out line 6 from /opt/bro/share/bro/securityonion/config-bro.bro:
>
> ##! This script reconfigures some of the builtin Bro scripts to suit certain SecurityOnion uses.
>
> redef PacketFilter::all_packets = F;
> redef capture_filters = { ["bpf.conf"] = "ip or not ip" };
>
> ## redef Notice::emailed_types += { BPFConf::InvalidFilter };
>
> After another borctl install & start, bro fired up without error.
>
> The original issue continues, we are still not seeing http.log, conn.log, etc..... in the bro output.

[Vincent]

On Friday, February 15, 2013 12:58:46 PM UTC-5, Joseph Crain wrote:
> Hi Doug,
>

> Nope, I am not using multiple PF_RING workers. I followed the steps in the discussion you linked just to be sure.
>
> I have installed a new SO infrastructure using 2 new VM instances (fwiw, The original problem I posted is on a physical server), one as an SO server and one as a sensor. Bro logging is working correctly on the new SO sensor and showing up in ELSA on the server.
>
> Additionally, I tried several re-installs on a blank VM using standalone and server+sensor (on same box) and in all cases bro behaved as mentioned in the original post...no useful data. For these tests I used the 12.04 ISO and ran updates.
>
> Eventually I will be proficient enough with bro and broctl to troubleshoot the problem. It appears to be a very simple mis-configuration. At this point I am assuming I am missing something really obvious. I will post back if I ever find out the solution and of course welcome any other ideas / troubleshooting help.
>
> Thanks for your help and work on this
>
> -joe

Joseph,

Did you ever discover a resolution to this issue? I am experiencing the same problem.

Thank you,

Vincent