kernel panic in pf

Michal Purzynski

unread,

Mar 28, 2013, 8:27:20 PM3/28/13

to securit...@googlegroups.com

Hi.

I'm getting a kernel panic every time I start something using pf_ring.

Everything is updated, in fact, the installation is a week old (fresh?).

There was a lot of trying, testing, etc and what can I say is:

1. the panic is there everytime. Sometimes seconds sometimes longer, but
it cannot keep up even 5 minutes.
2. netsniff-ng does not cause it.
3. Snort makes the box die quickly, no matter how many snort processes
do I run.
4. The box is a HP 360 G8 with 16GB RAM and dual (single-port) Intel
NICs based on 82599.
5. Traffic to the box is around 2Gbits/sec, divided (uneven) between two
interfaces.

And something _very_ interesting - the box does not die when booted in
UP mode (I've made a test).

Also, made sure that irq balancing daemon isn't running, but it didn't
change anything.

The full output is at

http://pastebin.com/sgLMrr49

If you need more information, I'm here.

Heine Lysemose

unread,

Mar 29, 2013, 3:44:01 AM3/29/13

to securit...@googlegroups.com

Hi

Just to get some more info about your system. Can you run

sudo sostat

Thanks,
Lysemose

--
You received this message because you are subscribed to the Google Groups "security-onion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to security-onion+unsubscribe@googlegroups.com.
To post to this group, send email to security-onion@googlegroups.com.
Visit this group at http://groups.google.com/group/security-onion?hl=en-US.
For more options, visit https://groups.google.com/groups/opt_out.

Michal Purzynski

unread,

Mar 29, 2013, 5:30:57 AM3/29/13

to securit...@googlegroups.com

On 3/29/13 8:44 AM, Heine Lysemose wrote:

Hi

Just to get some more info about your system. Can you run

sudo sostat

Thanks,
Lysemose

Here you are, I have to keep interfaces down, so there's no pf_ring statistics, etc.

=========================================================================
Service Status
=========================================================================
Status: HIDS
* ossec_agent (sguil)[ OK ]
Status: Bro
Name       Type       Host       Status        Pid    Peers Started
manager    manager    10.22.75.93 running       59497 1      29 Mar 09:19:13
proxy      proxy      10.22.75.93 running       59544 1      29 Mar 09:19:15
Status: nsm1.hostname-eth4
* netsniff-ng (full packet data)[ FAIL ]
* pcap_agent (sguil)[ OK ]
* snort_agent-1 (sguil)[ OK ]
* snort_agent-2 (sguil)[ OK ]
* snort_agent-3 (sguil)[ OK ]
* snort_agent-4 (sguil)[ OK ]
* snort_agent-5 (sguil)[ OK ]
* snort_agent-6 (sguil)[ OK ]
* snort_agent-7 (sguil)[ OK ]
* snort_agent-8 (sguil)[ OK ]
* snort-1 (alert data)[ FAIL ]
* stale PID file found, process will be restarted at the next 5-minute interval!
* snort-2 (alert data)[ FAIL ]
* stale PID file found, process will be restarted at the next 5-minute interval!
* snort-3 (alert data)[ FAIL ]
* stale PID file found, process will be restarted at the next 5-minute interval!
* snort-4 (alert data)[ FAIL ]
* stale PID file found, process will be restarted at the next 5-minute interval!
* snort-5 (alert data)[ FAIL ]
* stale PID file found, process will be restarted at the next 5-minute interval!
* snort-6 (alert data)[ FAIL ]
* stale PID file found, process will be restarted at the next 5-minute interval!
* snort-7 (alert data)[ FAIL ]
* stale PID file found, process will be restarted at the next 5-minute interval!
* snort-8 (alert data)[ FAIL ]
* stale PID file found, process will be restarted at the next 5-minute interval!
* barnyard2-1 (spooler, unified2 format)[ OK ]
* barnyard2-2 (spooler, unified2 format)[ OK ]
* barnyard2-3 (spooler, unified2 format)[ OK ]
* barnyard2-4 (spooler, unified2 format)[ OK ]
* barnyard2-5 (spooler, unified2 format)[ OK ]
* barnyard2-6 (spooler, unified2 format)[ OK ]
* barnyard2-7 (spooler, unified2 format)[ OK ]
* barnyard2-8 (spooler, unified2 format)[ OK ]
* prads (sessions/assets)[ FAIL ]
* sancp_agent (sguil)[ OK ]
* pads_agent (sguil)[ OK ]
* argus[ FAIL ]
* stale PID file found, process will be restarted at the next 5-minute interval!
* http_agent (sguil)[ OK ]
Status: nsm1.hostname-eth5
* netsniff-ng (full packet data)[ FAIL ]
* pcap_agent (sguil)[ OK ]
* snort_agent-1 (sguil)[ OK ]
* snort_agent-2 (sguil)[ OK ]
* snort_agent-3 (sguil)[ OK ]
* snort_agent-4 (sguil)[ OK ]
* snort_agent-5 (sguil)[ OK ]
* snort_agent-6 (sguil)[ OK ]
* snort_agent-7 (sguil)[ OK ]
* snort_agent-8 (sguil)[ OK ]
* snort-1 (alert data)[ FAIL ]
* stale PID file found, process will be restarted at the next 5-minute interval!
* snort-2 (alert data)[ FAIL ]
* stale PID file found, process will be restarted at the next 5-minute interval!
* snort-3 (alert data)[ FAIL ]
* stale PID file found, process will be restarted at the next 5-minute interval!
* snort-4 (alert data)[ FAIL ]
* stale PID file found, process will be restarted at the next 5-minute interval!
* snort-5 (alert data)[ FAIL ]
* stale PID file found, process will be restarted at the next 5-minute interval!
* snort-6 (alert data)[ FAIL ]
* stale PID file found, process will be restarted at the next 5-minute interval!
* snort-7 (alert data)[ OK ]
* snort-8 (alert data)[ FAIL ]
* stale PID file found, process will be restarted at the next 5-minute interval!
* barnyard2-1 (spooler, unified2 format)[ OK ]
* barnyard2-2 (spooler, unified2 format)[ OK ]
* barnyard2-3 (spooler, unified2 format)[ OK ]
* barnyard2-4 (spooler, unified2 format)[ OK ]
* barnyard2-5 (spooler, unified2 format)[ OK ]
* barnyard2-6 (spooler, unified2 format)[ OK ]
* barnyard2-7 (spooler, unified2 format)[ OK ]
* barnyard2-8 (spooler, unified2 format)[ OK ]
* prads (sessions/assets)[ FAIL ]
* sancp_agent (sguil)[ OK ]
* pads_agent (sguil)[ OK ]
* argus[ FAIL ]
* stale PID file found, process will be restarted at the next 5-minute interval!
* http_agent (sguil)[ OK ]

=========================================================================
Interface Status
=========================================================================
eth0      Link encap:Ethernet HWaddr ac:16:2d:6f:75:00
          inet addr:10.22.75.93 Bcast:10.22.75.255 Mask:255.255.255.0
          inet6 addr: fe80::ae16:2dff:fe6f:7500/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:503574 errors:0 dropped:0 overruns:0 frame:0
          TX packets:216626 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:100841822 (100.8 MB) TX bytes:131387807 (131.3 MB)
          Interrupt:32

lo        Link encap:Local Loopback
          inet addr:127.0.0.1 Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING MTU:16436 Metric:1
          RX packets:57942 errors:0 dropped:0 overruns:0 frame:0
          TX packets:57942 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:32383523 (32.3 MB) TX bytes:32383523 (32.3 MB)

=========================================================================
Disk Usage
=========================================================================
Filesystem      Size Used Avail Use% Mounted on
/dev/sda2        37G 3.4G   32G 10% /
udev            7.9G 4.0K 7.9G   1% /dev
tmpfs           3.2G 576K 3.2G   1% /run
none            5.0M     0 5.0M   0% /run/lock
none            7.9G     0 7.9G   0% /run/shm
/dev/sda1       460M   59M 378M 14% /boot
/dev/sda4       6.3T 3.3T 2.8T 55% /nsm

=========================================================================
Network Sockets
=========================================================================
COMMAND     PID         USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
rsyslogd    927         root    3u IPv4   1315      0t0 UDP *:514
rsyslogd    927         root    4u IPv6   1316      0t0 UDP *:514
rsyslogd    927         root    9u IPv4    168      0t0 UDP *:34892
rsyslogd    927         root   11u IPv4    169      0t0 UDP *:51558
rsyslogd    927         root   13u IPv4 12311      0t0 UDP *:60189
rsyslogd    927         root   16u IPv4   2989      0t0 UDP *:38805
rsyslogd    927         root   17u IPv4 23742      0t0 UDP *:41503
rsyslogd    927         root   18u IPv4 23743      0t0 UDP *:45054
rsyslogd    927         root   20u IPv4 20966      0t0 UDP *:41474
sshd       1045         root    3r IPv4   1337      0t0 TCP *:22 (LISTEN)
sshd       1045         root    4u IPv6   1339      0t0 TCP *:22 (LISTEN)
mysqld     1217        mysql   10u IPv4    184      0t0 TCP 127.0.0.1:50000 (LISTEN)
ntpd       1442          ntp   16u IPv4   9735      0t0 UDP *:123
ntpd       1442          ntp   17u IPv6   9736      0t0 UDP *:123
ntpd       1442          ntp   18u IPv4   9742      0t0 UDP 127.0.0.1:123
ntpd       1442          ntp   19u IPv4   9743      0t0 UDP 10.22.75.93:123
ntpd       1442          ntp   20u IPv6   9744      0t0 UDP [fe80::ae16:2dff:fe6f:7500]:123
ntpd       1442          ntp   21u IPv6   9745      0t0 UDP [::1]:123
ruby       1817         root    5u IPv4    218      0t0 TCP 10.22.75.93:46842->10.22.75.36:61613 (ESTABLISHED)
master     1956         root   12u IPv4    229      0t0 TCP *:25 (LISTEN)
master     1956         root   13u IPv6    230      0t0 TCP *:25 (LISTEN)
snmpd      1979         snmp    8u IPv4 20811      0t0 UDP 127.0.0.1:161
snmpd      1979         snmp    9u IPv4 20809      0t0 UDP *:39068
searchd    1988 sphinxsearch    6u IPv4 16822      0t0 TCP *:9306 (LISTEN)
searchd    1988 sphinxsearch    7u IPv4 16823      0t0 TCP *:3307 (LISTEN)
/usr/sbin 2098         root    4u IPv4 20819      0t0 TCP *:443 (LISTEN)
/usr/sbin 2098         root    5u IPv4 20822      0t0 TCP *:9876 (LISTEN)
/usr/sbin 2098         root    6u IPv4 20824      0t0 TCP *:444 (LISTEN)
collectd   2132         root    5u IPv4   2036      0t0 TCP 10.22.75.93:47287->10.22.75.142:2003 (ESTABLISHED)
ssh        2197         root    3r IPv4 30022      0t0 TCP 10.22.75.93:52310->10.22.75.99:22 (ESTABLISHED)
ssh        2197         root    4u IPv6 30025      0t0 TCP [::1]:3306 (LISTEN)
ssh        2197         root    5u IPv4 30026      0t0 TCP 127.0.0.1:3306 (LISTEN)
ssh        2197         root    6u IPv4 697982      0t0 TCP 127.0.0.1:3306->127.0.0.1:60558 (ESTABLISHED)
ssh        2197         root    7u IPv4 672375      0t0 TCP 127.0.0.1:3306->127.0.0.1:60580 (ESTABLISHED)
ssh        2197         root    8u IPv4 710718      0t0 TCP 127.0.0.1:3306->127.0.0.1:60582 (ESTABLISHED)
ssh        2197         root    9u IPv4 710720      0t0 TCP 127.0.0.1:3306->127.0.0.1:60584 (ESTABLISHED)
ssh        2197         root   10u IPv4 711696      0t0 TCP 127.0.0.1:3306->127.0.0.1:60586 (ESTABLISHED)
ssh        2197         root   11u IPv4 711697      0t0 TCP 127.0.0.1:3306->127.0.0.1:60588 (ESTABLISHED)
ssh        2197         root   12u IPv4 692864      0t0 TCP 127.0.0.1:3306->127.0.0.1:60590 (ESTABLISHED)
ssh        2197         root   13u IPv4 692865      0t0 TCP 127.0.0.1:3306->127.0.0.1:60592 (ESTABLISHED)
ssh        2197         root   14u IPv4 692866      0t0 TCP 127.0.0.1:3306->127.0.0.1:60594 (ESTABLISHED)
nrpe      10040       nagios    3u IPv4 47455      0t0 TCP *:5666 (LISTEN)
sshd      57909         root    3r IPv4 697813      0t0 TCP 10.22.75.93:22->10.22.75.6:36082 (ESTABLISHED)
sshd      57925   mpurzynski    3u IPv4 697813      0t0 TCP 10.22.75.93:22->10.22.75.6:36082 (ESTABLISHED)
tclsh     59436         root    3u IPv4 697914      0t0 TCP 10.22.75.93:42521->10.22.75.99:7736 (ESTABLISHED)
bro       59497         root    4u IPv4 697917      0t0 UDP 10.22.75.93:50230->10.22.75.40:53
bro       59505         root    0u IPv4 668593      0t0 TCP *:47761 (LISTEN)
bro       59505         root    1u IPv6 668594      0t0 TCP *:47761 (LISTEN)
bro       59505         root    2u IPv4 688535      0t0 TCP 10.22.75.93:47761->10.22.75.93:48374 (ESTABLISHED)
bro       59505         root    4u IPv4 697917      0t0 UDP 10.22.75.93:50230->10.22.75.40:53
bro       59544         root    4u IPv4 697918      0t0 UDP 10.22.75.93:58558->10.22.75.40:53
bro       59551         root    0u IPv4 687888      0t0 TCP 10.22.75.93:48374->10.22.75.93:47761 (ESTABLISHED)
bro       59551         root    1u IPv4 687891      0t0 TCP *:47762 (LISTEN)
bro       59551         root    2u IPv6 687892      0t0 TCP *:47762 (LISTEN)
bro       59551         root    4u IPv4 697918      0t0 UDP 10.22.75.93:58558->10.22.75.40:53
tclsh     59596         root    3u IPv4 697921      0t0 TCP 10.22.75.93:42523->10.22.75.99:7736 (ESTABLISHED)
tclsh     59616         root    3u IPv4 697924      0t0 TCP 10.22.75.93:42524->10.22.75.99:7736 (ESTABLISHED)
tclsh     59616         root    4u IPv4 672281      0t0 TCP 127.0.0.1:8001 (LISTEN)
tclsh     59616         root    6u IPv4 688547      0t0 TCP 127.0.0.1:8001->127.0.0.1:36821 (ESTABLISHED)
tclsh     59634         root    3u IPv4 697927      0t0 TCP 10.22.75.93:42525->10.22.75.99:7736 (ESTABLISHED)
tclsh     59634         root    4u IPv4 697928      0t0 TCP 127.0.0.1:8002 (LISTEN)
tclsh     59652         root    3u IPv4 697931      0t0 TCP 10.22.75.93:42526->10.22.75.99:7736 (ESTABLISHED)
tclsh     59652         root    4u IPv4 697932      0t0 TCP 127.0.0.1:8003 (LISTEN)
tclsh     59670         root    3u IPv4 697935      0t0 TCP 10.22.75.93:42527->10.22.75.99:7736 (ESTABLISHED)
tclsh     59670         root    4u IPv4 697936      0t0 TCP 127.0.0.1:8004 (LISTEN)
tclsh     59688         root    3u IPv4 697939      0t0 TCP 10.22.75.93:42528->10.22.75.99:7736 (ESTABLISHED)
tclsh     59688         root    4u IPv4 697940      0t0 TCP 127.0.0.1:8005 (LISTEN)
tclsh     59706         root    3u IPv4 697943      0t0 TCP 10.22.75.93:42529->10.22.75.99:7736 (ESTABLISHED)
tclsh     59706         root    4u IPv4 697944      0t0 TCP 127.0.0.1:8006 (LISTEN)
tclsh     59724         root    3u IPv4 697948      0t0 TCP 10.22.75.93:42530->10.22.75.99:7736 (ESTABLISHED)
tclsh     59724         root    4u IPv4 697949      0t0 TCP 127.0.0.1:8007 (LISTEN)
tclsh     59742         root    3u IPv4 697952      0t0 TCP 10.22.75.93:42531->10.22.75.99:7736 (ESTABLISHED)
tclsh     59742         root    4u IPv4 697953      0t0 TCP 127.0.0.1:8008 (LISTEN)
barnyard2 59990         root    3u IPv4 705704      0t0 TCP 127.0.0.1:36821->127.0.0.1:8001 (ESTABLISHED)
barnyard2 59990         root    4u IPv4 705707      0t0 TCP 127.0.0.1:60558->127.0.0.1:3306 (ESTABLISHED)
barnyard2 60008         root    3u IPv4 686009      0t0 TCP 127.0.0.1:40896->127.0.0.1:8002 (ESTABLISHED)
barnyard2 60026         root    3u IPv4 705708      0t0 TCP 127.0.0.1:54448->127.0.0.1:8003 (ESTABLISHED)
barnyard2 60043         root    3u IPv4 691439      0t0 TCP 127.0.0.1:55000->127.0.0.1:8004 (ESTABLISHED)
barnyard2 60060         root    3u IPv4 697035      0t0 TCP 127.0.0.1:60688->127.0.0.1:8005 (ESTABLISHED)
barnyard2 60077         root    3u IPv4 706585      0t0 TCP 127.0.0.1:36642->127.0.0.1:8006 (ESTABLISHED)
barnyard2 60094         root    3u IPv4 697985      0t0 TCP 127.0.0.1:59601->127.0.0.1:8007 (ESTABLISHED)
barnyard2 60111         root    3u IPv4 690491      0t0 TCP 127.0.0.1:32772->127.0.0.1:8008 (ESTABLISHED)
tclsh     60143         root    3u IPv4 672288      0t0 TCP 10.22.75.93:42541->10.22.75.99:7736 (ESTABLISHED)
tclsh     60160         root    3u IPv4 672289      0t0 TCP 10.22.75.93:42542->10.22.75.99:7736 (ESTABLISHED)
tclsh     60197         root    3u IPv4 697992      0t0 TCP 10.22.75.93:42543->10.22.75.99:7736 (ESTABLISHED)
tclsh     60241         root    3u IPv4 697998      0t0 TCP 10.22.75.93:42544->10.22.75.99:7736 (ESTABLISHED)
tclsh     60261         root    3u IPv4 698001      0t0 TCP 10.22.75.93:42545->10.22.75.99:7736 (ESTABLISHED)
tclsh     60261         root    4u IPv4 698002      0t0 TCP 127.0.0.1:8101 (LISTEN)
tclsh     60261         root    6u IPv4 703819      0t0 TCP 127.0.0.1:8101->127.0.0.1:52558 (ESTABLISHED)
tclsh     60279         root    3u IPv4 698005      0t0 TCP 10.22.75.93:42546->10.22.75.99:7736 (ESTABLISHED)
tclsh     60279         root    4u IPv4 698006      0t0 TCP 127.0.0.1:8102 (LISTEN)
tclsh     60279         root    6u IPv4 693830      0t0 TCP 127.0.0.1:8102->127.0.0.1:38962 (ESTABLISHED)
tclsh     60297         root    3u IPv4 698009      0t0 TCP 10.22.75.93:42547->10.22.75.99:7736 (ESTABLISHED)
tclsh     60297         root    4u IPv4 698010      0t0 TCP 127.0.0.1:8103 (LISTEN)
tclsh     60297         root    6u IPv4 686979      0t0 TCP 127.0.0.1:8103->127.0.0.1:45279 (ESTABLISHED)
tclsh     60315         root    3u IPv4 698013      0t0 TCP 10.22.75.93:42548->10.22.75.99:7736 (ESTABLISHED)
tclsh     60315         root    4u IPv4 698014      0t0 TCP 127.0.0.1:8104 (LISTEN)
tclsh     60315         root    6u IPv4 703848      0t0 TCP 127.0.0.1:8104->127.0.0.1:45817 (ESTABLISHED)
tclsh     60333         root    3u IPv4 698017      0t0 TCP 10.22.75.93:42549->10.22.75.99:7736 (ESTABLISHED)
tclsh     60333         root    4u IPv4 698018      0t0 TCP 127.0.0.1:8105 (LISTEN)
tclsh     60333         root    6u IPv4 698072      0t0 TCP 127.0.0.1:8105->127.0.0.1:36908 (ESTABLISHED)
tclsh     60351         root    3u IPv4 698021      0t0 TCP 10.22.75.93:42550->10.22.75.99:7736 (ESTABLISHED)
tclsh     60351         root    4u IPv4 698022      0t0 TCP 127.0.0.1:8106 (LISTEN)
tclsh     60351         root    6u IPv4 703859      0t0 TCP 127.0.0.1:8106->127.0.0.1:34581 (ESTABLISHED)
tclsh     60369         root    3u IPv4 698025      0t0 TCP 10.22.75.93:42551->10.22.75.99:7736 (ESTABLISHED)
tclsh     60369         root    4u IPv4 698026      0t0 TCP 127.0.0.1:8107 (LISTEN)
tclsh     60369         root    6u IPv4 703866      0t0 TCP 127.0.0.1:8107->127.0.0.1:34874 (ESTABLISHED)
tclsh     60387         root    3u IPv4 698029      0t0 TCP 10.22.75.93:42552->10.22.75.99:7736 (ESTABLISHED)
tclsh     60387         root    4u IPv4 698030      0t0 TCP 127.0.0.1:8108 (LISTEN)
tclsh     60387         root    6u IPv4 698075      0t0 TCP 127.0.0.1:8108->127.0.0.1:56225 (ESTABLISHED)
barnyard2 61174         root    3u IPv4 693796      0t0 TCP 127.0.0.1:52558->127.0.0.1:8101 (ESTABLISHED)
barnyard2 61174         root    4u IPv4 693799      0t0 TCP 127.0.0.1:60580->127.0.0.1:3306 (ESTABLISHED)
barnyard2 61235         root    3u IPv4 707832      0t0 TCP 127.0.0.1:38962->127.0.0.1:8102 (ESTABLISHED)
barnyard2 61235         root    4u IPv4 707835      0t0 TCP 127.0.0.1:60582->127.0.0.1:3306 (ESTABLISHED)
barnyard2 61296         root    3u IPv4 691685      0t0 TCP 127.0.0.1:45279->127.0.0.1:8103 (ESTABLISHED)
barnyard2 61296         root    4u IPv4 691688      0t0 TCP 127.0.0.1:60584->127.0.0.1:3306 (ESTABLISHED)
barnyard2 61431         root    3u IPv4 691710      0t0 TCP 127.0.0.1:45817->127.0.0.1:8104 (ESTABLISHED)
barnyard2 61431         root    4u IPv4 691713      0t0 TCP 127.0.0.1:60586->127.0.0.1:3306 (ESTABLISHED)
barnyard2 61512         root    3u IPv4 709809      0t0 TCP 127.0.0.1:36908->127.0.0.1:8105 (ESTABLISHED)
barnyard2 61512         root    4u IPv4 709812      0t0 TCP 127.0.0.1:60588->127.0.0.1:3306 (ESTABLISHED)
barnyard2 61609         root    3u IPv4 693889      0t0 TCP 127.0.0.1:34581->127.0.0.1:8106 (ESTABLISHED)
barnyard2 61609         root    4u IPv4 693892      0t0 TCP 127.0.0.1:60590->127.0.0.1:3306 (ESTABLISHED)
barnyard2 61670         root    3u IPv4 691775      0t0 TCP 127.0.0.1:34874->127.0.0.1:8107 (ESTABLISHED)
barnyard2 61670         root    4u IPv4 691778      0t0 TCP 127.0.0.1:60592->127.0.0.1:3306 (ESTABLISHED)
barnyard2 61731         root    3u IPv4 693903      0t0 TCP 127.0.0.1:56225->127.0.0.1:8108 (ESTABLISHED)
barnyard2 61731         root    4u IPv4 693906      0t0 TCP 127.0.0.1:60594->127.0.0.1:3306 (ESTABLISHED)
tclsh     61850         root    3u IPv4 701357      0t0 TCP 10.22.75.93:42570->10.22.75.99:7736 (ESTABLISHED)
tclsh     61910         root    3u IPv4 713926      0t0 TCP 10.22.75.93:42571->10.22.75.99:7736 (ESTABLISHED)
/usr/sbin 61996     www-data    4u IPv4 20819      0t0 TCP *:443 (LISTEN)
/usr/sbin 61996     www-data    5u IPv4 20822      0t0 TCP *:9876 (LISTEN)
/usr/sbin 61996     www-data    6u IPv4 20824      0t0 TCP *:444 (LISTEN)
/usr/sbin 61997     www-data    4u IPv4 20819      0t0 TCP *:443 (LISTEN)
/usr/sbin 61997     www-data    5u IPv4 20822      0t0 TCP *:9876 (LISTEN)
/usr/sbin 61997     www-data    6u IPv4 20824      0t0 TCP *:444 (LISTEN)
/usr/sbin 61998     www-data    4u IPv4 20819      0t0 TCP *:443 (LISTEN)
/usr/sbin 61998     www-data    5u IPv4 20822      0t0 TCP *:9876 (LISTEN)
/usr/sbin 61998     www-data    6u IPv4 20824      0t0 TCP *:444 (LISTEN)
/usr/sbin 61999     www-data    4u IPv4 20819      0t0 TCP *:443 (LISTEN)
/usr/sbin 61999     www-data    5u IPv4 20822      0t0 TCP *:9876 (LISTEN)
/usr/sbin 61999     www-data    6u IPv4 20824      0t0 TCP *:444 (LISTEN)
/usr/sbin 62000     www-data    4u IPv4 20819      0t0 TCP *:443 (LISTEN)
/usr/sbin 62000     www-data    5u IPv4 20822      0t0 TCP *:9876 (LISTEN)
/usr/sbin 62000     www-data    6u IPv4 20824      0t0 TCP *:444 (LISTEN)
tclsh     62112         root    3u IPv4 698096      0t0 TCP 10.22.75.93:42572->10.22.75.99:7736 (ESTABLISHED)

=========================================================================
IDS Rules Update
=========================================================================
Fri Mar 29 07:01:01 UTC 2013
Backing up current downloaded.rules file before it gets overwritten.
Cleaning up downloaded.rules backup files older than 30 days.
Backing up current local.rules file before it gets overwritten.
Cleaning up local.rules backup files older than 30 days.
Sleeping for 5 minutes to allow master time to download new rules.
Copying rules from nsmserver1.hostname.
Restarting Barnyard2.
Restarting: nsm1.hostname-eth4
* stopping: barnyard2-1 (spooler, unified2 format)[ OK ]
* starting: barnyard2-1 (spooler, unified2 format)[ OK ]
* stopping: barnyard2-2 (spooler, unified2 format)[ OK ]
* starting: barnyard2-2 (spooler, unified2 format)[ OK ]
* stopping: barnyard2-3 (spooler, unified2 format)[ OK ]
* starting: barnyard2-3 (spooler, unified2 format)[ OK ]
* stopping: barnyard2-4 (spooler, unified2 format)[ OK ]
* starting: barnyard2-4 (spooler, unified2 format)[ OK ]
* stopping: barnyard2-5 (spooler, unified2 format)[ OK ]
* starting: barnyard2-5 (spooler, unified2 format)[ OK ]
* stopping: barnyard2-6 (spooler, unified2 format)[ OK ]
* starting: barnyard2-6 (spooler, unified2 format)[ OK ]
* stopping: barnyard2-7 (spooler, unified2 format)[ OK ]
* starting: barnyard2-7 (spooler, unified2 format)[ OK ]
* stopping: barnyard2-8 (spooler, unified2 format)[ OK ]
* starting: barnyard2-8 (spooler, unified2 format)[ OK ]
Restarting: nsm1.hostname-eth5
* stopping: barnyard2-1 (spooler, unified2 format)[ OK ]
* starting: barnyard2-1 (spooler, unified2 format)[ OK ]
* stopping: barnyard2-2 (spooler, unified2 format)[ OK ]
* starting: barnyard2-2 (spooler, unified2 format)[ OK ]
* stopping: barnyard2-3 (spooler, unified2 format)[ OK ]
* starting: barnyard2-3 (spooler, unified2 format)[ OK ]
* stopping: barnyard2-4 (spooler, unified2 format)[ OK ]
* starting: barnyard2-4 (spooler, unified2 format)[ OK ]
* stopping: barnyard2-5 (spooler, unified2 format)[ OK ]
* starting: barnyard2-5 (spooler, unified2 format)[ OK ]
* stopping: barnyard2-6 (spooler, unified2 format)[ OK ]
* starting: barnyard2-6 (spooler, unified2 format)[ OK ]
* stopping: barnyard2-7 (spooler, unified2 format)[ OK ]
* starting: barnyard2-7 (spooler, unified2 format)[ OK ]
* stopping: barnyard2-8 (spooler, unified2 format)[ OK ]
* starting: barnyard2-8 (spooler, unified2 format)[ OK ]
Restarting IDS Engine.
Restarting: nsm1.hostname-eth4
* stopping: snort-1 (alert data) (not running)[ WARN ]
    - stale PID file found, deleting!
* starting: snort-1 (alert data)[ OK ]
* stopping: snort-2 (alert data) (not running)[ WARN ]
    - stale PID file found, deleting!
* starting: snort-2 (alert data)[ OK ]
* stopping: snort-3 (alert data) (not running)[ WARN ]
    - stale PID file found, deleting!
* starting: snort-3 (alert data)[ OK ]
* stopping: snort-4 (alert data) (not running)[ WARN ]
    - stale PID file found, deleting!
* starting: snort-4 (alert data)[ OK ]
* stopping: snort-5 (alert data) (not running)[ WARN ]
    - stale PID file found, deleting!
* starting: snort-5 (alert data)[ OK ]
* stopping: snort-6 (alert data)[ FAIL ]
* starting: snort-6 (alert data) (already running)[ WARN ]
* stopping: snort-7 (alert data) (not running)[ WARN ]
    - stale PID file found, deleting!
* starting: snort-7 (alert data)[ OK ]
* stopping: snort-8 (alert data) (not running)[ WARN ]
    - stale PID file found, deleting!
* starting: snort-8 (alert data)[ OK ]
Restarting: nsm1.hostname-eth5
* stopping: snort-1 (alert data) (not running)[ WARN ]
    - stale PID file found, deleting!
* starting: snort-1 (alert data)[ OK ]
* stopping: snort-2 (alert data) (not running)[ WARN ]
    - stale PID file found, deleting!
* starting: snort-2 (alert data)[ OK ]
* stopping: snort-3 (alert data) (not running)[ WARN ]
    - stale PID file found, deleting!
* starting: snort-3 (alert data)[ OK ]
* stopping: snort-4 (alert data) (not running)[ WARN ]
    - stale PID file found, deleting!
* starting: snort-4 (alert data)[ OK ]
* stopping: snort-5 (alert data) (not running)[ WARN ]
    - stale PID file found, deleting!
* starting: snort-5 (alert data)[ OK ]
* stopping: snort-6 (alert data) (not running)[ WARN ]
    - stale PID file found, deleting!
* starting: snort-6 (alert data)[ OK ]
* stopping: snort-7 (alert data)[ FAIL ]
* starting: snort-7 (alert data) (already running)[ WARN ]
* stopping: snort-8 (alert data) (not running)[ WARN ]
    - stale PID file found, deleting!
* starting: snort-8 (alert data)[ OK ]

=========================================================================
CPU Usage
=========================================================================
top - 09:21:52 up 8:59, 1 user, load average: 0.80, 1.02, 0.92
Tasks: 300 total,   1 running, 299 sleeping,   0 stopped,   0 zombie
Cpu(s): 0.8%us, 1.9%sy, 0.2%ni, 97.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 16394156k total, 1367668k used, 15026488k free,   167852k buffers
Swap:   976636k total,        0k used,   976636k free,   299952k cached

PID USER      PR NI VIRT RES SHR S %CPU %MEM    TIME+ COMMAND
59551 root      25   5 62832 19m 952 S   10 0.1   0:14.61 bro
59505 root      25   5 133m 19m 936 S    8 0.1   0:15.14 bro
61670 root      20   0 129m 31m 1776 S    2 0.2   0:05.18 barnyard2
62695 root      20   0 17468 1404 916 R    2 0.0   0:00.01 top
    1 root      20   0 24336 2324 1364 S    0 0.0   0:03.48 init
    2 root      20   0     0    0    0 S    0 0.0   0:00.00 kthreadd
    3 root      20   0     0    0    0 S    0 0.0   0:00.01 ksoftirqd/0
    5 root      20   0     0    0    0 S    0 0.0   0:00.24 kworker/u:0
    6 root      RT   0     0    0    0 S    0 0.0   0:00.07 migration/0
    7 root      RT   0     0    0    0 S    0 0.0   0:00.04 watchdog/0
    8 root      RT   0     0    0    0 S    0 0.0   0:00.07 migration/1
    9 root      20   0     0    0    0 S    0 0.0   0:00.20 kworker/1:0
   10 root      20   0     0    0    0 S    0 0.0   0:00.01 ksoftirqd/1
   12 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/1
   13 root      RT   0     0    0    0 S    0 0.0   0:00.07 migration/2
   14 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/2:0
   15 root      20   0     0    0    0 S    0 0.0   0:00.00 ksoftirqd/2
   16 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/2
   17 root      RT   0     0    0    0 S    0 0.0   0:00.07 migration/3
   18 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/3:0
   19 root      20   0     0    0    0 S    0 0.0   0:00.01 ksoftirqd/3
   20 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/3
   21 root      RT   0     0    0    0 S    0 0.0   0:00.07 migration/4
   22 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/4:0
   23 root      20   0     0    0    0 S    0 0.0   0:00.01 ksoftirqd/4
   24 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/4
   25 root      RT   0     0    0    0 S    0 0.0   0:00.07 migration/5
   26 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/5:0
   27 root      20   0     0    0    0 S    0 0.0   0:00.01 ksoftirqd/5
   28 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/5
   29 root      RT   0     0    0    0 S    0 0.0   0:00.07 migration/6
   31 root      20   0     0    0    0 S    0 0.0   0:00.01 ksoftirqd/6
   32 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/6
   33 root      RT   0     0    0    0 S    0 0.0   0:00.07 migration/7
   34 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/7:0
   35 root      20   0     0    0    0 S    0 0.0   0:00.00 ksoftirqd/7
   36 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/7
   37 root      RT   0     0    0    0 S    0 0.0   0:00.07 migration/8
   38 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/8:0
   39 root      20   0     0    0    0 S    0 0.0   0:00.01 ksoftirqd/8
   40 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/8
   41 root      RT   0     0    0    0 S    0 0.0   0:00.07 migration/9
   42 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/9:0
   43 root      20   0     0    0    0 S    0 0.0   0:00.00 ksoftirqd/9
   44 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/9
   45 root      RT   0     0    0    0 S    0 0.0   0:00.07 migration/10
   46 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/10:0
   47 root      20   0     0    0    0 S    0 0.0   0:00.01 ksoftirqd/10
   48 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/10
   49 root      RT   0     0    0    0 S    0 0.0   0:00.07 migration/11
   50 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/11:0
   51 root      20   0     0    0    0 S    0 0.0   0:00.01 ksoftirqd/11
   52 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/11
   53 root      RT   0     0    0    0 S    0 0.0   0:00.08 migration/12
   54 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/12:0
   55 root      20   0     0    0    0 S    0 0.0   0:00.02 ksoftirqd/12
   56 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/12
   57 root      RT   0     0    0    0 S    0 0.0   0:00.20 migration/13
   58 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/13:0
   59 root      20   0     0    0    0 S    0 0.0   0:00.11 ksoftirqd/13
   60 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/13
   61 root      RT   0     0    0    0 S    0 0.0   0:00.19 migration/14
   62 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/14:0
   63 root      20   0     0    0    0 S    0 0.0   0:00.16 ksoftirqd/14
   64 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/14
   65 root      RT   0     0    0    0 S    0 0.0   0:00.14 migration/15
   66 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/15:0
   67 root      20   0     0    0    0 S    0 0.0   0:00.18 ksoftirqd/15
   68 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/15
   69 root      RT   0     0    0    0 S    0 0.0   0:00.47 migration/16
   71 root      20   0     0    0    0 S    0 0.0   0:00.09 ksoftirqd/16
   72 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/16
   73 root      RT   0     0    0    0 S    0 0.0   0:00.10 migration/17
   74 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/17:0
   75 root      20   0     0    0    0 S    0 0.0   0:00.04 ksoftirqd/17
   76 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/17
   77 root      RT   0     0    0    0 S    0 0.0   0:00.10 migration/18
   78 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/18:0
   79 root      20   0     0    0    0 S    0 0.0   0:00.05 ksoftirqd/18
   80 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/18
   81 root      RT   0     0    0    0 S    0 0.0   0:00.24 migration/19
   82 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/19:0
   83 root      20   0     0    0    0 S    0 0.0   0:00.13 ksoftirqd/19
   84 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/19
   85 root      RT   0     0    0    0 S    0 0.0   0:00.18 migration/20
   86 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/20:0
   87 root      20   0     0    0    0 S    0 0.0   0:00.14 ksoftirqd/20
   88 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/20
   89 root      RT   0     0    0    0 S    0 0.0   0:00.14 migration/21
   90 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/21:0
   91 root      20   0     0    0    0 S    0 0.0   0:00.16 ksoftirqd/21
   92 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/21
   93 root      RT   0     0    0    0 S    0 0.0   0:00.12 migration/22
   94 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/22:0
   95 root      20   0     0    0    0 S    0 0.0   0:00.12 ksoftirqd/22
   96 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/22
   97 root      RT   0     0    0    0 S    0 0.0   0:00.12 migration/23
   98 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/23:0
   99 root      20   0     0    0    0 S    0 0.0   0:00.05 ksoftirqd/23
100 root      RT   0     0    0    0 S    0 0.0   0:00.02 watchdog/23
101 root       0 -20     0    0    0 S    0 0.0   0:00.00 cpuset
102 root       0 -20     0    0    0 S    0 0.0   0:00.00 khelper
103 root      20   0     0    0    0 S    0 0.0   0:00.00 kdevtmpfs
104 root       0 -20     0    0    0 S    0 0.0   0:00.00 netns
105 root      20   0     0    0    0 S    0 0.0   0:00.26 kworker/u:1
106 root      20   0     0    0    0 S    0 0.0   0:00.02 sync_supers
107 root      20   0     0    0    0 S    0 0.0   0:00.00 bdi-default
108 root       0 -20     0    0    0 S    0 0.0   0:00.00 kintegrityd
109 root       0 -20     0    0    0 S    0 0.0   0:00.00 kblockd
110 root       0 -20     0    0    0 S    0 0.0   0:00.00 ata_sff
111 root      20   0     0    0    0 S    0 0.0   0:00.00 khubd
112 root       0 -20     0    0    0 S    0 0.0   0:00.00 md
113 root      20   0     0    0    0 S    0 0.0   0:00.40 kworker/5:1
115 root      20   0     0    0    0 S    0 0.0   0:00.00 khungtaskd
116 root      20   0     0    0    0 S    0 0.0   0:00.00 kswapd0
117 root      20   0     0    0    0 S    0 0.0   0:00.00 kswapd1
118 root      25   5     0    0    0 S    0 0.0   0:00.00 ksmd
119 root      39 19     0    0    0 S    0 0.0   0:00.00 khugepaged
120 root      20   0     0    0    0 S    0 0.0   0:00.00 fsnotify_mark
121 root      20   0     0    0    0 S    0 0.0   0:00.00 ecryptfs-kthrea
122 root       0 -20     0    0    0 S    0 0.0   0:00.00 crypto
130 root       0 -20     0    0    0 S    0 0.0   0:00.00 kthrotld
131 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/6:1
132 root      20   0     0    0    0 S    0 0.0   0:00.02 scsi_eh_0
133 root      20   0     0    0    0 S    0 0.0   0:00.00 scsi_eh_1
155 root       0 -20     0    0    0 S    0 0.0   0:00.00 devfreq_wq
157 root      20   0     0    0    0 S    0 0.0   0:00.18 kworker/23:1
158 root      20   0     0    0    0 S    0 0.0   0:00.17 kworker/22:1
159 root      20   0     0    0    0 S    0 0.0   0:00.17 kworker/21:1
160 root      20   0     0    0    0 S    0 0.0   0:00.17 kworker/20:1
161 root      20   0     0    0    0 S    0 0.0   0:01.12 kworker/19:1
162 root      20   0     0    0    0 S    0 0.0   0:00.17 kworker/18:1
163 root      20   0     0    0    0 S    0 0.0   0:00.18 kworker/17:1
164 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/16:1
165 root      20   0     0    0    0 S    0 0.0   0:00.18 kworker/15:1
166 root      20   0     0    0    0 S    0 0.0   0:00.18 kworker/14:1
167 root      20   0     0    0    0 S    0 0.0   0:00.19 kworker/13:1
168 root      20   0     0    0    0 S    0 0.0   0:00.18 kworker/12:1
169 root      20   0     0    0    0 S    0 0.0   0:00.17 kworker/11:1
170 root      20   0     0    0    0 S    0 0.0   0:00.20 kworker/10:1
171 root      20   0     0    0    0 S    0 0.0   0:00.17 kworker/9:1
172 root      20   0     0    0    0 S    0 0.0   0:00.18 kworker/8:1
173 root      20   0     0    0    0 S    0 0.0   0:00.16 kworker/7:1
174 root      20   0     0    0    0 S    0 0.0   0:00.17 kworker/4:1
175 root      20   0     0    0    0 S    0 0.0   0:00.17 kworker/3:1
176 root      20   0     0    0    0 S    0 0.0   0:00.18 kworker/2:1
452 root      20   0     0    0    0 S    0 0.0   0:00.00 scsi_eh_2
454 root      20   0     0    0    0 S    0 0.0   0:00.00 hpsa
472 root      20   0     0    0    0 S    0 0.0   0:00.48 jbd2/sda2-8
473 root       0 -20     0    0    0 S    0 0.0   0:00.00 ext4-dio-unwrit
556 root      20   0     0    0    0 S    0 0.0   0:00.21 kworker/6:2
573 root      20   0     0    0    0 S    0 0.0   0:00.00 jbd2/sda1-8
574 root       0 -20     0    0    0 S    0 0.0   0:00.00 ext4-dio-unwrit
587 root      20   0 18040 1428 532 S    0 0.0   0:00.12 upstart-udev-br
594 root      20   0 21656 1468 804 S    0 0.0   0:00.08 udevd
601 root      20   0     0    0    0 S    0 0.0   0:00.38 jbd2/sda4-8
603 root       0 -20     0    0    0 S    0 0.0   0:00.00 ext4-dio-unwrit
676 root      20   0 21596 1000 456 S    0 0.0   0:00.16 udevd
764 root       0 -20     0    0    0 S    0 0.0   0:00.00 edac-poller
818 root       0 -20     0    0    0 S    0 0.0   0:00.00 kpsmoused
927 root      20   0 311m 1964 1180 S    0 0.0   0:08.21 rsyslogd
943 messageb 20   0 23816 696 440 S    0 0.0   0:00.01 dbus-daemon
946 root      20   0     0    0    0 S    0 0.0   0:00.34 flush-8:0
947 root      20   0     0    0    0 S    0 0.0   0:00.18 kworker/16:2
1040 root      20   0 15188 392 196 S    0 0.0   0:00.00 upstart-socket-
1045 root      20   0 49956 2872 2264 S    0 0.0   0:00.07 sshd
1158 root      20   0 14504 964 800 S    0 0.0   0:00.00 getty
1162 root      20   0 14504 968 800 S    0 0.0   0:00.00 getty
1167 root      20   0 14504 968 800 S    0 0.0   0:00.00 getty
1168 root      20   0 14504 956 800 S    0 0.0   0:00.00 getty
1171 root      20   0 14504 956 800 S    0 0.0   0:00.00 getty
1179 root      20   0 19112 1020 780 S    0 0.0   0:00.09 cron
1180 daemon    20   0 16908 376 216 S    0 0.0   0:00.00 atd
1217 mysql     20   0 537m 46m 7280 S    0 0.3   0:03.81 mysqld
1442 ntp       20   0 33456 2088 1492 S    0 0.0   0:00.50 ntpd
1447 root      20   0 213m 1396 664 S    0 0.0   0:01.27 hpasmlited
1800 root      20   0 51148 1580 976 S    0 0.0   0:04.90 lldpd
1803 _lldpd    20   0 46908 700 208 S    0 0.0   0:02.89 lldpd
1817 root      20   0 153m 12m 2232 S    0 0.1   0:00.02 ruby
1956 root      20   0 25108 1668 1356 S    0 0.0   0:00.05 master
1975 postfix   20   0 27336 1796 1452 S    0 0.0   0:00.02 qmgr
1979 snmp      20   0 48804 5188 2568 S    0 0.0   0:06.83 snmpd
1987 sphinxse 20   0 102m 5460 208 S    0 0.0   0:00.00 searchd
1988 sphinxse 20   0 183m 18m 7476 S    0 0.1   0:07.70 searchd
2032 root      16 -4 46232 972 604 S    0 0.0   0:03.30 auditd
2034 root      12 -8 80264 860 688 S    0 0.0   0:05.38 audispd
2035 root      16 -4 19152 1120 908 S    0 0.0   0:12.68 audisp-cef
2037 root      20   0     0    0    0 S    0 0.0   0:01.59 kauditd
2098 root      20   0 178m 15m 8808 S    0 0.1   0:00.46 /usr/sbin/apach
2130 root      20   0 4292 292 200 S    0 0.0   0:00.00 collectdmon
2132 root      20   0 572m 7588 3632 S    0 0.0   0:46.94 collectd
2153 root      20   0 14504 960 800 S    0 0.0   0:00.00 getty
2195 root      20   0 4308 316 216 S    0 0.0   0:00.00 autossh
2197 root      20   0 44996 4416 2480 S    0 0.0   0:01.16 ssh
2245 root      20   0 4340 360 280 S    0 0.0   0:00.00 tail
2877 root      20   0 4344 612 504 S    0 0.0   0:00.00 tail
2895 root      20   0 4344 608 504 S    0 0.0   0:00.00 tail
2913 root      20   0 4344 360 280 S    0 0.0   0:00.00 tail
2931 root      20   0 4344 360 280 S    0 0.0   0:00.00 tail
2949 root      20   0 4344 608 504 S    0 0.0   0:00.00 tail
2974 root      20   0 4344 612 504 S    0 0.0   0:00.00 tail
2992 root      20   0 4344 360 280 S    0 0.0   0:00.00 tail
3010 root      20   0 4344 360 280 S    0 0.0   0:00.00 tail
3456 root      20   0 4340 608 512 S    0 0.0   0:00.00 tail
3521 root      20   0 4344 360 280 S    0 0.0   0:00.00 tail
3539 root      20   0 4344 360 280 S    0 0.0   0:00.00 tail
3557 root      20   0 4344 608 504 S    0 0.0   0:00.00 tail
3575 root      20   0 4344 356 280 S    0 0.0   0:00.00 tail
3593 root      20   0 4344 608 504 S    0 0.0   0:00.00 tail
3611 root      20   0 4344 612 504 S    0 0.0   0:00.00 tail
3629 root      20   0 4344 356 280 S    0 0.0   0:00.00 tail
3647 root      20   0 4344 612 504 S    0 0.0   0:00.00 tail
4100 root      20   0 4340 612 516 S    0 0.0   0:00.00 tail
6822 root      20   0 21408 832 408 S    0 0.0   0:00.00 udevd
10020 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/1:1
10040 nagios    20   0 25464 1204 688 S    0 0.0   0:00.82 nrpe
41842 root      20   0     0    0    0 S    0 0.0   0:00.02 kworker/0:0
43201 postfix   20   0 27172 1528 1248 S    0 0.0   0:00.00 pickup
51894 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/0:2
54533 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/0:3
57909 root      20   0 79536 3620 2784 S    0 0.0   0:00.01 sshd
57925 mpurzyns 20   0 79536 1744 908 S    0 0.0   0:00.04 sshd
57926 mpurzyns 20   0 25472 7992 1692 S    0 0.0   0:00.46 bash
59436 root      20   0 43716 5240 2680 S    0 0.0   0:00.00 tclsh
59437 root      20   0 5912 616 520 S    0 0.0   0:00.00 tail
59488 root      20   0 11056 1528 1284 S    0 0.0   0:00.00 bash
59497 root      20   0 491m 21m 3916 S    0 0.1   0:01.22 bro
59535 root      20   0 11056 1528 1284 S    0 0.0   0:00.00 bash
59544 root      20   0 203m 20m 3924 S    0 0.1   0:01.20 bro
59596 root      20   0 39232 5336 3148 S    0 0.0   0:00.04 tclsh
59616 root      20   0 38816 5052 3140 S    0 0.0   0:00.03 tclsh
59618 root      20   0 5916 612 520 S    0 0.0   0:00.00 tail
59634 root      20   0 38816 5040 3132 S    0 0.0   0:00.02 tclsh
59636 root      20   0 5916 616 520 S    0 0.0   0:00.00 tail
59652 root      20   0 38816 5040 3132 S    0 0.0   0:00.03 tclsh
59654 root      20   0 5916 616 520 S    0 0.0   0:00.00 tail
59670 root      20   0 38816 5044 3132 S    0 0.0   0:00.03 tclsh
59672 root      20   0 5916 616 520 S    0 0.0   0:00.00 tail
59688 root      20   0 38816 5040 3132 S    0 0.0   0:00.02 tclsh
59690 root      20   0 5916 612 520 S    0 0.0   0:00.00 tail
59706 root      20   0 38816 5044 3132 S    0 0.0   0:00.02 tclsh
59708 root      20   0 5916 612 520 S    0 0.0   0:00.00 tail
59724 root      20   0 38816 5048 3136 S    0 0.0   0:00.02 tclsh
59726 root      20   0 5916 616 520 S    0 0.0   0:00.00 tail
59742 root      20   0 38816 5044 3132 S    0 0.0   0:00.03 tclsh
59744 root      20   0 5916 612 520 S    0 0.0   0:00.00 tail
59990 root      20   0 129m 31m 1776 S    0 0.2   0:05.16 barnyard2
60008 root      20   0 29956 4112 1256 S    0 0.0   0:00.54 barnyard2
60010 root      20   0     0    0    0 S    0 0.0   0:00.00 kworker/0:1
60026 root      20   0 29956 4108 1256 S    0 0.0   0:00.54 barnyard2
60043 root      20   0 29956 4116 1256 S    0 0.0   0:00.50 barnyard2
60060 root      20   0 29956 4112 1256 S    0 0.0   0:00.52 barnyard2
60077 root      20   0 29956 4108 1256 S    0 0.0   0:00.54 barnyard2
60094 root      20   0 29956 4108 1256 S    0 0.0   0:00.51 barnyard2
60111 root      20   0 29956 4108 1256 S    0 0.0   0:00.51 barnyard2
60143 root      20   0 38792 5020 3132 S    0 0.0   0:00.02 tclsh
60145 root      20   0 5900 360 280 S    0 0.0   0:00.00 cat
60160 root      20   0 38808 4940 3116 S    0 0.0   0:00.02 tclsh
60197 root      20   0 38836 5052 3136 S    0 0.0   0:00.03 tclsh
60199 root      20   0 5912 684 584 S    0 0.0   0:00.00 tail
60241 root      20   0 39232 5332 3144 S    0 0.0   0:00.03 tclsh
60261 root      20   0 38816 5048 3140 S    0 0.0   0:00.02 tclsh
60263 root      20   0 5916 616 520 S    0 0.0   0:00.00 tail
60279 root      20   0 38816 5052 3140 S    0 0.0   0:00.03 tclsh
60281 root      20   0 5916 616 520 S    0 0.0   0:00.00 tail
60297 root      20   0 38816 5052 3140 S    0 0.0   0:00.03 tclsh
60299 root      20   0 5916 612 520 S    0 0.0   0:00.00 tail
60315 root      20   0 38816 5052 3140 S    0 0.0   0:00.03 tclsh
60317 root      20   0 5916 616 520 S    0 0.0   0:00.00 tail
60333 root      20   0 38816 5048 3140 S    0 0.0   0:00.03 tclsh
60335 root      20   0 5916 612 520 S    0 0.0   0:00.00 tail
60351 root      20   0 38816 5052 3140 S    0 0.0   0:00.03 tclsh
60353 root      20   0 5916 616 520 S    0 0.0   0:00.00 tail
60369 root      20   0 38816 5048 3140 S    0 0.0   0:00.02 tclsh
60371 root      20   0 5916 612 520 S    0 0.0   0:00.00 tail
60387 root      20   0 38816 5052 3144 S    0 0.0   0:00.03 tclsh
60389 root      20   0 5916 616 520 S    0 0.0   0:00.00 tail
61174 root      20   0 129m 31m 1776 S    0 0.2   0:05.17 barnyard2
61235 root      20   0 129m 31m 1776 S    0 0.2   0:05.15 barnyard2
61296 root      20   0 129m 31m 1776 S    0 0.2   0:05.12 barnyard2
61431 root      20   0 129m 31m 1776 S    0 0.2   0:05.34 barnyard2
61512 root      20   0 129m 31m 1776 S    0 0.2   0:05.12 barnyard2
61609 root      20   0 129m 31m 1776 S    0 0.2   0:05.21 barnyard2
61731 root      20   0 129m 31m 1776 S    0 0.2   0:05.30 barnyard2
61850 root      20   0 38792 5024 3136 S    0 0.0   0:00.03 tclsh
61852 root      20   0 5900 360 280 S    0 0.0   0:00.00 cat
61910 root      20   0 38808 4940 3116 S    0 0.0   0:00.02 tclsh
61957 root      20   0 215m 2052 1772 S    0 0.0   0:00.00 PassengerWatchd
61961 root      20   0 288m 2288 2000 S    0 0.0   0:00.02 PassengerHelper
61963 root      20   0 108m 8200 2164 S    0 0.1   0:00.07 ruby1.9.1
61967 nobody    20   0 165m 4676 3644 S    0 0.0   0:00.01 PassengerLoggin
61996 www-data 20   0 178m 8064 1296 S    0 0.0   0:00.00 /usr/sbin/apach
61997 www-data 20   0 178m 7340 632 S    0 0.0   0:00.00 /usr/sbin/apach
61998 www-data 20   0 178m 7340 632 S    0 0.0   0:00.00 /usr/sbin/apach
61999 www-data 20   0 178m 7340 632 S    0 0.0   0:00.00 /usr/sbin/apach
62000 www-data 20   0 178m 7340 632 S    0 0.0   0:00.00 /usr/sbin/apach
62112 root      20   0 38836 4956 3112 S    0 0.0   0:00.02 tclsh
62114 root      20   0 5912 684 584 S    0 0.0   0:00.00 tail
62147 root      20   0 48224 1880 1448 S    0 0.0   0:00.00 sudo
62148 root      20   0 11036 1476 1252 S    0 0.0   0:00.00 sostat

=========================================================================
Log Archive
=========================================================================
/nsm/sensor_data/nsm1.hostname-eth4/dailylogs/
659G    .
637G    ./2013-03-23
7.2G    ./2013-03-28
15G    ./2013-03-29

/nsm/sensor_data/nsm1.hostname-eth5/dailylogs/
2.6T    .
2.6T    ./2013-03-23
6.8G    ./2013-03-28
2.2G    ./2013-03-29

/nsm/bro/logs/
1.6M    .
36K    ./20--
236K    ./2013-03-28
268K    ./2013-03-29
1.1M    ./stats

=========================================================================
IDS Engine (snort) packet drops
=========================================================================
/nsm/sensor_data/nsm1.hostname-eth4/snort-1.stats last reported pkt_drop_percent as 49.253
/nsm/sensor_data/nsm1.hostname-eth4/snort-2.stats last reported pkt_drop_percent as 42.387
/nsm/sensor_data/nsm1.hostname-eth4/snort-3.stats last reported pkt_drop_percent as 58.134
/nsm/sensor_data/nsm1.hostname-eth4/snort-4.stats last reported pkt_drop_percent as 64.234
/nsm/sensor_data/nsm1.hostname-eth4/snort-5.stats last reported pkt_drop_percent as 67.864
/nsm/sensor_data/nsm1.hostname-eth4/snort-6.stats last reported pkt_drop_percent as 57.978
/nsm/sensor_data/nsm1.hostname-eth4/snort-7.stats last reported pkt_drop_percent as 0.000
/nsm/sensor_data/nsm1.hostname-eth4/snort-8.stats last reported pkt_drop_percent as 0.000
/nsm/sensor_data/nsm1.hostname-eth5/snort-10.stats last reported pkt_drop_percent as 0.000
/nsm/sensor_data/nsm1.hostname-eth5/snort-11.stats last reported pkt_drop_percent as 0.000
/nsm/sensor_data/nsm1.hostname-eth5/snort-12.stats last reported pkt_drop_percent as 0.000
/nsm/sensor_data/nsm1.hostname-eth5/snort-13.stats last reported pkt_drop_percent as 0.000
/nsm/sensor_data/nsm1.hostname-eth5/snort-14.stats last reported pkt_drop_percent as 0.000
/nsm/sensor_data/nsm1.hostname-eth5/snort-15.stats last reported pkt_drop_percent as 0.000
/nsm/sensor_data/nsm1.hostname-eth5/snort-16.stats last reported pkt_drop_percent as 0.000
/nsm/sensor_data/nsm1.hostname-eth5/snort-1.stats last reported pkt_drop_percent as 42.677
/nsm/sensor_data/nsm1.hostname-eth5/snort-2.stats last reported pkt_drop_percent as 36.215
/nsm/sensor_data/nsm1.hostname-eth5/snort-3.stats last reported pkt_drop_percent as 0.000
/nsm/sensor_data/nsm1.hostname-eth5/snort-4.stats last reported pkt_drop_percent as 58.137
/nsm/sensor_data/nsm1.hostname-eth5/snort-5.stats last reported pkt_drop_percent as 24.771
/nsm/sensor_data/nsm1.hostname-eth5/snort-6.stats last reported pkt_drop_percent as 33.672
/nsm/sensor_data/nsm1.hostname-eth5/snort-7.stats last reported pkt_drop_percent as 67.527
/nsm/sensor_data/nsm1.hostname-eth5/snort-8.stats last reported pkt_drop_percent as 33.540
/nsm/sensor_data/nsm1.hostname-eth5/snort-9.stats last reported pkt_drop_percent as 0.000

=========================================================================
pf_ring stats
=========================================================================

On Mar 29, 2013 1:28 AM, "Michal Purzynski" <mic...@rsbac.org> wrote:

Hi.

I'm getting a kernel panic every time I start something using pf_ring.

Everything is updated, in fact, the installation is a week old (fresh?).

There was a lot of trying, testing, etc and what can I say is:

1. the panic is there everytime. Sometimes seconds sometimes longer, but it cannot keep up even 5 minutes.
2. netsniff-ng does not cause it.
3. Snort makes the box die quickly, no matter how many snort processes do I run.
4. The box is a HP 360 G8 with 16GB RAM and dual (single-port) Intel NICs based on 82599.
5. Traffic to the box is around 2Gbits/sec, divided (uneven) between two interfaces.

And something _very_ interesting - the box does not die when booted in UP mode (I've made a test).

Also, made sure that irq balancing daemon isn't running, but it didn't change anything.

The full output is at

http://pastebin.com/sgLMrr49

If you need more information, I'm here.

--
You received this message because you are subscribed to the Google Groups "security-onion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
To post to this group, send email to securit...@googlegroups.com.

Visit this group at http://groups.google.com/group/security-onion?hl=en-US.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "security-onion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
To post to this group, send email to securit...@googlegroups.com.

Doug Burks

unread,

Mar 29, 2013, 7:26:55 AM3/29/13

to securit...@googlegroups.com

Hi Michal,

Are you sure you're running the latest packages?
sudo apt-get update && sudo apt-get dist-upgrade

Have you tried going down to 1 PF_RING instance just to see if that
makes any difference?

Since you're monitoring multiple interfaces, have you disabled Bro's
PF_RING load balancing?
http://securityonion.blogspot.com/2013/02/important-note-for-those-monitoring.html

Have you tried switching to Suricata just to see if that makes any difference?
https://code.google.com/p/security-onion/wiki/FAQ#I'm_currently_running_Snort.__How_do_I_switch_to_Suricata?

Thanks,
Doug

Doug Burks
http://securityonion.blogspot.com

Michal Purzynski

unread,

Mar 29, 2013, 6:00:33 PM3/29/13

to securit...@googlegroups.com

On 3/29/13 12:26 PM, Doug Burks wrote:
> Hi Michal,
>
> Are you sure you're running the latest packages?
> sudo apt-get update && sudo apt-get dist-upgrade

Just to make sure, I've reinstalled the box today from ground zero,
including the OS.

> Have you tried going down to 1 PF_RING instance just to see if that
> makes any difference?

First monitoring interface up (that one with less traffic), second down.

Bro setup to have 1 process, same with snort, tripple checked.

> Since you're monitoring multiple interfaces, have you disabled Bro's
> PF_RING load balancing?

Yup.

> http://securityonion.blogspot.com/2013/02/important-note-for-those-monitoring.html
>
> Have you tried switching to Suricata just to see if that makes any difference?
> https://code.google.com/p/security-onion/wiki/FAQ#I'm_currently_running_Snort.__How_do_I_switch_to_Suricata?

Just tried, the box died after 14 seconds.

Interesting - running everything but snort/suricata and the box has been
running happily for over an hour.

Do you think it's a pf_ring bug? I'm ready to help debugging it, give 7+
years of kernel development experience and a few boxes with tons (a few
Gbit/sec) of traffic. I just have to get it running :)

Should I ask on the pf_ring list, or are some developers here?

Doug Burks

unread,

Mar 29, 2013, 8:25:54 PM3/29/13

to securit...@googlegroups.com

Strange. We've had reports of kernel panics with Suricata, but I don't remember any reports with Snort.

Have you tried stopping Bro altogether?

sudo broctl stop

Just to confirm, what's your PF_RING version number and what's your kernel version?

You're running 64-bit, right?

Did you start with our ISO image or your own Ubuntu?

Have you tried updating your NIC drivers?

I don't think there are any PF_RING developers on this list, so you can follow up with their list if necessary.

Thanks,

Doug

Michal Purzynski

unread,

Apr 1, 2013, 4:54:39 PM4/1/13

to securit...@googlegroups.com

On 3/30/13 1:25 AM, Doug Burks wrote:

Strange. We've had reports of kernel panics with Suricata, but I don't remember any reports with Snort.

Have you tried stopping Bro altogether?

sudo broctl stop

Yes, and what's interesting, running with Bro only (or Bro, Argus, PADS, etc but without snort) didn't make the box crash.

Snort/Suricata crash the box even if running alone.

Just to confirm, what's your PF_RING version number and what's your kernel version?

PF_RING Version : 5.5.2 ($Revision: $)

Linux nsm1 3.2.0-40-generic #64-Ubuntu SMP Mon Mar 25 21:22:10 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

You're running 64-bit, right?

Sure thing.

Did you start with our ISO image or your own Ubuntu?

With a clean 12.04 Ubuntu server ISO.

Have you tried updating your NIC drivers?

Not yet, I'll do that next. BTW - I had exactly the same results with netxen.

Doug Burks

unread,

Apr 1, 2013, 5:01:14 PM4/1/13

to securit...@googlegroups.com

On Mon, Apr 1, 2013 at 4:54 PM, Michal Purzynski <mic...@rsbac.org> wrote:
> Linux nsm1 3.2.0-40-generic #64-Ubuntu SMP Mon Mar 25 21:22:10 UTC 2013
> x86_64 x86_64 x86_64 GNU/Linux

Where did you get 3.2.0-40 from? All of my boxes are showing that the
most recent kernel in Ubuntu's repo is 3.2.0-39.

> You're running 64-bit, right?
>
> Sure thing.
>
>
> Did you start with our ISO image or your own Ubuntu?
>
> With a clean 12.04 Ubuntu server ISO.

If you haven't already, you might want to try our ISO image to see if
you get any different results there.

Thanks,
Doug

Michal Purzynski

unread,

Apr 3, 2013, 7:46:00 AM4/3/13

to securit...@googlegroups.com

On 4/1/13 11:01 PM, Doug Burks wrote:
> On Mon, Apr 1, 2013 at 4:54 PM, Michal Purzynski <mic...@rsbac.org> wrote:
>> Linux nsm1 3.2.0-40-generic #64-Ubuntu SMP Mon Mar 25 21:22:10 UTC 2013
>> x86_64 x86_64 x86_64 GNU/Linux
> Where did you get 3.2.0-40 from? All of my boxes are showing that the
> most recent kernel in Ubuntu's repo is 3.2.0-39.

Interesting, all I did was just running apt-get update && apt-get
dist-upgrade

>
>> You're running 64-bit, right?
>>
>> Sure thing.
>>
>>
>> Did you start with our ISO image or your own Ubuntu?
>>
>> With a clean 12.04 Ubuntu server ISO.
> If you haven't already, you might want to try our ISO image to see if
> you get any different results there.

Will do that next, but my preliminary results with Xubuntu ISO image (in
home network) are "interesting". Everything seems to be running... but
I've got no alerts at all after like an hour after setup has been done.

Rules are in place, but snort uses 0% cpu so it sounds to good to be
true (verified that rules were downloaded, checked logs, re-run
rules-update, etc).
Bro uses some CPU and seems to do its job. Elsa shows me 15788 logs
indexed and 30174 archived, but even a simple query for "google.com"
returns no results.

And yes, there's traffic on the monitored interface.

Hence, I'd rather get the test setup working before I have someone
install it on wannabe-production ;)
> Thanks,
> Doug
>

Doug Burks

unread,

Apr 3, 2013, 8:05:04 AM4/3/13

to securit...@googlegroups.com

On Wed, Apr 3, 2013 at 7:46 AM, Michal Purzynski <mic...@rsbac.org> wrote:
>> Where did you get 3.2.0-40 from? All of my boxes are showing that the
>> most recent kernel in Ubuntu's repo is 3.2.0-39.
>
> Interesting, all I did was just running apt-get update && apt-get
> dist-upgrade

Interesting. What repos are you using?

>> If you haven't already, you might want to try our ISO image to see if
>> you get any different results there.
>
> Will do that next, but my preliminary results with Xubuntu ISO image (in
> home network) are "interesting". Everything seems to be running... but I've
> got no alerts at all after like an hour after setup has been done.
>
> Rules are in place, but snort uses 0% cpu so it sounds to good to be true
> (verified that rules were downloaded, checked logs, re-run rules-update,
> etc).

If monitoring non-RFC1918 address space, have you updated your
HOME_NET variable and restarted Snort?

Are you sure there's traffic that should result in IDS alerts? Have
you tried "curl testmyids.com"?

> Bro uses some CPU and seems to do its job. Elsa shows me 15788 logs indexed
> and 30174 archived, but even a simple query for "google.com" returns no
> results.

If you search for "class=BRO_HTTP groupby:site", do you see most of
your web browsing?

> And yes, there's traffic on the monitored interface.

Please send the output of the following (redacting sensitive info as necessary):
sudo sostat

Thanks,
Doug

Michal Purzynski

unread,

Apr 3, 2013, 9:02:30 AM4/3/13

to securit...@googlegroups.com

On 4/3/13 2:05 PM, Doug Burks wrote:
> On Wed, Apr 3, 2013 at 7:46 AM, Michal Purzynski<mic...@rsbac.org> wrote:
>>> Where did you get 3.2.0-40 from? All of my boxes are showing that the
>>> most recent kernel in Ubuntu's repo is 3.2.0-39.
>> Interesting, all I did was just running apt-get update && apt-get
>> dist-upgrade
> Interesting. What repos are you using?

I think I've spot the bug now - backports repo was enabled and has most
likely caused it.

Anyway, I'm sure it wasn't enabled during previous tests. And just for
sure will re-test with Xubuntu provided SO ISO.

That's was was used and was the wrong one:

###### Ubuntu Main Repos ######
deb http://us.archive.ubuntu.com/ubuntu/ precise main restricted
universe multiverse
deb-src http://us.archive.ubuntu.com/ubuntu/ precise main restricted
universe multiverse

###### Ubuntu Update Repos ######

#### Updates
deb http://us.archive.ubuntu.com/ubuntu/ precise-updates main restricted
universe multiverse
deb-src http://us.archive.ubuntu.com/ubuntu/ precise-updates main
restricted universe multiverse

#### Security
deb http://us.archive.ubuntu.com/ubuntu/ precise-security main
restricted universe multiverse
deb-src http://us.archive.ubuntu.com/ubuntu/ precise-security main
restricted universe multiverse

#### Proposed
deb http://us.archive.ubuntu.com/ubuntu/ precise-proposed main
restricted universe multiverse
deb-src http://us.archive.ubuntu.com/ubuntu/ precise-proposed main
restricted universe multiverse

#### Backports
deb http://us.archive.ubuntu.com/ubuntu/ precise-backports main
restricted universe multiverse
deb-src http://us.archive.ubuntu.com/ubuntu/ precise-backports main
restricted universe multiverse

#### Ubuntu Partner Repo
deb http://archive.canonical.com/ubuntu precise partner
deb-src http://archive.canonical.com/ubuntu precise partner

#### Ubuntu Extras Repo
deb http://extras.ubuntu.com/ubuntu precise main
deb-src http://extras.ubuntu.com/ubuntu precise main

But previously I had:

# File managed by puppet 3.1.1

deb https://our_own_mirror/apt/archive.ubuntu.com/ubuntu precise main
multiverse restricted universe
deb-src https://our_own_mirror/apt/archive.ubuntu.com/ubuntu precise
main multiverse restricted universe

deb https://our_own_mirror/apt/archive.ubuntu.com/ubuntu
precise-backports main multiverse restricted universe
deb-src https://our_own_mirror/apt/archive.ubuntu.com/ubuntu
precise-backports main multiverse restricted universe

deb https://our_own_mirror/apt/archive.ubuntu.com/ubuntu
precise-security main multiverse restricted universe
deb-src https://our_own_mirror/apt/archive.ubuntu.com/ubuntu
precise-security main multiverse restricted universe

deb https://our_own_mirror/apt/archive.ubuntu.com/ubuntu precise-updates
main multiverse restricted universe
deb-src https://our_own_mirror/apt/archive.ubuntu.com/ubuntu
precise-updates main multiverse restricted universe

>>> If you haven't already, you might want to try our ISO image to see if
>>> you get any different results there.
>> Will do that next, but my preliminary results with Xubuntu ISO image (in
>> home network) are "interesting". Everything seems to be running... but I've
>> got no alerts at all after like an hour after setup has been done.
>>
>> Rules are in place, but snort uses 0% cpu so it sounds to good to be true
>> (verified that rules were downloaded, checked logs, re-run rules-update,
>> etc).
> If monitoring non-RFC1918 address space, have you updated your
> HOME_NET variable and restarted Snort?

I've updated it in snort and pads. The later started to give me data,
the former still ignores everything. The line now looks like:

ipvar HOME_NET [192.168.0.0/16,10.0.0.0/8,172.16.0.0/12,198.18.0.0/15]

I use 198.18.0.0/15 internally here (good if you VPN a lot :).

It's got to be this problem, SO was working here with 198.16.0.0/16 subnet.

> Are you sure there's traffic that should result in IDS alerts? Have
> you tried "curl testmyids.com"?

Yes, I did it over 40 times today, according to Bro logs. Also, the
snort process does not even show using any CPU with >10Mbit of traffic.

>> Bro uses some CPU and seems to do its job. Elsa shows me 15788 logs indexed
>> and 30174 archived, but even a simple query for "google.com" returns no
>> results.
> If you search for "class=BRO_HTTP groupby:site", do you see most of
> your web browsing?

Yes, I do now. Elsa does not return anything if the start date is before
installation date (hence no logs for some period). Bug or a feature ;)

I've reinstalled SO today, told ELSA to search for something and the
"from" was set two days ago. I though it would proceed anyway and just
return results from today, but it's not the case.

Doug Burks

unread,

Apr 3, 2013, 9:15:44 AM4/3/13

to securit...@googlegroups.com

On Wed, Apr 3, 2013 at 9:02 AM, Michal Purzynski <mic...@rsbac.org> wrote:
>> Interesting. What repos are you using?
>
> I think I've spot the bug now - backports repo was enabled and has most
> likely caused it.

Yep, that would do it. I'd recommend sticking with the standard repos
as we don't do any testing with the non-standard repos.

>>>> If you haven't already, you might want to try our ISO image to see if
>>>> you get any different results there.
>>>
>>> Will do that next, but my preliminary results with Xubuntu ISO image (in
>>> home network) are "interesting". Everything seems to be running... but
>>> I've
>>> got no alerts at all after like an hour after setup has been done.
>>>
>>> Rules are in place, but snort uses 0% cpu so it sounds to good to be true
>>> (verified that rules were downloaded, checked logs, re-run rules-update,
>>> etc).
>>
>> If monitoring non-RFC1918 address space, have you updated your
>> HOME_NET variable and restarted Snort?
>
> I've updated it in snort and pads. The later started to give me data, the
> former still ignores everything. The line now looks like:
>
> ipvar HOME_NET [192.168.0.0/16,10.0.0.0/8,172.16.0.0/12,198.18.0.0/15]
>
> I use 198.18.0.0/15 internally here (good if you VPN a lot :).
>
> It's got to be this problem, SO was working here with 198.16.0.0/16 subnet.

Did you restart Snort after changing HOME_NET?

When you ran Setup, did you let it configure your network interfaces?

>> Are you sure there's traffic that should result in IDS alerts? Have
>> you tried "curl testmyids.com"?
>
> Yes, I did it over 40 times today, according to Bro logs. Also, the snort
> process does not even show using any CPU with >10Mbit of traffic.
>
>>> Bro uses some CPU and seems to do its job. Elsa shows me 15788 logs
>>> indexed
>>> and 30174 archived, but even a simple query for "google.com" returns no
>>> results.
>>
>> If you search for "class=BRO_HTTP groupby:site", do you see most of
>> your web browsing?
>
> Yes, I do now. Elsa does not return anything if the start date is before
> installation date (hence no logs for some period). Bug or a feature ;)
>
> I've reinstalled SO today, told ELSA to search for something and the "from"
> was set two days ago. I though it would proceed anyway and just return
> results from today, but it's not the case.

In my experience, it should return logs in this case.

>>> And yes, there's traffic on the monitored interface.
>>
>> Please send the output of the following (redacting sensitive info as
>> necessary):
>> sudo sostat

I didn't see the output of "sudo sostat" in your email. If you'd like
us to help you further, please provide this output so that we get a
better understanding of what your system looks like.

Thanks,

Michal Purzynski

unread,

Apr 3, 2013, 9:45:56 AM4/3/13

to securit...@googlegroups.com

Oh, sorry, my bad. Here it goes.

All I've cut are the public IPv6 addresses and nothing more.

=========================================================================
Service Status
=========================================================================
Status: securityonion
* sguil server[ OK ]

Status: HIDS
* ossec_agent (sguil)[ OK ]
Status: Bro
Name Type Host Status Pid Peers Started

manager manager 198.18.3.50 running 12201 2 03 Apr
12:39:20
proxy proxy 198.18.3.50 running 12251 2 03 Apr
12:39:22
so1-eth1-1 worker 198.18.3.50 running 12297 2 03 Apr
12:39:24
Status: so1-eth1
* netsniff-ng (full packet data)[ OK ]

* pcap_agent (sguil)[ OK ]
* snort_agent-1 (sguil)[ OK ]

* snort-1 (alert data)[ OK ]

* barnyard2-1 (spooler, unified2 format)[ OK ]

* prads (sessions/assets)[ OK ]

* sancp_agent (sguil)[ OK ]
* pads_agent (sguil)[ OK ]

* argus[ OK ]

* http_agent (sguil)[ OK ]

=========================================================================
Interface Status
=========================================================================

eth0 Link encap:Ethernet HWaddr 00:0c:29:e5:33:f0
inet addr:198.18.3.50 Bcast:198.18.3.255 Mask:255.255.255.0

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:31799 errors:0 dropped:0 overruns:0 frame:0
TX packets:21504 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:30722050 (30.7 MB) TX bytes:7131753 (7.1 MB)

eth1 Link encap:Ethernet HWaddr 00:1f:29:5b:e0:36
UP BROADCAST RUNNING NOARP PROMISC MULTICAST MTU:1500 Metric:1
RX packets:730399 errors:0 dropped:6 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:539368263 (539.3 MB) TX bytes:0 (0.0 B)
Interrupt:16 Memory:d2640000-d2660000

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1

RX packets:23724 errors:0 dropped:0 overruns:0 frame:0
TX packets:23724 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:14513829 (14.5 MB) TX bytes:14513829 (14.5 MB)

=========================================================================
Disk Usage
=========================================================================
Filesystem Size Used Avail Use% Mounted on

/dev/sda1 12G 6.2G 5.1G 55% /
udev 2.0G 4.0K 2.0G 1% /dev
tmpfs 791M 768K 791M 1% /run

none 5.0M 0 5.0M 0% /run/lock

none 2.0G 216K 2.0G 1% /run/shm

=========================================================================
Network Sockets
=========================================================================
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME

avahi-dae 949 avahi 12u IPv4 8731 0t0 UDP *:5353
avahi-dae 949 avahi 13u IPv6 8732 0t0 UDP *:5353
avahi-dae 949 avahi 14u IPv4 8733 0t0 UDP *:40191
avahi-dae 949 avahi 15u IPv6 8734 0t0 UDP *:49704
cupsd 971 root 8u IPv6 8724 0t0 TCP [::1]:631
(LISTEN)
cupsd 971 root 9u IPv4 8725 0t0 TCP
127.0.0.1:631 (LISTEN)
sshd 1217 root 3u IPv4 9626 0t0 TCP *:22 (LISTEN)
sshd 1217 root 4u IPv6 9628 0t0 TCP *:22 (LISTEN)
syslog-ng 1332 root 9u IPv4 3874 0t0 TCP *:514 (LISTEN)
syslog-ng 1332 root 10u IPv4 3875 0t0 UDP *:514
mysqld 1375 mysql 10u IPv4 4030 0t0 TCP
127.0.0.1:3306 (LISTEN)
mysqld 1375 mysql 34u IPv4 248383 0t0 TCP
127.0.0.1:3306->127.0.0.1:38975 (ESTABLISHED)
mysqld 1375 mysql 116u IPv4 49164 0t0 TCP
127.0.0.1:3306->127.0.0.1:38823 (ESTABLISHED)
mysqld 1375 mysql 118u IPv4 38826 0t0 TCP
127.0.0.1:3306->127.0.0.1:38835 (ESTABLISHED)
mysqld 1375 mysql 121u IPv4 41844 0t0 TCP
127.0.0.1:3306->127.0.0.1:38862 (ESTABLISHED)
mysqld 1375 mysql 122u IPv4 51384 0t0 TCP
127.0.0.1:3306->127.0.0.1:38855 (ESTABLISHED)
searchd 1844 sphinxsearch 6u IPv4 9894 0t0 TCP *:9306
(LISTEN)
searchd 1844 sphinxsearch 7u IPv4 9895 0t0 TCP *:3307
(LISTEN)
/usr/sbin 2282 root 5u IPv6 13544 0t0 TCP *:443 (LISTEN)
/usr/sbin 2282 root 7u IPv6 13548 0t0 TCP *:9876
(LISTEN)
/usr/sbin 2282 root 9u IPv6 13552 0t0 TCP *:3154
(LISTEN)
/usr/sbin 2282 root 11u IPv6 13560 0t0 TCP *:444 (LISTEN)
/usr/sbin 2332 www-data 5u IPv6 13544 0t0 TCP *:443 (LISTEN)
/usr/sbin 2332 www-data 7u IPv6 13548 0t0 TCP *:9876
(LISTEN)
/usr/sbin 2332 www-data 9u IPv6 13552 0t0 TCP *:3154
(LISTEN)
/usr/sbin 2332 www-data 11u IPv6 13560 0t0 TCP *:444 (LISTEN)
/usr/sbin 2333 www-data 5u IPv6 13544 0t0 TCP *:443 (LISTEN)
/usr/sbin 2333 www-data 7u IPv6 13548 0t0 TCP *:9876
(LISTEN)
/usr/sbin 2333 www-data 9u IPv6 13552 0t0 TCP *:3154
(LISTEN)
/usr/sbin 2333 www-data 11u IPv6 13560 0t0 TCP *:444 (LISTEN)
/usr/sbin 2334 www-data 5u IPv6 13544 0t0 TCP *:443 (LISTEN)
/usr/sbin 2334 www-data 7u IPv6 13548 0t0 TCP *:9876
(LISTEN)
/usr/sbin 2334 www-data 9u IPv6 13552 0t0 TCP *:3154
(LISTEN)
/usr/sbin 2334 www-data 11u IPv6 13560 0t0 TCP *:444 (LISTEN)
/usr/sbin 2335 www-data 5u IPv6 13544 0t0 TCP *:443 (LISTEN)
/usr/sbin 2335 www-data 7u IPv6 13548 0t0 TCP *:9876
(LISTEN)
/usr/sbin 2335 www-data 9u IPv6 13552 0t0 TCP *:3154
(LISTEN)
/usr/sbin 2335 www-data 11u IPv6 13560 0t0 TCP *:444 (LISTEN)
/usr/sbin 2335 www-data 31u IPv4 41493 0t0 TCP
127.0.0.1:44619->127.0.0.1:9306 (CLOSE_WAIT)
/usr/sbin 2335 www-data 37u IPv4 51411 0t0 TCP
127.0.0.1:38862->127.0.0.1:3306 (ESTABLISHED)
ntpd 2920 ntp 16u IPv4 14325 0t0 UDP *:123
ntpd 2920 ntp 17u IPv6 14326 0t0 UDP *:123
ntpd 2920 ntp 18u IPv4 14332 0t0 UDP 127.0.0.1:123
ntpd 2920 ntp 19u IPv4 14333 0t0 UDP
198.18.3.50:123
ntpd 2920 ntp 20u IPv6 14334 0t0 UDP
[2001:470:7868:9:20c:29ff:fee5:33f0]:123
ntpd 2920 ntp 21u IPv6 14335 0t0 UDP
[fe80::20c:29ff:fee5:33f0]:123
ntpd 2920 ntp 22u IPv6 14336 0t0 UDP
[2001:470:7868:9:10f9:d741:696c:bd13]:123
ntpd 2920 ntp 23u IPv6 17409 0t0 UDP [::1]:123
tclsh 3211 root 13u IPv4 17994 0t0 TCP *:7734
(LISTEN)
tclsh 3211 root 14u IPv4 17995 0t0 TCP *:7736
(LISTEN)
tclsh 3211 root 15u IPv4 184956 0t0 TCP
127.0.0.1:7736->127.0.0.1:56693 (ESTABLISHED)
tclsh 3211 root 16u IPv4 248174 0t0 TCP
127.0.0.1:7736->127.0.0.1:56697 (ESTABLISHED)
tclsh 3211 root 17u IPv4 248226 0t0 TCP
127.0.0.1:7736->127.0.0.1:56698 (ESTABLISHED)
tclsh 3211 root 18u IPv4 246347 0t0 TCP

127.0.0.1:7736->127.0.0.1:56701 (ESTABLISHED)

tclsh 3211 root 19u IPv4 247254 0t0 TCP
127.0.0.1:7736->127.0.0.1:56702 (ESTABLISHED)
tclsh 3211 root 20u IPv4 245658 0t0 TCP
127.0.0.1:7736->127.0.0.1:56703 (ESTABLISHED)
/usr/sbin 3959 www-data 5u IPv6 13544 0t0 TCP *:443 (LISTEN)
/usr/sbin 3959 www-data 7u IPv6 13548 0t0 TCP *:9876
(LISTEN)
/usr/sbin 3959 www-data 9u IPv6 13552 0t0 TCP *:3154
(LISTEN)
/usr/sbin 3959 www-data 11u IPv6 13560 0t0 TCP *:444 (LISTEN)
/usr/sbin 3959 www-data 30u IPv4 38827 0t0 TCP
127.0.0.1:44640->127.0.0.1:9306 (CLOSE_WAIT)
/usr/sbin 3959 www-data 35u IPv4 38825 0t0 TCP
127.0.0.1:38835->127.0.0.1:3306 (ESTABLISHED)
ruby1.9.1 4250 www-data 12u IPv4 20879 0t0 TCP
127.0.0.1:49477 (LISTEN)
/usr/sbin 4261 www-data 5u IPv6 13544 0t0 TCP *:443 (LISTEN)
/usr/sbin 4261 www-data 7u IPv6 13548 0t0 TCP *:9876
(LISTEN)
/usr/sbin 4261 www-data 9u IPv6 13552 0t0 TCP *:3154
(LISTEN)
/usr/sbin 4261 www-data 11u IPv6 13560 0t0 TCP *:444 (LISTEN)
/usr/sbin 4261 www-data 31u IPv4 38728 0t0 TCP
127.0.0.1:44628->127.0.0.1:9306 (CLOSE_WAIT)
/usr/sbin 4261 www-data 36u IPv4 38727 0t0 TCP
127.0.0.1:38823->127.0.0.1:3306 (ESTABLISHED)
/usr/sbin 4264 www-data 5u IPv6 13544 0t0 TCP *:443 (LISTEN)
/usr/sbin 4264 www-data 7u IPv6 13548 0t0 TCP *:9876
(LISTEN)
/usr/sbin 4264 www-data 9u IPv6 13552 0t0 TCP *:3154
(LISTEN)
/usr/sbin 4264 www-data 11u IPv6 13560 0t0 TCP *:444 (LISTEN)
/usr/sbin 4264 www-data 31u IPv4 50202 0t0 TCP
127.0.0.1:44660->127.0.0.1:9306 (CLOSE_WAIT)
/usr/sbin 4264 www-data 36u IPv4 50201 0t0 TCP
127.0.0.1:38855->127.0.0.1:3306 (ESTABLISHED)
sshd 4930 root 3u IPv4 27180 0t0 TCP
198.18.3.50:22->198.18.3.12:40656 (ESTABLISHED)
sshd 5065 michal 3u IPv4 27180 0t0 TCP
198.18.3.50:22->198.18.3.12:40656 (ESTABLISHED)
/usr/sbin 9384 www-data 5u IPv6 13544 0t0 TCP *:443 (LISTEN)
/usr/sbin 9384 www-data 7u IPv6 13548 0t0 TCP *:9876
(LISTEN)
/usr/sbin 9384 www-data 9u IPv6 13552 0t0 TCP *:3154
(LISTEN)
/usr/sbin 9384 www-data 11u IPv6 13560 0t0 TCP *:444 (LISTEN)
/usr/sbin 9388 www-data 5u IPv6 13544 0t0 TCP *:443 (LISTEN)
/usr/sbin 9388 www-data 7u IPv6 13548 0t0 TCP *:9876
(LISTEN)
/usr/sbin 9388 www-data 9u IPv6 13552 0t0 TCP *:3154
(LISTEN)
/usr/sbin 9388 www-data 11u IPv6 13560 0t0 TCP *:444 (LISTEN)
/usr/sbin 9389 www-data 5u IPv6 13544 0t0 TCP *:443 (LISTEN)
/usr/sbin 9389 www-data 7u IPv6 13548 0t0 TCP *:9876
(LISTEN)
/usr/sbin 9389 www-data 9u IPv6 13552 0t0 TCP *:3154
(LISTEN)
/usr/sbin 9389 www-data 11u IPv6 13560 0t0 TCP *:444 (LISTEN)
tclsh 11326 root 3u IPv4 228217 0t0 TCP
127.0.0.1:56693->127.0.0.1:7736 (ESTABLISHED)
bro 12201 root 4u IPv4 245194 0t0 UDP
198.18.3.50:59973->198.18.4.10:53
bro 12209 root 0u IPv4 245929 0t0 TCP *:47761
(LISTEN)
bro 12209 root 1u IPv6 245930 0t0 TCP *:47761
(LISTEN)
bro 12209 root 2u IPv4 185334 0t0 TCP
198.18.3.50:47761->198.18.3.50:45678 (ESTABLISHED)
bro 12209 root 4u IPv4 245194 0t0 UDP
198.18.3.50:59973->198.18.4.10:53
bro 12209 root 8u IPv4 246836 0t0 TCP
198.18.3.50:47761->198.18.3.50:45680 (ESTABLISHED)
bro 12251 root 4u IPv4 245206 0t0 UDP
198.18.3.50:42609->198.18.4.10:53
bro 12258 root 0u IPv4 163416 0t0 TCP
198.18.3.50:45678->198.18.3.50:47761 (ESTABLISHED)
bro 12258 root 1u IPv4 163419 0t0 TCP *:47762
(LISTEN)
bro 12258 root 2u IPv6 163420 0t0 TCP *:47762
(LISTEN)
bro 12258 root 4u IPv4 245206 0t0 UDP
198.18.3.50:42609->198.18.4.10:53
bro 12258 root 7u IPv4 163431 0t0 TCP
198.18.3.50:47762->198.18.3.50:43515 (ESTABLISHED)
bro 12297 root 4u IPv4 163427 0t0 UDP
198.18.3.50:51991->198.18.4.10:53
bro 12300 root 0u IPv4 246011 0t0 TCP
198.18.3.50:43515->198.18.3.50:47762 (ESTABLISHED)
bro 12300 root 1u IPv4 246012 0t0 TCP
198.18.3.50:45680->198.18.3.50:47761 (ESTABLISHED)
bro 12300 root 2u IPv4 246015 0t0 TCP *:47763
(LISTEN)
bro 12300 root 4u IPv4 163427 0t0 UDP
198.18.3.50:51991->198.18.4.10:53
bro 12300 root 8u IPv6 246016 0t0 TCP *:47763
(LISTEN)
tclsh 12376 root 3u IPv4 245260 0t0 TCP
127.0.0.1:56697->127.0.0.1:7736 (ESTABLISHED)
tclsh 12415 root 3u IPv4 245309 0t0 TCP
127.0.0.1:56698->127.0.0.1:7736 (ESTABLISHED)
tclsh 12415 root 4u IPv4 246893 0t0 TCP
127.0.0.1:8001 (LISTEN)
tclsh 12415 root 6u IPv4 246265 0t0 TCP
127.0.0.1:8001->127.0.0.1:48464 (ESTABLISHED)
barnyard2 12530 root 3u IPv4 246264 0t0 TCP
127.0.0.1:48464->127.0.0.1:8001 (ESTABLISHED)
barnyard2 12530 root 4u IPv4 246268 0t0 TCP
127.0.0.1:38975->127.0.0.1:3306 (ESTABLISHED)
tclsh 12608 root 3u IPv4 247206 0t0 TCP
127.0.0.1:56701->127.0.0.1:7736 (ESTABLISHED)
tclsh 12645 root 3u IPv4 246459 0t0 TCP
127.0.0.1:56702->127.0.0.1:7736 (ESTABLISHED)
tclsh 12722 root 3u IPv4 246532 0t0 TCP
127.0.0.1:56703->127.0.0.1:7736 (ESTABLISHED)

=========================================================================
IDS Rules Update
=========================================================================

=========================================================================
CPU Usage
=========================================================================
top - 13:29:02 up 1:21, 2 users, load average: 0.09, 0.15, 0.20
Tasks: 209 total, 2 running, 206 sleeping, 0 stopped, 1 zombie
Cpu(s): 3.5%us, 2.2%sy, 0.4%ni, 92.9%id, 1.0%wa, 0.0%hi, 0.0%si,
0.0%st
Mem: 4049568k total, 3784132k used, 265436k free, 150716k buffers
Swap: 6217032k total, 1288k used, 6215744k free, 1109616k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

12297 root 20 0 282m 100m 71m R 16 2.5 10:46.76 bro
12258 root 25 5 64876 18m 956 S 14 0.5 7:50.19 bro
12300 root 25 5 127m 82m 64m S 14 2.1 7:45.33 bro
12209 root 25 5 135m 18m 940 S 12 0.5 7:54.17 bro
12251 root 20 0 275m 21m 3972 S 2 0.5 0:47.73 bro
1 root 20 0 24592 2532 1384 S 0 0.1 0:00.94 init

2 root 20 0 0 0 0 S 0 0.0 0:00.00 kthreadd

3 root 20 0 0 0 0 S 0 0.0 0:00.14 ksoftirqd/0
5 root 20 0 0 0 0 S 0 0.0 0:00.26 kworker/u:0
6 root RT 0 0 0 0 S 0 0.0 0:00.03 migration/0
7 root RT 0 0 0 0 S 0 0.0 0:00.01 watchdog/0
8 root RT 0 0 0 0 S 0 0.0 0:00.02 migration/1
9 root 20 0 0 0 0 S 0 0.0 0:00.11 kworker/1:0
10 root 20 0 0 0 0 S 0 0.0 0:00.14 ksoftirqd/1
11 root 20 0 0 0 0 S 0 0.0 0:00.37 kworker/0:1
12 root RT 0 0 0 0 S 0 0.0 0:00.01 watchdog/1
13 root RT 0 0 0 0 S 0 0.0 0:00.02 migration/2
15 root 20 0 0 0 0 S 0 0.0 0:00.20 ksoftirqd/2
16 root RT 0 0 0 0 S 0 0.0 0:00.01 watchdog/2
17 root RT 0 0 0 0 S 0 0.0 0:00.02 migration/3
18 root 20 0 0 0 0 S 0 0.0 0:00.32 kworker/3:0
19 root 20 0 0 0 0 S 0 0.0 0:00.12 ksoftirqd/3
20 root RT 0 0 0 0 S 0 0.0 0:00.01 watchdog/3
21 root 0 -20 0 0 0 S 0 0.0 0:00.00 cpuset
22 root 0 -20 0 0 0 S 0 0.0 0:00.00 khelper
23 root 20 0 0 0 0 S 0 0.0 0:00.00 kdevtmpfs
24 root 0 -20 0 0 0 S 0 0.0 0:00.00 netns
25 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/u:1
26 root 20 0 0 0 0 S 0 0.0 0:00.00 sync_supers
27 root 20 0 0 0 0 S 0 0.0 0:00.00 bdi-default
28 root 0 -20 0 0 0 S 0 0.0 0:00.00 kintegrityd
29 root 0 -20 0 0 0 S 0 0.0 0:00.00 kblockd
30 root 0 -20 0 0 0 S 0 0.0 0:00.00 ata_sff
31 root 20 0 0 0 0 S 0 0.0 0:00.00 khubd
32 root 0 -20 0 0 0 S 0 0.0 0:00.00 md
34 root 20 0 0 0 0 S 0 0.0 0:00.00 khungtaskd
35 root 20 0 0 0 0 S 0 0.0 0:00.14 kswapd0
36 root 25 5 0 0 0 S 0 0.0 0:00.00 ksmd
37 root 39 19 0 0 0 S 0 0.0 0:00.00 khugepaged
38 root 20 0 0 0 0 S 0 0.0 0:00.00 fsnotify_mark
39 root 20 0 0 0 0 S 0 0.0 0:00.00
ecryptfs-kthrea
40 root 0 -20 0 0 0 S 0 0.0 0:00.00 crypto
48 root 0 -20 0 0 0 S 0 0.0 0:00.00 kthrotld
49 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_0
50 root 20 0 0 0 0 S 0 0.0 0:00.03 scsi_eh_1
71 root 0 -20 0 0 0 S 0 0.0 0:00.00 devfreq_wq
72 root 20 0 0 0 0 S 0 0.0 0:00.15 kworker/1:1
74 root 20 0 0 0 0 S 0 0.0 0:00.56 kworker/2:1
75 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/2:2
225 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_2
226 root 0 -20 0 0 0 S 0 0.0 0:00.00
vmw_pvscsi_wq_2
236 root 0 -20 0 0 0 S 0 0.0 0:00.00 ttm_swap
338 root 20 0 0 0 0 S 0 0.0 0:00.05 kworker/0:2
346 root 20 0 0 0 0 S 0 0.0 0:00.40 jbd2/sda1-8
347 root 0 -20 0 0 0 S 0 0.0 0:00.00
ext4-dio-unwrit
439 root 20 0 17232 640 448 S 0 0.0 0:00.04
upstart-udev-br
468 root 20 0 21952 1780 824 S 0 0.0 0:00.04 udevd
613 root 0 -20 0 0 0 S 0 0.0 0:00.00 kmpathd
616 root 20 0 21900 1276 364 S 0 0.0 0:00.00 udevd
617 root 20 0 22020 1288 328 S 0 0.0 0:00.00 udevd
619 root 0 -20 0 0 0 S 0 0.0 0:00.00
kmpath_handlerd
673 root 0 -20 0 0 0 S 0 0.0 0:00.00 kpsmoused
842 root 20 0 15188 400 200 S 0 0.0 0:00.00
upstart-socket-
928 messageb 20 0 24724 1712 832 S 0 0.0 0:00.14 dbus-daemon
940 root 20 0 21320 1724 1440 S 0 0.0 0:00.00 bluetoothd
949 avahi 20 0 32308 1760 1448 S 0 0.0 0:00.01 avahi-daemon
952 avahi 20 0 32180 468 212 S 0 0.0 0:00.00 avahi-daemon
971 root 20 0 101m 3956 2960 S 0 0.1 0:00.00 cupsd
1080 root 10 -10 0 0 0 S 0 0.0 0:00.00 krfcommd
1167 root 20 0 0 0 0 S 0 0.0 0:00.24 flush-8:0
1217 root 20 0 49956 2856 2248 S 0 0.1 0:00.00 sshd
1304 root 20 0 20024 960 800 S 0 0.0 0:00.00 getty
1310 root 20 0 20024 964 800 S 0 0.0 0:00.00 getty
1325 root 20 0 20024 952 800 S 0 0.0 0:00.00 getty
1327 root 20 0 20024 964 800 S 0 0.0 0:00.00 getty
1331 root 20 0 26780 432 196 S 0 0.0 0:00.00 syslog-ng
1332 root 20 0 70532 4264 2876 S 0 0.1 0:00.93 syslog-ng
1333 root 20 0 20024 960 800 S 0 0.0 0:00.00 getty
1355 root 20 0 4460 816 560 S 0 0.0 0:00.00 acpid
1360 root 20 0 280m 4332 3524 S 0 0.1 0:00.02 lightdm
1361 root 20 0 19112 1024 780 S 0 0.0 0:00.01 cron
1362 daemon 20 0 16908 372 216 S 0 0.0 0:00.00 atd
1366 root 20 0 15980 688 512 S 0 0.0 0:00.34 irqbalance
1375 mysql 20 0 2334m 77m 8300 S 0 2.0 0:09.52 mysqld
1388 root 20 0 197m 39m 12m S 0 1.0 0:03.94 Xorg
1402 root 20 0 4400 616 512 S 0 0.0 0:00.00 sh
1404 root 20 0 202m 36m 3832 S 0 0.9 0:02.91 perl
1433 root 20 0 132m 4496 3720 S 0 0.1 0:00.07
accounts-daemon
1436 root 20 0 207m 4856 3636 S 0 0.1 0:00.15 polkitd
1473 root 20 0 2042m 3956 2852 S 0 0.1 0:00.07
console-kit-dae
1543 root 20 0 12804 536 348 S 0 0.0 0:00.00 ossec-execd
1553 ossec 20 0 14508 2384 800 S 0 0.1 0:02.75
ossec-analysisd
1559 root 20 0 4528 552 408 S 0 0.0 0:00.00
ossec-logcollec
1583 root 20 0 214m 4336 3372 S 0 0.1 0:00.01 upowerd
1741 root 20 0 185m 4924 3908 S 0 0.1 0:00.02 lightdm
1777 root 20 0 5752 2108 640 S 0 0.1 0:10.79
ossec-syscheckd
1781 ossec 20 0 13060 544 364 S 0 0.0 0:00.00
ossec-monitord
1843 sphinxse 20 0 102m 5440 204 S 0 0.1 0:00.00 searchd
1844 sphinxse 20 0 565m 26m 6668 S 0 0.7 0:05.66 searchd
2282 root 20 0 176m 12m 6584 S 0 0.3 0:00.15
/usr/sbin/apach
2288 root 20 0 215m 1928 1684 S 0 0.0 0:00.00
PassengerWatchd
2291 root 20 0 736m 2368 2004 S 0 0.1 0:02.32
PassengerHelper
2297 root 20 0 108m 9276 2240 S 0 0.2 0:00.04 ruby1.9.1
2300 nobody 20 0 165m 4656 3652 S 0 0.1 0:00.00
PassengerLoggin
2332 www-data 20 0 368m 97m 5804 S 0 2.5 0:01.56
/usr/sbin/apach
2333 www-data 20 0 177m 9152 2492 S 0 0.2 0:00.02
/usr/sbin/apach
2334 www-data 20 0 368m 98m 6008 S 0 2.5 0:01.49
/usr/sbin/apach
2335 www-data 20 0 373m 102m 7980 S 0 2.6 0:01.32
/usr/sbin/apach
2348 root 20 0 20024 960 800 S 0 0.0 0:00.00 getty
2920 ntp 20 0 37696 2260 1628 S 0 0.1 0:00.19 ntpd
2955 michal 20 0 232m 4652 3808 S 0 0.1 0:00.04
gnome-keyring-d
2966 michal 20 0 4400 700 584 S 0 0.0 0:00.00 sh
2998 michal 20 0 12492 316 0 S 0 0.0 0:00.00 ssh-agent
3001 michal 20 0 26556 788 476 S 0 0.0 0:00.00 dbus-launch
3002 michal 20 0 25616 1920 620 S 0 0.0 0:00.08 dbus-daemon
3010 michal 20 0 47600 2744 2204 S 0 0.1 0:00.02 xfconfd
3015 michal 20 0 63856 2668 2044 S 0 0.1 0:00.35 xscreensaver
3017 michal 20 0 158m 6512 5128 S 0 0.2 0:00.02 xfce4-session
3023 michal 20 0 154m 10m 8168 S 0 0.3 0:00.42 xfwm4
3025 michal 20 0 297m 20m 10m S 0 0.5 0:00.32 xfce4-panel
3027 michal 20 0 233m 7652 6144 S 0 0.2 0:00.00 Thunar
3029 michal 20 0 128m 4004 2768 S 0 0.1 0:00.00 xfsettingsd
3030 michal 20 0 305m 17m 11m S 0 0.4 0:00.43 xfdesktop
3036 michal 20 0 185m 5612 4492 S 0 0.1 0:00.01
polkit-gnome-au
3038 michal 20 0 256m 23m 11m S 0 0.6 0:00.10 applet.py
3040 michal 20 0 451m 15m 11m S 0 0.4 0:00.05 nm-applet
3042 michal 20 0 577m 31m 14m S 0 0.8 0:00.16
blueman-applet
3047 michal 20 0 529m 8968 6108 S 0 0.2 0:00.02 xfce4-volumed
3051 michal 20 0 383m 12m 9596 S 0 0.3 0:00.11
update-notifier
3055 michal 20 0 52420 2444 2044 S 0 0.1 0:00.00 gvfsd
3058 michal 20 0 215m 3608 2992 S 0 0.1 0:00.00
gvfs-fuse-daemo
3060 michal 20 0 150m 3808 2432 S 0 0.1 0:00.01
xfce4-settings-
3063 michal 9 -11 270m 3992 2800 S 0 0.1 0:00.00 pulseaudio
3065 rtkit 21 1 164m 1316 1084 S 0 0.0 0:00.02 rtkit-daemon
3066 michal 20 0 212m 4532 3236 S 0 0.1 0:00.02
xfce4-power-man
3070 root 20 0 116m 3584 2848 S 0 0.1 0:00.02 udisks-daemon
3071 root 20 0 45516 804 448 S 0 0.0 0:00.00 udisks-daemon
3089 michal 20 0 57124 2704 1972 S 0 0.1 0:00.01 gconfd-2
3099 michal 20 0 80852 4340 3512 S 0 0.1 0:00.00
gvfs-gdu-volume
3101 michal 20 0 138m 2516 2020 S 0 0.1 0:00.00
gvfs-afc-volume
3104 michal 20 0 60344 2408 1908 S 0 0.1 0:00.00
gvfs-gphoto2-vo
3109 michal 20 0 69556 3896 3276 S 0 0.1 0:00.00 gvfsd-trash
3110 michal 20 0 149m 7188 5648 S 0 0.2 0:00.01
panel-4-systray
3116 michal 20 0 415m 12m 9796 S 0 0.3 0:00.06
xfce4-indicator
3117 michal 20 0 148m 8944 7156 S 0 0.2 0:00.05
panel-7-datetim
3118 michal 20 0 169m 9.8m 7280 S 0 0.2 0:00.02
panel-9-xfsm-lo
3122 michal 20 0 190m 10m 7804 S 0 0.3 0:00.03
panel-24-thunar
3129 michal 20 0 514m 6736 5144 S 0 0.2 0:00.01
indicator-messa
3131 michal 20 0 411m 4804 3804 S 0 0.1 0:00.01
indicator-appli
3133 michal 20 0 524m 7596 5980 S 0 0.2 0:00.02
indicator-sound
3148 michal 20 0 57824 2612 2156 S 0 0.1 0:00.00
obex-data-serve
3211 root 20 0 123m 11m 3832 S 0 0.3 0:00.88 tclsh
3215 root 20 0 118m 3592 928 S 0 0.1 0:00.11 tclsh
3216 root 20 0 118m 3224 564 S 0 0.1 0:00.00 tclsh
3254 michal 20 0 536m 70m 38m S 0 1.8 0:06.02
chromium-browse
3290 michal 20 0 258m 7292 2116 S 0 0.2 0:00.07
chromium-browse
3293 michal 20 0 6464 408 320 S 0 0.0 0:00.00
chromium-browse
3294 michal 20 0 274m 17m 12m S 0 0.4 0:00.01
chromium-browse
3320 michal 20 0 282m 5940 720 S 0 0.1 0:00.00
chromium-browse
3577 michal 20 0 875m 48m 19m S 0 1.2 0:07.29
chromium-browse
3959 www-data 20 0 371m 98m 6392 S 0 2.5 0:01.48
/usr/sbin/apach
4204 root 20 0 4344 608 512 S 0 0.0 0:00.00 tail
4250 www-data 20 0 355m 88m 3512 S 0 2.2 0:01.02 ruby1.9.1
4261 www-data 20 0 372m 101m 8000 S 0 2.6 0:01.37
/usr/sbin/apach
4264 www-data 20 0 373m 101m 6368 S 0 2.6 0:01.25
/usr/sbin/apach
4278 www-data 20 0 422m 87m 3672 S 0 2.2 0:03.25 ruby
4358 michal 20 0 53644 2036 1672 S 0 0.1 0:00.00
gnome-keyring-d
4930 root 20 0 101m 4336 3288 S 0 0.1 0:00.01 sshd
5065 michal 20 0 102m 2320 952 S 0 0.1 0:00.34 sshd
5066 michal 20 0 31024 8076 1740 S 0 0.2 0:00.21 bash
5210 root 20 0 78152 2380 1776 S 0 0.1 0:00.00 sudo
5211 root 20 0 27504 4556 1744 S 0 0.1 0:00.16 bash
8058 root 20 0 11440 684 584 S 0 0.0 0:00.00 tail
9384 www-data 20 0 177m 8532 2208 S 0 0.2 0:00.00
/usr/sbin/apach
9388 www-data 20 0 177m 8532 2192 S 0 0.2 0:00.00
/usr/sbin/apach
9389 www-data 20 0 176m 7808 1564 S 0 0.2 0:00.00
/usr/sbin/apach
11326 root 20 0 46092 6424 2792 S 0 0.2 0:00.02 tclsh
11327 root 20 0 11436 616 520 S 0 0.0 0:00.00 tail
12191 root 20 0 16572 1520 1284 S 0 0.0 0:00.00 bash
12201 root 20 0 1284m 22m 3992 S 0 0.6 0:51.28 bro
12242 root 20 0 16576 1528 1284 S 0 0.0 0:00.00 bash
12288 root 20 0 16576 1528 1284 S 0 0.0 0:00.00 bash
12335 root 20 0 267m 254m 239m S 0 6.4 0:02.13 netsniff-ng
12376 root 20 0 40524 5360 3088 S 0 0.1 0:00.02 tclsh
12415 root 20 0 40528 5256 3080 S 0 0.1 0:00.02 tclsh
12417 root 20 0 11436 612 520 S 0 0.0 0:00.00 tail
12475 sguil 20 0 954m 581m 11m S 0 14.7 0:15.83 snort
12530 root 20 0 155m 57m 1780 S 0 1.4 0:08.74 barnyard2
12573 sguil 20 0 25864 7116 3764 S 0 0.2 0:03.01 prads
12608 root 20 0 40084 4956 3076 S 0 0.1 0:00.02 tclsh
12610 root 20 0 11420 360 280 S 0 0.0 0:00.00 cat
12645 root 20 0 41212 6208 3116 S 0 0.2 0:00.25 tclsh
12684 sguil 20 0 111m 7184 1140 S 0 0.2 0:05.19 argus
12722 root 20 0 40632 5484 3084 S 0 0.1 0:00.24 tclsh
12724 root 20 0 11440 680 584 S 0 0.0 0:00.00 tail
13308 root 20 0 59148 5864 3396 S 0 0.1 0:00.09 vim
19398 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/3:1
19972 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/3:2
19980 root 20 0 202m 34m 788 S 0 0.9 0:00.01 perl
19992 root 20 0 4400 612 512 S 0 0.0 0:00.00 sh
19995 root 20 0 4400 316 216 S 0 0.0 0:00.00 sh
20000 root 20 0 4308 356 276 S 0 0.0 0:00.00 sleep
20006 michal 20 0 259m 14m 10m S 0 0.4 0:00.10 xfce4-terminal
20007 michal 20 0 0 0 0 Z 0 0.0 0:00.00
xfce4-terminal <defunct>
20008 michal 20 0 27452 4440 1684 S 0 0.1 0:00.10 bash
20064 root 20 0 78400 2540 1812 S 0 0.1 0:00.00 sudo
20066 root 20 0 27492 4540 1740 S 0 0.1 0:00.10 bash
20161 root 20 0 16560 1480 1252 S 0 0.0 0:00.00 sostat
20400 root 20 0 17336 1332 916 R 0 0.0 0:00.00 top

=========================================================================
Log Archive
=========================================================================
/nsm/sensor_data/so1-eth1/dailylogs/
1.5G .
1.5G ./2013-04-03

/nsm/bro/logs/
1.6M .
1.5M ./2013-04-03
104K ./stats

=========================================================================
IDS Engine (snort) packet drops
=========================================================================

/nsm/sensor_data/so1-eth1/snort-1.stats last reported pkt_drop_percent

as 0.000

=========================================================================
pf_ring stats
=========================================================================

Appl. Name : <unknown>
Tot Packets : 292499
Tot Pkt Lost : 0
TX: Send Errors : 0
Reflect: Fwd Errors: 0
Appl. Name : snort-cluster-51-socket-0
Tot Packets : 289647
Tot Pkt Lost : 0
TX: Send Errors : 0
Reflect: Fwd Errors: 0

=========================================================================
Sguil Uncategorized Events
=========================================================================
COUNT(*)
80

=========================================================================
Sguil events summary for yesterday
=========================================================================
Total
0

=========================================================================
Top 50 All time Sguil Events
=========================================================================
Totals GenID:SigID Signature
11 10000:1 PADS New Asset - ssl TLS 1.0 Client Hello
3 10000:1 PADS New Asset - unknown @https
3 10000:2 PADS Changed Asset - ssl TLS 1.0 Client Hello
3 10000:1 PADS New Asset - unknown @domain
2 10000:2 PADS Changed Asset - unknown @domain
2 10000:1 PADS New Asset - http iTunes/11.0.2 (Macintosh; OS X
10.8.3) AppleWebKit/536.28.10
2 10000:2 PADS Changed Asset - unknown @ssh
2 10000:1 PADS New Asset - unknown @syslog
2 10000:2 PADS Changed Asset - ssl Generic TLS 1.0 SSL
2 10000:1 PADS New Asset - unknown @ntp
1 10000:2 PADS Changed Asset - unknown @https
1 10000:1 PADS New Asset - ssh OpenSSH 5.9 (Protocol 2.0)
1 10000:1 PADS New Asset - domain DNS SQR No Error
1 10000:2 PADS Changed Asset - ssh PuTTY Release_0.62 (Protocol 2.0)
1 10000:1 PADS New Asset - http Mozilla/5.0 (Android; Mobile;
rv:20.0) Gecko/20.0 Firefox/20.0
1 10000:1 PADS New Asset - http curl/7.24.0 (x86_64-apple
(darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5)
1 10000:2 PADS Changed Asset - domain DNS SQR No Error
1 10000:1 PADS New Asset - http Dalvik/1.6.0 (Linux; U; Android
4.2.1; GT (I9300 Build/JOP40D))
1 10000:1 PADS New Asset - http Mozilla/5.0 (Windows NT 6.2;
WOW64; rv:19.0) Gecko/20100101 Firefox/19.0
1 10000:1 PADS New Asset - unknown @www
1 10000:2 PADS Changed Asset - unknown @imaps
1 10000:1 PADS New Asset - unknown @imaps
1 10000:1 PADS New Asset - http Microsoft BITS/7.6
1 10000:1 PADS New Asset - dns TCP DNS Server
1 10000:1 PADS New Asset - ssh OpenSSH 5.3 (Protocol 2.0)
1 10000:1 PADS New Asset - http iTunes (iPad/6.1 (2; 16GB; dt:74))
1 10000:1 PADS New Asset - http Microsoft (CryptoAPI/6.2)
Total
49

=========================================================================
Top 50 URLs for yesterday
=========================================================================
Total
0

=========================================================================
Snorby Events Summary for yesterday
=========================================================================
Total
0

=========================================================================
Top 50 All Time Snorby Events
=========================================================================
Total
0

Doug Burks

unread,

Apr 3, 2013, 10:11:45 AM4/3/13

to securit...@googlegroups.com

After changing HOME_NET, did you restart Snort?

Did you update Bro’s network configuration in
/opt/bro/etc/networks.cfg and then restart Bro?

Please send a copy of your /etc/network/interfaces file.

Thanks,
Doug

> --
> You received this message because you are subscribed to the Google Groups
> "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at http://groups.google.com/group/security-onion?hl=en-US.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

Michal Purzynski

unread,

Apr 3, 2013, 11:08:05 AM4/3/13

to securit...@googlegroups.com

On 2013-04-03 16:11, Doug Burks wrote:
> After changing HOME_NET, did you restart Snort?

Yes, I did. Just restarted a whole VM to make sure.

root@so1:~# grep HOME /etc/nsm/so1-eth1/snort.conf
ipvar HOME_NET [192.168.0.0/16,10.0.0.0/8,172.16.0.0/12,198.18.0.0/15]

> Did you update Broï¿½s network configuration in

> /opt/bro/etc/networks.cfg and then restart Bro?

cat /opt
/bro/etc/networks.cfg
# List of local networks in CIDR notation, optionally followed by a
# descriptive tag.
# For example, "10.0.0.0/8" or "fe80::/64" are valid prefixes.

10.0.0.0/8 Private IP space
192.168.0.0/16 Private IP space
172.16.0.0/12 Private IP space
198.18.0.0/15 Private IP space

>
> Please send a copy of your /etc/network/interfaces file.

cat /etc/network/interfaces
# This configuration was created by the Security Onion setup script.
The original network
# interface configuration file was backed up to
/etc/networking/interfaces.bak.

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# loopback network interface
auto lo
iface lo inet loopback

# Management network interface
auto eth0
iface eth0 inet static
address 198.18.3.50
gateway 198.18.3.1
netmask 255.255.255.0
dns-nameservers 198.18.4.10
dns-domain private.corp.nusec.eu

auto eth1
iface eth1 inet manual
up ifconfig $IFACE -arp up
up ip link set $IFACE promisc on
down ip link set $IFACE promisc off
down ifconfig $IFACE down
post-up ethtool -G $IFACE rx 4096; for i in rx tx sg tso ufo gso gro
lro; do ethtool -K $IFACE $i off; done
post-up echo 1 > /proc/sys/net/ipv6/conf/$IFACE/disable_ipv6

Doug Burks

unread,

Apr 3, 2013, 11:15:13 AM4/3/13

to securit...@googlegroups.com

On Wed, Apr 3, 2013 at 11:08 AM, Michal Purzynski <mic...@rsbac.org> wrote:
> Yes, I did. Just restarted a whole VM to make sure.

So you're running Security Onion in a VM?

What VM software are you using?

Have you configured it to allow promiscuous mode on the proper interface?

Thanks,
Doug

Michal Purzynski

unread,

Apr 3, 2013, 11:37:33 AM4/3/13

to securit...@googlegroups.com

Well that test-bed installation for my home, yeah. The production is on
a real (and beefy) hardware (the one that kernel panics).

For the VM:

I can see the packets on the monitor interface without any problem - and
Bro seems to work. The problem is with just a snort. Pads is also great
and I can see assets discovered.

The hypervisor is a ESXi, but it does not matter much, since the card is
passed to the VM, to avoid loosing too much performance. And it was
working on my old security onion installation nicely.

The only difference now is that I've got the network renumbered (and SO
reinstalled) and that how it looks like.

Doug Burks

unread,

Apr 4, 2013, 7:04:35 AM4/4/13

to securit...@googlegroups.com

Have you checked the Snort log file for any clues?

Thanks,
Doug

Michal Purzynski

unread,

Apr 4, 2013, 12:53:15 PM4/4/13

to securit...@googlegroups.com

So, I've just tried it on SO provided ISO and the results are identical
- kernel panic after a few minutes.

The installation has been done, than apt-get update && apt-get
dist-upgrade, and than sosetup and a standalone setup.

I get the same results no matter what I do - running snort only, single
interface, single process also crashes the box.

Output from sostat and /proc/net/pf_ring follows

=========================================================================
Service Status
=========================================================================
Status: securityonion
* sguil server[ FAIL ]

Status: HIDS
* ossec_agent (sguil)[ OK ]
Status: Bro
Name Type Host Status Pid Peers Started

nsm1-eth4-1 worker xxx.xxx.xxx.xxx crashed
manager manager xxx.xxx.xxx.xxx running 13367 2 04
Apr 11:51:12
proxy proxy xxx.xxx.xxx.xxx running 13419 2 04
Apr 11:51:14
nsm1-eth5-1 worker xxx.xxx.xxx.xxx running 13482 2 04
Apr 11:51:16
Status: nsm1-eth4

* netsniff-ng (full packet data)[ FAIL ]
* pcap_agent (sguil)[ OK ]
* snort_agent-1 (sguil)[ OK ]

* snort-1 (alert data)[ FAIL ]
* stale PID file found, process will be restarted at the next
5-minute interval!

* barnyard2-1 (spooler, unified2 format)[ OK ]

* prads (sessions/assets)[ FAIL ]
* sancp_agent (sguil)[ OK ]
* pads_agent (sguil)[ OK ]
* argus[ FAIL ]
* stale PID file found, process will be restarted at the next
5-minute interval!
* http_agent (sguil)[ OK ]

Status: nsm1-eth5
* netsniff-ng (full packet data)[ OK ]

* pcap_agent (sguil)[ OK ]
* snort_agent-1 (sguil)[ OK ]

* snort-1 (alert data)[ OK ]

* barnyard2-1 (spooler, unified2 format)[ OK ]

* prads (sessions/assets)[ OK ]

* sancp_agent (sguil)[ OK ]
* pads_agent (sguil)[ OK ]

* argus[ OK ]

* http_agent (sguil)[ OK ]

=========================================================================
Interface Status
=========================================================================
eth0 Link encap:Ethernet HWaddr ac:16:2d:6f:75:00

inet addr:xxx.xxx.xxx.xxx Bcast:xxx.xxx.xxx.xxx
Mask:255.255.255.0

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:12425 errors:0 dropped:0 overruns:0 frame:0
TX packets:6351 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2027063 (2.0 MB) TX bytes:1549067 (1.5 MB)
Interrupt:32

eth5 Link encap:Ethernet HWaddr 90:e2:ba:2c:ca:3c
UP BROADCAST RUNNING NOARP PROMISC MULTICAST MTU:1500
Metric:1
RX packets:37375989 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:38272704118 (38.2 GB) TX bytes:0 (0.0 B)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1

RX packets:16707 errors:0 dropped:0 overruns:0 frame:0
TX packets:16707 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:31349849 (31.3 MB) TX bytes:31349849 (31.3 MB)

=========================================================================
Disk Usage
=========================================================================
Filesystem Size Used Avail Use% Mounted on

/dev/sda2 6.4T 30G 6.0T 1% /

udev 7.9G 4.0K 7.9G 1% /dev

tmpfs 3.2G 976K 3.2G 1% /run

none 5.0M 0 5.0M 0% /run/lock
none 7.9G 0 7.9G 0% /run/shm

=========================================================================
Network Sockets
=========================================================================
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME

avahi-dae 1347 avahi 12u IPv4 9699 0t0 UDP *:5353
avahi-dae 1347 avahi 13u IPv6 9700 0t0 UDP *:5353
avahi-dae 1347 avahi 14u IPv4 9701 0t0 UDP *:42130
avahi-dae 1347 avahi 15u IPv6 9702 0t0 UDP *:54943
mysqld 1924 mysql 10u IPv4 339 0t0 TCP
127.0.0.1:3306 (LISTEN)
syslog-ng 1929 root 14u IPv4 293 0t0 TCP *:514
(LISTEN)
syslog-ng 1929 root 15u IPv4 294 0t0 UDP *:514
/usr/sbin 3143 root 4u IPv4 30324 0t0 TCP *:443
(LISTEN)
/usr/sbin 3143 root 5u IPv4 30327 0t0 TCP *:9876
(LISTEN)
/usr/sbin 3143 root 6u IPv4 30329 0t0 TCP *:3154
(LISTEN)
/usr/sbin 3143 root 7u IPv4 30333 0t0 TCP *:444
(LISTEN)
/usr/sbin 3209 www-data 4u IPv4 30324 0t0 TCP *:443
(LISTEN)
/usr/sbin 3209 www-data 5u IPv4 30327 0t0 TCP *:9876
(LISTEN)
/usr/sbin 3209 www-data 6u IPv4 30329 0t0 TCP *:3154
(LISTEN)
/usr/sbin 3209 www-data 7u IPv4 30333 0t0 TCP *:444
(LISTEN)
/usr/sbin 3210 www-data 4u IPv4 30324 0t0 TCP *:443
(LISTEN)
/usr/sbin 3210 www-data 5u IPv4 30327 0t0 TCP *:9876
(LISTEN)
/usr/sbin 3210 www-data 6u IPv4 30329 0t0 TCP *:3154
(LISTEN)
/usr/sbin 3210 www-data 7u IPv4 30333 0t0 TCP *:444
(LISTEN)
/usr/sbin 3211 www-data 4u IPv4 30324 0t0 TCP *:443
(LISTEN)
/usr/sbin 3211 www-data 5u IPv4 30327 0t0 TCP *:9876
(LISTEN)
/usr/sbin 3211 www-data 6u IPv4 30329 0t0 TCP *:3154
(LISTEN)
/usr/sbin 3211 www-data 7u IPv4 30333 0t0 TCP *:444
(LISTEN)
/usr/sbin 3213 www-data 4u IPv4 30324 0t0 TCP *:443
(LISTEN)
/usr/sbin 3213 www-data 5u IPv4 30327 0t0 TCP *:9876
(LISTEN)
/usr/sbin 3213 www-data 6u IPv4 30329 0t0 TCP *:3154
(LISTEN)
/usr/sbin 3213 www-data 7u IPv4 30333 0t0 TCP *:444
(LISTEN)
/usr/sbin 3214 www-data 4u IPv4 30324 0t0 TCP *:443
(LISTEN)
/usr/sbin 3214 www-data 5u IPv4 30327 0t0 TCP *:9876
(LISTEN)
/usr/sbin 3214 www-data 6u IPv4 30329 0t0 TCP *:3154
(LISTEN)
/usr/sbin 3214 www-data 7u IPv4 30333 0t0 TCP *:444
(LISTEN)
sshd 3587 root 3r IPv4 23714 0t0 TCP
xxx.xxx.xxx.xxx:22->xxx.xxx.xxx.xxx:48256 (ESTABLISHED)
sshd 3901 michal 3u IPv4 23714 0t0 TCP
xxx.xxx.xxx.xxx:22->xxx.xxx.xxx.xxx:48256 (ESTABLISHED)
tclsh 6019 root 3u IPv4 32656 0t0 TCP
127.0.0.1:8002 (LISTEN)
tclsh 6019 root 5u IPv4 8849 0t0 TCP
127.0.0.1:8002->127.0.0.1:46817 (ESTABLISHED)
tclsh 6037 root 3u IPv4 32662 0t0 TCP
127.0.0.1:8003 (LISTEN)
tclsh 6037 root 5u IPv4 34891 0t0 TCP
127.0.0.1:8003->127.0.0.1:60888 (ESTABLISHED)
tclsh 6055 root 3u IPv4 32668 0t0 TCP
127.0.0.1:8004 (LISTEN)
tclsh 6055 root 5u IPv4 36096 0t0 TCP
127.0.0.1:8004->127.0.0.1:59986 (ESTABLISHED)
tclsh 6073 root 3u IPv4 32674 0t0 TCP
127.0.0.1:8005 (LISTEN)
tclsh 6073 root 5u IPv4 36097 0t0 TCP
127.0.0.1:8005->127.0.0.1:43299 (ESTABLISHED)
tclsh 6092 root 3u IPv4 34859 0t0 TCP
127.0.0.1:8006 (LISTEN)
tclsh 6092 root 5u IPv4 18367 0t0 TCP
127.0.0.1:8006->127.0.0.1:45405 (ESTABLISHED)
tclsh 6110 root 3u IPv4 34865 0t0 TCP
127.0.0.1:8007 (LISTEN)
tclsh 6110 root 5u IPv4 26873 0t0 TCP
127.0.0.1:8007->127.0.0.1:58550 (ESTABLISHED)
tclsh 6128 root 3u IPv4 34871 0t0 TCP
127.0.0.1:8008 (LISTEN)
tclsh 6128 root 5u IPv4 13072 0t0 TCP
127.0.0.1:8008->127.0.0.1:56660 (ESTABLISHED)
barnyard2 6389 root 3u IPv4 21794 0t0 TCP
127.0.0.1:46817->127.0.0.1:8002 (ESTABLISHED)
barnyard2 6406 root 3u IPv4 18362 0t0 TCP
127.0.0.1:60888->127.0.0.1:8003 (ESTABLISHED)
barnyard2 6423 root 3u IPv4 34892 0t0 TCP
127.0.0.1:59986->127.0.0.1:8004 (ESTABLISHED)
barnyard2 6440 root 3u IPv4 25709 0t0 TCP
127.0.0.1:43299->127.0.0.1:8005 (ESTABLISHED)
barnyard2 6458 root 3u IPv4 13069 0t0 TCP
127.0.0.1:45405->127.0.0.1:8006 (ESTABLISHED)
barnyard2 6475 root 3u IPv4 34923 0t0 TCP
127.0.0.1:58550->127.0.0.1:8007 (ESTABLISHED)
barnyard2 6493 root 3u IPv4 15596 0t0 TCP
127.0.0.1:56660->127.0.0.1:8008 (ESTABLISHED)
tclsh 7187 root 3u IPv4 21893 0t0 TCP
127.0.0.1:8102 (LISTEN)
tclsh 7187 root 5u IPv4 26314 0t0 TCP
127.0.0.1:8102->127.0.0.1:36782 (ESTABLISHED)
tclsh 7246 root 3u IPv4 29174 0t0 TCP
127.0.0.1:8103 (LISTEN)
tclsh 7246 root 5u IPv4 27114 0t0 TCP
127.0.0.1:8103->127.0.0.1:58244 (ESTABLISHED)
tclsh 7334 root 3u IPv4 15258 0t0 TCP
127.0.0.1:8104 (LISTEN)
tclsh 7334 root 5u IPv4 39213 0t0 TCP
127.0.0.1:8104->127.0.0.1:51255 (ESTABLISHED)
tclsh 7640 root 3u IPv4 34931 0t0 TCP
127.0.0.1:8105 (LISTEN)
tclsh 7640 root 5u IPv4 23241 0t0 TCP
127.0.0.1:8105->127.0.0.1:39849 (ESTABLISHED)
tclsh 7701 root 3u IPv4 36937 0t0 TCP
127.0.0.1:8106 (LISTEN)
tclsh 7701 root 5u IPv4 34664 0t0 TCP
127.0.0.1:8106->127.0.0.1:59929 (ESTABLISHED)
tclsh 7762 root 3u IPv4 36946 0t0 TCP
127.0.0.1:8107 (LISTEN)
tclsh 7762 root 5u IPv4 22229 0t0 TCP
127.0.0.1:8107->127.0.0.1:36199 (ESTABLISHED)
tclsh 7819 root 3u IPv4 26062 0t0 TCP
127.0.0.1:8108 (LISTEN)
tclsh 7819 root 5u IPv4 36338 0t0 TCP
127.0.0.1:8108->127.0.0.1:59873 (ESTABLISHED)
barnyard2 8342 root 3u IPv4 26313 0t0 TCP
127.0.0.1:36782->127.0.0.1:8102 (ESTABLISHED)
barnyard2 8360 root 3u IPv4 27113 0t0 TCP
127.0.0.1:58244->127.0.0.1:8103 (ESTABLISHED)
barnyard2 8377 root 3u IPv4 39212 0t0 TCP
127.0.0.1:51255->127.0.0.1:8104 (ESTABLISHED)
barnyard2 8394 root 3u IPv4 23240 0t0 TCP
127.0.0.1:39849->127.0.0.1:8105 (ESTABLISHED)
barnyard2 8411 root 3u IPv4 34663 0t0 TCP
127.0.0.1:59929->127.0.0.1:8106 (ESTABLISHED)
barnyard2 8428 root 3u IPv4 23988 0t0 TCP
127.0.0.1:36199->127.0.0.1:8107 (ESTABLISHED)
barnyard2 8445 root 3u IPv4 30984 0t0 TCP
127.0.0.1:59873->127.0.0.1:8108 (ESTABLISHED)
sshd 12734 root 3r IPv4 45894 0t0 TCP *:22 (LISTEN)
sshd 12734 root 4u IPv6 45896 0t0 TCP *:22 (LISTEN)
ntpd 13364 ntp 16u IPv4 45970 0t0 UDP *:123
ntpd 13364 ntp 17u IPv6 45971 0t0 UDP *:123
ntpd 13364 ntp 18u IPv4 45977 0t0 UDP 127.0.0.1:123
ntpd 13364 ntp 19u IPv4 45978 0t0 UDP
xxx.xxx.xxx.xxx:123
ntpd 13364 ntp 21u IPv6 45980 0t0 UDP [::1]:123
bro 13367 root 4u IPv4 48509 0t0 UDP
xxx.xxx.xxx.xxx:44731->xxx.xxx.xxx.xxx:53
bro 13375 root 0u IPv4 47364 0t0 TCP *:47761
(LISTEN)
bro 13375 root 1u IPv6 47365 0t0 TCP *:47761
(LISTEN)
bro 13375 root 2u IPv4 21423 0t0 TCP
xxx.xxx.xxx.xxx:47761->xxx.xxx.xxx.xxx:60994 (ESTABLISHED)
bro 13375 root 4u IPv4 48509 0t0 UDP
xxx.xxx.xxx.xxx:44731->yyy.yyy.yyy.yyy:53
bro 13375 root 8u IPv4 21425 0t0 TCP
xxx.xxx.xxx.xxx:47761->xxx.xxx.xxx.xxx:60997 (ESTABLISHED)
bro 13419 root 4u IPv4 48511 0t0 UDP
xxx.xxx.xxx.xxx:57164->yyy.yyy.yyy.yyy:53
bro 13426 root 0u IPv4 47371 0t0 TCP
xxx.xxx.xxx.xxx:60994->xxx.xxx.xxx.xxx:47761 (ESTABLISHED)
bro 13426 root 1u IPv4 47374 0t0 TCP *:47762
(LISTEN)
bro 13426 root 2u IPv6 47375 0t0 TCP *:47762
(LISTEN)
bro 13426 root 4u IPv4 48511 0t0 UDP
xxx.xxx.xxx.xxx:57164->yyy.yyy.yyy.yyy:53
bro 13426 root 7u IPv4 21424 0t0 TCP
xxx.xxx.xxx.xxx:47762->xxx.xxx.xxx.xxx:58268 (ESTABLISHED)
bro 13482 root 4u IPv4 48512 0t0 UDP
xxx.xxx.xxx.xxx:42657->yyy.yyy.yyy.yyy:53
bro 13485 root 0u IPv4 47381 0t0 TCP
xxx.xxx.xxx.xxx:58268->xxx.xxx.xxx.xxx:47762 (ESTABLISHED)
bro 13485 root 1u IPv4 47382 0t0 TCP
xxx.xxx.xxx.xxx:60997->xxx.xxx.xxx.xxx:47761 (ESTABLISHED)
bro 13485 root 2u IPv4 47385 0t0 TCP *:47764
(LISTEN)
bro 13485 root 4u IPv4 48512 0t0 UDP
xxx.xxx.xxx.xxx:42657->yyy.yyy.yyy.yyy:53
bro 13485 root 8u IPv6 47386 0t0 TCP *:47764
(LISTEN)
tclsh 13607 root 3u IPv4 56663 0t0 TCP
127.0.0.1:8001 (LISTEN)
tclsh 13607 root 5u IPv4 37174 0t0 TCP
127.0.0.1:8001->127.0.0.1:44416 (ESTABLISHED)
barnyard2 13716 root 3u IPv4 48626 0t0 TCP
127.0.0.1:44416->127.0.0.1:8001 (ESTABLISHED)
tclsh 13993 root 3u IPv4 48724 0t0 TCP
127.0.0.1:8101 (LISTEN)
tclsh 13993 root 5u IPv4 48905 0t0 TCP
127.0.0.1:8101->127.0.0.1:36391 (ESTABLISHED)
barnyard2 14093 root 3u IPv4 10709 0t0 TCP
127.0.0.1:36391->127.0.0.1:8101 (ESTABLISHED)
sshd 14776 root 3r IPv4 60736 0t0 TCP
xxx.xxx.xxx.xxx:22->zzz.zzz.zzz.zzz:47391 (ESTABLISHED)
sshd 14936 michal 3u IPv4 60736 0t0 TCP
xxx.xxx.xxx.xxx:22->zzz.zzz.zzz.zzz:47391 (ESTABLISHED)

=========================================================================
IDS Rules Update
=========================================================================

=========================================================================
CPU Usage
=========================================================================
top - 11:52:45 up 9 min, 2 users, load average: 4.20, 2.16, 1.14
Tasks: 304 total, 5 running, 299 sleeping, 0 stopped, 0 zombie
Cpu(s): 7.5%us, 1.3%sy, 0.1%ni, 89.9%id, 0.6%wa, 0.0%hi, 0.5%si,
0.0%st
Mem: 16394156k total, 16225672k used, 168484k free, 105724k
buffers
Swap: 24936144k total, 0k used, 24936144k free, 11884320k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

14114 sguil 20 0 45388 26m 3780 R 102 0.2 1:08.90 prads
14047 sguil 20 0 956m 629m 11m R 100 3.9 1:07.92 snort
13482 root 20 0 1652m 1.4g 75m R 98 9.2 1:27.44 bro
13916 root 20 0 365m 353m 337m R 96 2.2 0:58.39
netsniff-ng
14224 sguil 20 0 303m 215m 1160 S 94 1.3 1:00.07 argus
13375 root 25 5 135m 20m 944 S 14 0.1 0:09.93 bro
13485 root 25 5 127m 84m 64m S 14 0.5 0:09.84 bro
116 root 20 0 0 0 0 S 8 0.0 0:00.08 kswapd0
13426 root 25 5 64884 20m 952 S 8 0.1 0:09.59 bro
117 root 20 0 0 0 0 S 6 0.0 0:00.06 kswapd1
2070 root 20 0 5392 1408 488 S 4 0.0 0:03.50
ossec-syscheckd
15329 root 20 0 17468 1408 916 R 4 0.0 0:00.02 top
574 root 20 0 0 0 0 S 2 0.0 0:00.33
jbd2/sda2-8
618 root 20 0 0 0 0 S 2 0.0 0:00.09 flush-8:0
1929 root 20 0 83612 16m 2880 S 2 0.1 0:02.73 syslog-ng
2039 ossec 20 0 14508 2320 768 S 2 0.0 0:01.76
ossec-analysisd
13367 root 20 0 1646m 24m 3948 S 2 0.2 0:03.26 bro
13419 root 20 0 278m 23m 3980 S 2 0.1 0:01.16 bro
14775 root 20 0 17472 1568 1052 S 2 0.0 0:00.16 top
1 root 20 0 24592 2580 1384 S 0 0.0 0:03.33 init

2 root 20 0 0 0 0 S 0 0.0 0:00.00 kthreadd

3 root 20 0 0 0 0 S 0 0.0 0:00.02
ksoftirqd/0
5 root 20 0 0 0 0 S 0 0.0 0:00.37

kworker/u:0
6 root RT 0 0 0 0 S 0 0.0 0:00.07
migration/0

7 root RT 0 0 0 0 S 0 0.0 0:00.00 watchdog/0
8 root RT 0 0 0 0 S 0 0.0 0:00.35
migration/1
9 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/1:0
10 root 20 0 0 0 0 S 0 0.0 0:00.00
ksoftirqd/1
11 root 20 0 0 0 0 S 0 0.0 0:00.07
kworker/0:1
12 root RT 0 0 0 0 S 0 0.0 0:00.00 watchdog/1

13 root RT 0 0 0 0 S 0 0.0 0:00.07
migration/2
14 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/2:0
15 root 20 0 0 0 0 S 0 0.0 0:00.00
ksoftirqd/2

16 root RT 0 0 0 0 S 0 0.0 0:00.00 watchdog/2

17 root RT 0 0 0 0 S 0 0.0 0:00.07
migration/3
18 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/3:0

19 root 20 0 0 0 0 S 0 0.0 0:00.00
ksoftirqd/3
20 root RT 0 0 0 0 S 0 0.0 0:00.00 watchdog/3

21 root RT 0 0 0 0 S 0 0.0 0:00.07
migration/4
22 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/4:0

23 root 20 0 0 0 0 S 0 0.0 0:00.00
ksoftirqd/4
24 root RT 0 0 0 0 S 0 0.0 0:00.00 watchdog/4

25 root RT 0 0 0 0 S 0 0.0 0:00.07
migration/5
26 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/5:0

27 root 20 0 0 0 0 S 0 0.0 0:00.00
ksoftirqd/5
28 root RT 0 0 0 0 S 0 0.0 0:00.00 watchdog/5

29 root RT 0 0 0 0 S 0 0.0 0:00.07
migration/6

30 root 20 0 0 0 0 S 0 0.0 0:00.06
kworker/6:0
31 root 20 0 0 0 0 S 0 0.0 0:00.00
ksoftirqd/6
32 root RT 0 0 0 0 S 0 0.0 0:00.00 watchdog/6

33 root RT 0 0 0 0 S 0 0.0 0:00.07
migration/7
34 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/7:0
35 root 20 0 0 0 0 S 0 0.0 0:00.00
ksoftirqd/7

36 root RT 0 0 0 0 S 0 0.0 0:00.00 watchdog/7

37 root RT 0 0 0 0 S 0 0.0 0:00.07
migration/8
38 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/8:0

39 root 20 0 0 0 0 S 0 0.0 0:00.00
ksoftirqd/8
40 root RT 0 0 0 0 S 0 0.0 0:00.00 watchdog/8

41 root RT 0 0 0 0 S 0 0.0 0:00.07
migration/9
42 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/9:0
43 root 20 0 0 0 0 S 0 0.0 0:00.00
ksoftirqd/9

44 root RT 0 0 0 0 S 0 0.0 0:00.00 watchdog/9

45 root RT 0 0 0 0 S 0 0.0 0:00.07
migration/10
46 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/10:0

47 root 20 0 0 0 0 S 0 0.0 0:00.00
ksoftirqd/10
48 root RT 0 0 0 0 S 0 0.0 0:00.00
watchdog/10
49 root RT 0 0 0 0 S 0 0.0 0:00.11

migration/11
50 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/11:0

51 root 20 0 0 0 0 S 0 0.0 0:00.00
ksoftirqd/11
52 root RT 0 0 0 0 S 0 0.0 0:00.00

watchdog/11
53 root RT 0 0 0 0 S 0 0.0 0:00.08
migration/12
54 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/12:0

55 root 20 0 0 0 0 S 0 0.0 0:00.00
ksoftirqd/12
56 root RT 0 0 0 0 S 0 0.0 0:00.00
watchdog/12
57 root RT 0 0 0 0 S 0 0.0 0:00.08

migration/13
58 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/13:0

59 root 20 0 0 0 0 S 0 0.0 0:00.02
ksoftirqd/13
60 root RT 0 0 0 0 S 0 0.0 0:00.00
watchdog/13
61 root RT 0 0 0 0 S 0 0.0 0:00.08

migration/14
62 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/14:0

63 root 20 0 0 0 0 S 0 0.0 0:00.00
ksoftirqd/14
64 root RT 0 0 0 0 S 0 0.0 0:00.00
watchdog/14
65 root RT 0 0 0 0 S 0 0.0 0:00.07

migration/15
66 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/15:0

67 root 20 0 0 0 0 S 0 0.0 0:00.01
ksoftirqd/15
68 root RT 0 0 0 0 S 0 0.0 0:00.00
watchdog/15
69 root RT 0 0 0 0 S 0 0.0 0:00.07
migration/16
70 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/16:0
71 root 20 0 0 0 0 S 0 0.0 0:00.00
ksoftirqd/16
72 root RT 0 0 0 0 S 0 0.0 0:00.00
watchdog/16
73 root RT 0 0 0 0 S 0 0.0 0:00.94

migration/17
74 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/17:0

75 root 20 0 0 0 0 S 0 0.0 0:00.00
ksoftirqd/17
76 root RT 0 0 0 0 S 0 0.0 0:00.00
watchdog/17
77 root RT 0 0 0 0 S 0 0.0 0:00.07

migration/18
78 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/18:0

79 root 20 0 0 0 0 S 0 0.0 0:00.00
ksoftirqd/18
80 root RT 0 0 0 0 S 0 0.0 0:00.00
watchdog/18
81 root RT 0 0 0 0 S 0 0.0 0:00.08

migration/19
82 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/19:0

83 root 20 0 0 0 0 S 0 0.0 0:00.01
ksoftirqd/19
84 root RT 0 0 0 0 S 0 0.0 0:00.00
watchdog/19
85 root RT 0 0 0 0 S 0 0.0 0:00.08

migration/20
86 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/20:0

87 root 20 0 0 0 0 S 0 0.0 0:00.00
ksoftirqd/20
88 root RT 0 0 0 0 S 0 0.0 0:00.00
watchdog/20
89 root RT 0 0 0 0 S 0 0.0 0:00.07

migration/21
90 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/21:0

91 root 20 0 0 0 0 S 0 0.0 0:00.00
ksoftirqd/21
92 root RT 0 0 0 0 S 0 0.0 0:00.00
watchdog/21
93 root RT 0 0 0 0 S 0 0.0 0:00.07
migration/22
95 root 20 0 0 0 0 S 0 0.0 0:00.00
ksoftirqd/22
96 root RT 0 0 0 0 S 0 0.0 0:00.00
watchdog/22
97 root RT 0 0 0 0 S 0 0.0 0:00.07

migration/23
98 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/23:0

99 root 20 0 0 0 0 S 0 0.0 0:00.00
ksoftirqd/23
100 root RT 0 0 0 0 S 0 0.0 0:00.00

watchdog/23
101 root 0 -20 0 0 0 S 0 0.0 0:00.00 cpuset
102 root 0 -20 0 0 0 S 0 0.0 0:00.00 khelper
103 root 20 0 0 0 0 S 0 0.0 0:00.00 kdevtmpfs
104 root 0 -20 0 0 0 S 0 0.0 0:00.00 netns

105 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/u:1
106 root 20 0 0 0 0 S 0 0.0 0:00.00

sync_supers
107 root 20 0 0 0 0 S 0 0.0 0:00.00
bdi-default
108 root 0 -20 0 0 0 S 0 0.0 0:00.00
kintegrityd
109 root 0 -20 0 0 0 S 0 0.0 0:00.00 kblockd
110 root 0 -20 0 0 0 S 0 0.0 0:00.00 ata_sff
111 root 20 0 0 0 0 S 0 0.0 0:00.00 khubd
112 root 0 -20 0 0 0 S 0 0.0 0:00.00 md

113 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/3:1

115 root 20 0 0 0 0 S 0 0.0 0:00.00 khungtaskd

118 root 25 5 0 0 0 S 0 0.0 0:00.00 ksmd
119 root 39 19 0 0 0 S 0 0.0 0:00.00 khugepaged
120 root 20 0 0 0 0 S 0 0.0 0:00.00
fsnotify_mark
121 root 20 0 0 0 0 S 0 0.0 0:00.00
ecryptfs-kthrea
122 root 0 -20 0 0 0 S 0 0.0 0:00.00 crypto
130 root 0 -20 0 0 0 S 0 0.0 0:00.00 kthrotld

132 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_0
133 root 20 0 0 0 0 S 0 0.0 0:00.02 scsi_eh_1

155 root 0 -20 0 0 0 S 0 0.0 0:00.00 devfreq_wq

156 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/1:2
355 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/23:1
356 root 20 0 0 0 0 S 0 0.0 0:00.92
kworker/0:2
465 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/22:1
470 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/21:1
471 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/20:1
472 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_2
474 root 20 0 0 0 0 S 0 0.0 0:00.00 hpsa
476 root 20 0 0 0 0 S 0 0.0 0:00.02
kworker/19:1
477 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/18:1
478 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/17:1
479 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/16:1
482 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/15:1
487 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/14:1
488 root 20 0 0 0 0 S 0 0.0 0:00.01
kworker/13:1
489 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/12:1
490 root 20 0 0 0 0 S 0 0.0 0:00.01
kworker/9:1
491 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/5:1
492 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/4:1
493 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/6:2
499 root 20 0 0 0 0 S 0 0.0 0:00.02
kworker/2:1
501 root 20 0 0 0 0 S 0 0.0 0:00.01
kworker/7:1
502 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/8:1
503 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/10:1
507 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/11:1
575 root 0 -20 0 0 0 S 0 0.0 0:00.00
ext4-dio-unwrit
617 root 20 0 0 0 0 S 0 0.0 0:00.00
flush-251:0
715 root 20 0 17232 636 448 S 0 0.0 0:00.07
upstart-udev-br
814 root 20 0 22720 2500 824 S 0 0.0 0:00.10 udevd
1185 root 20 0 22716 2036 360 S 0 0.0 0:00.00 udevd
1244 messageb 20 0 24268 1364 812 S 0 0.0 0:00.05
dbus-daemon
1271 root 0 -20 0 0 0 S 0 0.0 0:00.00 kmpathd
1272 root 0 -20 0 0 0 S 0 0.0 0:00.00
kmpath_handlerd
1284 root 0 -20 0 0 0 S 0 0.0 0:00.00 kpsmoused
1300 root 0 -20 0 0 0 S 0 0.0 0:00.00
edac-poller
1334 root 20 0 21188 1712 1436 S 0 0.0 0:00.00 bluetoothd
1347 avahi 20 0 32312 1780 1440 S 0 0.0 0:00.01
avahi-daemon
1348 avahi 20 0 32180 472 216 S 0 0.0 0:00.00
avahi-daemon
1376 root 20 0 101m 3768 2800 S 0 0.0 0:00.00 cupsd
1381 root 10 -10 0 0 0 S 0 0.0 0:00.00 krfcommd
1411 root 20 0 0 0 0 S 0 0.0 0:00.01
kworker/22:2
1708 root 20 0 15188 404 200 S 0 0.0 0:00.00
upstart-socket-
1868 root 20 0 20024 964 800 S 0 0.0 0:00.00 getty
1874 root 20 0 20024 956 800 S 0 0.0 0:00.00 getty
1891 root 20 0 20024 968 800 S 0 0.0 0:00.00 getty
1892 root 20 0 20024 964 800 S 0 0.0 0:00.00 getty
1895 root 20 0 20024 956 800 S 0 0.0 0:00.00 getty
1903 root 20 0 4460 820 560 S 0 0.0 0:00.00 acpid
1910 root 20 0 19112 1020 780 S 0 0.0 0:00.00 cron
1911 daemon 20 0 16908 376 216 S 0 0.0 0:00.00 atd
1919 root 20 0 280m 4256 3492 S 0 0.0 0:00.02 lightdm
1923 root 20 0 15980 744 544 S 0 0.0 0:00.19 irqbalance
1924 mysql 20 0 1634m 82m 8220 S 0 0.5 0:53.86 mysqld
1928 root 20 0 26780 436 196 S 0 0.0 0:00.00 syslog-ng
1951 root 20 0 217m 19m 9832 S 0 0.1 0:01.70 Xorg
2027 root 20 0 12804 536 348 S 0 0.0 0:00.00
ossec-execd
2043 root 20 0 4528 520 380 S 0 0.0 0:00.00
ossec-logcollec
2075 root 20 0 185m 4672 3688 S 0 0.0 0:00.02 lightdm
2078 ossec 20 0 13060 548 364 S 0 0.0 0:00.00
ossec-monitord
2094 root 20 0 132m 4324 3656 S 0 0.0 0:00.02
accounts-daemon
2097 root 20 0 207m 4812 3612 S 0 0.0 0:00.04 polkitd
2115 root 20 0 4154m 3948 2844 S 0 0.0 0:00.04
console-kit-dae
2188 root 20 0 4400 612 512 S 0 0.0 0:00.00 sh
2190 root 20 0 203m 37m 3796 S 0 0.2 0:04.79 perl
2200 lightdm 20 0 4400 612 504 S 0 0.0 0:00.00
lightdm-greeter
2207 lightdm 20 0 23952 852 560 S 0 0.0 0:00.00
dbus-daemon
2208 lightdm 20 0 244m 13m 10m S 0 0.1 0:00.92
lightdm-gtk-gre
2268 lightdm 20 0 52420 2388 1988 S 0 0.0 0:00.00 gvfsd
2312 lightdm 20 0 215m 3604 2992 S 0 0.0 0:00.00
gvfs-fuse-daemo
2349 root 20 0 214m 4296 3336 S 0 0.0 0:00.02 upowerd
2510 root 20 0 94656 2596 1904 S 0 0.0 0:00.00 lightdm
3128 root 20 0 4400 612 512 S 0 0.0 0:00.00 sh
3131 root 20 0 4400 316 216 S 0 0.0 0:00.00 sh
3137 root 20 0 4308 352 276 S 0 0.0 0:00.00 sleep
3143 root 20 0 176m 12m 6588 S 0 0.1 0:00.08
/usr/sbin/apach
3148 root 20 0 215m 1940 1684 S 0 0.0 0:00.00
PassengerWatchd
3160 root 20 0 288m 2284 1996 S 0 0.0 0:00.00
PassengerHelper
3179 root 20 0 108m 8200 2164 S 0 0.1 0:00.05 ruby1.9.1
3182 nobody 20 0 165m 4684 3656 S 0 0.0 0:00.00
PassengerLoggin
3203 root 20 0 20024 956 800 S 0 0.0 0:00.00 getty
3209 www-data 20 0 176m 6904 664 S 0 0.0 0:00.00
/usr/sbin/apach
3210 www-data 20 0 176m 6904 664 S 0 0.0 0:00.00
/usr/sbin/apach
3211 www-data 20 0 176m 6904 664 S 0 0.0 0:00.00
/usr/sbin/apach
3213 www-data 20 0 176m 6904 664 S 0 0.0 0:00.00
/usr/sbin/apach
3214 www-data 20 0 176m 6904 664 S 0 0.0 0:00.00
/usr/sbin/apach
3587 root 20 0 101m 4364 3304 S 0 0.0 0:00.01 sshd
3901 michal 20 0 101m 1984 920 S 0 0.0 0:00.16 sshd
3902 michal 20 0 30924 7828 1612 S 0 0.0 0:00.34 bash
4002 root 20 0 78392 2528 1808 S 0 0.0 0:00.01 sudo
4030 root 20 0 27480 4528 1736 S 0 0.0 0:00.22 bash
6019 root 20 0 32304 3884 2592 S 0 0.0 0:00.02 tclsh
6021 root 20 0 4344 360 280 S 0 0.0 0:00.00 tail
6037 root 20 0 32304 3884 2592 S 0 0.0 0:00.03 tclsh
6039 root 20 0 4344 608 504 S 0 0.0 0:00.00 tail
6055 root 20 0 32304 3884 2592 S 0 0.0 0:00.02 tclsh
6057 root 20 0 4344 608 504 S 0 0.0 0:00.00 tail
6073 root 20 0 32304 3880 2592 S 0 0.0 0:00.02 tclsh
6075 root 20 0 4344 608 504 S 0 0.0 0:00.00 tail
6092 root 20 0 32304 3880 2592 S 0 0.0 0:00.02 tclsh
6094 root 20 0 4344 608 504 S 0 0.0 0:00.00 tail
6110 root 20 0 32304 3880 2592 S 0 0.0 0:00.03 tclsh
6112 root 20 0 4344 360 280 S 0 0.0 0:00.00 tail
6128 root 20 0 32304 3884 2592 S 0 0.0 0:00.02 tclsh
6130 root 20 0 4344 356 280 S 0 0.0 0:00.00 tail
6389 root 20 0 32732 6808 1256 S 0 0.0 0:02.44 barnyard2
6406 root 20 0 32740 6816 1256 S 0 0.0 0:02.67 barnyard2
6423 root 20 0 32732 6808 1256 S 0 0.0 0:02.54 barnyard2
6440 root 20 0 32732 6804 1256 S 0 0.0 0:02.47 barnyard2
6458 root 20 0 32736 6808 1256 S 0 0.0 0:02.61 barnyard2
6475 root 20 0 32736 6812 1256 S 0 0.0 0:02.49 barnyard2
6493 root 20 0 32740 6816 1256 S 0 0.0 0:02.47 barnyard2
7187 root 20 0 32296 3872 2592 S 0 0.0 0:00.02 tclsh
7189 root 20 0 4344 612 504 S 0 0.0 0:00.00 tail
7246 root 20 0 32296 3872 2592 S 0 0.0 0:00.02 tclsh
7248 root 20 0 4344 608 504 S 0 0.0 0:00.00 tail
7334 root 20 0 32296 3876 2592 S 0 0.0 0:00.03 tclsh
7342 root 20 0 4344 360 280 S 0 0.0 0:00.00 tail
7640 root 20 0 32296 3872 2592 S 0 0.0 0:00.02 tclsh
7650 root 20 0 4344 612 504 S 0 0.0 0:00.00 tail
7701 root 20 0 32296 3872 2592 S 0 0.0 0:00.02 tclsh
7703 root 20 0 4344 608 504 S 0 0.0 0:00.00 tail
7762 root 20 0 32296 3872 2592 S 0 0.0 0:00.03 tclsh
7764 root 20 0 4344 608 504 S 0 0.0 0:00.00 tail
7819 root 20 0 32296 3876 2592 S 0 0.0 0:00.03 tclsh
7821 root 20 0 4344 604 504 S 0 0.0 0:00.00 tail
8342 root 20 0 32728 6808 1256 S 0 0.0 0:02.54 barnyard2
8360 root 20 0 32728 6808 1256 S 0 0.0 0:02.59 barnyard2
8377 root 20 0 32728 6808 1256 S 0 0.0 0:02.70 barnyard2
8394 root 20 0 32728 6808 1256 S 0 0.0 0:02.72 barnyard2
8411 root 20 0 32728 6808 1256 S 0 0.0 0:02.53 barnyard2
8428 root 20 0 32732 6812 1256 S 0 0.0 0:02.52 barnyard2
8445 root 20 0 32728 6808 1256 S 0 0.0 0:02.60 barnyard2
8584 www-data 20 0 426m 91m 3744 S 0 0.6 0:01.77 ruby
8632 root 20 0 22716 1992 316 S 0 0.0 0:00.00 udevd
10324 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/0:0
12306 root 20 0 0 0 0 S 0 0.0 0:00.00
kworker/9:2
12734 root 20 0 49956 2852 2240 S 0 0.0 0:00.00 sshd
12787 root 20 0 44308 4048 2140 S 0 0.0 0:00.00 tclsh
13332 root 20 0 16572 1520 1284 S 0 0.0 0:00.00 bash
13364 ntp 20 0 37696 2204 1576 S 0 0.0 0:00.00 ntpd
13410 root 20 0 16572 1520 1284 S 0 0.0 0:00.00 bash
13465 root 20 0 16576 1524 1284 S 0 0.0 0:00.00 bash
13569 root 20 0 39396 3904 2640 S 0 0.0 0:00.02 tclsh
13607 root 20 0 39392 3940 2652 S 0 0.0 0:00.02 tclsh
13609 root 20 0 11436 616 520 S 0 0.0 0:00.00 tail
13716 root 20 0 32728 6812 1256 S 0 0.0 0:02.58 barnyard2
13787 root 20 0 39420 3880 2644 S 0 0.0 0:00.02 tclsh
13789 root 20 0 11420 356 280 S 0 0.0 0:00.00 cat
13823 root 20 0 39384 3892 2640 S 0 0.0 0:00.02 tclsh
13899 root 20 0 39408 3896 2644 S 0 0.0 0:00.02 tclsh
13955 root 20 0 39396 3904 2640 S 0 0.0 0:00.01 tclsh
13993 root 20 0 39392 3952 2652 S 0 0.0 0:00.02 tclsh
13995 root 20 0 11436 612 520 S 0 0.0 0:00.00 tail
14093 root 20 0 32728 6808 1256 S 0 0.0 0:02.76 barnyard2
14149 root 20 0 39816 4264 2644 S 0 0.0 0:00.27 tclsh
14151 root 20 0 11420 360 280 S 0 0.0 0:00.00 cat
14185 root 20 0 39384 3896 2640 S 0 0.0 0:00.02 tclsh
14261 root 20 0 39408 3900 2644 S 0 0.0 0:00.01 tclsh
14776 root 20 0 101m 4360 3304 S 0 0.0 0:00.02 sshd
14936 michal 20 0 101m 1980 924 S 0 0.0 0:00.00 sshd
14937 michal 20 0 30928 7832 1608 S 0 0.0 0:00.43 bash
15035 root 20 0 78392 2524 1808 S 0 0.0 0:00.02 sudo
15036 root 20 0 16556 1476 1252 S 0 0.0 0:00.00 sostat
15162 root 20 0 203m 34m 772 S 0 0.2 0:00.00 perl

=========================================================================
Log Archive
=========================================================================
/nsm/sensor_data/nsm1-eth4/dailylogs/
8.0K .
4.0K ./2013-04-04

/nsm/sensor_data/nsm1-eth5/dailylogs/
26G .
26G ./2013-04-04

/nsm/bro/logs/
3.9M .
3.9M ./2013-04-04
20K ./stats

=========================================================================
IDS Engine (snort) packet drops
=========================================================================

ERROR: No stats found in /nsm/sensor_data/nsm1-eth4/snort-1.stats
ERROR: No stats found in /nsm/sensor_data/nsm1-eth4/snort-2.stats
ERROR: No stats found in /nsm/sensor_data/nsm1-eth4/snort-3.stats
ERROR: No stats found in /nsm/sensor_data/nsm1-eth4/snort-4.stats
ERROR: No stats found in /nsm/sensor_data/nsm1-eth4/snort-5.stats
ERROR: No stats found in /nsm/sensor_data/nsm1-eth4/snort-6.stats
ERROR: No stats found in /nsm/sensor_data/nsm1-eth4/snort-7.stats
ERROR: No stats found in /nsm/sensor_data/nsm1-eth4/snort-8.stats
ERROR: No stats found in /nsm/sensor_data/nsm1-eth5/snort-1.stats
ERROR: No stats found in /nsm/sensor_data/nsm1-eth5/snort-2.stats
ERROR: No stats found in /nsm/sensor_data/nsm1-eth5/snort-3.stats
ERROR: No stats found in /nsm/sensor_data/nsm1-eth5/snort-4.stats
ERROR: No stats found in /nsm/sensor_data/nsm1-eth5/snort-5.stats
ERROR: No stats found in /nsm/sensor_data/nsm1-eth5/snort-6.stats
ERROR: No stats found in /nsm/sensor_data/nsm1-eth5/snort-7.stats
ERROR: No stats found in /nsm/sensor_data/nsm1-eth5/snort-8.stats

=========================================================================
pf_ring stats
=========================================================================
Appl. Name : <unknown>

Tot Packets : 24901912
Tot Pkt Lost : 17705139

TX: Send Errors : 0
Reflect: Fwd Errors: 0

Appl. Name : snort-cluster-52-socket-0
Tot Packets : 14733285
Tot Pkt Lost : 12874940

TX: Send Errors : 0
Reflect: Fwd Errors: 0

=========================================================================
Sguil Uncategorized Events
=========================================================================
COUNT(*)

233

=========================================================================
Sguil events summary for yesterday
=========================================================================
Total
0

=========================================================================
Top 50 All time Sguil Events
=========================================================================
Totals GenID:SigID Signature

75 10000:1 PADS New Asset - http Apache
19 10000:2 PADS Changed Asset - http Apache
14 10000:1 PADS New Asset - http Apache 2.2.15 (Red Hat)
14 10000:1 PADS New Asset - http HTTP (Monitor/1.1)
13 10000:1 PADS New Asset - ssh OpenSSH 5.3 (Protocol 2.0)
11 10000:1 PADS New Asset - unknown @www
11 10000:1 PADS New Asset - ssh OpenSSH 4.3 (Protocol 2.0)
10 10000:1 PADS New Asset - ssh OpenSSH 5.4 (Protocol 2.0)
7 10000:2 PADS Changed Asset - http HTTP (Monitor/1.1)
6 10000:1 PADS New Asset - unknown @ftp
4 10000:2 PADS Changed Asset - http Apache 2.2.3 (Red Hat)
4 10000:1 PADS New Asset - unknown @domain
4 10000:2 PADS Changed Asset - ssl Generic TLS 1.0 SSL
3 10000:2 PADS Changed Asset - http Apache 2.2.15 (Red Hat)
3 10000:1 PADS New Asset - ssl Generic TLS 1.0 SSL

3 10000:1 PADS New Asset - unknown @https

3 10000:2 PADS Changed Asset - http Server: nginx
3 10000:1 PADS New Asset - ssh OpenSSH 5.2 (Protocol 2.0)
3 10000:1 PADS New Asset - http Server: nginx
2 10000:2 PADS Changed Asset - ssh OpenSSH 5.3 (Protocol 2.0)
2 10000:1 PADS New Asset - http Apache 2.2.3 (Red Hat)
2 10000:1 PADS New Asset - ssh libssh 0.1 (Protocol 2.0)
2 10000:1 PADS New Asset - dns TCP DNS Server
1 10000:1 PADS New Asset - http Apache 2.2.14 (Ubuntu)

1 10000:1 PADS New Asset - ssl TLS 1.0 Client Hello

1 10000:1 PADS New Asset - http Apache 1.3.41 (Darwin)
1 10000:1 PADS New Asset - http Mozilla/5.0 (Windows; U; Windows
NT 5.1; de (DE; rv:1.4) Gecko/20030619 Netscape/7.1 (ax))
1 10000:1 PADS New Asset - http Mozilla/5.0 (Macintosh; Intel Mac OS X
10.8; rv:23.0) Gecko/20130403 Firefox/23.0
1 10000:2 PADS Changed Asset - http Mozilla/5.0 (Windows NT 5.1;
rv:19.0) Gecko/20100101 Firefox/19.0
1 10000:1 PADS New Asset - unknown @syslog
1 10000:1 PADS New Asset - http Mozilla/5.0 (Windows NT 6.0; rv:20.0)
Gecko/20100101 Firefox/20.0
1 10000:2 PADS Changed Asset - http Mozilla/5.0 (X11; Linux x86_64;
rv:16.0) Gecko/20121011 Thunderbird/16.0.1 Lightning/1.8
1 10000:2 PADS Changed Asset - unknown @ssh

1 10000:2 PADS Changed Asset - unknown @https

1 10000:1 PADS New Asset - http Mozilla/5.0 (Windows; U; Windows NT
6.1; is; rv:1.9.2.18) Gecko/20110614 Firefox/3.6.18
1 10000:1 PADS New Asset - http Mozilla/4.0 (compatible; MSIE 8.0;
Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR
3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0)
1 10000:1 PADS New Asset - unknown @ssh
Total
233

=========================================================================
Top 50 URLs for yesterday
=========================================================================
Total
0

=========================================================================
Snorby Events Summary for yesterday
=========================================================================
Total
0

=========================================================================
Top 50 All Time Snorby Events
=========================================================================
Total
0

Bound Device(s) : eth5
Active : 1
Breed : Non-DNA
Sampling Rate : 1
Capture Direction : RX+TX
Socket Mode : RX+TX
Appl. Name : <unknown>
IP Defragment : No
BPF Filtering : Enabled
# Sw Filt. Rules : 0
# Hw Filt. Rules : 0
Poll Pkt Watermark : 1
Num Poll Calls : 6
Channel Id Mask : 0xFFFFFFFF
Cluster Id : 21
Slot Version : 14 [5.5.2]
Min Num Slots : 8159
Bucket Len : 8192
Slot Len : 8224 [bucket+header]
Tot Memory : 67108864
Tot Packets : 69502755
Tot Pkt Lost : 50060714
Tot Insert : 19459186
Tot Read : 19388518
Insert Offset : 61074974
Remove Offset : 61090936
TX: Send Ok : 0
TX: Send Errors : 0
Reflect: Fwd Ok : 0
Reflect: Fwd Errors: 0
Num Free Slots : 0
Bound Device(s) : eth5
Active : 1
Breed : Non-DNA
Sampling Rate : 1
Capture Direction : RX+TX
Socket Mode : RX only
Appl. Name : snort-cluster-52-socket-0
IP Defragment : No
BPF Filtering : Enabled
# Sw Filt. Rules : 0
# Hw Filt. Rules : 0
Poll Pkt Watermark : 128
Num Poll Calls : 2
Channel Id Mask : 0xFFFFFFFF
Cluster Id : 52
Slot Version : 14 [5.5.2]
Min Num Slots : 4889
Bucket Len : 1514
Slot Len : 1714 [bucket+header]
Tot Memory : 8388608
Tot Packets : 60966874
Tot Pkt Lost : 50827261
Tot Insert : 10151167
Tot Read : 10144168
Insert Offset : 1996226
Remove Offset : 1999184
TX: Send Ok : 0
TX: Send Errors : 0
Reflect: Fwd Ok : 0
Reflect: Fwd Errors: 0
Num Free Slots : 0

PF_RING Version : 5.5.2 ($Revision: $)

Ring slots : 4096
Slot version : 14
Capture TX : Yes [RX+TX]
IP Defragment : No
Socket Mode : Standard
Transparent mode : Yes (mode 0)
Total rings : 2
Total plugins : 0
Name: eth5
Index: 7
Address: 90:E2:BA:2C:CA:3C
Polling Mode: NAPI/TNAPI
Type: Ethernet
Family: Standard NIC
# Bound Sockets: 2
Max # TX Queues: 24
# Used RX Queues: 24

--
Michal Purzynski
RSBAC Team

Eric Ooi

unread,

Apr 4, 2013, 1:08:32 PM4/4/13

to securit...@googlegroups.com

Have you tried running suricata instead of snort?

Heine Lysemose

unread,

Apr 4, 2013, 2:03:00 PM4/4/13

to securit...@googlegroups.com

Hi Michael

I don't see the Interface Status for eth4... Only eth0 and eth5. Can we see the full output of sostat were you only hiding your sensitive information.

Thanks,
Lysemose

Michal Purzynski

unread,

Apr 4, 2013, 2:44:42 PM4/4/13

to securit...@googlegroups.com

On 2013-04-04 20:03, Heine Lysemose wrote:

Hi Michael

I don't see the Interface Status for eth4... Only eth0 and eth5. Can we see the full output of sostat were you only hiding your sensitive information.

That's because eth4 was ifdown-ed. I've only redcted IP addresses, nothing more.

Michal Purzynski

unread,

Apr 4, 2013, 2:43:35 PM4/4/13

to securit...@googlegroups.com

On 2013-04-04 19:08, Eric Ooi wrote:

Have you tried running suricata instead of snort?

Sure, kernel panic was even quicker.

Doug Burks

unread,

Apr 4, 2013, 6:49:33 PM4/4/13

to securit...@googlegroups.com

Did you ever get around to updating your NIC drivers?

Could you try different NICs?

Thanks,
Doug

Michal Purzynski

unread,

Apr 5, 2013, 6:14:57 AM4/5/13

to securit...@googlegroups.com

On 2013-04-05 00:49, Doug Burks wrote:
> Did you ever get around to updating your NIC drivers?

Yes. I'm running the newest ixgbe drivers now with LRO disabled. The
only thing it has changed is that a single process of snort does not
kill the box, but two processes do it right away (in seconds).

> Could you try different NICs?

Not really, I've tried a various brands and nothing changes. The servers
aren't exactly under my desk, so every time I change something there's
got to be a task assigned to another team (that also services other
teams), etc.

And, I've got a really good quality NICs here - Intel X520-SR1 (two in
each server). You cannot go better than that :-)

Just a wild guess - what if snort is overwhelmed and cannot remove
packets fast enough from the ring and the pf_ring code goes wild? I
guess it should be dropping packets than.

Doug Burks

unread,

Apr 5, 2013, 8:12:07 AM4/5/13

to securit...@googlegroups.com

Really strange. We have lots of folks running this on lots of
different hardware with no problems. You should probably go ahead and
start a thread on the PF_RING mailing list.

Thanks,
Doug

Michal Purzynski

unread,

Apr 5, 2013, 2:46:41 PM4/5/13

to securit...@googlegroups.com

On 2013-04-05 14:12, Doug Burks wrote:
> Really strange. We have lots of folks running this on lots of
> different hardware with no problems. You should probably go ahead and
> start a thread on the PF_RING mailing list.
>
> Thanks,
> Doug
>
>

OK, so pf_ring developers wants me to test on 5.5.3-dev version, and
that's what I'm doing now.

So far the box is stable with over 2.5Gbits / 300k pps from 30 minutes,
something that never happened before. Far from declaring victory here,
of course, and watching the "top/iostat/logs" like a hawk.

Running without BRO (waiting for the upgrade so it can load balance), 2
snort processes for eth4 (up to 300Mbits/sec) and 16 x snort on eth5 (a
bit more than 2Gbit/sec).

BTW is 16GB of memory enough? I don't worry so much about the cores (12
physical and HT enabled) but the memory.

Also, prads is using a single core, 100% non-stop.

Argus is on 70% on average, and so is netsniff-ng (which spikes up to
450MB/sec in writes).

Doug Burks

unread,

Apr 5, 2013, 2:55:51 PM4/5/13

to securit...@googlegroups.com

On Fri, Apr 5, 2013 at 2:46 PM, Michal Purzynski <mic...@rsbac.org> wrote:
> OK, so pf_ring developers wants me to test on 5.5.3-dev version, and that's
> what I'm doing now.
>
> So far the box is stable with over 2.5Gbits / 300k pps from 30 minutes,
> something that never happened before. Far from declaring victory here, of
> course, and watching the "top/iostat/logs" like a hawk.

Sounds like we're making progress!

> Running without BRO (waiting for the upgrade so it can load balance), 2
> snort processes for eth4 (up to 300Mbits/sec) and 16 x snort on eth5 (a bit
> more than 2Gbit/sec).

Nice!

> BTW is 16GB of memory enough? I don't worry so much about the cores (12
> physical and HT enabled) but the memory.

I'd recommend maxing out the RAM in the box.

> Also, prads is using a single core, 100% non-stop.
>
> Argus is on 70% on average, and so is netsniff-ng (which spikes up to
> 450MB/sec in writes).

Depending on how much you can tune and other variables, you may want
to consider a flow-based load balancer to evenly distribute the 2Gb
pipe to multiple sensors.

Thanks,
Doug

Michal Purzynski

unread,

Apr 8, 2013, 2:32:16 PM4/8/13

to securit...@googlegroups.com

18:08:05 up 3 days, 28 min

RX bytes:5079048321829 (5.0 TB) TX bytes:0 (0.0 B)
RX bytes:40286266376192 (40.2 TB) TX bytes:0 (0.0 B)

OK, so it seems the box is stable.

The traffic is from 0.6Gbit/sec and 130k pps up to 3.5Gbit/sec and 400k
pps. The other box will receive similar amount of traffic.

A huge thank you all for making me try new things to troubleshoot this
problem!

So the resolution was to:
1. upgrade pf_ring to development version (5.5.3 from SVN) along with
its libraries
2. update the NIC driver to version 3.14.5
3. load it with LRO=0

The stream5 memory had to be increased to 1GB per snort instance.

Now on to building a fully distributed setup. Oh, and already ordered
some RAM, so each server will have 64GB divided equally between all
memory channels.

Doug, I'll be running it with pf_ring from SVN for now.

Doug Burks

unread,

Apr 8, 2013, 2:49:03 PM4/8/13

to securit...@googlegroups.com

Glad it's working for you! Hopefully there will be a new PF_RING stable release soon with the fixes from SVN so that I can package it and get it out to all our users.

Thanks,

Doug

--
You received this message because you are subscribed to the Google Groups "security-onion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
To post to this group, send email to securit...@googlegroups.com.
Visit this group at http://groups.google.com/group/security-onion?hl=en-US.
For more options, visit https://groups.google.com/groups/opt_out.

Reply all

Reply to author

Forward

kernel panic in pf_ring

Michal Purzynski

Heine Lysemose

Michal Purzynski

Doug Burks

Michal Purzynski

Doug Burks

Michal Purzynski

Doug Burks

Michal Purzynski

Doug Burks

Michal Purzynski

Doug Burks

Michal Purzynski

Doug Burks

Michal Purzynski

Doug Burks

Michal Purzynski

Doug Burks

Michal Purzynski

Eric Ooi

Heine Lysemose

Michal Purzynski

Michal Purzynski

Doug Burks

Michal Purzynski

Doug Burks

Michal Purzynski

Doug Burks

Michal Purzynski

Doug Burks