PRADS and the performance

[Michal Purzynski]

Hi,

I've noticed that PRADS is a single threaded application, and we don't
really cluster it in any way, like snort.

Prads is eating a whole core, most of the time here, so I'm looking into
some way to pf_ring cluster it.

I would disable it, but I will lose the session information in sguil,
something I'd definitely want to have running.

Thoughts, ideas, suggestions, blames?

[Jeremy Hoel]

I do'nt use SO specifically, but with sguil Prads (pads) doesn't do
session tracking.. it does asset tracking (windows, web server, dns,
etc..). The session tracking would come from cxtracker or sancp.

> --
> You received this message because you are subscribed to the Google Groups
> "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at http://groups.google.com/group/security-onion?hl=en-US.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Doug Burks]

Hi Jeremy,

PRADS does indeed do both asset tracking *and* session tracking. We
configure Sguil's sancp_agent and pads_agent to read data produced by
PRADS.

Thanks,
Doug

--
Doug Burks
http://securityonion.blogspot.com

[Doug Burks]

Hi Michal,

There are two main options here:

1. If you don't necessarily need to have the session or asset data in
Sguil, then you can just disable PRADS and use the session data in
ELSA. It comes from the Bro conn.log and Bro takes advantage of
PF_RING clustering.

OR

2. If you really want to keep PRADS and have it take advantage of
PF_RING clustering, you'll need to:
- compile PRADS against PF_RING
- update the NSM scripts to spin up $IDS_LB_PROCS instances of PRADS,
each writing to unique files/directories
- update the NSM scripts to spin up $IDS_LB_PROCS instances of
sancp_agent and pads_agent, each reading from the corresponding PRADS
output
- do lots of testing!

Hope that helps!

Thanks,
Doug

[Michal Purzynski]

One more question - are we missing something important, not having
PRADS? Like, when I request Sguil to provide a session transcript, how
does it know what data to use? Is is the sancp mysql table and sessions
from there, or something else?

Asking, because I remember having PRADS disabled, and still having it
working :>

[Doug Burks]

In Sguil, you can pivot from an IDS alert to full transcript without
having any session data in the sancp table. Sguil takes the sensor
name from the alert to determine which sensor to request the pcap from
and then locates the proper stream using timestamp and src/dst
IP/port.

The one caveat here is CapMe. If you use CapMe to pivot to full
transcript in ELSA, it will default to searching the sancp table. In
this case, we use the sancp table like an index to look up the
NET_GROUP for the session, which tells sguild which sensor to request
the pcap from. So if you disable PRADS, then CapME will no longer be
able to pivot to full transcript from ELSA.

Doug

[Michal Purzynski]

Thanks!

I'll see what I can do to recompile prads and have it use the pf_ring
libpcap. If anything will let everyone know.

BTW, the answers like the one below, are awesome, it's fast, it's 100%
complete, and I couldn't expect it from a NSM vendor for 100 000 USD
(you can add zeros if you want).

Thanks again :)

SO For The Win.

[Doug Burks]

I should also mention that one disadvantage of PRADS session data is
that it all goes to a single central database on the master server
regardless of how many sensors you have. That single central database
may then become bloated and slow.

By contrast, the advantage of using Bro's conn.log as your session
data is that it's stored in ELSA, which is a distributed database by
default. Each ELSA log node has its own local MySQL database. So as
you add nodes, you're automatically distributing your database load.

Doug

Doug Burks
http://securityonion.blogspot.com

[Michal Purzynski]

OK, what you said makes me wonder if there's any reason to go for the
PRADS at all - I'd rather not :) In fact PRADS is disabled here, for all
the reasons. I got a feeling, that it isn't maintained any more.

One thing - as my sancp table is empty, I cannot query it from a
CapMe/Snorby - and that means, no arbitrary queries for packet capture
data. Sometimes it's useful to look into a session which didn't trigger
any Snort alert.

I would have to go to the capture data dir, figure out which pcaps have
the necessary information, merge them, load into wireshark - all
manually. Do you have some ideas how to do it in a best way? Xplico
isn't really a good tool for that, as it ignores the data sent on ports
it does not know.

Of course everything still is fine when there was a Snort alert, but if
not - am I on my own, or is there anything I can do to simplify it?

[Doug Burks]

On Fri, May 3, 2013 at 5:27 AM, Michal Purzynski <mic...@rsbac.org> wrote:
> OK, what you said makes me wonder if there's any reason to go for the PRADS
> at all - I'd rather not :) In fact PRADS is disabled here, for all the
> reasons. I got a feeling, that it isn't maintained any more.

For some networks, a single instance of PRADS may be totally
sufficient and they may not have any database issues. You're
monitoring more traffic than most of our users are, so that's why
you're running into these issues.

> One thing - as my sancp table is empty, I cannot query it from a
> CapMe/Snorby

You should still be able to pivot to CapMe from Snorby because that
should default to querying the event table instead of the sancp table.

> and that means, no arbitrary queries for packet capture data.
> Sometimes it's useful to look into a session which didn't trigger any Snort
> alert.

Agreed, at some point we need to rework CapMe so that it's no longer
dependent on the sancp table.

Doug

[psy phii]

I have recompiled prads against PF_RING for awhile now and have attached my git diff for anyone interested. This is against commit 3c751c87190731304baf699ce8b0aa5329bec82e but the short and skinny of it is
;

1. change ldflags to include pfring shared lib. Newer versions of pf_ring seem to depend on librt (-lrt ) so id say try with or without up to you.

In src/Makefile change
LDFLAGS+=-lpcap -lpcre -lresolv
to
LDFLAGS+=-lpcap -lpcre -lresolv -lpfring -lrt

2. Add header includes and api call to change packet polling watermark from 1 to 128. I found that without this prads will still have high (but lower) load and will generate tons of context switches polling the ring all the time.

src/prads.c

#include "dhcp.h"
+#include <pcap.h>
+#include <pcap/pcap.h>
+#include <pfring.h>
//#include "output-plugins/log_init.h"

and

olog("[*] Sniffing...\n");
+ pcap_set_watermark(config.handle, 128);
pcap_loop(config.handle, -1, got_packet, NULL);

game_over();

Also you can set clusters up by using the PF_RING libpcap environment variables like

PCAP_PF_RING_APPNAME=
PCAP_PF_RING_USE_CLUSTER_PER_FLOW_5_TUPLE=
PCAP_PF_RING_RSS_REHASH=
PCAP_PF_RING_CLUSTER_ID=

For more options try

strings /usr/local/lib/libpcap.so.1.1.1 | grep PCAP_

or wherever your pcap is.

++++++++++++SNIP++++++++++++++

diff --git a/Makefile b/Makefile
index 9c8b550..36a49e2 100644
--- a/Makefile
+++ b/Makefile
@@ -1,6 +1,6 @@
PREFIX=/usr/local
BINDIR=${PREFIX}/bin
-CONFDIR=${PREFIX}/etc/prads
+CONFDIR=/etc/prads
MANDIR=${PREFIX}/share/man/man1
DOCUTIL=rst2man
INSTALLGROUP=root
diff --git a/src/Makefile b/src/Makefile
index ee0cf37..da040b8 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -32,7 +32,7 @@ endif
ifeq (${TCMALLOC},y)
LDFLAGS+=-ltcmalloc
endif
-LDFLAGS+=-lpcap -lpcre -lresolv
+LDFLAGS+=-lpcap -lpcre -lresolv -lpfring -lrt
CFLAGS+=-DCONFDIR='"${CONFDIR}/"'

else
@@ -68,7 +68,7 @@ ifneq (${DEBUG_PACKET},)
CFLAGS+= -DDEBUG_PACKET
endif
ifeq ($(UNAME), FreeBSD)
-LDFLAGS= -L/usr/local/lib -lpcre -lpcap
+LDFLAGS= -L/usr/local/lib -lpcre -lpcap -lpfring -lrt
CPPFLAGS= -I/usr/local/include
endif

diff --git a/src/prads.c b/src/prads.c
index 34503fc..5c10398 100644
--- a/src/prads.c
+++ b/src/prads.c
@@ -41,6 +41,9 @@
#include "tcp.h"
#include "dump_dns.h"
#include "dhcp.h"
+#include <pcap.h>
+#include <pcap/pcap.h>
+#include <pfring.h>
//#include "output-plugins/log_init.h"
#include "output-plugins/log.h"

@@ -1466,6 +1469,7 @@ int main(int argc, char *argv[])

cxt_init();
olog("[*] Sniffing...\n");
+ pcap_set_watermark(config.handle, 128);
pcap_loop(config.handle, -1, got_packet, NULL);

game_over();

[Mandeep Baidwan]

Are there any plans for the above to be included in SO at any point? We make use of PRADS to allow visibility of all traffic between hosts regardless of a snort alert occurring. PRADS pretty much always runs close to 100%.

I'm weighing up bringing up a new sensor with a faster/less cores cpu but I'd wait if multithreaded PRADS will be implemented at some point.

Thanks!

[Doug Burks]

Hi Mandeep,

Please see the previous discussion in this thread:
https://groups.google.com/d/topic/security-onion/bYBjG8259dU/discussion

Even if PRADS were compiled with PF_RING, it would still be outputting
to a single central MySQL database. That database would get big and
bloated quickly, especially if you have lots of traffic and/or lots of
sensors.

Instead of getting your session data from PRADS, you can leverage the
session data from Bro's conn.log. Bro is already compiled with
PF_RING and outputs to ELSA which is a scalable distributed database.

> --
> You received this message because you are subscribed to the Google Groups "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.

> Visit this group at http://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/d/optout.



--
Doug Burks
Need Security Onion Training or Commercial Support?
http://securityonionsolutions.com
Last day to register for 3-Day Training Class in Augusta GA is 12/11!

[Mandeep Baidwan]

Doug,

Thanks for your reply, I must have missed that bit.

I did start to use bro, and whilst it was performed a lot better than PRADS I couldn't find a way to arbitrarily view pcap data between hosts. Is that expected or am I missing something in my setup? e.g. A host has communicated with a known malicious IP but we want to review the pcaps for the past week to see what else has gone to or come from that IP address.

If that's not possible I'll probably go down the route of a more powerful cpu.

Mandeep.

[Doug Burks]

- Log into ELSA.

- Use the "Connections" links on the left side OR manually search for
the IP address in question in the Bro conn logs like this:
class=BRO_CONN 1.2.3.4

- Find an individual connection that you'd like to pivot to full packet capture.

- Click Info.

- "Log Info" window appears. Click the Plugin drop-down box and then
click getPcap.

- CapME screen appears. Use the same credentials you used to login to ELSA.

- CapME should then render the pcap as an ASCII transcript and provide
links to download the pcap itself.

[psy phii]

Does SO include a way to build the host_attribute table used to enable target based defrag support in snort? As far as I know prads2snort is the only way to build that passively using free tools. There is another tool that uses nmap output to do this as well but I cant remember the name.

That is the main reason I continue to use prads. With pf_ring libpcap you can enable load balancing using the PCAP_PF_RING_CLUSTER_ID environment variable.

Use the same ID integer for instances you want to share a load.