Best practice to report multiple syslog into a snort database

[Cyril Peponnet]

Firt of All, Happy New Year for all sagan crew and users :)

Here is the dilemia,

I discovered sagan and how it could be powerfull to reports events
from syslog into snort database for reporting them with snorby.
I was wondering how to reports multiples devices at the same time in
the same database without mixing all alert together.

For example, I've a wireless AP, a switch, a firewall, two Hypervisor
hosts and some webservers. I've also a snort applicance analysing the
incomming traffic from outside.

The idea is to create a rsyslog server to gather all the syslog from
the devices into dedicated files and send them to sagan for analysis
and reports.
As I understand, the only way to do that is to create one mixed file
and use sagan as a sensor for snorby.

I was wondering how it could be possible to create on the same machine
multiple sensors for example : network, hypervisors and servers as
following
-Network with firewall and switch and wireless AP log analysis
-Hypervisors with the two virtualiation hosts
-Servers with the webservers

The idea behind is to create some custom rules per groups ans don't
apply for exemple apache rules to the hypervisor syslog.


Is though about this kind of mecanism :

1° All devices are remote logging into a single rsyslog machine
(dedicated). In the rsyslog rules I gather the device into categories
and I write them to differents fifo (3).
2° Next, I set 3 differentes sagan config files according to the 3
differents fifo with differents rules to output to 3 unified2 files
(network, server, virtu).
3° To the end, I use barnyard2 to send thoses files to an mysql
database.

I a dreaming world it will apperars in snorby as 3 new sensors
(network, server and virtu).


Is this realistic ? What is the best way to gather syslog and avoir to
mix from they came from ?

Thanks a lot,

Regards CP

[Champ Clark III [Quadrant]]

Is this realistic ? What is the best way to gather syslog and avoir to
mix from they came from ?

Completely realistic and I've set this up multiple times.  It's actually a nice way to do it.   

One of the things I've been meaning to do to "get the word out" about Sagan is a regular blog/wiki posting.   This way,  I could cover installations like you've described and other

nifty things Sagan can do.   I'll give you a brief run down of how I've done it. 

The first step is to get the multiple devices reporting to syslog (syslog-ng/rsyslog).   For example sake,  we'll call our resources "firewalls,  routers,  servers".  

You'll want to setup your syslog daemon to receive syslog messages on different ports.    For example:

routers:  514/UDP

firewalls: 1514/UDP

servers: 2514/UDP

Of course,  you could use syslog over TCP,  or TCP with TLS,  etc,  however that's out side of the scope and I'm trying to keep things simple for the example.   When the syslog daemon receives messages per-UDP port,  they get stored into a sagan FIFO.  

For example,  you might have your syslog daemon store to the follow FIFO's per inbound UDP/port:

sagan-routers.fifo

sagan-firewalls.fifo

sagan-servers.fifo 

You then create 3 configurations files (sagan-routers.config,  sagan-firewalls.config,  sagan-servers.conf).  Each config file need point to there respected FIFO.   You'll also need to make some other minor changes.  For example, you'll want to change the LOCKFILE  and a few other things.

You can then start Sagan with:

sagan --config /usr/local/etc/sagan-routers.config -D

sagan --config /usr/local/etc/sagan-firewalls.config -D

sagan --config /usr/local/etc/sagan-servers.config -D

The rest depends on if you're doing direct database writes or Barnyard2.

If you're doing barnyard2,  then make sure you setup each sagan configuration with it's own unified2 output format.  (ie - "sagan-routers.u2").   Then setup the instances of barnyard2 needed to read from each unified2 output. 

One thing to note here.  If the Sagan is running on the same machine as the syslog receiver,  you might want to do direct database writes.    This lets Sagan records a little bit of extra information in the "Alert" field.   If the Sagan instance is running remote or you're worried about losing events if Sagan dies,  then use Barnyard2.  

It's actually pretty nice once it's setup.  I have setup's in the field that have both Snort data and Sagan data all in the same place.  You can generate some really nice reports with Snorby based on that information :)

I hope this helps,  and let me know if you have any other questions!

Champ Clark III
(office) 904.253.7856

(mobile) 850.443.2440 

(SOC) 800.538.9357 ext 101

ccl...@quadrantsec.com
www.quadrantsec.com

[Champ Clark III [Quadrant]]

Oh yeah, sorry I missed your question on IRC :)

[Cyril Peponnet]

That's what I call a nice response !!! 

Rsyslog and Sagan will be on the same host, barnyard2 was only to avoid losing event if I loose the SQL connection. But as I have read that some informations are left behind when going through barnyard2, I will send directly so SQL.

An other thing, can I mix unified output and external process ? The main idea is to add a little script reacting on specific events (let's say for example, restart a remote service when crashing) and keep the database logging events.

Thanks a lot.

One more thing, it's really missing of use case as you said ! On the wiki or blog, put some real use case as example could help a lot of people like me.

Snort,snorby and sagan is the wining combination for full monitoring health of your architecture.

<quadrant.png>

[Champ Clark III [Quadrant]]

On Jan 6, 2012, at 1:12 PM, Cyril Peponnet wrote:

That's what I call a nice response !!! 

Rsyslog and Sagan will be on the same host, barnyard2 was only to avoid losing event if I loose the SQL connection. But as I have read that some informations are left behind when going through barnyard2, I will send directly so SQL.

You don't really loose a lot of information.  For example,  instead of seeing "[OpenSSH] Authentication Failure [uid: 0] [username: root]",  it'll just show up as "[OpenSSH Authentication Failure". 

It's a annoying problem with no work around (  at this fime ) with Unified2/Barnyard2

An other thing, can I mix unified output and external process ? The main idea is to add a little script reacting on specific events (let's say for example, restart a remote service when crashing) and keep the database logging events.

Yep.  You can mix output plugins without an issue :) 

Thanks a lot.

One more thing, it's really missing of use case as you said ! On the wiki or blog, put some real use case as example could help a lot of people like me.

Yep. I've been pondering on that for a while now :)  Just need to find time to start writing. 

Snort,snorby and sagan is the wining combination for full monitoring health of your architecture.

Woo!  :)

Let me know if you run into any issues.