Detecting mysql disconnects

[DigiAngel]

Hey all!

So...I was wondering if we could get sagan to detect/log when it drops
connection to the mysql server it's connected to. Would be nice to
get this functionality if possible. Thank you.

James

[Champ Clark III [Quadrant]]

Yeah..  I could probably add that pretty easy.  I'm surprised it doesn't already do that.  Have you checked your sagan.log file?

Champ Clark III
(office) 904.253.7856

(mobile) 850.443.2440 

(SOC) 800.538.9357 ext 101

ccl...@quadrantsec.com
www.quadrantsec.com

[DigiAngel]

Well...I see the below:

[*] [11/09/2011 03:59:49] - [output-plugins/sagan-snort.c, line 183]
Lost connection to MySQL database. Trying 1
[*] [11/09/2011 03:59:51] - [output-plugins/sagan-snort.c, line 183]
Lost connection to MySQL database. Trying 2
[*] [11/09/2011 03:59:53] - [output-plugins/sagan-snort.c, line 183]
Lost connection to MySQL database. Trying 3
[E] [11/24/2011 10:26:30] - [output-plugins/sagan-snort.c, line 113]
MySQL Error 2003: "Can't connect to MySQL server on
'10.10.254.110' (113)"

That helps...issue is the switch that the box was connected to was
rebooted on the third of this month..I don't see any entries for
December. I AM however running it in the foreground in debug
(normalize), so maybe that had something to do with it. And as I'm
looking at this, I don't think it dropped connection...I think it ran
out of threads:

[*] [12/03/2011 23:56:09] - Snort database thread handler: Out of
threads

At that point it never logged anything until I stopped and started
sagan again. How can I avoid that in the future?

James

On Dec 5, 1:31 pm, "Champ Clark III [Quadrant]"

<ccl...@quadrantsec.com> wrote:
> Yeah..  I could probably add that pretty easy.  I'm surprised it doesn't already do that.  Have you checked your sagan.log file?
>
> On Dec 5, 2011, at 3:25 PM, DigiAngel wrote:
>
> > Hey all!
>
> > So...I was wondering if we could get sagan to detect/log when it drops
> > connection to the mysql server it's connected to.  Would be nice to
> > get this functionality if possible.  Thank you.
>
> > James
>

[Champ Clark III [Quadrant]]

I think it'll try X number of times in a row,  then give up.  However,  I'd have to look at that code. 

When it can't connect,  that also keeps that thread alive.  So each new event creates a new thread and eventually you run out of

threads all together. 

The best way to handle this is to run Sagan with Unified2 output and let barnyard2 take over the process of sending the data to the MySQL database.  This way,  if the connection goes down,  barnyard2 will queue it until it comes back online.

[DigiAngel]

Ok...that makes sense. Any way we can have those messages go to
syslog? Thanks Champ.

James

On Dec 5, 2:10 pm, "Champ Clark III [Quadrant]"

[Champ Clark III [Quadrant]]

Hahaha,  Yep,  but don't create a rule for it,  or you'll end up in a feed back loop! :)

I haven't had a chance to complete it,  but I plan on adding "processors" to Sagan.   For example,  a processor that can determine if Sagan hasn't heard events from a particular machine for X amount of time,  a processor to trigger when a host starts generating more events than normal,  etc. 

The first processor,  and the most simple,  will be to generate "Sagan" related events.    This would allow you,  from the console,  to "see" that Sagan ran out of threads,  or has some other issues.   

I've always been hesitant to put Sagan related information _in_ syslog in fear that some users might unwittingly create a feedback loop. 

[Champ Clark III [Quadrant]]

I should note,  I _have_ re-structured  Sagan and it's ready for these new "processors",  I just haven't actually wrote any processors yet! 

On Dec 5, 2011, at 4:14 PM, DigiAngel wrote:

[DigiAngel]

Hehe...good deal...I be getting the latest git this week and testing
it out. Thanks again Champ.

James

On Dec 5, 2:29 pm, "Champ Clark III [Quadrant]"