Processing logs that have been relayed through other syslog server

[Russell Fulton]

HI

We have central syslog server set up and for various reasons we can not set up elsa nodes on these machine. What we are doing is forwarding selected logs to the machine running elsa. For some things this does not matter — all the information we need is in the actual log record. For other things like sshd logs the only way of knowing which machine is involved is by the source IP.

getting syslog-ng to spoof the source addresses is not really an option because of the network and firewall set ups.

Is there anyway to get syslog-ng to forward logs while preserving the identity of the original source other than using UPP and spoofing the source addresses ?

Russell

[C. L. Martinez]

What syslog product do you use in your central syslog server??

[Martin Holste]

This can be done, but it is dangerous because it can make ELSA do lots of DNS lookups (which are a blocking operation) and can cause the system to drop logs.  To do this, you can switch $SOURCEIP in the syslog-ng.conf template for ELSA to be $HOST.  Assuming the forwarder is using proper syslog protocol forwarding, and the hostnames can be resolved, then this should work.

--
You received this message because you are subscribed to the Google Groups "enterprise-log-search-and-archive" group.
To unsubscribe from this group and stop receiving emails from it, send an email to enterprise-log-search-...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Dan]

If your sending server is syslog-ng as well, it's easy. Just move the host rewrite logic to the source server instead.

On ELSA node's syslog-ng.conf, comment out the line "rewrite(r_host)"
Then, on the source server, add that r_host rewrite and apply it to the log destination pointing at ELSA.

[Russell Fulton]

we are decided to go with syslog-ng so If if did become feasible to run ELSA on the central system we could.

Russell

[Russell Fulton]

On 14/03/2013, at 6:44 AM, Dan <pete...@uwosh.edu> wrote:

> If your sending server is syslog-ng as well, it's easy. Just move the host rewrite logic to the source server instead.

This sounds promising! I'll have a proper look when I get in to work. I was somewhat surprised that syslog-ng did not explicly provide any except spoofing addresses.

>
> On ELSA node's syslog-ng.conf, comment out the line "rewrite(r_host)"
> Then, on the source server, add that r_host rewrite and apply it to the log destination pointing at ELSA.
>

Russell

[r.fu...@auckland.ac.nz]

Finally getting back to this...

On Thursday, March 14, 2013 2:45:09 AM UTC+13, Martin wrote:

This can be done, but it is dangerous because it can make ELSA do lots of DNS lookups (which are a blocking operation) and can cause the system to drop logs.  To do this, you can switch $SOURCEIP in the syslog-ng.conf template for ELSA to be $HOST.  Assuming the forwarder is using proper syslog protocol forwarding, and the hostnames can be resolved, then this should work.

I have just looked carefully at the templates in the latest syslog-ng.conf and I find:

rewrite r_host { set("$SOURCEIP", value("HOST")); };

template t_db_parsed { template("$R_UNIXTIME\t$HOST\t$PROGRAM\t${.classifier.class}\t$MSGONLY\t${i0}\t${i1}\t${i2}\t${i3}\t${i4}\t${i5}\t${s0}\t${s1}\t${s2}\t${s3}\t${s4}\t${s5}\n"); };

 

I am confused -- my older conf files have $SOURCEIP in the template and no r_host rewrite.  

Has something changed in the last few weeks?

I did change my test template that was using $SOURCEIP to use $HOST and that produced the host names in that output stream.

I am about to try Dan' suggestion of moving the host rewrite on one of the feed syslogs as well.

[Russell Fulton]

On 4/04/2013, at 1:26 PM, <r.fu...@auckland.ac.nz>
wrote:

> Finally getting back to this...
>
> On Thursday, March 14, 2013 2:45:09 AM UTC+13, Martin wrote:
> This can be done, but it is dangerous because it can make ELSA do lots of DNS lookups (which are a blocking operation) and can cause the system to drop logs. To do this, you can switch $SOURCEIP in the syslog-ng.conf template for ELSA to be $HOST. Assuming the forwarder is using proper syslog protocol forwarding, and the hostnames can be resolved, then this should work.
>
> I have just looked carefully at the templates in the latest syslog-ng.conf and I find:
>
> rewrite r_host { set("$SOURCEIP", value("HOST")); };

having RTFM I see this does exactly the opposite of what I expected — now things make sense.

So I can manipulate the things by varying the r_host pattern.

Russell

[Russell Fulton]

Oh dear, apologies for a string of reply to self!

On 4/04/2013, at 2:26 PM, Russell Fulton <r.fu...@auckland.ac.nz>
wrote:

>

Having got the $HOST into the template and got rid of <hostname>/<hostname> by tweaking r_host I find that ELSA shows all Host as 0.0.0.0.

I am guessing that this is because the perl script is expecting an IP not a name.

I have now moved the rewrite to the sending machine and overridden it on the elsa node and it all looks good! I finally have real source IPs showing up in host!

I think this method of handling forwarding deserves a explicit note in the docs…

BTW I am using TCP to forward ….

Russell

[Martin Holste]

I'm sure some more documentation is needed.  Can you write a few lines of what you're looking for?

Russell

[Russell Fulton]

On 4/04/2013, at 4:59 PM, Martin Holste <mcho...@gmail.com>
wrote:

> I'm sure some more documentation is needed. Can you write a few lines of what you're looking for?
>

most certainly!

Not quite sure where it should go… an FAQ or Tips section ?

Tip:

When relaying logs from another syslog-ng server where you can not spoof the source IP address but still want to retain the correct host attribute in ELSA you can rewrite the HOST macro on the sending system:

rewrite r_elsa { set("$SOURCEIP", value("HOST")); }; # put the sourceIP into the HOST macro

and include this in your elsa log definition:

log { source(s_net); filter(f_elsa_tcp); rewrite( r_elsa); destination(elsa_tcp); };

On your elsa node in your /etc/elsa_syslog-ng.conf you need to override r_host:

rewrite r_host {};

Note that this is *global* and will affect all sources.

If one wanted to do this for a subset of sources then one would need to have separate source log definitions and name the rewrite rule something other than r_host. Doable but not pretty and it will mean that you would need to carefully check your syslog setup after each update.

======================================================

comments anyone? you can probably do this with rsyslog as well...

Russell