Rewrite syslog tag to the source hostname

[Jesper Eneberg]

Hi,

I use a central fluentd to receive logs from several servers using syslog and then route those logs to other systems such as papertrail via the plugin: fluent-plugin-remote_syslog. 

However, when the logs reaches its destination (papertrail) they all have the same tag. The tag is set by the plugin, eg:

<match syslog.**>

 type remote_syslog

 tag fluentd

 host XXXX

 port XXXX

</match>

Is it possible to set the tag to the "host" value I get from the syslog? Eg: The tag changes depending on the source server - if I send syslog message from the server logtest01 the tag should be logtest01, if I send from logtest02, the tag should be logtest02.

The syslog looks like this:  {"host":"SERVERHOSTNAME","ident":"USER","pid":"9999","message":"MESSAGE_123"}

I did try the rewrite_tag_filter plugin, but I think I fail with the regexp.... This is how my config looks at the moment:

###Syslog input - this tags all with "syslog"

<source>

  @type syslog

  protocol_type tcp

  port 5140

  bind 0.0.0.0

  tag syslog

</source>

###Rewrite the tag to: syslog."SourceHostname" depending on the HOST value in the logentry.

<match syslog.**>

  @type rewrite_tag_filter

  rewriterule1 host ^(\w+) syslog.$1 

</match>

###Sends all syslog-tagged logs to papertrail

<match syslog.**>

 type remote_syslog

 tag ${tag}

 host XXXX

 port XXXX

</match>

Regards,
Jesper

[Mr. Fiber]

> I did try the rewrite_tag_filter plugin, but I think I fail with the regexp....

It means ^(\w+) doesn't match your hostname value?

How about testing regexp by using irb or ruby script?

Masahiro

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jesper Eneberg]

Hi,

Yeah the regexp doesn't seem to work. I'm kinda bad with regexp so any help would be appreciated.

This is the current workflow: 

1. Logs get sent from my client websrv01 via rsyslog to fluentd.

2. Fluentd receives the syslog message via syslog input and tags the message with "rsyslog". 

3. Here I want to rename that tag to the client hostname, so the tag will be: rsyslog.websrv01. - The syslog message has a "Host" value which contains the hostname and this is what i've been trying to get with rewrite_tag_filter

4. Send the matching rsyslog.** to the destinations. The destinations can now see the client hostname (websrv01) as tag.

Thank you

Jesper

[Mr. Fiber]

You can test you regexp on this site: http://rubular.com/

BTW how about using ^(.+) instead of ^(\w+)?

On Fri, Jul 22, 2016 at 5:49 PM, Jesper Eneberg <jes...@eneberg.nu> wrote:

Hi,

Yeah the regexp doesn't seem to work. I'm kinda bad with regexp so any help would be appreciated.

This is the current workflow: 

1. Logs get sent from my client websrv01 via rsyslog to fluentd.

2. Fluentd receives the syslog message via syslog input and tags the message with "rsyslog". 

3. Here I want to rename that tag to the client hostname, so the tag will be: rsyslog.websrv01. - The syslog message has a "Host" value which contains the hostname and this is what i've been trying to get with rewrite_tag_filter

4. Send the matching rsyslog.** to the destinations. The destinations can now see the client hostname (websrv01) as tag.

Thank you

Jesper

Den torsdag 21 juli 2016 kl. 17:36:05 UTC+2 skrev repeatedly: