Grok with if statement

[joaquin.sil...@gmail.com]

Hi, 

I'm new in Fluentd and I came form the Logstash world. I need to know that if this Logstash conf code is possible to transforme it into a Fluentd conf.

filter{ 

     if "a" in [message] 
     { 
              grok { match => { "message" => "grok pattern A" } } 
     }

     else if "b" in [message] 
     { 
              grok { match => { "message" => "grok pattern B" } } 
     }
}

I installed the grok parser plugin, but my problem is, how to do I insert the IF statement in the <source>?

My current code is:

<source>
  type tail
  path /home/ubuntu/datos-prueba/*.log
  pos_file /var/log/td-agent/tmp/access.log.pos
  tag input
  format grok
  <grok>
    pattern A
  </grok>
  read_from_head true
</source>

Thanks and regards,

[Mr. Fiber]

Hi,

You can use rewrite-tag-filter plugin for such purpose.
In addition, if your pattern is different, multi-forma-parser is another candidate.

Masahiro

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[joaquin.sil...@gmail.com]

Hi Masahiro,

Fluent-plugin-rewrite-tag-filter use regex and I need grok, because all my logstash conf file use it. Do you have an example of doing that (with grok)?, that will be very helpful.

[Mr. Fiber]

From your example, is "a" of 'if "a" in [message]' grok pattern?

[joaquin.sil...@gmail.com]

I will explain it better. I have to do differents grok pattern depending of the current message that arrives. For example if I have 2 types one that contains the string "string1" and the other that contains "string2". For each one i have to do a different grok pattern (grok_pattern_A and grok_pattern_B respectively), because they have different format. It is something like this:

filter{
	if "String1" in [message]
	{
	grok { match => { "message" => "grok_pattern_A" } }
	}
	else if "String2" in [message]
	{
	grok { match => { "message" => "grok_pattern _B" } }
	}
	}

[Mr. Fiber]

Following confiruation is one example for your case:

<match tag.pattern>

  @type rewrite_tag_filter

  rewriterule1 message string1 a.tag.pattern

  rewriterule1 message string2 b.tag.pattern

</match>

<filter a.tag.pattern>

  @type filter

  parser grok

  # grok configuration

</filter>

<filter b.tag.pattern>

  @type filter

  parser grok

  # grok configuration

</filter>

# other matches.

If you don't need to separate data flow for string1 and string2,

using multiple grok patterns seems enough.

https://github.com/kiyoto/fluent-plugin-grok-parser#how-it-works

[joaquin.sil...@gmail.com]

Thanks! i will try that.

[joaquin.sil...@gmail.com]

I didn't found a filter plugin called filter. All that I found was https://github.com/muddydixon/fluent-plugin-filter and it don't seems to do what I want. Cand you tell me please what plugin is that?.

Regards 

El domingo, 24 de abril de 2016, 2:11:08 (UTC-3), repeatedly escribió:

[Mr. Fiber]

Sorry this is a typo. Use parser filter, @type parser,  instead of `@type filter`.

[joaquin.sil...@gmail.com]

Thanks for the reply, now I'm testing the rewrite tag filter, but i don't know what's wrong, it don't assign any tag. This is my conf file:

	<source> @type tail
	path /home/ubuntu/datos-prueba/SG04_Itau_LogsInformes_ssl__37_160413122345.log
	pos_file /var/log/td-agent/tmp/access.log.pos
	tag input
	format none
	read_from_head true
	</source>
	#assign sslyes tag if the log contains the string "ssl", otherwise assign sslno
	<match input>
	@type rewrite_tag_filter
	capitalize_regex_backreference no
	rewriterule1 message ssl sslyes
	rewriterule2 message ^((?!ssl).)*$ sslno
	</match>
	<match sslyes sslno>
	@type amqp
	host xxxxxx
	port 5672
	user xxxxx
	password xxxxxx
	vhost /
	exchange xxxxxx
	exchange_type topic
	exchange_durable true
	payload_only true
	content_type application/octet-stream
	</match>