fluent-plugin-grok-parser: Fail to parse multiple fields

[Ziva Zhang]

I am trying to use fluent-plugin-grok-parser to parse multiple fields, following the usage in https://github.com/kiyoto/fluent-plugin-grok-parser.

However it is found that only the first parsing could succeed.

My configs:

<source>

  type tail

  format grok

  path /var/log/opt/cisco/elk_collect/elk_collect_stats.log

  pos_file /var/log/td-agent/elk_collect_stats.log.pos

  <grok>

    pattern "CPUUtil %{NUMBER:pam_cpu_util:float}"

  </grok>

  <grok>

    pattern "KiBMemTotal %{INT:pam_mem_total_kb:integer}"

  </grok>

  tag "#{Socket.gethostname}.elk_collect_pam_stats.log"

</source>  

Sample Log Messages:

ELKDataNo 132, 2016-09-18T08:26:44 UTC, EventCount 0, ErrorCount 2932, WarningCount 2932, CPUUtil 1.4, KiBMemTotal 16433044, KiBMemUsed 3555320, KiBMemFree 12877604, KiBMemBuffer 1308

Result:

The first field pam_cpu_util could be parsed and indexed as value 1.4 successfully, while the 2nd field pam_mem_total_kb can't be parsed at all.

Is there anything wrong with my configs?

BTW, it also works if I use grok_pattern "CPUUtil %{NUMBER:pam_cpu_util:float}, KiBMemTotal %{INT:pam_mem_total_kb:integer}" instead. But this way has dependency on neighbor fields, which is not what I desires.

[Mr. Fiber]

> Is there anything wrong with my configs?

https://github.com/kiyoto/fluent-plugin-grok-parser#how-it-works

Documentation says "If you want to try multiple grok patterns and use the first matched one",

so the result seems expected behaviour.

It seems intuitive implementation.

If you want logstash grok's 'match' feature, grok-parser seems to not support yet.

https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html

Masahiro

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ziva Zhang]

Got it. Thanks, Masahiro!

To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+u...@googlegroups.com.