Parsing audit.logs on Fluentd Client side

[Zeal Vora]

Hi

We are using grok to parse the audit.logs . The grok pattern is correct ( verified in grok debugger ) . The configuration seems to be working perfectly after restart, but on the Kibana, the message still aren't parsed. Everything comes in one line.

Any help would be appreciated.  This is the current fluentd configuration side 

<source>

  type config_expander

  <config>

    type tail

    format none

    path /var/log/audit/audit.log

    pos_file /var/log/td-agent/tmp/audit.log.pos

    tag sys.test

 </config>

    format grok_pure

    grok_pattern type=%{WORD:audit_type} msg=audit\(%{NUMBER:audit_epoch}:%{NUMBER:audit_counter}\): pid=%{NUMBER:audit_pid} uid=%{NUMBER:audit_uid} auid=%{NUMBER:audit_auid} ses=%{NUMBER:audit_ses} msg=%{GREEDYDATA:audit_message}

    grok_pattern_path /etc/td-agent/plugin/fluent-plugin-grok-parser/patterns/grok-patterns

</source>

<match sys.*>

  type forest

  subtype copy

  <template>

    <store>

      type elasticsearch

      logstash_format true

      host 192.168.25.100

      port 9200

    </store>

  </template>

</match>

[Zeal Vora]

I fixed it.

Basically, i decided to use some other plugin which had more downloads.  I switched to : fluent-plugin-grok-parser  ;  https://github.com/kiyoto/fluent-plugin-grok-parser 

With the newer plugin, i did not have to mention grok_pattern_path and other details, which solved the issue.

[Kiyoto Tamura]

Glad to hear (I am the author of the plugin)

Let us know if you have any question. Also, it'd be nice if you could give Treasure Data a shot (disclaimer: I am their head of marketing)

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Find out how Wish.com, the world's largest mobile shopping site, builds its recommendation engine.

[Zeal Vora]

Thanks alot Kiyoto. I'll check Treasure Data also :)

--
You received this message because you are subscribed to a topic in the Google Groups "Fluentd Google Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/fluentd/LUUjKdl2FNU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to fluentd+u...@googlegroups.com.