Possible to output fluentd system logs in JSON format?

446 views
Skip to first unread message

jmo...@zendesk.com

unread,
Dec 19, 2017, 3:41:22 PM12/19/17
to Fluentd Google Group
I was looking through the docs about the logs that fluentd itself generates: https://docs.fluentd.org/v0.12/articles/logging

When I look at the logs it's generating, I see lines like this:

2017-12-19 20:21:29 +0000 [warn]: buffer flush took longer time than slow_flush_log_threshold: plugin_id="object:3fcf14b0f6f8" elapsed_time=38.915493082 slow_flush_log_threshold=20.0

Is it possible to configure fluentd to output its own logs in JSON format?

I'm forwarding all the logs to a EFK stack, and it would be great if those values like elapsed_time and slow_flush_log_threshold were something I could easily index and search on.

In theory I could try to reverse-engineer regexs to parse those logs and extract the values, but it would be a lot easier to just get those logs in a structured way.

Thanks,
Jon

Mr. Fiber

unread,
Dec 19, 2017, 3:52:56 PM12/19/17
to Fluentd Google Group
Capturing fluentd logs with <match fluent.**> is one approach: https://docs.fluentd.org/v0.12/articles/logging#capture-fluentd-logs
You can process fluentd logs by plugins.

In v1.0, you can use <log> directive in <system> to change log format: https://github.com/fluent/fluentd/pull/1644
I forgot to add this parameter to logging article. I will do it.


Thanks,
Masahiro 


--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Jon Moter

unread,
Dec 19, 2017, 4:11:21 PM12/19/17
to flu...@googlegroups.com
Perfect, thank you!

Any word when the fluentd daemonset will support v1.0, so I can run that in my Kubernetes clusters?


Reply all
Reply to author
Forward
0 new messages