How to log debug level messages in default td-agent.log file (only getting info level).

3,319 views
Skip to first unread message

Corey H.

unread,
Apr 17, 2018, 7:08:41 PM4/17/18
to Fluentd Google Group
Hi Everyone

I would think this one would be easy but no luck thus far. When I run td-agent -vv I see info level and debug level logs. 

Example:
2018-04-13 16:59:03 +0000 [info]: following tail of /nsm/bro/logs/current/dns.log
2018-04-13 17:09:23 +0000 [debug]: Write chunk 569bdeea7bbfa1f84455e5041cab60ab /  73 records /   31 KB
2018-04-13 17:09:24 +0000 [debug]: Finish writing chunk
2018-04-13 17:09:28 +0000 [debug]: Write chunk 569bdef2fcbb7d52e85ae034e68006d1 /   2 records /    0 KB
2018-04-13 17:09:29 +0000 [debug]: Finish writing chunk
2018-04-13 17:09:33 +0000 [debug]: Write chunk 569bdef6cdb1f9299093036e9a2af1ed /   7 records /    3 KB

That works great, but I need those Debug logs to appear in a log file, preferably the default location (var/log/td-agent/td-agent.log). 
I'd also need this to work without having to run in verbose mode. Currently, based off the above output, td-agent.log will only 
retain the first record which is of the [info] type. 

Thanks

The version I'm running: td-agent-2.3.5-1.el7

My config file:

<system>
 @log_level debug
</system>

<source>
   @type tail
   path /nsm/bro/logs/current/http.log
   pos_file /var/log/td-agent/http.log.pos
   format none
   tag bro.http
 </source>

<filter bro.http>
   @type grep
 <exclude>
   key message
   pattern ^#
 </exclude>
</filter>

<filter bro.http>
   @type record_transformer
   <record>
    hostname "#{Socket.gethostname}"
   </record>
</filter>

<match bro.http>
 @type kinesis_firehose
 @log_level debug
 slow_flush_log_threshold 35
 ssl_verify_peer false
 delivery_stream_name xxx
 http_proxy xxx
 aws_key_id xxx
 aws_sec_key xxx
 region us-east-1
 flush_interval 5s
</match>

Mr. Fiber

unread,
Apr 19, 2018, 12:04:21 AM4/19/18
to Fluentd Google Group
> <system>
>  @log_level debug
> </system>


In system, use log_level instead of @log_level


Masahiro


--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Corey H.

unread,
Apr 25, 2018, 12:22:58 PM4/25/18
to Fluentd Google Group
In my case, this did not work. I ended up having make changes to the unit file as seen here:


Thank you for your reply!
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+u...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages