Fluentd statsd plugin

[felippe...@corp.globo.com]

Hi!

This is my conf:

<source>

  type serverstatus #https://gist.github.com/chrischang12/943a69b02f3435281557

  uri mongodb://user:pass@localhost:27017/admin

  stats_interval 2s

</source>

<match serverstatus.**>

  type statsd

  host udp.mystatsdserver.com

  port 8125

</match>

I saw the mongo logs and the metrics are being collected by td-agent but the problem is that, apparently, td-agent is not sending the metrics to statsd (confirm it by running "sudo tcpdump -nn -i any udp and src host <your_ip>" ).

Does anybody went trough this problem?

Another doubt I have is that how can I configure a "statsdkey" param inside the conf?

Regards,
Felippe Raposo.

[Masahiro Nakagawa]

Hi,

https://github.com/fakechris/fluent-plugin-statsd

statsd plugin is BufferedOutput so need some time to flush the buffer.

Default flush_interval is 60s.

http://docs.fluentd.org/articles/output-plugin-overview

http://docs.fluentd.org/articles/buffer-plugin-overview

No flush after 1 minute passed?

Is there no error logs in /var/log/td-agent/td-agent.log?

Thanks,

Masahiro

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Felippe da Motta Raposo]

Hi,

thanks by your response but there are no error longs in  /var/log/td-agent/td-agent.log.

The process ran all the night and this is the log output. 

2014-10-09 09:52:42 -0300 [info]: starting fluentd-0.10.53

2014-10-09 09:52:42 -0300 [info]: reading config file path="/etc/td-agent/td-agent.conf"

2014-10-09 09:52:42 -0300 [info]: gem 'fluent-mixin-config-placeholders' version '0.2.4'

2014-10-09 09:52:42 -0300 [info]: gem 'fluent-mixin-plaintextformatter' version '0.2.6'

2014-10-09 09:52:42 -0300 [info]: gem 'fluent-plugin-flume' version '0.1.1'

2014-10-09 09:52:42 -0300 [info]: gem 'fluent-plugin-mongo' version '0.7.3'

2014-10-09 09:52:42 -0300 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.4.1'

2014-10-09 09:52:42 -0300 [info]: gem 'fluent-plugin-s3' version '0.4.0'

2014-10-09 09:52:42 -0300 [info]: gem 'fluent-plugin-scribe' version '0.10.10'

2014-10-09 09:52:42 -0300 [info]: gem 'fluent-plugin-statsd' version '1.0.3'

2014-10-09 09:52:42 -0300 [info]: gem 'fluent-plugin-td' version '0.10.21'

2014-10-09 09:52:42 -0300 [info]: gem 'fluent-plugin-td-monitoring' version '0.1.3'

2014-10-09 09:52:42 -0300 [info]: gem 'fluent-plugin-webhdfs' version '0.2.2'

2014-10-09 09:52:42 -0300 [info]: gem 'fluentd' version '0.10.53'

2014-10-09 09:52:42 -0300 [info]: using configuration file: <ROOT>

  <source>

    type serverstatus

    uri mongodb://user:pass@localhost:27017/admin

    stats_interval 2s

  </source>

  <match serverstatus.**>

    type statsd

    host udp.statsd.fake.com

    port 8125

  </match>

</ROOT>

2014-10-09 09:52:42 -0300 [info]: adding source type="serverstatus"

2014-10-09 09:52:42 -0300 [info]: adding match pattern="serverstatus.**" type="statsd"

--

Best,

Felippe Raposo.

[Felippe da Motta Raposo]

Hi!

I figured out the solution and it is quite simple. I had to install the fluentd reformer plugin[1] and set up the configuration to send statsd specific keys(as show in the example below).

    <source>

      type serverstatus #https://gist.github.com/chrischang12/943a69b02f3435281557

      uri mongodb://user:pass@localhost:27017/admin

      stats_interval 2s

    </source>

    <match serverstatus.reformer>

      type copy

      <store>

        type statsd

        host udp.mystatsdserver.com

        port 8125

        flush_interval 4s

      </store>

    </match>

    <match serverstatus.**>

      type record_reformer

      tag serverstatus.reformer

      statsd_key mykey

      statsd_count ${my_field}

      statsd_type ${"count"}

    </match>

[1]: https://github.com/sonots/fluent-plugin-record-reformer

--

Best,

Felippe Raposo.

[Masahiro Nakagawa]

Good! Enjoy Fluentd :)