Integration of fluentd-Redis-Logstash

[Parima Soni]

Hello All, 

I am working on fluentd and logstash, Elasticsearch, and kibana integration, To integrate fluentd and logstash, I am using " Fluentd-Redis-Logstash".

I have done all the configuration for redis, fluentd and logstash. Isolated containers are up and running fine, but when I am passing the logs I am not getting the logs in fluentd.

I have added redis output plugin also. still I am unable to integrate "FRL" pipeline.

Can you please guide me on the same.

Thanks & Regards:

Parima Soni  

[Mr. Fiber]

>  I am passing the logs I am not getting the logs in fluentd.

This is unclear.

Which part is the problem? Input doesn't work or output doesn't write events to redis?

Write more detailed information here for help.

Masahiro

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Parima Soni]

Hi Masahiro, 

1 Fluentd configuration:

you can see below configuration for fluent.conf file.when I am passing the logs, I am not getting logs in fluentd.

2017-06-26 10:25:29 +0000 [info]: reading config file path="/fluentd/etc/fluent.conf"

2017-06-26 10:25:29 +0000 [info]: starting fluentd-0.12.37

2017-06-26 10:25:29 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '1.9.5'

2017-06-26 10:25:29 +0000 [info]: gem 'fluent-plugin-redis' version '0.2.3'

2017-06-26 10:25:29 +0000 [info]: gem 'fluent-plugin-redis-store' version '0.1.1'

2017-06-26 10:25:29 +0000 [info]: gem 'fluent-plugin-secure-forward' version '0.4.5'

2017-06-26 10:25:29 +0000 [info]: gem 'fluentd' version '0.12.37'

2017-06-26 10:25:29 +0000 [info]: adding match pattern="pattern.**" type="copy"

2017-06-26 10:25:30 +0000 [info]: adding source type="syslog"

2017-06-26 10:25:30 +0000 [info]: using configuration file: <ROOT>

  <source>

    @type syslog

    port 42185

    bind 0.0.0.0

    tag syslog

    with_priority true

  </source>

  <match pattern.**>

    @type copy

    <store>

      @type stdout

    </store>

    <store>

      @type redis_store

      key userdata

      host REDIS_HOST_ADDRESS(string, all , 172.17.0.1)

      port REDIS_PORT(int, all, 6379)

      timeout TIMEOUT(float, all , 5.0)

      key_prefix prefix_(string, all, '')

      key_suffix _suffix(string, all , '')

      store_type zset(zset|set|list|string, all, zset)

      key_expire 604800(int, all, nil)

      value_expire 86400(int , zset, nil)

      value_length 100(int, zset|list, nil)

      order asc(asc|desc, zset/list, nil)

    </store>

  </match>

</ROOT>

2017-06-26 10:25:30 +0000 [info]: listening syslog socket on 0.0.0.0:42185 with udp

after that I have run my aaplication and route logs to the fluentd container,but fluetd is up and running but I am not getting logs after running fluentd container.

Redis server is up and running with port 6379.

Please check and let me know if any other information required.

Thanks & Regards:

Parima

To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+u...@googlegroups.com.

[Mr. Fiber]

> <match pattern.**>

Is this actual configuration?

in_syslog uses "#{@tag}.#{facility}.#{priority}" for the tag.

http://docs.fluentd.org/v0.12/articles/in_syslog#example-usage

To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+unsubscribe@googlegroups.com.