add log name as well like tomcat Catalina.out. I am not able to filter name of logs on elasticsearch

[Kuldeep Yadav]

Hi,

i am trying to monitor all logs via fluentd-elasticsearch. showing my config file

<source>

 type secure_forward

 shared_key sUlWoNk

 secure false

 self_hostname ${hostname}

 cert_auto_generate no

 buffer_type file

 buffer_path /var/log/td-agent/secure_buffer

 flush_interval 15s

 buffer_queue_limit 256

 buffer_chunk_limit 64m

 port 2514

</source>

<match app.**>

 type forest

 #remove_prefix app

 subtype copy

 <template>

   <store>

     type elasticsearch

     host localhost

     port 9200

     flush_interval 5s

     logstash_format true

     utc_index false

   </store>

 </template>

</match>

When i am seeing these logs on kibana interface. I am not able to filter which logs are these means i am able to find the server name but not able to find log name like...access.log, error.log, Catalina.out etc...

[Kuldeep Yadav]

and please help me regarding tomcat logs template.

[Mr. Fiber]

See this article: http://www.fluentd.org/guides/recipes/apache-add-hostname

record_transformer filter with dynamic configuration, "#{}", may help.

> please help me regarding tomcat logs template.

https://github.com/GoogleCloudPlatform/fluentd-catch-all-config/blob/master/configs/config.d/tomcat.conf

One idea is using none like above.

Other approaches are writing regex for tomcat with fluentd-ui or

using grok parser: http://blog.lanyonm.org/articles/2014/01/12/logstash-multiline-tomcat-log-parsing.html

Masahiro

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kuldeep Yadav]

Hi,

Thanks for your quick response.Need some more help

1. remove old indices more than one week

2. how i write format of any logs

Kuldeep Yadav   i3clogic pvt. ltd

Mobile: 9999070782 Website: http://www.3clogic.com Skype: kuldeepyadav.221

HAPPY NEW YEAR!

Get a signature like this: Click here!

--
You received this message because you are subscribed to a topic in the Google Groups "Fluentd Google Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/fluentd/pahqBZmRyU0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to fluentd+u...@googlegroups.com.

[Mr. Fiber]

> 1. remove old indices more than one week

This is non fluentd question.

Please post such question on Elastic forum.

> 2. how i write format of any logs

fluentd-ui (td-agent-ui in td-agent) provides interactive regexp editor.