forward the logs to s3

[NIKHIL SHARMA]

Hi,

We are using different servers and forwarded their log to log aggregator then forwarded their logs  to s3 buckets.

I am facing one challenge can we differentiate the stored logs on s3 with host name logs coming ,in that way we can easily track the logs.

Regards,

Nikhil

[NIKHIL SHARMA]

i have an idea for the forest plugin will that work in my case as i want logs which i am forwarding to s3 can be differentiated with host name when  they stored on  
my bucket.

Regards,

Nikhil

[Lance N.]

This is how I do it, with the forest plugin. In my Fluentd startup script i get the host's IP and pass that in as an environment variable IP. Inside the forest description I fetch it from the environment when describing the full path name.

One point: you can buffer output data in memory or on disk. If you buffer on disk, writers inside a forest plugin DO NOT go and find their buffer files after a program restart.

[NIKHIL SHARMA]

Hi thanks for the reply.

I am new to fluentd if you can share the kind of configuration it is very beneficial for me.So i can use for my environment.

Thanks

Nikhil