Running all the plugins at the start forcefully

[meru....@gmail.com]

Hi

I am working on an application which would have a run_interval of 12 hours. When I start the FluentD, I have to wait till 12 hours elapse to get to see any data from the box. Is there any setting which forcefully runs the plugin. The conf file looks like the following:

 <source>
  type exec
  command /work/folder/RunThis.py > /dev/null && cat /work/folder/target/Completed.xml
  format tsv
  keys xml
  run_interval 12h
</source>
<filter test_filter>
  type filter_process_xml
  log_level trace
</filter>

Also, does fluentD maintain state when restarting? When I restart fluentD, the time-counter resets.

Any pointers would be very helpful.

Thanks in advance.

Regards,

Varad

[Eduardo Silva]

the run_interval will always start counting since Fluentd starts. Note that the main job of Fluentd is about log collection instead of job scheduling.

Since you have a python script that generate your .xml, I would suggest you schedule that program in your Crontab service and let Fluentd 'tail' the target file/directory where your script is updating the content.

is this for OMS ?

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Eduardo Silva

Open Source, Treasure Data

http://www.treasuredata.com/opensource

 

[meru....@gmail.com]

Thanks for your response. Yes, this is for OMS.

The problem of doing the cron initiated python workload comes when the agent restarts and the cron become independent. We might want to turn the agents off, or restart together but then a separate wrappers are needed for the multiple items being restarted.

Also, I do understand that the primary task of Fluentd is log collection but then is there a way to control or change fluentd's scheduler?

Thanks,

Varad