monitoring via python

[skac...@gmail.com]

Hi,
Could you point me to api py classes and packages that I can extend to monitor workflow and task execution and to stop or archive workflows.

Also, it seems that no matter how I setup a rapidfire to run or store launch directories, I can't never execute binary wrapper from the relative path.

I mean this:

cd  /cube/apps/
./myapp.sh


I always need to execute absolute path to wrapper.
I guess I could use to tasks, in a workflow but I am trying to do this via rapidfire or lpad setup....
Regards,
 

[Anubhav Jain]

Hi Sasha,

Most of the code to manage the database is in launchpad.py. You can search for keywords such as "archive" or "defuse" (pause/stop) and "reignite" (resume).

I am pretty confused about your question regarding rapidfire. The only point of rapidfire is to create a new directory and pull a job from the database, and then repeat this multiple times. It should not be calling any other scripts; as you mentioned, that should go inside the task. Similarly, the only point of lpad setup is to create database credential files. Neither of these commands should be used to launch external scripts.

Note that if you want to run tasks in a particular directory, you can set the _launch_dir keyword as specified here (although I'm not sure this is your question):

https://pythonhosted.org/FireWorks/controlworker.html

--
You received this message because you are subscribed to the Google Groups "fireworkflows" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fireworkflow...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fireworkflows/8e9ca530-7cd8-43dd-a960-1bd1ee42617b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sasha Kacanski]

Hi Anubhav,
Thanks for pointers.
I Created pretty extensive wrapper around fireworks.
Everything works.
As model I use script based template to submit tons of tasks as oen or many workflows. after that workflow is submitted to mongoDB.
That all works. I actually create task from my own manifest passing arguments line at the time.
Fw client just register  itself and waits for jobs to process.

I read the page you are suggesting.

The question is this:

launch_dir key is setup via launchpad module, correct?

I am not using yaml, everything is done via python templates ...

I will check launchpad.py for bellow methods ...

I mentioned rapidfire because of the m_dir key that I am using to specify where the log files are going (loop is running)

My apologies for not clarifying exactly what I am trying to achieve...

[Anubhav Jain]

Hi Sasha,

Thanks for the update. I am not sure how much I am able to help, however in regard to your specific question about "launch_dir", that is set in the FW spec itself as per the docs:

"set the _launch_dir key in your Firework spec to the full path of the directory you want to execute the Firework in"

i.e., my_fw = Firework([tasks], spec={"_launch_dir":"/where/I/want/to/run/the/job"})

Best,

Anubhav

To view this discussion on the web visit https://groups.google.com/d/msgid/fireworkflows/CAFWiiko9zwhZkej7SM40xmjpRFG7VbnJ7-HrjWOouP3ze1JK9w%40mail.gmail.com.

[skac...@gmail.com]

Cool thanks, now it is working exactly as I want...