Hi Anubhav,
Thanks on the answer,
I am familiar with MPI framework, I used it in the past.
Maybe I am not enough familiar with fireworks, but if I am understanding you correctly, fireworks clients or computer setup node could be setup to listen for jobs.
My only concern is following:
I want to programmatically setup a node (computer resource) to listen for jobs on each node.
Let's say I have four servers, they all share fireworks and python via NFS. fifth node is running mongoDB and has also programmatically setup firework framework to submit same job as four tasks or workflows.
You are right, jobs are same python app with instructions on how to slice big data
All I want to is to submit these tasks or work flows and as schedule them for parallel execution on each node.
If you are telling me that fireworks has nothing that synchronize this but relies on the external protocol framework like MPI to do that via hooks, then I will need to go that way.
I understood that fireworks has also a built in queue, possibly via mongodb or otherwise. I apologies I did not look at the source yet...
Let me know if I am correct, and I will use your mlaunch tutorial!
Thanks again, --sasha