Getting nodes to pull jobs from fireworks

[Michael B]

Hello,

I'm trying to run jobs on remote computing clusters that pull from the fireworks database on our local server. I've set this up using ssh tunnels to secure the connection and have gotten rlaunch to work fine on the remote cluster. However, qlaunch is causing errors because the ssh tunnel only connects our local server to the head node, not any of the other nodes that the job may be sent to through the queue. My question is how can you get all nodes on a remote cluster to be able to read from a local fireworks database through a secure connection? My initial idea is to include a tunnel-activating script in the pre-rocket (and a tunnel-closing script in the post-rocket) of the qadapter but I want to know if there's something I'm overlooking that would make this simpler.

Thanks,

Michael

[Anubhav Jain]

Hi Michael

Three options that I can think of (none of them ideal) are:

1. Use offline mode, which is not ideal but should work:

https://pythonhosted.org/FireWorks/offline_tutorial.html

2. Put the tunnel scripts in the pre and post rocket like you suggest

3. Relocate your fireworks database to somewhere that is accessible by your workers

Sorry there aren't any other better options at the moment -

Best,

Anubhav

--
You received this message because you are subscribed to the Google Groups "fireworkflows" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fireworkflows+unsubscribe@googlegroups.com.
To post to this group, send email to firewo...@googlegroups.com.
Visit this group at https://groups.google.com/group/fireworkflows.
To view this discussion on the web visit https://groups.google.com/d/msgid/fireworkflows/14325733-d02f-4e92-b127-c8ee98d1fcf2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Best,
Anubhav

[Michael B]

 Hi Anubhav,

Thank you for  the suggestions, I will take a look at offline mode.

Michael