ScriptTask and stored_data

[Jeff Graves]

Is it possible for a ScriptTask to pass variables from stored_data as command line parameters to the script?

[Anubhav Jain]

Hi Jeff,

No, the ScriptTask can only take parameters from the "spec" (if use_global_spec is set True) or from the initialization parameters from the task (default). If you have a previous task/Firework that needs to pass some parameter to the following ScriptTask (e.g. the "script" command), you can have the previous Firework pass the an "update_spec" command to the downstream ScriptTask to use the new "script". i.e. this will modify the spec of the downstream ScriptTask

If what you're doing is more complicated than that, I would suggest upgrading to PyTask or writing your own FireTask

Best,

Anubhav

On Mon, Aug 8, 2016 at 12:00 PM, Jeff Graves <jagra...@gmail.com> wrote:

Is it possible for a ScriptTask to pass variables from stored_data as command line parameters to the script?

--
You received this message because you are subscribed to the Google Groups "fireworkflows" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fireworkflows+unsubscribe@googlegroups.com.
To post to this group, send email to firewo...@googlegroups.com.
Visit this group at https://groups.google.com/group/fireworkflows.
To view this discussion on the web visit https://groups.google.com/d/msgid/fireworkflows/880e948c-96fe-4413-ae96-b55ff7a3a6db%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Best,
Anubhav

[Jeff Graves]

Thank you for your feed back.

I have one more question pertaining to ScriptTask.  I have a FireServer and several FireWorkers running across 20 machines.  I have a custom binary (compiled C++ program) I am trying to execute with ScriptTask.  Is there any way to get the executable to the FireWorker that will need to execute it with out putting it on an NFS or pre-staging it to every possible worker?

I thought I could accomplish this by writing a Firework that contained the FileTransferTask followed by the ScriptTask to first pull the executable before running it.  However, as far as I can tell, I can only use the FileTransferTask to send data, not pull data.  Is this correct?

On Monday, August 8, 2016 at 3:07:33 PM UTC-4, ajain wrote:

Hi Jeff,

No, the ScriptTask can only take parameters from the "spec" (if use_global_spec is set True) or from the initialization parameters from the task (default). If you have a previous task/Firework that needs to pass some parameter to the following ScriptTask (e.g. the "script" command), you can have the previous Firework pass the an "update_spec" command to the downstream ScriptTask to use the new "script". i.e. this will modify the spec of the downstream ScriptTask

If what you're doing is more complicated than that, I would suggest upgrading to PyTask or writing your own FireTask

Best,

Anubhav

On Mon, Aug 8, 2016 at 12:00 PM, Jeff Graves <jagra...@gmail.com> wrote:

Is it possible for a ScriptTask to pass variables from stored_data as command line parameters to the script?

--
You received this message because you are subscribed to the Google Groups "fireworkflows" group.

To unsubscribe from this group and stop receiving emails from it, send an email to fireworkflow...@googlegroups.com.

To post to this group, send email to firewo...@googlegroups.com.
Visit this group at https://groups.google.com/group/fireworkflows.
To view this discussion on the web visit https://groups.google.com/d/msgid/fireworkflows/880e948c-96fe-4413-ae96-b55ff7a3a6db%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Best,
Anubhav

[Anubhav Jain]

Hi Jeff

You are correct that the FileTransferTask implementation only allows for transfer to remote servers, and not from remote servers. This was not done intentionally; if you wanted to update the code to allow two-way transfers, I would be happy to include it in the repo. Unfortunately, I don't have much time at the moment to implement this myself.

If you have passwordless SSH set up, will it work to just include a ScriptTask that will scp the needed file from wherever it is located? i.e., use a ScriptTask instead of FileTransferTask to do the file transfer with scp.

Best,

Anubhav

[Jeff Graves]

I was wrapped up with trying to fine a native FireWorks solution that I didn't think of using scp.  That should work just fine for now.