remote execution problem

[Foivos Gypas]

Hi guys

I have  a new question regarding anduril.
If you remember in a previous post (https://groups.google.com/forum/#!topic/anduril-dev/V2IBC_PMPrs) I try to transfer everything from anduril 1.2.5 to 1.2.22

Now everything seems to work properly apart from the remote execution.

The hosts.conf file is same. I did not change anything there.

In the previous version I was running anduril like this:

anduril run "/path/to/workflow.and" --execution-dir="/path/to/exec/dir/" --component-dir="/path/to/components/dir" --hosts="/path/to/hosts.conf"

In the new version I do the following:

anduril run "/path/to/workflow.and" --execution-dir "/path/to/exec/dir/" -b "/path/to/bundle" --hosts "/path/to/hosts.conf"

But in this case I get the following error message:

Local execution mode incompatible with non local @host=long_hm


I get a result but by local run. It is not executed remotely.


Do you have any idea? I am a little confused.


Thank you in advance
Foivos

[lauri...@gmail.com]

Not that I have used this feature in the new Anduril version, ... but I would guess you can get it to work by specifying "--exec-mode remote". See "anduril run" default output for the flags. The different exec modes were introduced to support remote execution with systems such as Slurm. The default mode probably changed at that point. There probably was only one mode at that time, so the guy who implemented this just did not maintain backwards compatibility by default, and nobody here noticed since everyone moved to the new system.

Slurm does a much better job of managing resource allocations than Anduril, so I would recommend installing it if possible. Afterwards you may use --exec-mode slurm. The hosts defined in slurm are still usable with the same @host annotation, so there are no changes required in the Anduril script. If you don't like Slurm, you could use any other command as a prefix for running Anduril component instances.

The main thing that Anduril's remote execution has that Slurm doesn't have is copying the files necessary for remote execution. Though, for big files this would be a considerable overhead in any case. You can work around this e.g. by mounting file systems so that file system paths map identically.

[Foivos Gypas]

Thank you for the reply.

In the future the remote exec mode will change. Propably we can try slurm.

For the moment I just want a working version with remote exec in order to test the current pipeline. It's not possible to perform it locally.


When I specify the "--exec-mode remote" I get the following error message:


[ERROR] Component XXXXX: Non-zero exit code with remote command: 1

Any ideas?

Thank you in advance for the support

[Foivos Gypas]

Acrtually what is wrong is that when the remote anduril script is called (http://anduril.googlecode.com/hg-history/2299c7ea1f380b02c5154e6f817119f912626b88/bin/anduril-remote) I get this error message "Invalid number of arguments (6)".

[Foivos Gypas]

Hi guys

Just to let you know, the problem was fixed.
You just need to use "--exec-mode remote" as lauri mentioned and in my case I had to make small changes in the hosts.conf file.
Actually I had to remove some quotes from the paths.

Foivos