Error while launching SCOOP subprocesses due to No module named scoop.broker

[Devarajulu Deva]

Hi Yannick Hold-Geoffroy,
Thanks for quick response,
As per your suggestion I have re-installed scoop 0.7.1.1 in python installation area, now it is working fine, but we don't want to install scoop module in python installation area.
Actually we are installing modules in separate directory, modules directory path added to the environmental variable 'PYTHONPATH', like as PYTHONPATH = /root/da/python/modules/Scoop/lib/python2.7/site-packages.

When I run  "python -m scoop --host krivi-sys12 EX06_scoop_hostName.py -vv -n 6 pythonpath=$PYTHONPATH" am getting the follow error,

[2016-04-05 18:48:43,780] launcher  INFO    SCOOP 0.7 1.1 on linux2 using Python 2.7.6 (default, Mar 21 2015, 22:37:16) [GCC 4.4.7 20120313 (Red Hat 4.4.7-4)], API: 1013
[2016-04-05 18:48:43,780] launcher  INFO    Deploying 6 worker(s) over 2 host(s).
[2016-04-05 18:48:43,780] launcher  DEBUG   Using hostname/ip: "root-sys12" as external broker reference.
[2016-04-05 18:48:43,780] launcher  DEBUG   The python executable to execute the program with is: /root/da/sw/python/src/python/

2.7.6/bin/python2.7.
[2016-04-05 18:48:43,780] launcher  INFO    Worker distribution:
[2016-04-05 18:48:43,781] launcher  INFO       root-sys12:    2 + origin
[2016-04-05 18:48:43,781] launcher  INFO       EX06_scoop_hostName.py:    2 + origin
[2016-04-05 18:48:43,781] brokerLaunch DEBUG   Launching remote broker: ssh -x -n -oStrictHostKeyChecking=no root-sys12 /root/da/sw/python/src/python/2.7.6/bin/python2.7 -m scoop.broker.__main__ --echoGroup --echoPorts --backend ZMQ
devjel01@root-sys12's password:
ERROR:root:Error while launching SCOOP subprocesses:
ERROR:root:Traceback (most recent call last):
  File "build/bdist.linux-x86_64/egg/scoop/launcher.py", line 479, in main
    rootTaskExitCode = thisScoopApp.run()
  File "build/bdist.linux-x86_64/egg/scoop/launcher.py", line 260, in run
    backend=self.backend,
  File "build/bdist.linux-x86_64/egg/scoop/launch/brokerLaunch.py", line 157, in __init__
    "SSH process stderr:\n{stderr}".format(**locals()))
Exception: Could not successfully launch the remote broker.
Requested remote broker ports, received:

Port number decoding error:
need more than 1 value to unpack
SSH process stderr:
/root/da/sw/python/src/python/2.7.6/bin/python2.7: No module named scoop.broker


[2016-04-05 18:48:50,798] launcher  INFO    Finished cleaning spawned subprocesses.
INFO:launcherLogger:Finished cleaning spawned subprocesses.

Pleas help me if any one knows about this issue.

[Yannick Hold-Geoffroy]

Hello,

There are multiple ways to fix this issue, either:

1. Try using the --pythonpath flag (you are currently missing two minus signs -- before the flag) and without an equal = sign. That should do the trick.
   Or
2. Create a Python virtual environment with the same paths on every system. This will allow you to install any package in it without messing with the global Python installation area (the site-packages), which I recommend.
   Or
3. Use the --prolog flag to execute a shell script before SCOOP launches Python on the remote nodes. This will allow you to set up any environment variable you like.

Wish you a good day,

Yannick