socket.gaierror [Errno -2] Name or service not known when remotely launching from (and to) AWS EC2 nodes

[Sjoerd van Steenkiste]

Hi,

I am experimenting with SCOOP and Amazon EC2. I have launched two nodes (named ec1 & ec2) accessible from anywhere (i.e. open firewall) and configured password-less ssh between them. I am running the "Hello World" example and have verified that each node is able to run the example individually. 

I am currently trying to launch the example on node ec2 from node ec1. The filesystem between the nodes is identical and the file hosts.txt contains a single line "ec2 1". 

When running "python -m scoop --hostfile hosts.txt example.py" I obtain the following error

[2016-07-02 12:55:40,094] launcher  INFO    SCOOP 0.7 1.1 on linux2 using Python 2.7.10 (default, Dec  8 2015, 18:25:23) [GCC 4.8.3 20140911 (Red Hat 4.8.3-9)], API: 1013

[2016-07-02 12:55:40,094] launcher  INFO    Deploying 1 worker(s) over 1 host(s).

[2016-07-02 12:55:40,094] launcher  INFO    Worker distribution: 

[2016-07-02 12:55:40,094] launcher  INFO       ec2: 0 + origin

[2016-07-02 12:55:40,444] __main__  INFO    Worker(s) launched using /bin/bash

Traceback (most recent call last):

  File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main

    "__main__", fname, loader, pkg_name)

  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code

    exec code in run_globals

  File "/usr/local/lib/python2.7/site-packages/scoop/bootstrap/__main__.py", line 302, in <module>

    b.main()

  File "/usr/local/lib/python2.7/site-packages/scoop/bootstrap/__main__.py", line 92, in main

    self.run()

  File "/usr/local/lib/python2.7/site-packages/scoop/bootstrap/__main__.py", line 290, in run

    futures_startup()

  File "/usr/local/lib/python2.7/site-packages/scoop/bootstrap/__main__.py", line 271, in futures_startup

    run_name="__main__"

  File "/usr/local/lib/python2.7/site-packages/scoop/futures.py", line 64, in _startup

    result = _controller.switch(rootFuture, *args, **kargs)

  File "/usr/local/lib/python2.7/site-packages/scoop/_control.py", line 176, in runController

    execQueue = FutureQueue()

  File "/usr/local/lib/python2.7/site-packages/scoop/_types.py", line 256, in __init__

    self.socket = Communicator()

  File "/usr/local/lib/python2.7/site-packages/scoop/_comm/scoopzmq.py", line 62, in __init__

    s.connect((scoop.BROKER.externalHostname, scoop.BROKER.task_port))

  File "/usr/lib64/python2.7/socket.py", line 228, in meth

    return getattr(self._sock,name)(*args)

socket.gaierror: [Errno -2] Name or service not known

[2016-07-02 12:55:40,497] launcher  INFO    Root process is done.

Killed by signal 15.

[2016-07-02 12:55:40,766] launcher  INFO    Finished cleaning spawned subprocesses.

 Interestingly enough I have observed this error before when using scoop cross platform (when launching on an ubuntu from a mac), where as back then launching from ubuntu to ubuntu with identical file systems worked fine. I have verified that when supplying any of the (python) path flags the same error is obtained. 

Would you have any suggestions in how to resolve this? Let me know if you wish me to re-run with additional verbosity turned on.

Thanks,

Sjoerd

[Yannick Hold-Geoffroy]

Hello,

I'm not quite sure about why you're seeing this particular issue. Basically, SCOOP tries to create a socket and the OS reports back an error because it cannot find the target.

There are a couple of things that you could try:
1) Ensure that the current node is the first one of your hosts.txt (It will help SCOOP by launching the broker on the local node)
2) Ensure that the names you gave to your nodes in your hosts.txt file are routable (meaning you can do 'ssh <node name>' and it works). If not, add them to your /etc/hosts file with the format: "<ip>  <node name>" .

I am curious, what do you mean by "supplying any of the (python) path flags"?

Have a nice day,
Yannick