Getting started

[jlezard]

Hello,

Spark looks amazing.

After getting the latest trunk for mesos. When run the mesos-ec2
script I get the following error:

ssh: connect to host ec2-xxx-21-xx-xx.compute-1.amazonaws.com port 22:
Connection refused
rsync: connection unexpectedly closed (0 bytes received so far)
[sender]
rsync error: error in rsync protocol data stream (code 12) at
io.c(601) [sender=3.0.8]
Traceback (most recent call last):
File "./mesos_ec2.py", line 541, in <module>
main()
File "./mesos_ec2.py", line 450, in main
setup_cluster(conn, master_nodes, slave_nodes, zoo_nodes, opts,
True)
File "./mesos_ec2.py", line 304, in setup_cluster
deploy_files(conn, "deploy." + opts.os, opts, master_nodes,
slave_nodes, zoo_nodes)
File "./mesos_ec2.py", line 415, in deploy_files
subprocess.check_call(command, shell=True)
File "/usr/lib/python2.7/subprocess.py", line 511, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'rsync -rv -e 'ssh -o
StrictHostKeyChecking=no -i ../../AmazonKeys/sparkUS.pem' '/tmp/
tmpyIF3M8/' 'ro...@ec2-107-21-79-48.compute-1.amazonaws.com:/''
returned non-zero exit status 12

Whats really word is that the deployment worked fine once ..... :(

Help please,

Thanks,

Joe

[Matei Zaharia]

Oh, this can happen sometimes if the EC2 machine takes a while to come up. Just run the launch command again but with a --resume parameter.

Matei

[jlezard]

Thanks for the quick reply.

So I reran the command and I got:

Found 1 master(s), 1 slaves, 0 ZooKeeper nodes
Deploying files to master...

ssh: connect to host ec2-xxx-21-xx-xx.compute-1.amazonaws.com port 22:
Connection refused
rsync: connection unexpectedly closed (0 bytes received so far)
[sender]
rsync error: error in rsync protocol data stream (code 12) at
io.c(601) [sender=3.0.8]
Traceback (most recent call last):

File "./mesos_ec2.py", line 525, in <module>
main()
File "./mesos_ec2.py", line 434, in main

setup_cluster(conn, master_nodes, slave_nodes, zoo_nodes, opts,
True)
File "./mesos_ec2.py", line 304, in setup_cluster
deploy_files(conn, "deploy." + opts.os, opts, master_nodes,
slave_nodes, zoo_nodes)

File "./mesos_ec2.py", line 399, in deploy_files

subprocess.check_call(command, shell=True)
File "/usr/lib/python2.7/subprocess.py", line 511, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'rsync -rv -e 'ssh -o

StrictHostKeyChecking=no -i ../../AmazonKeys/keyName.pem' '/tmp/
tmpopJ36F/' 'ro...@ec2-xxx-21-xx-xx.compute-1.amazonaws.com:/''

returned non-zero exit status 12

Very bizar

Thanks

> > tmpyIF3M8/' 'r...@ec2-107-21-79-48.compute-1.amazonaws.com:/''

[jlezard]

Aha it just worked!! Thanks!!

> tmpopJ36F/' 'r...@ec2-xxx-21-xx-xx.compute-1.amazonaws.com:/''

[walrus...@gmail.com]

Is a --resume parameter preferable to extending the wait time, e.g. -w 240 ? 

Thanks

> tmpyIF3M8/' 'root@ec2-107-21-79-48.compute-1.amazonaws.com:/''

[walrus...@gmail.com]

Please disregard, thanks.