Problem launching spark ec2 (While cloning spark-ec2.git)

[sku...@lucirix.com]

All, I am unable to resolve this error msg. It's caused in spark-ec2.py by the following line.

I am running this in windows 7 and the error occurs in a git bash window or a ms dos window

ssh(master, opts, "rm -rf spark-ec2 && git clone http://github.com/mesos/spark-ec2.git")

(The original was https, I changed it to http, but I am still getting the same error)

can anyone throw some light on this? I googled around for the git error msg and found that

git-upload-pack was indeed in the path

--

Cloning into 'spark-ec2.git''...

fatal: http://github.com/mesos/spark-ec2.git'/info/refs?service=git-upload-pack

not found: did you run git update-server-info on the server?

Traceback (most recent call last):

  File "spark_ec2.py", line 713, in <module>

    main()

  File "spark_ec2.py", line 587, in main

    setup_cluster(conn, master_nodes, slave_nodes, zoo_nodes, opts, True)

  File "spark_ec2.py", line 389, in setup_cluster

    ssh(master, opts, "rm -rf spark-ec2 && git clone http://github.com/mesos/spa

rk-ec2.git")

  File "spark_ec2.py", line 547, in ssh

    (opts.identity_file, opts.user, host, command), shell=True)

  File "C:\Python27\lib\subprocess.py", line 542, in check_call

    raise CalledProcessError(retcode, cmd)

subprocess.CalledProcessError: Command 'ssh -t -o StrictHostKeyChecking=no -i ku

mar-aws.pem ro...@ec2-54-242-143-210.compute-1.amazonaws.com 'rm -rf spark-ec2 &&

 git clone http://github.com/mesos/spark-ec2.git'' returned non-zero exit status

 128

[Shivaram Venkataraman]

Hmm - from the error message it looks like there is an extra quote in
the repo being accessed. ( http://github.com/mesos/spark-ec2.git' ). I
am wondering if this is due to some differences in the windows Python
escaping rules.

You could also try to ssh into the master node and run the git clone
command directly to see if there is a problem with git installed on
the machine.

Shivaram

> --
> You received this message because you are subscribed to the Google Groups
> "Spark Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to spark-users...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[sku...@lucirix.com]

It seems to be a problem with the ssh that's part of the git bash install.

Long story short: don't try to invoke the spark_ec2.py script from msdog.

The path on the remote server seems to break on non-interactive shells.

I finally installed a Ubuntu image on Virtual box (this approach is a bit heavy handed) and it worked flawlessly from there.