trying to run cap deploy:update and getting "no such file or directory" error on server or github

[Dean]

Hi:

I'm completely new to using capistrano and I hope to soon deploy my
first rails application to production. I'm having trouble getting
cap:deploy going. Here's my scenario:

development machine: windows box running XP Professional. Stuck behind
our corporate firewall. HTTP_PROXY is set correctly, so that I can
push/pull to my private repo on github.

rails -v: 2.3.5
git --version: 1.7.2.3.msysgit.0
cap --version: v2.5.19

remote (deploy) server: linux box running Ubunto 10.04 Server (also
behind our corporate firewall)
rails version on server: 2.3.5
git version on server: 1.7.1
capistrano version on server: v2.5.19

deploy.rb on local (development) machine:

http://pastie.org/1647866

I've successfully run cap deploy:setup.
When I run cap deploy:check, I get the success message: "You appear to
have all necessary dependencies installed."

But when I try to run cap deploy:update I get the following error
output:

http://pastie.org/1647904

My diagnosis is that when capistrano gets to my server, and tries to
send an ls-remote command to my repo on github, it can't connect due
to an authorization or proxy firewall problem.

When I login to my remote server as the deploy user, and attempt to
issue the same command issued by capistrano at the command line:

$ /usr/bin/git ls-remote g...@github.com:git_username/app_name.git
master

I get the error message:

ssh: connect to host github.com port 22: Connection timed out
fatal: the remote end hung up unexpectedly

When I try to ssh -v github.com, I get basicaly the same error:
connection timed out.

When I try to clone my github repository locally on the remote server,
the process seems to proceed, but terminates with a 401 error:

$ git clone http://github.com/git_username/app_name.git
Initalized empty Git repository in home/username/rails_projects/
app_name/.git/
error: The requested URL returned error: 401 while accessing
http://github.com/git_username/app_name.git/info/refs
fatal: HTTP request failed

My copy of git on the remote server is installed at /usr/bin/git.

My .gitconfig file on the remote server contains this code:

[http]
proxy = http://proxy_username:proxy_password@proxy_server.com:port

Any suggestions on what I should try next?

Thanks,

Dean Richardson
Molex.com

[Lee Hambley]

Line #3 of your deploy.rb instructs Git to use SSH, which is apparently blocked by your firewall, so the HTTP proxy you have configured in your .gitconfig isnt' taking effect. See http://www.kernel.org/pub/software/scm/git/docs/git-clone.html#_git_urls_a_id_urls_a

- Lee

[Christian Eager]

Try:

:repository,  "https://github.com/username/my_app_name.git"

in deploy.rb, and in your remote .gitconfig:

    [https]

            proxy = http://proxy_username:proxy_password@proxy_server.com:port

git@github... isn't using the same protocol as ssh://, but the underlying issue is the same. To get around firewalls using ssh, I like corkscrew [1] (requires Cygwin on Windows).

The reason why

$ git clone http://github.com/git_username/app_name.git
Initalized empty Git repository in home/username/rails_projects/
app_name/.git/
error: The requested URL returned error: 401 while accessing
http://github.com/git_username/app_name.git/info/refs
fatal: HTTP request failed

failed is probably because your repository is set to private, and git tried to authenticate using your local username (deploy?). If you try

   git clone http://git_us...@github.com/git_username/app_name.git (or even better, https://...)

that should work remotely, assuming you add https.proxy to your remote .gitconfig.

Best,

Chris

[1] http://www.agroman.net/corkscrew/

--
* You received this message because you are subscribed to the Google Groups "Capistrano" group.
* To post to this group, send email to capis...@googlegroups.com
* To unsubscribe from this group, send email to capistrano+...@googlegroups.com For more options, visit this group at http://groups.google.com/group/capistrano?hl=en

[Lee Hambley]

git@github... isn't using the same protocol as ssh://

Yes, it is, but your advice to use the http:// scheme is right, I had hoped that was implicit in my answer, perhaps leading him to check the docs!

- Lee

[Dean]

Hi Christian:

I've made some progress, thanks to help from you and Lee.

I've successfully managed to git clone my github repository to my
linux box. I've also got corkscrew to work so I can perform a local
git ls-remote successfully from the remote server .Along the way, I
created a deploy key and attached it to my private repository. And
I've adjusted my deploy.rb file to reflect what I've learned.

But alas, cap deploy:update still generates the same "no such file or
directory" error as before.

Here's the new deploy.rb:

http://pastie.org/1661119

.gitconfig now has:

[https]
proxy = https://proxy_username:pass...@proxy.ip.com:80

... as you suggested.

But I found that I was only able to corkscrew ssh and ls-remote
locally using http://git_us...@github.com/git_username/my_app.git
rather than https.

My ~/.ssh/config file now contains:

Host github.com
ProxyCommand /usr/bin/corkscrew my.proxy.com 80 %h %p ~/.ssh/
proxyauth

User git
Port 443
Hostname ssh.github.com
IdentityFile "location of my private key"
TCPKeepAlive yes
IdentitiesOnly yes

Any further suggestions for me?

Thanks again,

Dean Richardson
Molex Fiber Optics

On Mar 8, 2:13 pm, Christian Eager <cea...@gmail.com> wrote:
> Try:
>
>  :repository,  "https://github.com/username/my_app_name.git"
>
> in deploy.rb, and in your remote .gitconfig:
>
>     [https]

>             proxy =http://proxy_username:proxy_password@proxy_server.com:

> port
>
> git@github... isn't using the same protocol as ssh://, but the underlying
> issue is the same. To get around firewalls using ssh, I like corkscrew [1]
> (requires Cygwin on Windows).
>
> The reason why
>

>  $ git clonehttp://github.com/git_username/app_name.git

>
> > Initalized empty Git repository in home/username/rails_projects/
> > app_name/.git/
> > error: The requested URL returned error: 401 while accessing
> >http://github.com/git_username/app_name.git/info/refs
> > fatal: HTTP request failed
>
> failed is probably because your repository is set to private, and git tried
> to authenticate using your local username (deploy?). If you try
>

>    git clone http://git_usern...@github.com/git_username/app_name.git<http://github.com/git_username/app_name.git>

> (or
> even better, https://...)
>
> that should work remotely, assuming you add https.proxy to your remote
> .gitconfig.
>
> Best,
> Chris
>
> [1]http://www.agroman.net/corkscrew/
>

> > $ git clonehttp://github.com/git_username/app_name.git

> > Initalized empty Git repository in home/username/rails_projects/
> > app_name/.git/
> > error: The requested URL returned error: 401 while accessing
> >http://github.com/git_username/app_name.git/info/refs
> > fatal: HTTP request failed
>
> > My copy of git on the remote server is installed at /usr/bin/git.
>
> > My .gitconfig file on the remote server contains this code:
>
> > [http]

> >         proxy =http://proxy_username:proxy_password@proxy_server.com:port

[Lee Hambley]

Dean,

Can we get a full trace output - what happens end-to-end when you try to deploy, please try not to nerf it too badly if you don't want addresses/etc shared with the public)

- Lee

[Dean]

Lee:

Sure! Here you go, nothing hidden:

http://pastie.org/1661238

--Dean

P.S. "nerf"? As in, obscure crucial details as a side-effect of trying
to anonymize directory/account info? :)

[Lee Hambley]

P.S. "nerf"? As in, obscure crucial details as a side-effect of trying to anonymize directory/account info? :)

Absolutely… it looks Dean like your git installation is broken, git commands come in two parts, they historyicaly were called things like `git-ls-remote`, and they had some changes to make it `git ls-remote`, as a side effect the `git` command often has it's own $PATH in which to look for it's command counterparts (`git-ls-remote` in this case).

Check how it's installed, and try sshing into the server directly and running a few Git commands, see if you can't work out if it's working or not.

- Lee

[Dean]

Lee:

OK, I uninstalled git on my Ubuntu box, then reinstalled it (1.7.1).
I'm able to successfully issue the command:
/usr/bin/git ls-remote http://hika...@github.com/hikari17/molex_app.git
master.

(The result is a hash followed by refs/heads/master, so git seems to
work there.)

I can also ssh into my Ubuntu box from the windows machine that I want
to deploy from:

ssh de...@ubuntu.server.ip

gets me to the command line on my deploy server, and

/usr/bin/git ls-remote http://hika...@github.com/hikari17/molex_app.git
master

is successful too, producing the same refs/heads/master result as for
the command issued at the command line on the server itself.

So I can ssh into the server directly. But capistrano still fails to.
As a review, here's the key line of the error message I get:

*** [deploy:update_code] rolling back
* executing "rm -rf /home/dean/rails_projects/releases/
20110314162408; true"
servers: ["10.24.5.50"]
[10.24.5.50] executing command
command finished
c:/RubyonRails/Ruby187/lib/ruby/gems/1.8/gems/capistrano-2.5.19/lib/
capistrano/r
ecipes/deploy.rb:98:in ``': No such file or directory - /usr/bin/
git ls-remote h
ttp://hika...@github.com/hikari17/molex_app.git master
(Errno::ENOENT)

Here's the output when I ssh directly into the deploy server:

$ ssh de...@10.24.5.50
de...@10.24.5.50's password:
Linux dgwjasonfried 2.6.35-22-server #33-Ubuntu SMP Sun Sep 19
20:48:58 UTC 2010
x86_64 GNU/Linux
Ubuntu 10.10

Welcome to the Ubuntu Server!
* Documentation: http://www.ubuntu.com/server/doc

*** System restart required ***
Last login: Mon Mar 14 13:53:40 2011 from dgwhelpdesk02.molex.com

And here's the output when I issue the ls-remote command on the deploy
server after I've successfully ssh'd into it:

dean@dgwjasonfried:~$ /usr/bin/git ls-remote http://hika...@github.com/hikari1
7/molex_app.git master
Password:
48b813a4a42eebf4e5f5aa384c4a2ded19bfb1d8 refs/heads/master

Thanks for your help!

--Dean

[Lee Hambley]

Dean, quick debuging task, try this:

def :remote_env do

  puts capture("env")

  puts capture("which git")

end

Put that in your capfile, and run with `cap remote_env` - examine the output and see if you get any clues!

- Lee

[Dean]

Lee:

OK, put your suggested code in my Capfile, which now looks like this:

load 'deploy' if respond_to?(:namespace)
Dir['vendor/plugins/*/recipes/*.rb'].each { |plugin|
load(plugin) }

load 'config/deploy'

def :remote_env do
puts capture("env")
puts capture("which git")
end

I tried running this from the command line using $ cap remote_env
and got various syntax error messages:

Capfile:9:in 'load': compile error (SyntaxError)
Capfile:6: syntax error, unexpected kDO, expecting '\n' or ';'
Capfile:9 syntax error, unexpected kEND, expecting $end

I tried taking out the : before remove_env, got similar error
messages. I tried adding an extra end (don't the 'do' and the 'def'
both need 'end' statements to close them off?) and experienced similar
issues.

Tried to eliminate typing errors as the reason... did I parse your
code incorrectly?

Thanks...

--Dean

[Lee Hambley]

Sorry dean, should have been `task` not `def` - been hacking away from Cap for too long :)

- Lee

[Dean]

Success!

Both commands executed, with the expected results:

remote_env displayed my remote server username, home directory, proxy
settings, etc.. It also responded to "which git" with /usr/bin/git.
Ssh on my XP box uses port 3468 to tunnel to my ubuntu server, where
it uses port 22.

Nothing looks particularly surprising or odd to me, though I'm very
much a newbie. The full content is here:

http://pastie.org/1671765

--Dean

[Lee Hambley]

Dean, any friends with *nix knowledge, this is starting to smell like something that 5 minutes pairing with someone could fix; if they know enough to fill in gaps in your knowledge (and you can bring them up to speed on what we've been doing here) - it's almost certainly something & nothing… 

- Lee

[Christian Eager]

I might be reading the stack trace incorrectly, but isn't the error happening locally?

Excerpt from around line 14 of http://pastie.org/1661238:

c:/RubyonRails/Ruby187/lib/ruby/gems/1.8/gems/capistrano-2.5.19/lib/capistrano/r

ecipes/deploy.rb:98:in ``': No such file or directory - /usr/bin/git ls-remote h

ttp://hika...@github.com/hikari17/molex_app.git master (Errno::ENOENT)

       from c:/RubyonRails/Ruby187/lib/ruby/gems/1.8/gems/capistrano-2.5.19/lib

/capistrano/recipes/deploy.rb:98:in `run_locally'

Dean, what's the output of `which git' on your local?

[Lee Hambley]

Christian, nice catch - Dean… you *do* have Git installed locally, right, on your workstation? (and it's in the Path?)

- Lee

[Dean]

Christian/Lee:

Git is indeed installed locally:

drichardson@DGWHELPDESK02 /c/rails_projects/molex_app (master)
$ which git
/bin/git

Issuing the ls-remote command from the local machine with /usr/bin/git
also works:

http://pastie.org/1675883

Still, I think you're on to something.

BTW, C:\Program Files\Git\cmd\; is in my Windows PATH environment
variable.

--Dean

On Mar 15, 2:15 pm, Christian Eager <cea...@gmail.com> wrote:
> I might be reading the stack trace incorrectly, but isn't the error
> happening locally?
>
> Excerpt from around line 14 ofhttp://pastie.org/1661238:
>

> c:/RubyonRails/Ruby187/lib/ruby/gems/1.8/gems/capistrano-2.5.19/lib/capistr­ano/r

>
> ecipes/deploy.rb:98:in ``': No such file or directory - /usr/bin/git
>
> > ls-remote h
>

> ttp://hikar...@github.com/hikari17/molex_app.git master (Errno::ENOENT)

>
>        from
>
> > c:/RubyonRails/Ruby187/lib/ruby/gems/1.8/gems/capistrano-2.5.19/lib
>
> /capistrano/recipes/deploy.rb:98:in `run_locally'
>
> Dean, what's the output of `which git' on your local?
>
>
>
>
>
> On Tue, Mar 15, 2011 at 1:01 PM, Lee Hambley <lee.hamb...@gmail.com> wrote:
> > Dean, any friends with *nix knowledge, this is starting to smell like
> > something that 5 minutes pairing with someone could fix; if they know
> enough
> > to fill in gaps in your knowledge (and you can bring them up to speed on
> > what we've been doing here) - it's almost certainly something & nothing…
> > - Lee
>
> > --
> > * You received this message because you are subscribed to the Google
> Groups
> > "Capistrano" group.
> > * To post to this group, send email to capis...@googlegroups.com
> > * To unsubscribe from this group, send email to
> > capistrano+...@googlegroups.com For more options, visit this group

> > athttp://groups.google.com/group/capistrano?hl=en- Hide quoted text -
>
> - Show quoted text -

[Lee Hambley]

Hrm… I'm outta ideas Dean, sorry man - try #capistrano on freenode… they might be able to help you in realtime (I haven't touched a windows machine in a few years… not even sure i'd know what one looked like anymore)

- Lee

[Christian Eager]

Dean,

I don't have a ton of Windows experience either, but I do happen to have a VM running with MinGW, git (at /bin/git), and ruby, so I think I match your environment pretty well. Even though `/usr/bin/git ls-remote' works from the shell, it doesn't work for me if I run it from irb, in a similar manner to capistrano:

$ irb

irb(main):001:0> `/usr/bin/git ls-remote`

Errno::ENOENT: No such file or directory - /usr/bin/git ls-remote

        from (irb):1:in ``'

        from (irb):1

irb(main):002:0> `git ls-remote`

From ssh://...

Try setting :scm_command to "git" instead of "/usr/bin/git" in your deploy.rb.

Chris

On Tue, Mar 15, 2011 at 4:38 PM, Lee Hambley <lee.h...@gmail.com> wrote:

Hrm… I'm outta ideas Dean, sorry man - try #capistrano on freenode… they might be able to help you in realtime (I haven't touched a windows machine in a few years… not even sure i'd know what one looked like anymore)

- Lee

--

* You received this message because you are subscribed to the Google Groups "Capistrano" group.
* To post to this group, send email to capis...@googlegroups.com

[Dean]

Yes... that did it... marvelous!

Thanks you Chris (and Lee) for taking the time to dive deep on this
issue and tease out the solution.

--Dean