Capistrano & Hudson: getting Authentication Failed even though ssh keys set up correctly

[Rory Gibson]

Hi guys,
We're using Capistrano for a few things here at Technophobia, and I'm
aware that I'm pushing the boundaries with this usage, but please bear
with me, it's got us all stumped here.

Basically, I'm attempting to use Cap to deploy a PHP application from
with the Hudson CI server.

I have a cap script which works perfectly from the command line, using
the "copy" method to tar up, copy via sftp and unpack the PHP app on
the remote server.
SSH Keys are set up correctly so that I never have to enter a
password.

However, when I set up a Hudson job to run the cap script (by cd'ing
to the directory containing the Capfile and issuing cap deploy), I get
a "Net::SSH::AuthenticationFailed" error.

This may or may not be caused by the 'stty: invalid argument' messages
above; however there are no stty commands in any of /etc/profile, /etc/
bash.bashrc, ~/.bashrc or ~/.profile so I can't work out what that's
about.

Also, I'm not sure if the apparent 'Password:' prompt should be there
even though I'm using SSH keys; it certainly doesn't appear if I run
the script from the command line.

Logs are below (slightly anonymised), huge thanks if anyone can help
me out.

Cheers,
Rory Gibson

------- Hudson console output ------------
Started by user anonymous
Updating http://subversion/repos/TSB_Holly-Sugar/trunk
At revision 131
no change for http://subversion/repos/TSB_Holly-Sugar/trunk since the
previous build
[workspace] $ /bin/sh -xe /usr/local/tomcat/temp/
hudson2895655324317047058.sh
+ cd '/home/tomcat/.hudson/jobs/Hudson SugarCRM Deployment/workspace/
Sugar/deploy'
+ /var/lib/gems/1.8/bin/cap sandbox deploy
* executing `sandbox'
triggering start callbacks for `deploy'
* executing `multistage:ensure'
* executing `deploy'
triggering before callbacks for `deploy'
* executing `local:log_config'
=========================================================
Loading Sandbox environment
CRM host : rory-sandbox
Remote copy tmp : /home/deploy/tmp
Deploy to : /home/deploy/sites/crm.rory-sandbox/non-secure/
htdocs
DB name : sugarcrm
DB user : tsb-dev
DB admin name : sugarcrm
DB admin user : tsb-dev
=========================================================
* executing `deploy:update'
** transaction: start
* executing `deploy:update_code'
executing locally: "svn info
http://subversion.technophobia.int/repos/TSB_Holly-Sugar/trunk/SugarCE-5.2.0e/
--username hudson --password XXPASSWORD_ELIDEDXX --no-auth-cache -
rHEAD"
* refreshing local cache to revision 124 at /tmp/sugar_cap_cache
executing locally: svn update -q --username hudson --password
GamCaFrouft4 --no-auth-cache -r124 /tmp/sugar_cap_cache
* copying cache to deployment staging area /tmp/sugar_copy/
20090806100323
compressing /tmp/sugar_copy/20090806100323 to /tmp/sugar_copy/
20090806100323.tar.gz
executing locally: tar czf 20090806100323.tar.gz 20090806100323
servers: ["rory-sandbox"]
Password: stty: standard input: Invalid argument
stty: standard input: Invalid argument
stty: standard input: Invalid argument

*** [deploy:update_code] rolling back
* executing "rm -rf /home/deploy/sites/crm.rory-sandbox/non-secure/
htdocs/releases/20090806100323; true"
servers: ["rory-sandbox"]
** [deploy:update_code] exception while rolling back:
Capistrano::ConnectionError, connection failed for: rory-sandbox
(Net::SSH::AuthenticationFailed: deploy)
connection failed for: rory-sandbox (Net::SSH::AuthenticationFailed:
deploy)
Finished: FAILURE

[Lee Hambley]

Rory,

Always good to see people pushing the boundaries, for deploying PHP you may want to take a look at my railsless deploy gem; it's the default deploy recipe, with all the rails-ish bits like migrations, mongrel, restart, etc removed - you can find it on github [1].

With regard to your problem deploying from Hudson, I'm not familiar with the technology; but I suspect the problem has to do with whos keys it is using.

Depending how Hudson is configured, I would expect cron tasks linked with it to run as your Apache (or other web server, of course) user (usually www or nobody) - or root.

With this in mind, you might be able to try something like this:

[rory@workstation]$ ssh www@hudson 'cd /app/directory && cap deploy'

That is as close to the cron environment for that user as you can get over SSH, by doing that - you'll be using a non-login shell, and not relying on settings in any files like /etc/profile or similar, and also subject to any curious permissions your machine may have with regard to non-login processes.

You may find that your www,root and nobody users, whichever you need to run the job as... have a non-login shell, check your O/S docs for how to change that, and make sure there's a good way to set it back when you are done debugging!

Please come back if you have any problems, frustratingly this probably isn't even a Capistrano problem, but don't let that stop you asking for help, we're getting used to supporting SSH configuration issues here :-D

Thanks for pushing the envelope!

-- Lee Hambley

Twitter: @leehambley
Blog: http://lee.hambley.name/
Working with Rails: http://is.gd/1s5W1

2009/8/6 Rory Gibson <ro...@deadcrow.net>

[Rory Gibson]

Lee,
Thanks very much for getting back to me.
The scripts I've come up with cover a lot of the same ground as your
Railsless-deploy setup, it turns out, but they're not as nice :-)
I'm using the multi-stage extension and a few other little tricks that
are specific to my environment.

Looks like my problem is something specific to how Hudson sets up the
environment for its forked builds; I'd already tried logging in as
you suggested via SSH and it worked fine (tested it again to be
certain anyway!)

I've managed to solve/hack/workaround (delete as appropriate) the
problem anyway; I've set Hudson up to run a Rake task as a build step.
The rakefile exec's a shell script which calls Cap; this huge amount
of indirection seems to allow the correct environment to somehow
exist.
I'm not happy with it, but it works, and for now that'll do.

Cheers for your help, it's very much appreciated.

Rory Gibson



2009/8/6 Lee Hambley <lee.h...@gmail.com>: