[Rocks-Discuss] ssh root@compute-node requires password

425 views
Skip to first unread message

Tent Pig

unread,
Jan 15, 2010, 10:35:53 AM1/15/10
to npaci-rocks...@sdsc.edu
I just finished a fresh install of Rocks 5.2 on a 500-node cluster, pxebooting all compute-nodes by hand, insert-ethers running, etc. to get a functional cluster.

All of my nodes are up and running.

My problem is if I attempt, as root on the head-end, to ssh root@compute-whatever, I'm being asked for a password... on every single compute node.


This is annoyingly aggravating. I have several post-install scripts to run on each of the compute nodes, and I really don't feel like typing the root password several thousand times. It will take me months to complete this rollout, and I was supposed to be finished by the end of business today.

I googled this problem and it seems pretty common, but I can't get a clear resolution. .ssh folder permissions, regening the ssh key, yada yada yada... all of which I've tried and nothing fixes it.

I'm absolutely dead in the water here at the moment.

Is there a simple solution to this problem? Reinstalling the compute nodes (one of the suggested solutions) doesn't seem a reasonable answer, since I just installed the compute nodes, haven't even touched them after they rebooted, and I have this issue. (Nor do I really feel like sitting there to reboot/PXEboot manually all over again.)



jean-francois prieur

unread,
Jan 15, 2010, 11:23:30 AM1/15/10
to Discussion of Rocks Clusters
Could you try the command 'rocks sync users', on the head node as root and
see if it fixes this issue?

Regards,
JF Prieur

2010/1/15 Tent Pig <ten...@yahoo.com>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.sdsc.edu/pipermail/npaci-rocks-discussion/attachments/20100115/7f3761d8/attachment.html

Tim Carlson

unread,
Jan 15, 2010, 11:39:44 AM1/15/10
to Discussion of Rocks Clusters
On Fri, 15 Jan 2010, Tent Pig wrote:

Read this thread. It should help. Most likely the permissions on /root
were to strict when you installed your nodes.

https://lists.sdsc.edu/pipermail/npaci-rocks-discussion/2009-November/043680.html

You have a 500 node cluster installed with no IPMI capability? That could
be painful when you are in production mode.

You are not dead in the water. You could use "expect" to loop through all
your nodes and type in the password just once.

Tim
-------------------------------------------
Tim Carlson, PhD
Senior Research Scientist
Environmental Molecular Sciences Laboratory

Greg Bruno

unread,
Jan 15, 2010, 11:54:01 AM1/15/10
to Discussion of Rocks Clusters


here's a little background on how rocks distributes the root user's
public key and on ssh login in general.

in order to login to a host without having to type a password with
ssh, the public key for the user must be in the file
$HOME/.ssh/authorized_keys on the remote host and the user's private
key must be loaded into the user's environment. you can see what
private keys are currently loaded by executing:

# ssh-add -l

in rocks, when a user logs in and if the file $HOME/.ssh/id_rsa
doesn't exist, then the user is prompted to create their ssh keys.
this is also true for the root user, which is why you create ssh keys
for the root user when you login to the frontend the first time after
the frontend is installed. also, in rocks, after your ssh keys are
created, the public key is copied into $HOME/.ssh/authorized_keys.

on the frontend, the root user's home directory is not NFS mounted on
all the compute nodes, so in rocks, we distribute root's
authorized_keys file inside the kickstart file when a compute node
installs.

so, if you change root's ssh keys on the frontend, then the normal way
authorized_keys is updated on the compute nodes is through
reinstallation.

but, the good news is, i believe there is a workaround.

on the frontend, let's generate new ssh keys for root:

# rm -rf /root/.ssh/id_rsa*

then logout and log back in. this will prompt you to generate new ssh
keys for root.

now, let's distribute root's authorized_keys file with 411:

# cd /var/411

edit 'Files.mk' and change the line:

# FILES += /my/file

to:

FILES += /root/.ssh/authorized_keys

now tell 411 to push all the files under its control out to the compute nodes:

# rocks sync users

now try to login to a compute node.

- gb


- gb


- gb

Tent Pig

unread,
Jan 15, 2010, 2:53:24 PM1/15/10
to Discussion of Rocks Clusters

> but, the good news is, i believe there is a workaround.

(snip)

Worked like a charm! Thanks!



Reply all
Reply to author
Forward
0 new messages