remote kernel trouble

[Ian]

I'm unable to start a remote kernel, and have not found a solution to
my problem the archives. I'm running Mathematica 7.0 on Windows Vista,
trying to start a remote kernel on a Linux server also running 7.0.

I'm relying on WolframSSH.jar to make the connection, so I haven't
fiddled with the advanced settings under kernel configuration
options. (Well, I've tried without success to use putty to make the
connection). When I evaluate the first command in a notebook using the
remote kernel, I get the authentication key question and then enter a
password, and then nothing happens. The notebook claims to be running
but never evaluates, there's no error message, and Mathematica locks
up.

I can see that a MathKernel does start on the remote machine, so the
login at least is successful. And force closing Mathematica locally
kills the remote kernel.

The only clue I can discover is that Mathematica wants to put the RSA
key in the wrong place. My username on both machines is
"carroll" (which I can confirm Mathematica knows with $UserName), but
it creates the known_hosts file in "C:\Users\Ian\.ssh\" which wouldn't
otherwise exist. I also don't see any key created on the remote
machine. Could this cause the problem? What else should I try doing?

Thanks
-Ian
(* apologies if this is a double post, the first vanished *)

[Ian]

I have followed the very simple instructions for setting up a remote
kernel in Mathematica 7.0 on Windows Vista, but my connection is not
working. I believe the source of the problem is the location of the
ssh keys. After telling a notebook to start the remote kernel, I get
to a dialog which says:

"C:\Users\Ian\.ssh\known hosts does not exist Are you sure you want to
create it?"

My username is "carroll" not "Ian", and Mathematica knows this:

In = $UserName
Out = carroll

"carroll" is also my username on the remote machine. How do I force
Mathematica to create and look for the known_hosts file in "C:\Users
\carroll\.ssh\"?

And yes, even if I let the known_hosts file get created in Users\Ian
\.ssh, the connection still does not work.

Thanks
-Ian

[Ian]

Thanks to help from bj...@mit, but I'm still not there. I'm now
getting a msg "The kernel cruncher failed to connect to the front end.
(Error = MLECONNECT). You should try running the kernel connection
outside the front end." I tried, but what sort of behavior is good or
bad?

My configuration on a Windows Vista machine running Mathematica 7.0 is

Arguments to MLOpen:
-LinkMode Listen -LinkProtocol TCPIP -LinkOptions MLDontInteract

Shell command to launch kernel:
plink -i keyfile carroll@cruncher -x math -mathlink -linkmode Connect -
linkprotocol TCPIP -linkname "`linkname`" -linkhost `ipaddress`

When I run exactly this from cmd line in Vista, I get
> bash: linkname: command not found
> bash: ipaddress: command not found

> LinkOpen::linke: --Message text not found -- (MathLink linkname missing or malformed.)

When from the cmd line in Vista I run
plink -i keyfile carroll@cruncher -x math -mathlink -linkmode Connect -
linkprotocol TCPIP -linkname 53...@local.machine

there's no message, and I have to ctrl-C to get back to the cmd
prompt. a MathKernel does start on cruncher. should something else
happen?

thanks for any help!
-Ian

[Zach Bjornson]

I've never used the WolframSSH.jar utility, but I have used plink to do
this. Instructions are below (it sounds like you've done some of these
steps).
-Zach

1. Enable sshd on the slave. At a terminal, type
# sudo apt-get install openssh-server openssh-client

On your host, in this order...
1. Install Plink. Download it from
http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
and save them to e.g. the Program Files directory. You might want PuTTY
as well, but probably won't need it.

2. Open Mathematica and go to Evaluation>Parallel Kernel Configuration.
Click on the Remote Kernels tab. Click "Add Host."

3. Type the hostname or IP address of the slave (both will work) in the
Hostname box.

4. Select "Use custom launch command" and change it to this:
C:\Progra~2\plink.exe user...@192.168.0.1 -pw 123456 -x "math -mathlink
-linkmode Connect `4` -linkname `2` -subkernel -noinit >& /dev/null &"

where C:\Progra~2\plink.exe is the location of plink.exe (it might be
Progra~1 if you're not on a 64-bit machine), username is your username
on the slave, 192.168.0.1 is the hostname or IP address of the slave and
123456 is the password for your login on the slave.

5. Restart Mathematica. (This is not really necessary, you can just
evaluate LaunchKernels[] but sometimes Mathematica bawks at your
already-open kernels.) All of your kernels should launch. Test it with
e.g. ParallelEvaluate[2^2048;//Timing] to see how your kernel speeds
roughly compare.

If you get an error about your remote kernel(s) not launching, try
running the command from step 4 from a command shell. (Open cmd.exe and
type C:\Progra~2\plink.exe user...@192.168.0.1 -pw 123456 "math" with
the replacements described above. You should get In[1]:= in a shell.) If
that doens't work, either PuTTY or your slave isn't configured properly.

[John Fultz]

You say that the MathKernel is starting on the remote machine, so the ssh
machinery worked. Don't worry about where known_hosts got created...there may
be a minor issue here, but it's not causing your principal problem. And neither
WolframSSH nor ssh itself will create a key on another machine. There are, of
course, options for you to do this manually and for these ssh implementations to
use that key...but they don't work by creating anything on the remote system.
There seems to be a little bit of confusion in general about ssh here, but as I
said, let's set all of that aside. You have confirmed (and I'll trust you did
this via reliable means) that MathKernel was launched, and that means the ssh
hurdle was cleared.

So, what are the other possible problems? These are the ones I can think of...

* The command-line might not identify the machine running the FE correctly. If
you can get the command-line which was run on the remote machine, check to see
how it identified your machine, and then ping that location. Did it work?

* You might have multiple network adapters on the FE machine, and the kernel
might not be connecting to the same adapter that the FE is listening on.

* There might be an intervening firewall blocking traffic on arbitrary ports.

The first two can be fixed by fiddling with the command-line which is sent via
ssh. The third, I'm afraid, is a real problem, for which there is no good
solution.

Also, if you talk to Wolfram Tech Support, they can walk you through a process
of manually launching an FE on one side, a kernel on the other, and hooking them
up step by step in a way that will hopefully produce better feedback to what the
real problem is (I could do this, too, but it'd about triple the length of this
email, and I'm really very, very busy right now).

Sincerely,

John Fultz
jfu...@wolfram.com
User Interface Group
Wolfram Research, Inc.

[Zach Bjornson]

Hi Ian,

It sounds like you're doing more than is necessary, but I might be
misunderstanding your e-mail. My replies are interspersed below.

On 11/6/2009 5:15 AM, Ian wrote:
> Thanks to help from bj...@mit, but I'm still not there. I'm now
> getting a msg "The kernel cruncher failed to connect to the front end.
> (Error = MLECONNECT). You should try running the kernel connection
> outside the front end." I tried, but what sort of behavior is good or
> bad?
>
> My configuration on a Windows Vista machine running Mathematica 7.0 is
>
> Arguments to MLOpen:
> -LinkMode Listen -LinkProtocol TCPIP -LinkOptions MLDontInteract
>

Try using the *exact* ML string that I sent in my instructions. The
linkmode needs to be connect; you need to specify a host name, ML needs
to interact, etc. This looks more like a slave string, but I don't know
where you got it.

> Shell command to launch kernel:
> plink -i keyfile carroll@cruncher -x math -mathlink -linkmode Connect -
> linkprotocol TCPIP -linkname "`linkname`" -linkhost `ipaddress`
>
> When I run exactly this from cmd line in Vista, I get
>
>> bash: linkname: command not found
>> bash: ipaddress: command not found
>>

You shouldn't be running anything from the Windows shell except for
diagnostics. Mmeca launches the remote kernel on its own. The syntax you
wrote is specific to Mmeca as well (the `linkname` part, at least)

>
>> LinkOpen::linke: --Message text not found -- (MathLink linkname missing or malformed.)
>>
> When from the cmd line in Vista I run
> plink -i keyfile carroll@cruncher -x math -mathlink -linkmode Connect -
> linkprotocol TCPIP -linkname 53...@local.machine
>
> there's no message, and I have to ctrl-C to get back to the cmd
> prompt. a MathKernel does start on cruncher. should something else
> happen?
>
>

If your connection is successful from a command line (e.g. if you "ssh
yourserver math" you should get a Mathematica command line interface ">"
and be able to e.g. type 1+1 and get 2).

> thanks for any help!
> -Ian
>
>
>

Try following my instructions exactly; I think that will clear up your
problem. You don't need to do anything outside of Mmeca to use the
remote kernel after you set it up.
Cheers,
Zach

> On Nov 5, 1:24 am, Ian<carroll....@gmail.com> wrote:
>

[Mark Perrin]

We use a lot of remote kernels at work, and from home to work. The
configurations we have are Mac to Mac, Mac to Linux, Mac to Windows
and Windows to Windows. Had trouble with them all until we found this:

"Remote Kernel Strategies"
http://library.wolfram.com/infocenter/Conferences/7250/

It is a useful set of programs to easily connect to remote kernels. It
has worked beautifully every time. Give it a go, it might work for you.

Regards,

Mark Perrin