RSA_verify failed (PuTTy maybe?)

[Pete Flugstad]

Hello,

I've been getting some strange behavior when I attempt to connect to
OpenSSH 3.1p1 (on a Mandrake Linux box, glibc-2.2.2). My client is
PuTTY 0.52, running on Win2k. I've got Pagent running with ssh1
and ssh-rsa keys installed.

The first time I open up a PuTTY window and connect to the server,
the RSA (SSHv2) authentication fails, and it falls back to password.
If I immediately try again however, RSA (SSHv2) works. This seems
to happen the first time I use SSH in the morning, and everything
continues to work fine the rest of the day. So far, I haven't gotten
an Event Log from putty yet when the failure occurs. I will be trying
that tomorrow.

On the server side, in /var/log/messages, I see the following line
from the failure:

Mar 18 11:00:11 taz sshd[29707]: error: ssh_rsa_verify:
RSA_verify failed: error:04077077:lib(4):func(119):reason(119)

This seems to be releated to SSL_R_BAD_RSA_ENCRYPT in the OpenSSL
sources, but I don't see how, as the mapping from func/reason to a
particular place in the code is not clear at all.

I'm posting this to both comp.security.ssh and the Putty mailing
list, since I've upgraded BOTH recently. The version of Putty I
moved to now has SSH2 support, so I switched to that (rather than
SSH1). At this point in time, I tend to suspect Win2k/Putty, but
I can't rule out OpenSSH issues.

So, has anyone seen this before? Is it a known problem. Can
anyone decrypt the RSA_verify error code?

Thanks,
Pete Flugstad

PS: You have to edit my email to reply directly to me.

[Simon Tatham]

Pete Flugstad <pete_f...@nospam.icon-labs.com> wrote:
> Mar 18 11:00:11 taz sshd[29707]: error: ssh_rsa_verify:
> RSA_verify failed: error:04077077:lib(4):func(119):reason(119)

Sorry I missed this when it was posted.

I occasionally see this error myself, and I'm reasonably sure it's a
latent minor problem in the PuTTY bignum code. In order to debug it
I probably need to prepare a trick PuTTY with a controllable random
number generator, so that once I see it happen I can fix the random
seed and repeat it as many times as I like. I haven't got round to
doing that yet though. Anyone who feels like helping is welcome to,
as usual :-)
--
Simon Tatham "_shin_, n. An ingenious device for
<ana...@pobox.com> finding tables and chairs in the dark."