ssh error invalid digest

[Tom Goodman]

Hello OpenSSL,

Can openssl 3.5.71-1 be coerced to connect to an old ssh server?

I am seeing this problem on cygwin, but it looks like an openssl issue,
not a cygwin issue.

After updating my cygwin environment to their current recommended
version of openssl, 3.5.71-1, I am seeing this error:

tgoodman@tester04 ~
$ ssh -o HostKeyAlgorithms=+ssh-rsa notsure
ssh_dispatch_run_fatal: Connection to 192.168.11.3 port 22: error
in libcrypto: invalid digest

The destination host is running Ubuntu 10.04.4 LTS. (Yes, it is
incredibly old, but this is an interior machine and it still works, so
no one wants to touch it.)

tgoodman@notsure:~$ openssl version
OpenSSL 0.9.8k 25 Mar 2009

tgoodman@notsure:~$ ssh -version
OpenSSH_5.3p1 Debian-3ubuntu7.1, OpenSSL 0.9.8k 25 Mar 2009

Several years ago I added the HostKeyAlgorithms option that was needed
to support the old crypto. (Actually added to ~/.ssh/config, but for
debugging I've temporarily deleted config so everything is on the
command line.)

Claude recommended:

tgoodman@tester04 ~
$ ssh -o HostKeyAlgorithms=+ssh-rsa \
-o PubkeyAcceptedKeyTypes=+ssh-rsa \
-o
KexAlgorithms=+diffie-hellman-group14-sha1,diffie-hellman-group1-sha1 \
notsure
ssh_dispatch_run_fatal: Connection to 192.168.11.3 port 22: error
in libcrypto: invalid digest

Since that advice came from Claude, I did not have high hopes. As
expected, that did not fix the issue.

The unsatisfying 'fix' I have implemented is to downgrade the openssl
and libssl3 packages from 3.5.71-1 to the older 3.0.19-1, but that is
not a good, long term solution.

So my question:

Can openssl 3.5.71-1 be coerced to make the connection to this old ssh
server? If so, what options?

--
Tom Goodman
TomGo...@FBE-inc.com
https://fbe-inc.com/
FBE Associates, Inc.

[Viktor Dukhovni]

On Sun, Jun 28, 2026 at 06:18:42PM -0400, Tom Goodman wrote:

> Can openssl 3.5.71-1 be coerced to connect to an old ssh server?

The "3.5.71-1" is a vendor-specific version number. You can ask your
vendor about that version. The latest upstream OpenSSL 3.5 patch
release is openssl-3.5.7.

> $ ssh -o HostKeyAlgorithms=+ssh-rsa notsure
> ssh_dispatch_run_fatal: Connection to 192.168.11.3 port 22: error in libcrypto: invalid digest

You'll need to explain in more detail specifically which digest the SSH
client is attempting to use. With "ssh-rsa" I'd expect RSA with SHA1,
which still works in OpenSSL 3.5.7. Demo in a fresh build tree with
a 2048 bit RSA key:

$ ./util/wrap.pl ./apps/openssl dgst -sha1 \
-sign test/certs/serverkey.pem <(printf foo) | wc -c
256

$ ./util/wrap.pl ./apps/openssl dgst -sha1 \
-sign test/certs/serverkey.pem <(printf foo) |
./util/wrap.pl ./apps/openssl dgst -sha1 -signature /dev/stdin \
-verify test/certs/serverkey.pem <(printf foo)
Verified OK

So SHA1 with RSA produces the expected signature length = modulus
length, and verifies OK.

> libcrypto: invalid digest

You'll need to figure out whether your code is linked with the right
libcrypto, and what digest it is trying and failing to compute or
verify.

> Can openssl 3.5.71-1 be coerced to make the connection to this old ssh
> server? If so, what options?

Well, that's a question for your vendor, perhaps they've disabled SHA1
in this build, or made other changes.

--
Viktor. 🇺🇦 Слава Україні!