SSH with RSA and SHA2 Support in Gerrit?

[Michael Tughan]

Hello all,

Over the weekend, we have had some people upgrade from Git 2.33 to 2.34. On Windows machines, this also installed OpenSSH 8.8p1 over top of OpenSSH 8.7p1 which had been previously installed. After this upgrade, these people are not able to authenticate to Gerrit using RSA keys anymore. Other key types (or at least Ed25519) don't seem to be affected. The error message, visible when enabling debug logs, is "debug1: send_pubkey_test: no mutual signature algorithm"

After some research, I believe it's because OpenSSH 8.8 removed support for using RSA signatures with SHA-1 hashes (see the "Potentially-incompatible changes" section of the OpenSSH 8.8 release notes [1]). I can confirm that adding "PubkeyAcceptedAlgorithms +ssh-rsa" to the SSH configuration for the Gerrit server resolves the issue.

Given the above, this appears to be a shortcoming in the Gerrit server and I'm not seeing a configuration option to enable something like the rsa-sha2-256 or rsa-sha2-512 algorithms. Is this something that we can enable in our existing environment? Does it require a SSHD upgrade and a new Gerrit release?

Gerrit 3.4.1

Ubuntu 18.04.6

[1] https://www.openssh.com/txt/release-8.8

[Han-Wen Nienhuys]

I know David Ostrovsky twiddled with some of the algorithms. (See eg.
https://gerrit.googlesource.com/gerrit/+/a0f77df3aa3536a0b25c8db0fc607d629786ffb0).

When I run ssh -vvv locally against Gerrit from master , I see

debug2: peer server KEXINIT proposal
..
debug2: host key algorithms:
ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa

ie. rsa-sha2 is accepted for host keys. Maybe it needs to be enabled
for user authentication separately?

--
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--

Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

[Oswald Buddenhagen]

On Tue, Nov 23, 2021 at 11:54:50AM +0100, 'Han-Wen Nienhuys' via Repo and Gerrit Discussion wrote:
>ie. rsa-sha2 is accepted for host keys. Maybe it needs to be enabled
>for user authentication separately?
>

yes. https://bugs.chromium.org/p/gerrit/issues/detail?id=13930

[Clark Boylan]

And these are the two related MINA bugs:

https://issues.apache.org/jira/browse/SSHD-1141
https://issues.apache.org/jira/browse/SSHD-1216

MINA 2.7.0 fixes this for MINA ssh clients (SSHD-1141) which I assume
is needed by the replication plugin. Then the next release of MINA
(2.8.0?) will fix this on the sshd side so that openssh users can push
to Gerrit using RSA keys. It may also be possible to vendor the fixed
MINA code into Gerrit without a new MINA release. I personally stopped
trying to work on this after it became clear MINA is intertwined with
JGit and needed more time and understanding than I could give it. But
I think this is still possible.

From the openssh release notes:

For example, the following stanza in ~/.ssh/config will enable
RSA/SHA1 for host and user authentication for a single destination
host:

Host old-host
HostkeyAlgorithms +ssh-rsa
PubkeyAcceptedAlgorithms +ssh-rsa

We recommend enabling RSA/SHA1 only as a stopgap measure until
legacy implementations can be upgraded or reconfigured with another
key type (such as ECDSA or Ed25519).

I have personally updated my key.

[Michael Tughan]

That was the recommendation we made to people affected by this too, to upgrade to a Ed25519 key, which is more secure in the long run anyway. I think that's what we'll be sticking with, but it does mean that people who are using older SSH versions (which don't disable the ssh-rsa key exchange algorithm) may still continue to use ssh-rsa even if they support stronger exchange algorithms. It'd probably still be good to get this resolved.

Thanks all for your responses and your input. So we can determine that Gerrit (by way of JGit and its MINA backend) does not support the stronger RSA/SHA-2 key exchange algorithms right now and it may be a little bit before they do (as JGit would need to update to MINA 2.8.0 [or whatever version includes the fix for SSHD-1216] and then Gerrit would need to update to the new JGit version). In the meantime, the workarounds identified before (to use a different, stronger private key algorithm or to enable ssh-rsa) appear to be valid and what should be done in the meantime. Of course, HTTP(S) can always be used in replacement for SSH and will sidestep all of this.

Michael

[David Ostrovsky]

clark....@gmail.com schrieb am Dienstag, 23. November 2021 um 16:53:37 UTC+1:

On Tue, Nov 23, 2021 at 4:25 AM Oswald Buddenhagen
<oswald.bu...@gmx.de> wrote:
>
> On Tue, Nov 23, 2021 at 11:54:50AM +0100, 'Han-Wen Nienhuys' via Repo and Gerrit Discussion wrote:
> >ie. rsa-sha2 is accepted for host keys. Maybe it needs to be enabled
> >for user authentication separately?
> >
> yes. https://bugs.chromium.org/p/gerrit/issues/detail?id=13930

And these are the two related MINA bugs:

https://issues.apache.org/jira/browse/SSHD-1141
https://issues.apache.org/jira/browse/SSHD-1216

MINA 2.7.0 fixes this for MINA ssh clients (SSHD-1141) which I assume
is needed by the replication plugin. Then the next release of MINA
(2.8.0?) will fix this on the sshd side so that openssh users can push
to Gerrit using RSA keys. It may also be possible to vendor the fixed
MINA code into Gerrit without a new MINA release. I personally stopped
trying to work on this after it became clear MINA is intertwined with
JGit and needed more time and understanding than I could give it. But
I think this is still possible.

SSHD 2.8.0 upgrade is done in Gerrit: [1], that also includes this pending 

JGit change: [2] for this issue: [3].

[1] https://gerrit-review.googlesource.com/c/gerrit/+/327079

[2] https://git.eclipse.org/r/c/jgit/jgit/+/188371

[3] https://bugs.eclipse.org/bugs/show_bug.cgi?id=577542