TLS1.3 with ECC certificate fails with "CertificateVerify uses unsupported signature algorithm"

[Arun Kumar]

Hi Team, 

We are using Rabbitmq 3.9.13 with Erlang 24.2.1, and when we try to enable TLSv1.3 with ECC certificates (secp256r1/secp381r1 curves) a continuous error is thrown in Rabbitmq logs - tls_handshake_1_3.erl:2082 "CertificateVerify uses unsupported signature algorithm".

But Rabbitmq works well with TLSv1.3 and RSA certificates and also TLS1.2 works  successfully with same ECC and RSA certificates. Only when we enable TLSv1.3 with ECC certificates it is causing issue. 

Please find attached the Rabbitmq log & config file, wireshark trace of TLSv1.3 with ECC cert handshake failure, Erlang crypto command output, and also the Java client with Tls1.3 trying to connect to Rabbitmq, 

Can you kindly check and let me know if there is any issue with TLSv1.3+ECC on Rabbitmq?

[Lemons, Terry]

Hi Arun

 

You mentioned, “Rabbitmq works well with TLSv1.3 and RSA certificates”. https://www.rabbitmq.com/ssl.html#enabling-tls describes how to specify the certificate information that RabbitMQ TLS will use. I believe that ciphers that use RSA certificates and ciphers that use ECC certificates use different certificates. From my reading of https://www.rabbitmq.com/ssl.html#enabling-tls, it seems that only one certificate ‘type’ (RSA or ECC, in your case) can be specified at a time. This would mean that you could use RSA-type ciphers or ECC-type ciphers, but not both at the same time.

 

But I’d be happy for others to point out that I’m wrong!

 

Thanks

tl

 

From: rabbitm...@googlegroups.com <rabbitm...@googlegroups.com> On Behalf Of Arun Kumar
Sent: Tuesday, October 25, 2022 2:00 AM
To: rabbitmq-users
Subject: [rabbitmq-users] TLS1.3 with ECC certificate fails with "CertificateVerify uses unsupported signature algorithm"

 

[EXTERNAL EMAIL]

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/afcfe11c-9630-49ad-865b-70e3c2561971n%40googlegroups.com [groups.google.com].

 

Internal Use - Confidential

[Arun Kumar]

Hi Terry,

Yes, as you mentioned we can set only 1 certificate type(RSA or ECC) in the ssl_options config, but we have enabled and set both RSA and ECC ciphers in the config which you can see in the attached advanced.config file.
Based on the need, the certificate will be switched between RSA or ECC.
When the RSA cert was set, the RSA ciphers were used for communication, and when ECC cert was set later the ECC ciphers were used for communication.
We are using TLSv1.2 since almost 5 years now and both RSA and ECC certificates have worked with TLSv1.2 without any issue and it is still working fine for TLSv1.2.
Also we have multiple server-client systems with individual Rabbitmq servers on them, where server can use ECC cert and client uses RSA cert and this also has been working fine with TLSv1.2 without any issue in all these years.

Recently we received requirement to enable TLSv1.3 along with TLSv1.2 in Rabbitmq so that we can move towards TLSv1.3 in the future.
As part of this change for TLSv1.3, Rabbitmq with RSA certificates seems to work fine even with TLSv1.3 but when we try to switch to ECC certificates, the rabbitmq throws an error as mentioned in my previous message.

TLSv1.2 is working fine with both RSA and ECC certificates and TLSv1.3 works fine ONLY for RSA certificate.
When we switch to ECC certificate  with TLSv1.3 it is not working and is throwing the error.

You can check the advanced.config and the rabbitmq.log file attached for your reference.

[Luke Bakken]

Hi Arun,

My first suggestion is to re-try with the latest version of Erlang, 25.1.2

If you are using openssl to generate your certificates, can you provide the commands and configuration you are using so I can generate certs in exactly the same manner for testing?

Thanks -

Luke

[Arun Kumar]

Hi Luke, 

I checked this link (https://www.rabbitmq.com/which-erlang.html) and we are already upgrading the Rabbitmq to 3.11.2 and Erlang to 25.1 on one of our systems and we shall let you know the test results for TLSv1.3 + ECC certificates. Is this fine?

To answer the other question related to how we generate our certificates, we are using Java BouncyCastle APIs to programatically generate our certificates (we do not use openssl). I have attached both the RSA and ECC end entity certificates for your reference. Can you please check the certificates?

[Luke Bakken]

Thanks for testing.

Certs are pretty useless without private keys so I can't check the files you sent me. I think we have ECC support in our cert gen project so I'll use that - https://github.com/rabbitmq/tls-gen

[Arun Kumar]

Hi Luke, I have attached the rootCA, server end entity ECC certificate and server ECC private key in pem format which we used in our Rabbitmq. You can use these ECC certificate in your testing too.

Also I have upgraded my system with  Rabbitmq to 3.11.2 and Erlang to 25.1. I will test the TLSv1.3+ECC on this system in a couple of days and share with you the results.

[Arun Kumar]

Hi Luke, 

I tested on that upgraded system which has  RabbitMQ 3.11.2 and Erlang 25.1. With TLSv1.3 + ECC certificates on RabbitMQ, the same error still persists and RabbitMQ continuously throws the error into the rabbitmq.log file - ""CertificateVerify uses unsupported signature algorithm""

But TLSv1.3 + RSA certificates it is working fine without any issue. 

Attached is the rabbitmq.log file which has the error, the rabbitmq version snapshot and the Java Spring client bean which we use to connect to  RabbitMQ 

Kindly share your views once you test TLSv1.3 + ECC at your end.

[Luke Bakken]

Hi Arun,

To take RabbitMQ out of the equation, I have the following test code:

https://github.com/lukebakken/rabbitmq-users-tls1.3-ecc-certs-4qqaOO-QZf4

In one terminal, run this command, which will start up an Erlang TLS server -

./run-tls-server.sh

...in another terminal, run this command, which uses openssl s_client to connect to the Erlang TLS server

./run-tls-client.sh

In my environment, an up-to-date Arch Linux workstation, the test succeeds.

I'm now testing with RabbitMQ.

On Wednesday, November 2, 2022 at 6:09:01 AM UTC-7 Arun Kumar wrote:

Hi Luke, 

[Luke Bakken]

Hi again Arun,

In the same project you will see run-rabbitmq-server.sh and run-rabbitmq-client.sh. The first command starts RabbitMQ using your certificates as well as your settings, and the run-rabbitmq-client.sh command connects to port 5671 using openssl s_client

As with the other test, everything succeeds in my Arch Linux environment.

Based on your configuration file, you appear to be running on Windows. I will add equivalent Powershell scripts to the repository so you can run the same thing in your environment.

When Erlang runs on Windows, it uses OpenSSL for cryptographic functions, so I doubt that I will see a different behavior in my Windows environment. However, your Java virtual machine may be using the Windows TLS implementation ("schannel") that may be the cause of the issue. Which Java VM are you using?

Thanks,

Luke

[Luke Bakken]

Note: I am using Erlang version 25.1.2

[Luke Bakken]

Hi Arun,

As expected, running my tests on Windows using your ECC certificates works correctly. Please see the output attached from the two scripts on Windows.

At this point the issue lies in your environment. Please run the following two scripts in your Windows server and provide the FULL output as file attachments. If RabbitMQ is already running, STOP it first because my script downloads and runs RabbitMQ itself from the zip archive (it will not install anything):

.\run-rabbitmq-server.ps1

...in another terminal, after RabbitMQ starts

.\run-rabbitmq-client.ps1

Thank you,

Luke

[Arun Kumar]

Hi Luke, 

Thank you getting back with the elaborate results. 
We are using Java OpenJDK "11.0.16.1" with Java version details as below in our windows system. 

  openjdk version "11.0.16.1" 2022-08-12
  OpenJDK Runtime Environment Temurin-11.0.16.1+1 (build 11.0.16.1+1)

  OpenJDK 64-Bit Server VM Temurin-11.0.16.1+1 (build 11.0.16.1+1, mixed mode) 
  

Yes, we are running on Windows and we are using the Java + Spring and the Spring class="org.springframework.amqp.rabbit.connection.RabbitConnectionFactoryBean" is used in Spring XML configuration file to create the RabbitMQ client bean which tries to connect to the RabbitMQ upon our application startup.

How do we check if our Java virtual machine is using the Windows TLS implementation ("schannel") or its internal Java TLS implementation.

In our application we have tried to set the Java system properties ( System.setProperty("jdk.tls.client.protocols", "TLSv1.3,TLSv1.2"); System.setProperty("https.protocols","TLSv1.3,TLSv1.2");) to use TLS1.3 with TLS1.2. Even After setting this property in Java we are getting the same error. 

I will check my system environment further as you suggested and also your powershell scripts and update you on the status. 

[Luke Bakken]

Arun,

I am using this Java version on my Windows workstation:

> java -version
openjdk version "19" 2022-09-20
OpenJDK Runtime Environment Temurin-19+36 (build 19+36)
OpenJDK 64-Bit Server VM Temurin-19+36 (build 19+36, mixed mode, sharing)

My guess is that, on Windows, Java uses the Windows SSL/TLS implementation.

I just added two more Powershell scripts to the repository -

https://github.com/lukebakken/rabbitmq-users-tls1.3-ecc-certs-4qqaOO-QZf4/blob/main/setup-java-keystores.ps1

https://github.com/lukebakken/rabbitmq-users-tls1.3-ecc-certs-4qqaOO-QZf4/blob/main/run-perftest.ps1

The first one will set up the Java key and trust stores. They are already committed to the certs/ dir in the repo, so you don't have to run that, but I thought you'd be interested to see how I'm doing it.

The run-perftest.ps1 script will download and run PerfTest using your certs.

Guess what, I see the same error as you in the RabbitMQ logs -

2022-11-04 07:39:36.953551-07:00 [notice] <0.605.0> TLS server: In state wait_cv at tls_handshake_1_3.erl:2071 generated SERVER ALERT: Fatal - Handshake Failure
2022-11-04 07:39:36.953551-07:00 [notice] <0.605.0>  - "CertificateVerify uses unsupported signature algorithm"

Now I'm getting somewhere! I'll follow up.

[Luke Bakken]

Hi Arun,

Please see this issue - https://github.com/erlang/otp/issues/6435

[Arun Kumar]

Luke,
Wow! You are amazingly quick. Thank you for taking so much efforts and reproducing it in your system and raising this bug. 

I tried to upgrade my system Java to 18 and 19, but I still see the same error. So even with the latest Java version the issue is not resolved. 

I will monitor the status on the bug you raised too. Please let me know if anything is needed from my end and I will be happy to share with you. 

Thanks and cheers, 

[Luke Bakken]

Hi Arun,

https://github.com/erlang/otp/issues/6435#issuecomment-1317210750

The problem is that your ECC certs use sha as a "hash signing algorithm". This is not supported by TLS 1.3

See the discussion starting here - https://github.com/erlang/otp/issues/6435#issuecomment-1317210750

Solution is to fix this in your certs if you wish to use TLS 1.3, otherwise use 1.2.

[Luke Bakken]

Actually, my answer may be incorrect.

Currently you are using the following setting for versions:

['tlsv1.3', 'tlsv1.2']

Try changing that to

['tlsv1.3']

Let me know if that resolves your issue.

[Arun Kumar]

Hi Luke, 

I will try out the options listed by you and will update you the result of the test soon.

[Arun Kumar]

Hi Luke, 

I tried to change ['tlsv1.3', 'tlsv1.2'] to ['tlsv1.3'] and I am getting a different error now as follows 

When Java Client uses TLSv1.3 and RabbitMQ is configured only with ['tlsv1.3'] and ECC Certificate , I am continuously getting the below error -  

2022-11-21 16:29:47.215000+05:30 [notice] <0.10773.0> TLS server: In state wait_cv at tls_handshake_1_3.erl:604 generated SERVER ALERT: Fatal - Internal Error

2022-11-21 16:29:47.215000+05:30 [notice] <0.10773.0>  - badarg

As to the other point, sha is the standard for "Signing Hash Algorithm" irrespective of RSA/ECC and in the certificate the sha256 is used for sign hash.

[Luke Bakken]

Hello -

OK yes, badarg is what I see as well with my test code and is a bug. See the comment here:  https://github.com/erlang/otp/issues/6435#issuecomment-1319875377

I'm trying my test project (https://github.com/lukebakken/erlang-otp-6435/) with a custom Erlang built with this PR - https://github.com/erlang/otp/pull/6490

[Luke Bakken]

Hello,

Yes, PR #6490 fixes the issue you report. It should ship in an upcoming Erlang 25 release. I suggest watching it!

Thanks,

Luke

[Arun Kumar]

Hi Luke, 

okay thank you. I will also monitor the Bug # 6435 and  PR #6490.

[Luke Bakken]

Check out this comment - https://github.com/erlang/otp/issues/6435#issuecomment-1326227133

It is Java that is choosing to use sha1 during the handshake, which tripped up Erlang.

Your workaround at this time is to use TLS 1.2

Thanks,

Luke

On Wednesday, November 23, 2022 at 10:53:43 PM UTC-8 Arun Kumar wrote:

Hi Luke, 

[Arun Kumar]

Hi Luke, 

We will continue to use TLS1.2 as you suggested.

Also, should we raise a bug or notify Java about this issue?