Java upgrade with TLS 1.3 support causes rabbitmq TLS handshake failures

1,657 views
Skip to first unread message

Terry Lemons

unread,
Jan 13, 2021, 4:50:41 PM1/13/21
to rabbitmq-users
Hi

Our engineering team has a system that is using RabbitMQ java client (com.rabbitmq:amqp-client:5.4.3) to connect over TLS 1.2 to a RabbitMQ server (version 3.7.16) on another system. This connection broke with a TLS handshake failure when we upgraded our system from java-1_8_0-openjdk-1.8.0.252-27.45.6.x86_64 to Nov 5, 2020: java-1_8_0-openjdk-1.8.0.272-27.48.1.x86_64.rpm. The changelog for this later release (https://centos.pkgs.org/7/centos-updates-x86_64/java-1.8.0-openjdk-1.8.0.272.b10-1.el7_9.x86_64.rpm.html) contains a note about the inclusion of TLS 1.3. Looking in the Java 8u261 Release Notes (https://www.oracle.com/java/technologies/javase/8u261-relnotes.html), I see that this release includes support for TLS 1.3.

When we did a packet capture and analyzed the conversation between the RabbitMQ client and server, we noticed that the server was failing the TLS handshake:
In the trace we see the client receiving - javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

Further analysis of the Client Hello packet being generated by our system showed that the ‘Version’ value is ‘TLS 1.2’ but the packet contains several extensions including ‘signature_algorithrms_cert’ and ‘supported_versions’. We found that these two extensions are NOT part of the TLS 1.2 RFC (https://tools.ietf.org/html/rfc5246) but are part of the TLS 1.3 RFC (https://tools.ietf.org/html/rfc8446).

So, my current theory is that the RabbitMQ server on the other system is forcing a handshake failure, because it finds that the Client Hello packet contains extensions that it doesn’t understand.

This feels like a bug to me, in that the TLS packet has a ‘Version’ value of ‘TLS 1.2’, but the extensions are NOT part of TLS 1.3.

1. Do you agree that this is a bug that needs to be fixed?
2. Is there any way to get our application to work with the current software stack? We see that this handshake works with java-1_8_0-openjdk-1.8.0.252-27.45.6.x86_64 but not with java-1_8_0-openjdk-1.8.0.272-27.48.1.x86_64.rpm.

Thanks!
Terry Lemons

Gabriele Santomaggio

unread,
Jan 14, 2021, 2:56:41 AM1/14/21
to rabbitmq-users
RabbitMQ supports TLS 1.3 starting from  v3.8.10 ( the version should be released soon).
See: TLSv1.3 is no longer considered experimental (on Erlang 23) and excluded from TLS listener configuration.


Please give a try with 3.8.10. (Erlang must be updated too)

Cheers
-
Gabriele Santomaggio

Arnaud Cogoluegnes

unread,
Jan 14, 2021, 3:35:52 AM1/14/21
to rabbitm...@googlegroups.com
3.7.x has reached EOL [1], so you should upgrade to 3.8.9 and upgrade Erlang as well.

If you are really stuck with 3.7.x you may try to bump the Erlang version you are using. Erlang handles the TLS handshake, not RabbitMQ, so you may be hitting some incompatibility between the JDK and the Erlang you are using (you did not specify the Erlang version you are using BTW). This would not be the first this kind of incompatibility pops up [2].

And as Gabriele explained, if you want to use TLS 1.3, you need to test the last RC of RabbitMQ 3.8.10.

[1] https://www.rabbitmq.com/versions.html
[2] https://bugs.erlang.org/browse/ERL-973

________________________________________
From: rabbitm...@googlegroups.com <rabbitm...@googlegroups.com> on behalf of Terry Lemons <terry....@dell.com>
Sent: Wednesday, January 13, 2021 10:50 PM
To: rabbitmq-users
Subject: [Suspected Spam] [rabbitmq-users] Java upgrade with TLS 1.3 support causes rabbitmq TLS handshake failures
--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com<mailto:rabbitmq-user...@googlegroups.com>.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/2ba6f22f-0463-4604-bbbd-e3f9766229e5n%40googlegroups.com<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Frabbitmq-users%2F2ba6f22f-0463-4604-bbbd-e3f9766229e5n%2540googlegroups.com%3Futm_medium%3Demail%26utm_source%3Dfooter&data=04%7C01%7Cacogoluegnes%40vmware.com%7C3d9a7f854283455ad5ec08d8b80d4987%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637461714502119719%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=7agkw81B3mzUfxw08X4Lp0Hhf%2Fv5BB1lkfD7GhYM64s%3D&reserved=0>.

Terry Lemons

unread,
Jan 14, 2021, 12:05:44 PM1/14/21
to rabbitmq-users
Hi

Thanks Gabriele and Arnaud for your replies!

I should have been clearer with my question. Our system is the ‘client’ side of the RabbitMQ conversation, so our problem seems to be with the rabbitmq client – where the Client Hello seems to have additional fields from TLS 1.3 even though we specify TLS 1.2 as the protocol to follow. We’re currently using version 5.4.3 of the RabbitMQ Java client (com.rabbitmq:amqp-client:5.4.3) – note that we also tried this with (com.rabbitmq:amqp-client:5.10.0) which seems to be the latest. Is there a later version of the ampq-client that has been updated to handle TLS 1.3?
Specifically, in the outgoing rabbitmq client-generated Client Hello, we want to use TLS 1.2 (as specified in the TLS 1.2 RFC); we don't want to use TLS 1.3 or "TLS 1.2 plus TLS 1.3 additions".

Thanks!
Terry Lemons

Arnaud Cogoluegnes

unread,
Jan 14, 2021, 12:31:18 PM1/14/21
to rabbitm...@googlegroups.com
You can force the TLS version to use with ConnectionFactory#useSslProtocol(SSLContext) and SSLContext#getInstance(String). But I don't think that would be useful in your case, the Java client favors TLS 1.2 by default anyway.

As stated in my previous email, the JDK and Erlang versions may not be compatible (because of a bug somewhere), this is why I'd like to know the *Erlang version* in use. The upgrade of the JDK may have broken your system, because Erlang does not understand the TLS of this very JDK. I've seen that before [1]. If you cannot find out about the Erlang version, you may have to downgrade the JDK or find another version (Java 11+ ?) that would work with this Erlang.

[1] https://bugs.erlang.org/browse/ERL-973

________________________________________
From: rabbitm...@googlegroups.com <rabbitm...@googlegroups.com> on behalf of Terry Lemons <terry....@dell.com>

Sent: Thursday, January 14, 2021 6:05 PM
To: rabbitmq-users
Subject: [Suspected Spam] Re: [Suspected Spam] [rabbitmq-users] Java upgrade with TLS 1.3 support causes rabbitmq TLS handshake failures

Hi

Thanks Gabriele and Arnaud for your replies!

I should have been clearer with my question. Our system is the ‘client’ side of the RabbitMQ conversation, so our problem seems to be with the rabbitmq client – where the Client Hello seems to have additional fields from TLS 1.3 even though we specify TLS 1.2 as the protocol to follow. We’re currently using version 5.4.3 of the RabbitMQ Java client (com.rabbitmq:amqp-client:5.4.3) – note that we also tried this with (com.rabbitmq:amqp-client:5.10.0) which seems to be the latest. Is there a later version of the ampq-client that has been updated to handle TLS 1.3?
Specifically, in the outgoing rabbitmq client-generated Client Hello, we want to use TLS 1.2 (as specified in the TLS 1.2 RFC); we don't want to use TLS 1.3 or "TLS 1.2 plus TLS 1.3 additions".

Thanks!
Terry Lemons

On Thursday, January 14, 2021 at 3:35:52 AM UTC-5 Arnaud Cogoluègnes wrote:
3.7.x has reached EOL [1], so you should upgrade to 3.8.9 and upgrade Erlang as well.

If you are really stuck with 3.7.x you may try to bump the Erlang version you are using. Erlang handles the TLS handshake, not RabbitMQ, so you may be hitting some incompatibility between the JDK and the Erlang you are using (you did not specify the Erlang version you are using BTW). This would not be the first this kind of incompatibility pops up [2].

And as Gabriele explained, if you want to use TLS 1.3, you need to test the last RC of RabbitMQ 3.8.10.

[1] https://www.rabbitmq.com/versions.html<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.rabbitmq.com%2Fversions.html&data=04%7C01%7Cacogoluegnes%40vmware.com%7Cf4f70d28dd014bd6c51a08d8b8aea58c%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637462407531261829%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=yT6qlaMxNOHIlAcMcTaZD1unQBn7VSwkvHRzqEWVtHE%3D&reserved=0>
[2] https://bugs.erlang.org/browse/ERL-973<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.erlang.org%2Fbrowse%2FERL-973&data=04%7C01%7Cacogoluegnes%40vmware.com%7Cf4f70d28dd014bd6c51a08d8b8aea58c%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637462407531271829%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=t9ydghiRz2RrZNytHnrqKTqgW0bQQ%2FFLZms0TzEgj%2Fo%3D&reserved=0>

________________________________________
From: rabbitm...@googlegroups.com <rabbitm...@googlegroups.com> on behalf of Terry Lemons <terry....@dell.com>
Sent: Wednesday, January 13, 2021 10:50 PM
To: rabbitmq-users
Subject: [Suspected Spam] [rabbitmq-users] Java upgrade with TLS 1.3 support causes rabbitmq TLS handshake failures

Hi

Our engineering team has a system that is using RabbitMQ java client (com.rabbitmq:amqp-client:5.4.3) to connect over TLS 1.2 to a RabbitMQ server (version 3.7.16) on another system. This connection broke with a TLS handshake failure when we upgraded our system from java-1_8_0-openjdk-1.8.0.252-27.45.6.x86_64 to Nov 5, 2020: java-1_8_0-openjdk-1.8.0.272-27.48.1.x86_64.rpm. The changelog for this later release (https://centos.pkgs.org/7/centos-updates-x86_64/java-1.8.0-openjdk-1.8.0.272.b10-1.el7_9.x86_64.rpm.html<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcentos.pkgs.org%2F7%2Fcentos-updates-x86_64%2Fjava-1.8.0-openjdk-1.8.0.272.b10-1.el7_9.x86_64.rpm.html&data=04%7C01%7Cacogoluegnes%40vmware.com%7Cf4f70d28dd014bd6c51a08d8b8aea58c%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637462407531271829%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=MlZzWzbftshDQ6THYZxcJiXATMb09Xxvlaflx8p0wfo%3D&reserved=0>) contains a note about the inclusion of TLS 1.3. Looking in the Java 8u261 Release Notes (https://www.oracle.com/java/technologies/javase/8u261-relnotes.html<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.oracle.com%2Fjava%2Ftechnologies%2Fjavase%2F8u261-relnotes.html&data=04%7C01%7Cacogoluegnes%40vmware.com%7Cf4f70d28dd014bd6c51a08d8b8aea58c%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637462407531281822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=GMxPYYdK%2BjYftRSAX8TNYI46rViJlWiHlmRnQgw8jCk%3D&reserved=0>), I see that this release includes support for TLS 1.3.

When we did a packet capture and analyzed the conversation between the RabbitMQ client and server, we noticed that the server was failing the TLS handshake:
In the trace we see the client receiving - javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

Further analysis of the Client Hello packet being generated by our system showed that the ‘Version’ value is ‘TLS 1.2’ but the packet contains several extensions including ‘signature_algorithrms_cert’ and ‘supported_versions’. We found that these two extensions are NOT part of the TLS 1.2 RFC (https://tools.ietf.org/html/rfc5246<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Frfc5246&data=04%7C01%7Cacogoluegnes%40vmware.com%7Cf4f70d28dd014bd6c51a08d8b8aea58c%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637462407531281822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=6NiMYGWB%2BuFo9WEA0GH67yE7Mo47DiHLow4Koo%2Bgr6s%3D&reserved=0>) but are part of the TLS 1.3 RFC (https://tools.ietf.org/html/rfc8446<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Frfc8446&data=04%7C01%7Cacogoluegnes%40vmware.com%7Cf4f70d28dd014bd6c51a08d8b8aea58c%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637462407531281822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=t%2FQGZF6iMEiJPc6t1CDWQ6h1NPUotbHGldscIWvTMIc%3D&reserved=0>).

So, my current theory is that the RabbitMQ server on the other system is forcing a handshake failure, because it finds that the Client Hello packet contains extensions that it doesn’t understand.

This feels like a bug to me, in that the TLS packet has a ‘Version’ value of ‘TLS 1.2’, but the extensions are NOT part of TLS 1.3.

1. Do you agree that this is a bug that needs to be fixed?
2. Is there any way to get our application to work with the current software stack? We see that this handshake works with java-1_8_0-openjdk-1.8.0.252-27.45.6.x86_64 but not with java-1_8_0-openjdk-1.8.0.272-27.48.1.x86_64.rpm.

Thanks!
Terry Lemons


--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com<mailto:rabbitmq-user...@googlegroups.com>.

To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/2ba6f22f-0463-4604-bbbd-e3f9766229e5n%40googlegroups.com<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Frabbitmq-users%2F2ba6f22f-0463-4604-bbbd-e3f9766229e5n%2540googlegroups.com&data=04%7C01%7Cacogoluegnes%40vmware.com%7Cf4f70d28dd014bd6c51a08d8b8aea58c%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637462407531291814%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=52g%2FHOBaDpkq34Ri3MF4CZ4C53hz%2BNeDHFz7kzM3jLs%3D&reserved=0><https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Frabbitmq-users%2F2ba6f22f-0463-4604-bbbd-e3f9766229e5n%2540googlegroups.com%3Futm_medium%3Demail%26utm_source%3Dfooter&data=04%7C01%7Cacogoluegnes%40vmware.com%7C3d9a7f854283455ad5ec08d8b80d4987%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637461714502119719%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=7agkw81B3mzUfxw08X4Lp0Hhf%2Fv5BB1lkfD7GhYM64s%3D&reserved=0<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Frabbitmq-users%2F2ba6f22f-0463-4604-bbbd-e3f9766229e5n%2540googlegroups.com%3Futm_medium%3Demail%26utm_source%3Dfooter&data=04%7C01%7Cacogoluegnes%40vmware.com%7Cf4f70d28dd014bd6c51a08d8b8aea58c%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637462407531291814%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Cl%2Bc1O78vgd9CuzXgbjh7Kw%2FJI92tbcLD1F5e%2FDIF1U%3D&reserved=0>>.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com<mailto:rabbitmq-user...@googlegroups.com>.

To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/a68a40e3-a0f4-41ee-a249-89a3e29aaae7n%40googlegroups.com<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Frabbitmq-users%2Fa68a40e3-a0f4-41ee-a249-89a3e29aaae7n%2540googlegroups.com%3Futm_medium%3Demail%26utm_source%3Dfooter&data=04%7C01%7Cacogoluegnes%40vmware.com%7Cf4f70d28dd014bd6c51a08d8b8aea58c%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637462407531301809%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=tyOnxE%2F34wo2ip9qkWvLn8V1J2RIB1Z6GHO69I16wj0%3D&reserved=0>.

Terry Lemons

unread,
Jan 14, 2021, 5:02:28 PM1/14/21
to rabbitmq-users
Hi Arnaud

On the RabbitMQ *server* (which is the 'other end' of the conversation from my team's system),  erlang/OTP 22 is being used. On my team's system, we're using erlang 22.3.4.5-1 for a different connection (not part of this discussion).

But, as I mentioned, my team's system is using the RabbitMQ Java client (which, I believe, does not use erlang?). Regarding the RabbitMQ Java client's formation of the TLS Client Hello packet, my understanding is that the RabbitMQ Java client delegates to the Java Secure Socket Extension (JSSE) API for TLS communication. That would mean that the RabbitMQ java client is not the problem. It is the JSSE provider that came with build SLES 12 openjdk 8u272 and 8u275.

Thoughts?

Thanks
Terry Lemons

Arnaud Cogoluegnes

unread,
Jan 15, 2021, 3:53:24 AM1/15/21
to rabbitm...@googlegroups.com
> On the RabbitMQ *server* (which is the 'other end' of the conversation from my team's system), erlang/OTP 22 is being used. On my team's system, we're using erlang 22.3.4.5-1 for a different connection (not part of this discussion).

This is confusing, you mention Erlang 22 and Erlang 22.3.4.5, on possibly 2 different systems. We need to know exactly on which versions of RabbitMQ and Erlang you're encountering the problem (and all the digits for each version).

And yes, the Java client uses the JDK for TLS and RabbitMQ uses Erlang for TLS, so it's likely a problem between the runtimes (JDK and Erlang) that causes the problem.

I tried the programs from ERL-973 [1] (which originates from a similar issue I had with the Java client on Java 11 and Erlang 22.0.2) with Java 8-252, 8-272, 8-275 and Erlang 22.3.4.5 but I could not reproduce, the handshake succeeded. Nevertheless I see the extensions you mention show up only on 272 and 275.

With 272:

*** ClientHello, TLSv1.2
RandomCookie: GMT: 1610699286 bytes = { 172, 210, 206, 220, 64, 184, 190, 151, 152, 42, 236, 243, 211, 144, 101, 153, 118, 151, 139, 217, 221, 50, 78, 88, 187, 128, 136, 12 }
Session ID: {}
Cipher Suites: [TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
Compression Methods: { 0 }
Extension elliptic_curves, curve names: {secp256r1, secp384r1, secp521r1}
Extension ec_point_formats, formats: [uncompressed]
Extension signature_algorithms, signature_algorithms: SHA512withECDSA, SHA512withRSA, SHA384withECDSA, SHA384withRSA, SHA256withECDSA, SHA256withRSA, SHA256withDSA, SHA224withECDSA, SHA224withRSA, SHA224withDSA, SHA1withECDSA, SHA1withRSA, SHA1withDSA
Extension extended_master_secret
***
main, WRITE: TLSv1.2 Handshake, length = 185
main, READ: TLSv1.2 Handshake, length = 87
check handshake state: server_hello[2]
*** ServerHello, TLSv1.2
RandomCookie: GMT: 1610699286 bytes = { 212, 170, 193, 81, 160, 20, 143, 114, 73, 75, 199, 170, 202, 161, 94, 197, 50, 228, 202, 163, 123, 213, 0, 19, 220, 55, 201, 193 }
Session ID: {84, 178, 166, 96, 19, 204, 152, 199, 154, 3, 189, 79, 87, 167, 38, 136, 177, 133, 139, 199, 12, 32, 215, 244, 68, 28, 175, 40, 188, 171, 231, 159}
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
Compression Method: 0
Extension renegotiation_info, renegotiated_connection: <empty>
Extension ec_point_formats, formats: [uncompressed]
***

With 272 and 275:

javax.net.ssl|FINE|01|main|2021-01-15 09:33:07.910 CET|Logger.java:765|Produced ClientHello handshake message (
"ClientHello": {
"client version" : "TLSv1.2",
"random" : "AE 88 6C A7 12 01 B0 44 13 66 20 B3 A0 00 DC BA BA A7 6F 6F C5 11 FE A2 FE 12 0D D2 60 4F 7A 59",
"session id" : "",
"cipher suites" : "[TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384(0xC02C), TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256(0xC02B), TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384(0xC030), TLS_RSA_WITH_AES_256_GCM_SHA384(0x009D), TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384(0xC02E), TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384(0xC032), TLS_DHE_RSA_WITH_AES_256_GCM_SHA384(0x009F), TLS_DHE_DSS_WITH_AES_256_GCM_SHA384(0x00A3), TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256(0xC02F), TLS_RSA_WITH_AES_128_GCM_SHA256(0x009C), TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256(0xC02D), TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256(0xC031), TLS_DHE_RSA_WITH_AES_128_GCM_SHA256(0x009E), TLS_DHE_DSS_WITH_AES_128_GCM_SHA256(0x00A2), TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384(0xC024), TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384(0xC028), TLS_RSA_WITH_AES_256_CBC_SHA256(0x003D), TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384(0xC026), TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384(0xC02A), TLS_DHE_RSA_WITH_AES_256_CBC_SHA256(0x006B), TLS_DHE_DSS_WITH_AES_256_CBC_SHA256(0x006A), TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA(0xC00A), TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA(0xC014), TLS_RSA_WITH_AES_256_CBC_SHA(0x0035), TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA(0xC005), TLS_ECDH_RSA_WITH_AES_256_CBC_SHA(0xC00F), TLS_DHE_RSA_WITH_AES_256_CBC_SHA(0x0039), TLS_DHE_DSS_WITH_AES_256_CBC_SHA(0x0038), TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256(0xC023), TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256(0xC027), TLS_RSA_WITH_AES_128_CBC_SHA256(0x003C), TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256(0xC025), TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256(0xC029), TLS_DHE_RSA_WITH_AES_128_CBC_SHA256(0x0067), TLS_DHE_DSS_WITH_AES_128_CBC_SHA256(0x0040), TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA(0xC009), TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA(0xC013), TLS_RSA_WITH_AES_128_CBC_SHA(0x002F), TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA(0xC004), TLS_ECDH_RSA_WITH_AES_128_CBC_SHA(0xC00E), TLS_DHE_RSA_WITH_AES_128_CBC_SHA(0x0033), TLS_DHE_DSS_WITH_AES_128_CBC_SHA(0x0032), TLS_EMPTY_RENEGOTIATION_INFO_SCSV(0x00FF)]",
"compression methods" : "00",
"extensions" : [
"supported_groups (10)": {
"versions": [secp256r1, secp384r1, secp521r1, ffdhe2048, ffdhe3072, ffdhe4096, ffdhe6144, ffdhe8192]
},
"ec_point_formats (11)": {
"formats": [uncompressed]
},
"signature_algorithms (13)": {
"signature schemes": [ecdsa_secp256r1_sha256, ecdsa_secp384r1_sha384, ecdsa_secp521r1_sha512, rsa_pss_rsae_sha256, rsa_pss_rsae_sha384, rsa_pss_rsae_sha512, rsa_pss_pss_sha256, rsa_pss_pss_sha384, rsa_pss_pss_sha512, rsa_pkcs1_sha256, rsa_pkcs1_sha384, rsa_pkcs1_sha512, dsa_sha256, ecdsa_sha224, rsa_sha224, dsa_sha224, ecdsa_sha1, rsa_pkcs1_sha1, dsa_sha1]
},
"signature_algorithms_cert (50)": {
"signature schemes": [ecdsa_secp256r1_sha256, ecdsa_secp384r1_sha384, ecdsa_secp521r1_sha512, rsa_pss_rsae_sha256, rsa_pss_rsae_sha384, rsa_pss_rsae_sha512, rsa_pss_pss_sha256, rsa_pss_pss_sha384, rsa_pss_pss_sha512, rsa_pkcs1_sha256, rsa_pkcs1_sha384, rsa_pkcs1_sha512, dsa_sha256, ecdsa_sha224, rsa_sha224, dsa_sha224, ecdsa_sha1, rsa_pkcs1_sha1, dsa_sha1]
},
"extended_master_secret (23)": {
<empty>
},
"supported_versions (43)": {
"versions": [TLSv1.2, TLSv1.1, TLSv1]
}
]
}
)
javax.net.ssl|FINE|01|main|2021-01-15 09:33:07.924 CET|Logger.java:765|Consuming ServerHello handshake message (
"ServerHello": {
"server version" : "TLSv1.2",
"random" : "60 01 53 43 B0 F7 E1 5A 70 CB AF BA 95 3D B1 9B 02 29 35 C3 44 C0 A4 13 52 B7 FF CC 4B 4D 37 5A",
"session id" : "DC C3 72 A1 00 78 30 A0 B9 BD 9A FE 5B D5 20 95 30 F8 3E 40 9C C4 A4 6B FF 6F 4B BE F3 DF 93 64",
"cipher suite" : "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384(0xC030)",
"compression methods" : "00",
"extensions" : [
"renegotiation_info (65,281)": {
"renegotiated connection": [<no renegotiated connection>]
},
"ec_point_formats (11)": {
"formats": [uncompressed]
}
]
}
)

To move forward:
* please provide the exact versions of RabbitMQ and Erlang where you hit the problem
* please provide the configuration of RabbitMQ and the Java client
* ideally provide a standalone repository with code samples that would allow us to reproduce the problem

[1] https://bugs.erlang.org/browse/ERL-973

Avinash Reddy

unread,
Feb 27, 2021, 8:15:48 PM2/27/21
to rabbitmq-users
I work for same engineering team. Here is output we got when using the RabbitMQ Java client with Java 1.8.0_281 version.
As you can see there is no "ServerHello" happened in our call. I'm curious how it worked in your environment. If possible, can you please share your sample code which is working with Java8u271 onwards?

Issue here, the same code works fine with Java 8u252 or lower version (tested with 8u191 too).
But fails from Java 8u271 onwards (no change in the code). Let us know if any java command arguments I should use here to make it working?
Basically we want the RabbitMQ Java Client work with any RabbitMQ server version >= 3.6.x (and with any erlang version >= 20)

Server Details:
RabbitMQ version: 3.7.16
Erlang version:  

[{release,"Erlang/OTP","22","10.4",

          [{kernel,"6.4","/usr/lib/erlang/lib/kernel-6.4"},

           {stdlib,"3.9","/usr/lib/erlang/lib/stdlib-3.9"},

           {sasl,"3.4","/usr/lib/erlang/lib/sasl-3.4"}],

          permanent}].

Java version: 1.8.0_241-b07


Java Client Details:
Spring Boot version: 2.4.1 (using spring-amqp to test java client connection to server on port 5671)
AMQP version: 5.9.0
Java version: 1.8.0_281-b09 (we also tried with 8u272 & 8u275, got same error as shown below)
Java options used here for debugging and troubleshooting are:
-Djavax.net.debug=all -Djdk.tls.client.protocols=TLSv1.2 -Djdk.tls.client.protocols=TLSv1.2 -Djavax.net.debug=ssl:handshake:verbose -Djdk.tls.server.protocols=TLSv1.2 -Djsse.enableFFDHE=false -Djavax.net.ssl.sessionCacheSize=0 -Djdk.tls.client.enableStatusRequestExtension=true -Djdk.tls.namedGroups="secp521r1, secp256r1, secp256k1"

Exception/Output:
javax.net.ssl|FINE|11|restartedMain|2021-02-27 17:52:28.591 MST|SSLCipher.java:438|jdk.tls.keyLimits:  entry = AES/GCM/NoPadding KeyUpdate 2^37. AES/GCM/NOPADDING:KEYUPDATE = 137438953472
javax.net.ssl|FINE|11|restartedMain|2021-02-27 17:52:28.933 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 for TLS11
javax.net.ssl|FINE|11|restartedMain|2021-02-27 17:52:28.934 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 for TLS11
javax.net.ssl|FINE|11|restartedMain|2021-02-27 17:52:28.934 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 for TLS11
javax.net.ssl|FINE|11|restartedMain|2021-02-27 17:52:28.934 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 for TLS11
javax.net.ssl|FINE|11|restartedMain|2021-02-27 17:52:28.934 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_RSA_WITH_AES_256_GCM_SHA384 for TLS11
javax.net.ssl|FINE|11|restartedMain|2021-02-27 17:52:28.934 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_RSA_WITH_AES_256_CBC_SHA256 for TLS11
javax.net.ssl|FINE|11|restartedMain|2021-02-27 17:52:28.934 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_RSA_WITH_AES_128_CBC_SHA256 for TLS11
javax.net.ssl|FINE|11|restartedMain|2021-02-27 17:52:28.934 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 for TLS10
javax.net.ssl|FINE|11|restartedMain|2021-02-27 17:52:28.934 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 for TLS10
javax.net.ssl|FINE|11|restartedMain|2021-02-27 17:52:28.935 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 for TLS10
javax.net.ssl|FINE|11|restartedMain|2021-02-27 17:52:28.935 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 for TLS10
javax.net.ssl|FINE|11|restartedMain|2021-02-27 17:52:28.935 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_RSA_WITH_AES_256_GCM_SHA384 for TLS10
javax.net.ssl|FINE|11|restartedMain|2021-02-27 17:52:28.935 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_RSA_WITH_AES_256_CBC_SHA256 for TLS10
javax.net.ssl|FINE|11|restartedMain|2021-02-27 17:52:28.935 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_RSA_WITH_AES_128_CBC_SHA256 for TLS10
javax.net.ssl|WARNING|11|restartedMain|2021-02-27 17:52:28.958 MST|SignatureScheme.java:297|Signature algorithm, ed25519, is not supported by the underlying providers
javax.net.ssl|WARNING|11|restartedMain|2021-02-27 17:52:28.958 MST|SignatureScheme.java:297|Signature algorithm, ed448, is not supported by the underlying providers
javax.net.ssl|ALL|11|restartedMain|2021-02-27 17:52:28.961 MST|SignatureScheme.java:373|Ignore unsupported signature scheme: ed25519
javax.net.ssl|ALL|11|restartedMain|2021-02-27 17:52:28.961 MST|SignatureScheme.java:373|Ignore unsupported signature scheme: ed448
javax.net.ssl|ALL|11|restartedMain|2021-02-27 17:52:28.962 MST|SignatureScheme.java:393|Ignore disabled signature scheme: rsa_md5
javax.net.ssl|INFO|11|restartedMain|2021-02-27 17:52:28.962 MST|AlpnExtension.java:161|No available application protocols
javax.net.ssl|FINE|11|restartedMain|2021-02-27 17:52:28.962 MST|SSLExtensions.java:259|Ignore, context unavailable extension: application_layer_protocol_negotiation
javax.net.ssl|FINE|11|restartedMain|2021-02-27 17:52:28.964 MST|ClientHello.java:567|Produced ClientHello handshake message (
"ClientHello": {
  "client version"      : "TLSv1.2",
  "random"              : "F5 9A 19 FA A2 41 9F EF 9B B3 DD 30 7E A5 63 85 17 84 D0 1F F5 07 F9 73 59 EB 4D 07 98 E0 D9 40",
  "session id"          : "",
  "cipher suites"       : "[TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384(0xC030), TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384(0xC028), TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256(0xC02F), TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256(0xC027), TLS_RSA_WITH_AES_256_GCM_SHA384(0x009D), TLS_RSA_WITH_AES_256_CBC_SHA256(0x003D), TLS_RSA_WITH_AES_128_CBC_SHA256(0x003C), TLS_RSA_WITH_AES_128_CBC_SHA(0x002F), TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384(0xC02C), TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384(0xC024), TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256(0xC02B), TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256(0xC023)]",
  "compression methods" : "00",
  "extensions"          : [
    "server_name (0)": {
      type=host_name (0), value=rabbitserver.local.com
    },
    "status_request (5)": {
      "certificate status type": ocsp
      "OCSP status request": {
        "responder_id": <empty>
        "request extensions": {
          <empty>
        }
      }
    },
    "supported_groups (10)": {
      "versions": [secp521r1, secp256r1, secp256k1]
    },
    "ec_point_formats (11)": {
      "formats": [uncompressed]
    },
    "signature_algorithms (13)": {
      "signature schemes": [ecdsa_secp256r1_sha256, ecdsa_secp384r1_sha384, ecdsa_secp521r1_sha512, rsa_pss_rsae_sha256, rsa_pss_rsae_sha384, rsa_pss_rsae_sha512, rsa_pss_pss_sha256, rsa_pss_pss_sha384, rsa_pss_pss_sha512, rsa_pkcs1_sha256, rsa_pkcs1_sha384, rsa_pkcs1_sha512, dsa_sha256, ecdsa_sha224, rsa_sha224, dsa_sha224, ecdsa_sha1, rsa_pkcs1_sha1, dsa_sha1]
    },
    "signature_algorithms_cert (50)": {
      "signature schemes": [ecdsa_secp256r1_sha256, ecdsa_secp384r1_sha384, ecdsa_secp521r1_sha512, rsa_pss_rsae_sha256, rsa_pss_rsae_sha384, rsa_pss_rsae_sha512, rsa_pss_pss_sha256, rsa_pss_pss_sha384, rsa_pss_pss_sha512, rsa_pkcs1_sha256, rsa_pkcs1_sha384, rsa_pkcs1_sha512, dsa_sha256, ecdsa_sha224, rsa_sha224, dsa_sha224, ecdsa_sha1, rsa_pkcs1_sha1, dsa_sha1]
    },
    "status_request_v2 (17)": {
      "cert status request": {
        "certificate status type": ocsp_multi
        "OCSP status request": {
          "responder_id": <empty>
          "request extensions": {
            <empty>
          }
        }
      }
    },
    "extended_master_secret (23)": {
      <empty>
    },
    "supported_versions (43)": {
      "versions": [TLSv1.2, TLSv1.1, TLSv1]
    },
    "renegotiation_info (65,281)": {
      "renegotiated connection": [<no renegotiated connection>]
    }
  ]
}
)
javax.net.ssl|FINE|11|restartedMain|2021-02-27 17:52:29.067 MST|Alert.java:238|Received alert message (
"Alert": {
  "level"      : "fatal",
  "description": "handshake_failure"
}
)
javax.net.ssl|SEVERE|11|restartedMain|2021-02-27 17:52:29.068 MST|TransportContext.java:345|Fatal (HANDSHAKE_FAILURE): Received fatal alert: handshake_failure (
"throwable" : {
  javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
  at sun.security.ssl.Alert.createSSLException(Alert.java:131)
  at sun.security.ssl.Alert.createSSLException(Alert.java:117)
  at sun.security.ssl.TransportContext.fatal(TransportContext.java:340)
  at sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:293)
  at sun.security.ssl.TransportContext.dispatch(TransportContext.java:186)
  at sun.security.ssl.SSLTransport.decode(SSLTransport.java:154)
  at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1279)
  at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1188)
  at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:401)
  at sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketImpl.java:808)
  at sun.security.ssl.SSLSocketImpl.access$200(SSLSocketImpl.java:75)
  at sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1093)
  at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
  at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
  at java.io.DataOutputStream.flush(DataOutputStream.java:123)
  at com.rabbitmq.client.impl.SocketFrameHandler.sendHeader(SocketFrameHandler.java:160)
  at com.rabbitmq.client.impl.SocketFrameHandler.sendHeader(SocketFrameHandler.java:170)
  at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:314)
  at com.rabbitmq.client.impl.recovery.RecoveryAwareAMQConnectionFactory.newConnection(RecoveryAwareAMQConnectionFactory.java:64)
  at com.rabbitmq.client.impl.recovery.AutorecoveringConnection.init(AutorecoveringConnection.java:156)
  at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1130)
  at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1087)
  at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1223)
...
...
javax.net.ssl|FINE|11|restartedMain|2021-02-27 17:52:29.068 MST|SSLSocketImpl.java:1500|close the underlying socket
javax.net.ssl|FINE|11|restartedMain|2021-02-27 17:52:29.069 MST|SSLSocketImpl.java:1519|close the SSL connection (initiative)
2021-02-27 17:52:29.069 ERROR 15517 --- [  restartedMain] c.r.client.impl.SocketFrameHandler       : TLS connection failed: Received fatal alert: handshake_failure


Arnaud Cogoluegnes

unread,
Mar 1, 2021, 3:03:11 AM3/1/21
to rabbitm...@googlegroups.com
3.7 has reached EOL [1], please upgrade to 3.8.13 to see if you can reproduce the error.

[1] https://www.rabbitmq.com/versions.html

________________________________________
From: rabbitm...@googlegroups.com <rabbitm...@googlegroups.com> on behalf of Avinash Reddy <avinas...@gmail.com>
Sent: Sunday, February 28, 2021 2:15 AM
To: rabbitmq-users
Subject: [Suspected Spam] Re: [rabbitmq-users] Java upgrade with TLS 1.3 support causes rabbitmq TLS handshake failures
[1] https://bugs.erlang.org/browse/ERL-973<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.erlang.org%2Fbrowse%2FERL-973&data=04%7C01%7Cacogoluegnes%40vmware.com%7Ce7583bc87158489e11fd08d8db8665fd%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C1%7C0%7C637500717596201741%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=umFzrLUXdiEjJfmCrg8EvZleAm0xlc4Cd3q%2B1XCKbKE%3D&reserved=0>

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com<mailto:rabbitmq-user...@googlegroups.com>.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/7e141977-92b7-4aa0-bb5d-e06136c9bb29n%40googlegroups.com<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Frabbitmq-users%2F7e141977-92b7-4aa0-bb5d-e06136c9bb29n%2540googlegroups.com%3Futm_medium%3Demail%26utm_source%3Dfooter&data=04%7C01%7Cacogoluegnes%40vmware.com%7Ce7583bc87158489e11fd08d8db8665fd%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C1%7C0%7C637500717596211732%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=StNVDRINGdkopsSgGrqc%2BqV6AnsfC10DHIrEcJwL4QQ%3D&reserved=0>.

Avinash Reddy

unread,
Mar 4, 2021, 12:42:02 AM3/4/21
to rabbitmq-users
Hi there,

Here is the matrix of RabbitMQ and Erlang version, and using the Java Client version 5.9.0 showing TLS connection status.

RabbitMQ version          erlang version           Java  8u252 (TLS via Java Client on 5671)     Java  8u281 (TLS via Java Client on 5671)

3.6.14                                     20                                        success                                                            success

3.6.14                                     20                                        success                                                            success

3.6.14                                     20                                        success                                                            success

3.6.14                                     20                                       success                                                            success

3.7.16                                     22                                       success                                                            Failed

3.7.16                                     22                                       success                                                            Failed

3.8.3                                      22.0                                    success                                                             Failed


Java options used are: -Djavax.net.debug=all -Djavax.net.debug=ssl:handshake:verbose -Djdk.tls.client.protocols="TLSv1,TLSv1.1,TLSv1.2" -Djdk.tls.namedGroups="secp256r1, secp384r1, secp521r1, sect283k1, sect283r1, sect409k1, sect409r1, sect571k1, sect571r1, secp256k1" -Djsse.enableFFDHE=false -Djavax.net.ssl.sessionCacheSize=0

Please let us know if there is any better way to debug the handshake failure?
Also any ideas on how to fix it.?

Here is the output for RabbitMQ server 3.8.3 version
javax.net.ssl|FINE|0F|restartedMain|2021-03-03 22:12:53.235 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 for TLS11
javax.net.ssl|FINE|0F|restartedMain|2021-03-03 22:12:53.236 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 for TLS11
javax.net.ssl|FINE|0F|restartedMain|2021-03-03 22:12:53.236 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 for TLS11
javax.net.ssl|FINE|0F|restartedMain|2021-03-03 22:12:53.236 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 for TLS11
javax.net.ssl|FINE|0F|restartedMain|2021-03-03 22:12:53.236 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_RSA_WITH_AES_256_GCM_SHA384 for TLS11
javax.net.ssl|FINE|0F|restartedMain|2021-03-03 22:12:53.236 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_RSA_WITH_AES_256_CBC_SHA256 for TLS11
javax.net.ssl|FINE|0F|restartedMain|2021-03-03 22:12:53.236 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_RSA_WITH_AES_128_CBC_SHA256 for TLS11
javax.net.ssl|FINE|0F|restartedMain|2021-03-03 22:12:53.237 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 for TLS10
javax.net.ssl|FINE|0F|restartedMain|2021-03-03 22:12:53.237 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 for TLS10
javax.net.ssl|FINE|0F|restartedMain|2021-03-03 22:12:53.237 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 for TLS10
javax.net.ssl|FINE|0F|restartedMain|2021-03-03 22:12:53.237 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 for TLS10
javax.net.ssl|FINE|0F|restartedMain|2021-03-03 22:12:53.237 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_RSA_WITH_AES_256_GCM_SHA384 for TLS10
javax.net.ssl|FINE|0F|restartedMain|2021-03-03 22:12:53.237 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_RSA_WITH_AES_256_CBC_SHA256 for TLS10
javax.net.ssl|FINE|0F|restartedMain|2021-03-03 22:12:53.237 MST|HandshakeContext.java:297|Ignore unsupported cipher suite: TLS_RSA_WITH_AES_128_CBC_SHA256 for TLS10
javax.net.ssl|FINE|0F|restartedMain|2021-03-03 22:12:53.250 MST|SSLExtensions.java:259|Ignore, context unavailable extension: status_request
javax.net.ssl|WARNING|0F|restartedMain|2021-03-03 22:12:53.253 MST|SignatureScheme.java:297|Signature algorithm, ed25519, is not supported by the underlying providers
javax.net.ssl|WARNING|0F|restartedMain|2021-03-03 22:12:53.253 MST|SignatureScheme.java:297|Signature algorithm, ed448, is not supported by the underlying providers
javax.net.ssl|ALL|0F|restartedMain|2021-03-03 22:12:53.256 MST|SignatureScheme.java:373|Ignore unsupported signature scheme: ed25519
javax.net.ssl|ALL|0F|restartedMain|2021-03-03 22:12:53.256 MST|SignatureScheme.java:373|Ignore unsupported signature scheme: ed448
javax.net.ssl|ALL|0F|restartedMain|2021-03-03 22:12:53.257 MST|SignatureScheme.java:393|Ignore disabled signature scheme: rsa_md5
javax.net.ssl|INFO|0F|restartedMain|2021-03-03 22:12:53.257 MST|AlpnExtension.java:161|No available application protocols
javax.net.ssl|FINE|0F|restartedMain|2021-03-03 22:12:53.257 MST|SSLExtensions.java:259|Ignore, context unavailable extension: application_layer_protocol_negotiation
javax.net.ssl|FINE|0F|restartedMain|2021-03-03 22:12:53.258 MST|SSLExtensions.java:259|Ignore, context unavailable extension: status_request_v2
javax.net.ssl|FINE|0F|restartedMain|2021-03-03 22:12:53.260 MST|ClientHello.java:567|Produced ClientHello handshake message (
"ClientHello": {
  "client version"      : "TLSv1.2",
  "random"              : "D8 CF 48 A3 89 B9 61 1A 4C D5 8F 83 35 B6 28 28 69 4A C8 73 5F E3 82 DA 1C AC AD 4A 5E 71 B4 AF",
  "session id"          : "",
  "cipher suites"       : "[TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384(0xC030), TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384(0xC028), TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256(0xC02F), TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256(0xC027), TLS_RSA_WITH_AES_256_GCM_SHA384(0x009D), TLS_RSA_WITH_AES_256_CBC_SHA256(0x003D), TLS_RSA_WITH_AES_128_CBC_SHA256(0x003C), TLS_RSA_WITH_AES_128_CBC_SHA(0x002F), TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384(0xC02C), TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384(0xC024), TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256(0xC02B), TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256(0xC023)]",
  "compression methods" : "00",
  "extensions"          : [
    "server_name (0)": {
      type=host_name (0), value=rabbitmqlatest.local.com
    },
    "supported_groups (10)": {
      "versions": [secp256r1, secp384r1, secp521r1, sect283k1, sect283r1, sect409k1, sect409r1, sect571k1, sect571r1, secp256k1]
    },
    "ec_point_formats (11)": {
      "formats": [uncompressed]
    },
    "signature_algorithms (13)": {
      "signature schemes": [ecdsa_secp256r1_sha256, ecdsa_secp384r1_sha384, ecdsa_secp521r1_sha512, rsa_pss_rsae_sha256, rsa_pss_rsae_sha384, rsa_pss_rsae_sha512, rsa_pss_pss_sha256, rsa_pss_pss_sha384, rsa_pss_pss_sha512, rsa_pkcs1_sha256, rsa_pkcs1_sha384, rsa_pkcs1_sha512, dsa_sha256, ecdsa_sha224, rsa_sha224, dsa_sha224, ecdsa_sha1, rsa_pkcs1_sha1, dsa_sha1]
    },
    "signature_algorithms_cert (50)": {
      "signature schemes": [ecdsa_secp256r1_sha256, ecdsa_secp384r1_sha384, ecdsa_secp521r1_sha512, rsa_pss_rsae_sha256, rsa_pss_rsae_sha384, rsa_pss_rsae_sha512, rsa_pss_pss_sha256, rsa_pss_pss_sha384, rsa_pss_pss_sha512, rsa_pkcs1_sha256, rsa_pkcs1_sha384, rsa_pkcs1_sha512, dsa_sha256, ecdsa_sha224, rsa_sha224, dsa_sha224, ecdsa_sha1, rsa_pkcs1_sha1, dsa_sha1]
    },
    "extended_master_secret (23)": {
      <empty>
    },
    "supported_versions (43)": {
      "versions": [TLSv1.2, TLSv1.1, TLSv1]
    },
    "renegotiation_info (65,281)": {
      "renegotiated connection": [<no renegotiated connection>]
    }
  ]
}
)
javax.net.ssl|FINE|0F|restartedMain|2021-03-03 22:12:53.316 MST|Alert.java:238|Received alert message (
"Alert": {
  "level"      : "fatal",
  "description": "handshake_failure"
}
)
javax.net.ssl|SEVERE|0F|restartedMain|2021-03-03 22:12:53.318 MST|TransportContext.java:345|Fatal (HANDSHAKE_FAILURE): Received fatal alert: handshake_failure (

Arnaud Cogoluegnes

unread,
Mar 4, 2021, 3:47:27 AM3/4/21
to rabbitm...@googlegroups.com
This matrix does not help me to help you, because the following elements are missing: exact version of Erlang, server configuration (including certificates, keys), client configuration, and code samples to reproduce.

It would be really helpful if you could provide a standalone project to reproduce.

Meanwhile, I re-used the reproduce.sh script I mentioned earlier [1]. It uses only Erlang and Java (not RabbitMQ broker or client involved), on Erlang 22.3.4.5 (mentioned by you in a previous post) and on Java 8u281. This scripts starts an Erlang server:

cd /tmp
mkdir erlang-22-java-11-tls-handshare-failure
cd erlang-22-java-11-tls-handshare-failure
git clone https://github.com/michaelklishin/tls-gen.git
make -C tls-gen/basic
echo '-module(tls_cipher_negotiation).
-export([start/0]).

start() ->
ssl:start(),
{ok, ListenSocket} = ssl:listen(9999, [
{cacertfile, "/tmp/erlang-22-java-11-tls-handshare-failure/tls-gen/basic/result/ca_certificate.pem"},
{certfile, "/tmp/erlang-22-java-11-tls-handshare-failure/tls-gen/basic/result/server_certificate.pem"},
{keyfile, "/tmp/erlang-22-java-11-tls-handshare-failure/tls-gen/basic/result/server_key.pem"},
{verify, verify_peer},
{fail_if_no_peer_cert, false}
]),
{ok, TLSTransportSocket} = ssl:transport_accept(ListenSocket),
{ok, _} = ssl:handshake(TLSTransportSocket),
ok.' > /tmp/erlang-22-java-11-tls-handshare-failure/tls_cipher_negotiation.erl

erlc tls_cipher_negotiation.erl
erl -noshell -s tls_cipher_negotiation start -s init stop


Then a Java client tries to connect:

echo 'import javax.net.ssl.*;
import java.net.Socket;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;

public class TlsClient {

public static void main(String[] args) throws Exception {
System.setProperty("javax.net.debug", "ssl:handshake:verbose");
SSLContext sslContext = SSLContext.getInstance("TLSv1.2");
sslContext.init(null, new TrustManager[]{
new X509TrustManager() {
@Override
public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException {
}

@Override
public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException {
}

@Override
public X509Certificate[] getAcceptedIssuers() {
return new X509Certificate[0];
}
}
}, null);
SSLSocketFactory ssf = sslContext.getSocketFactory();
Socket s = ssf.createSocket("127.0.0.1", 9999);
((SSLSocket) s).getSession();
}

}' > TlsClient.java
javac TlsClient.java
java TlsClient


The connection succeeded, so I could not reproduce the handshake failure.

If you don't want to provide a standalone project to reproduce the error you're hitting, you can try this script on the different environments you have and see if the error comes up.

[1] https://bugs.erlang.org/browse/ERL-973

________________________________________
From: rabbitm...@googlegroups.com <rabbitm...@googlegroups.com> on behalf of Avinash Reddy <avinas...@gmail.com>

Sent: Thursday, March 4, 2021 6:42 AM
To: rabbitmq-users
Subject: Re: [Suspected Spam] Re: [rabbitmq-users] Java upgrade with TLS 1.3 support causes rabbitmq TLS handshake failures

M K

unread,
Mar 4, 2021, 4:59:27 AM3/4/21
to rabbitmq-users
There is now a TLS 1.3-specific example in the docs [1] which passes a widely
used OpenSSL-based suite [2] which you can run against a node.

Erlang support for TLS 1.3 is new and can only be considered mature as of 23.x.
Which is why previously RabbitMQ marked it as experimental internally and excluded
from the list of enabled protocols.

RabbitMQ itself does not implement TLS so we cannot help with anything except narrowing things
down to be reported to the Erlang maintainers.

Lemons, Terry

unread,
Mar 4, 2021, 9:28:10 AM3/4/21
to rabbitm...@googlegroups.com

Hi

 

Thank you both for this help.

 

RabbitMQ itself does not implement TLS so we cannot help with anything except narrowing things down to be reported to the Erlang maintainers.” Yes, good point; if there is an erlang mailing list/forum, we should probably move this conversation to there.

 

We’re guessing that the use of openjdk beyond 8u272 (which introduces TLS 1.3 support) on the client end is causing erlang versions before 23 on the server end to generate TLS handshake failures.

 

Actually, we are not interested in using TLS 1.3 (at least, not at this time). So, on the server end, with erlang versions before 23, is there a way to disable use of TLS 1.3?

 

Thanks

tl

 

From: rabbitm...@googlegroups.com <rabbitm...@googlegroups.com> On Behalf Of M K
Sent: Thursday, March 4, 2021 4:59 AM
To: rabbitmq-users
Subject: Re: [Suspected Spam] Re: [rabbitmq-users] Java upgrade with TLS 1.3 support causes rabbitmq TLS handshake failures

 

[EXTERNAL EMAIL]

--

You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/58f317cc-4d2c-47ca-b64a-88d49bec872an%40googlegroups.com.

Arnaud Cogoluegnes

unread,
Mar 4, 2021, 10:31:07 AM3/4/21
to rabbitm...@googlegroups.com
There is the ssl_options.versions configuration entry [1] to limit the version of TLS used, see the examples in the documentation.

This may indeed be an Erlang/Java incompatibility, but I could not reproduce it. Again, you should try the script I mentioned earlier on your environments to see if you can reproduce. This script is standalone and does not imply RabbitMQ, just Erlang and Java.

[1] https://www.rabbitmq.com/ssl.html#tls-versions-why-limit

________________________________________
From: rabbitm...@googlegroups.com <rabbitm...@googlegroups.com> on behalf of Lemons, Terry <Terry....@dell.com>
Sent: Thursday, March 4, 2021 3:27 PM
To: rabbitm...@googlegroups.com
Subject: RE: [Suspected Spam] Re: [rabbitmq-users] Java upgrade with TLS 1.3 support causes rabbitmq TLS handshake failures

Hi

Thank you both for this help.

“RabbitMQ itself does not implement TLS so we cannot help with anything except narrowing things down to be reported to the Erlang maintainers.” Yes, good point; if there is an erlang mailing list/forum, we should probably move this conversation to there.

We’re guessing that the use of openjdk beyond 8u272 (which introduces TLS 1.3 support) on the client end is causing erlang versions before 23 on the server end to generate TLS handshake failures.

Actually, we are not interested in using TLS 1.3 (at least, not at this time). So, on the server end, with erlang versions before 23, is there a way to disable use of TLS 1.3?

Thanks
tl

From: rabbitm...@googlegroups.com <rabbitm...@googlegroups.com> On Behalf Of M K
Sent: Thursday, March 4, 2021 4:59 AM
To: rabbitmq-users
Subject: Re: [Suspected Spam] Re: [rabbitmq-users] Java upgrade with TLS 1.3 support causes rabbitmq TLS handshake failures


[EXTERNAL EMAIL]
There is now a TLS 1.3-specific example in the docs [1] which passes a widely
used OpenSSL-based suite [2] which you can run against a node.

Erlang support for TLS 1.3 is new and can only be considered mature as of 23.x.
Which is why previously RabbitMQ marked it as experimental internally and excluded
from the list of enabled protocols.

RabbitMQ itself does not implement TLS so we cannot help with anything except narrowing things
down to be reported to the Erlang maintainers.

1. https://www.rabbitmq.com/ssl.html#tls1.3<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.rabbitmq.com%2Fssl.html%23tls1.3&data=04%7C01%7Cacogoluegnes%40vmware.com%7C5af6b8d4f9244713500608d8df19bc72%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C1%7C0%7C637504648923609125%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=nvZ4nq%2F7OyBAxHilM5chrbk51aTaRJHOlioRRPvOHns%3D&reserved=0>
2. https://www.rabbitmq.com/ssl.html#tls-evaluation-tools<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.rabbitmq.com%2Fssl.html%23tls-evaluation-tools&data=04%7C01%7Cacogoluegnes%40vmware.com%7C5af6b8d4f9244713500608d8df19bc72%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C1%7C0%7C637504648923619119%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2BijR1CR6M6hVV%2FhrKvKCrmSt3diyn%2BkAABoSq3N20rM%3D&reserved=0>
On Thursday, March 4, 2021 at 11:47:27 AM UTC+3 Arnaud Cogoluègnes wrote:
This matrix does not help me to help you, because the following elements are missing: exact version of Erlang, server configuration (including certificates, keys), client configuration, and code samples to reproduce.

It would be really helpful if you could provide a standalone project to reproduce.

Meanwhile, I re-used the reproduce.sh script I mentioned earlier [1]. It uses only Erlang and Java (not RabbitMQ broker or client involved), on Erlang 22.3.4.5 (mentioned by you in a previous post) and on Java 8u281. This scripts starts an Erlang server:

cd /tmp
mkdir erlang-22-java-11-tls-handshare-failure
cd erlang-22-java-11-tls-handshare-failure
git clone https://github.com/michaelklishin/tls-gen.git<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmichaelklishin%2Ftls-gen.git&data=04%7C01%7Cacogoluegnes%40vmware.com%7C5af6b8d4f9244713500608d8df19bc72%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C1%7C0%7C637504648923619119%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=4WpvakMYYgcv3zVjhUfqiEuRNzwdOZce7wHBzSTepNw%3D&reserved=0>
[1] https://bugs.erlang.org/browse/ERL-973<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.erlang.org%2Fbrowse%2FERL-973&data=04%7C01%7Cacogoluegnes%40vmware.com%7C5af6b8d4f9244713500608d8df19bc72%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C1%7C0%7C637504648923629104%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Blo0qgRdY8S%2Bedpq1M9sPDAALoU8%2B5e07OoYz2gaa8o%3D&reserved=0>
type=host_name (0), value=rabbitmqlatest.local.com<https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Frabbitmqlatest.local.com%2F&data=04%7C01%7Cacogoluegnes%40vmware.com%7C5af6b8d4f9244713500608d8df19bc72%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C1%7C0%7C637504648923629104%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=bLXPR%2B0Ld2u7NlDIWi4uBNm6V9qQrCKPDnLtCRgff4g%3D&reserved=0>
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com<mailto:rabbitmq-user...@googlegroups.com>.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/58f317cc-4d2c-47ca-b64a-88d49bec872an%40googlegroups.com<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Frabbitmq-users%2F58f317cc-4d2c-47ca-b64a-88d49bec872an%2540googlegroups.com%3Futm_medium%3Demail%26utm_source%3Dfooter&data=04%7C01%7Cacogoluegnes%40vmware.com%7C5af6b8d4f9244713500608d8df19bc72%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C1%7C0%7C637504648923639101%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RO0%2BJZAZs2IMtM1Hccf6zbJ9Cjx4D%2BSV8xClN9gcm9Y%3D&reserved=0>.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com<mailto:rabbitmq-user...@googlegroups.com>.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/SJ0PR19MB4493AF6FD09DA6A33C15A5EAE4979%40SJ0PR19MB4493.namprd19.prod.outlook.com<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Frabbitmq-users%2FSJ0PR19MB4493AF6FD09DA6A33C15A5EAE4979%2540SJ0PR19MB4493.namprd19.prod.outlook.com%3Futm_medium%3Demail%26utm_source%3Dfooter&data=04%7C01%7Cacogoluegnes%40vmware.com%7C5af6b8d4f9244713500608d8df19bc72%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C1%7C0%7C637504648923639101%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=xVIokaGIQthgtFgWgCZVZOmA%2BJ4LvRTcBiw6IP7jWaA%3D&reserved=0>.
Reply all
Reply to author
Forward
0 new messages