SSL Handshake failure no Logs generated

455 views
Skip to first unread message

Nicolas Valdiviezo

unread,
Jun 13, 2019, 4:04:33 PM6/13/19
to rabbitmq-users
Sorry my lack of rabbitMQ lingo, but I am fairly new to RabbitMQ community. For the past month, I have been using RabbitMQ Server v3.7.14 on Erlang/OTP 21 running on Windows Server 2016.
Since yesterday I have been trying to set up the TLS support, I got my certificates from DigiCert (I skipped doing the self-signing ones) but other than that I filled the TLS Support guide to the letter.

The issue is that when I try to connect with a client to the server, the connection failed to create. From Wireshark capture I can tell that TCP accepts the connection, the client says hello and the server responds to Client Hello TLSv1.2 with RST ACK. (figure1) So I went to the TLS Troubleshooting guide:
  1. When using the .bat tool from RabbitMQ status or diagnostics listeners the ports show as listening.
  2. netsta -ab shows that erl.exe is listening on 5671 and 5672; however netstat -ano shows that the correct pid listens on the ports (5672 and 5671.
  3. I tested the certs using the openssl tool for windows (v1.1.1). On the WinServer 2016 I did: openssl s_server <domainInCN>:8433 -verify 8 -prexit -debug and connected from the other machine using the client call, and the connection is established, I verified that Wireshark recorded the traffic and that the handshake completed using tlsv1.2. (figure2) using the cert documents that were in my config file for rabbitMQ.
  4. Then I tested to connect from openssl s_client to port 5671. But again the handshake fails after Client Hello.
  5. telnet <domainInCN> 5672 does work and Wireshark sees the message sent as AMQP protocol (as it should)
I have seen other people have this issue, but what prompted me to post a new question is that there is no record on the log file that there was a connection attempt. (NOTHING!) I reset the server, again nothing.
I moved the certs from their original location c:/certs/ to %RABBIT_HOME%/etc/certs/ thinking that maybe rabbitMQ could not access the c root folders; but again same result as above. 

Any recommendations on how to troubleshoot that is NOT the TLS troubleshoot guide would be super helpful!

Best,

Nicolas




config.PNG
netstat_ano.PNG
opensslRabbitCapture.PNG
opensslTest.PNG
status.PNG

Luke Bakken

unread,
Jun 14, 2019, 1:05:12 AM6/14/19
to rabbitmq-users
Hi Nicolas,

This is an issue I haven't seen before. Start by simplifying your RabbitMQ configuration and, for now do not limit to TLS 1.2 and set verify to verify_none.

Then, read this doc to enable all cipher suites:


Please see this message for how to list all available cipher suites. You will use this list in your configuration file:


Then, please re-try the openssl test to port 5671.

Luke Bakken

unread,
Jun 14, 2019, 10:00:35 AM6/14/19
to rabbitmq-users
Hi again Nicholas -

Please see this discussion as that user is having the same issue -


If it's possible to try the latest Windows version of Erlang that would be a good place to start.

Thanks -
Luke

Nicolas Valdiviezo

unread,
Jun 14, 2019, 10:48:14 AM6/14/19
to rabbitmq-users
Hello Luke,

Thanks for the reply! So I did the changes you suggested: Kept the config file to a minimum and added all the cipher suites that appeared using the rabbitmqctl eval "ssl:cipher_suites(all)."  Dame issue, I cannot see logs for the attempt to connect and the handshake still fails. I am attaching images showing the results as well as a wireshark capture.
Again, I am very thankful for your help!

Best,
Nicolas
wiresharkCaptureFailedSSLHandshake.pcapng
restartListenersVerify.PNG
cipherSuitesAll.PNG
configNew.PNG
clientResultOpensslTest.PNG
logsreboot.PNG

Nicolas Valdiviezo

unread,
Jun 14, 2019, 10:57:17 AM6/14/19
to rabbitmq-users
Very similar. However, what is concerning to me is that rabbitmq is not logging the attempt to connect. Maybe because the handshake fails. I guess the main goal for me right now is to identify what part of the communication is breaking, haha. I checked my Erlang version and it is 21.3  (located on C:\Program Files\erl10.3\releases\21\OTP_VERSION)

Luke Bakken

unread,
Jun 14, 2019, 11:06:13 AM6/14/19
to rabbitmq-users
Hi Nicolas -

There's no log since the connection never proceeds to the point where RabbitMQ begins to handle it. The TLS code within the Erlang VM kills the connection before then.

I suspect you're running into a TLS issue that was fixed in recent 21.3 / 22 versions of Erlang. Unfortunately the Erlang team doesn't provide patch releases in binary form for Windows. I'll ask to see what options you have.

Thanks -
Luke

Nicolas Valdiviezo

unread,
Jun 14, 2019, 11:25:47 AM6/14/19
to rabbitmq-users
Then, should it appeal on the Erlang logs? I will check just in case.

Thank you, Luke.
Best,
Nicolas Valdiviezo

Nicolas Valdiviezo

unread,
Jun 19, 2019, 2:49:45 PM6/19/19
to rabbitmq-users
Hello Luke,

Is there a way I can see where the runtime is failing? I cannot find erlang logs or anything like that. The other alternative is uninstalling my current instance of rabbitMQ and Erlang/OPT and starting fresh with rabbitMQ 3.7.15 and Erlang 21.3 for Windows 64bit, would that give me the suspected patch that I need? Do I need to uninstall RabbitMQ before I uninstall Erlang?

Thanks,
Nico

Luke Bakken

unread,
Jun 24, 2019, 4:20:01 PM6/24/19
to rabbitmq-users
Hi Nicolas,

You can see from your openssl test that the TLS connection isn't working at all.

Could you please try an older Erlang release for Windows, like 20.3?


I will follow up if we're able to get patch release builds for Windows.

Thanks,
Luke
Reply all
Reply to author
Forward
0 new messages