Issue with TLS for rabbitmq management UI on Windows 2016 server

[pak 149]

HI there

I'm running rabbitmq_server-3.7.16 with Erlang/OTP 21 [erts-10.3] [64-bit) on windows 2016

I setup TLS for management UI:

rabbitmq.conf:

management.ssl.port       = 15671

management.ssl.cacertfile = E:/Program Files/RabbitMQ Server/certs/rootca.pem

management.ssl.certfile   = E:/Program Files/RabbitMQ Server/certs/server.pem

management.ssl.keyfile    = E:/Program Files/RabbitMQ Server/certs/serverkey.pem

My log file shows that the listener is up

...Management plugin: HTTPS listener started on port 15671

i can access the management UI on HTTP port, overview page shows:

Listening ports

Protocol	Bound to	Port
amqp	0.0.0.0	5672
amqp	::	5672
clustering	::	25672
http	0.0.0.0	15672
http	::	15672
https	0.0.0.0	15671
https	::	15671

however i am unable to connect to https management UI - browser returns this error:

This page can’t be displayed

Turn on TLS 1.0, TLS 1.1, and TLS 1.2 in Advanced settings and try connecting to https://localhost:15671 again.

If this error persists, it is possible that this site uses an unsupported protocol or cipher suite such as RC4 (link for the details), which is not considered secure. Please contact your site administrator.

i followed all the SSL troubleshooting steps, i'm able to test with openssl when running local server 

>openssl s_server -accept 8443 -cert server.pem -key serverkey.pem -CAfile rootca.pem

here is the output of the validation 

>openssl s_client -connect localhost:8443 -CAfile rootca.pem  -verify 8 -verify_hostname server_name"

verify depth is 8

CONNECTED(00000154)

---truncated here---

---

Certificate chain

---truncated here---

---

Server certificate

-----BEGIN CERTIFICATE-----

---truncated here---

-----END CERTIFICATE-----

subject=/C=CA/ST=Quebec/L=Montreal/O=CN/CN=server

issuer=/DC=com/DC=Domain/CN=IssuingCA

---

No client certificate CA names sent

Peer signing digest: SHA512

Server Temp Key: X25519, 253 bits

---

SSL handshake has read 6092 bytes and written 269 bytes

Verification: OK

Verified peername: server_name

---

New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384

Server public key is 2048 bit

Secure Renegotiation IS supported

Compression: NONE

Expansion: NONE

No ALPN negotiated

SSL-Session:

    Protocol  : TLSv1.2

    Cipher    : ECDHE-RSA-AES256-GCM-SHA384

    Session-ID: 58166A3E0F08B8535AED647BF089B9114A109080D8A90D0780403B1D37F8821E

    Session-ID-ctx:

    Master-Key: 75D8D2D5A3B86C16B8866E3337410D5035EFEC72C2900EAF5E616B1B8920E360EDE2F9E6E7A01C99E3C8552C1DF0C5F5

    PSK identity: None

    PSK identity hint: None

    SRP username: None

    TLS session ticket lifetime hint: 7200 (seconds)

    TLS session ticket:

---truncated here---

    Start Time: 1583952425

    Timeout   : 7200 (sec)

    Verify return code: 0 (ok)

    Extended master secret: yes

---

When running same validation on port 15671 i get this result:

>openssl s_client -connect 127.0.0.1:15671

 

CONNECTED(000000E8)

write:errno=0

---

no peer certificate available

---

No client certificate CA names sent

---

SSL handshake has read 0 bytes and written 176 bytes

Verification: OK

---

New, (NONE), Cipher is (NONE)

Secure Renegotiation IS NOT supported

Compression: NONE

Expansion: NONE

No ALPN negotiated

SSL-Session:

    Protocol  : TLSv1.2

    Cipher    : 0000

    Session-ID:

    Session-ID-ctx:

    Master-Key:

    PSK identity: None

    PSK identity hint: None

    SRP username: None

    Start Time: 1583952874

    Timeout   : 7200 (sec)

    Verify return code: 0 (ok)

    Extended master secret: no

---

Are there any know issues in setting up Management UI with TLS on Windows servers?

Thanks for your help

[Luke Bakken]

Hello,

You should be using a more recent version of Erlang, preferably 22.2 (the latest). RabbitMQ 3.7.16 is also old, but that version won't affect TLS.

You should use openssl s_client to connect to port 15671. It will show the TLS handshake output. Please attach the output you see.

Thanks,

Luke

[pak 149]

Hi Luke

here is the log entry that states that the port is Up:

2020-03-12 07:47:39.933 [info] <0.587.0> Management plugin: HTTP (non-TLS) listener started on port 15672
2020-03-12 07:47:39.964 [info] <0.587.0> Management plugin: HTTPS listener started on port 15671

here is the output. Its like the port is up but no certificate is attached

>openssl s_client -connect 127.0.0.1:15671

 

verify depth is 8
CONNECTED(00000140)

write:errno=0
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 176 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000
    Session-ID:
    Session-ID-ctx:
    Master-Key:
    PSK identity: None
    PSK identity hint: None
    SRP username: None

    Start Time: 1584013682

    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
---

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/3f18c61b-91c0-400f-af32-df6f061f53b7%40googlegroups.com.

[Luke Bakken]

Hello,

Please take the time to do the troubleshooting steps here: https://www.rabbitmq.com/troubleshooting-ssl.html

Perhaps your RabbitMQ configuration has an error in a path to a certificate. I suggest using a location other than E:\Program Files. Try the ProgramData directory, it will be on your system drive and will be readable / writable by the RabbitMQ service.

We know that TLS works on Windows, so there is a configuration error or an error with your certificates.

Thanks,

Luke

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.

[pak 149]

Hi Luke

 

yes i did follow all the troubleshooting steps as described in https://www.rabbitmq.com/troubleshooting-ssl.html

Its because it is still failing that i opened this post :-)

I moved the certificates to c:\programdata, but still have the same issue

i also suspect the certificate to be problematic, however running openssl tests does not show any issue.

what bothers me too is that the debug log does not show any error, it actually shows that the HTTPS is loaded properly:

2020-03-12 10:53:42.013 [debug] <0.545.0> Starting HTTP[S] listener with transport ranch_ssl, options [{cacertfile,"C:/ProgramData/Certs/RootCA.pem"},{certfile,"C:/ProgramData/Certs/rabbitmqdev.cert.pem"},{ciphers,["ECDHE-ECDSA-AES256-GCM-SHA384","ECDHE-RSA-AES256-GCM-SHA384","ECDHE-ECDSA-AES256-SHA384","ECDHE-RSA-AES256-SHA384","ECDH-ECDSA-AES256-GCM-SHA384","ECDH-RSA-AES256-GCM-SHA384","ECDH-ECDSA-AES256-SHA384","ECDH-RSA-AES256-SHA384","DHE-RSA-AES256-GCM-SHA384"]},{client_renegotiation,false},{depth,2},{honor_cipher_order,true},{honor_ecc_order,true},{keyfile,"C:/ProgramData/Certs/rabbitmqdev.key.pem"},{password,"*****"},{port,15671},{secure_renegotiate,true},{versions,['tlsv1.2']},{versions,['tlsv1.2','tlsv1.1',tlsv1]}] and protocol options #{sendfile => false}, stream handlers [rabbit_cowboy_stream_h,cowboy_compress_h,cowboy_stream_h]

2020-03-12 10:53:42.029 [info] <0.593.0> Management plugin: HTTPS listener started on port 15671

Could it be the certificate key usage that is not correct?

this is what i have:

key usage: Digital Signature, Key Encipherment (a0)

Enhanced key usage:

Smart Card Logon (1.3.6.1.4.1.311.20.2.2)

Server Authentication (1.3.6.1.5.5.7.3.1)

Client Authentication (1.3.6.1.5.5.7.3.2)

This type of certificate is our standard for all HTTPS setups

Thanks

[Luke Bakken]

Hello,

Unless someone says otherwise I can only assume they haven't done sufficient troubleshooting based on our procedure.

The output provided doesn't match the configuration file you shared earlier. Specifically, you are setting ciphers and tls versions, which I don't see in your first message. I also see a password and depth setting. Are you using an intermediate CA?

In order for us to help, you have to provide the exact configuration you're using. Please attach your complete RabbitMQ configuration file to your next message.

I suggest your next step is to remove your settings for TLS versions and ciphers.

Thanks,

Luke

[pak 149]

Hi Luke

Here are two config files that i have tried enabling SSL with. 

Yes i'm using an intermediate CA, the RooCA.pem certificate file has both issuing and root CA (concatenated as described in https://www.rabbitmq.com/ssl.html#peer-verification-trusted-certificates)

I do confirm that I  followed all the step sin the TLS troubleshooting guide. 

attached is the output of openssl "s_client connect" test

The diagnostic command all returned correct data:

>rabbitmq-diagnostics listeners

Asking node rabbit@mtlh-polwb01ad to report its protocol listeners ...

Interface: [::], port: 25672, protocol: clustering, purpose: inter-node and CLI tool communication

Interface: [::], port: 5672, protocol: amqp, purpose: AMQP 0-9-1 and AMQP 1.0

Interface: 0.0.0.0, port: 5672, protocol: amqp, purpose: AMQP 0-9-1 and AMQP 1.0

Interface: [::], port: 15672, protocol: http, purpose: HTTP API

Interface: 0.0.0.0, port: 15672, protocol: http, purpose: HTTP API

Interface: [::], port: 15671, protocol: https, purpose: HTTP API over TLS (HTTPS)

Interface: 0.0.0.0, port: 15671, protocol: https, purpose: HTTP API over TLS (HTTPS)

>rabbitmq-diagnostics.bat --silent tls_versions

tlsv1.2

tlsv1.1

tlsv1

sslv3

Thanks

Patrick

[Luke Bakken]

Hello,

Please use your "lean" configuration and test with a depth of 99.

If that doesn't work, re-test with certificates generated by our test project's "basic" profile, which does not use intermediates: https://github.com/michaelklishin/tls-gen

I can't tell by the output provided, but you may also have to enable all ciphers - https://www.rabbitmq.com/ssl.html#cipher-suites

Please run rabbitmq-diagnostics help cipher_suites to see the argument to list all ciphers (I think it's --all). You would then use that list in your configuration file.

Thanks,

Luke

[pak 149]

Hi Luke

Any change you have test certificate files handy that you could attach ?. I'm on windows and don't think i can run that "make" command

I tried with depth 99 and enabling all the ciphers, did not work either

Thanks

[Luke Bakken]

Hello,

I can generate the certs but they are for a specific host name.

On Windows, you can install msys2, then install the "make" and "python" packages and it will work.

Thanks,

Luke

[pak 149]

Hi Luke

we are having talks about upgrading to newer versions (3.8) in the near future.

I will wait until we have that new version in place to continue troubleshooting this, hopefully it will work with the new versions

thanks for your help

Patrick

On Wednesday, 11 March 2020 14:59:48 UTC-4, pak 149 wrote:

[Luke Bakken]

Hi Patrick,

Upgrading RabbitMQ will not solve this issue. Upgrading Erlang to 22.2 might fix this if there is something fixed in Erlang with regard to intermediate certs.

If you can share your certificates, or equivalents, perhaps we can help diagnose.

The tls-gen project has examples that use intermediate certificates. We have tested RabbitMQ with those and know they work.

Thanks,

Luke