TLS Configuration

[Richard]

Hi

I have been struggling to get TLS working. I'm using the latest docker image, and have used tls-gen to generate the certificates/keys. 

It seems to start ok, but I just can't connect to it. The folder containing the certificates is mounted on a volume (to /tmp/tls) and it seems happy with them.

Any help really appreciated. I have tried to provide as much diagnostic information below as I can but if more is needed, I can provide it. 

Versions: RabbitMQ 3.7.28 on Erlang 22.3.4.7

The listener seems ok:

started TLS (SSL) listener on [::]:5671

If I try to check the hostname using 

openssl x509 -noout -subject -in server_certificate.pem, 

I get this: subject=CN = <my-hostname>, O = server 

If I try to connect using

openssl s_client -connect <my-hostname>:5671 -cert client_certificate.pem -key client_key.pem -CAfile ca_certificate.pem

I get 

Enter pass phrase for client_key.pem:

CONNECTED(00000005)

write:errno=0

---

no peer certificate available

---

No client certificate CA names sent

---

SSL handshake has read 0 bytes and written 310 bytes

Verification: OK

---

New, (NONE), Cipher is (NONE)

Secure Renegotiation IS NOT supported

Compression: NONE

Expansion: NONE

No ALPN negotiated

Early data was not sent

Verify return code: 0 (ok)

---

If I follow the troubleshooting guide and run these two commands in different windows

1) openssl s_server -accept 5671 -cert server_certificate.pem -key server_key.pem -CAfile ca_certificate.pem

2) openssl s_client -connect <my-hostname>:5671 -cert client_certificate.pem -key client_key.pem -CAfile ca_certificate.pem -verify 8 -verify_hostname <my-hostname>

I get this

#

# Output of connecting window

#

verify depth is 8

Enter pass phrase for client_key.pem:

CONNECTED(00000005)

depth=1 CN = TLSGenSelfSignedtRootCA, L = $$$$

verify return:1

depth=0 CN = <my-hostname>, O = server

verify return:1

---

Certificate chain

 0 s:CN = <my-hostname>, O = server

   i:CN = TLSGenSelfSignedtRootCA, L = $$$$

 1 s:CN = TLSGenSelfSignedtRootCA, L = $$$$

   i:CN = TLSGenSelfSignedtRootCA, L = $$$$

---

Server certificate

-----BEGIN CERTIFICATE-----

MIIDazCCAlOgAwIBAgIBATANBgkqhkiG9w0BAQsFADAxMSAwHgYDVQQDDBdUTFNH

ZW5TZWxmU2lnbmVkdFJvb3RDQTENMAsGA1UEBwwEJCQkJDAeFw0yMDEyMTgxMjA1

MTNaFw0zMDEyMTYxMjA1MTNaMCQxETAPBgNVBAMMCFZJUi1MMDAyMQ8wDQYDVQQK

DAZzZXJ2ZXIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDtJAco8Nyw

ClWBpFIPBQcqp0jbdLlpJMdepUc207/4OSeR292few0p4o6R6/L/0Rp2U2TD03sY

5obkTrikn1eaBKYTS9Qh7ERF1i4GX7JWa5o6PsLZguEtUrlToAWouY+3qqxKZTTY

yL8HrdZ+lPlSuM0whBLaknl6W2AaVE7BWbSnnZCcvCMR0IvVejnPZw6Inm8nRerq

iQO4X163Tjmc58Hcw8rjMA8yQxbAL3ah+HoqqOQ7oA/XhGeu6Fza+3CQr/4I/wxa

sRPuRasiWZnQrjGG2zWybGLsxBYsmbnttBocrjkkZvAxDCAXT0bKiyrZ8hUATumD

4Ayj1jYa06mPAgMBAAGjgZowgZcwCQYDVR0TBAIwADALBgNVHQ8EBAMCBaAwEwYD

VR0lBAwwCgYIKwYBBQUHAwEwKAYDVR0RBCEwH4IIVklSLUwwMDKCCFZJUi1MMDAy

gglsb2NhbGhvc3QwHQYDVR0OBBYEFEH4kcPC3fcaGpjZLVwLBHytSarkMB8GA1Ud

IwQYMBaAFN/dT878JMVPUNBlxT7/s+b/F+qAMA0GCSqGSIb3DQEBCwUAA4IBAQBC

V0kx5wusjgfhKFYBBLaZWmtTkx3oXKkwW8v0QU9x2xhcDZT9JRO7Abu5IruGcsDH

i03KTEUiOj8Byee1AaYGEDcx8UkVRr7jAHUWoWe52ba2TJqoCFDDDL8apkUC+v5c

nxBc+zjizwO8gCTRr0vKCoEkjYScaW/F/h5IbLmRF9QbsbhLWmI/eB1+haXmjj0Q

toL9AamL2P1Y+HCxZ8QU+M+KO6V1rQ9hopPJrXZHhsoM1A4w//TXi7LYdM7zI/+U

SLkkpPR34p7kkaaaDxBRc1DJkZCA70l6BFTvQ6/hCEM2T2MvFi6/SHM4JnoQcJbD

6aLTUtLi4PuhW7+/rfoO

-----END CERTIFICATE-----

subject=CN = <my-hostname>, O = server

issuer=CN = TLSGenSelfSignedtRootCA, L = $$$$

---

No client certificate CA names sent

Peer signing digest: SHA256

Peer signature type: RSA-PSS

Server Temp Key: X25519, 253 bits

---

SSL handshake has read 2292 bytes and written 390 bytes

Verification: OK

Verified peername: <my-hostname>

---

New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384

Server public key is 2048 bit

Secure Renegotiation IS NOT supported

Compression: NONE

Expansion: NONE

No ALPN negotiated

Early data was not sent

Verify return code: 0 (ok)

---

---

Post-Handshake New Session Ticket arrived:

SSL-Session:

    Protocol  : TLSv1.3

    Cipher    : TLS_AES_256_GCM_SHA384

    Session-ID: 2FC5272F46B57D83BADFB327E4CD7FFC1DFC821E384FCF036F6F3FF2C1338DD5

    Session-ID-ctx:

    Resumption PSK: 091C8D82B0121589C5D7684E29651BC79C3EFAC52BF5E3C6A53DB66CA5592ED8BB374735FE1D8A14796EB42F918A1F14

    PSK identity: None

    PSK identity hint: None

    SRP username: None

    TLS session ticket lifetime hint: 7200 (seconds)

    TLS session ticket:

    0000 - b6 78 6e 24 b4 e6 0d bd-93 ab 1f 64 4b 10 5e 1b   .xn$.......dK.^.

    0010 - 15 7e 36 6d cd 5e 07 f1-9e 45 8c a9 2d 69 a7 eb   .~6m.^...E..-i..

    0020 - c2 1b e6 c9 a0 39 37 65-33 f2 ef bf 6f f7 61 eb   .....97e3...o.a.

    0030 - 93 b6 ae 47 6b 7e 83 ff-80 49 d5 84 80 61 ba 73   ...Gk~...I...a.s

    0040 - fe 3d f1 58 26 35 0f 5f-e6 27 93 25 14 67 26 a0   .=.X&5._.'.%.g&.

    0050 - 58 d0 9c 8c 79 3a 87 0f-ea 03 db a5 1f 89 40 d6   X...y:........@.

    0060 - c7 fa 37 d2 23 be 7b 97-92 ee e2 9f 3a bc 3c 30   ..7.#.{.....:.<0

    0070 - de d6 92 72 8e 31 b0 f6-55 2a 8a 39 39 f5 93 83   ...r.1..U*.99...

    0080 - ef 34 db 47 77 2d 88 25-b6 47 c1 5e 2a f2 01 45   .4.Gw-.%.G.^*..E

    0090 - 62 98 33 d8 ec 2b 8a ed-93 d6 a7 d8 0e 6c 56 11   b.3..+.......lV.

    00a0 - fa eb 4d b2 1f 11 25 b9-f9 9c 13 be 49 ac 00 0c   ..M...%.....I...

    00b0 - 4d 07 2f 86 b4 cd 0c e0-58 84 b8 ed 07 09 4d 85   M./.....X.....M.

    Start Time: 1608295418

    Timeout   : 7200 (sec)

    Verify return code: 0 (ok)

    Extended master secret: no

    Max Early Data: 0

---

read R BLOCK

---

Post-Handshake New Session Ticket arrived:

SSL-Session:

    Protocol  : TLSv1.3

    Cipher    : TLS_AES_256_GCM_SHA384

    Session-ID: FBC2234E9CB28085F7C78466AA13BFCD19A09EA03A2423113273C3CE0D0CF394

    Session-ID-ctx:

    Resumption PSK: 00F84917198B546B79A65FC9C1B511152AF74714ACF88E32B27D3E07F960EA8CC5E27CBAD02027F9DE646B2BA2ED596D

    PSK identity: None

    PSK identity hint: None

    SRP username: None

    TLS session ticket lifetime hint: 7200 (seconds)

    TLS session ticket:

    0000 - b6 78 6e 24 b4 e6 0d bd-93 ab 1f 64 4b 10 5e 1b   .xn$.......dK.^.

    0010 - d0 55 6b 22 be 48 86 97-d6 a4 7b 12 39 73 f0 c9   .Uk".H....{.9s..

    0020 - ca 2c e1 44 8b 63 ee b3-ce 40 9f 73 28 b0 36 62   .,.D.c...@.s(.6b

    0030 - a8 d1 db 80 9d 90 80 df-aa b5 c7 dd 9b f7 4c d9   ..............L.

    0040 - b6 f6 99 54 52 0c 48 59-f0 76 ce 01 d0 3d 3d b5   ...TR.HY.v...==.

    0050 - b3 e9 ff 0f 7b 2b c5 cb-59 a8 54 10 cd c4 61 4e   ....{+..Y.T...aN

    0060 - dd 5c d4 48 83 33 ae 14-27 4a 59 9e 2d 6c 06 48   .\.H.3..'JY.-l.H

    0070 - 98 13 96 1c b4 2a 5f ff-bc f2 79 6d 93 3d 39 76   .....*_...ym.=9v

    0080 - cd 74 8d 1a c6 e9 b3 c7-ab 40 49 1a 95 27 43 a1   .t.......@I..'C.

    0090 - d9 82 4c 5f bd 62 74 d1-d6 7a 99 9d df b0 91 48   ..L_.bt..z.....H

    00a0 - 77 7a 03 0c ff 8b 5d b4-e4 49 c9 56 fe e4 30 e5   wz....]..I.V..0.

    00b0 - a5 1d f1 37 11 85 cb 35-65 12 44 af f1 2a 02 4e   ...7...5e.D..*.N

    00c0 - 49 60 2c 1b e6 5e 16 ea-b3 be 49 96 1d c2 30 da   I`,..^....I...0.

    Start Time: 1608295418

    Timeout   : 7200 (sec)

    Verify return code: 0 (ok)

    Extended master secret: no

    Max Early Data: 0

---

read R BLOCK

#

# Output of listening window

#

Using default temp DH parameters

ACCEPT

-----BEGIN SSL SESSION PARAMETERS-----

MH4CAQECAgMEBAITAgQgC2yh91SBRZLwmFOOvx+wpHYdQxVb2ox+kQgBVQBt6s0E

MAD4SRcZi1RreaZfycG1ERUq90cUrPiOMrJ9Pgf5YOqMxeJ8utAgJ/neZGsrou1Z

baEGAgRf3KP6ogQCAhwgpAYEBAEAAACuBwIFAL06hAY=

-----END SSL SESSION PARAMETERS-----

Shared ciphers:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES256-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA

Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:ECDSA+SHA1:RSA+SHA224:RSA+SHA1:DSA+SHA224:DSA+SHA1:DSA+SHA256:DSA+SHA384:DSA+SHA512

Shared Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:ECDSA+SHA1:RSA+SHA224:RSA+SHA1

Supported Elliptic Groups: X25519:P-256:X448:P-521:P-384

Shared Elliptic groups: X25519:P-256:X448:P-521:P-384

---

No server certificate CA names sent

CIPHER is TLS_AES_256_GCM_SHA384

Secure Renegotiation IS supported

#

# Full config (rabbitmq.conf)

#

listeners.ssl.default = 5671

ssl_options.cacertfile = /tmp/tls/tls-gen/basic/result/ca_certificate.pem

ssl_options.certfile = /tmp/tls/tls-gen/basic/result/server_certificate.pem

ssl_options.keyfile = /tmp/tls/tls-gen/basic/result/server_key.p12

loopback_users = none

management.load_definitions = /tmp/conf/broker_definitions.json

ssl_options.versions.1 = tlsv1.3

ssl_options.versions.2 = tlsv1.2

ssl_options.versions.3 = tlsv1.1

ssl_options.ciphers.1  = ECDHE-ECDSA-AES256-GCM-SHA384

ssl_options.ciphers.2  = ECDHE-RSA-AES256-GCM-SHA384

ssl_options.ciphers.3  = ECDHE-ECDSA-AES256-SHA384

ssl_options.ciphers.4  = ECDHE-RSA-AES256-SHA384

ssl_options.ciphers.5  = ECDH-ECDSA-AES256-GCM-SHA384

ssl_options.ciphers.6  = ECDH-RSA-AES256-GCM-SHA384

ssl_options.ciphers.7  = ECDH-ECDSA-AES256-SHA384

ssl_options.ciphers.8  = ECDH-RSA-AES256-SHA384

ssl_options.ciphers.9  = DHE-RSA-AES256-GCM-SHA384

ssl_options.ciphers.10 = DHE-DSS-AES256-GCM-SHA384

ssl_options.ciphers.11 = DHE-RSA-AES256-SHA256

ssl_options.ciphers.12 = DHE-DSS-AES256-SHA256

ssl_options.ciphers.13 = ECDHE-ECDSA-AES128-GCM-SHA256

ssl_options.ciphers.14 = ECDHE-RSA-AES128-GCM-SHA256

ssl_options.ciphers.15 = ECDHE-ECDSA-AES128-SHA256

ssl_options.ciphers.16 = ECDHE-RSA-AES128-SHA256

ssl_options.ciphers.17 = ECDH-ECDSA-AES128-GCM-SHA256

ssl_options.ciphers.18 = ECDH-RSA-AES128-GCM-SHA256

ssl_options.ciphers.19 = ECDH-ECDSA-AES128-SHA256

ssl_options.ciphers.20 = ECDH-RSA-AES128-SHA256

ssl_options.ciphers.21 = DHE-RSA-AES128-GCM-SHA256

ssl_options.ciphers.22 = DHE-DSS-AES128-GCM-SHA256

ssl_options.ciphers.23 = DHE-RSA-AES128-SHA256

ssl_options.ciphers.24 = DHE-DSS-AES128-SHA256

ssl_options.ciphers.25 = ECDHE-ECDSA-AES256-SHA

ssl_options.ciphers.26 = ECDHE-RSA-AES256-SHA

ssl_options.ciphers.27 = DHE-RSA-AES256-SHA

ssl_options.ciphers.28 = DHE-DSS-AES256-SHA

ssl_options.ciphers.29 = ECDH-ECDSA-AES256-SHA

ssl_options.ciphers.30 = ECDH-RSA-AES256-SHA

ssl_options.ciphers.31 = ECDHE-ECDSA-AES128-SHA

ssl_options.ciphers.32 = ECDHE-RSA-AES128-SHA

ssl_options.ciphers.33 = DHE-RSA-AES128-SHA

ssl_options.ciphers.34 = DHE-DSS-AES128-SHA

ssl_options.ciphers.35 = ECDH-ECDSA-AES128-SHA

ssl_options.ciphers.36 = ECDH-RSA-AES128-SHA

[Richard]

Apologies, I was using the wrong docker image. 

Now I'm using the latest: Starting RabbitMQ 3.8.9 on Erlang 23.2

And I still see the same behaviour

[Richard]

Still struggling with this. I got into my docker container to try rabbitmq-diagnostics and I couldn't see any problems. Listeners all looked good. Is there a way to get even more logging to try and figure out why the connection wasn't even entertained?

[Michal Kuratczyk]

Can you create a repo with all the files and commands so that we can reproduce the problem?

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/3c50dc02-e1c0-45e4-a0cd-80b60bd03627n%40googlegroups.com.

--

Michał

[Richard]

Sure, and thanks for taking the time to look at this problem for me. 

I've attached the folder structure I'm using, together with a readme with all the commands I'm running. 

[Michal Kuratczyk]

You have a password-protected key but you didn't tell RabbitMQ what the password is. Add `ssl_options.password = bunnies` to the config file.

To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/4121b098-d4ed-4a2e-9e6d-19d08f806902n%40googlegroups.com.

--

Michał

[Richard]

Hmm, I added that in but I still seem to get exactly the same behaviour unfortunately (even after deleting the docker containers, just in case). 

I think I may have stripped that out of my example by accident. 

[Michal Kuratczyk]

Oh, one more thing - change '.p12' to '.pem`. It works for me now.

To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/1cb4758c-395b-498f-9347-109a8583e03an%40googlegroups.com.

--

Michał

[Richard]

It works! 

Thanks so much for helping me with this Michal, I really appreciate it.

[Tobias Schöeneberg]

Hi Michal,

hi Richard,

I have the same problem. I downloaded Richard's zip file and did the two fixes (use .pem instead of p12, and add ssl_options.password).

But for me the problem persists. 

I build on Richards work and added it all into this little repo: https://github.com/metas-ts/rabbitmq_tls_problem

It would be really great if someone could help me.

Best regards Tobias

[Luke Bakken]

Hello,

Remove these lines and re-try -

https://github.com/metas-ts/rabbitmq_tls_problem/blob/main/volumes/config/rabbitmq.conf#L10-L57

Also you have to be absolutely certain that the user that RabbitMQ runs as can read the certs and has access to all intermediate directories. Please confirm that by fully listing directories and attach the output here.

Thanks,

Luke

[Tobias Schöeneberg]

Hello Luke, 

thx for the hint. I removed them and retried - same error.

Meanwhile our admin set up a solution with nginx as teminating TLS proxy. But I could still try out other things, in case it helps to avoid the problem for others..