ssl_error_rx_record_too_long from Management Console

[Robert Greathouse]

I secured the management console as described at https://www.rabbitmq.com/ssl.html. And confirmed my configuration by following the directions at https://www.rabbitmq.com/troubleshooting-ssl.html. I do get a successful handshake from the broker. But when I try accessing the management console with Firefox, I get a ssl_error_rx_record_too_long error. When I try with Internet Explorer 11 I get an error stating to tunr on TLS 1.2, 1.1, and 1.0. But they are enabled. I've pasted my rabbitmq.config below.

%% -*- mode: erlang -*-

[

 {ssl, [

        {versions, ['tlsv1.2', 'tlsv1.1', tlsv1]}

       ]

 },

 {rabbit, [

           {tcp_listeners, [5672]},

           {ssl_listeners, [5671]},

           {ssl_options, [

                          {cacertfile, "/etc/rabbitmq/testca/cacert.pem"},

                          {certfile, "/etc/rabbitmq/server/cert.pem"},

                          {keyfile, "/etc/rabbitmq/server/key.pem"},

                          {password, "11235813"},

                          {verify, verify_peer},

                          {fail_if_no_peer_cert, false}

                          ]

           }

          ]

 },

 {rabbitmq_management, [

                        {listener, [

                                    {port, 15672},

                                    {ssl, true}

                                   ]

                        }

                       ]

 }

].

[Jean-Sébastien Pédron]

On 21/01/2016 03:10, Robert Greathouse wrote:
> But when I try accessing the
> management console with Firefox, I get a ssl_error_rx_record_too_long
> error.

Hi!

Is "ssl_error_rx_record_too_long" is the entire message Firefox
displays? If not, could you please send the entire message?

Also, can you post RabbitMQ log file?

--
Jean-Sébastien Pédron
Pivotal / RabbitMQ

[Michael Klishin]

On 21 January 2016 at 05:10:08, Robert Greathouse (rob...@opensourcearchitect.co) wrote:
> But when I try accessing the management console with Firefox,
> I get a ssl_error_rx_record_too_long error.

on what Erlang version? We highly recommend using at least 17.5 with TLS. 
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[Robert Greathouse]

Erlang 18.1

=INFO REPORT==== 20-Jan-2016::21:15:42 ===
Starting RabbitMQ 3.5.7 on Erlang 18.1
Copyright (C) 2007-2015 Pivotal Software, Inc.
Licensed under the MPL.  See http://www.rabbitmq.com/

=INFO REPORT==== 20-Jan-2016::21:15:42 ===
node           : rabbit@<hostname>
home dir       : /var/lib/rabbitmq
config file(s) : /etc/rabbitmq/rabbitmq.config
cookie hash    : <hash>
log            : /var/log/rabbitmq/rabbit@<hostname>.log
sasl log       : /var/log/rabbitmq/rabbit@<hostname>-sasl.log
database dir   : /var/lib/rabbitmq/mnesia/rabbit@<hostname>


=INFO REPORT==== 20-Jan-2016::21:15:42 ===
Memory limit set to 6380MB of 15950MB total.

=INFO REPORT==== 20-Jan-2016::21:15:42 ===
Disk free limit set to 50MB

=INFO REPORT==== 20-Jan-2016::21:15:42 ===
Limiting to approx 16284 file handles (14653 sockets)

=INFO REPORT==== 20-Jan-2016::21:15:42 ===
FHC read buffering:  ON
FHC write buffering: ON

=INFO REPORT==== 20-Jan-2016::21:15:42 ===
Priority queues enabled, real BQ is rabbit_variable_queue

=INFO REPORT==== 20-Jan-2016::21:15:43 ===
Management plugin: using rates mode 'basic'

=INFO REPORT==== 20-Jan-2016::21:15:43 ===
msg_store_transient: using rabbit_msg_store_ets_index to provide index

=INFO REPORT==== 20-Jan-2016::21:15:43 ===
msg_store_persistent: using rabbit_msg_store_ets_index to provide index

=INFO REPORT==== 20-Jan-2016::21:15:43 ===
started TCP Listener on 0.0.0.0:5672

=INFO REPORT==== 20-Jan-2016::21:15:43 ===
started SSL Listener on 0.0.0.0:5671

=INFO REPORT==== 20-Jan-2016::21:15:43 ===
rabbit on node 'rabbit@<hostname-node2>' up

=INFO REPORT==== 20-Jan-2016::21:15:43 ===
rabbit on node 'rabbit@<hostname-node3>' up

=INFO REPORT==== 20-Jan-2016::21:15:43 ===
Management plugin started. Port: 15672

=INFO REPORT==== 20-Jan-2016::21:15:43 ===
Statistics database started.

=INFO REPORT==== 20-Jan-2016::21:15:43 ===
Server startup complete; 9 plugins started.
 * rabbitmq_federation_management
 * rabbitmq_management_visualiser
 * rabbitmq_management
 * rabbitmq_management_agent
 * rabbitmq_federation
 * amqp_client
 * rabbitmq_web_dispatch
 * webmachine
 * mochiweb

[Michael Klishin]

Does it work if you use the same certificate with openssl s_server or nginx/Apache.

If so, this is an issue to bring up on erlang-questions.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Robert Greathouse]

Yes, it does work with openssl, both as a server to test the certs, and as a client to the broker.

[Michael Klishin]

On 22 January 2016 at 00:02:57, Robert Greathouse (rob...@opensourcearchitect.co) wrote:
> Yes, it does work with openssl, both as a server to test the certs,
> and as a client to the broker.

Earlier you said that

> I do get a successful handshake from the broker. 

you mean you can connect using TLS to port 5672 but not 15672, correct?

Can you try with a nightly build to compare ? (it uses a different HTTP server in the management plugin)
http://www.rabbitmq.com/nightlies/rabbitmq-server/v3.7.0.121/

[Robert Greathouse]

This is my response when connecting to port 5671

openssl s_client -connect localhost:5671 -cert client/cert.pem -key client/key.pem -CAfile testca/cacert.pem
CONNECTED(00000003)
depth=1 CN = MyTestCA
verify return:1
depth=0 CN = icl-tibco-vm01, O = server
verify return:1
---
Certificate chain
 0 s:/CN=icl-tibco-vm01/O=server
   i:/CN=MyTestCA
 1 s:/CN=MyTestCA
   i:/CN=MyTestCA
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIC5zCCAc+gAwIBAgIBATANBgkqhkiG9w0BAQUFADATMREwDwYDVQQDEwhNeVRl
c3RDQTAeFw0xNjAxMjAyMDM2MTJaFw0xNzAxMTkyMDM2MTJaMCoxFzAVBgNVBAMM
DmljbC10aWJjby12bTAxMQ8wDQYDVQQKDAZzZXJ2ZXIwggEiMA0GCSqGSIb3DQEB
AQUAA4IBDwAwggEKAoIBAQDFe8Sf01i5BaCz6uMjnJMiqwrm4rl2LxVRDDPIf4rV
Pka/GHJn/nF12pGVGyVU9j1VKHE3thqXS6HZPM3y39TBHnPcQE6MvH+a5Y5WbaEj
KPKyre510HzIF2j6LPnAvQRMu0BCqtYYYkIcrlMhoG8xaXuCWT0EnrowjLzZcmqF
uyGHzU/eipIMP+2kQQM4RxcHBS1e/t7uk4iAs/ZlFiMqOB+3oiscWWwni/F1CXiC
YUo5JyRjF/Kp8LT1xYhSws97ZUgQ3on/LBXIBi4ou7y2NWyZvsvy6DlerxnXamq4
wiNEas4+CNAQb0gI1JyZkjJTfUM+GYCxrHpXU9G7uM5hAgMBAAGjLzAtMAkGA1Ud
EwQCMAAwCwYDVR0PBAQDAgUgMBMGA1UdJQQMMAoGCCsGAQUFBwMBMA0GCSqGSIb3
DQEBBQUAA4IBAQCUvvvO7tzYP2F1y21RMC3atijDjFRWzS6xZeHCjMvHpqE1rO+w
KboZ9zJANcHs9zDUnHe7S5Ab3CtFPMfRj/s28HCN8RyfzcUDiNyqFC0GAxKxcjKA
/7KyPS3N/LpOFpfasvGZ0NAYgEX34oAdPw0SaQBxQ1jWQ0JCjkiYgu7PY2K95Nn7
Sbl0EZ6s97x5kMLf67bMfhgREL9jBqTKaHlC+F+hsY4QMrOwI0SNhYQR12VNCKvg
3tKlwwnrCug1wQLf0rdqH911wzE/ktFTXJQPXc2d3i0brQtFkq7Ak7Yx3DXcd0L3
+sz8npoQcaDcVF9TizR6iflgYJyLh1Rj8QBV
-----END CERTIFICATE-----
subject=/CN=icl-tibco-vm01/O=server
issuer=/CN=MyTestCA
---
Acceptable client certificate CA names
/CN=MyTestCA
---
SSL handshake has read 1695 bytes and written 2315 bytes
---
New, TLSv1/SSLv3, Cipher is AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : AES256-GCM-SHA384
    Session-ID: 3746E323C84FE5124B2A92BFDC29E47F7C9A9BCB7AC8611015E93434D09AD811
    Session-ID-ctx:
    Master-Key: BE9EE374F4CA41F0886113A148D1A8BCDDCEB33D9F5705EE16A74B8096119FDED2EECC309108EAF5C5CE30A70BCE8EC9
    Key-Arg   : None
    Krb5 Principal: None
    PSK identity: None
    PSK identity hint: None
    Start Time: 1453414157
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

[Robert Greathouse]

This is my response when connecting to port 15672

openssl s_client -connect localhost:15672 -cert client/cert.pem -key client/key.pem -CAfile testca/cacert.pem
CONNECTED(00000003)
139859393967944:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:s23_clnt.c:769:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 249 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
---

[Michael Klishin]

On 22 January 2016 at 01:11:54, Robert Greathouse (rob...@opensourcearchitect.co) wrote:
> openssl s_client -connect localhost:15672 -cert client/cert.pem
> -key client/key.pem -CAfile testca/cacert.pem CONNECTED(00000003)
> 139859393967944:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown
> protocol:s23_clnt.c:769: --- no peer certificate available

OK, thanks.

In "Configuring the HTTP listener", rabbitmq_management.ssl_opts is set as well as rabbit.ssl_options:
http://www.rabbitmq.com/management.html

This is because RabbitMQ messaging protocol listeners use a certain internal library and assume
they will share TLS options. But HTTP API uses an HTTP server which is a completely separate
thing and is entirely unaware of RabbitMQ.

In other words, you need to configure certificates for rabbitmq_management again, and the key
is slightly different.

I understand that this is confusing.  We'll see if we can improve this for 3.7.0.

[Robert Greathouse]

My new rabbitmq.config

%% -*- mode: erlang -*-

[

 {ssl, [

        {versions, ['tlsv1.2', 'tlsv1.1', tlsv1]}

       ]

 },

 {rabbit, [

 %%          {tcp_listeners, [5672]},

           {ssl_listeners, [5671]},

           {ssl_options, [

                          {cacertfile, "/etc/rabbitmq/testca/cacert.pem"},

                          {certfile, "/etc/rabbitmq/server/cert.pem"},

                          {keyfile, "/etc/rabbitmq/server/key.pem"},

                          {password, "11235813"},

                          {verify, verify_peer},

                          %{verify},

                          {fail_if_no_peer_cert, false}

                         ]

           }

          ]

 },

 {rabbitmq_mangement, [

                       {listener, [

                                   {port, 15672},

                                   {ssl, true}

                                   ,{ssl_opts, [

                                               {cacertfile, "/etc/rabbitmq/testca/cacert.pem"},

                                               {certfile, "/etc/rabbitmq/server/cert.pem"},

                                               {keyfile, "/etc/rabbitmq/server/key.pem"},

                                               {verify, verify_peer}

                                               ,{verify_peer}

                                              ]

                                   }

                                  ]

                       }

                      ]

 }

].

ANd the test with the new config. 

openssl s_client -connect localhost:5672 -cert client/cert.pem -key client/key.pem -CAfile testca/cacert.pem
CONNECTED(00000003)
139775589451592:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:s23_clnt.c:769:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 249 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
---
bash-4.1$ openssl s_client -connect localhost:15672 -cert client/cert.pem -key client/key.pem -CAfile testca/cacert.pem
CONNECTED(00000003)
140545848366920:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:s23_clnt.c:769:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 249 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
---

[Michael Klishin]

 On 22 January 2016 at 01:49:43, Robert Greathouse (rob...@opensourcearchitect.co) wrote:
> openssl s_client -connect localhost:5672 -cert client/cert.pem
> -key client/key.pem -CAfile testca/cacert.pem

is this a typo or your tried port 5672 instead of 15672? 5672 does not use TLS in your config
(and typically), and thus it provides no certificate to the client. In fact, I'm surprised that openssl s_client
doesn't hang for a while in this case, as some TLS clients do.
I'd recommend using 15671 for TLS, it leaves less room for mistakes like this.

Another note: peer verification from the server rarely makes sense when HTTP(S) is concerned.
Browsers are rarely set up for two-way peer verification.

[Robert Greathouse]

Yes, that was a typo. But shouldn't I get a SSL handshake on 15672 with this configuration?

[Robert Greathouse]

What is the proper configuration for securing the RabbitMQ console? I tried including the verify_peer for usage with scripted access to the RESTful API. But I haven't gotten to that point yet.

[Michael Klishin]

On 22 January 2016 at 04:54:15, Robert Greathouse (rob...@opensourcearchitect.co) wrote:
> shouldn't I get a SSL handshake on 15672 with this configuration?

Your config looks correct.

RabbitMQ HTTP API is used with TLS in at least one Pivotal product, so we are confident it
works fine.

How were your certificates produced, e.g. can you reproduce this with https://github.com/michaelklishin/tls-gen? 

[Robert Greathouse]

They were generated using the directions from Pivotal at https://www.rabbitmq.com/ssl.html.

[Michael Klishin]

On 22 January 2016 at 14:34:11, Robert Greathouse (rob...@opensourcearchitect.co) wrote:
> They were generated using the directions from Pivotal at https://www.rabbitmq.com/ssl.html.

Which is the same thing as tls-gen.

What Erlang version and OS do you use? We'll try reproducing. 

[Robert Greathouse]

Erlang 18.1 RHEL 7.1