Problems with SSL cipher suites

[quotenjug...@anti-discofestival.de]

We are having issues with SSL cipher suites on a Windows installation of RabbitMQ/Erlang. All the connections to the RabbitMQ server (Linux and Windows clients) only get "tlsv1.2 rsa aes_256_cbc sha256" for the SSL connection. They should be using a ecdhe_* or dhe_* cipher instead which is way more secure. For some reason it only uses simple rsa cipher suites although the other ones are .

This issue doesn't happen with a Linux RabbitMQ server that is using the exact same configuration (I actually got this configuration from the Linux one).

The ssl_options part of the rabbitmq.conf looks like this:

{cacertfile,"C:\\Program Files (x86)\Application\\ssl\\..\\RabbitMQ\\Data\\ssl\\cacert.pem"},

                    {certfile,"C:\\Program Files (x86)\\Application\\ssl\\..\\RabbitMQ\\Data\\ssl\\cert.pem"},

                    {keyfile,"C:\\Program Files (x86)\\Application\\ssl\\..\\RabbitMQ\\Data\\ssl\\key.pem"},

                    {verify,verify_none},

                    {fail_if_no_peer_cert,false},

{ciphers,[

{ecdhe_ecdsa,aes_256_cbc,sha384},

{ecdhe_rsa,aes_256_cbc,sha384},

{ecdh_ecdsa,aes_256_cbc,sha384},

{ecdh_rsa,aes_256_cbc,sha384},

{dhe_rsa,aes_256_cbc,sha256},

{dhe_dss,aes_256_cbc,sha256},

{rsa,aes_256_cbc,sha256},

{ecdhe_ecdsa,aes_128_cbc,sha256},

{ecdhe_rsa,aes_128_cbc,sha256},

{ecdh_ecdsa,aes_128_cbc,sha256},

{ecdh_rsa,aes_128_cbc,sha256},

{dhe_rsa,aes_128_cbc,sha256},

{dhe_dss,aes_128_cbc,sha256},

{rsa,aes_128_cbc,sha256},

{dhe_rsa,aes_256_cbc,sha},

{rsa,aes_128_cbc,sha}]},

{client_renegotiation, false},

{versions, ['tlsv1.2', 'tlsv1.1']},

{honor_cipher_order, true}

cipher_suites/0 lists this:

9> ssl:cipher_suites().

[{ecdhe_ecdsa,aes_256_cbc,sha384},

 {ecdhe_rsa,aes_256_cbc,sha384},

 {ecdh_ecdsa,aes_256_cbc,sha384},

 {ecdh_rsa,aes_256_cbc,sha384},

 {dhe_rsa,aes_256_cbc,sha256},

 {dhe_dss,aes_256_cbc,sha256},

 {rsa,aes_256_cbc,sha256},

 {ecdhe_ecdsa,aes_128_cbc,sha256},

 {ecdhe_rsa,aes_128_cbc,sha256},

 {ecdh_ecdsa,aes_128_cbc,sha256},

 {ecdh_rsa,aes_128_cbc,sha256},

 {dhe_rsa,aes_128_cbc,sha256},

 {dhe_dss,aes_128_cbc,sha256},

 {rsa,aes_128_cbc,sha256},

 {ecdhe_ecdsa,aes_256_cbc,sha},

 {ecdhe_rsa,aes_256_cbc,sha},

 {dhe_rsa,aes_256_cbc,sha},

 {dhe_dss,aes_256_cbc,sha},

 {ecdh_ecdsa,aes_256_cbc,sha},

 {ecdh_rsa,aes_256_cbc,sha},

 {rsa,aes_256_cbc,sha},

 {ecdhe_ecdsa,'3des_ede_cbc',sha},

 {ecdhe_rsa,'3des_ede_cbc',sha},

 {dhe_rsa,'3des_ede_cbc',sha},

 {dhe_dss,'3des_ede_cbc',sha},

 {ecdh_ecdsa,'3des_ede_cbc',sha},

 {ecdh_rsa,'3des_ede_cbc',...},

 {rsa,...},

 {...}|...]

Using sslyze shows this:

  * TLSV1_2 Cipher Suites:

      Preferred:                       

        TLS_RSA_WITH_AES_256_CBC_SHA256                   -              256 bits                                                                  

      Accepted:                        

        TLS_RSA_WITH_AES_256_CBC_SHA256                   -              256 bits                                                                  

        TLS_RSA_WITH_AES_128_CBC_SHA256                   -              128 bits                                                                  

        TLS_RSA_WITH_AES_128_CBC_SHA                      -              128 bits  

For the Linux server sslyze shows this (although there's a weird error with one cipher which doesn't happen with other servers using SSL like httpd):

  * TLSV1_2 Cipher Suites:

      Preferred:                       

        TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384             ECDH-256 bits  256 bits                                                                  

      Accepted:                        

        TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384             ECDH-256 bits  256 bits                                                                  

        TLS_DHE_RSA_WITH_AES_256_CBC_SHA256               DH-1024 bits   256 bits                                                                  

        TLS_DHE_RSA_WITH_AES_256_CBC_SHA                  DH-1024 bits   256 bits                                                                  

        TLS_RSA_WITH_AES_256_CBC_SHA256                   -              256 bits                                                                  

        TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256             ECDH-256 bits  128 bits                                                                  

        TLS_DHE_RSA_WITH_AES_128_CBC_SHA256               DH-1024 bits   128 bits                                                                  

        TLS_RSA_WITH_AES_128_CBC_SHA256                   -              128 bits                                                                  

        TLS_RSA_WITH_AES_128_CBC_SHA                      -              128 bits                                                                  

      Undefined - An unexpected error happened: 

        TLS_DHE_DSS_WITH_AES_256_CBC_SHA256               OpenSSLError - 

error:1409017F:SSL routines:ssl3_get_server_certificate:wrong certificate type

I also think there's a small error in the documentation on https://www.rabbitmq.com/ssl.html since it says you should call "ssl:cipher_suites(openssl)." to get a list of all supported cipher suites, but that returns the ciphers in the wrong format. It should just be the call without any parameter.

[Jean-Sébastien Pédron]

On 14/06/2016 16:28, quotenjug...@anti-discofestival.de wrote:
> We are having issues with SSL cipher suites on a Windows installation of
> RabbitMQ/Erlang. All the connections to the RabbitMQ server (Linux and
> Windows clients) only get "tlsv1.2 rsa aes_256_cbc sha256" for the SSL
> connection. They should be using a ecdhe_* or dhe_* cipher instead which
> is way more secure. For some reason it only uses simple rsa cipher
> suites although the other ones are .

Hi!

What version of Erlang are you using on both Windows and Linux?

--
Jean-Sébastien Pédron
Pivotal / RabbitMQ

[quotenjug...@anti-discofestival.de]

I just realized I forgot to add that information. Sorry about that.

Erlang is 18.0 on Windows and at least 18.0 on Linux.

RabbitMQ server 3.5.4 on Windows and should be at least 3.5.4 on Linux.

The Windows version is Windows Server 2012 (an upgrade to R2 should happen soon) and Linux is Centos 6.x.

I am not in charge of the Linux system - I just know it was updated several times when the Windows one wasn't.

[Jean-Sébastien Pédron]

On 14/06/2016 16:47, quotenjug...@anti-discofestival.de wrote:
> I just realized I forgot to add that information. Sorry about that.

No problem!

> Erlang is 18.0 on Windows and at least 18.0 on Linux.

Good. This is important because the "honor_cipher_order" option is only
available in Erlang 17.0 and later. Without this option, the client-side
cipher order is used.

> RabbitMQ server 3.5.4 on Windows and should be at least 3.5.4 on Linux.
> The Windows version is Windows Server 2012 (an upgrade to R2 should
> happen soon) and Linux is Centos 6.x.

So, I tried to reproduce the problem on Windows because I don't know if
TLS support in Erlang is different on this platform. However, I get the
exact same behavior when using the same configuration on both Unix and
Windows.

I couldn't use sslyze but I played with testssl.sh [1]. I kept the
default ciphersuites and just enabled "honor_cipher_order". On both
RabbitMQ, here is the reported cipher order:

Cipher order
TLSv1: AES256-SHA DES-CBC3-SHA AES128-SHA
TLSv1.1: AES256-SHA DES-CBC3-SHA AES128-SHA
TLSv1.2: AES256-GCM-SHA384 AES256-SHA256 AES128-GCM-SHA256
AES128-SHA256 AES256-SHA DES-CBC3-SHA AES128-SHA

Can you double check and compare the version of Erlang on the Windows
and Linux hosts? There were some regressions with TLS in the 18.x
branch, maybe you hit one of them.

Try to upgrade to a newer Erlang on the Windows host too; the latest one
is 18.3. Of course, do that on a test environment because you could hit
another less pleasant regression.

[1] https://testssl.sh/

[quotenjug...@anti-discofestival.de]

So, I tried to reproduce the problem on Windows because I don't know if
TLS support in Erlang is different on this platform. However, I get the
exact same behavior when using the same configuration on both Unix and
Windows.

I couldn't use sslyze but I played with testssl.sh [1]. I kept the
default ciphersuites and just enabled "honor_cipher_order". On both
RabbitMQ, here is the reported cipher order:

Yeah, I realized (again after posting) that I forgot to include the reference to sslyze - https://github.com/nabla-c0d3/sslyze

 

 Cipher order
     TLSv1:     AES256-SHA DES-CBC3-SHA AES128-SHA
     TLSv1.1:   AES256-SHA DES-CBC3-SHA AES128-SHA
     TLSv1.2:   AES256-GCM-SHA384 AES256-SHA256 AES128-GCM-SHA256
                AES128-SHA256 AES256-SHA DES-CBC3-SHA AES128-SHA

That's exactly the behavior I am getting - no DHE or ECDHE ciphers. Could you try cipher_suites/0 to verify they are actually available for you?

I used the default RabbitMQ configuration and got this from sslyze:

  * TLSV1_2 Cipher Suites:

      Preferred:                       

        TLS_RSA_WITH_AES_256_CBC_SHA256                   -              256 bits                                                                  

      Accepted:                        

        TLS_RSA_WITH_AES_256_CBC_SHA256                   -              256 bits                                                                  

        TLS_RSA_WITH_AES_256_CBC_SHA                      -              256 bits                                                                  

        TLS_RSA_WITH_AES_128_CBC_SHA256                   -              128 bits                                                                  

        TLS_RSA_WITH_AES_128_CBC_SHA                      -              128 bits                                                                  

        TLS_RSA_WITH_3DES_EDE_CBC_SHA                     -              112 bits                                                                  

        TLS_RSA_WITH_DES_CBC_SHA                          -              56 bits   

The DES ciphers are not in my configured cipher list, so that's the reason they were not listed in the ciphers I initially posted. So the ciphers list and honor_cipher_order *are* working. Just for some reason the DHE and ECDHE ciphers are not getting through from Erlang to the RabbitMQ server in some cases.

Interestingly I am not getting any GCM ciphers at all, but that could be because I am using an older Erlang version.

FYI The Linux server is running Erlang 18.1 and RabbitMQ 3.5.6. 

[Jean-Sébastien Pédron]

On 17/06/2016 11:16, quotenjug...@anti-discofestival.de wrote:
> Cipher order
> TLSv1: AES256-SHA DES-CBC3-SHA AES128-SHA
> TLSv1.1: AES256-SHA DES-CBC3-SHA AES128-SHA
> TLSv1.2: AES256-GCM-SHA384 AES256-SHA256 AES128-GCM-SHA256
> AES128-SHA256 AES256-SHA DES-CBC3-SHA AES128-SHA
>
> That's exactly the behavior I am getting - no DHE or ECDHE ciphers.
> Could you try cipher_suites/0 to verify they are actually available for you?

Yes, they are available, but you're right, I didn't notice the selected
ciphersuite was not the expected one.

If I only select the ECDHE-* ciphersuites, openssl s_client(1) fails to
connect to the broker because of "insufficient security". If I use the
same certificates/keys with openssl s_server(1), then,
ECDHE-RSA-AES256-GCM-SHA384 is selected as expected. So this is not a
problem with the certificate.

I tested with Erlang 18.0 on Windows and Erlang 18.3.3 on FreeBSD. I
will test with Erlang 19 because there were bug fixes in SSL.

[quotenjug...@anti-discofestival.de]

On Friday, June 17, 2016 at 5:03:25 PM UTC+2, Jean-Sébastien Pédron wrote:

I tested with Erlang 18.0 on Windows and Erlang 18.3.3 on FreeBSD. I
will test with Erlang 19 because there were bug fixes in SSL.

Thanks! Good somebody can reproduce it.

I usually check the release notes of Erlang and I didn't see anything related to this in any of the recent versions.

It's still weird that we have a Linux system where it works as expected. The packages we are using on CentOS are esl-erlang_18.1-1-centos-6_amd64.rpm and rabbitmq-server-3.5.6-1.noarch.rpm which look like the "official" packages to me. So it's unlikely there's additional patches involved, that are not yet applied upstream which might have fixed this.

[Michael Klishin]

Note that there were multiple 18.x releases since then, technically the most recent Erlang/OTP release is 18.3.4

although it is not yet available in binary form. We are pushing for at least Erlang Solutions to build those patch

versions as well as "feature" release (18.3.0, 19.0, etc). JFYI.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Michael Klishin]

Binary builds of patch Erlang versions are now available from the ESL site, see

https://groups.google.com/forum/#!topic/rabbitmq-users/ad8Mz9Gg9NQ.

On Friday, June 17, 2016 at 7:04:17 PM UTC+3, Michael Klishin wrote:

Note that there were multiple 18.x releases since then, technically the most recent Erlang/OTP release is 18.3.4

although it is not yet available in binary form. We are pushing for at least Erlang Solutions to build those patch

versions as well as "feature" release (18.3.0, 19.0, etc). JFYI.

On Fri, Jun 17, 2016 at 6:54 PM wrote:

On Friday, June 17, 2016 at 5:03:25 PM UTC+2, Jean-Sébastien Pédron wrote:

I tested with Erlang 18.0 on Windows and Erlang 18.3.3 on FreeBSD. I
will test with Erlang 19 because there were bug fixes in SSL.

Thanks! Good somebody can reproduce it.

I usually check the release notes of Erlang and I didn't see anything related to this in any of the recent versions.

It's still weird that we have a Linux system where it works as expected. The packages we are using on CentOS are esl-erlang_18.1-1-centos-6_amd64.rpm and rabbitmq-server-3.5.6-1.noarch.rpm which look like the "official" packages to me. So it's unlikely there's additional patches involved, that are not yet applied upstream which might have fixed this.

--