Error connecting with SSL (unknown CA)

[Nick]

So i followed the guide here : https://www.rabbitmq.com/ssl.html

to create my own CA and my certificates.

My rabbit config looks like this:

[
  {ssl, [{versions, ['tlsv1.2', 'tlsv1.1']}]},
  {rabbit, [
     {ssl_listeners, [5671]},
     {auth_mechanisms, ['EXTERNAL', 'PLAIN']},
     {ssl_apps,[asn1,crypto,public_key,ssl]},
     {ssl_options, [{cacertfile,"/etc/rabbitmq/cert/ca/cacert.pem"},
                    {cacerts,"/etc/rabbitmq/cert/ca/cacert.pem"},
                    {certfile,"/etc/rabbitmq/cert/server_cert.pem"},
                    {keyfile,"/etc/rabbitmq/cert/server_key.pem"},
                    {versions, ['tlsv1.2', 'tlsv1.1']},
                    {ssl_cert_login, true},
                    {ssl_cert_login_from, common_name},
                    {verify,verify_peer},
                    {fail_if_no_peer_cert,false}]},
     {log_levels, [{connection, debug}]}
   ]}
].

I started debugging with: https://www.rabbitmq.com/troubleshooting-ssl.html

I can connect in the shell same as mentioned in paragraph "Check keys and certificates with OpenSSL"

But where i fail is when i try to connect to the broker directly like this:

openssl s_client  -connect 127.0.0.1:5002 -cert client/cert.pem -key client/key.pem -CAfile cacert.pem -tls1_2

The output from the server is:

Error on AMQP connection <0.1147.0>:
{ssl_upgrade_error,{tls_alert,"unknown ca"}}

     Supervisor: {local,tls_connection_sup}
     Context:    child_terminated
     Reason:     {badarg,
                     [{ets,update_counter,[381005,#Ref<0.0.0.86782>,-1],[]},
                      {ssl_pkix_db,ref_count,3,
                          [{file,"ssl_pkix_db.erl"},{line,184}]},
                      {ssl_connection,handle_trusted_certs_db,1,
                          [{file,"ssl_connection.erl"},{line,1799}]},
                      {ssl_connection,terminate,3,
                          [{file,"ssl_connection.erl"},{line,912}]},
                      {tls_connection,terminate,3,
                          [{file,"tls_connection.erl"},{line,326}]},
                      {gen_fsm,terminate,7,[{file,"gen_fsm.erl"},{line,595}]},
                      {proc_lib,init_p_do_apply,3,
                          [{file,"proc_lib.erl"},{line,237}]}]}
     Offender:   [{pid,<0.1148.0>},
                  {name,undefined},
                  {mfargs,{tls_connection,start_link,undefined}},
                  {restart_type,temporary},
                  {shutdown,4000},
                  {child_type,worker}]

The client throws this error:

140735329915728:error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca:s3_pkt.c:1472:SSL alert number 48
140735329915728:error:1409E0E5:SSL routines:ssl3_write_bytes:ssl handshake failure:s3_pkt.c:656:

I have consulted google, but so far nothing that helped me...

Cheers,

Nick

[Michael Klishin]

 On 20 July 2015 at 16:45:29, Nick (springe...@gmail.com) wrote:
> Error on AMQP connection <0.1147.0>:
> {ssl_upgrade_error,{tls_alert,"unknown ca"}}

This likely means that your "client CA" is different from your "server CA",
in which case your server node(s) needs to have the client CA on the list of trusted authorities.

However that is done depends on the OS, e.g. on Linux you typically add a certificate
under a directory in /etc.
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[Nick]

The CA of the client and server are the same (i used the example of the manual in which they are the same).

I thought the following line in the rabbitmq.cnf makes the "client" CA trusted:

 {cacerts,"/etc/rabbitmq/cert/ca/cacert.pem"}

So no need to store it seperately under a secont location on the machine (Linux), correct?

It is stored under /etc/rabbitmq/cert/ca/cacert.pem  though, as specified in the config.

Cheers,

Nick

[Michael Klishin]

On 20 July 2015 at 16:56:09, Nick (springe...@gmail.com) wrote:
> The CA of the client and server are the same (i used the example
> of the manual in which they are the same).
> I thought the following line in the rabbitmq.cnf makes the "client"
> CA trusted:
> {cacerts,"/etc/rabbitmq/cert/ca/cacert.pem"}
>
>
>
> So no need to store it seperately under a secont location on the
> machine (Linux), correct?
> It is stored under /etc/rabbitmq/cert/ca/cacert.pem though,
> as specified in the config.

I'm not sure how you can get that TLS alert when the certificate is the same
and it is correctly specified for the server. Adding it system-wide shouldn't
be necessary.

You have an exception in the log that suggests there was an exception
in the Erlang's  ssl app. What RabbitMQ and Erlang versions do you use?

[Nick]

Yes, that is why i am puzzled, hours of googeling have led to nowhere :S

RabbitMQ 3.5.3, Erlang 17.5.3

[Michael Klishin]

On 20 July 2015 at 17:06:19, Nick (springe...@gmail.com) wrote:
> Yes, that is why i am puzzled, hours of googeling have led to nowhere
> :S
>
> RabbitMQ 3.5.3, Erlang 17.5.3

I'm not sure what may be going on. Try 17.4 or 17.3 or even 17.1?

Inspecting our test suites on Linux, they generate a new certificate pair every time
and do not install them system-wide. However, on Windows we do that.

In any case, the exception you're seeing in the TLS implementation is not something
expected .

[Nick]

So just tried it with Erlang 17.1, 17.3, 17.4, 17.5 and 18.0, still getting the same error :(

[Michael Klishin]

On 21 July 2015 at 14:27:38, Nick (springe...@gmail.com) wrote:
> So just tried it with Erlang 17.1, 17.3, 17.4, 17.5 and 18.0,
> still getting the same error :(

Since your certificates are generated and self-signed, can you try
reproducing the issue with tls-gen [1]? Feel free to modify it in your fork
as needed.

1. https://github.com/michaelklishin/tls-gen 

[Nick]

i created the files doing this

PASSWORD=foopass make

after that i copied ca_certification.pem, server_certificate.pem and server_key.pem to my rabbitMQ instance and added the password option in rabbitmq.cnf (not sure if that was necessary)

ran 

openssl s_client  -connect 127.0.0.1:5002 -cert client_certificate.pem -key client_key.pem -CAfile ca_certificate.pem -tls1_2

still getting the same result.

[Michael Klishin]

On 21 July 2015 at 14:56:55, Nick (springe...@gmail.com) wrote:
> i created the files doing this
>
> PASSWORD=foopass make
>
>
>
> after that i copied ca_certification.pem, server_certificate.pem
> and server_key.pem to my rabbitMQ instance and added the password
> option in rabbitmq.cnf (not sure if that was necessary)
>
> ran
>
> openssl s_client -connect 127.0.0.1:5002 -cert client_certificate.pem
> -key client_key.pem -CAfile ca_certificate.pem -tls1_2
> still getting the same result.

This does not happen for me with 3.5.3, tls-gen-produced certificates and
Erlang 17.4/17.5 on 64-bit OS X. Our CI machines use self-signed certificates
on Linux.

So there is something environment-specific. I have OpenSSL 1.0.2a here. 

[Nick]

On the client side i have 1.0.2d and on the server side 1.0.1e

[Nick]

even when i upgrade my server to 1.0.2a, the problem still persists

[joseph.y...@erlang-solutions.com]

Hello Nick,

We may have uncovered the problem.

The SSL settings listed in the configuration above contain both the cacertfile and the cacerts option:

[{cacertfile,"/etc/rabbitmq/cert/ca/cacert.pem"},
 {cacerts,"/etc/rabbitmq/cert/ca/cacert.pem"},

 ...

]

The cacerts option overrides the cacertfile option. The cacerts tuple then needs type [public_key:der_encoded()], but it gets the "/etc/rabbitmq/cert/ca/cacert.pem" string, so removing the cacerts option might help.

Reference @ http://www.erlang.org/doc/man/ssl.html