SSL - alert 'unknown CA'

[Gabriele Paggi]

Hello :)

I'm in the process of replacing SSL certificates in a working setup. 

The original SSL certificates were signed by the same single CA, whereas the new ones are signed different sub-CA signed by the same sub-CA, which is then signed by the root-CA.

I proceeded doing the following:

- created a cabundle.pem by bundling all CA certificates

- set depth: 4 in the ssl_options

- replaced cacertfile with cacerts  in the ssl_options (is it needed?)

- fixed the path in cacerts to point to cabundle.pem

The error I get when connecting to the broker is:

=ERROR REPORT==== 23-Jan-2015::16:27:30 ===

SSL: certify: ssl_alert.erl:92:Fatal error: unknown ca

=INFO REPORT==== 23-Jan-2015::16:27:33 ===

accepting AMQP connection <0.433.0> (10.15.30.4:55609 -> 10.15.20.4:5671)

=ERROR REPORT==== 23-Jan-2015::16:27:33 ===

SSL: certify: ssl_handshake.erl:1401:Fatal error: unknown ca

if I test with openssl everything seems to work fine:

===client

[root@server ~]# openssl s_client -connect 10.15.20.4:5671 -cert /etc/pki/tls/certs/server.pem -key /etc/pki/tls/private/server.key -CAfile /etc/pki/tls/certs/cabundle.pem

[...]

    Krb5 Principal: None

    PSK identity: None

    PSK identity hint: None

    Start Time: 1422026439

    Timeout   : 300 (sec)

    Verify return code: 0 (ok)

---

===server

root@server2 private]# openssl s_server -accept 5671 -CAfile /etc/pki/tls/certs/cabundle.pem -cert /etc/pki/tls/certs/server.pem -key /etc/pki/tls/private/server.key -state

Using default temp DH parameters

Using default temp ECDH parameters

ACCEPT

SSL_accept:before/accept initialization

SSL_accept:SSLv3 read client hello A

SSL_accept:SSLv3 write server hello A

SSL_accept:SSLv3 write certificate A

SSL_accept:SSLv3 write key exchange A

SSL_accept:SSLv3 write server done A

SSL_accept:SSLv3 flush data

SSL_accept:SSLv3 read client key exchange A

SSL_accept:SSLv3 read finished A

SSL_accept:SSLv3 write session ticket A

SSL_accept:SSLv3 write change cipher spec A

SSL_accept:SSLv3 write finished A

SSL_accept:SSLv3 flush data

-----BEGIN SSL SESSION PARAMETERS-----

MFUCAQECAgMDBALAMAQABDDsRpNLmoA0nL+hR6FLZs1gjvKqWadNXTtPyqyZ4A81

ubZOqY/T4DpyrxO1/VwIQuyhBgIEVMJmd6IEAgIBLKQGBAQBAAAA

-----END SSL SESSION PARAMETERS-----

Shared ciphers:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-

[...]

Secure Renegotiation IS supported

What is the correct way to configure ssl_options to use a CA bundle file instead of one a single CA certificate?

Thanks!!!!

--- 

Gabriele

[Michael Klishin]

Search for "Verification Depth" on [1]. Make sure the root CA is on the appropriate list of
trusted CA's on your system (this varies between OSes and platforms, e.g. JDK has its own store).

1. http://www.rabbitmq.com/ssl.html
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[Gabriele Paggi]

Hi Michael,

On 23 Jan 2015 17:16, "Michael Klishin" <mkli...@pivotal.io> wrote:
>
> On 23 January 2015 at 18:32:42, Gabriele Paggi (gabrie...@gmail.com) wrote:
> > I'm in the process of replacing SSL certificates in a working
> > setup.

[...]

> > - set depth: 4 in the ssl_options
> > - replaced cacertfile with cacerts in the ssl_options (is it
> > needed?)

[...]

> Search for "Verification Depth" on [1]. Make sure the root CA is on the appropriate list of
> trusted CA's on your system (this varies between OSes and platforms, e.g. JDK has its own store).

I read that paragraph and that's why I set depth: 4 in ssl_options.
My client is another RabbitMQ server which has this one as upstream in a federation; I don't think I need a keystore, right?
Do I need to use cacertfile or cacerts in ssl_options to specify my ca bundle (it doesn't work either way)?

thanks!

--
Gabriele

[Michael Klishin]

On 23 January 2015 at 19:32:11, Gabriele Paggi (gabriel...@gmail.com) wrote:
> My client is another RabbitMQ server which has this one as upstream
> in a federation; I don't think I need a keystore, right?

You don't. Note that BOTH server and client must use a sufficient verification depth.
For Federation this means configuring the Erlang client (amqp_client.ssl_options IIRC).

> Do I need to use cacertfile or cacerts in ssl_options to specify
> my ca bundle (it doesn't work either way)?

In ssl_options of what app? If you configure amqp_client.ssl_options
and ssl.ssl_options, then anything that uses TLS as a client should use the
settings you want.

[Gabriele Paggi]

On Friday, 23 January 2015 17:36:25 UTC+1, Michael Klishin wrote:

On 23 January 2015 at 19:32:11, Gabriele Paggi (gabriel...@gmail.com) wrote:
> My client is another RabbitMQ server which has this one as upstream  
> in a federation; I don't think I need a keystore, right?

You don't. Note that BOTH server and client must use a sufficient verification depth.
For Federation this means configuring the Erlang client (amqp_client.ssl_options IIRC).

By configuring the Erlang client I guess you mean adding options under {rabbit, [ {ssl_options .... } ] } in rabbitmq.config, right?

 

> Do I need to use cacertfile or cacerts in ssl_options to specify  
> my ca bundle (it doesn't work either way)?

In ssl_options of what app? If you configure amqp_client.ssl_options
and ssl.ssl_options, then anything that uses TLS as a client should use the
settings you want.

ssl_options in rabbitmq.config. What is the difference between amqp_client.ssl_options  and ssl.ssl_options and how (where) do I configure both?

Thanks!

-- 

Gabriele

[Michael Klishin]

 On 23 January 2015 at 23:32:17, Gabriele Paggi (gabrie...@gmail.com) wrote:
> By configuring the Erlang client I guess you mean adding options
> under {rabbit, [ {ssl_options .... } ] } in rabbitmq.config,
> right?

Under {amqp_client, [{ssl_options, …}]}, since `rabbit` is for RabbitMQ server.

> > > Do I need to use cacertfile or cacerts in ssl_options to specify
> > > my ca bundle (it doesn't work either way)?
> >
> > In ssl_options of what app? If you configure amqp_client.ssl_options
> > and ssl.ssl_options, then anything that uses TLS as a client
> should use the
> > settings you want.
>
> ssl_options in rabbitmq.config. What is the difference between
> amqp_client.ssl_options and ssl.ssl_options and how (where)
> do I configure both?

amqp_client is for RabbitMQ Erlang client while ssl is for the standard library TLS support app.

In the same config file.

[Gabriele Paggi]

On Friday, 23 January 2015 21:35:48 UTC+1, Michael Klishin wrote:

 On 23 January 2015 at 23:32:17, Gabriele Paggi (gabrie...@gmail.com) wrote:
> By configuring the Erlang client I guess you mean adding options  
> under {rabbit, [ {ssl_options .... } ] } in rabbitmq.config,  
> right?

Under {amqp_client, [{ssl_options, …}]}, since `rabbit` is for RabbitMQ server.

Ok, that should help for the federation, but I get the same error if I try a connection via stomp, with openssl being the client on the other end.

Both client and server use the same ca bundle and their certificate is actually signed by the same ca.

=ERROR REPORT==== 23-Jan-2015::21:56:01 ===

SSL: certify: ssl_handshake.erl:1401:Fatal error: unknown ca

=ERROR REPORT==== 23-Jan-2015::21:56:06 ===

STOMP detected TLS certificate verification error on <0.429.0> (10.15.30.4:41642 -> 10.15.20.4:61614): alert 'unknown CA'

==stomp plugin configuration

  {rabbitmq_stomp, [

    {ssl_listeners, [61614]} ,

    {ssl_cert_login, true}

  ]},

The client returns Verify return code: 0 (ok) and same does openssl verify -verbose -purpose sslserver -CAfile cabundle.pem server_cert.pem

Is it there a particular format for the CA bundle other than appending pem files with the root CA being the last one in the file?

Thanks for the help!

-- 

Gabriele

[Michael Klishin]

On 24 January 2015 at 00:02:58, Gabriele Paggi (gabrie...@gmail.com) wrote:
> Ok, that should help for the federation, but I get the same error
> if I try a connection via stomp, with openssl being the client
> on the other end.
> Both client and server use the same ca bundle and their certificate
> is actually signed by the same ca.
>
> =ERROR REPORT==== 23-Jan-2015::21:56:01
> ===
> SSL: certify: ssl_handshake.erl:1401:Fatal error: unknown
> ca
>
> =ERROR REPORT==== 23-Jan-2015::21:56:06
> ===
> STOMP detected TLS certificate verification error on <0.429.0>
> (10.15.30.4:41642 -> 10.15.20.4:61614): alert 'unknown CA'
>
>
> ==stomp plugin configuration
> {rabbitmq_stomp, [
> {ssl_listeners, [61614]} ,
> {ssl_cert_login, true}
> ]},

Please post the entire configuration.

> The client returns Verify return code: 0 (ok) and same does openssl
> verify -verbose -purpose sslserver -CAfile cabundle.pem server_cert.pem
>
> Is it there a particular format for the CA bundle other than appending
> pem files with the root CA being the last one in the file?

Not that I'm aware of, simply concat .pem files together in order.

[Gabriele Paggi]

On Friday, 23 January 2015 22:06:00 UTC+1, Michael Klishin wrote:

On 24 January 2015 at 00:02:58, Gabriele Paggi (gabrie...@gmail.com) wrote:

Please post the entire configuration.

Here it is, hope it can help:

[

  {rabbit, [

    {cluster_nodes, {['rabbit@xxx', 'rabbit@xxx'], ram}},

    {cluster_partition_handling, autoheal},

    {tcp_listeners, []},

    {ssl_listeners, [5671]},

    {ssl_options, [{cacerts,"/etc/pki/tls/certs/cabundle.pem"},

                    {certfile,"/etc/pki/tls/certs/xxx_server.pem"},

                    {keyfile,"/etc/pki/tls/private/xxx_server.key"},

                    {depth, 4},

                    {verify,verify_peer},

                    {fail_if_no_peer_cert,true}]},

    {auth_mechanisms, ['AMQPLAIN', 'EXTERNAL']},

    {ssl_cert_login_from, common_name},

    {default_user, <<"xxxx">>},

    {default_pass, <<"xxxx">>}

  ]},

  {kernel, [

  ]}

,

  {rabbitmq_management, [

    {listener, [

      {port, 15672},

      {ssl, true},

      {ssl_opts, [{cacertfile, "/etc/pki/tls/certs/cabundle.pem"},                                {certfile, "/etc/pki/tls/certs/xxx_server.pem"},

                  {keyfile, "/etc/pki/tls/private/xxx_server.key"}]}

    ]}

  ]}

,

  {rabbitmq_stomp, [

    {ssl_listeners, [61614]} ,

    {ssl_cert_login, true}

  ]},

  {amqp_client, [

        {ssl_options, [{cacerts,"/etc/pki/tls/certs/cabundle.pem"},

                    {certfile,"/etc/pki/tls/certs/xxx.pem"},

                    {keyfile,"/etc/pki/tls/private/xxx.key"},

                                        {depth, 4}

        ]}

  ]}

].

% EOF

 

> The client returns Verify return code: 0 (ok) and same does openssl
> verify -verbose -purpose sslserver -CAfile cabundle.pem server_cert.pem
>
> Is it there a particular format for the CA bundle other than appending
> pem files with the root CA being the last one in the file?

Not that I'm aware of, simply concat .pem files together in order.

Ok, then I should be good to go :)

-- 

Gabriele

 

[Michael Klishin]

On 24 January 2015 at 00:10:46, Gabriele Paggi (gabrie...@gmail.com) wrote:
> {rabbit, [
> {cluster_nodes, {['rabbit@xxx', 'rabbit@xxx'], ram}},
> {cluster_partition_handling, autoheal},
> {tcp_listeners, []},
> {ssl_listeners, [5671]},
> {ssl_options, [{cacerts,"/etc/pki/tls/certs/cabundle.pem"},
> {certfile,"/etc/pki/tls/certs/xxx_server.pem"},
> {keyfile,"/etc/pki/tls/private/xxx_server.key"},
> {depth, 4},
>
> {verify,verify_peer},
> {fail_if_no_peer_cert,true}]},
> {auth_mechanisms, ['AMQPLAIN', 'EXTERNAL']},
> {ssl_cert_login_from, common_name},
> {default_user, <<"xxxx">>},
> {default_pass, <<"xxxx">>}
> ]},
> {kernel, [
>
> ]}

OK, so STOMP uses its own TLS listeners but options from rabbit.ssl_options (this is both reduce
duplication in the config and due to networking modules shared by the core and some plugins).

I suspect that you indeed may need to concatenate CA certificates.

[Gabriele Paggi]

What do you mean with concatenate? 

If I verify the certificate against the bundle, openssl returns OK, both for the client and server certificates:

[root@server private]# openssl verify -verbose -purpose sslserver -CAfile /etc/pki/tls/certs/cabundle.pem /etc/pki/tls/certs/xxx_server.pem

/etc/pki/tls/certs/xxx_server.pem: OK

[root@server private]# 

--

Gabriele

[Michael Klishin]

On 24 January 2015 at 00:26:08, Gabriele Paggi (gabriel...@gmail.com) wrote:
> What do you mean with concatenate?
> If I verify the certificate against the bundle, openssl returns
> OK, both for the client and server certificates:
>
> [root@server private]# openssl verify -verbose -purpose sslserver
> -CAfile /etc/pki/tls/certs/cabundle.pem /etc/pki/tls/certs/xxx_server.pem
> /etc/pki/tls/certs/xxx_server.pem: OK

and the entire chain is in the CA bundle file, correct? I was referring to that.

tls-gen can produce certificates with chained CAs:
https://github.com/michaelklishin/tls-gen/

May I ask you to try your setup with such certificates? This may be a STOMP (and, in turn, MQTT) plugin
limitations. Since I can't ask you to hand me over the certificates/keys ;) tls-gen is out best bet to verify this.

[Gabriele Paggi]

Hi Michael,

On Friday, 23 January 2015 22:32:59 UTC+1, Michael Klishin wrote:

On 24 January 2015 at 00:26:08, Gabriele Paggi (gabriel...@gmail.com) wrote:
> What do you mean with concatenate?
> If I verify the certificate against the bundle, openssl returns
> OK, both for the client and server certificates:
>
> [root@server private]# openssl verify -verbose -purpose sslserver
> -CAfile /etc/pki/tls/certs/cabundle.pem /etc/pki/tls/certs/xxx_server.pem
> /etc/pki/tls/certs/xxx_server.pem: OK

and the entire chain is in the CA bundle file, correct? I was referring to that.

Yes it is :)

 

tls-gen can produce certificates with chained CAs:
https://github.com/michaelklishin/tls-gen/

May I ask you to try your setup with such certificates? This may be a STOMP (and, in turn, MQTT) plugin
limitations. Since I can't ask you to hand me over the certificates/keys ;) tls-gen is out best bet to verify this.

Sure, I just tried using tls-gen with the "intermediates" setup (root CA, 2 intermediate CAs) and I got the same error with STOMP and openssl as client:

=ERROR REPORT==== 23-Jan-2015::22:51:35 ===

SSL: certify: ssl_handshake.erl:1401:Fatal error: unknown ca

=ERROR REPORT==== 23-Jan-2015::22:51:40 ===

STOMP detected TLS certificate verification error on <0.441.0> (10.15.30.4:41743 -> 10.15.20.4:61614): alert 'unknown CA'

Didn't try with AMQP as I need to get STOMP going first. 

As you mention a possible STOMP plugin limitation, do you think this is related to the depth of the chain?

-- 

Gabriele

[Michael Klishin]

On 24 January 2015 at 00:57:19, Gabriele Paggi (gabrie...@gmail.com) wrote:
> Sure, I just tried using tls-gen with the "intermediates" setup
> (root CA, 2 intermediate CAs) and I got the same error with STOMP
> and openssl as client:
>
> =ERROR REPORT==== 23-Jan-2015::22:51:35
> ===
> SSL: certify: ssl_handshake.erl:1401:Fatal error: unknown
> ca
>
> =ERROR REPORT==== 23-Jan-2015::22:51:40
> ===
> STOMP detected TLS certificate verification error on <0.441.0>
> (10.15.30.4:41743 -> 10.15.20.4:61614): alert 'unknown CA'
>
> Didn't try with AMQP as I need to get STOMP going first.
>
> As you mention a possible STOMP plugin limitation, do you think
> this is related to the depth of the chain?

Yes, most likely. In fact, you can try the "basic" setup with tls-gen and see
if that works.

Trying with openssl s_server and the chained certificates is another good idea.

[Gabriele Paggi]

On Friday, 23 January 2015 23:01:51 UTC+1, Michael Klishin wrote:

On 24 January 2015 at 00:57:19, Gabriele Paggi (gabrie...@gmail.com) wrote:

> As you mention a possible STOMP plugin limitation, do you think
> this is related to the depth of the chain?

Yes, most likely. In fact, you can try the "basic" setup with tls-gen and see
if that works.

Yes, that works and is the same setup I had in place till today.

 

Trying with openssl s_server and the chained certificates is another good idea.

openssl s_server works already with my certificates and bundle, see first email.

I'll test tomorrow / Monday the federation.

Where can I file a bug / request for enhancement for the stomp plugin? :)

Thanks again for the help :)

-- 

Gabriele  

 

[Michael Klishin]

On 24 January 2015 at 01:11:14, Gabriele Paggi (gabrie...@gmail.com) wrote:
> Where can I file a bug / request for enhancement for the stomp
> plugin? :)
>
> Thanks again for the help :)

On this list. Consider it already filed :)

[Michael Klishin]

On 24 January 2015 at 01:20:24, Michael Klishin (mkli...@pivotal.io) wrote:
> On this list. Consider it already filed :)

Gabriele,

I cannot reproduce your issue.

I have a bunch of certificates freshly generated with tls-gen (using the "intermediate" setup). It has

[Root CA] --- [intermediate CA1] --- [intermediate CA2] --- [server certificate]

and my system is configured to trust the Root CA (using Keychain on OS X; this typically involves adding
.pem files in a well-known /etc/* directory on Linux).

I then start RabbitMQ and connect to ports 5671 and 61614 just fine using `openssl s_client`:
https://gist.github.com/michaelklishin/8506acff3d73ba9d1cf0

and RabbitMQ logs a successful connection:

=INFO REPORT==== 25-Jan-2015::23:25:40 ===
accepting AMQP connection <0.606.0> (127.0.0.1:58483 -> 127.0.0.1:5671)

So, STOMP plugin does use rabbit.ssl_options as expected. This can be verified easily with the
-showcerts option and other `s_client` output.

I'm pretty convinced this is a system-specific issue and not a bug. I'm happy to help
you to get to the bottom of it, though. 

[Philip Seidel]

Has anyone found a fix for this?  From what I have seen across the message boards it appears that SSL is broken in erlang 17+ if you are trying to use intermediary certs.  I tried using tls-gen and I see the same issues when using intermediate certificates.  If I set verify to verify_none there are no issues but that kind of defeats the point.  s_client seems to work fine but when connecting from the ruby amqp client it throws the unknown ca error.

# openssl s_client -connect localhost:5671 -cert /etc/rabbitmq/ssl/client/cert.pem -key /etc/rabbitmq/ssl/client/key.pem -CAfile /etc/rabbitmq/ssl/cacert.pem
CONNECTED(00000003)
depth=3 CN = MyTestRootCA
verify return:1
depth=2 CN = MyIntermediateCA1
verify return:1
depth=1 CN = MyIntermediateCA2
verify return:1
depth=0 CN = WASE-AM02004389, O = server
verify return:1
---
Certificate chain
 0 s:/CN=WASE-AM02004389/O=server
   i:/CN=MyIntermediateCA2
 1 s:/CN=MyIntermediateCA2
   i:/CN=MyIntermediateCA1
 2 s:/CN=MyIntermediateCA1
   i:/CN=MyTestRootCA
 3 s:/CN=MyTestRootCA
   i:/CN=MyTestRootCA
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIC8TCCAdmgAwIBAgIBAzANBgkqhkiG9w0BAQUFADAcMRowGAYDVQQDDBFNeUlu
dGVybWVkaWF0ZUNBMjAeFw0xNTA5MTUxNzQ5MjdaFw0yNTA5MTIxNzQ5MjdaMCsx
GDAWBgNVBAMTD1dBU0UtQU0wMjAwNDM4OTEPMA0GA1UEChMGc2VydmVyMIIBIjAN
BgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA1QrIEaqPwSVk3T/wlOi2Rf7XfvJI
MwscIorI4f2yCYGx09CJO3+er1xh8dOFhoTDKVOljAYXy3qL79nwWVC7x1KxqbWe
cj8x1YmxzKsDMDsqp7thtUTEqIf0A/nI4+oiZWMyCkd6Taqe7b+21K9bLlxlgSDH
evGoSV66JrgMADi8KJCUfbJcwHnIzIccT7AoDxDoLImV4eRqcgvpkEdGo23ZlCZu
j/K82/OU64/T+mYnPuYa6I3nwibAMK6n/RcxRrHOfqoeeLfCBRaQMb/J2YSAaOXe
OjQDCOwXia8AkULLaVI0PoipIPm9Jyqrfu5mBB3ITn6Uge5yRQ0aLLBEewIDAQAB
oy8wLTAJBgNVHRMEAjAAMAsGA1UdDwQEAwIFIDATBgNVHSUEDDAKBggrBgEFBQcD
ATANBgkqhkiG9w0BAQUFAAOCAQEAS29yGJlgeH7TBX/RZpTJXhQI+h70ttLGNzhz
Nr87iFQ3+asmKNdAA0DM59H1Bd8R+ysrSGW+bx/zoIbWc+SBqnbsWHeWPKljPgH8
rEOcM0ffDfdSZJopeudW3YnnFVovASfidPucGNcHeOb0iaA0ilVVksjE2LDQSly+
QaEAcAApMs+vGp+AhzRo5pmCZdiy4MslyUfChMurMApS3yHTCPtQghKzpCeFWndN
1ptrTU6sb7j8yBOwlpNqDZivtc+7pNsInDqEbtipFMMRv7/5iDJwMqOdFlILlq30
d0LC5FW0s4i/rHMQwYjTYWzm/WfLnguD9WfiEXZsW1m3f+8Vwg==
-----END CERTIFICATE-----
subject=/CN=WASE-AM02004389/O=server
issuer=/CN=MyIntermediateCA2
---
Acceptable client certificate CA names
/CN=MyTestRootCA
/CN=MyIntermediateCA2
/CN=MyIntermediateCA1
---
SSL handshake has read 3538 bytes and written 4090 bytes
---
New, TLSv1/SSLv3, Cipher is AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : AES256-GCM-SHA384
    Session-ID: 0DFEFC1936E60F7B920C500DB5CD66F8F68924C59C944C246B3EA2ED37B37B02
    Session-ID-ctx:
    Master-Key: 9639EC65656435F245A1B938ECB5F775FB5CFB4C15B9A37235CF00B9794FB86B7F4AFA7CCC32221A509011BF05660A3D
    Key-Arg   : None

    Krb5 Principal: None
    PSK identity: None
    PSK identity hint: None

    Start Time: 1442350735
    Timeout   : 300 (sec)

    Verify return code: 0 (ok)

---
12121212
AMQP    closed

[Michael Klishin]

On 16 Sep 2015 at 00:01:36, Philip Seidel (philip...@gmail.com) wrote:
> From what I have seen across the message boards it appears that
> SSL is broken in erlang 17+ if you are trying to use intermediary
> certs. I tried using tls-gen and I see the same issues when using
> intermediate certificates. If I set verify to verify_none there
> are no issues but that kind of defeats the point. s_client seems
> to work fine but when connecting from the ruby amqp client it throws
> the unknown ca error.

It sounds like you are just guessing. No, TLS is NOT broken in Erlang 17+, in fact,
many things around elliptic curve cipher suites were fixed in 17.0 onwards.
If we ever encounter issues with TLS in Erlang/OTP, it is always with versions
older than 17.x.

“Unknown CA” means what it says on the tin: your client does not trust the server’s CA
(could be vice versa: depends on which side logs this error). 
Either add the root CA to the OS’ trusted list or specify a list of trusted
CAs to the client. It can be (you haven’t posted any error messages) a client
end issue: the *client* doesn’t trust the peer, so it has nothing to do with
TLS support in Erlang.

Another thing to be aware of is TLS verification depth: how many certificates
in the chain will be trusted. This default varies from client to client but
is typically 1.

Bunny specifically has documentation sections on how to manage trusted certs,
or at least pointers in the right direction:
http://rubybunny.info/articles/tls.html

Please don’t trust every single thing you read on the Internet. Instead, learn
a bit about PKI and how TLS works.

[Philip Seidel]

Michael --

Thanks for the quick response.  I didn't know if this thread was still alive so I didn't provide any detailed debug logs, etc.  Thanks for clarifying that there aren't any known issues in erlang 17+ as I hadn't seen any responses in the posts that mentioned reverting to a version prior appeared to solve the issue.  I figured I would ping one of these threads to see if that was indeed true which you just confirmed so thanks for that. 

The error I am seeing is in the rabbitmq server log (snippet below).  I also included the rabbitmq configuration file which does have the verification depth set to 3.  The certs were created using tls-gen (very handy, btw).  I certainly understand what the 'unknown ca' means but the issue I was dealing with was trying to figure out why I was getting it since the cacert.pem included the proper ca certs listed in the proper order (copied from tls-gen).  I was under the assumption that if the client cert was signed with the same ca chain that was in the cacert.pem and loaded by rabbitmq it would be able to verify the peer.  I even tried setting the SSL_CERT_FILE environment variable, which ruby appears to load based on the strace output, to the cacert.pem but the AMQP library still can't establish the connection.

I know I shouldn't trust everything on the Internet but I took your advice and checked out the bunny client.  While working with this client I can establish a connection but only when I configure the 'tls_ca_certificates' attribute so it would appear that in order to get the AMQP client to work with EventMachine I would have to provide a similar setting but it doesn't seem to exist (at least I couldn't find one).  I have tried concatenating the client.pem along with the cacert.pem but that doesn't work because rabbitmq server throws a bad certificate error.  Thanks for the pointers but it seems like I might have to chalk this up to an inability to provide a ca chain when using the AMQP client and EventMachine.  Unfortunately, the system I am using (sensu) is based on EventMachine so I don't believe I could use the bunny library.

Thanks again for the info and let me know if you have any other ideas.

=ERROR REPORT==== 16-Sep-2015::01:36:26 ===
SSL: certify: ssl_handshake.erl:1490:Fatal error: unknown ca

=ERROR REPORT==== 16-Sep-2015::01:36:31 ===
Error on AMQP connection <0.2238.0>:
{ssl_upgrade_error,{tls_alert,"unknown ca"}}

[
  {kernel, [

  ]},
  {rabbit, [
    {cluster_nodes, {['rabbit@localhost'], disc}},
    {cluster_partition_handling,pause_minority},
    {ssl_listeners, [5671]},
    {ssl_options, [{cacertfile,"/etc/rabbitmq/ssl/cacert.pem"},
                    {certfile,"/etc/rabbitmq/ssl/cert.pem"},
                    {keyfile,"/etc/rabbitmq/ssl/key.pem"},
                    {verify,verify_peer},
                    {fail_if_no_peer_cert,true},
                    {depth,3}

                    ]},
    {tcp_listen_options, [binary, {packet,raw},
                                  {reuseaddr,true},
                                  {backlog,128},
                                  {nodelay,true},
                                  {exit_on_close,false},
                                  {keepalive,false}]},

    {default_user, <<"guest">>},
    {default_pass, <<"guest">>},
    {heartbeat, 580}

  ]}
].

[Michael Klishin]

 On 16 September 2015 at 05:26:21, Philip Seidel (philip...@gmail.com) wrote:
> I know I shouldn't trust everything on the Internet but I took
> your advice and checked out the bunny client. While working with
> this client I can establish a connection but only when I configure
> the 'tls_ca_certificates' attribute so it would appear that
> in order to get the AMQP client to work with EventMachine I would
> have to provide a similar setting but it doesn't seem to exist
> (at least I couldn't find one). I have tried concatenating the
> client.pem along with the cacert.pem but that doesn't work because
> rabbitmq server throws a bad certificate error. Thanks for the
> pointers but it seems like I might have to chalk this up to an inability
> to provide a ca chain when using the AMQP client and EventMachine.
> Unfortunately, the system I am using (sensu) is based on EventMachine
> so I don't believe I could use the bunny library.
>
> Thanks again for the info and let me know if you have any other ideas.
>
> =ERROR REPORT==== 16-Sep-2015::01:36:26 ===
> SSL: certify: ssl_handshake.erl:1490:Fatal error: unknown
> ca
>
> =ERROR REPORT==== 16-Sep-2015::01:36:31 ===
> Error on AMQP connection <0.2238.0>:
> {ssl_upgrade_error,{tls_alert,"unknown ca"}}

amqp gem (and possibly EventMachine) indeed may not have options for
specifying a list of trusted certificates.

It's a shame that Sensu uses amqp gem. I'll take a look at how trusted
certificates are configured with EventMachine but since it's abandonware, I'm
not holding my breath.

If there is absolutely no way to fix amqp gem, you can disable TLS authentication (verification)
in both the server and the client. This is not great but at least your traffic
will be encrypted.

See https://github.com/rabbitmq/rabbitmq-server/blob/master/docs/rabbitmq.config.example#L66
and Bunny docs.

[Philip Seidel]

Thanks Michael.  I appreciate the help.  Let me know if you find anything out.

[Jeff Garrett]

Phillip.  Did you end up getting Sensu working in this environment?  I think I am having the same problems (unknown ca, bad certificate) 

[Michael Klishin]

There is a separate guide on the basics of TLS troubleshooting:

http://www.rabbitmq.com/troubleshooting-ssl.html

I stated earlier in this thread, Sensu still uses Ruby amqp gem, which is

quite ancient and based on an old networking I/O library that seems to lack

support for recent TLS versions (or it has issues).

We've recommended the Sensu team to migrate to Bunny a long time ago.

I don't know if that went anywhere.

As an alternative, stick an HAproxy in front of your cluster and make it perform

TLS termination. Indeed, our TLS guide now lists it as an option:

http://www.rabbitmq.com/ssl.html.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.