SSL peer certificate verification

[Adam Bloom]

Hello,

 

Hoping that one of you knows some secrets you can impart in me to get SSL peer verification working from RabbitMQ. (I also posted this to the erlang mailing list, since it seems to be more related to the underlying Erlang SSL library than Rabbitmq, but reposting here as well just in case someone else has gotten this to work.) I’ve been digging through the source code of the underlying Erlang SSL library to no avail. In particular, I’m struggling to understand the depth setting. According to the docs (http://erlang.org/doc/man/ssl.html):

 

{depth, integer()}

Maximum number of non-self-issued intermediate certificates that can follow the peer certificate in a valid certification path. So, if depth is 0 the PEER must be signed by the trusted ROOT-CA directly; if 1 the path can be PEER, CA, ROOT-CA; if 2 the path can be PEER, CA, CA, ROOT-CA, and so on. The default value is 1.

 

I have two certificates (client and server) signed by the same intermediate CA. I would like to restrict rabbitmq to only accept connections from this (and potentially one more) intermediate CA. My initial thought was to set depth to 0 and place the intermediate ca (followed by the root ca) in the CA cert file. If I do this, the client is unable to connect and the server logs the following error:

SSL: certify: ssl_handshake.erl:1627:Fatal error: handshake failure - {bad_cert,max_path_length_reached}.

 

If I change depth to 1, the connection works. Rabbitmq also allows a cert signed by another intermediate ca (same root) to connect, which is what I’d like to prevent. This follows from the ssl man page though.

 

Can someone please advise me on the proper use of the “depth” setting in the Erlang ssl library such that only certificates signed by the trusted intermediate are able to connect? Do I need to remove the root CA from the CA cert file and explore partial_chain handlers?

 

Here are my current ssl_options settings from rabbitmq:

{ssl_options, [
            {cacertfile, "/etc/rabbitmq/certs/cacert.pem"},
            {certfile, "/etc/rabbitmq/certs/cert.pem"},
            {keyfile, "/etc/rabbitmq/certs/key.pem"},
            {verify, verify_peer},
            {depth, 1},
            {fail_if_no_peer_cert, true},
            {versions, ['tlsv1.2']}
      ]},

 

Thanks,

 

Adam

[Michael Klishin]

Sounds like the value of 0 is what you are looking for.

If you have a chain of

Peer => Intermediate A => Intermediate B => Root

then Intermediate A must be a trusted CA on both ends. You can still place the rest of the chain

and the root CA into the same .pem files, in theory they should be ignored.

IIRC there is no idea of "root CAs" in x509 certificates, only certificate usage restrictions (e.g. some

certificates can be used to sign others, they are CA certificates). This is to say that TLS

implementations can check for that but otherwise there is no special indication that a certificate

is "root."

https://tools.ietf.org/html/rfc5280 can clarify that and also describe the basics of certificate

chain verification.

You can implement any certificate chain verification strategy using an Erlang function but

I don't see any need for that.

Take a look at https://github.com/rabbitmq/rabbitmq-trust-store/tree/stable as well.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Adam Bloom]

Any ideas why a depth value of zero would result in the error I included in my original message? The intermediate is at the top of the CA cert file at both ends.

Thanks for the plugin link. I noticed that as well - wanted to get a static config doing what I wanted first, but then planned to take a closer look at that.

Adam

To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

--
You received this message because you are subscribed to a topic in the Google Groups "rabbitmq-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rabbitmq-users/D1-Xr0DuTOk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rabbitmq-users+unsubscribe@googlegroups.com.

[Michael Klishin]

The fact that a certificate is included into "CA certificates"

makes it trusted in some TLS implementations (I think Erlang is one of them)

but not all (Python is an exception I remember).

Why don't you try using only one CA certificate (the intermediary you want)

and making it trusted system-wide?

rabbitmq_trust_store ignores most of your TLS config. It overrides

certificate chain verification function to use a dynamic set of certificates

available from a local directory (or an HTTP(S) resource).

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.

[Michael Klishin]

Have you seen our TLS troubleshooting guide? It's of limited use in this case

because certificate chain verification algorithm can vary from implementation to implementation (just noticed RFC 5280 section 6 basically allow that) but

typically OpenSSL's s_server and s_client are very helpful in narrowing

the problem down by elimination:

http://www.rabbitmq.com/troubleshooting-ssl.html

On 5 Jun 2017, at 23:13, Adam Bloom <adam...@gmail.com> wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.

[Michael Klishin]

Managed to find some relevant functions in Erlang's public_key app:

https://github.com/erlang/otp/blob/maint-19/lib/public_key/src/public_key.erl#L716

which suggests they use the RFC 5280 algorithm.

The algorithm relies on certificate extensions and policy mappings quite a bit,

so consider posting some details about your certificate.

The max_path_length_reached error marker only used once, as expected, when the chain

is fully traversed:

https://github.com/erlang/otp/blob/maint-19/lib/public_key/src/public_key.erl#L1008

and the test suite has some examples that use self-signed certificates, with CRL and without:

https://github.com/erlang/otp/blob/maint-19/lib/public_key/test/pkits_SUITE.erl#L231

https://github.com/erlang/otp/tree/maint-19/lib/public_key/test/pkits_SUITE_data/pkits/smime-pem

If you can reproduce the same error with a chain generated with

https://github.com/michaelklishin/tls-gen/tree/master/separate_intermediates,

I'd be happy to try to dig deeper.

[Adam Bloom]

I reproduced the same error with your separate intermediate generator. Within rabbitmq, I used the server chain for CA file and the server cert/key. I then attempted to connect to that using openssl s_client (with the client chain, cert, and key) with a depth of 0 - no luck, same error. However, given that the intermediates are separate and the depth is zero, that makes sense.

Connecting to the server with the exact same cert/key as the server doesn't work either. So signed by the same intermediate CA, should be trusted. Tried with depth 1, got a new error this time:

SSL: certify: ssl_handshake.erl:1627:Fatal error: handshake failure - {bad_cert,invalid_ext_key_usage}

Didn't did into your TLS generator enough to see what's throwing that.

Example of how I was trying connections:

/usr/local/Cellar/openssl/1.0.2l/bin/openssl s_client -connect localhost:5671 -CAfile chained_server_ca_certificate.pem -cert server_certificate.pem -key server_key.pem

Any ideas?

[Michael Klishin]

You can't use client certificates for the server or vice versa, at least when they are produced

by tls-gen.

Why? When a certificate is generated it can be given a set of constraints on how it can be used

(this is called the Extended Key Usage extension). tls-gen uses that extension.

If you run

gmake info | grep -B 2 -A 2 "X509v3 Extended Key Usage"

in the tls-gen profile directory you used you'd see

X509v3 Extended Key Usage:

                TLS Web Client Authentication

for one certificate and

X509v3 Extended Key Usage:

                TLS Web Server Authentication

for the other.

[Adam Bloom]

Makes sense - just didn't check the extensions you were setting. Regardless, the "regular" case didn't work either. Were you expecting it to with depth 0? The client CA is unknown to the server, so I'd expect the error. What configuration would you try with depth 0 to get the client and server to work?

Just a note - I tried a self signed CA with two certs - depth 0 works. It seems to get wonky with the intermediate in the middle.

[Michael Klishin]

Adam,

tls-gen is a convenience tool. It generates CA bundles that include the entire chain because it's the root CA

that is used as trust anchor in most cases. That's not true in your case, so taking what tls-gen produces

as is isn't going to work.

I already mentioned the fact that as far as I know there are no "root CAs". There are CAs trusted by the peer

performing verification and those that aren't.

In other words, a self-signed root CA (the basic profile in tls-gen) should be no different from

an intermediary CA as long as it is on the trusted list on the machine that performs peer verification.

The algorithm described in RFC 5280 section 6 depends on certificate extensions and policies, which makes

it a lot more complicated but ultimately it just walks up the chain — starting from the trust anchor, a trusted certificate —

and makes sure that every chain element is a valid certificate (not expired, not revoked,

not violating any extensions, and so on).

So what I'd do is to use tls-gen or any other way of generating a chain, then taking the first intermediary you want

and using it as "root CA" that you add to the trusted set, plus leaf certificates that client and server will use. Everything above the first

intermediary shouldn't be necessary with depth = 0.

[Michael Klishin]

By the way, I suggested trying both `openssl s_client` and `openssl s_server` (with a RabbitMQ client of choice

or `openssl s_client`) to compare.

Those tools have a more intuitive definition of the depth (-verify) parameter as it covers the entire chain,

not just intermediaries (this is according to the man page of s_client and s_server).

They also print out the chain and verification status for each certificate in the chain.

[Chris]

I had the same problem. As far as I can tell, the cert was signed with the wrong capabilities.