Different roots for server and client certificates

[Sergey Krasilnikov]

Hello,

I have a case where I have 2 different roots for client and server certificates:

root1 > intermediate1.1 > intermediate1.2 > server_certificate
root2 > intermediate2.1 > intermediate2.2 > client_certificate

Those were generated using https://github.com/michaelklishin/tls-gen for simplicity.

Server SSL  configuration:

{ssl_options, [{cacertfile,"/etc/rabbitmq/keys/client/chained_ca_certificate.pem"},
                     {certfile,"/etc/rabbitmq/keys/server/server_certificate.pem"},
                     {keyfile,"/etc/rabbitmq/keys/server/server_key.pem"},
                     {depth,2},
                     {verify,verify_peer},
                     {fail_if_no_peer_cert,true},
                     {versions, ['tlsv1.2', 'tlsv1.1', tlsv1]}]}

Here is openssl output:

openssl s_client -connect localhost:5671 -cert client/client_certificate.pem -key client/client_key.pem -CAfile server/chained_ca_certificate.pem
CONNECTED(00000003)
depth=3 CN = MyTestRootCA
verify return:1
depth=2 CN = MyIntermediateCA1
verify return:1
depth=1 CN = MyIntermediateCA2
verify return:1
depth=0 CN = illapikov, O = server
verify return:1
139799658641048:error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca:s3_pkt.c:1472:SSL alert number 48
139799658641048:error:140790E5:SSL routines:ssl23_write:ssl handshake failure:s23_lib.c:177:

Server output into logs:

=ERROR REPORT==== 30-Jan-2017::18:04:04 ===
SSL: certify: ssl_handshake.erl:1606:Fatal error: unknown ca

I assume server does not trust client, but all configuration seems to be OK and 'almost' like in example.

Any comments would be appreciated.

Thank you in advance,
Sergey

[Michael Klishin]

If you use different root CAs they must be trusted on the peer machine.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sergey Krasilnikov]

I've added root CAs to the Ubuntu trust store, but it still is not working.

In case where I have next structure do I need to add intermediate CAs to Ubuntu trust store:

root > intermediate1 > server_certificate
root > intermediate2 > client_certificate

Also, I've tried simple case without intermediates and it was working without adding it anywhere:

root1 > server_certificate
root2 > client_certificate

Any comments and thoughts would be appreciated.

[Michael Klishin]

Well, I don't know what exactly you did but server certificate's root CA must be trusted

on client hosts and client root CAs must be trusted by RabbitMQ node hosts.

There was a lengthy discussion that's not the same case as yours but

provides some hints:

https://groups.google.com/forum/#!searchin/rabbitmq-users/intermediate$20CA%7Csort:relevance/rabbitmq-users/KApzsHndBaY/wDt4EvlwCQAJ

One of them is a TLS troubleshooting guide:

http://www.rabbitmq.com/troubleshooting-ssl.html

If removing intermediate certificates makes a difference, perhaps you need

to take a look at the verification depth used, see Certificate Chains and Verification Depth

in http://www.rabbitmq.com/ssl.html.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Sergey Krasilnikov]

The depth for 2 intermediate certificates was 2 (which is correct according to the docs).

I managed to make it work by concatenating all certificates in a way:

cat client_intermediate_ca root_ca server_intermediate_ca > all.pem

I've set it as a 'cacertfile' in RabbitMQ configuration and in 'CAfile' for openssl s_client.

Is that an expected CA chain?

Also, thank you for your quick response.

[Michael Klishin]

Chains go from the least "authoritative" certificates up to the most authoritative (the root).

So usually you'd have a leaf certificate at the top and the root at the bottom. In practice this can vary
from tool to tool but I believe most tools, e.g. Nginx, use this order:
http://nginx.org/en/docs/http/configuring_https_servers.html#chains

Also you can provide your own chain verification function — that's what our trust-store plugin uses
— in which case the traversal order or logic applied can be anything. Just mentioning it
because it's not obvious unless you read TLS RFCs to wind down ;)

> >> For more options, visit https://groups.google.com/d/optout.
> >>
> >
> >
> >
> > --
> > MK
> >
> > Staff Software Engineer, Pivotal/RabbitMQ
> >
>

> --
> You received this message because you are subscribed to the Google Groups "rabbitmq-users"
> group.
> To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.

> To post to this group, send an email to rabbitm...@googlegroups.com.

[Michael Klishin]

Right, so if you have a working example, can you please provide us with a shell transcript
of what you had to do (including tls-gen's `make` runs). I then can update the docs
to include an example. Previously we didn't mention tls-gen in the docs but now that we do,
there is no reason why we can't also use it to provide more specific examples in "advanced"
sections. 

On 31 January 2017 at 18:45:40, Sergey Krasilnikov (illapi...@gmail.com) wrote:

> >>> wrote:
> >>>
> >>> Hello,
> >>>
> >>> I have a case where I have 2 different roots for client and server
> >>> certificates:
> >>>
> >>> root1 > intermediate1.1 > intermediate1.2 > server_certificate
> >>> root2 > intermediate2.1 > intermediate2.2 > client_certificate
> >>>
> >>> Those were generated using https://github.com/michaelklishin/tls-gen for
> >>> simplicity.
> >>>
> >>> Server SSL configuration:
> >>> {ssl_options, [{cacertfile,
> >>> "/etc/rabbitmq/keys/client/chained_ca_certificate.pem"},
> >>> {certfile,
> >>> "/etc/rabbitmq/keys/server/server_certificate.pem"},
> >>> {keyfile,"/etc/rabbitmq/keys/server/server_key.pem"
> >>> },
> >>> {depth,2},
> >>> {verify,verify_peer},
> >>> {fail_if_no_peer_cert,true},
> >>> {versions, ['tlsv1.2', 'tlsv1.1', tlsv1]}]}
> >>>
> >>>
> >>>
> >>>
> >>> Here is openssl output:

> >>> openssl s_client -connect localhost:5671 -cert client/client_certificate
> >>> .pem -key client/client_key.pem -CAfile server/chained_ca_certificate.
> >>> pem

> >> For more options, visit https://groups.google.com/d/optout.
> >>
> >
> >
> >
> > --
> > MK
> >
> > Staff Software Engineer, Pivotal/RabbitMQ
> >
>

> --
> You received this message because you are subscribed to the Google Groups "rabbitmq-users"
> group.
> To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.

> To post to this group, send an email to rabbitm...@googlegroups.com.

[Sergey Krasilnikov]

So, the last example was not from tls-gen tool, but manually.

I've create a couple of gists to show how things are working.

https://gist.github.com/Illapikov/5940c1ad39e113bf2befe41eed274158 - certificates creation with minimal configuration possible.

https://gist.github.com/Illapikov/315efcbf12b863e136519d9107d9b20d - proof that these certificates are fine enough to work with openssl s_server & s_client.

https://gist.github.com/Illapikov/7df7cd2df96bd771841c82c9b0992fc0 - RabbitMQ cases.

In my opinion, I'd expect RabbitMQ certificates configuration to be the same as for openssl s_server, but it is not working.

Any thoughts?

[Michael Klishin]

I'd expect all TLS implementations to interoperate perfectly, approach every single

decision the same way, and support every cipher suite and extension ever known to mankind

but it doesn't work that way in practice.

Erlang's TLS implementation only uses OpenSSL for crypto functions. This has its

benefits, too: many of the OpenSSL TLS state machine exploits do not affect it.

Your "weird but working" example bundles every certificate generated

into a single file, so the problem is somewhere in how the certificates

are bundled, traversed, or trusted, and not certificates themselves or oddly manifested

cipher suites incompatibilities.

We extracted tls-gen from our test suites and ask to be used it for troubleshooting for a reason: it drastically

reduces the amount of time spent investigating environment-specific problems, which is

well over 90% of TLS-related questions on this list and in general.

So consider making things work with tls-gen generated pairs first.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Sergey Krasilnikov]

Well, I did some work in this area.

https://github.com/Illapikov/tls-gen/tree/different-chains

In the end this works the same way as my generated certificates.

[Sergey Krasilnikov]

A bit more details about this branch:

I've changed structure to have client and root certificates signed with different intermediate certificates (which is my case).

root - intermediate_ca1 - client and root - intermediate_ca2 - server.

In my configuration server trusts intermediate_ca1 + root, while client trusts intermediate_ca2 + root.

Unfortunate, this was not working for some reason. Error message was still 'unknown ca' and nothing else.

Depth is 1, so no issues with that.

Any thoughts? Can you please verify that this behavior is correct or not?

Thank you in advance.

[Michael Klishin]

Hi Sergey,

I'm afraid our team cannot afford to try reproducing every single TLS issue on this list.
We provide a troubleshooting guide, tools such as tls-gen and general advice as time permits but reproducing every case
it too time consuming for our small team.

I'm confused by your proposed configuration. Root certificates are not signed by any intermediates
(that's why they are root certificates). Do you mean that your fork uses two different root certificates?

If so, then both client and server must trust each other's (or both) root CAs.
Every OS has a list of certificates that it trusts (on Linux it's typically a protected directory, Windows and OS X have
dedicated applications). Have you considered adding your root CAs there?

It shouldn't take long to try verification depth of more than 1.

[Michael Klishin]

Note that it is perfectly fine to make an HAproxy or similar to perform TLS termination. If that's an easier
or more familiar way, just do it. 

On 2 February 2017 at 12:53:31, Sergey Krasilnikov (illapi...@gmail.com) wrote:

[Michael Klishin]

That said, we are willing to invest some time into introducing more "profiles" into tls-gen

because that's an investment that will help more than one user in the future.

I'll take a look at adding the following:

 * root CA => intermediate1 => client certificate/key pair

 * root CA => intermediate2 => server certificate/key pair

plus concatenated files and give it a try with Bunny and `openssl s_client`.

> To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.

> To post to this group, send an email to rabbitmq-users@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.
>

--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[Sergey Krasilnikov]

Looking forward to see our opinion about this case.

You can start with my changes: https://github.com/Illapikov/tls-gen/tree/different-chains

 * root CA => intermediate1 => client certificate/key pair

 * root CA => intermediate2 => server certificate/key pair

> To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.

> To post to this group, send an email to rabbitm...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

--
MK

Staff Software Engineer, Pivotal/RabbitMQ

--

[Sergey Krasilnikov]

Hello Michael,

Any progress on this one so far?

Do you need some help with integration this example into tls-gen? 

Do you have any ETA for this?

Looking forward with hope,

Sergey

[Michael Klishin]

I will not provide an ETA. Please adjust your expectations, all I agreed

on was adding one more profile/scenario to tls-gen that I'd verify. This is by no

means a priority, in particular when we have a release delayed by several weeks.

Our small team cannot afford getting to the root cause of every TLS question on this list.

We've previously recommended using HAproxy or similar for TLS termination as an alternative.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Michael Klishin]

I would rather not bet your project's future on a response from someone on a community mailing list.

Use HAproxy (if you cannot get that to work as well then it's a good indication that

it's not an issue with RabbitMQ or Erlang/OTP) for TLS termination.

To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Michael Klishin]

I've added a third profile to tls-gen:

https://github.com/michaelklishin/tls-gen/blob/master/README.md

Haven't had a chance to test the produced artifacts yet.

[Sergey Krasilnikov]

Good day, Michael

Thank you for your work.
Currently I have some room to wait your comments, so there is no any push.

Last time I misunderstood and was excited too much :)

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.

To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Dmitry Andrianov]

Michael,
My understanding is that HAProxy is not an option if you want to verify client certificate subject on the broker to do any access control based on client certificate

I am afraid there IS an issue with RabbitMQ+Erlang/OTP combo. Rabbit's role in that issue is minimal, but it still plays its part. I can easily prove that just upgrading erlang from R16 to R17 without touching a single bit of RabbitMQ or its configuration results in these "unknown ca" errors and RabbitMQ server stopping accepting SSL connections.
The breaking change was introduced in Erlang 3 years ago so, to be honest, I am really surprised you are not getting lots of similar reports. Probably there are not too many people out there using SSL client auth with intermediate certificates when connecting to RabbitMQ. Or they did not have to migrate from R16 to newer version. I even found my own thread 3 years ago where I ran into that problem - https://groups.google.com/forum/#!topic/rabbitmq-users/7vpFGpvPh4w That time I just gave up when found that it is R17 causing issues and decided to stick with R16 without digging deeper.

So, the problem:
* In R17, Erlang team decided to "fix" certificate validation and now the client is REQUIRED to send the entire cert chain to the server, not just its certificate alone. If there is an intermediate certificate, client MUST present it.
* it does not matter if client's intermediate is in the server's trusted list (cacertfile) - if client does not send its intermediate over the SSL connection, you will see "unknown ca" in the server log

This is where I found about this - http://erlang.2086793.n4.nabble.com/Figuring-out-proper-ssl-certificate-settings-with-17-3-td4710363.html

To my personal taste, they should not have done breaking changes like that. Or at least a workaround should have been provided as it may be just impossible to update already deployed clients. I 100% agree to every argument the guy who started that thread has. But it does not look Erlang team listened to him.

Now, where is Rabbit's role in all this. (You probably inherit 100% of that from Erlang but still...) There is only one trust option - "cacertfile" which used for two unrelated things:
* to provide server's cert chain. (only one certificate is read from "certfile" option. If I want broker to announce its intermediates certs too - I need to include them into "cacertfile")
* it also serves as a list of trusted CAs that client certificates must be signed with. These will be announced to the client over SSL connection during handshake and any client certificates signed by these will be accepted.

Why is this bad? Because I lose control to what I trust and what I do not. Imagine two chains

* Server's Root => Brokers => broker1
* Client's Root => AMQP Clients => client1

The broker is configured with broker1 but I also have to put its intermediate "Brokers" into cacertfile to make broker announce its full chain not just "broker1".
Now for that broker to trust my clients, I also need to include "Client's Root" into the very same file. I would include "AMQP Clients" but as I mentioned above, it won't be trusted anyway - I need to include "Client's Root" and make clients send their intermediate "AMQP Clients" during handshake.
In reality I want to accept only certificates signed with "AMQP Clients" CA but because of my cacertfile configuration, I will be accepting anything signed either by "Brokers" (which is not supposed to be a client certificate at all) or "Client's Root" (which can be something else, like Client's Root => Firmware CA => fw01)

Ideally, server chain and CAs that we trust clients from should be configured independently of each other.

However this issue is not that big as inability to accept connections from clients not sending the whole chain. The thread http://erlang.2086793.n4.nabble.com/Figuring-out-proper-ssl-certificate-settings-with-17-3-td4710363.html has some discussion on how to work around that but to be honest I have no idea how to apply this to RabbitMQ


Cheers
Dmitry

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.

To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

--

[Michael Klishin]

Hi Dmitry,

The x509 certificate-based authentication mechanism scenario is a good point.

I did a few things recently:

 * Added a profile that replicates certificate chains described in this thread [1]

 * Ported tls-gen to Python so that it is much easier to reuse code [2] and add more profiles

 * Tested TLS connections with the certificates and keys in [1][3]

As you can see, in [3], both `openssl s_client` and Bunny succeed with root + intermediate

CA bundle files. I didn't have to bundle all CA (or even all period) certificates is a single bundle

to get it to work. This is on OS X Sierra, Erlang 19.2.1 and with OpenSSL 0.9.zh, Ruby 2.3.1.

I encourage others to use tls-gen to generate the same certificates/key and try for themselves.

Now that tls-gen is a more extensible tool, there is no reason to not use it more. In fact, it's come full

circle and is used in some of RabbitMQ test suites, potentially replacing the original Makefile generator

some day.

1. https://github.com/michaelklishin/tls-gen/tree/master/separate_intermediates

2. https://github.com/michaelklishin/tls-gen/tree/master/tls_gen 

3. https://gist.github.com/michaelklishin/8dd18f810ec7a9d0a4c2c828566b13d3

hivehome.com

Hive | London | Cambridge | Houston | Toronto

The information contained in or attached to this email is confidential and intended only for the use of the individual(s) to which it is addressed. It may contain information which is confidential and/or covered by legal professional or other privilege. The views expressed in this email are not necessarily the views of Centrica plc, and the company, its directors, officers or employees make no representation or accept any liability for their accuracy or completeness unless expressly stated to the contrary. 

Hive is the trading name of Centrica Connected Home Limited (company no: 5782908), registered in England and Wales with its registered office at Millstream, Maidenhead Road, Windsor, Berkshire SL4 5GD.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Dmitry Andrianov]

Michael,
apologies for the late reply - I was on holidays.
Your example does work as expected but you ARE sending the intermediary ones because your chains have it!
This command:

openssl s_client -tls1 -connect 127.0.0.1:5671 -CAfile ./chained_client_ca_certificate.pem -cert ./client_certificate.pem -key ./client_key.pem -verify 2

will fail if you replace your -CAfile option with -CAfile ./ca_certificate.pem (keeping the -verify 2) or do

openssl s_client -tls1 -connect 127.0.0.1:5671 -CAfile ../intermediate_ca_server/certs/cacert.pem -cert ./client_certificate.pem -key ./client_key.pem

or omit the -CAfile completely (without -verify, openssl will just tell you it is unhappy with the remote side but still establishes the connection).
But in any of these cases, it is the server making the decision to abort the connection. And this gets logged:

=ERROR REPORT==== 28-Feb-2017::22:54:20 ===
SSL: certify: ssl_alert.erl:92:Fatal error: unknown ca

I haven't tested your Bunny example but I believe it is the same story there because of this line - https://gist.github.com/michaelklishin/8dd18f810ec7a9d0a4c2c828566b13d3#file-chains-rb-L13 The SSL library will include client intermediate CA into the chain and send it together with the client certificate. (The chained_client_ca_certificate has client's intermediate_ca_client in it)

And with Erlang 16 that was not required - the client only needed its own cert + key and the server intermediate in the cacertfile to trust the server. And server, configured with client's intermediate in its cacertfile would accept that connection.

Now the problem is that some clients may be extremely difficult/painful to update which leaves no option but to do some hacks on the RabbitMQ side. And it is not even obvious what to do in this case. Is that doable with a plugin? The last thing in the world I want is running a custom built version of RabbitMQ broker :)

Cheers

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.

To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dmitry Andrianov]

Hi again, Michael.

So do you think there are any ways of changing how certificate chain is validated that would make it possible to accept connection from a client that does not supply the intermediate cert?

If it can be done with configuration that would be great. Custom plugin still doable. Running a custom patched RabbitMQ is less fun :)

Thanks!

[Michael Klishin]

you can provide a custom verify_fun, that's how rabbitmq-trust-store operates.

Staff Software Engineer, Pivotal/RabbitMQ

[Michael Klishin]

To be more specific: search for "verify_fun" on http://erlang.org/doc/man/ssl.html

and take a look at https://github.com/rabbitmq/rabbitmq-trust-store/blob/7534df870136e5d5ed8af851899beab175eb843f/src/rabbit_trust_store_app.erl#L65

for a specific example.

rabbitmq-trust-store verifies leaf certificates by checking them against a predefined list

of certificates that can be obtained from a local filesystem or HTTPS sources

(in the tip of stable).

HTH.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Staff Software Engineer, Pivotal/RabbitMQ

[Sergey Krasilnikov]

Hi again Michael,

I'm sorry to bring this topic once again, but could you please clarify one to me?

I've been looking at https://github.com/michaelklishin/tls-gen and wondering why it has to be the same Root CA for both client and server?

Thanks in advance.

Sergey

> >>>> > >> I've added root CAs to the Ubuntu trust store, but it still is not<br class="m_-8585800597342799526gmail_ms

[Michael Klishin]

It doesn't have to be as long as

 * The client trusts the server CA (its in the trusted certificate store on the client machine)

 * The server trusts the client CA (same idea)

Having both share a CA makes both items above true in most cases even without explicitly adding any certificates

to the trust store (Python is an exception)

--

You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

[Sergey Krasilnikov]

Thanks for your reply, Michael

Could you please elaborate on your idea with trust store for RabbitMQ server?

I just thought that simple configuration as documentation would be enough.

Currently, we have pretty much the same.

Our desire is to have server certificate from one CA and clients from another CA.

I've tried my certificates with openssl s_client and s_server and it was working perfectly, but when I've changed RabbitMQ configuration server refused to trust client's CA (also, configuration was literally the same).

I can provide extra: how I generated certificates, what RabbitMQ configuration looked like and what openssl commands I executed.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Luke Bakken]

Hi Sergey,

A "trust store" is a generic term for a collection of trusted certificates, or CA certificates. For instance, Windows and OS X provide system-wide collections of certificates that are trusted and manages these certificates using special tools. On Linux, the "trust store" is usually a file containing many certificates and stored somewhere. On my Arch Linux workstation this file is /etc/ssl/cert.pem

You can add your own CA files to the system-level trust stores if needed. A simpler method is to concatenate the files into a new one. You should be able to concatenate the two CA files into a single one, like this:

cat ca_file_one.pem ca_file_two.pem > ca_certificates.pem

Then, you can use ca_certificates.pem as the argument to cacertfile for RabbitMQ and as the CA file for your client applications.

Thanks,

Luke

[Sergey Krasilnikov]

Hi Luke,

Thanks for your reply.

The thing is that was exactly what I've done - I've concatenated Client Intermediate CA with Client Root CA and passed to the RabbitMQ sever as cacertfile.

In addition to that I've concatenated Server Intermediate CA with Server Root CA and used it as CAfile for openssl s_client (which I've used to connect to RabbitMQ).

And, for some reason that didn't work - RabbitMQ server did not accept client certificate.

[Luke Bakken]

Hi Sergey,

Have you provided your complete RabbitMQ configuration file? In a previous message, you mentioned that your configuration worked successfully if you set the "depth" option:

https://groups.google.com/d/msg/rabbitmq-users/1Pyowi3exro/nuG07Ud9DQAJ

Please provide the configuration you are using. I don't have a set of certificates that use intermediate CAs at the moment, but I'll see about adding it to tls-gen.

Thanks,

Luke

[Sergey Krasilnikov]

Hi Luke,

It was working not because of depth option, but because I've concatenated both Server Root CA and Client Root CA, plus their intermediates into one file and specifying them as CA file for both server and client.

[

  {rabbit, [

      {loopback_users, []},

      {default_user, <<"guest">>},

      {default_pass, <<"guest">>},

      {vm_memory_high_watermark, 0.7},

      {auth_mechanisms, ['PLAIN','EXTERNAL']},

      {ssl_listeners, [5671]},

      {auth_backends, [rabbit_auth_backend_internal]},

      {ssl_handshake_timeout, 15000},

      {ssl_options, [{cacertfile,"/home/illapikov/rabbitmq/client/int+root.crt"},

                     {certfile,"/home/illapikov/rabbitmq/server/broker.crt"},

                     {keyfile,"/home/illapikov/rabbitmq/server/broker.key"},

                     {verify,verify_peer}

   ]},

  {rabbitmq_management, [{load_definitions, "/etc/rabbitmq/definitions.json"}]}

].

Server and client certificates:

ls server/

broker.crt  broker.csr  broker+intermediate.crt  broker.key  intermediate.crt  intermediate.csr  intermediate.key  int+root.crt  root.crt  root.key

ls client/

client.crt  client.csr  client.key  intermediate.crt  intermediate.csr  intermediate.key  int+root.crt  root.crt  root.key

And here is openssl s_client command:

openssl s_client -connect localhost:5671 -cert client/client.crt -key client/client.key -CAfile server/int+root.crt

And as a result I have RabbitMQ server complaining about client having unknown CA.

[Luke Bakken]

Hi Sergey,

Thanks for providing your configuration. Could you please re-add the depth setting to see if that resolves your issue? Give the following a try:

{depth, 4}

within the ssl_options section.

I am going to find time to add an intermediate CA profile to the tls-gen project.

Thanks,

Luke

On Monday, April 16, 2018 at 1:07:28 AM UTC-7, Sergey Krasilnikov wrote:

Hi Luke,

[Sergey Krasilnikov]

Hi Luke, 

{depth, 4} didn't help at all, error is still the same - Unknown CA.

Also, I'm not sure I was clear enough - these root.crt are different from each other.

openssl x509 -in client/root.crt -text

Certificate:

    Data:

        Version: 3 (0x2)

        Serial Number: 12089268137388763119 (0xa7c5b551bb8167ef)

    Signature Algorithm: sha256WithRSAEncryption

        Issuer: CN=Client Root CA

        Validity

            Not Before: Apr 12 14:17:09 2018 GMT

            Not After : Apr  9 14:17:09 2028 GMT

        Subject: CN=Client Root CA

openssl x509 -in server/root.crt -text

Certificate:

    Data:

        Version: 3 (0x2)

        Serial Number: 16915780918223294684 (0xeac0ed1f1907d0dc)

    Signature Algorithm: sha256WithRSAEncryption

        Issuer: CN=Server Root CA

        Validity

            Not Before: Apr 12 14:16:26 2018 GMT

            Not After : Apr  9 14:16:26 2028 GMT

        Subject: CN=Server Root CA

In tls-gen there is already an example with Intermediate CAs, but Root CA is always the same.

Not so sure, but it feels like RabbitMQ server wants to know not only about Client CAs, but about Server CAs as well.

[Luke Bakken]

Hi Sergey,

Thanks for clarifying. I re-reviewed this mailing list thread - can you confirm this is your setup?

Root CA #1 -> Intermediate CA #1 -> Server Certificate

Root CA #2 -> Intermediate CA #2 -> Client Certificate

Finally, your goal is to use client certificate authentication to connect applications to RabbitMQ, correct?

Thanks for being patient!

Luke

On Monday, April 16, 2018 at 10:37:13 AM UTC-7, Sergey Krasilnikov wrote:

Also, I'm not sure I was clear enough - these root.crt are different from each other.

Not so sure, but it feels like RabbitMQ server wants to know not only about Client CAs, but about Server CAs as well.

[Sergey Krasilnikov]

Exactly, that is my setup.

Yes, correct.

Thank you for helping me :) 

[Sergey Krasilnikov]

Hi Luke,

Do you need anything from me in addition what I've provided already?
Like openssl commands to create certificates or anything else?

Thanks, Sergey

[Luke Bakken]

Hi Sergey,

I have been working on higher priority items. I've started a branch of tls-gen with some changes if you'd like to follow along - https://github.com/lukebakken/tls-gen/tree/lrb-separate-root-and-intermediate

I should have time to investigate further today or tomorrow.

Thanks,

Luke

[Michael Klishin]

Note that tls-gen is not a particularly sophisticated tool. It's just automation of commonly

performed operations on common certificate chain

patterns using openssl CLI tools.

Contributions (including additional profiles) are welcome.

--

You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Luke Bakken]

Hi Sergey -

I have been working on this and have been taking notes here - https://gist.github.com/lukebakken/582806c8844d3fc08c22e898b272325e

Basically, what I am finding is that when client certificate authentication is used, the client must send it's certificate and the intermediate CAs during the handshake phase (you reported the same thing here - https://groups.google.com/d/msg/rabbitmq-users/1Pyowi3exro/89mgJjDSBQAJ)

Including the Intermediate CAs for the client certificate in the server-side CA bundle doesn't work. I'm not sure why, but it's something in how the Erlang TLS implementation works.

Let me know if you have further questions. There are a few more scenarios I'm going to try and will update that gist with my results.

Thanks,

Luke