RabbitMQ Mutual Authentication

[Domenico Amelino]

Dear All,

I have installed RabbitMQ on a Google Cloud Platform in order to use the Rabbit MQTT Message Broker together with Agosto IoT Adapter in order to connect IoT Devices to the Google Cloud Pub/Sub Service.

The device/client part is able to connect to the Rabbit MQTT Broker easily just following the instruction and setting up a MQTT Client, but i am currently having issues to the mutual ssl connection to the RabbitMQ message broker. guessing that i did some mistakes in Broker Configurations, i restarted from the default settings in order to setup the RabbitMQ Broker well. 

In order to be clear, i will describe my operational steps:

1. I installed tls-gen  from Git and i used it for generate custom CA, Server and Client Ecc-based key pairs and certificate. 

2. Then i created rabbit.config file:

[
 {rabbit,  [ 
    {ssl_listeners, [5671]},
    {ssl_options, [{cacertfile,"D:/RabbitMQ/certs/MyCA.pem"},
                  {certfile,"D:/RabbitMQ/certs/MyRabbitServer.pem"},
                  {keyfile,"D:/RabbitMQ/certs/MyRabbitServer.key"},
                  {verify,verify_none},
                  {fail_if_no_peer_cert,false}]}
  ]}
].

My focus is to establish a mutual authentication, but i started with this setting in order to allow the connection to clients without a certificate. Of course i have already check all the paths and they are correct.

3. I created an user named "ssltest" and gave permissions:

sudo rabbitmqctl add_user ssltest password

sudo rabbitmqctl set_permissions ssltest ".*" ".*" ".*"

4. So, before going on with the setting, i tried to check if my settings were good and i follow the description in https://www.rabbitmq.com/troubleshooting-ssl.html.

    But my check fails at this point: 

   Attempt SSL connection to broker

   Once you have a RabbitMQ broker listening on an SSL port you can again use the OpenSSL s_client to verify SSL connection establishment, this time against the broker. This check establishes whether the broker is likely     to be configured correctly, without needing to configure an AMQPS client. The example assumes a broker with an "ssl_listeners" configuration directive set to listen for SSL connections on localhost port 5671:

 openssl s_client -connect localhost:5671 -cert client/cert.pem -key client/key.pem \
  -CAfile testca/cacert.pem

And i am still stuck to this point. I don't know what point i missed and i am trying to figure out but since i am not including any complicated authentication of credentials, i don't know why i'm having this kind of trouble. 

Kind Regards,

Domenico. 

[Michael Klishin]

Your config only contains TLS configuration for AMQP (0-9-1, 1.0)

but not MQTT. See MQTT plugin docs for an example that is MQTT-specific.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Domenico Amelino]

Thank you for your answer. I enabled the mqtt_plugin:

rabbitmq-plugins enable rabbitmq_mqtt

Then, i created the client certificate signed by the same custom CA installed into the server. My configuration setting is now this:

[

 {rabbit,  [

        {tcp_listeners, [5672] },

        {ssl_listeners, [5673] },

        {auth_mechanisms, ['EXTERNAL', 'PLAIN']},

        {ssl_options, [{cacertfile,"/home/domenico_amelino/ca_certificate.pem"},

                  {certfile,"/home/domenico_amelino/server_certificate.pem"},

                  {keyfile,"/home/domenico_amelino/server_key.pem"},

                  {verify,verify_none},

                  {fail_if_no_peer_cert,false}]}

  ]},

  {rabbitmq_mqtt, [

                  {ssl_listeners,    [8883]},

                  {tcp_listeners,    [1883]}

                  ]}

].

But i don't understand what are the credentials that i have to use for the mqtt client (since i am using ssl authentication) and if i need to create a specific user. 

P.S. I fixed the trubleshotting with ssl using RSA-based certificates instead of ECC-based cryptography, did i something wrong or there is some issues in the ecc support?

Kind Regards,

Domenico. 

[Michael Klishin]

RSA-based certificates are fine, as are ECC-based ones (on Erlang 19.3, earlier versions

have fairly significant limitations around ECC implementation).

To use TLS with MQTT you need to

 * Enable the plugin

 * Configure a TL listener

You can also enable peer verification (authentication) via TLS listener options.

To *only* use TLS verification for authentication, one'd normally use https://github.com/rabbitmq/rabbitmq-auth-mechanism-ssl

but it is for AMQP 0-9-1, which has the idea of different authentication mechanisms.

MQTT does not (hey, authentication is entirely optional in MQTT, we are all friends on the Internet, right?) but there is

a workaround that you should be able to use.

If you enable TLS peer verification and ensure that it works you then can rely on MQTT plugin's default

credentials (or even try optional authentication with username/password pairs). This is covered

in http://www.rabbitmq.com/mqtt.html. Of course, if you do this, your plugin must not have

TCP listeners configured as leaving them enabled would allow anyone to connect

anonymously.

Let me know if my explanation isn't very clear.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Michael Klishin]

And while it is probably obvious to you, in case someone else discovers
this thread: configuring the server to reject clients without certificates
(by setting `rabbit. fail_if_no_peer_cert` to true) is also important

for TLS/x509-only authentication scenarios.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Domenico Amelino]

Thank you for the answer, now i'll start to try as you have described. 

Ok, clear, so i should be able to set up a mutual authentication using server key and certificate installed on the Server together with the Trusted CA. It's not important if clients without a valid certificate will not be able to connect, that's fine for my test. 

What kind of configuration do you think i have to set in the rabbitmq.config? 

Kind Regards,

Domenico. 

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Michael Klishin]

Roughly what you have + require peer verification + reject clients without certificates.

MQTT plugin has default credentials of guest/guest, perhaps create a new user, configure its vhost

access and permissions and use them for credentials. This will support connections

that are not from localhost (see http://www.rabbitmq.com/access-control.html for details).

Anonymous authentication is something you can try but it makes me cringe when I recommend it :)

Then configure your MQTT clients to performs TLS peer verification,

use a set of known credentials (or anonymous authentication)

and try to break the setup e.g. by producing a separate set

of certificates (signed by a CA your server does not trust).

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Domenico Amelino]

Unfortunately..It doesn't work on my settings, I'll send you all the configuration of the Broker. 

I tried to use a client mqtt which supports the tls/ssl client authentication and it doesn't work, the connections on 8883 ports fail. Otherwise, despite the settings on other port, for example 1883, client success to connect using credentials. 

The Macbook-Pro account has the same name of the common name inside the client Certificate, and I used the same RootCA to sign both the Client and the Server Certificate. 

This is my current config setup. 

I guess is something with the port, because If I try to check che logger in var/log/rabbitmq, when he doesn't show any connection refusal on the mqtt/ssl ports. 

There are maybe other log file that I can check to get more informations? 

I really hope you know what I'm doing wrong with the Broker Settings, this mutual mqtt authentication it's tricky to reach. 

Kind Regards, Domenico. 

[Michael Klishin]

I'm afraid I don't have the time (or enough information about your environment, apps and so on) to tell you what specifically is wrong.

Most TLS issues are environment-specific and take some time to dig out.

Our team did produce tls-gen, a troubleshooting guide that uses OpenSSL and reasonably extensive documentation

on TLS in RabbitMQ.

If I had to investigate this I'd do the following:

 * Make sure that your config is valid and loaded (covered in http://www.rabbitmq.com/configure.html, `rabbitmqctl environment`)

 * Make sure that the TLS listener is actually enabled (`rabbitmqctl status`, management UI lists enabled listeners)

 * Use `openssl s_client` and `s_server` to see if you can open a TLS connection to a given port and whether it succeeds with your particular certificate/key pair: http://www.rabbitmq.com/troubleshooting-ssl.html.

 

Taking a traffic capture with Wireshark can reveal a lot of useful information as well:

https://www.rabbitmq.com/amqp-wireshark.html. Wireshark can decrypt traffic if you give it

a private key.

Once your MQTT client can connect using TLS, you can proceed to using peer verifications

and the kind of credentials you want.

--

You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Michael Klishin]

RabbitMQ won't log refused TCP connections. It will, however, log

 * Any inbound TCP connection on a client listener port that sends at least some data

 * What ports it binds to (what listeners it starts, in RabbitMQ parlance) on boot

 * Any AMQP 0-9-1, AMQP 1.0, MQTT or STOMP connections that fail e.g. due to failed authentication or authorization (vhost access)

 * Any TLS alert (e.g. peer verification issue) during TLS upgrade

Wireshark is the best way to track down refused TCP connections.

Don't forget that configuration changes require a node restart.

--

You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Domenico Amelino]

I tried more or less everything, and i'm almost sure that the setting on the device and the broker are right. 

I checked the port configurations and looks like the issues are on my cloud platform, since i'm using Rabbit MQ on a Google Platform for the Agosto IoT Adapter.

Do you know if there are some specific settings for this kind of environment?

Thank you. 

Il giorno giovedì 13 luglio 2017 20:52:42 UTC+2, Michael Klishin ha scritto:

RabbitMQ won't log refused TCP connections. It will, however, log

 * Any inbound TCP connection on a client listener port that sends at least some data

 * What ports it binds to (what listeners it starts, in RabbitMQ parlance) on boot

 * Any AMQP 0-9-1, AMQP 1.0, MQTT or STOMP connections that fail e.g. due to failed authentication or authorization (vhost access)

 * Any TLS alert (e.g. peer verification issue) during TLS upgrade

Wireshark is the best way to track down refused TCP connections.

Don't forget that configuration changes require a node restart.

On Thu, Jul 13, 2017 at 9:37 PM, Domenico Amelino <domenico...@gmail.com> wrote:

Unfortunately..It doesn't work on my settings, I'll send you all the configuration of the Broker. 

I tried to use a client mqtt which supports the tls/ssl client authentication and it doesn't work, the connections on 8883 ports fail. Otherwise, despite the settings on other port, for example 1883, client success to connect using credentials. 

The Macbook-Pro account has the same name of the common name inside the client Certificate, and I used the same RootCA to sign both the Client and the Server Certificate. 

This is my current config setup. 

I guess is something with the port, because If I try to check che logger in var/log/rabbitmq, when he doesn't show any connection refusal on the mqtt/ssl ports. 

There are maybe other log file that I can check to get more informations? 

I really hope you know what I'm doing wrong with the Broker Settings, this mutual mqtt authentication it's tricky to reach. 

Kind Regards, Domenico. 

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Michael Klishin]

No and besides firewalls, there are very few infrastructure-specific bits that affect

RabbitMQ.

Well, some IaaS'es have less CPU throttling and more predictable network bandwidth/quality

than others but even in the worst case scenario (which is not Google Cloud, by the way) it should

not prevent connections from going through.

I don't see a way around using tcpdump on the server to find out what's going on.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Domenico Amelino]

Ok, my current settings on rabbitmq.config are:

[ {rabbit, [ {tcp_listeners, [5672] }, {ssl_listeners, [5673] }, {loopback_users, []}, {auth_mechanisms, ['EXTERNAL']}, {ssl_options, [{cacertfile,"/home/domenico_amelino/ca_certificate.pem"}, {certfile,"/home/domenico_amelino/server_certificate.pem"}, {keyfile,"/home/domenico_amelino/server_key.pem"}, {verify,verify_none}, {fail_if_no_peer_cert,false}]} ]}, {rabbitmq_mqtt, [ {default_user, <<"guest">>}, {default_pass, <<"guest">>}, {allow_anonymous, true}, {vhost, <<"/">>}, {tcp_listeners, [1884]}, {ssl_listeners, [1883]}, {ssl_cert_login, true} ]}].

On a client on another remote device i'm using openssl with client data and certificate authority data (all the stuff is generated with tls-gen).

damelino@INT-COM-119:/mnt/c/Users/amelino/Desktop/gcloudtest$ openssl s_client -connect 35.184.162.224:1883 -cert client_cert.pem -key client_key.pem -CAfile ca_cert.pem

CONNECTED(00000003)

139896077485728:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:177:

---

no peer certificate available

---

No client certificate CA names sent

---

SSL handshake has read 0 bytes and written 295 bytes

---

New, (NONE), Cipher is (NONE)

Secure Renegotiation IS NOT supported

Compression: NONE

Expansion: NONE

---

At the same time, i'm running tcpdump On my server on the Google Cloud Platform, and this is the output.

Looks like the checksum check fails for the first two messages made by Rabbit MQ broker. It's very weird and i'm still stuck. 

 Thank you for the attentions. 

[Michael Klishin]

It's a red herring (and TCP checksums can be disabled: https://wiki.wireshark.org/TCP_Checksum_Verification).

Your config disables peer verification and allows clients without certificates. I'm pretty sure

certificates won't be exchanged in that case (and openssl s_client output confirms that by saying

that no certificate names were sent and no peer certificate is available).