rabbitmq SSL uses guest and so can't connect outside of localhost

[Daud Nadeem]

I have a rabbitmq server and use the pika library with Python to produce/consume messages. For development purposes, I was simply using 

```credentials = pika.PlainCredentials(<user-name>, <password>)``` 

I want to change that to use pika.ExternalCredentials or TLS. 

I have set up my rabbitmq server to listen for TLS on port 5671, and have configured it correctly. I am able to communicate with rabbitmq from localhost, but the moment I try to communicate with it from outside the localhost it doesn't like that. I have a feeling my "credentials" are based on the "guest" user in rabbitmq. 

###rabbitmq.config

```

%% -*- mode: erlang -*-

[

 {rabbit,

  [

   {ssl_listeners, [5671]},

   {auth_mechanisms, ['PLAIN', 'AMQPLAIN', 'EXTERNAL']},

   {ssl_options, [{cacertfile,"~/tls-gen/basic/result/ca_certificate.pem"},

                  {certfile,"~/tls-gen/basic/result/server_certificate.pem"},

                  {keyfile,"~/tls-gen/basic/result/server_key.pem"},

                  {verify,verify_none},

                  {ssl_cert_login_from, common_name},

                  {fail_if_no_peer_cert,false}]}

   

  ]}

].

```

I can confirm this works, since in my logs for rabbitmq I see:

```

2019-08-21 15:34:47.663 [info] <0.442.0> started TLS (SSL) listener on [::]:5671

```

Server-side everything seems to be set up, I have also generated certificates and all the .pem files required. 

###test_rabbitmq.py

```

import pika

import ssl

from pika.credentials import ExternalCredentials

context = ssl.create_default_context(cafile="~/tls-gen/basic/result/ca_certificate.pem")

context.load_cert_chain("~/tls-gen/basic/result/client_certificate.pem",

                            "~/tls-gen/basic/result/client_key.pem")

ssl_options = pika.SSLOptions(context, "10.154.0.27")

params = pika.ConnectionParameters(port=5671,ssl_options=ssl_options, credentials = ExternalCredentials())

connection = pika.BlockingConnection(params)

channel = connection.channel()

```

###When I run the script locally

```

(<Basic.GetOk(['delivery_tag=1', 'exchange=', 'message_count=0', 'redelivered=False', 'routing_key=foobar'])>, <BasicProperties>, b'Hello, world!')

```

###When I run the script from another instance

```

Traceback (most recent call last):

  File "pbbarcode.py", line 200, in <module>

    main()

  File "pbbarcode.py", line 187, in main

    connection = pika.BlockingConnection(params)

  File "/usr/local/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 359, in __init__

    self._impl = self._create_connection(parameters, _impl_class)

  File "/usr/local/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 450, in _create_connection

    raise self._reap_last_connection_workflow_error(error)

pika.exceptions.AMQPConnectionError

```

###When I run the script locally, and delete the guest user

```

Traceback (most recent call last):

  File "test_mq.py", line 12, in <module>

    with pika.BlockingConnection(conn_params) as conn:

  File "/home/daudn/.local/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 359, in __init__

    self._impl = self._create_connection(parameters, _impl_class)

  File "/home/daudn/.local/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 450, in _create_connection

    raise self._reap_last_connection_workflow_error(error)

pika.exceptions.ProbableAuthenticationError: ConnectionClosedByBroker: (403) 'ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.'

```

It seems like SSL is configured with the user "guest" and rabbitmq doesn't allow connections to guest outside of localhost. How can I use SSL with a different user?

When I delete the guest user, this is what the rabbitmq log says:

```

2019-08-22 10:14:40.054 [info] <0.735.0> accepting AMQP connection <0.735.0> (127.0.0.1:59192 -> 127.0.0.1:5671)

2019-08-22 10:14:40.063 [error] <0.735.0> Error on AMQP connection <0.735.0> (127.0.0.1:59192 -> 127.0.0.1:5671, state: starting):

PLAIN login refused: user 'guest' - invalid credentials

2019-08-22 10:14:40.063 [warning] <0.735.0> closing AMQP connection <0.735.0> (127.0.0.1:59192 -> 127.0.0.1:5671):

client unexpectedly closed TCP connection

2019-08-22 10:15:12.613 [info] <0.743.0> Creating user 'guest'

2019-08-22 10:15:28.370 [info] <0.750.0> Setting user tags for user 'guest' to [administrator]

2019-08-22 10:15:51.352 [info] <0.768.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*'

2019-08-22 10:15:54.237 [info] <0.774.0> accepting AMQP connection <0.774.0> (127.0.0.1:59202 -> 127.0.0.1:5671)

2019-08-22 10:15:54.243 [info] <0.774.0> connection <0.774.0> (127.0.0.1:59202 -> 127.0.0.1:5671): user 'guest' authenticated and granted access to vhost '/'

```

This also clearly means the SSL is still using the username and password to connect to rabbitmq? HELP!

References:

[tls_official_example](https://pika.readthedocs.io/en/stable/examples/tls_mutual_authentication.html)

[pika_official_tls_docs](https://www.rabbitmq.com/ssl.html)

[added_authentication_external](https://github.com/rabbitmq/rabbitmq-auth-mechanism-ssl/blob/rabbitmq_v3_6_1/README.md)

[Daud Nadeem]

Click here for better format: https://stackoverflow.com/questions/57595427/authenticating-rabbitmq-using-externalcredentials/57596906#57596906

I just copy-pasted my stackoverflow question here

[Luke Bakken]

Hello,

My understanding is that you wish to use the CN= value from your client certificate as the user's login, because RabbitMQ is configured as following:

{ssl_cert_login_from, common_name}

The first action you must take is to create a user in RabbitMQ without a password whose username matches the CN= value in your client certificate. Have you done this?

Can you please run the following command on your client certificate:

openssl x509 -noout -text -in client_certificate.pem | fgrep CN

import pika

import ssl

from pika.credentials import ExternalCredentials

context = ssl.create_default_context(cafile="~/tls-gen/basic/result/ca_certificate.pem")

context.load_cert_chain("~/tls-gen/basic/result/client_certificate.pem",

                            "~/tls-gen/basic/result/client_key.pem")

ssl_options = pika.SSLOptions(context, "10.154.0.27")

params = pika.ConnectionParameters(port=5671,ssl_options=ssl_options, credentials = ExternalCredentials())

connection = pika.BlockingConnection(params)

channel = connection.channel()

This code looks correct. If I have time today I will test this out locally.

Thanks,

Luke

[Luke Bakken]

Hello,

You also need to enable the SSL auth mechanism in RabbitMQ:

rabbitmq-plugins enable rabbitmq_auth_mechanism_ssl

Thanks,

LUke

[Daud Nadeem]

daudn@filestore-manager:~/tls-gen/basic/result$ openssl x509 -noout -text -in client_certificate.pem | fgrep CN

        Issuer: CN = TLSGenSelfSignedtRootCA, L = $$$$

        Subject: CN = filestore-manager, O = client

I just created a new user “filestore-manager” with no password, and admin tag + * permissions. 

2019-08-22 14:42:21.763 [info] <0.651.0> Creating user 'filestore-manager'

2019-08-22 14:42:40.546 [info] <0.659.0> Setting user tags for user 'filestore-manager' to [administrator]

2019-08-22 14:43:04.415 [info] <0.666.0> Setting permissions for 'filestore-manager' in '/' to '.', '.', '.*'

2019-08-22 14:46:51.219 [info] <0.701.0> RabbitMQ is asked to stop...

SSL auth mechanism is enabled as you can see from:

2019-08-22 14:47:06.021 [info] <0.8.0> Server startup complete; 4 plugins started.

 * rabbitmq_management

 * rabbitmq_web_dispatch

 * rabbitmq_auth_mechanism_ssl

 * rabbitmq_management_agent

When I run my script with “credentials=ExternalCredentials()” I get:

2019-08-22 14:47:37.873 [info] <0.623.0> connection <0.623.0> (127.0.0.1:60838 -> 127.0.0.1:5671): user 'guest' authenticated and granted access to vhost '/'

2019-08-22 14:47:37.887 [warning] <0.623.0> closing AMQP connection <0.623.0> (127.0.0.1:60838 -> 127.0.0.1:5671, vhost: '/', user: 'guest'):

When I add cred = ExtCred():

Traceback (most recent call last):

  File "test_mq.py", line 13, in <module>

    with pika.BlockingConnection(conn_params) as conn:

  File "/home/daudn/.local/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 359, in __init__

    self._impl = self._create_connection(parameters, _impl_class)

  File "/home/daudn/.local/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 450, in _create_connection

    raise self._reap_last_connection_workflow_error(error)

pika.exceptions.ProbableAuthenticationError: ConnectionClosedByBroker: (403) 'ACCESS_REFUSED - Login was refused using authentication mechanism EXTERNAL. For details see the broker logfile.’

And rabbitMq Log output:

2019-08-22 14:48:35.356 [info] <0.644.0> accepting AMQP connection <0.644.0> (127.0.0.1:60846 -> 127.0.0.1:5671)

2019-08-22 14:48:35.371 [error] <0.644.0> Error on AMQP connection <0.644.0> (127.0.0.1:60846 -> 127.0.0.1:5671, state: starting):

EXTERNAL login refused: user 'O=client,CN=filestore-manager' - invalid credentials

2019-08-22 14:48:35.371 [warning] <0.644.0> closing AMQP connection <0.644.0> (127.0.0.1:60846 -> 127.0.0.1:5671):

client unexpectedly closed TCP connection

I think the problem is with user ‘0=client’ 

Daud Nadeem​ Bioinformatics Developer

d  0207 424 1328 Daud....@anthonynolan.org  |  www.anthonynolan.org Connect with us on Twitter and Facebook  The Royal Free Hospital , Pond Street, London, NW3 2QG, United Kingdom

Anthony Nolan is a registered charity no. 803716 / SCO38827 and is registered as a company limited by guarantee in England and Wales no. 02379280, having its registered office at Royal Free Hospital, Pond Street, London NW3 2QG. Any opinions expressed in this message are those of the individual and not necessarily those of the company. This message and any files transmitted with it are confidential and solely for the use of the intended recipient. The message and files may not be disclosed to, or used by, anyone other than the addressee, without the consent of the sender. If you receive this message in error, please advise the sender immediately. It is the responsibility of the recipient to ensure that this message and its attachments are virus free. ​ We couldn't save lives without keeping in touch with you. That's why we take looking after information you give us seriously.  Our privacy policy sets out how we collect, use and store your personal information. If you have any questions about this then please don't hesitate to get in touch.​

[Luke Bakken]

Hello,

The user must be named O=client,CN=filestore-manager. Once you create a user with that name in RabbitMQ this will work.

I am a bit surprised by that and it may be a bug in how RabbitMQ extracts the CN= value from the certificate. I will make time to investigate that and will follow up here.

Thanks,

Luke

[Daud Nadeem]

Much to my surprise, that does work. However, when I run the same code on a different instance (outside localhost) and pass in the IP address of the RabbitMQ, I still error out directly. 

Is there a way to generate the CA with respect to an IP address? 

When I run this:

import pika

import ssl

from pika.credentials import ExternalCredentials

context = ssl.create_default_context(

    cafile="/home/daudn/tls-gen/basic/result/ca_certificate.pem")

context.load_cert_chain("/home/daudn/tls-gen/basic/result/client_certificate.pem",

                        "/home/daudn/tls-gen/basic/result/client_key.pem")

ssl_options = pika.SSLOptions(context,"10.154.0.27")

conn_params = pika.ConnectionParameters(port=5671,

                                        ssl_options=ssl_options,credentials = ExternalCredentials())

with pika.BlockingConnection(conn_params) as conn:

    ch = conn.channel()

    ch.queue_declare("foobar")

    ch.basic_publish("", "foobar", "Hello, world!")

    print(ch.basic_get("foobar"))

I get this output:

Traceback (most recent call last):

  File "test_mq.py", line 13, in <module>

    with pika.BlockingConnection(conn_params) as conn:

  File "/home/daudn/.local/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 359, in __init__

    self._impl = self._create_connection(parameters, _impl_class)

  File "/home/daudn/.local/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 450, in _create_connection

    raise self._reap_last_connection_workflow_error(error)

  File "/home/daudn/.local/lib/python3.7/site-packages/pika/adapters/utils/io_services_utils.py", line 636, in _do_ssl_handshake

    self._sock.do_handshake()

  File "/usr/lib/python3.7/ssl.py", line 1117, in do_handshake

    self._sslobj.do_handshake()

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: IP address mismatch, certificate is not valid for '10.154.0.27'. (_ssl.c:1056)

[Daud Nadeem]

Also. If I run the exact same code as above on another instance, the output is:

Traceback (most recent call last):

  File "test_mq.py", line 13, in <module>

    with pika.BlockingConnection(conn_params) as conn:

  File "/home/daudn/.local/lib/python3.5/site-packages/pika/adapters/blocking_connection.py", line 360, in __init__

    self._impl = self._create_connection(parameters, _impl_class)

  File "/home/daudn/.local/lib/python3.5/site-packages/pika/adapters/blocking_connection.py", line 451, in _create_connection

    raise self._reap_last_connection_workflow_error(error)

pika.exceptions.AMQPConnectionError

[Luke Bakken]

Hi Daud,

You should be using a host name for the CN= value of your server certificate, and then the client code would connect via that host name, not an IP address.

Since it looks like you're using tls-gen you would have to re-generate certificates using the IP address for the CN value of your other server. It's worth a try.

At this point, this discussion has veered from being Pika-specific to more about RabbitMQ and x509 certificates. I suggest asking on the RabbitMQ mailing list:

Luke

[Luke Bakken]

There's not much information to work with here, unfortunately. Most likely the client can't reach the server.

[Daud Nadeem]

Yes, sorry about that Luke. I’ve already posted in the rabbitmq mailing list as well.

I believe the IP address would work, however it seems like the name becomes (O=client, CN=<IP-Address> so it may not work.

Anyway, thank you for your help. I’ll wait for a reply from the rabbitmq mailing list! You’ve been great help!

Kindest regards,

[Luke Bakken]

Hi Daud,

The basic profile of the tls-gen project allows you to customize some values in the certs. You could do the following I believe (untested!):

make CN=IP-ADDRESS CLIENT_ALT_NAME=filestore-manager

But, at some point, you'll have to use a "real" certificate provider as tls-gen is for testing, mainly. You'll also want to use host names for your connections and not IP addresses.

Thanks,

Luke

[Luke Bakken]

Hi Daud,

You have an issue with your configuration file as well. The paths to the certificates must be absolute paths and not use ~ or environment variables:

[Luke Bakken]

Hi Daud,

I have verified that using client certificate authentication works as intended, and extracting the common name works as well.

Here are my enabled plugins:

2019-08-23 14:46:44.480 [info] <0.8.0> Server startup complete; 4 plugins started.

 * rabbitmq_management

 * rabbitmq_web_dispatch

 * rabbitmq_management_agent

 * rabbitmq_auth_mechanism_ssl

I have attached the RabbitMQ 3.7.17 configuration and python script I used to test. Note that both correct a couple errors you have in your configuration as well as Python code. I have used absolute paths to the certificate PEM files.

I used tls-gen/basic and just ran make to generate the certificates. This is what CN looks like from my client certificate:

$ openssl x509 -noout -text -in client_certificate.pem | fgrep CN

        Issuer: CN = TLSGenSelfSignedtRootCA, L = $$$$

        Subject: CN = shostakovich, O = client

My local host name is shostakovich, and when RabbitMQ runs the node name is rabbit@shostakovich. I have an entry in /etc/hosts like this:

127.0.0.1    shostakovich

Please let me know if you have any more questions.

Thanks,

Luke

[Daud Nadeem]

Hey Luke!! Thank you so much for your thorough response, I wanted to update you for future reference.


My confusion was that this:

ssl_options = pika.SSLOptions(context, ‘filestore-manager')
params = pika.ConnectionParameters(port=5671, ssl_options=ssl_options, credentials=ExternalCredentials())

Worked from localhost and not from another instance. I thought “filestore-manager” was the host. But turns out I (could and) had to supply the host in the connection parameters like this:

ssl_options = pika.SSLOptions(context, ‘filestore-manager')
params = pika.ConnectionParameters(host, <ip-address>, port=5671, ssl_options=ssl_options, credentials=ExternalCredentials())

I’ve got it working, but I am extremely impressed with Pika’s support. Thank you so much Luke.

Kindest regards,

Daud.