certficate revocation via CRL is not honoured by RabbitMQ nodes

[కృష్ణ చైతన్య]

Hi,

I have configured rabbitmq to use TLS and as part out trying out certificate revocation through CRL by adding {crl_check, true}, {crl_cache, {ssl_crl_cache, {internal, [{http, 50000}]}}} in advanced.config. Post this, revoking the certificate by issuer has no impact on the establishment of the links between RabbitMQ nodes. Is there any additional configuration that needs to be done? Are there any logs that I need to check for getting additional information?

Regards,
Krishna Chaitanya.

[Luke Bakken]

Hi Krishna,

Any time you ask a question about RabbitMQ (or any software, really) we need to know important details like...

• RabbitMQ version
• Erlang version
• Operating system running RabbitMQ, and version
• Are you using a cluster, and if so, how many nodes
• Full configuration files and environment variable (attached to your question, not pasted!)

For instance, I can't check if you correctly added those settings to advanced.config

When you say "establishment of the links between RabbitMQ nodes", are you talking about TLS-encrypted connections between nodes in your RabbitMQ cluster?

Thanks,

Luke

[కృష్ణ చైతన్య]

Hi Luke,

RabbitMQ version:  3.8.9

Erlang version:  23.2.2

Operating System:  Official rabbitmq container based on alpine.

We are not using cluster.

advanced.config attached to the email. 

[Luke Bakken]

Hello,

So you know, RabbitMQ 3.8.9 is out of support.

Everything appears to be configured correctly. Have you verified that HTTP requests to your CRL server are actually being made and that your CRL server is sending the correct response?

Thanks,

Luke

[కృష్ణ చైతన్య]

Hi Luke,

I have moved to RabbitMQ version 3.10.7 with Erlang Version 25.0.4.  I still see the same issue.  I have checked by downloading the crl at the url present on the certificate from the machine where rabbitmq is running and I am able to download the same.  It has the list of revoked certificates and the appears in the output when the crl file is viewed using certtool.  Rabbitmq logs do not have anything relevant.  Is there any other place I should be checking to see that the crl download is atleast being attempted when the certificate is presented.

Regards,

Krishna Chaitanya.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/2ec9c529-9363-47ab-bfd3-6d548d6caf0dn%40googlegroups.com.

[Luke Bakken]

Hi  Krishna,

You would have to use a packet capture or check the logs of the HTTP server that serves the CRL files to verify that RabbitMQ is making requests. I suggest doing the latter - enable HTTP access logs and see what requests are being made.

Thanks,

Luke

[కృష్ణ చైతన్య]

Hi Luke,

I have checked the logs of HTTP server and I dont see entry pertaining to the ip address of the rabbitmq node as a source in the web server logs.  Nevertheless, when I execute wget with crl url as argument from the rabbitmq node, I see that the crl file is downloaded and the corresponding IP as the source shows up in the webserver logs.

I have also captured the pcap and dont see the CA server’s IP as the destination address in the capture, indicating that RMQ is not making an attempt to download the crl.

Any further pointers please.

Regards,

Krishna Chaitanya.

To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/8289c142-ff9a-48f3-a14e-259a50481be5n%40googlegroups.com.

[కృష్ణ చైతన్య]

Hi Luke,

Is there a known defect around this?  What is the procedure to file a defect for this?

Regards,

Krishna Chaitanya.

[Luke Bakken]

Hello  Krishna,

Please be patient. I have to find the time to use your configuration to reproduce the issue.

I do know that CRLs work with Erlang (and thus RabbitMQ) so most likely there is an issue in your environment. I last worked with CRLs in April of this year for a paying RabbitMQ customer.

If this issue is urgent, consider paying for support - https://www.rabbitmq.com/#support

Thanks,
Luke

On Monday, November 21, 2022 at 8:53:31 PM UTC-8 k.krishna...@gmail.com wrote:

Hi Luke,

Is there a known defect around this?  What is the procedure to file a defect for this?

Regards,

Krishna Chaitanya.

[Luke Bakken]

Hello Krishna,

Have you validated that your certificates have the correct HTTP URI in their "X509v3 CRL Distribution Points" extension to specify the URI for your CRL server?

To check this:

openssl x509 -text -in certificate-that-may-be-revoked.pem

The output should include a section like this:

X509v3 CRL Distribution Points:
    Full Name:
        URI:http://localhost:8000/basic.crl

If you don't see X509v3 CRL Distribution Points in the output, or if you see the wrong URI, you are not generating your certificates correctly to be used with a CRL.

Here is the configuration for an OpenSSL CA that we use to generate certs for testing - https://github.com/rabbitmq/tls-gen/blob/main/basic/openssl.cnf

I'm putting together a complete example using the above project and Docker - https://github.com/lukebakken/rabbitmq-users-crl-6LjcEo9cn98

When it's done I'll let you know.

Thanks,

Luke

[Luke Bakken]

The following example project is complete:

https://github.com/lukebakken/rabbitmq-users-crl-6LjcEo9cn98

Clone it (be sure to include submodules), and run it via this command:

docker compose up --build

You will see a CRL HTTP request in the output, like this:

rabbitmq-users-crl-6ljceo9cn98-crl-1       | 172.19.0.3 - - [26/Nov/2022 03:41:30] "GET /basic.crl HTTP/1.1" 200 -

Both the Python producer and consumer applications authenticate using client certificates. If you add the client cert to the CRL, the connection will be rejected. I will add a script that shows how to do that. Note that CRLs are only checked when connections are established, so I the apps will have to be restarted as well (that will be part of the script).

Between that project and rabbitmq/tls-gen, everything you need to generate certs correctly and configure your apps and RabbitMQ correctly to use client certificate authentication and CRLs is there.

Thanks,

Luke

[కృష్ణ చైతన్య]

Hi Luke,

Thanks for your email and all the help.

I see the output contains the CRL on the ceritificate.  Attached the certificate and the output of openssl "x509 -noout -in CNkchaitan029demolocalOUC\(2\).pem -text”. Th can the pass the CRL URI as the argument to wget and get the CRL file downloaded.

[Luke Bakken]

Hello,

Are you trying to use a CRL for your application's client certificates or something else? In your very first message you make this statement:

"no impact on the establishment of the links between RabbitMQ nodes"

If you are using inter-node TLS between your RabbitMQ cluster nodes you do NOT want to use a CRL because you always want your nodes to be able to connect to one another. I don't even know if you can configure a CRL for inter-node TLS.

At any rate, your cert looks correct and your configuration should work for applications connecting to RabbitMQ on port 5671. This is what my example project demonstrates.

Thanks,

Luke

[Luke Bakken]

Hello,

Yes, you can configure CRL checks for inter-node TLS - https://www.erlang.org/doc/apps/ssl/ssl_distribution.html#specifying-tls-options

But really, don't do that.

[Krishna Chaitanya]

Hi Luke,

Thanks for all the help.  Thanks again for the example program that you have shared.

As you have rightly identified I am looking for the CRL revocation check to happen when the connection is established between rabbitmq nodes.  We use federation plugin.

I shall go over the erlang documentation that you shared to achieve the purpose.

Is there any more reasons why you are advocating not to use CRL for connection between rabbitmq nodes?

Our application is a multi-node deployment with each node running an instance of rabbitmq.  Publishers and Subscribers can connect to any node in the deployment and those nodes can be widely dispersed across geographies.

Thanks again for all your help.

Regards,

Krishna Chaitanya.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/8d63312e-b48d-4fee-9eb6-315517266e7fn%40googlegroups.com.

[Luke Bakken]

Responses in-line...

 

As you have rightly identified I am looking for the CRL revocation check to happen when the connection is established between rabbitmq nodes.  We use federation plugin.

This is the first time you have mentioned the Federation plugin! This should have been stated in your first message.

 

I shall go over the erlang documentation that you shared to achieve the purpose.

No, I linked to the documentation for inter-node distributed Erlang communication, which is used for forming RabbitMQ clusters. This has nothing to do with Federation. There's no need for you to read that documentation.

Is there any more reasons why you are advocating not to use CRL for connection between rabbitmq nodes?

"connection between RabbitMQ nodes" is a vague statement. So is "links between RabbitMQ nodes". You should have said "Federation" right from the beginning of this discussion.

Our application is a multi-node deployment with each node running an instance of rabbitmq.  Publishers and Subscribers can connect to any node in the deployment and those nodes can be widely dispersed across geographies.

Yes, you can use CRLs for Federation but you must configure the Federation AMQP URI correctly - https://www.rabbitmq.com/federation.html#tls-connections

I will modify my example project to bring up two RabbitMQ nodes using Federation and CRLs for the federation links. I'll report back here when it's ready. It may be a few days.

Thanks,
Luke

 

[Luke Bakken]

Hello yet again!

Please see the project, which I have now updated to use two RabbitMQ nodes connected via Federation - https://github.com/lukebakken/rabbitmq-users-crl-6LjcEo9cn98

The Federation links are configured to authenticate with client certificates. The CRL server is checked when the Federation link starts.

Let me know if you have any questions.

To see it in action:

docker compose up --build

Thanks,

Lue