HA configuration of queues using RabbitMQ JMS

[Matyas Boronyai]

Dear Community,

Already received good advice here, so I try my luck again :).

Our use case is the following: in our existing system,  a handful of components communicate over ActiveMQ, using Camel and JMS. In the planned next version, the company-wise used stack contains RabbitMQ as the message broker, so our current task is to migrate existing applications to this new environment (more details about our integration in the below thread).

https://groups.google.com/g/rabbitmq-users/c/VlrSul8y_A0/m/ptp7l2z6AwAJ

The base situation is that we have certain long lasting services listening for commands on specific JMS queues/topics. When testing in a 3-node clustered environment, with a simple test setup continuously sending and consuming messages, we had trouble reconnecting listeners after shutting down and restarting cluster nodes, which is something I keep seeing complained about. We obviously didn't have queue mirroring setup, so we bumped into the classic exceptions like

Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=404, reply-text=NOT_FOUND - home node 'rab...@rabbitmq-2.rabbitmq-headless.rabbitmq.svc.cluster.local' of durable queue 'OMC.ILC.CMD.IRNS' in vhost '/' is down or inaccessible, class-id=50, method-id=10)

As we have some HA requirements anyway (I intentionally left this statement vague, it's not always obvious what do you ACTUALLY need in a specific situation), I then installed a policy that actually made our queues mirrored, which seemed to solve the problem, I was able to shut down and restart nodes in the cluster without the client noticing, exactly how I would expected this to be transparently done.

But then, I came upon the statement that the classic mirrored queue functionality is deprecated and to be removed soon, which is I would say, a bad sign, already at the beginning in your migration phase :). Quorum queues are the recommended substitute, and here is when I get into trouble:

-  with Camel and JMS, I wasn't able to find a proper way to inject this x-queue-type argument when implicitly declaring the queue. The only halfway-acceptable solution I could come up with is a custom ConnectionFactory, which sets this parameter explicitly when creating a JMS session. Is there a more convenient way to do this?

-  after using this custom connection factory, which always "forces" quorum queues, I bumped into the problem that JMS consumers declare the queue to be exclusive if the subscription is not durable, but that is not allowed with quorum queues and the declaration will be rejected. To circumvent this, I used a different connectionfactory for listeners which will declare classic queues instead of quorum ones.

-  with all this tweaks in place, I was able to run my tests and make successful recoveries after shutting down nodes.

Nevertheless, I'm not sure whether I'm at the right place at the right time :). Unfortunately, completely getting rid of JMS, as suggested previously is not an option for us at the time being. I'm kind of missing an opportunity to configure JMS with the supported HA queues. The current situation seemingly leaves us with two options, either I use the mirrored classic queues, that are already deprecated, or I literally hack my way through to use quorum queues with JMS somehow.

 Have I missed something? And if not, which option should I prefer in your opinion? 

Thanks,

Matyas

[Arnaud Cogoluègnes]

I'd say your approach is correct (sorry for the struggle  :-)  ).

You should definitely use quorum queues instead of classic mirrored queues.

To force the creation of quorum queues, you can also cast the JMS Session to RMQSession and use setQueueDeclareArguments(Map), it was introduced a while ago for the exact same purpose [1] (not ideal but good enough). The Javadoc provides some hints to use it.

Note using different connections for publishers and consumers is also considered a good practice.

[1] https://github.com/rabbitmq/rabbitmq-jms-client/pull/120

[Matyas Boronyai]

Thanks for the reassurance! :)

In fact, in my custom connection factory, I did exactly what you suggest with casting the session and calling the  setQueueDeclareArguments method.

Matyas