RabbitMQ Queue Aliasing - Zero Down Time Support

[Vyom Tandon]

Hi All,

We are planning to use RabbitMQ in our current Application and we have a typical requirement for ZDD (Zero Down Time). 

Our ZDD process has the Lite Side (Active) Application deployed in production and Dark Side (Passive) application used for other application version testing ready to be deployed in production. We are using IBM MQ in our existing environment where the Publisher is posting to real queues and subscribers are reading from Alias Queue. During the ZDD switch the Subscriber_1 exiting production code goes to dark side becomes passive and the dark side Subscriber_2 comes to lite side and becomes active. This happens only by changing the Alias queue references.

We want to achieve this Alias Queue concept in RabbitMQ so that when we do the switch there is no message loss and somehow Subscriber is pointing to Alias Queue rather than real queue so that we can make the switch easily.

I am attaching the 2 slides to show how the switch happens in the exiting system. Please suggest the similar solution for RabbitMQ.

Thanks,

Vyom

[Michael Klishin]

To be honest I don't understand the purpose of aliasing on the diagram, so I'll assume you are looking to have

a hot standby.

See https://groups.google.com/d/msg/rabbitmq-users/4bu1qTCnniU/T3HTdQLzAQAJ and [1] (in particular exchange federation).

1. http://www.rabbitmq.com/federation.html

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Vyom Tandon]

Hi Michael,

Thanks for the reply. 

An alias queue is a WebSphere® MQ object that you can use to access another queue or a topic. This means that more than one program can work with the same queue, accessing it using different names. The details of the IBM MQ Alias Queue is here : https://www.ibm.com/support/knowledgecenter/SSFKSJ_7.5.0/com.ibm.mq.pro.doc/q003120_.htm

We have the exact same requirement where subscribers will be connected to the Alias Queue from same or different machines instead of the real queues. The advantage of this topology is whenever we have to switch for hot deployment we just needs to change the alias links as shown in the diagram. 

Best Regards,

Vyom

[Michael Klishin]

There is no queue aliasing in RabbitMQ but you can move messages between queues with the Shovel plugin

(shovels can be created dynamically via HTTP API, for example).

[Vyom Tandon]

In that case what is the recommendation for hot deployment, we do not want any queues directly connected to subscriber. Is there anyway Subscriber can connect to Exchange instead of queue to pull the messages. This will help us switch between the queues. We are using Spring Integration AMQP for our Client applications.

[Michael Klishin]

Now that I think about it, given that it is apps that declare queues and topologies in RabbitMQ, I think the issue is

more of a consumer coordination one. Therefore using a tool such as ZooKeeper or etcd to communicate

up-to-date queue name changes to apps is what I'd recommend.

Apps then can consume from the old queue for N seconds (e.g. to make sure no deliveries are left behind) and switch to the new ones.

A dynamic Shovel could move messages between queues as well but it won't make the consumers switch to the new one(s).

[Vyom Tandon]

Appreciate your quick response.

Yes the Issue is coordination from the Consumer Side. Publisher can publish to Exchange and Consumers can consume from the real queue's. So when we have to switch from Consumer A (1.0 : lower version) to Consumer A (2.0 : higher version) as part of Code promotion this has to go as part of hot deployment. Both the consumer is tightly bounded with the physical queue and we cannot stop the Consumer A (1.0) for the switch. We want the Consumer A (2.0) get the reference of the queue which Consumer A (1.0) was using and start pulling the message from where the Consumer A(1.0) has left.

Is there anyway Consumer can take messages from Exchanges referencing the queue instead of Queue directly. This will help the switch ?

Regarding Zookeeper we might not want to use this for service discovery. We can explore the possibility for Shovel but we wanted to have a simple solution instead of complicating the infrastructure.

Thanks,

Vyom

[Michael Klishin]

You cannot consume from an exchange, exchanges do not store messages. You can add a single consumer to multiple queues

and discover exchange bindings via HTTP API, which together get you more or less what you want.