RabbitMQ - DR Scenario

[Paul Sanders]

Good Morning All,

I am hoping you can point me in the right direction, as I am having a few concerns on a RabbitMQ design. 

At present, we have a RabbitMQ cluster which our internall application uses to pass messages between the different components. This works well, it can manage node failure well. 

The issues I am having is when it comes to DR. How could I build a solution so that in the event of DR, any messages that hadn't been processed on my main Cluster will be available at the DR Site?

Thanks

Paul  

[Michael Klishin]

 On 1 December 2014 at 12:55:59, Paul Sanders (paul.sa...@gmail.com) wrote:
> The issues I am having is when it comes to DR. How could I build
> a solution so that in the event of DR, any messages that hadn't
> been processed on my main Cluster will be available at the DR Site?

Use exchange federation to replicate all messages to a stand-by cluster (C2). Then in
C2, use a policy to make all queues have a TTL for messages of a few hours or whatever
time window fits your case.

After failing over, delete the policy. Until RabbitMQ gets an AP version of mirroring, this is
the least painful option.
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[Laing, Michael]

For a more painful option... :)

We run active/active(/active) and duplicate all important messages to at least 2 core clusters.

The core clusters process in parallel, including database operations to a global Cassandra cluster, and then exchange outputs.

Each core dedups using Cassandra (coarse grain) as a final step, passing messages to the gateways, which also dedup (fine grain).

In extreme cases, a client may still receive a duplicate. They expect this.

In our case, core clusters are on different continents. 

We can run on one core cluster if necessary and have done so. No client, external or internal, noticed.

ml

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send an email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Paul Sanders]

Thanks Both for the answers. A federated exchange sounds like the solution. Although I would love to implement something like ML talks about :)

We are having some kickback from the business regarding our current setup (a 4x node cluster with mirrored queues), with people arguing it makes more sense to simply have a single box that would be restored from snapshot if it was to die. The application would deal with any failures. 

Has anyone got an opinion on this? I would ideally prefer a cluster with mirrored queues, as loosing the queing system is highly unlikely. I imagine the risk of loosing messages on a cluster is minimal?

[Skrzypek, Jonathan]

Hi,

The annoying one with this is that acks are not propagated back to the upstream cluster.
Which means that upon failover, the consuming application needs to know "where it is" in terms of sequence, and will have to ignore a certain amount
of messages and then start processing the ones it hasn't seen before.
Overall this means that the application needs to maintain state instead of the broker which slightly defeats the purpose.

However I understand a synchronous replication would be costly and painfully slow down the product.

[Laing, Michael]

It's certainly easy to dedup by unique message id, which is what we do.

In general, "at least once" delivery is what messaging systems are best at.

So it makes sense to design your apps with that in mind.

ml