Ack/Nack via different channel

[Simon O'Beirne]

Hi,

We are considering RabbitMQ as an internal message queue for a pub/sub application we are developing, where Rabbit would sit behind a REST API which provides messaging-type concepts to ensure guaranteed at-least-once delivery - i.e. peek (basic get/nack), peek lock (basic get), complete (ack), abandon (nack).

The problem with this is that Rabbit only accepts ack/nack on the same channel as the message was consumed on originally. As this is two separate REST calls, one to peek lock the message (basic get), and another to ack/nack the message, we don't hold the channel open and the ack/nack fails (silently). Furthermore, the lock on the message is released early due to the channel being closed once the message has been returned to the client by the REST service.

Solutions we have currently identified are:

1. Keep channel open either within REST API (i.e. make the web service stateful) or move Rabbit connections another stateful separate application, to allow ack/nack.
• Stateful REST API is a no-no due to load-balancing/scaling complexities, and keeping connections open would also result in challenges at scale as we will have multiple vhosts, each of which would require a separate connection.
  
2. Move message to message-specific "lock" queue and ack from original queue. When nack arrives, retrieve message from specific queue and move back to original queue, or if ack then remove from message-specific queue and allow RabbitMQ to remove the queue automatically
• This would potentially mean a large number of queues being created/removed - is this something RabbitMQ would handle well? Are there any performance limitations we would need to consider?
  
• We would need to have an additional process which moved the message from the message-specific queue back to the original queue after a pre-defined length of time (lock timeout period)
  
3. Place message in some other storage (database), remove message from RabbitMQ via ack. If message is subsequently nack'd then push back to original queue.
• This is feasible but we would prefer to keep messages in one place if possible

Are there any other ways in which we could solve this problem in a straightforward manner, ideally using only the tools provided out-of-the-box in RabbitMQ?

Many thanks,

Simon

[Michael Klishin]

On 26 March 2015 at 15:43:51, Simon O'Beirne (trialsm...@gmail.com) wrote:
> The problem with this is that Rabbit only accepts ack/nack on
> the same channel as the message was consumed on originally. As
> this is two separate REST calls, one to peek lock the message (basic
> get), and another to ack/nack the message, we don't hold the channel
> open and the ack/nack fails (silently). Furthermore, the lock
> on the message is released early due to the channel being closed
> once the message has been returned to the client by the REST service.
>
> Solutions we have currently identified are:
> Keep channel open either within REST API (i.e. make the web service
> stateful) or move Rabbit connections another stateful separate
> application, to allow ack/nack.
> Stateful REST API is a no-no due to load-balancing/scaling complexities,
> and keeping connections open would also result in challenges
> at scale as we will have multiple vhosts, each of which would require
> a separate connection.

I should point out that the problem is inherently stateful. You can push state around
but so will the point of contention as you add more instances at every layer.

> Move message to message-specific "lock" queue and ack from original
> queue. When nack arrives, retrieve message from specific queue
> and move back to original queue, or if ack then remove from message-specific
> queue and allow RabbitMQ to remove the queue automatically
> This would potentially mean a large number of queues being created/removed
> - is this something RabbitMQ would handle well? Are there any
> performance limitations we would need to consider?
> We would need to have an additional process which moved the message
> from the message-specific queue back to the original queue after
> a pre-defined length of time (lock timeout period)
> Place message in some other storage (database), remove message
> from RabbitMQ via ack. If message is subsequently nack'd then
> push back to original queue.
> This is feasible but we would prefer to keep messages in one place
> if possible
>
> Are there any other ways in which we could solve this problem in
> a straightforward manner, ideally using only the tools provided
> out-of-the-box in RabbitMQ?

Acking must happen on the same channel. Channels should be seen as multiplex connections: why would
one connection accept acks on other connections? How can this work technically given that there may be
conflicts between delivery tags?

You can either have a bit of state (which can be made instance local, not global) to your service, or you can
use a stateful tool that was created to handle this exact problem, e.g. ZooKeeper or etcd.
I'd recommend doing the latter.
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[Roth Jianping]

I am trying to solve the same problem as you described here and I wonder what approach you went for please? 

Thank you! 

Jianping