Reject with requeue when prefetch greater than 1

[V Z]

When consumer rejects a message with requeue=true but prefetch is greater than 1, say 10, does the message go into the prefetch buffer on the consumer side and is immediately dispatched to the consumer again, i.e. without communication with the broker, or does the message go back to the broker, thus consumer sees it again after it has processed what was in the prefetch buffer at the time of rejecting the message?

[Michael Klishin]

"Prefetch buffer" is not a thing in RabbitMQ. Requeued messages go back to the queue in its original position when possible. It will be delivered to a different or the same consumer just like before.

Prefetch value is an integer that every channel has. It is decremented by 1 when a message is rejected or acknowledged.

> On 20 mar 2016, at 1:29, V Z <uvzu...@gmail.com> wrote:
>
> When consumer rejects a message with requeue=true but prefetch is greater than 1, say 10, does the message go into the prefetch buffer on the consumer side and is immediately dispatched to the consumer again, i.e. without communication with the broker, or does the message go back to the broker, thus consumer sees it again after it has processed what was in the prefetch buffer at the time of rejecting the message?
>

> --
> You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
> To post to this group, send an email to rabbitm...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[V Z]

What does "prefetch buffer is not a thing" mean? Did you mean that messages are not buffered on the client?

I understand the statement of "messages going back into original position whenever possible". It's "whenever possible" that's ambiguous.

Assuming messages are buffered on the client (up to prefetch count) as much of the literature (ex: https://www.rabbitmq.com/blog/2012/05/11/some-queuing-theory-throughput-latency-and-bandwidth) seem to suggest, sending the rejected (requeue=true) message to the server will not get it into the original position. Sticking it back into the buffer will. However, same literature suggests messages go to the server.

What am I missing? Thanks

[Michael Klishin]

Prefetch count is a part of server state (each channel has one). It is a counter. There is NO message buffer that corresponds to it in client libraries, at least it is certainly not required for them to have one. QueueingConsumer in the Java client works similarly but you don't have to use it to use prefetch. In fact, it was added for a very different reason and is now obsolete.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send an email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[V Z]

What was added for a very different reason and is now obsolete?

If there is no buffer on the client, specifically Java client, more specifically DefaultConsumer, then how is it assured that the consumer does not wait for an ack to travel back to the server and next message traveling from the server before the consumer can start processing that message?

Is this (https://www.rabbitmq.com/blog/2012/05/11/some-queuing-theory-throughput-latency-and-bandwidth) not accurate to suggest that consumer buffer is meant to improve consumer utilization by reducing or eliminating the wait time for messages to be dispatched from the broker?

[Michael Klishin]

> On 21 mar 2016, at 2:27, V Z <uvzu...@gmail.com> wrote:
>
> What was added for a very different reason and is now obsolete?

Queuing consumer.

>
> If there is no buffer on the client, specifically Java client, more specifically DefaultConsumer, then how is it assured that the consumer does not wait for an ack to travel back to the server and next message traveling from the server before the consumer can start processing that message?
> Is this (https://www.rabbitmq.com/blog/2012/05/11/some-queuing-theory-throughput-latency-and-bandwidth) not accurate to suggest that consumer buffer is meant to improve consumer utilization by reducing or eliminating the wait time for messages to be dispatched from the broker?

Consumers on a channel are given up to prefetch value messages before they acknowledge some of them. Consumers can ack multiple messages at a time. RabbitMQ will deliver messages on a channel until it hits the limit, without waiting for acks.

Like I said earlier, there is no buffer per se. RabbitMQ will push messages until it hits the limit, then it stops until one or more unacknowledged deliveries get acknowledged or rejected. All of this happens asynchronously: there are no "acks for acks". TCP buffers is the only kind of buffering that usually happens in client libraries. The QueueingConsumer and its .NET counterpart are exceptions (they first put deliveries in an in-memory queue).

Consumers that would like to improve latency under a constant stream of deliveries are supposed to ack multiple messages at once. Using higher prefetch values increases how many messages are "in flight" but that only helps to a point.

Client libraries can choose whatever's strategy they please to dispatch deliveries. Most use some variation of a thread pool with per-channel ordering guarantee.

I hope things are a little clearer now.