Avoid requeue/redelivery loop when rejecting messages

[Simone B]

Hi all,
the documentation at https://www.rabbitmq.com/confirms.html states:

Requeued messages may be immediately ready for redelivery depending on their position in the queue and the prefetch value used by the channels with active consumers. This means that if all consumers requeue because they cannot process a delivery due to a transient condition, they will create a requeue/redelivery loop. Such loops can be costly in terms of network bandwidth and CPU resources. Consumer implementations can track the number of redeliveries and reject messages for good (discard them) or schedule requeueing after a delay.

and well, I'm hitting the issue with the requeue/redelivery loop.

In my use case, I want to use a quorum queue to keep track of messages that will be processed much later (days).

What does "schedule requeueing after a delay" mean? I don't understand how waiting  could help in avoiding the requeue loop (except by making the loop larger); also, I think that any artificial delay in rejecting messages would also slow down the incoming messages, so it will greatly decrease the overall throughput.

Moreover, when using a quorum queue it is advised to avoid requeueing excessively. I was wondering that a possible workaround for both scenarios (excessive requeueing and redelivery loop) could be to ack the message and publish it again (possibly after a certain number of tentatives, or randomly). This could possibly be achieved by setting the queue to dead-letter to the same queue itself.
Does this make any sense?

Thanks,

Simone

[Brown Purchase]

I had to create another service to manage what we call 'retries'.  Essentially, you ack the message after you have saved to another queue.  I had to add a wrapper class around all my messages for retry count, next try date, data (message), max retry count, error message, and queue.  Queue is for knowing where to insert back into when the next try date occurs.

The retry service takes an item out of the retry queue and puts into db.  It also pulls out of db to send into which ever queue it failed.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/1bb68d34-b26a-4971-9bfb-884260d98445n%40googlegroups.com.

[Karl Nilsson]

You don't even need the consumers to track the number of redeliveries - instead, you can use the QQs poison handling strategy to set a maximum number of redeliveries [1] after which it will drop or dead letter the message for you.

1. https://www.rabbitmq.com/quorum-queues.html#poison-message-handling

---

From: rabbitm...@googlegroups.com <rabbitm...@googlegroups.com> on behalf of Simone B <simoneb...@gmail.com>
Sent: 15 September 2020 3:37 PM
To: rabbitmq-users <rabbitm...@googlegroups.com>
Subject: [rabbitmq-users] Avoid requeue/redelivery loop when rejecting messages

 

--

[Simone B]

I can't really drop the messages, I still want to process them. In this scenario it would be necessary to wait days or even more before processing them.

So far, both suggestions involve the use of a second queue (either by moving everything to the second queue, or dead lettering to it). But I don't understand what the second queue does differently from the first queue that helps fixing the issue. I still need to scan the second queue and wait before processing, possibly hitting the same requeue/redelivery loop. 
The only difference I see in the first proposed solution is that, by using a second queue, the message is "enriched" with the retry count, so I think this could mean that the messages is acked and re-inserted every time with an increased retry count.

Or am I missing a different method of scanning the second queue that will avoid the loop?

To my understanding, basically acking the message and re-queueing it in the same queue seems the only workaround.
The docs seem to suggest another solution with the phrase "schedule requeueing after a delay" but I'm still confused at what that means or how it helps.

Thanks all,

Simone

[Luke Bakken]

Hello,

My assumption is that a delay is suggested to give your consumers or your environment time to "fix" whatever condition caused the message to be re-queued.

Thanks,

Luke

[diego unali]

Hi all,

do you think that ack the message and republish it in the same queue is a good practice for a quorum queue?

I also have some doubt about publishing and ack many times, because the internal transition log of the queue can grow very fast, causing problem while replicating to other nodes.

Thanks,

Diego

--

You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/2a4dd2d2-9dce-41d8-be4a-c586d7e2eafbo%40googlegroups.com.

[Brown Purchase]

I dont.  There is an option to send to a dead letter exchange or you can send to another queue manually.  I suggest having another process requeue those items at an interval of your choice.  i believe the dead letter exchange also tracks requeue/retry counts but it doesnt track the last try date or a retry delay.

In my system we have many apps that have many needs for retry.  ex 5 retries every 5m for jobs that send to 3rd party. or 72 retries every 10m for jobs that go over internet.  This is setup because of business needs.

Because of that I wrote my own wrapper class around a message, tracking retry info and can also give a UI to retries that have expired so users can manually retry after automatic retries are done.  This is because the systems i deal with can have outages.

I take my dead letters and send to a db so i can have another process requeue the items based upon an interval the business desires.

To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/CAEbt9DUvrJCQOZz71DTM9P7ykymj4v1_0-nbEXJjYAHkxwYbow%40mail.gmail.com.