Messages drops mysteriously when trying to get an automatic retry logic going

[Tommy Helin]

Hi,

I am trying to get an automatic dead letter cycle by having my message queue (Q1) declared with 

x-dead-letter-exchange = "" (default)

x-dead-letter-routing-key = DLQ

And for the queue DLQ (dead letter queue) I declare: 

x-dead-letter-exchange = Q1Exchange (header exchange)

x-dead-letter-routing-key = Q1

x-message-ttl = 30 seconds (the effect being that there is a delay until message is sent back to Q1)

Everything works great. So if:

• message can't be interpreted by consumer
• consumer throws an exception
• message expires in DLQ

the message gets dead lettered and is sent to the declared queue and I the retry-cycle is complete. HOWEVER...

When I put a max-length on Q1 to say 1 message (x-max-length = 1) messages just starts to disappear after a while. I have seen in the logs that broker drops messages due to:

"Message dropped. Dead-letter queues cycle detected: [<<"confirmation.ready">>,<<"confirmation.ready.retry">>,<<"confirmation.ready">>]”"

However that is not always shown in the logs when I do new tests so I don't think that is the whole truth or is it? Also I don't understand why message is dropped since:

"It is possible to form a cycle of dead-letter queues. For instance, this can happen when a queue dead-letters messages to the default exchange without specifiying a dead-letter routing key. Messages in such cycles (i.e. messages that reach the same queue twice) will be dropped if the entire cycle is due to message expiry."

Routing key is specified and message is not immediately sent from Q1 to Q1 again. Or does the statement mean that a cycle is formed as soon as a message is coming back to a queue twice even though the message has been in other queues in between? 

But when queue gets full oldest message is deleted with a reason other than "expired" and the statement above says that the entire cycle should be due to expires. Is the drop of a message when queue is full interpreted as an expire anyway even if the reason is not expired?

I really want to be able to have this automatic retry functionality so that you don't have to implement this logic programmatically for all queues. Everything works great is max-length is not reached but as soon as that happens messages starts disappearing  Could there be some other reason messages gets dropped when queue length is exceeded? I have really tried to get to the bottom of this but I can't figure out the last piece of the puzzle.

[Michael Klishin]

Dead lettering has a loop protection mechanism. Unfortunately there is no better way to debug its behavior than [1]

but there are no known false positives with it. Wireshark/tcpdump would not help because all of that republishing/routing is internal to the node.

1. https://www.rabbitmq.com/firehose.html

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ