Graceful shutdown of rabbit consumer

[Bouke Haarsma]

Hi all,

I've implemented a work queue, similarly to https://www.rabbitmq.com/tutorials/tutorial-two-dotnet.html. I've specified my QOS to 0/1 and no autoAck, just like the example. Now when the consumer is shutdown, the channel will be closed and if there was a message being processed it is returned to the queue. However now the following two things happen:

1. The actual processing of the task is still ongoing in the default task scheduler as long as the executable is still running. This can be observed by adding an additional `Console.ReadLine()` after the disposal of the `connection` and `channel`. This means that the message that was returned to the queue, might actually complete, but the queue won't be notified.

2. The actual processing will be killed when the executable is stopped. This means that it will be stopped at an arbitrary point in the execution.

The work I'm putting on the queue is not idempotent and handling of the messages doesn't occur in a database transaction. As a result, shutting down a consumer that's still processing work will cause data corruption. So, I'm looking for a way to shutdown the consumer that waits for completion of any messages that are already being processed. Pre-fetched items should be returned to the queue.

I think this has been discussed some time ago, but no solution was found: https://groups.google.com/d/topic/rabbitmq-users/sy1tG--JqA0/discussion.

Thank you,

Bouke

[Michael Klishin]

A redelivered message will have "redelivered" property (a boolean) set to true. Perhaps your system could discard/acknowledge such deliveries immediately.

With non-idempotent processing it is hard to recommend something specific as RabbitMQ relies on acks from the consumer to know whether a message has to be

marked for deletion/considered to be processed. I think it's a consumer implementation detail more than anything.

HTH.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Bouke Haarsma]

Thank you for your reply. Ideally we change the implementation of our processing to become idempotent.

However I think my question is more along the lines of: how do I shutdown the C# EventingBasicConsumer, but only after it has completed (ack'ed) all messages it's currently processing?

--

Bouke

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitm...@googlegroups.com.

To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Michael Klishin]

You can cancel the consumer IModel#BasicCancel. It won't get any more deliveries after that,

not including those that are already on the wire ("in flight"). Then your consumer has to determine when it is done with the last

remaining work item and shut down.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.

To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bouke Haarsma]

IModel#BasicCancel doesn't block until all tasks have finished. I need a way to wait until all messages that are currently being processed have finished processing and that those messages have been ack'ed on the queue. Only then do I want to close/dispose the consumer / channel. The consumer posts the work as tasks on the task scheduler. How do I know when all work has been processed, does the RabbitMQ client provide anything for this, or is this something I have to keep track of myself?

[Michael Klishin]

"When all messages have finished processing" is a function of your app. The client library cannot know. Cancelling a consumer makes sure

you do not get any new deliveries besides those already sent/dispatched. It makes sense to do that when you know the consumer should shut down.

"No new deliveries after N seconds" is a good enough channel shutdown criteria in many cases.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.

To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.