Consumer dropping from RabbitMQ queues

[Sunil Chaurha]

Hi All,

We have many applications running in production and sometimes we see below tow serious issues which lead to loss of business (sometimes).

1. Consumer gets dropped from Rabbit queue.
2. Consumer stops processing messages from the queue when there are millions of messages in the queue.

Description:

1.   In the first case, this happens suddenly and then our support team needs to get the notification about the transaction stuck then we re-bounce the TC instance and then again our war (interface) comes to normal situation and starts processing the message. 

I have done some analysis and seen some waiting threads into thread dump between rabbit and other resource and one more issue noticed was memory unavailability. 

Though we have been working to find root cause of this serious issue but still we don't have accurate information and no perfect solution available yet.

2.  - Consumer component stops processing the messages from the queue. 

In this case we have a transaction queue and a dead letter queue, consume polls messages from transaction queue and if the message is corrupted (or consumer did not accept the message after validation) then message goes to dead letter queue and based on policy message stays there for 30 mins and then message is flushed out from DL queue. 

We are looking for the solution for these two serious problems if anyone have noticed?

--

Regards,

Sunil  

[Michael Klishin]

On 16 Oct 2015 at 09:29:19, Sunil Chaurha (sunilc...@gmail.com) wrote:
> We are looking for the solution for these two serious problems
> if anyone have noticed?

What version do you run? Have you inspected RabbitMQ logs? Have you verified that
consumer connections are online?

I remember a couple of issues that could block consumer delivery and it can be rabbitmq/rabbitmq-server#227
and related issues with paging to disk being inefficient and constantly triggered due to false positives :
https://github.com/rabbitmq/rabbitmq-server/issues/227

So give 3.5.6 a try.
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[Michael Klishin]

+rabbitmq-users

On 16 Oct 2015 at 14:39:26, Sunil Chaurha (sunilc...@gmail.com) wrote:
> Hi Mike,
>
> we are using RabbitMQ 3.2.3, Erlang R15B02 version.

Have you inspected RabbitMQ logs? Have you verified that 
consumer connections are online? 

3.2.x is not getting any updates and is 19  releases behind:
http://www.rabbitmq.com/changelog.html