Connection is running but no channel

[JLi]

We have 4 consumers for processing some regular messages. 

The consumers have been run well for 2 months. But they suddenly stop consume messages from broker days ago.

On the broker console it shows the connection is running. But no channel exists.

I shutdown the connection and let it reconnect, the channel appears after reconnect. But all channels get lost after processing same messages next day.

On the consumers, I can see some messages been processed on all of the 4 nodes. And the messages still in the queue.

I suspect it might cause by some unexpected exceptions when processing the message. 

But I can't find a way to debug and identify which message caused the channel disconnection.

Does anyone have any idea how should I do to locate the root cause? Thanks.

Below is the consumer configuration:

ConnectionFactory factory = new ConnectionFactory();
factory.setUsername(userName);
factory.setPassword(password);
factory.setVirtualHost(virtualHost);
factory.setHost(hostName);
factory.setPort(portNumber);
factory.setRequestedHeartbeat(60);
factory.setAutomaticRecoveryEnabled(true);
factory.setNetworkRecoveryInterval(300000);

[Michael Klishin]

Answered in https://groups.google.com/forum/#!topic/rabbitmq-users/dnLJHUB41Wg.

In addition, applications can close their channels voluntarily. Please inspect the logs

and make sure your applications open new channels when it's appropriate to do so

(channels are generally meant to be long-lived, so no need to open a new one per operation).

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[JLi]

Thanks so much Michael. I thought set automaticRecoveryEnabled to true will get the channel reopen automatically, but it doesn't, right?

Also I have another question, the prefetched but unprocessed message on the crushed node still get processed but can't ack because the channel has close. So they sent to other consumers. How could I handle this situation?

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[JLi]

Let me rephrase my question.

It seems like, channel's shutdown won't stop the prefetched messages stop being processed.

But all those messages are requeued on the broker. So they get sent to other consumer nodes.

So those messages get processed multiple times.

Is there anyway to avoid this happen?

[Arnaud Cogoluègnes]

On Tue, Feb 14, 2017 at 2:19 AM, JLi <cunru...@gmail.com> wrote:

Thanks so much Michael. I thought set automaticRecoveryEnabled to true will get the channel reopen automatically, but it doesn't, right?

It doesn't. Automatic recovery kicks in in case of network errors, not when something goes wrong in a consumer.

 

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

[Arnaud Cogoluègnes]

There's a callback for channel/connection shutdown in Consumer, you can use it to stop or cancel the processing.

Try also to make the processing idempotent. Use of acknowledgements guarantees at-least-once delivery, not exactly-once. See [1].

[1] http://www.rabbitmq.com/reliability.html

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

[Michael Klishin]

Java client documentation explains what is covered by automatic recovery:

http://www.rabbitmq.com/api-guide.html

Channels get closed either by applications or due to a protocol exception. It is a very bad idea

to automatically reopen channels that were closed by RabbitMQ, therefore the client

does not do it.

Please start a new thread for your other question.

[Cunrui Lee]

Thanks Aranud. It's very helpful.

--
You received this message because you are subscribed to a topic in the Google Groups "rabbitmq-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rabbitmq-users/RyH81HGfA1w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rabbitmq-users+unsubscribe@googlegroups.com.