Rabbitmq consumer shutting down repeatedly

[Shuchi Gupta]

Hi,

I have configured a Dead Letter exchange with exponential back off policy. After making these changes, I have started getting an exception in the rabbitmq consumer getting shutdown repeatedly throwing the following exception:

Received shutdown signal for consumer tag=amq.ctag--Qn9jFNOd3vxhaHvEw8Nrw

com.rabbitmq.client.ShutdownSignalException: connection error

at com.rabbitmq.client.impl.AMQConnection.startShutdown(AMQConnection.java:715)

at com.rabbitmq.client.impl.AMQConnection.shutdown(AMQConnection.java:705)

at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:563)

at java.lang.Thread.run(Thread.java:745)

Caused by: java.io.EOFException

at java.io.DataInputStream.readUnsignedByte(DataInputStream.java:290)

at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:95)

at com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:139)

at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:532)

Can someone please give me some pointers regarding possible causes for this exception?

Thanks,

Shuchi

[Jean-Sébastien Pédron]

On 05.02.2015 10:01, Shuchi Gupta wrote:
> Hi,
>
> I have configured a Dead Letter exchange with exponential back off
> policy. After making these changes, I have started getting an exception
> in the rabbitmq consumer getting shutdown repeatedly throwing the
> following exception:

Hi!

Could you please post the RabbitMQ log files (both "normal" and sasl)
around the time you have this error?

--
Jean-Sébastien Pédron
Pivotal / RabbitMQ

[Michael Klishin]

 On 5 February 2015 at 12:01:15, Shuchi Gupta (shuchi....@gmail.com) wrote:
> I have configured a Dead Letter exchange with exponential back
> off policy. After making these changes, I have started getting
> an exception in the rabbitmq consumer getting shutdown repeatedly
> throwing the following exception:
>
>
> Received shutdown signal for consumer tag=amq.ctag--Qn9jFNOd3vxhaHvEw8Nrw
>
>
> com.rabbitmq.client.ShutdownSignalException: connection
> error
>
>
> at com.rabbitmq.client.impl.AMQConnection.startShutdown(AMQConnection.java:715)
>
>
> at com.rabbitmq.client.impl.AMQConnection.shutdown(AMQConnection.java:705)
>
>
> at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:563)
>
>
> at java.lang.Thread.run(Thread.java:745)
>
>
> Caused by: java.io.EOFException
>
>
> at java.io.DataInputStream.readUnsignedByte(DataInputStream.java:290)
>
>
> at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:95)

>

> Can someone please give me some pointers regarding possible
> causes for this exception?

See RabbitMQ log files to find more clues.

This looks like a network I/O  issue, do you use automatic connection recovery?
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[Shuchi Gupta]

There is no activity in sasl log file, here is the trace I found in the other log file:

=INFO REPORT==== 5-Feb-2015::09:46:06 ===

accepting AMQP connection <0.25858.13> (10.0.17.160:59865 -> 10.0.17.203:5672)

=WARNING REPORT==== 5-Feb-2015::09:46:06 ===

closing AMQP connection <0.25858.13> (10.0.17.160:59865 -> 10.0.17.203:5672):

connection_closed_abruptly

While consuming the message, I am getting this error before the shutdown exception:

2015-02-05 07:59:21,128  INFO [SimpleAsyncTaskExecutor-1565] (SimpleMessageListenerContainer.java:1154) - Restarting Consumer: tags=[[amq.ctag-XEN4F57JzeWrDhSqcoma7A]], channel=Cached Rabbit Channel: AMQChannel(amqp://gu...@54.169.121.224:5672/,2), acknowledgeMode=AUTO local queue size=0

2015-02-05 07:59:21,128  INFO [SimpleAsyncTaskExecutor-1565] (SimpleMessageListenerContainer.java:1154) - Restarting Consumer: tags=[[amq.ctag-XEN4F57JzeWrDhSqcoma7A]], channel=Cached Rabbit Channel: AMQChannel(amqp://gu...@54.169.121.224:5672/,2), acknowledgeMode=AUTO local queue size=0

2015-02-05 08:00:20,879 ERROR [pool-8-thread-8276] (CachingConnectionFactory.java:242) - Channel shutdown: connection error

2015-02-05 08:00:20,879 ERROR [pool-8-thread-8276] (CachingConnectionFactory.java:242) - Channel shutdown: connection error

2015-02-05 08:00:20,879 ERROR [pool-8-thread-8276] (CachingConnectionFactory.java:242) - Channel shutdown: connection error

2015-02-05 08:00:20,879 ERROR [pool-8-thread-8276] (CachingConnectionFactory.java:242) - Channel shutdown: connection error

--
You received this message because you are subscribed to a topic in the Google Groups "rabbitmq-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rabbitmq-users/UhbxjEQIMj0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send an email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Shuchi Gupta]

I actually deleted a queue from the rabbitmq publisher, could it be possible that the consumer is failing due to that? Though the queue has been regenerated. 

[Michael Klishin]

 On 5 February 2015 at 12:58:16, Shuchi Gupta (shuchi....@gmail.com) wrote:
> I actually deleted a queue from the rabbitmq publisher, could
> it be possible that the consumer is failing due to that? Though
> the queue has been regenerated.

Queues are not regenerated. If you delete a queue with consumers, they'd get a notification:
http://www.rabbitmq.com/consumer-cancel.html.

This means you app re-declared the queue.

> 2015-02-05 07:59:21,128  INFO [SimpleAsyncTaskExecutor-1565] (SimpleMessageListenerContainer.java:1154) - Restarting
> Consumer: tags=[[amq.ctag-XEN4F57JzeWrDhSqcoma7A]], channel=Cached Rabbit Channel: 
> AMQChannel(amqp://gu...@54.169.121.224:5672/,2), acknowledgeMode=AUTO local queue size=0

Yeah, so you use Spring-AMQP which you haven't mentioned.

Spring-AMQP does a bunch of opinionated stuff around error handling. Their support forum is on Stack Overflow,
tagged as [spring-amqp].