com.rabbitmq.client.AlreadyClosedException: channel is already closed due to previous clean channel

[Shweta Lokhande]

Hi Team,

Getting the below exception . Can you please suggest some quick fix. PS: No changes were done in queue declaration/setting and everything was running fine earlier.

at com.rabbitmq.client.impl.recovery.AutorecoveringConnection.access$000(AutorecoveringConnection.java:47)

        at com.rabbitmq.client.impl.recovery.AutorecoveringConnection$1.shutdownCompleted(AutorecoveringConnection.java:344)

        at com.rabbitmq.client.impl.ShutdownNotifierComponent.notifyListeners(ShutdownNotifierComponent.java:75)

        at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:572)

        at java.lang.Thread.run(Thread.java:722)

Caused by: com.rabbitmq.client.AlreadyClosedException: channel is already closed due to previous clean channel shutdown; reason: #method<channel.close>(reply-code=200, reply-text=OK, class-id=0, method-id=0)

        at com.rabbitmq.client.impl.AMQChannel.ensureIsOpen(AMQChannel.java:190)

        at com.rabbitmq.client.impl.AMQChannel.rpc(AMQChannel.java:223)

        at com.rabbitmq.client.impl.AMQChannel.privateRpc(AMQChannel.java:209)

        at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:118)

        at com.rabbitmq.client.impl.ChannelN.queueDeclare(ChannelN.java:779)

        at com.rabbitmq.client.impl.recovery.AutorecoveringChannel.queueDeclare(AutorecoveringChannel.java:230)

        at com.rabbitmq.client.impl.recovery.RecordedQueue.recover(RecordedQueue.java:36)

        at com.rabbitmq.client.impl.recovery.AutorecoveringConnection.recoverQueues(AutorecoveringConnection.java:446)

        ... 7 more

Caught an exception when recovering topology Caught an exception while recovering binding between amq.direct and TORCH_ETL_REMOVE_LOCKFILE

com.rabbitmq.client.TopologyRecoveryException: Caught an exception while recovering binding between amq.direct and TORCH_ETL_REMOVE_LOCKFILE

        at com.rabbitmq.client.impl.recovery.AutorecoveringConnection.recoverBindings(AutorecoveringConnection.java:469)

        at com.rabbitmq.client.impl.recovery.AutorecoveringConnection.recoverEntities(AutorecoveringConnection.java:424)

        at com.rabbitmq.client.impl.recovery.AutorecoveringConnection.beginAutomaticRecovery(AutorecoveringConnection.java:364)

        at com.rabbitmq.client.impl.recovery.AutorecoveringConnection.access$000(AutorecoveringConnection.java:47)

        at com.rabbitmq.client.impl.recovery.AutorecoveringConnection$1.shutdownCompleted(AutorecoveringConnection.java:344)

        at com.rabbitmq.client.impl.ShutdownNotifierComponent.notifyListeners(ShutdownNotifierComponent.java:75)

        at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:572)

        at java.lang.Thread.run(Thread.java:722)

Caused by: com.rabbitmq.client.AlreadyClosedException: channel is already closed due to previous clean channel shutdown; reason: #method<channel.close>(reply-code=200, reply-text=OK, class-id=0, method-id=0)

        at com.rabbitmq.client.impl.AMQChannel.ensureIsOpen(AMQChannel.java:190)

        at com.rabbitmq.client.impl.AMQChannel.rpc(AMQChannel.java:223)

        at com.rabbitmq.client.impl.AMQChannel.privateRpc(AMQChannel.java:209)

        at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:118)

        at com.rabbitmq.client.impl.ChannelN.queueBind(ChannelN.java:836)

        at com.rabbitmq.client.impl.ChannelN.queueBind(ChannelN.java:61)

        at com.rabbitmq.client.impl.recovery.RecordedQueueBinding.recover(RecordedQueueBinding.java:14)

        at com.rabbitmq.client.impl.recovery.AutorecoveringConnection.recoverBindings(AutorecoveringConnection.java:466)

[Michael Klishin]

There is an answer right in the stack trace:

> Caused by: com.rabbitmq.client.AlreadyClosedException: channel is already closed due to previous clean channel shutdown

Channels cannobe be used after they were closed, voluntarily or due to a channel-level exception.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Shweta Lokhande]

Hi Micheal,

I am new to RabbitMQ.
Can you please tell me what are the possible reasons of channel getting shutdown ?

Is it something more to do with client application or  more with doing RabbitMQ server ?


Regards,

Shweta

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Michael Klishin]

Well, the answer to that is also in the same stack trace:

> Caused by: com.rabbitmq.client.AlreadyClosedException: channel is already closed due to previous clean channel shutdown

Your channel was voluntarily closed by your application (or a library, but not the server). It is done by invoking Channel#close.
I highly recommend going through the protocol overview and RabbitMQ tutorials:

http://www.rabbitmq.com/tutorials/amqp-concepts.html
http://www.rabbitmq.com/getstarted.html

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Shweta Lokhande]

HI Micheal,

Thanks for the pointers. 
Well , I couldn't  figure out what exactly caused this "Caught an exception when recovering topology Caught an exception while recovering binding between amq.direct and TORCH_ETL_REMOVE_LOCKFILE"

Also , rabbitmq connection keeps resetting every 5 mins.

Can you suggest a quick fix?

Regards,

Shweta 

[Michael Klishin]

I am a big fan of suggesting things that let 

those asking questions understand how to get to the root cause. You aren't providing enough information to get a "quick fix". Also, quick fixes may be bad for your career and probably skin.

If a channel is closed by your app during connection recovery, it may suggest that you use very short lived channels that your application closes. You haven't provided any code snippets so that's the best  hypothesis I have.

As for connections resetting (as in TCP reset?) every so often, you haven't provided any logs so I again have to improvise and guess. See how we are not going to get to the quick fix town at this rate? Yeah.

If a proxy or load balancer of some kind is involved,

it may be closing TCP connections it believes are inactive. http://rabbitmq.com/heartbeats.html and plenty of threads in this list archive mention a feature found in most messaging protocols that also helps work around such unreasonably low timeouts even when you don't have control over your infrastructure and cannot bump the value: heartbeats.

In recent RabbitMQ versions the heartbeat timeout is 60 seconds but it used to be about 10 minutes in older versions, which means heartbeat frames are sent every roughly 5 minutes, which is close enough to a commonly used TCP connection inactivity value of 5 minutes.

Server logs, load balancer logs and a traffic capture

will help prove this hypothesis.

Staff Software Engineer, Pivotal/RabbitMQ