com.rabbitmq.utility.BlockingCell.get(BlockingCell.java:77)

[Vijaya Sagar]

We are using RabbitMQ 3.5.5, Erlang R16B02

at client end .. its spring-rabbit 1.5.0 , amqp-client 3.5.5 , spring amqp 1.5.0 ...

<rabbit:connection-factory id="connectionFactory"  cache-mode="CONNECTION" connection-cache-size="10" connection-factory="cf" />

We are seeing below exceptions on some threads only ... not sure why ... Pl. advice

03 Oct 2015 08:46:01,652 pool-4-thread-21 - Created new connection: SimpleConnection@775925c0 [delegate=amqp://dlc...@135.165.179.110:5673/]

   03 Oct 2015 08:46:01,652 pool-4-thread-21 - Adding new connection 'Dedicated Rabbit Connection: SimpleConnection@775925c0 [delegate=amqp://dlc...@135.165.179.110:5673/]'

   03 Oct 2015 08:46:01,757 pool-4-thread-21 - Creating cached Rabbit Channel from AMQChannel(amqp://dlc...@135.165.179.110:5673/,1)

   03 Oct 2015 08:46:01,770 pool-4-thread-21 - Executing callback on RabbitMQ Channel: Cached Rabbit Channel: AMQChannel(amqp://dlc...@135.165.179.110:5673/,1)

   03 Oct 2015 08:46:01,770 pool-4-thread-21 - Publishing message on exchange [dlcore.device.oidupdate.exchange], routingKey = [!.test7800.!.-540]

   03 Oct 2015 08:46:11,355 pool-4-thread-59 - org.springframework.amqp.AmqpTimeoutException: java.util.concurrent.TimeoutException

at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:70)

at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:276)

at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:526)

at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils$1.createConnection(ConnectionFactoryUtils.java:80)

at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.doGetTransactionalResourceHolder(ConnectionFactoryUtils.java:130)

at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.getTransactionalResourceHolder(ConnectionFactoryUtils.java:67)

at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:1278)

at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:1271)

at org.springframework.amqp.rabbit.core.RabbitTemplate.send(RabbitTemplate.java:619)

at org.springframework.amqp.rabbit.core.RabbitTemplate.send(RabbitTemplate.java:613)

at com.att.dlife.dlcore.mbus.client.publisher.AmqpMessageSenderImpl.send(AmqpMessageSenderImpl.java:98)

at com.att.dlife.dlcore.mbus.client.demo.MessageSendTask.run(MessageSendTask.java:35)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)

at java.lang.Thread.run(Thread.java:662)

Caused by: java.util.concurrent.TimeoutException

at com.rabbitmq.utility.BlockingCell.get(BlockingCell.java:77)

at com.rabbitmq.utility.BlockingCell.uninterruptibleGet(BlockingCell.java:111)

at com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:37)

at com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:367)

at com.rabbitmq.client.impl.AMQChannel.privateRpc(AMQChannel.java:234)

at com.rabbitmq.client.impl.AMQChannel.rpc(AMQChannel.java:212)

at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:327)

at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:648)

at com.rabbitmq.client.DLCoreRabbitMQConnectionFactory.newConnection(DLCoreRabbitMQConnectionFactory.java:158)

at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:695)

at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:264)

... 13 more

[Michael Klishin]

On 3 Oct 2015 at 21:35:43, Vijaya Sagar (sagar....@gmail.com) wrote:
> We are using RabbitMQ 3.5.5, Erlang R16B02
> at client end .. its spring-rabbit 1.5.0 , amqp-client 3.5.5
> , spring amqp 1.5.0 ...
>

> > connection-cache-size="10" connection-factory="cf" />
>
> We are seeing below exceptions on some threads only ... not sure
> why ... Pl. advice
>
> 03 Oct 2015 08:46:01,652 pool-4-thread-21 - Created new connection:

> SimpleConnection@775925c0 [delegate=amqp://dlc...@135.165.179.110:5673/(http://dlc...@135.165.179.110:5673/)]

> 03 Oct 2015 08:46:01,652 pool-4-thread-21 - Adding new connection
> 'Dedicated Rabbit Connection: SimpleConnection@775925c0

> [delegate=amqp://dlc...@135.165.179.110:5673/(http://dlc...@135.165.179.110:5673/)]'

> 03 Oct 2015 08:46:01,757 pool-4-thread-21 - Creating cached

> Rabbit Channel from AMQChannel(amqp://dlc...@135.165.179.110:5673/,1(http://dlc...@135.165.179.110:5673/,1))

> 03 Oct 2015 08:46:01,770 pool-4-thread-21 - Executing callback

> on RabbitMQ Channel: Cached Rabbit Channel: AMQChannel(amqp://dlc...@135.165.179.110:5673/,1(http://dlc...@135.165.179.110:5673/,1))

> 03 Oct 2015 08:46:01,770 pool-4-thread-21 - Publishing message
> on exchange [dlcore.device.oidupdate.exchange], routingKey
> = [!.test7800.!.-540]
> 03 Oct 2015 08:46:11,355 pool-4-thread-59 - org.springframework.amqp.AmqpTimeoutException:
> java.util.concurrent.TimeoutException
> at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:70)
> at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:276)
> at

Connection handshake takes more than the default, e.g. because TCP
connection backlog is too short and you have a surge in incoming TCP
connections.

This is mentioned in http://www.rabbitmq.com/networking.html
and configurable: https://github.com/rabbitmq/rabbitmq-server/blob/master/docs/rabbitmq.config.example#L182.

Finding the root cause would require a careful inspection of a  traffic capture (libpcap/Wireshark).

Handshake timeout is configurable as of Java client 3.5.5:
https://github.com/rabbitmq/rabbitmq-java-client/issues/81
--
MK

Staff Software Engineer, Pivotal/RabbitMQ