RabbitMQ AMQP operations fail using .NET Client

[Simone Musetti]

Erlang version: 21.2

RabbitMQ version: 3.7.12

Rabbit .Net Client: 3.6.10.0

Environment:

- Virtual machine with Windows Server 2016

- No antivirus, neither any endpoint policies

Hi all,

from time to time I'm experiencing an annoying problem on a RabbitMQ cluster (composed by two VMs). When performing an operation using .NET Client I get this type of exception:

Exception Type {RabbitMQ.Client.Exceptions.AlreadyClosedException}

Message {Already closed: The AMQP operation was interrupted: AMQP close-reason, initiated by Application, code=200, text="Goodbye", classId=0, methodId=0, cause=}

StackTrace {

   at RabbitMQ.Client.Impl.SessionBase.Transmit(Command cmd)

   at RabbitMQ.Client.Impl.ModelBase.ModelSend(MethodBase method, ContentHeaderBase header, Byte[] body)

   at RabbitMQ.Client.Framing.Impl.Model._Private_BasicGet(String queue, Boolean noAck)

   at RabbitMQ.Client.Impl.ModelBase.BasicGet(String queue, Boolean noAck)

RabbitMQ.Client.Exceptions.OperationInterruptedException: The AMQP operation was interrupted: AMQP close-reason, initiated by Application, code=200, text="Goodbye", classId=0, methodId=0, cause=

   at RabbitMQ.Client.Impl.SimpleBlockingRpcContinuation.GetReply(TimeSpan timeout)

   at RabbitMQ.Client.Impl.ModelBase.QueueDeclare(String queue, Boolean passive, Boolean durable, Boolean exclusive, Boolean autoDelete, IDictionary`2 arguments)

   at RabbitMQ.Client.Impl.ModelBase.QueueDeclare(String queue, Boolean durable, Boolean exclusive, Boolean autoDelete, IDictionary`2 arguments)

On RabbitMQ logs, at the same time, I can see this

[error] <0.1298.0> closing AMQP connection <0.1298.0> ([::1]:54074 -> [::1]:5672):

missed heartbeats from client, timeout: 60s

Seems like some application is closing the connection abruptly. No antivirus is engaged in this environment.

Do you might have any clue about the problem that the installation is experiencing?

Thank you in advance, Simone.

[Michael Klishin]

You have evidence of missed heartbeats reported by the server [1]. On top of that,

the client reports that it has closed the connection with exit code of 200 while (potentially another thread) tried to declare a queue:

> RabbitMQ.Client.Exceptions.OperationInterruptedException: The AMQP operation was interrupted: AMQP close-reason, initiated by Application

The "initiated by" part suggests that it was your application code and

not the peer (server) or the library that has initiated connection closure. So you have to take a look at what your own code does with this connection.

RabbitMQ .NET client 3.6.10 was released over 3 years ago. The latest version is 6.1.0.

Please upgrade before you put any more time into investigating this.

1. https://www.rabbitmq.com/heartbeats.html
   

---

From: rabbitm...@googlegroups.com <rabbitm...@googlegroups.com> on behalf of Simone Musetti <simone....@gmail.com>
Sent: Thursday, June 11, 2020 12:30
To: rabbitmq-users <rabbitm...@googlegroups.com>
Subject: [rabbitmq-users] RabbitMQ AMQP operations fail using .NET Client

 

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/3562a164-4f42-4f02-8f16-e231c2104593o%40googlegroups.com.

[Simone Musetti]

Il giorno giovedì 11 giugno 2020 14:11:01 UTC+2, Michael Klishin ha scritto:

You have evidence of missed heartbeats reported by the server [1]. On top of that,

the client reports that it has closed the connection with exit code of 200 while (potentially another thread) tried to declare a queue:

> RabbitMQ.Client.Exceptions.OperationInterruptedException: The AMQP operation was interrupted: AMQP close-reason, initiated by Application

The "initiated by" part suggests that it was your application code and

not the peer (server) or the library that has initiated connection closure. So you have to take a look at what your own code does with this connection.

RabbitMQ .NET client 3.6.10 was released over 3 years ago. The latest version is 6.1.0.

Please upgrade before you put any more time into investigating this.

1. https://www.rabbitmq.com/heartbeats.html
   

---

From: rabbitm...@googlegroups.com <rabbitm...@googlegroups.com> on behalf of Simone Musetti <simone...@gmail.com>

Thank you for your response. Unfortunately I cannot update the RabbitMQ .NET Client to the latest version. I found another strange behaviour.When I try to perform a queue bind creation, I get a Peer error even if the RabbitMQ instances is up and running. The strange part is: RabbitMQ seems to refuse the QueueBind operation cause the binding does not exist, while I'm creating it. Exception is the following:

RabbitMQ.Client.Exceptions.OperationInterruptedException: The AMQP operation was interrupted: AMQP close-reason, initiated by Peer, code=404, text="NOT_FOUND - no binding MyBinding between exchange 'queue1' in vhost 'myVhost' and queue 'queue1' in vhost 'myVhost'", classId=50, methodId=20, cause=

   at RabbitMQ.Client.Impl.SimpleBlockingRpcContinuation.GetReply(TimeSpan timeout)

   at RabbitMQ.Client.Impl.ModelBase.ModelRpc(MethodBase method, ContentHeaderBase header, Byte[] body)

   at RabbitMQ.Client.Framing.Impl.Model._Private_QueueBind(String queue, String exchange, String routingKey, Boolean nowait, IDictionary`2 arguments)

   at RabbitMQ.Client.Impl.ModelBase.QueueBind(String queue, String exchange, String routingKey)

I cannot understand why the operation is failing from time to time, while I think it is related to the previous exceptions. Do you have any idea?