Closing Connections

[Scott Fredrickson]

Hello,

I am wondering if anyone has some insight here, if so I would really appreciate it. 

I have a fairly simple queue setup, there's a publisher and say 10-25 consumers. Very frequently, even for say a small set of 100 requests I receive the following error:

The connection was not open (or was forcibly closed), attempting to reestablish it System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host..

I've tried all the things I can think of including putting the latest broker on the same machine as most of the consumers.

This is a C# process, there is a separate connection for the producers and the consumers. The consumers are listening on MyTopic.requests and publish the results out on MyTopic.results, acknowledgement is manual. When a message is received, the callback is 

        public void ProcessMessage(MyRequestObject myRequestObject)

        {

            var myCallbackProcessor = new MyCallbackProcessor(myRequestObject);

            _ = Task.Run(() => myCallbackProcessor.Calc());

            Logger($"{Status.WorkerName}:\tReady on {WorkTopic}");

        }

This is the stack trace below. Please note, the exception occurs when trying to publish the results back. If there's anything I've left out, or that would be helpful, please let me know.

Thank you,

Scott

at RabbitMQ.Client.Impl.ModelBase.BasicPublish(String exchange, String routingKey, Boolean mandatory, IBasicProperties basicProperties, ReadOnlyMemory`1 body)
at RabbitMQ.Client.Impl.ModelBase.ModelSend(MethodBase method, ContentHeaderBase header, ReadOnlyMemory`1 body)
at RabbitMQ.Client.Impl.SessionBase.Transmit(OutgoingCommand& cmd)
at RabbitMQ.Client.Framing.Impl.Connection.MainLoop()
at RabbitMQ.Client.Framing.Impl.Connection.MainLoopIteration()
at RabbitMQ.Client.Impl.SocketFrameHandler.ReadFrame()
at RabbitMQ.Client.Impl.InboundFrame.ReadFrom(Stream reader, Byte[] frameHeaderBuffer, ArrayPool`1 pool, UInt32 maxMessageSize)
at System.IO.BufferedStream.ReadByteSlow()
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
--- End of inner exception stack trace ---
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
---> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host.
RabbitMQ.Client.Exceptions.AlreadyClosedException: Already closed: The AMQP operation was interrupted: AMQP close-reason, initiated by Library, code=541, text='Unexpected Exception', classId=0, methodId=0, cause=System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host..

2023-01-23 18:15:59.553 ERROR Messaging.Producer {MqConnection.Name} - Failed publishing to MyServer w/ MyLibrary

at RabbitMQ.Client.Framing.Impl.Connection.MainLoop()
at RabbitMQ.Client.Framing.Impl.Connection.MainLoopIteration()
at RabbitMQ.Client.Impl.SocketFrameHandler.ReadFrame()
at RabbitMQ.Client.Impl.InboundFrame.ReadFrom(Stream reader, Byte[] frameHeaderBuffer, ArrayPool`1 pool, UInt32 maxMessageSize)
at System.IO.BufferedStream.ReadByteSlow()
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
--- End of inner exception stack trace ---
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
---> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host.

2023-01-23 18:15:59.551 WARN MyLibrary.Messaging.Producer The connection was not open (or was forcibly closed), attempting to reestablish it System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host..

[Scott Fredrickson]

Circling back on this - does anyone have any insights or thoughts here?

[Luke Bakken]

Hello,

I realize it has been a very long time since your original message.

Do you know what RabbitMQ logged at the same time as the "System.Net.Sockets.SocketException (10054)" exception?

Thanks,

Luke