RabbitMQ Dotnet Client ConnectionRecoveryError Callback IOException

[Boran Taylan BALCI]

Hello,
I have a running RabbitMQ container and I am trying an integration test. What I do is restarting container with certain shutdown time and observe the auto recovery (I enabled it) and ConnectionRecoveryError Callback.
While I observe ConnectionRecoveryError, I realized it is raising the handler and giving an IOException -> "connection.start never received ...." regardless it is successfully recovered or not. When I look for an answer I always see something related to ssl. I am reaching my container on localhost:5672.

Currently I am skipping(swallowing) that IOException which made me afraid that I might be missing an unhealthy status of the broker. 
Please inform me with further information,
Best regards,
Boran

[Michael Klishin]

Start with inspecting and sharing the logs [1], and providing some details on what exactly your integration test does.

Connection recovery will retry *connecting* every 5 seconds by default. Initial connection failures are up to you to handle.

1. https://www.rabbitmq.com/logging.html#logged-events

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/6b2b83ec-b975-41b4-858a-e0343d1204bf%40googlegroups.com.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Boran Taylan BALCI]

Hello Michael,
Apologies for the late reply, Logs can't tell me much since I am trashing the docker container that runs RabbitMQ. What I try with my integration test step by step is the following(using RabbitMQ .NET client):

- Step 1 Docker start

- Step 2 Setup connection, channel.

- Step 3 Assign a callback for ConnectionRecoveryError

- Step 4 Publish

- Step 5 Docker stop

- Step 6 Quickly after try to publish -> I didn't know how it will behave, I thought It d block the thread till 5 second until its sure connection is gone but instead it just throws exception(not reachable broker). Anyway still this is fine.

- Step 7 Quickly after docker start

- Step 8 Publish

- Step 9 Assert both Publishes have been succesfully made while docker running(Step 4 and step 8). -> passes

- Step 9 Assert Callback NOT called -> fails

It calls the callback with IOException -> "connection.start never received ....",

Could you explain why I am getting this? To my opinion the callback shouldn't be called with any exception at all since recovery was successful. And one other case is when I shutdown the container for long time it really cant recover the connection. So its calling the callback as expected but again with additional this IOException, so I receive one aggregate exception(only this expected) and additionally this IOException (not expected) with same message -> "connection.start never received ...."

It would be great if one can tell me how to approach this.

Best regards,
Boran

On Tuesday, January 7, 2020 at 4:06:23 PM UTC+1, Michael Klishin wrote:

Start with inspecting and sharing the logs [1], and providing some details on what exactly your integration test does.

Connection recovery will retry *connecting* every 5 seconds by default. Initial connection failures are up to you to handle.

1. https://www.rabbitmq.com/logging.html#logged-events

On Tue, Jan 7, 2020 at 4:43 PM Boran Taylan BALCI <borantay...@medel.com> wrote:

Hello,
I have a running RabbitMQ container and I am trying an integration test. What I do is restarting container with certain shutdown time and observe the auto recovery (I enabled it) and ConnectionRecoveryError Callback.
While I observe ConnectionRecoveryError, I realized it is raising the handler and giving an IOException -> "connection.start never received ...." regardless it is successfully recovered or not. When I look for an answer I always see something related to ssl. I am reaching my container on localhost:5672.

Currently I am skipping(swallowing) that IOException which made me afraid that I might be missing an unhealthy status of the broker. 
Please inform me with further information,
Best regards,
Boran

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitm...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/6b2b83ec-b975-41b4-858a-e0343d1204bf%40googlegroups.com.

[Michael Klishin]

See server logs, all connection attempts that RabbitMQ registers would be logged [1].

It can be that the restarted container does not boot fast enough, does not open ports fast enough and so on.

Having a port open does not mean that there's a fully booted node behind it ready to serve traffic.

Generally, a client should not assume that connections will succeed instantly, nor you the developer believe that events/peer unavailability in a distributed

system are/is detected momentarily [2].

1. https://www.rabbitmq.com/logging.html#logged-events

2. https://www.rabbitmq.com/heartbeats.html

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/ed097bc8-f115-4714-a902-1fd2773f7a17%40googlegroups.com.