Java client - CancelledKeyException in NioLoop leading to memory issue

[Rémi Baptiste]

Hello,

I'm investigating an issue raised by one of our user. It seems that after a certain amount of time, a CancelledKeyException is thrown in the NioLoop which stops the NioLoop thread and after this every 10 seconds, this exception appears :

ERROR c.r.c.impl.ForgivingExceptionHandler - Caught an exception during connection recovery!
java.util.concurrent.TimeoutException: null
at com.rabbitmq.utility.BlockingCell.get(BlockingCell.java:77)

The recovery connection loop creates each time a set of object like RecoveryAwareAMQConnection, SocketChannelFrameHandler, SocketChannelFrameHandlerState ... which lead to the memory issue.

We are using the amqp-client 5.14.2 with on top of it the reactor-rabbit in version 1.5.6.

I had a look at the NioLoop code and I found that this exception could only be thrown here https://github.com/rabbitmq/rabbitmq-java-client/blob/7ec9a56b6472b8c1177f008e2e98848bfeb743ef/src/main/java/com/rabbitmq/client/impl/nio/NioLoop.java#L139 or here https://github.com/rabbitmq/rabbitmq-java-client/blob/7ec9a56b6472b8c1177f008e2e98848bfeb743ef/src/main/java/com/rabbitmq/client/impl/nio/NioLoop.java#L225.

As I am not able able to reproduce the issue so far, I was wondering if there was a reason not to add the call to key.isReadable() or key.isWritable() inside the try/catch used just below ?

BR,
Rémi

[Arnaud Cogoluègnes]

Hi, Rémi.

This is a bug, thanks for the analysis. I pushed a fix [1], can you try out the latest snapshot [2]?

[1] https://github.com/rabbitmq/rabbitmq-java-client/pull/1493

[2] https://github.com/rabbitmq/rabbitmq-java-client?tab=readme-ov-file#snapshots

[Rémi Baptiste]

Thanks Arnaud for the quick reply and fix.
I am still trying to have a reproducer but I will keep you informed as soon as I have with a status for the fix.

Regards,
Rémi 

[Rémi Baptiste]

Hi Arnaud,

As I expected, I did not manage to reproduce the initial issue but I've made a lot of tests and run our integration tests without any issue with the snapshot.
So for me it's all good.

BR,

Rémi

[Arnaud Cogoluègnes]

Hi, Rémi.

Thanks for the follow-up, 5.24.0 is out [1] with the fix.

[1] https://github.com/rabbitmq/rabbitmq-java-client/releases/tag/v5.24.0