Java Client connection leak on recovery

283 views
Skip to first unread message

עילי ודגנית זידמן

unread,
Sep 7, 2017, 3:19:21 AM9/7/17
to rabbitmq-users
Hi All,
 

this leak seems to happen in our live system.

 we are using java client version: 4.1.0 ,server version: 3.6.1, erlang: 18.3

how I can solve this?

thanks!!!!

Arnaud Cogoluègnes

unread,
Sep 7, 2017, 8:22:56 AM9/7/17
to rabbitm...@googlegroups.com
I cannot reproduce with the instructions you gave: connection threads are finished (I use JVisualVM to look at them) and the connections are not accumulating in the management UI. I used the same versions as you.

It's really weird connections show up in the management UI after a complete shutdown of the broker.

Can you please provide us with some simple code (one Java class with a main method) that reproduces the issue? My test consists in opening a connection and consuming from a queue.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Michael Klishin

unread,
Sep 7, 2017, 9:59:53 AM9/7/17
to rabbitmq-users
The most authoritative source of what connections are actually around (and not just in the stats DB, which
is updated every N seconds) is `rabbitmqctl list_connections`. Management plugin stats DB cannot survive
a broker restart because it is entirely in memory, though.

What's the effective heartbeat value (http://www.rabbitmq.com/heartbeats.html)? Have you tried against 3.6.11
and/or a more recent Java client? (the server doesn't have to be updated at the same time)?

Note that all the connections mentioned in rabbitmq-java-client#299 is in the running state, both in the client
and in the server, both as far as the client and the server are concerned. My best guess at the moment

עילי ודגנית זידמן

unread,
Sep 8, 2017, 1:23:50 AM9/8/17
to rabbitmq-users
Hi all,
We found the root cause for the connection leak in our scenario, we are using lyra 0.5.4 with Rabbit client version 4.1.0 and both lyra and rabbit client do recovery to the connection but in different ways, lyra keeps the old connection and Rabbit client create new connection and this causes the connection leak. we reverted the lyra as you can see in this commit: https://github.com/foundation-runtime/communication/commit/dcdb8a1c544d4c4241e4e24e6a14551ea4d400b9 (this is our library that is wrapper around lyra and Rabbit client) and it solved this porblem.


בתאריך יום חמישי, 7 בספטמבר 2017 בשעה 10:19:21 UTC+3, מאת עילי ודגנית זידמן:

Michael Klishin

unread,
Sep 8, 2017, 11:50:27 AM9/8/17
to rabbitmq-users
If you use Lyra, RabbitMQ Java client connection recovery must be disabled.
Disabling Lyra recovery makes little sense because if you do it, you probably don't need Lyra at all.

עילי ודגנית זידמן

unread,
Sep 17, 2017, 2:39:41 AM9/17/17
to rabbitmq-users
Thanks! it works! I set it to false when creating the connection with Lyra:

final ConnectionFactory connectionFactory = options.getConnectionFactory();
connectionFactory
.setAutomaticRecoveryEnabled(false);
connectionFactory
.setTopologyRecoveryEnabled(false);



בתאריך יום שישי, 8 בספטמבר 2017 בשעה 18:50:27 UTC+3, מאת Michael Klishin:
Reply all
Reply to author
Forward
0 new messages