Possible memory leak in pika.BlockingConnection and pika.SelectConnection

563 views
Skip to first unread message

John Larson

unread,
Oct 21, 2020, 3:30:39 PM10/21/20
to Pika
Hello!

I've been using Pika for a few months and I'm loving it. The BlockingConnection makes for really easy, Pythonic use, and it's nice to have the SelectConnection option for when I need to use it in a non-blocking manner.

One thing I've only run into recently is that when a BlockingConnection object or SelectConnection object is created, it doesn't seem to let go of the memory it uses, even if the connection is closed and I don't keep a reference to the connection object.

A minimal reproduction I've created for BlockingConnection

```
import pika

while True:
    pika.BlockingConnection().close()
```

Environment:
OS: macOS Catalina version 10.15.7 (19H2)
Python: 3.9.0
Pika: 1.1.0
RabbitMQ: 3.8.9, installed using docker, with the command `docker run -d -p 5672:5672 rabbitmq:3.8.9`

Expected behavior: If I view the process's memory usage, it will stay fairly level.

Actual behavior: When I view the process in macOS Activity Monitor, the process's memory usage increases over time, at a rate of about 0.5-1 MB/minute.

It also eats memory in a similar way when I run it in Amazon Linux. If you want, I can provide the environment details for that as well.

I considered using SelectConnection as a workaround, but I got similar results:

```
import pika


def on_connected(connection):
    connection.close()


def on_connection_closed(connection, exc):
    connection.ioloop.stop()


while True:
    connection = pika.SelectConnection(on_open_callback=on_connected)
    connection.add_on_close_callback(on_connection_closed)
    connection.ioloop.start()
```

Environment (Same as for BlockingConnection test):
OS: macOS Catalina version 10.15.7 (19H2)
Python: 3.9.0
Pika: 1.1.0
RabbitMQ: 3.8.9, installed using docker, with the command `docker run -d -p 5672:5672 rabbitmq:3.8.9`


Expected behavior: If I view the process's memory usage, it will stay fairly level.

Actual behavior: When I view the process in macOS Activity Monitor, the process's memory usage increases over time, at a rate of about 0.5-1 MB/minute.

Thanks for taking the time to look at this; if you got this far, you read a somewhat long message. :) Is there something I'm not doing correctly here? Is this a known bug?

Thanks,
John

Luke Bakken

unread,
Oct 22, 2020, 11:43:36 AM10/22/20
to Pika
Hi John,

Thanks for the very detailed report. I'll try to find time to investigate.

Luke

John Larson

unread,
Dec 15, 2020, 5:23:36 PM12/15/20
to pika-...@googlegroups.com
Should I go ahead and file an issue on GitHub for this?

--
You received this message because you are subscribed to a topic in the Google Groups "Pika" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pika-python/ZMOZbiQs9cE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pika-python...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pika-python/4fdff392-2a2c-4db5-b26d-c5bcbe30869co%40googlegroups.com.

Luke Bakken

unread,
Dec 15, 2020, 5:55:28 PM12/15/20
to Pika
Please do. I haven't yet had time to reproduce this.
Reply all
Reply to author
Forward
0 new messages