Debugging memory leaks

95 views
Skip to first unread message

Anupama Goparaju

unread,
Jan 5, 2022, 8:21:36 PM1/5/22
to sqlalchemy
Hi,

We are trying to debug memory leaks in our flask app that is using SQLAlchemy version 1.3.16. When i was using tracemalloc python library and comparing the memory snapshots,
I found the below lines repetitively appearing in the snapshot differences: Could you please help us in providing us some pointers if we aren't using the lib properly or is there some caching going on that is causing this similar to what is mentioned here - https://giters.com/sqlalchemy/sqlalchemy/issues/7242.

sqlalchemy/util/langhelpers.py:1429: size=3687 KiB (+18.5 KiB), count=3395 (+17), average=1112 B
sqlalchemy/sql/visitors.py:95: size=22.8 KiB (+5376 B), count=147 (+56), average=159 B
sqlalchemy/sql/schema.py:1346: size=2301 KiB (+3944 B), count=10157 (+17), average=232 B
sqlalchemy/sql/schema.py:1345: size=2301 KiB (+3944 B), count=10157 (+17), average=232 B

Thanks,
Anupama
 

Mike Bayer

unread,
Jan 6, 2022, 8:28:00 AM1/6/22
to noreply-spamdigest via sqlalchemy
1.3.16 doesn't have a lot of caching going on compared to the 1.4 series.  it's expected that memory in an ORM application will grow for a few iterations as there is some caching of things, but a "leak" is defined as growth that never stops.   you'd want to verify that the growth you see does in fact continue without stopping until it gets into the tens or hundreds of gigs without ever going down, I usually use OS-level memory tools like top to verify this.

the line numbers shown seem to indicate schema objects like Table / Column / ForeignKey instances are being created.  These are usually created for a MetaData collection at the module level and would not be constructed on a continuous basis so you might want to look for patterns where Table objects are being created per-request, something like that.
--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
 
 
To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description.
---
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+...@googlegroups.com.

Anupama Goparaju

unread,
Jan 11, 2022, 8:03:01 PM1/11/22
to sqlalchemy
Thanks.
Do you have a minimalistic example on the lifecycle of table objects? Or best practices of create, use, and destroy them?
How do we identify if they are continuously created and not destroyed?

Mike Bayer

unread,
Jan 12, 2022, 9:49:41 AM1/12/22
to noreply-spamdigest via sqlalchemy
Table objects are normally created at the module level and are persistent for the full lifetime of the application.
Reply all
Reply to author
Forward
0 new messages