SQLAlchemy Memory Leak

[Ahmed Cheikh]

Hello Everybody, 

I used SQLAlchemy in one of my projects. And in this project, I loop over many objects. I use for each object the same engine and session object to query from my database. What I noticed is that for each loop, there is an increase in the memory usage. 

I checked all the other reasons that could potientially lead to a memory leak and thee was none. So I was wondering if any of you faced such a problem? And if so did you find any solution

PS: I tried closing all my sessions after each iteration, delete session object and then garbage collect it, dispose engine and even used the recently added sqlalchemy.orm.session.close_all_sessions()

Thx for your help.

[Jonathan Vanasco]

Memory leaks caused by SqlAlchemy are extremely rare. The code is in heavy production with tens of thousands of users.

If there is a memory leak, it is likely caused by another library, or your code in how you are using SqlAlchemy.

Your next step should be creating a "Short, Self Contained, Correct (Compilable), Example"  (http://sscce.org/) that reproduces the problem and others can analyze/fix/run.

[João Miguel Neves]

Hi,

When you go through the objects, do you remove them from the session with session.expire(obj)?

Hope this helps,

João

--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
 
http://www.sqlalchemy.org/
 
To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description.
---
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/26f16790-685c-42aa-9c36-27970c838ffe%40googlegroups.com.

[Ahmed Cheikh]

I tried the obj.expire(obj) and it did unfortunately not change anything. I still have accumulating used memory for whatever reason. When I did profile the code I saw that sometime engine objects use memory and sometimes session. 

Do any of  you guys have any other suggestions

Le jeudi 27 février 2020 05:17:08 UTC+1, João Miguel Neves a écrit :

Hi,

When you go through the objects, do you remove them from the session with session.expire(obj)?

Hope this helps,

João

On Wed, 26 Feb 2020 at 17:09, Ahmed Cheikh <hajer.ch...@gmail.com> wrote:

Hello Everybody, 

I used SQLAlchemy in one of my projects. And in this project, I loop over many objects. I use for each object the same engine and session object to query from my database. What I noticed is that for each loop, there is an increase in the memory usage. 

I checked all the other reasons that could potientially lead to a memory leak and thee was none. So I was wondering if any of you faced such a problem? And if so did you find any solution

PS: I tried closing all my sessions after each iteration, delete session object and then garbage collect it, dispose engine and even used the recently added sqlalchemy.orm.session.close_all_sessions()

Thx for your help.

--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
 
http://www.sqlalchemy.org/
 
To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description.
---
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sqlal...@googlegroups.com.

[Mike Bayer]

produce a script that reproduces the same results and share it here.   this means writing a small script that looks like the same thing you are doing, but without your whole environment.  if it does not leak memory, then you have to slowly change one element at a time until it looks more and more like your actual application until it starts to leak memory.    that would then provide a clue what the cause might be.

To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/18f34cfb-9d80-4747-9e4c-7e538f813344%40googlegroups.com.