Re: [gevent] Apparent memory leakage in gevent

[Denis Bilenko]

On Fri, Dec 14, 2012 at 9:06 AM, Joon-Kyu Park <segfa...@gmail.com> wrote:
> Hi. I think there's a memory leak in gevent. I reproduced this via following
> code.
>
...
> As you can see there are a lot of green threads created.
> According to objgraph.most_common_types(), Python heap seems to be reclaimed
> after every threads are finished but the actual memory usage remains
> unchanged.
> I suspect that gevent.sleep() call in f() leaves an amount of memory in
> native (libevent side or something?) memory. Leak doesn't occur after
> commenting that line out.
> Of course there's no need for explicit GC call but I wanted to make sure
> that the memory to be recollected properly.
> Is this a bug, or I just misused it?

I don't get what the script is supposed to show and what is not right.
I'll take another look, if you could simplify it by

- removing 3rdparty packages, like objgraph and psutil
- making the script loop infinitely.

(Use ps/top to see that the memory is leaking.)

Thanks,
Denis.

[Ralf Schmitt]

Heungsub Lee <h...@subl.ee> writes:

> Hi Denis, I'm Heungsub Lee, a co-worker of Joon-kyu.
>
> I simplified the code: https://gist.github.com/4265015#file-gevent_leak2-py
> It would be great if you take another look at our case again. Thank you.

If you wrap your main function in another loop, memory usage stays
constant at around 2.4GB. It wouldn't do that if there was a memory
leak.

[Joon-Kyu Park]

2012년 12월 17일 월요일 오후 4시 46분 12초 UTC+9, Ralf Schmitt 님의 말:

Then which part of code is causing Python to hold up to gigabytes of memory

after execution?

After every jobs are done they are gone (by jobs = None statement) and shouldn't

be used thereafter.

If this is obviously not a memory leak, am I missing something?

Regards,

Park Joon-Kyu

[Ralf Schmitt]

Joon-Kyu Park <segfa...@gmail.com> writes:

> Then which part of code is causing Python to hold up to gigabytes of memory
> after execution?
> After every jobs are done they are gone (by jobs = None statement) and
> shouldn't
> be used thereafter.
> If this is obviously not a memory leak, am I missing something?

The memory is not given back to the operating system. It's being reused
when needed by the process. The following stackoverflow page may contain
more information:
http://stackoverflow.com/questions/1421491/does-calling-free-or-delete-ever-release-memory-back-to-the-system