Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

python has memory leak?

6 views
Skip to first unread message

yzg...@gmail.com

unread,
Apr 22, 2008, 5:54:37 PM4/22/08
to
Hi all,

I feel that my python script is leaking memory. And this is a test I
have:

log.write("[" + timestamp() + "] " + "test() ... memory usage: " +
" ".join(repr(i/(1024*1024)) for i in getMemInfo()) + "\n")
m = {}
i = 1000*1000
while i > 0:
i = i - 1
m.setdefault(i, []).append(i)
log.write("[" + timestamp() + "] " + "test() ... memory usage: " +
" ".join(repr(i/(1024*1024)) for i in getMemInfo()) + "\n")
m = {}
log.write("[" + timestamp() + "] " + "test() done. memory usage: "
+ " ".join(repr(i/(1024*1024)) for i in getMemInfo()) + "\n")

From which I got:

[17:44:50] test() ... memory usage: 55L 55L
[17:44:53] test() ... memory usage: 143L 143L
[17:44:53] test() done. memory usage: 125L 143L

In the above code getMemInfo is my func to return current and peak
memory usage in bytes. Can some expert explain how python manages
memory?

The version of my python is:
Python 2.4.4 Stackless 3.1b3 060516 (#71, Oct 31 2007, 14:22:28) [MSC
v.1310 32 bit (Intel)] on win32

Many thanks,

GH

Christian Heimes

unread,
Apr 22, 2008, 6:59:54 PM4/22/08
to pytho...@python.org
yzg...@gmail.com schrieb:

> The version of my python is:
> Python 2.4.4 Stackless 3.1b3 060516 (#71, Oct 31 2007, 14:22:28) [MSC
^^^^^^^^^
This is the wrong list to ask for memory leaks of Stackless ;)

Christian

Jean-Paul Calderone

unread,
Apr 22, 2008, 9:58:39 PM4/22/08
to pytho...@python.org
On Tue, 22 Apr 2008 14:54:37 -0700 (PDT), yzg...@gmail.com wrote:
>Hi all,
>
>I feel that my python script is leaking memory. And this is a test I
>have:
>
> [snip]
>

The test doesn't demonstrate any leaks. It does demonstrate that memory
usage can remain at or near peak memory usage even after the objects for
which that memory was allocated are no longer live in the process. This
is only a leak if peak memory goes up again each time you create any new
objects. Try repeated allocations of a large dictionary and observe how
memory usage rises and falls.

Python 2.5 does a somewhat better job of releasing memory when actual use
falls below peak, but this is a difficult thing to do perfectly.

Jean-Paul

yzg...@gmail.com

unread,
Apr 23, 2008, 10:44:23 AM4/23/08
to
Thanks to Christian and Jean-Paul for your reply. I moved away from
the stackless version (which I don't understand what uniqueness it
comes with) and downloaded a fresh copy of Python 2.5.2 (r252:60911,
Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32. Following
J-P's suggestion, I included the block to be tested in a loop:

i=20
while i>0:
i=i-1
test(log)
log.flush()

And logged the memory usage in bytes instead of MB. The result as
included below shows the peak memory usage keeps going up in the first
7 iterations. Can someone point to some docs or books about how python
manages memory or share some experience?

Thanks,

-GH


[10:27:54] test() starting ... memory usage: 4599808L 4599808L
[10:27:56] test() ... memory usage: 107499520L 107499520L
[10:27:56] test() done. memory usage: 41754624L 107503616L
[10:27:56] test() starting ... memory usage: 41754624L 107503616L
[10:27:59] test() ... memory usage: 108216320L 108400640L
[10:27:59] test() done. memory usage: 58884096L 108400640L
[10:27:59] test() starting ... memory usage: 58884096L 108400640L
[10:28:02] test() ... memory usage: 108220416L 108433408L
[10:28:02] test() done. memory usage: 42233856L 108433408L
[10:28:02] test() starting ... memory usage: 42233856L 108433408L
[10:28:04] test() ... memory usage: 108228608L 108441600L
[10:28:05] test() done. memory usage: 75149312L 108441600L
[10:28:05] test() starting ... memory usage: 75149312L 108441600L
[10:28:07] test() ... memory usage: 108498944L 108707840L
[10:28:07] test() done. memory usage: 41992192L 108707840L
[10:28:07] test() starting ... memory usage: 41992192L 108707840L
[10:28:10] test() ... memory usage: 108122112L 108707840L
[10:28:10] test() done. memory usage: 58523648L 108707840L
[10:28:10] test() starting ... memory usage: 58523648L 108707840L
[10:28:13] test() ... memory usage: 108642304L 108855296L
[10:28:13] test() done. memory usage: 41873408L 108855296L
[10:28:13] test() starting ... memory usage: 41873408L 108855296L
[10:28:16] test() ... memory usage: 108224512L 108855296L
[10:28:16] test() done. memory usage: 58626048L 108855296L
[10:28:16] test() starting ... memory usage: 58626048L 108855296L
[10:28:18] test() ... memory usage: 108224512L 108855296L
[10:28:19] test() done. memory usage: 41975808L 108855296L
[10:28:19] test() starting ... memory usage: 41975808L 108855296L
[10:28:21] test() ... memory usage: 108224512L 108855296L
[10:28:22] test() done. memory usage: 58626048L 108855296L
[10:28:22] test() starting ... memory usage: 58626048L 108855296L
[10:28:24] test() ... memory usage: 108224512L 108855296L
[10:28:24] test() done. memory usage: 41975808L 108855296L
[10:28:24] test() starting ... memory usage: 41975808L 108855296L
[10:28:27] test() ... memory usage: 108224512L 108855296L
[10:28:27] test() done. memory usage: 58626048L 108855296L
[10:28:27] test() starting ... memory usage: 58626048L 108855296L
[10:28:30] test() ... memory usage: 108224512L 108855296L
[10:28:30] test() done. memory usage: 41975808L 108855296L
[10:28:30] test() starting ... memory usage: 41975808L 108855296L
[10:28:32] test() ... memory usage: 108224512L 108855296L
[10:28:33] test() done. memory usage: 58626048L 108855296L
[10:28:33] test() starting ... memory usage: 58626048L 108855296L
[10:28:35] test() ... memory usage: 108224512L 108855296L
[10:28:36] test() done. memory usage: 41975808L 108855296L
[10:28:36] test() starting ... memory usage: 41975808L 108855296L
[10:28:38] test() ... memory usage: 108224512L 108855296L
[10:28:38] test() done. memory usage: 58626048L 108855296L
[10:28:38] test() starting ... memory usage: 58626048L 108855296L
[10:28:41] test() ... memory usage: 108224512L 108855296L
[10:28:41] test() done. memory usage: 41975808L 108855296L
[10:28:41] test() starting ... memory usage: 41975808L 108855296L
[10:28:44] test() ... memory usage: 108224512L 108855296L
[10:28:44] test() done. memory usage: 58626048L 108855296L
[10:28:44] test() starting ... memory usage: 58626048L 108855296L
[10:28:46] test() ... memory usage: 108224512L 108855296L
[10:28:47] test() done. memory usage: 41975808L 108855296L
[10:28:47] test() starting ... memory usage: 41975808L 108855296L
[10:28:49] test() ... memory usage: 108224512L 108855296L
[10:28:50] test() done. memory usage: 58626048L 108855296L


0 new messages