python 3.0 memory leaking?
rupert.thurner
month for a memory leak nobody is able to find, see
http://groups.google.com/group/trac-dev/browse_thread/thread/116e519da54f16b
.
does python-3.0 improve something to avoid writing memory leaking
applications?
rupert.
Jeroen Ruigrok van der Werven
-On [20080210 20:16], rupert.thurner (rupert....@gmail.com) wrote:
>the edgewall trac release 0.11 is blocked now since more than one
>month for a memory leak nobody is able to find, see
>http://groups.google.com/group/trac-dev/browse_thread/thread/116e519da54f16b
>.
You are slightly mistaken that nobody is able to find anything. There is
currently a lot underway to fix this. The problem was not just one area, but
problems on a number of areas. Given the fact nobody uses a large enough
site with a 0.11 trunk installation these sort of things will start to show
during beta test, as it did. This is still beta period and I personally
would rather release a month later (even despite the already long release
time) than give our users a solution that eats away at their resources like
crazy. I am sure people will appreciate the latter more in the end. Already
the current fixes in trunk for Trac and Genshi stabilized a few large beta
deployment sites in terms of resource eating.
>does python-3.0 improve something to avoid writing memory leaking
>applications?
There are efforts underway to optimize the memory usage for 2.6 on a lot of
levels more than 2.5 already did. Check Python-dev of the last 2-3 weeks.
However, this is not interesting in the short term, since 2.4 and 2.5 (and
later on 2.6) will be versions that will be used for the coming 1-2 years
for sure. Avoidance is very difficult, especially given Python's dynamic
nature. But I'll let more knowledgeable minds talk about that.
For me, a relative newbie to Python, the entire memory allocation issue is
not transparent at all and information about it is scattered across the net.
One of the things I hope to contribute to in the coming year is to make sure
the entire area of Python memory use and proper coding is better understood.
--
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
イェルーン ラウフロック ヴァン デル ウェルヴェン
http://www.in-nomine.org/ | http://www.rangaku.org/
What is to be, will be. And what isn't to be sometimes happens...
Christian Heimes
> does python-3.0 improve something to avoid writing memory leaking
> applications?
No, it doesn't. Python 3.0 and maybe 2.6 are going to have some small
improvements but the improvements aren't related to memory leaking. I'm
working on the matter for a while now. I've limited some free lists and
I've a patch that gives allocated memory back to the OS earlier.
Have you followed my advice and checked all places that deal with
frames, tracebacks, exception objects and __del__ methods?
Christian
Christian Heimes
> For me, a relative newbie to Python, the entire memory allocation issue is
> not transparent at all and information about it is scattered across the net.
> One of the things I hope to contribute to in the coming year is to make sure
> the entire area of Python memory use and proper coding is better understood.
Python uses its own memory allocator for small objecst (< 257 bytes).
Larger objects are allocated directly with malloc, smaller objects end
up in arenas. The code is well documented in
http://svn.python.org/view/python/trunk/Objects/obmalloc.c?rev=56476&view=auto
Several objects keep a free list for performance reasons. Free list save
some extra mallocs and initialization of data structures. I've renamed
all free lists in Python 2.6 to "free_list".
Ints and floats are using their own block allocation algorithm. The code
predates Python's pymalloc code. I'm working on replacing the code with
pymalloc because pymalloc-ed memory is given back to the OS. The int
and float free lists keep their sizes until the Python process ends.
Christian
rupert.thurner
many thanks christian! do you have any code examples how to handle
frames, tracebacks, exception objects and __del__ correctly?
btw, i put your hints on http://wiki.python.org/moin/FreeMemory.
rupert.