python issue 4688

[Robert Schuppenies]

Hi guys.

I just tested pympler with the current trunk and py3k branch and one of my
tests is failing. Turns out that they changed the garbage collector behavior a
bit[1].

What happens it that in future Python versions container objects such as dict
and list will be handled by the garbage collector _only_ if they reference
other container objects.

Take the following example:

>>> import gc
>>> s = 'some string'
>>> d = {1: s, 2: 2}

In Python 2.6 and 3.0 gc.get_referrers would tell you that s is referred to by d:

>>> d in gc.get_referrers(s)
True
>>> s in gc.get_referents(d)
True

But not so in 2.7 and 3.1

>>> d in gc.get_referrers(s)
False
>>> s in gc.get_referents(d)
True

You'll notice that "gc.get_referents(d)" still works, though. Now adding a
container object to d will make d handled by the garbage collector, again:

>>> d[3] = []
>>> d in gc.get_referrers(s)
True
>>> s in gc.get_referents(d)
True

What are the implications for us? muppy has some problems which I think we can
fix, but I havn't dealt with it yet. And asizeof may again not return objects
referenced by iterators. The old heapmonitor code seems not to be affected.

Please let me know if you see any other effects this change may have on pympler.

cheers,
robert

[1]
http://bugs.python.org/issue4688http://bugs.python.org/issue4688http://bugs.python.org/issue4688

[Jean Brouwers]

Here is my take on this.  For sizing given objects, it makes no difference for asizeof, since only gc.get_referents is used in that case (and gc.get_referers is not).

But when called with the -all option to size all managed objects, asizeof uses gc.get_objects and that may not return all container objects since "immutable containers" are no longer managed.

Btw, it would nice if gc had an option, called, say manage_all.  If set to True, all objects would be managed.  If False, "immutable" objects are not managed as in <http://bugs.python.org/issue4688>.  The default for manage_all is False.

/Jean