Wow: list of immediate subclasses?
Roman Suzi
I am very surprised that modern classes (inherited from object)
in Python 2.3 show their ancestry, eg:
>>> object.__subclasses__()
[<type 'type'>, <type 'int'>, <type 'basestring'>, <type 'list'>, <type
'NoneType'>, <type 'NotImplementedType'>, <type 'module'>, <type
'zipimport.zipimporter'>, <type 'posix.stat_result'>, <type
'posix.statvfs_result'>, <type 'dict'>, <type 'function'>, <type 'dictproxy'>,
<type 'member_descriptor'>, <type 'getset_descriptor'>, <type 'tuple'>, <type
'wrapper_descriptor'>, <type 'builtin_function_or_method'>, <type
'method_descriptor'>, <type 'file'>]
This seems cool introspection tool but raises questions:
1- I wonder if it has security consequences (knowing what other
classes are in there, probably in some other modules). As it adds
a hole starting from base class "object". Earlier it was possible
to "chroot" to some custom builtin namespace and provide
restricted execution.
2- how do I get rid of a class completely?
This doesn't work:
>>> object.__subclasses__()
[<type 'type'>, <type 'int'>, <type 'basestring'>, <type 'list'>, <type
'NoneType'>, <type 'NotImplementedType'>, <type 'module'>, <type
'zipimport.zipimporter'>, <type 'posix.stat_result'>, <type
'posix.statvfs_result'>, <type 'dict'>, <type 'function'>]
>>> class A(object): pass
...
>>> object.__subclasses__()
[<type 'type'>, <type 'int'>, <type 'basestring'>, <type 'list'>, <type
'NoneType'>, <type 'NotImplementedType'>, <type 'module'>, <type
'zipimport.zipimporter'>, <type 'posix.stat_result'>, <type
'posix.statvfs_result'>, <type 'dict'>, <type 'function'>, <class
'__main__.A'>]
>>> del A
>>> object.__subclasses__()
[<type 'type'>, <type 'int'>, <type 'basestring'>, <type 'list'>, <type
'NoneType'>, <type 'NotImplementedType'>, <type 'module'>, <type
'zipimport.zipimporter'>, <type 'posix.stat_result'>, <type
'posix.statvfs_result'>, <type 'dict'>, <type 'function'>, <class
'__main__.A'>]
>>>
Sincerely yours, Roman Suzi
--
r...@onego.ru =\= My AI powered by GNU/Linux RedHat 7.3
Carl Banks
__subclasses__ is a list of weak references, so the class can be
collected even though it's listed in __subclasses__. For whatever
reason, class A isn't deleted by reference counts, but cyclic garbage
collection gets it.
Python 2.2.3c1 (#12, May 27 2003, 21:32:04)
[GCC 2.95.4 20011002 (Debian prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class A(object): pass
...
>>> object.__subclasses__()
[<type 'type'>, <type 'list'>, <type 'NoneType'>, <type 'NotImplementedT
ype'>, <type 'module'>, <type 'posix.stat_result'>, <type 'posix.statvfs
_result'>, <type 'dict'>, <type 'function'>, <type 'str'>, <type 'file'>
, <class '__main__.A'>]
>>> del A
>>> object.__subclasses__()
[<type 'type'>, <type 'list'>, <type 'NoneType'>, <type 'NotImplementedT
ype'>, <type 'module'>, <type 'posix.stat_result'>, <type 'posix.statvfs
_result'>, <type 'dict'>, <type 'function'>, <type 'str'>, <type 'file'>
, <class '__main__.A'>]
>>> 1
1
>>> 1
1
>>> 1
1
>>> import gc
>>> gc.collect()
6
>>> object.__subclasses__()
[<type 'type'>, <type 'list'>, <type 'NoneType'>, <type 'NotImplementedT
ype'>, <type 'module'>, <type 'posix.stat_result'>, <type 'posix.statvfs
_result'>, <type 'dict'>, <type 'function'>, <type 'str'>, <type 'file'>
]
--
CARL BANKS
Andrew Dalke
> __subclasses__ is a list of weak references, so the class can be
> collected even though it's listed in __subclasses__. For whatever
> reason, class A isn't deleted by reference counts, but cyclic garbage
> collection gets it.
But I want to know that reason. Why are there any cycles? After
all, don't weak-references help break cycles? And in my code
before, why do I need to do the gc a few times to force the cycle
to be broken?
>>> float.__subclasses__()
[]
>>> class Spam(float): pass
...
>>> float.__subclasses__()
[<class '__main__.Spam'>]
>>> del Spam
>>> _ = None # just in case ...
>>> float.__subclasses__()
[<class '__main__.Spam'>]
>>> gc.collect()
0
>>> float.__subclasses__()
[<class '__main__.Spam'>]
>>> gc.collect()
0
>>> float.__subclasses__()
[<class '__main__.Spam'>]
>>> gc.collect()
0
>>> gc.collect()
6
>>> gc.collect()
0
>>> float.__subclasses__()
[]
>>>
It's a bit too mysterious for my liking.
Andrew
da...@dalkescientific.com
Michael Hudson
> Carl Banks, responding to Roman Suzi:
> > __subclasses__ is a list of weak references, so the class can be
> > collected even though it's listed in __subclasses__. For whatever
> > reason, class A isn't deleted by reference counts, but cyclic garbage
> > collection gets it.
>
> But I want to know that reason. Why are there any cycles? After
> all, don't weak-references help break cycles? And in my code
> before, why do I need to do the gc a few times to force the cycle
> to be broken?
>
> >>> float.__subclasses__()
> []
> >>> class Spam(float): pass
> ...
> >>> float.__subclasses__()
> [<class '__main__.Spam'>]
> >>> del Spam
> >>> _ = None # just in case ...
This doesn't do what you think it does. The reference to Spam is in
__builtin__._; this only assigns to __main__._.
I don't know where the cycle is, off-hand. I suspect I did once...
[...]
> It's a bit too mysterious for my liking.
I can manage not to care :-)
Cheers,
mwh
--
It's actually a corruption of "starling". They used to be carried.
Since they weighed a full pound (hence the name), they had to be
carried by two starlings in tandem, with a line between them.
-- Alan J Rosenthal explains "Pounds Sterling" on asr
Michael Hudson
> "Andrew Dalke" <ada...@mindspring.com> writes:
>
> > Carl Banks, responding to Roman Suzi:
> > > __subclasses__ is a list of weak references, so the class can be
> > > collected even though it's listed in __subclasses__. For whatever
> > > reason, class A isn't deleted by reference counts, but cyclic garbage
> > > collection gets it.
> >
> > But I want to know that reason. Why are there any cycles?
[...]
>
> I don't know where the cycle is, off-hand. I suspect I did once...
Found it (them):
>>> class A(object): pass
...
>>> A.__dict__['__weakref__'].__objclass__ is A
True
praise-be-to-neils-ly y'rs
mwh
--
I never disputed the Perl hacking skill of the Slashdot creators.
My objections are to the editors' taste, the site's ugly visual
design, and the Slashdot community's raging stupidity.
-- http://www.cs.washington.edu/homes/klee/misc/slashdot.html#faq
Carl Banks
> Carl Banks, responding to Roman Suzi:
>> __subclasses__ is a list of weak references, so the class can be
>> collected even though it's listed in __subclasses__. For whatever
>> reason, class A isn't deleted by reference counts, but cyclic garbage
>> collection gets it.
>
> But I want to know that reason. Why are there any cycles? After
> all, don't weak-references help break cycles?
Take a look at "__mro__" method of types. __mro__ is a tuple
containing real references, and the first item is always the type
object, follwed by all it's bases. That makes at least one cycle.
There's probably more since gc collects 6 objects.
> And in my code
> before, why do I need to do the gc a few times to force the cycle
> to be broken?
Because, __subclasses__ is a list of weak references internally, it
gets converted to a regular list when called, and this list is stored
in _. Thus, if you call gc.collect right after evaluating
float.__subclasses__, there remains an active reference.
I think it's a very minor problem. There doesn't appear to be a
memory leak, and what's the point of making the effort to manually
break cycles when cycle garbage collection exists?
--
CARL BANKS
Andrew Dalke
> >>> A.__dict__['__weakref__'].__objclass__ is A
Ahhh...
Andrew
da...@dalkescientific.com
Roman Suzi
>Carl Banks, responding to Roman Suzi:
>> __subclasses__ is a list of weak references, so the class can be
>> collected even though it's listed in __subclasses__. For whatever
>> reason, class A isn't deleted by reference counts, but cyclic garbage
>> collection gets it.
>
>It's a bit too mysterious for my liking.
I wonder why to have __subclasses__() at all... I recall somebody
(Tim Peters?) tolds us that classes are essentially just
dictionaries - everything else is an imagination by OO-brains
and fluffy operations like computing MRO. __subclasses__ aren't
needed for OO to work. What for they really are? Just to have nice way
to know subclasses? Why not to use some external framework for that task,
hooking object manipulation to some routine if necessary?
I can't find any reference to __subclasses__ in standard library!
(I was looking for __subclasses__ string)
Tim Peters
> I wonder why to have __subclasses__() at all... I recall somebody
> (Tim Peters?) tolds us that classes are essentially just
> dictionaries - everything else is an imagination by OO-brains
> and fluffy operations like computing MRO. __subclasses__ aren't
> needed for OO to work. What for they really are? Just to have nice way
> to know subclasses? Why not to use some external framework for that
> task, hooking object manipulation to some routine if necessary?
>
> I can't find any reference to __subclasses__ in standard library!
> (I was looking for __subclasses__ string)
Looking for __subclasses__ won't answer it -- that's just Python exposing an
internal mechanism for the morbidly curious <wink>.
Look for uses of update_subclasses() instead, in typeobject.c. At the C
level, Python doesn't want to do long-winded MRO searches for every
attribute reference for every type implemented in C, so "percolates down"
type object slot definitions from parent classes to their children at the
time child classes are first created. Then the child class just has to look
at its own definitions.
This doesn't work, though, when a base class definition inherited by a child
class changes in the base class later, or when a new definition appears by
magic in the MRO (from the child's POV) before the base class originally
inherited from. Since Python allows changing class characteristics at
runtime, this happens (albeit not in most peoples' code). In such cases,
the freshly-mutated type object's tp_subclasses slot (that's the C spelling
of the Python-level __subclasses__) is used to figure out which derived
classes need to get their type object slots updated too.
In short, __subclasses__ exists to support efficient runtime class mutation
at the C level. I view it much more as an accidental detail of the current
implementation than as a property of the language (Python would still be
Python if tp_subclasses/__subclasses__ didn't exist; CPython would simply be
slower (and possibly much slower) for some kinds of code then -- at heart,
it's to support an MRO-lookup caching/memoization gimmick).
Michael Hudson
> Michael Hudson:
> > >>> A.__dict__['__weakref__'].__objclass__ is A
>
> Ahhh...
And *then* I find how I should have found <wink> the cycle:
>>> class A(object): pass
...
>>> pprint.pprint([r for r in gc.get_referrers(A) if r is not __main__.__dict__])
[<attribute '__dict__' of 'A' objects>,
<attribute '__weakref__' of 'A' objects>,
(<class '__main__.A'>, <type 'object'>)]
Forgot about the obvious A.__mro__[0] is A cycle!
Cheers,
mwh
--
A difference which makes no difference is no difference at all.
-- William James (I think. Reference anyone?)