<quote>
class Foo:
pass
class Bar(object):
pass
class Baz(Foo,Bar):
pass
# Monkey-patch Foo to add a special method
def my_nonzero(self):
print "my_nonzero called"
return False
Foo.__nonzero__ = my_nonzero
b = Baz()
print "doing the test on Baz(Foo,Bar). Should return false"
if b:
print "true"
else:
print "false"
</quote>
Produces this output:
doing the test on Baz(Foo,Bar). Should return false
true
With some experimentation it is clear that this behaviour only occurs
when you combine new+old-style multiple inheritance, monkey-patching
and special methods. If Foo and Bar are either old or new-style it
works. calling b.__nonzero__() directly works. Defining __nonzero__
within Foo works.
I know this level of messing with python internals is a bit risky but
I'm wondering why the above code doesn't work.
Any ideas?
Cheers,
Stephen.
You've already discovered why--you're mixing old and new style
classes.
Monkey patching is definitely unpythonic. You must be a Ruby guy. Why
don't you try doing something else to get the behavior you want,
something more explicit?
I think I can barely explain what happens here (but I may be
absolutely wrong! Please someone with greater knowledge of Python
innards correct whatever is wrong on my description!)
type objects contain "slots" (function pointers) corresponding to the
"magic" methods like __nonzero__ (this one is stored into the
nb_nonzero slot). new-style classes are types; old-style classes are
not (they're all instances of classobj).
The slots are populated when a new type is created (e.g., when
creating a new-style class) and are updated when a magic attribute is
set onto the type. By example, setting the __nonzero__ attribute on a
new-style class updates the nb_nonzero slot. old-style classes just
store the magic attribute in its __dict__. Note that if you patch Foo
*before* defining Baz, it works fine, because Baz sees the magic
attribute and can populate its nb_nonzero slot when the new type is
created.
When you define Baz, neither Foo nor Bar have a __nonzero__ at this
time, so the nb_nonzero slot on Baz is empty.
Later, when you alter Foo, Baz cannot notice it (Foo has no way to
notify Baz that something changed).
If Foo were a new-style class, things are different: new-style classes
maintain a list of subclasses, and the subclasses can then be notified
of changes. In particular, setting a magic attribute on a base class
notifies all its subclasses, and the corresponding slots are updated.
The problem seems to be exactly that: old-style base classes can't
notify its derived new-style classes when magic methods are added, so
the corresponding slots aren't updated. Always asking the base class
whether it has a magic method or not would slow down all method
lookups.
To force a slot update:
py> Baz.__nonzero__ = "xxx"
py> del Baz.__nonzero__
py> bool(b)
my_nonzero called
False
(setting or deleting __nonzero__ triggers the slot update)
This is *why* it happens. How to avoid this... well, don't do that in
the first place :) Or try to patch the base class *before* the new-
style derived class is defined. Or replace the derived class with a
new version of itself (once the base was patched). Or, if you know all
the derived classes, force a slot update on them as above.
--
Gabriel Genellina
New library? Geez, if people are dumb enough to do this, are you sure
you want your application to depend on their library?
Sometimes you have to work with code that's not up to your standards,
but come on.
Carl Banks
Time and place.
A well-considered, timely monkey-patch can sometimes save all kinds of
workarounds and other headaches.
Carl Banks
Indeed, I chuckled at the idea I was a Ruby programmer. Of course
it's a bit of a hack but monkey-patching has been common place in the
Zope world for years (maybe Zope3 has rid itself of the problem).
Agreed, mixing new and old-style classes in multiple inheritance is
asking for trouble (although the Guido's essay on the new-style
classes suggests you can: http://www.python.org/download/releases/2.2.3/descrintro/#subclassing).
I won't name and shame the library involved but unfortunately we can't
always pick and choose the tools we work with unless we want to write
(or rewrite) everything ourselves.
Cheers,
Stephen.
That sounds exactly right and your work-around provides confirmation.
This code is an expedient solution for us so I expect we'll go with
forcing a slot update. Hopefully the library concerned is going to
fix the problem in a future release.
Cheers,
Stephen.
To work with 3.0, they will have to, or rather it will be done for them,
since there will only be new style classes.
+1
Michele Simionato
Doing the armchair code reviewer without context is easy but my guess
would be that the old style library classes were written long before
the new style ones (perhaps even before the latter were introduced)
and/or written independently by different developers/groups, without
any plan to mix the two in the future. Also remember that up to 2.4
even the standard exception classes were old-style so it's not safe to
assume that you only have new style classes to worry about when the
very standard library includes lots of legacy code.
George
It's not the use of old-style classes, or even old-style and new-style
in the same package. It's multiple inheritance combining old- and new-
style classes that's so wrong here. They should have converted the
old-style to new when they decided to derive from both types (it's not
like that's terribly difficult).
Carl Banks
> They should have converted the
> old-style to new when they decided to derive from both types (it's not
> like that's terribly difficult).
Unless perhaps the old-style class is part of an stdlib or 3rd party
(or rather 4th party since "they" are the 3rd party here ;-)) module.
Even if it's technically possible and the change doesn't break other
things, I'd rather not have to maintain a patched version of the
stdlib.
George
Strange enough, I learned monkey-patching with Python, years before I
first heard of a language named Ruby.
> Why
> don't you try doing something else to get the behavior you want,
> something more explicit?
>
From experience, monkey-patching is sometimes the very best solution.
Indeed. I believe the term started at Zope Corp.
Ah, Wikipedia supports my recollection:
http://en.wikipedia.org/wiki/Monkey_patch
(And I swear that it did so before I checked it.)
It looks like some Ruby folks have taken to calling the practice "duck
punching", which has a certain charm of its own.
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco