invoking a method from two superclasses
Mitchell L Model
class A:
def __init__(self):
print('A')
class B:
def __init__(self):
print('B')
class C(A, B):
def __init__(self):
super().__init__()
print('C')
C()
Output is:
A
C
I've discovered the surprising fact described in the documentation of super
<http://docs.python.org/3.1/library/functions.html#super>
that specifying a class as the first argument of super means to skip that class when
scanning the mro so that if C.__init__ includes the line
super(A, self).__init__()
what gets called is B.__init__, so that if I want to call __init__ of both classes the
definition of C should have both of the following lines:
super().__init__()
super(A, self).__init__()
and that
super(B, self).__init__()
does nothing because B is the last class in the mro.
This seems weird. Would someone please give a clear example and explanation of
the recommended way of initializing both superclasses in a simple multiple inheritance
situation?
Note: I am EXTREMELY knowledgeable about OO, Python, and many OOLs.
I don't mean to be arrogant, I just want to focus the discussion not open it to a broad
interchange about multiple inheritance, the ways it can be used or avoided, etc. I just
want to know how to use super. The documentation states the following:
"There are two typical use cases for super. In a class hierarchy with single inheritance,
super can be used to refer to parent classes without naming them explicitly, thus
making the code more maintainable."
"The second use case is to support cooperative multiple inheritance in a dynamic
execution environment. This use case is unique to Python and is not found in
statically compiled languages or languages that only support single inheritance.
This makes it possible to implement "diamond diagrams" where multiple base
classes implement the same method."
"For both use cases, a typical superclass call looks like this:
class C(B):
def method(self, arg):
super().method(arg) # This does the same thing as:
# super(C, self).method(arg)
"
Though it claims to be demonstrating both cases, it is only demonstrating single
inheritance and a particular kind of multiple inheritance where the method is found
in only one class in the mro. This avoids situations where you want to call the
method anywhere it is found in the mro, or at least in the direct superclasses.
Perhaps __init__ is a special case, but I don't see how to figure out how to __init__
two superclasses of a class from the documentation. I often file "bug reports" about
documentation ambiguities, vagueness, incompletenesses, etc., but I don't want to
do so for this case until I've heard something definitive about how it should be
handled.
Thanks in advance.
Scott David Daniels
> In Python 3, how should super() be used to invoke a method defined in C
> that overrides its two superclasses A and B, in particular __init__?
> I've discovered the surprising fact described in the documentation of super
> <http://docs.python.org/3.1/library/functions.html#super>
> that specifying a class as the first argument of super means to skip that class when
>
> This seems weird. Would someone please give a clear example and explanation of
> the recommended way of initializing both superclasses in a simple multiple inheritance
> situation?
OK, in Diamond inheritance in Python (and all multi-inheritance is
diamond-shaped in Python), the common ancestor must have a method
in order to properly use super. The mro is guaranteed to have the
top of the split (C below) before its children in the mro, and the
join point (object or root below) after all of the classes from
which it inherits.
So, the correct way to do what you want:
class A:
def __init__(self):
super().__init__()
print('A')
class B:
def __init__(self):
super().__init__()
print('B')
class C(A, B):
def __init__(self):
super().__init__()
print('C')
C()
And, if you are doing it with a message not available in object:
class root:
def prints(self):
print('root') # or pass if you prefer
class A(root):
def prints(self):
super().prints()
print('A')
class B(root):
def prints(self):
super().prints()
print('B')
class C(A, B):
def prints(self):
super().prints()
print('C')
C().prints()
--Scott David Daniels
Scott....@Acm.Org
Mitchell L Model
methods can be called explicitly without resorting to super(), e.g.:
class C(A, B):
def __init__(self):
A.__init__(self)
B.__init__(self)
My question is really whether there is any way of getting around the
explicit class names by using super() and if not, shouldn't the documentation
of super point out that if more than one class on the mro defines a method
only the first will get called? What's strange is that it specifically mentions
diamond patterns, which is an important case to get right, but it doesn't show
how.
I suspect we should have a Multiple Inheritance HOWTO, though details and
recommendations would be controversial. I've accumulated lots of abstract
examples along the lines of my question, using multiple inheritance both to
create combination classes (the kinds that are probably best done with
composition instead of inheritance) and mixins. I like mixins, and I like abstract
classes. And yes I understand the horrors of working with a large component
library that uses mixins heavily, because I've experienced many of them, going
all the way back to Lisp-Machine Lisp's window system with very many combo
classes such as FancyFontScrollingTitledMinimizableWindow, or whatever.
Also, I understand that properties might be better instead of multiple inheritance
for some situations. What I'm trying to do is puzzle out what the reasonable uses
of multiple inheritance are in Python 3 and how classes and methods that follow
them should be written.
Carl Banks
> Allow me to add to my previous question that certainly the superclass
> methods can be called explicitly without resorting to super(), e.g.:
>
> class C(A, B):
> def __init__(self):
> A.__init__(self)
> B.__init__(self)
>
> My question is really whether there is any way of getting around the
> explicit class names by using super()
Yes there is: just make sure that all subclasses also call super.
class A:
def __init__(self):
super().__init__()
print('A')
class B:
def __init__(self):
super().__init__()
print('B')
class C(A, B):
def __init__(self):
super().__init__()
print('C')
Bam, that's it.
What's happening is that A's super calls B. That is likely to seem
wrong to someone who is very familiar with OOP, but it's how Python's
MI works.
Read this essay/rant that explains how super works and why the author
thinks it's not useful. Then ignore the last part, because I and many
others have found it very useful, desipte its drawbacks.
http://fuhm.net/super-harmful/
Carl Banks
Carl Banks
> On Jun 30, 5:34 pm, Mitchell L Model <MLMLi...@Comcast.net> wrote:
>
> > Allow me to add to my previous question that certainly the superclass
> > methods can be called explicitly without resorting to super(), e.g.:
>
> > class C(A, B):
> > def __init__(self):
> > A.__init__(self)
> > B.__init__(self)
>
> > My question is really whether there is any way of getting around the
> > explicit class names by using super()
>
> Yes there is: just make sure that all subclasses also call super.
And by subclasses I mean base classes, of course. ugh
Carl Banks
Mitchell L Model
>Date: Tue, 30 Jun 2009 16:49:18 -0700
>Message-ID: <ieudnfm_J89EP9fX...@pdx.net>
>Subject: Re: invoking a method from two superclasses
>
>Mitchell L Model wrote:
>>In Python 3, how should super() be used to invoke a method defined in C
> > that overrides its two superclasses A and B, in particular __init__?
>>...
>>I've discovered the surprising fact described in the documentation of super
>><http://docs.python.org/3.1/library/functions.html#super>
>>that specifying a class as the first argument of super means to skip that class when
>>scanning the mro so that ....
>>
>>This seems weird. Would someone please give a clear example and explanation of
>>the recommended way of initializing both superclasses in a simple multiple inheritance
>>situation?
>
>OK, in Diamond inheritance in Python (and all multi-inheritance is
>diamond-shaped in Python), the common ancestor must have a method
>in order to properly use super. The mro is guaranteed to have the
>top of the split (C below) before its children in the mro, and the
>join point (object or root below) after all of the classes from
>which it inherits.
>
>So, the correct way to do what you want:
> def __init__(self):
> super().__init__()
> print('A')
>
> class B:
> def __init__(self):
> super().__init__()
> print('B')
>
> def __init__(self):
> print('C')
>
> C()
>
>And, if you are doing it with a message not available in object:
>
> class root:
> def prints(self):
> print('root') # or pass if you prefer
>
> class A(root):
> def prints(self):
> super().prints()
> print('A')
>
> class B(root):
> def prints(self):
> super().prints()
>
> class C(A, B):
> super().prints()
> print('C')
>
> C().prints()
>
>--Scott David Daniels
>Scott....@Acm.Org
>
Great explanation, and 1/2 a "duh" to me. Thanks.
What I was missing is that each path up to and including the top of the diamond
must include a definition of the method, along with super() calls to move the method
calling on its way up. Is this what the documentation means by
"cooperative multiple inheritance"?
If your correction of my example, if you remove super().__init__ from B.__init__
the results aren't affected, because object.__init__ doesn't do anything and
B comes after A in C's mro. However, if you remove super().__init__ from
A.__init__, it stops the "supering" process dead in its tracks.
It would appear that "super()" really means something like CLOS's call-next-method.
I've seen various discussions in people's blogs to the effect that super() doesn't really
mean superclass, and I'm beginning to develop sympathy with that view. I realize that
implementationally super is a complicated proxy; understanding the practical
implications isn't so easy. While I've seen all sorts of arguments and discussions,
including the relevant PEP(s), I don't think I've ever seen anyone lay out an example
such as we are discussing with the recommendation that basically if you are using
super() in multiple inheritance situations, make sure that the methods of all the classes
in the mro up to at least the top of a diamond all call super() so it can continue to
move the method calls along the mro. The documentation of super(), for instance,
recommends that all the methods in the diamond should have the same signature, but
and it says that super() can be used to implement the diamond, but it never actually
comes out and says that each method below the top must call super() at the risk
of the chain of calls being broken. I do wonder whether this should go in the doc
of super, the tutorial, or a HOWTO -- it just seems to important and subtle to leave
for people to discover.
Again, many thanks for the quick and clear response.
Scott David Daniels
> ... Again, many thanks for the quick and clear response.
Since you know exactly what is confusing right now, and
what the resolution is, could I lean on you to take a stab
at improving the explanation? I think I know what the
solution is, but I don't have a good handle on what needs
explaining and what is easily understood. If you do something,
others can fix it where it doesn't work.
--Scott David Daniels
Scott....@Acm.Org
Gabriel Genellina
<MLML...@comcast.net> escribi�:
> Allow me to add to my previous question that certainly the superclass
> methods can be called explicitly without resorting to super(), e.g.:
>
> class C(A, B):
> def __init__(self):
> A.__init__(self)
> B.__init__(self)
>
> My question is really whether there is any way of getting around the
> explicit class names by using super() and if not, shouldn't the
> documentation
> of super point out that if more than one class on the mro defines a
> method
> only the first will get called?
super returns [a proxy to] the *next* class in the MRO chain; it may or
may not be a superclass of C (it may be a sibling class instead). If you
know that only A B and C are involved and no other subclass ever exists,
yes, you can explicitely invoke A.__init__ and B.__init__. If not (and
this covers the vast majority of cases) using super in all places is the
only way to get correct results (for details, see the "Harmful" article by
James Knight [1] that someone has already referenced)
Note that doing it right with __init__ is tricky: usually, cooperative
methods share the same signature so the super() call just passes the same
arguments to the next class in the chain. But isn't uncommon for __init__
to have different signatures for different subclasses; the article [1]
contains a recipe for this case.
> What's strange is that it specifically mentions diamond patterns, which
> is an important caseto get right, but it doesn't show how.
The "typical superclass call" in the 2.6 docs is, uhm, typical :) - in
principle you use it everywhere you have cooperative classes. For the sake
of completeness I'll copy it here:
class C(A, B):
def method(self, arg):
super(C, self).method(arg)
(you may simply use super().method(arg) in Python 3). Every implementation
of method() in the class hierarchy should contain a super call like above
(well, it gets somewhat more complex due to error handling, but you get
the idea...)
In addition to the "Harmful" document, consider reading the three-article
series by M. Simionato [2] which explains the problem from a different
point of view.
[1] Python's Super Considered Harmful
http://fuhm.net/super-harmful/
[2] http://www.artima.com/weblogs/viewpost.jsp?thread=236275
--
Gabriel Genellina
Carl Banks
wrote:
> En Tue, 30 Jun 2009 21:34:02 -0300, Mitchell L Model
>
> > Allow me to add to my previous question that certainly the superclass
> > methods can be called explicitly without resorting to super(), e.g.:
>
> > class C(A, B):
> > def __init__(self):
> > A.__init__(self)
> > B.__init__(self)
>
> > My question is really whether there is any way of getting around the
> > explicit class names by using super() and if not, shouldn't the
> > documentation
> > of super point out that if more than one class on the mro defines a
> > method
> > only the first will get called?
>
> super returns [a proxy to] the *next* class in the MRO chain; it may or
> may not be a superclass of C (it may be a sibling class instead).
It could be even a niece class.
Carl Banks
Scott David Daniels
Sorry, after looking over some other responses, I went back and re-read
your reply. I'm just making sure here, but:
> Scott David Daniels wrote:
Below compressed for readability in comparison:
>> class A:
>> def __init__(self): super().__init__(); print('A')
>> class B:
>> def __init__(self): super().__init__(); print('B')
>> class C(A, B):
>> def __init__(self): super().__init__(); print('C')
>> C()
>> And, if you are doing it with a message not available in object:
Renamed to disambiguate later discussion
>> class root:
>> def prints(self): print('root') # or pass if you prefer
>> class D(root):
>> def prints(self): super().prints(); print('D')
>> class E(root):
>> def prints(self): super().prints(); print('E')
>> class F(D, E):
>> def prints(self): super().prints(); print('F')
>> F().prints()
> What I was missing is that each path up to and including the top of the diamond
> must include a definition of the method, along with super() calls to move the method
> calling on its way up.
Actually, not really true. In the F through root example, any of the
"prints" methods except that on root may be deleted and the whole thing
works fine. The rootward (closer to object) end must contain the method
in question, possibly only doing a pass as the action, and _not_ calling
super. The other methods (those in D, E, and F above are all optional
(you can freely comment out those methods where you like), but each
should call super() in their bodies.
Note that you can also add a class:
class G(E, D):
def prints(self): super().prints(); print('G')
G().prints()
Also note that the inheritances above can be "eventually inherits from"
as well as direct inheritance.
> Is this what the documentation means by "cooperative multiple inheritance"?
Yes, the phrase is meant to convey "No magic (other than super itelf) is
involved in causing the other methods to be invoked. If you want "all"
prints methods called, make sure all but the last method do super calls.
Of course, any method that doesn't do the super call will be the last by
definition (no flow control then), but by putting the non-forwarding
version at or below the lower point of the diamond, the mro order
guarantees that you will have a "good" place to stop.
Think of the mro order as a solution to the partial order constraints
that a class must appear before any of its direct superclasses, and (by
implication) after any of its subclasses.
> If your correction of my example, if you remove super().__init__ from B.__init__
> the results aren't affected, because object.__init__ doesn't do anything and
> B comes after A in C's mro. However, if you remove super().__init__ from
> A.__init__, it stops the "supering" process dead in its tracks.
Removing the super from B.__init__ means that you don't execute
object.__init__. It turns out that object does nothing in its __init__,
but without knowing that, removing the super from B.__init__ is also a
mistake.
So, you may well already have it all right, but as long as I'm putting
in the effort to get the "operational rules about using super" out, I
thought I'd fill in this last little bit.
--Scott David Daniels
Scott....@Acm.Org
Michele Simionato
> I suspect we should have a Multiple Inheritance HOWTO, though details and
> recommendations would be controversial. I've accumulated lots of abstract
> examples along the lines of my question, using multiple inheritance both to
> create combination classes (the kinds that are probably best done with
> composition instead of inheritance) and mixins. I like mixins, and I like abstract
> classes. And yes I understand the horrors of working with a large component
> library that uses mixins heavily, because I've experienced many of them, going
> all the way back to Lisp-Machine Lisp's window system with very many combo
> classes such as FancyFontScrollingTitledMinimizableWindow, or whatever.
> Also, I understand that properties might be better instead of multiple inheritance
> for some situations. What I'm trying to do is puzzle out what the reasonable uses
> of multiple inheritance are in Python 3 and how classes and methods that follow
> them should be written.
You may be interest in my "Things to know about super":
http://www.artima.com/weblogs/viewpost.jsp?thread=236275
http://www.artima.com/weblogs/viewpost.jsp?thread=236278
http://www.artima.com/weblogs/viewpost.jsp?thread=237121
Mitchell L Model
The documentation of super points out that good design of diamond patterns require the methods to have the same signature throughout the diamond. That's fine for non-mixin classes where the diamond captures different ways of handling the same data. The classical example is
BufferedStram
/ \
/ \
/ \
BufInputStrm BufOutputStrm both have buffers, but use them differentlyu
\ /
\ /
\ /
RandomAccessStream or something like that
The idea of the diamond is to have just one buffer, rather than the two buffers that would result in C++ without making the base classes virtual. All four classes could define __init__ with the argument
filename, or whatever, and everything works fine.
The problems start with the use of mixins. In essence, mixins intentionally do NOT want to be part of diamond patterns. They are orthogonal to the "true" or "main" class hierarchy and just poke their heads in hear and there in that hierarchy. Moreover, a class could inherit from multiple mixins. Typical simple orthogonal mixins would be NamedObject, TrackedObject, LoggedObject, ColoredWidget, and other such
names compounded from an adjective, participle, or gerund and a completely meaningless name such as Object or Thing and which classes typically manage one action or piece of state to factor it out from the many other classes that need it where the pattern of which classes need them does not follow the regular class hierarchy. Suppose I have a class User that includes NamedObject, TrackedObject, and LoggedObject as base classes. (By Tracked I mean instances are automatically registered in a list or dictionary for use by class methods that search, count, or do other operations on them.)
The problem is that each mixin's __init__ is likely to have a different signature. User.__init__
would have arguments (self, name, log), but it would need to call each mixin class's __init__ with different arguments. Mixins are different than what the document refers to as cooperative multiple inheritance -- does that make them uncooperative multiple inheritance classes :-)? I think I'd be back to having to call each mixin class's __init__ explicitly:
class User(NamedObject, TrackedObject, LoggedObject)
def __init__(self, name, log):
NamedObject.__init__(self, name)
TrackedObject.__init__(self)
LoggedObject.__init__(self, log)
This is not terrible. It seems somehow appropriate that because mixin use is "orthogonal" to the "real" inheritance hierarchy, they shouldn't have the right to use super() and the full mro (after all, who knows where the mro will take these calls anyway). And in many cases, two mixins will join into another (a NamedTrackedObject) for convenience, and then classes inheriting from that have one less init to worry about.
I suspect this problem is largely with __init__. All the other __special__ fns have defined argument lists and so can be used with diamond patterns, mixins, etc.
Does this all sound accurate? (Again, not looking for design advice, just trying to ferret out the subtleties and implications of multiple inheritance and super().)
Carl Banks
> [Continuing the discussion about super() and __init__]
>
> The documentation of super points out that good design of
> diamond patterns require the methods to have the same
> signature throughout the diamond. That's fine for non-mixin
> classes where the diamond captures different ways of handling
> the same data.
[snip]
> The problem is that each mixin's __init__ is likely to have a
> different signature.
[snip]
Here are my recommendations for mixin classes.
1. Try to write mixins so that they don't need initializer arguments.
There's several approaches to this. I prefer to burden the subclass
to set any attributes the mixin needs before calling super().__init__
().
class A(SomeMixin,SomeBaseClass):
def __init__(self,arg1,arg2):
self.some_attribute_needed_by_mixin = 10
super().__init__(arg1,arg2)
That might seem a little weird to some (possibly because it is
impossible in C++ or Java to do that), but I find it to be a far more
versatile way for a subclass to send data to its parent. I use it for
regular single inheritance, even.
Another possibility is to have a separate mixin initializer that is
called separately.
2. The __init__ of a mixin, if it has one, should always take *args
and **kwargs arguments, and only those arguments, and pass them as-is
to the superclass.
class SomeMixin:
def __init__(self,*args,**kwargs):
super().__init__(*args,**kwargs)
self.use_somehow(self.some_attribute_needed_by_mixin)
If your mixin must have its own initializer argument, make it a
keyword-only argument, and have __init__ pop it off kwargs.
3. List mixin bases first among base classes.
I don't think Python provides an ideal way to do MI, BTW. I believe
that's by design: MI is somewhat troublesome specifically to
discourage overuse. As a power MI user, I am fine with this.
> I think I'd be back to having to call each mixin class's
> __init__ explicitly:
I'd say never mix super() and direct calls, for any reason. Pick one
and go with it.
Carl Banks
Lie Ryan
super() is short for two things:
1) superharmful
2) superuseful