Builtn super() function. How to use it with multiple inheritance? And why should I use it at all?
Lacrima
I have two super classes:
class SuperClass1(object):
def __init__(self, word):
print word
class SuperClass2(object):
def __init__(self, word, word2):
print word, word2
Also I have subclass of these classes:
class SubClass(SuperClass1, SuperClass2):
def __init__(self):
pass
I have two questions.
1) Inside __init__ of SubClass how can I firstly call __init__ of
SuperClass1, and then __init__ of SuperClass2, using builtin super()
function.
2) Why should I use super() at all, if it is very easy to call methods
of super class like this:
class SubClass(SuperClass1, SuperClass2):
def __init__(self):
SuperClass1.__init__(self, 'Python')
SuperClass2.__init__(self, 'Hello', 'world')
Thanks in advance.
Raymond Hettinger
> Hi!
>
> I have two super classes:
>
> class SuperClass1(object):
> def __init__(self, word):
> print word
>
> class SuperClass2(object):
> def __init__(self, word, word2):
> print word, word2
>
> Also I have subclass of these classes:
>
> class SubClass(SuperClass1, SuperClass2):
> def __init__(self):
> pass
>
> I have two questions.
> 1) Inside __init__ of SubClass how can I firstly call __init__ of
> SuperClass1, and then __init__ of SuperClass2, using builtin super()
> function.
I would write it like this:
class SuperClass1(object):
def __init__(self, **kwds):
word = kwds.pop('word')
print word
super(SuperClass1, self).__init__(**kwds)
class SuperClass2(object):
def __init__(self, **kwds):
word1 = kwds.pop('word1')
word2 = kwds.pop('word2')
print word1, word2
super(SuperClass2, self).__init__(**kwds)
class SubClass(SuperClass1, SuperClass2):
def __init__(self, **kwds):
super(SubClass, self).__init__(**kwds)
SubClass(word='Python', word1='Hello', word2='World')
> 2) Why should I use super() at all, if it is very easy to call methods
> of super class like this:
> class SubClass(SuperClass1, SuperClass2):
> def __init__(self):
> SuperClass1.__init__(self, 'Python')
> SuperClass2.__init__(self, 'Hello', 'world')
That works just fine in this case.
The challenge arises in "diamond diagrams"
such as A->B A->C B->D C->D where both B and C
are written independently of D and both need to call
A's __init__ but that method should only be called
once (not once by B and again by C).
In that case, the super() pattern shown above will
let each parent's method be called exactly once
and guarantee that parents are called before grandparents
and guarantee that the left-to-right ordering of multiple
bases is respected.
Raymond
Lacrima
wrote:
Hi, Raymond!
Thank you for your answer.
Some things are still not clear. Your example works great. But if I
remove "super(SuperClass1, self).__init__(**kwds)" from SuperClass1's
__init__, the example stops working. That is when I instantiate
SubClass only __init__ of SuperClass1 is called and __init__ of
SuperClass2 is omitted, i.e. only 'Python' is printed. Why is it so?
So as I understand every parent should necessarily call super() at the
end of its __init__ method in order for things to work properly.
But what if SuperClass1 is from third party library? Then I can't
modify it to follow this convention, that is when I instantiate my
SubClass only __init__ from SuperClass1 will be called, and __init__
from SuperClass2 will be omitted.
How to deal with that?
My post is quite intricate, but this is because of my English. Sorry.
Looking forward for help. Thank you.
Gregory Ewing
> But what if SuperClass1 is from third party library?
If it hasn't been designed for super(), then you
can't use super() with it.
super() only works when *every* class in the
hierarchy has been designed with it in mind.
--
Greg
Steven D'Aprano
That incorrect. You can certainly use super() with classic classes in the
hierarchy, and super() didn't even exist when they were created.
>>> class Parent:
... def method(self, arg):
... return repr(arg)
...
>>> class Classic(Parent):
... def method(self, arg):
... return "argument was %s" % Parent.method(self, arg)
...
>>> class New(object, Classic):
... def method(self, arg):
... return super(New, self).method(arg).upper()
...
>>> x = New()
>>> x.method('spam')
"ARGUMENT WAS 'SPAM'"
The problem isn't super(), and people who give glib advise "don't use
super()" are just compounding the problem. The problem is with multiple
inheritance where methods have different method signatures. If you don't
change the method signatures, super() will work fine.
Advising people not to use super() might make people feel virtuous, but
it doesn't do anything to help the reader write non-buggy MI hierarchies.
It pushes the effort of dealing with multiple inheritance onto them,
forcing them to re-implement the MRO, probably badly. Can you re-
implement the C3 algorithm? Have you even heard of it? If you answer No
to either of those questions, chances are high that trying to deal with
the MRO manually will lead to worse bugs than using super().
Should you use super()?
1. If you're doing multiple inheritance with metaclasses, you MUST use
super().
2. If you're using single inheritance only, and never modify method
signatures, there is no reason not to use super().
3. If you're using mixins, you should use super().
4. If you never modify method signatures, then you should use super().
5. If you do modify method signatures, you shouldn't do that (except
possibly in constructors, and even there only cautiously). But if you do
it anyway, then you should use super() *except* in the methods where you
modify the signature.
6. If you don't use super(), chances are that your class hierarchy is
still buggy, but instead of failing loudly and noisily with an exception,
it's silently giving the wrong results.
7. If you can avoid multiple inheritance in favour of another technique
(such as composition), you should strongly consider that.
--
Steven
Raymond Hettinger
> Thank you for your answer.
You're welcome.
> Some things are still not clear. Your example works great. But if I
> remove "super(SuperClass1, self).__init__(**kwds)" from SuperClass1's
> __init__, the example stops working. That is when I instantiate
> SubClass only __init__ of SuperClass1 is called and __init__ of
> SuperClass2 is omitted, i.e. only 'Python' is printed. Why is it so?
>
> So as I understand every parent should necessarily call super() at the
> end of its __init__ method in order for things to work properly.
Yes. That's correct. Python's super() was designed to support
cooperative multiple inheritance. The "cooperative" part means that
every class implementing the target method (such as __init__ in your
example) needs to call super() in order to trigger the next method in
the sequence (the method resolution order or MRO).
> But what if SuperClass1 is from third party library?
. . .
> How to deal with that?
Then, the odds are that that class isn't "cooperating". You can
either wrap the third-party library to add a super() call or you can
switch to an alternate design using composition instead of
inheritance.
Raymond
P.S. Outside of the simple case of single inheritance, the one key to
understanding super() is to forget about the concept of parent
classes. Instead, super() is all about the MRO which is computed
dynamically (unknowable at the time a class is written). Every class
in the MRO implementing the target method *must* call super() to give
the next class in the MRO a chance to run.
IOW, using super() means "I'm in the MRO and I got a chance to run;
now the next class in the MRO gets its chance." Since the MRO is
only knowable at runtime, the sole job of super() is to figure out
which is "the next class in the MRO".
Gregory Ewing
> Every class
> in the MRO implementing the target method *must* call super() to give
> the next class in the MRO a chance to run.
EXCEPT for the last one, which must NOT call super!
The posted example happens to work because object has
a default __init__ method that does nothing. But this
is not generally true of other methods, which means you
need a "terminating" class at the end of the MRO whose
methods don't call super.
--
Greg
Michele Simionato
http://micheles.googlecode.com/hg/artima/python/super.pdf
M.S.
Ethan Furman
Speaking of new-style classes only, don't they all end in object? And
if the MRO is only known at run-time, how is one to know at code-time
whether your (new-style) class is at the end of the line?
~Ethan~
Raymond Hettinger
> Speaking of new-style classes only, don't they all end in object? And
> if the MRO is only known at run-time, how is one to know at code-time
> whether your (new-style) class is at the end of the line?
That is a bit of a PITA. One way of handling it is to design your
diamond so that only one class inherits from object and that class
doesn't use super(). Or you can wrap the super call in a try/except
AttributeError.
Cooperative multiple inheritance isn't pretty -- which is just another
good reason to use composition rather that inheritance.
Raymond
Jean-Michel Pichavant
dangerous guess game. It makes implicit what would *really* need to be
explicit.
class Base1(object):
def foo(self):
print 'Base1'
class Base2(object):
def foo(self):
print 'Base1'
class Sub(Base1, Base2):
def foo(self):
# which base version to call ???
# choice A
# use super an pray that it will do what I need (better
know exactly how the MRO works)
# also pray that further readers know as much as I do
about super
# choice B
# use explicit calls so I can choose which algorithm I
want to use, calling Base1, Base2 or both of them
# If the choice is too difficult, that means one thing
=> my inheritance design is crap => rewrite it properly.
Super is known for being required for diamond inheritance, and this
reputation is well earned. Outside this scope, super's not required.
JM
Steven D'Aprano
> Steven D'Aprano wrote:
>> That incorrect. You can certainly use super() with classic classes in
>> the hierarchy, and super() didn't even exist when they were created.
>> The problem isn't super(), and people who give glib advise "don't use
>> super()" are just compounding the problem. The problem is with multiple
>> inheritance where methods have different method signatures. If you
>> don't change the method signatures, super() will work fine.
[...]
> I think you're missing the point that super for most of us is a
> dangerous guess game. It makes implicit what would *really* need to be
> explicit.
No, I'm not missing the point. I'm *disagreeing* with the point. I'm
saying that for simple inheritance hierarchies, super() does no harm, and
it is *required* for most complex inheritance hierarchies. Given that,
why not use it?
I suppose that, if you're inheriting from an existing class which doesn't
use super(), there may be issues (although I'm not convinced that those
issues go away if you avoid super!) but for a new class under your
control, there are no negatives to super() that didn't already exist with
the nature of multiple inheritance.
There is, as far as I can tell, ONE use-case where super() must be
avoided, and that is a use-case that is discouraged for other reasons:
where you change the method signature. If you're changing the method
signature, you're already in dubious territory and better know what
you're doing. You're violating the Liskov Substitution Principle, and if
so, you better have a good reason. (I'm a bit skeptical about the LSP,
but many people much smarter than I think it's virtually gospel, so who
knows?) But either way, if you change the method signature, you're going
to have problems somewhere unless you've really careful. The problems
with super() are merely symptoms of that change.
If you don't change the method signature, then I don't believe super()
does any harm, regardless of whether you have single inheritance or
multiple inheritance, whether it is a diamond diagram or not. The worst
that can be said about it is that super()'s signature is hard to
understand.
I would argue that your argument about "explicit" and "implicit" is
incorrect. You shouldn't want to explicitly specify which class you are
inheriting from, at least not under normal circumstances. It is enough to
say that you are inheriting behaviour. If you have a Widget class, and
you find yourself explicitly calling methods by *named* superclasses,
you're probably doing something wrong.
It should always be possible to insert a class higher up the hierarchy,
or rename a grandparent, without effecting your class. (Obviously you
can't expect to rename an immediate parent class.)
super() is just as explicit as len(), or str.upper(). It says,
explicitly, that it will call the method belonging to one or more
superclass of the given class. That's no more implicit than mylist[5] is
implicit merely because you didn't have to walk the list by hand.
> class Base1(object):
> def foo(self):
> print 'Base1'
>
>
> class Base2(object):
> def foo(self):
> print 'Base1'
>
>
> class Sub(Base1, Base2):
> def foo(self):
> # which base version to call ???
> # choice A
> # use super an pray that it will do what I need (better
> know exactly how the MRO works)
But the point is, super() knows exactly how the MRO works, so you don't
have to.
> # also pray that further readers know as much as I do
> about super
Now that's just silly. Do you write all your classes for the lowest
common denominator? Do you assume all you users are ignorant? If you
subclass dict, do you feel the need to "pray" that your users won't try
to use mutable objects as keys?
If you avoid super(), do you "pray" that your users won't try to use your
class in a multiple inheritance situation, and have a buggy class? I bet
you don't. If they misuse your class, that's their responsibility.
> # choice B
> # use explicit calls so I can choose which algorithm I
> want to use, calling Base1, Base2 or both of them
That's a fair point.
super() uses the standard MRO algorithm, it isn't a "Do What I Mean" mind-
reading function. If you want an unusual MRO, then you're responsible for
managing it yourself. Go right ahead, this is Python and you have all the
tools to do so. Nothing forces you to inherit behaviour from the
superclasses at all -- super() is designed for overloading (extending)
the behaviour of methods, and so the assumption is that you want to call
the method in each superclass. But if you're overriding the method
instead, or doing something unusual, then you're in charge.
> # If the choice is too difficult, that means one thing
> => my inheritance design is crap => rewrite it properly.
>
> Super is known for being required for diamond inheritance, and this
> reputation is well earned. Outside this scope, super's not required.
Of course it's not required. Nor does it do any harm. Far from it -- your
single inheritance subclass might be just what another user needs for a
multiple inheritance class, and by not using super(), you ruin it for him.
--
Steven
Jean-Michel Pichavant
> [snip]
>
> super() is just as explicit as len(), or str.upper(). It says,
> explicitly, that it will call the method belonging to one or more
> superclass of the given class.
>
Everybody can accurately guess what len and upper are doing without
looking at the documentation. No one can guess for super without closely
looking at the documention, or even at some good articles on the net
which try to clear things up about super. And note there is no such
article about len or upper.
As someone already said in this list, the main problem with super is
that it tends to refer to the superclass method while in fact it calls
the next MRO method.
"mro" would have been the proper name for "super".
JM
Steven D'Aprano
> Steven D'Aprano wrote:
>> [snip]
>>
>> super() is just as explicit as len(), or str.upper(). It says,
>> explicitly, that it will call the method belonging to one or more
>> superclass of the given class.
>>
> Come on Steven, you're better than this :) . Everybody can accurately
> guess what len and upper are doing without looking at the documentation.
> No one can guess for super without closely looking at the documention,
> or even at some good articles on the net which try to clear things up
> about super. And note there is no such article about len or upper.
super() is complex because the problem it is solving is a hard problem.
That doesn't make it implicit, any more than (say) itertools.groupby() is
implicit just because it's complex, or urllib2.request() is implicit just
because some people don't know much about web protocols and have to read
articles about it to learn.
> As someone already said in this list, the main problem with super is
> that it tends to refer to the superclass method while in fact it calls
> the next MRO method.
Why do you think that is a problem? That's what it is supposed to do,
because that's what is needed to correctly implement multiple inheritance.
> "mro" would have been the proper name for "super".
That's your opinion. In any case, whether super() was called super() or
mro() or aardvark() makes no difference to the functionality or whether
it is useful.
--
Steven
Gregory Ewing
> super() is just as explicit as len(), or str.upper(). It says,
> explicitly, that it will call the method belonging to one or more
> superclass of the given class.
That's not strictly true. It will call a method belonging to some
class in the mro of self, but that class is not necessarily in
the base list of the class mentioned in the super() call. It's
possible for a super() call to go "sideways" in the inheritance
graph.
--
Greg
Gregory Ewing
I think the point is that the name is misleading, because it
makes it *sound* like it's going to call a method in a superclass,
when it fact it might not.
--
Greg
Jean-Michel Pichavant
> On Thu, 29 Jul 2010 19:29:24 +0200, Jean-Michel Pichavant wrote:
>
>
>> As someone already said in this list, the main problem with super is
>> that it tends to refer to the superclass method while in fact it calls
>> the next MRO method.
>>
>
> Why do you think that is a problem? That's what it is supposed to do,
> because that's what is needed to correctly implement multiple inheritance.
>
>
>
>> "mro" would have been the proper name for "super".
>>
>
> That's your opinion. In any case, whether super() was called super() or
> mro() or aardvark() makes no difference to the functionality or whether
> it is useful.
>
>
>
>
called dogs and dogs called cats as I'm dealing with industrial
programming, not litterature nor poetry.
JM
Steven D'Aprano
> Steven D'Aprano wrote:
>
>> super() is just as explicit as len(), or str.upper(). It says,
>> explicitly, that it will call the method belonging to one or more
>> superclass of the given class.
>
> That's not strictly true. It will call a method belonging to some class
> in the mro of self, but that class is not necessarily in the base list
> of the class mentioned in the super() call.
Yes, that's what I said. super() can visit any superclass of the given
class, not just one of the immediate base class(es). That's why it's
called super() rather than base() or parent(). It would be rather
pointless if super() was limited to just the base classes.
> It's possible for a super()
> call to go "sideways" in the inheritance graph.
I doubt that very much. A class F can't inherit behaviour from a class E
merely by virtue of them both being subclasses of the same hierarchy. If
it did, that would be... disturbing.
Example:
E inherits from D.
D inherits from C and B.
C and B both inherit from A.
F also inherits from C.
F and E are "sideways" to each other (sibling classes?), but they don't
inherit from each other.
Perhaps you're referring to the angled lines in a diagram such as:
A
/ \
C B
\ /
D
/ \
E F
F and E don't inherit from each other, because they are sidewards to each
other (they are not in each other's MRO). Regardless of the angles we
draw the lines, all of D, C, B, A are above E (and F). Or downwards if
you prefer to reverse the diagram. Yes, a super call might jog left from
C to B, but only when being called from one of the lower classes D-F.
That's still an upwards call relative to the originator, not sidewards.
--
Steven
Steven D'Aprano
I'm not sure I understand your point here. If you call super() from a
method that doesn't exist in any superclass, then you are correct, it
won't call a method in a superclass, and will raise AttributeError.
But in the more sensible case that you only call super() when there is
actually something to inherit, then of course it calls the method in a
superclass. It certainly doesn't call methods from arbitrary unrelated
classes, only those which are in the MRO. That is, superclasses.
If Z is below A in the hierarchy, then we have no difficulty in
identifying Z as a subclass of A, and likewise we should have no problem
with identifying A as *a* (and not "the") superclass of Z, no matter how
distant they are or how tangled the DIG between them.
The exception would be if you could have loops in the class hierarchy, in
which case the concepts of super- and sub-classes breaks down completely.
But even if some other languages allowed that, Python doesn't, so we're
safe.
--
Steven
Jean-Michel Pichavant
"Readers familiar will single inheritance languages, such as Java or
Smalltalk, will have a clear concept of superclass in mind. This
concept, however, has /no useful meaning/ in Python or in other multiple
inheritance languages".
the complete article on super:
http://www.artima.com/weblogs/viewpost.jsp?thread=236275
super is a strange name for something that makes no sense in python
(talking about superclass here). It doesn't take away the usefulness of
super, but it's a proof that you can ruin a good feature with a bad name.
JM
Hrvoje Niksic
> I think the point is that the name is misleading, because it makes it
> *sound* like it's going to call a method in a superclass, when it fact
> it might not.
That is indeed confusing to some people, especially those who refuse to
to accept the notion that "superclass" means the same as "next in MRO",
maintaining instead that superclass refers to one of the base classes,
their bases, etc. -- IMO a defensible position.
super might have better been called next_in_mro, next_method, or
next_class, except those are harder to type, and way less catchy than
"super". The Dylan and CLOS operator that super is most closely based
on is called (call-)next-method.
Raymond Hettinger
That is an important point and it is what Guido does in his examples:
http://www.python.org/download/releases/2.2.3/descrintro/#cooperation
The design options are:
* if overriding a method provided by object() such as __init__(),
__getattribute__() or __setattr__(), then you should call super() in
each class that overrides or extends those methods.
* if you know the whole class structure in advance, you call super()
in every class except the last one -- that is what Guido does in the
save() example.
* if you don't know the whole class structure in advance, then you
can't be sure which is that last class in the mro with the target
method, so you need to wrap the super() call in a try / except
AttributeError
Raymond
Gregory Ewing
> A
> / \
> C B
> \ /
> D
> / \
> E F
>
> Yes, a super call might jog left from
> C to B, but only when being called from one of the lower classes D-F.
> That's still an upwards call relative to the originator, not sidewards.
But it's not an upward call relative to the class mentioned
in the super() call, which is why I say it's misleading.
--
Greg
Steven D'Aprano
Which class would that be?
I think I'm going to need an example that demonstrates what you mean,
because I can't make heads or tails of it. Are you suggesting that a call
to super(C, self).method() from within C might call B.method(self)?
--
Steven
Steven D'Aprano
> Quoting Michele's article (I think he's still hanging around this list)
>
> "Readers familiar will single inheritance languages, such as Java or
> Smalltalk, will have a clear concept of superclass in mind. This
> concept, however, has /no useful meaning/ in Python or in other multiple
> inheritance languages".
I have read Michelle Simionato's articles on super in Python. He has
taught me a lot. But on that specific matter, I think he is wrong.
Of course, he is right to say that the concept of *the* superclass is
meaningless in a MI language like Python. If MyClass inherits method
spam() from class A, and inherits method ham() from class B, which is
"the" superclass of MyClass?
But Michelle is wrong to conclude that the problem lies with the concept
of *superclass*. The problem lies with the idea that there is ONE
superclass. By dismissing the entire concept, he is throwing out the baby
with the bathwater.
The useful, and I would argue *correct*, concept of superclass is very
simple. It is a reflection of subclass: if Z is a subclass of A, then A
is a superclass of Z. This follows e.g. superset and subset. We don't
have any problem understanding that a class can have many subclasses. Why
the resistance to the idea that a class can have many superclasses?
Even in a single inheritance language, if we had a class hierarchy
A -> B -> C -> ... -> Y -> Z
it makes perfect sense to describe *all* of A-Y as superclasses of Z,
just as we describe all of B-Z as subclasses of A.
--
Steven
Ian Kelly
I have to chime in and agree that the name "super" is problematic.
I'm reading this thread with a sense of alarm because I apparently
never read the super() documentation too closely (why would I? "Oh,
it just accesses an attribute from a superclass. Moving on.") and
have been writing code for the past four years under the impression
that super() will always refer to a superclass of the current class.
Fortunately, I don't use multiple inheritance often, and when I do I
prefer to write the superclasses explicitly (perhaps because of the
same misconception), so I probably haven't really abused it terribly.
On a tangent, is it just me, or is the super() documentation
incorrect, or at least unclear? Quoting from the first two
paragraphs:
super(type[, object-or-type])
Return a proxy object that delegates method calls to a parent or
sibling class of type. This is useful for accessing inherited methods
that have been overridden in a class. The search order is same as that
used by getattr() except that the type itself is skipped.
The __mro__ attribute of the type lists the method resolution
search order used by both getattr() and super(). The attribute is
dynamic and can change whenever the inheritance hierarchy is updated.
In the first paragraph, "type" refers to "type", the first parameter
in the function signature. In the second paragraph, "type" refers to
the type instance or the type of the object passed as the second
argument. If it also referred to the first parameter, then super()
would always access a superclass as I initially thought; I wonder if
this might have been partially responsible for my confusion.
Cheers,
Ian
Steven D'Aprano
> I have to chime in and agree that the name "super" is problematic. I'm
> reading this thread with a sense of alarm because I apparently never
> read the super() documentation too closely (why would I? "Oh, it just
> accesses an attribute from a superclass. Moving on.") and have been
> writing code for the past four years under the impression that super()
> will always refer to a superclass of the current class.
In Python 2.x, super() doesn't know what the current class is. You have
to explicitly tell it. If you tell it a lie, surprising things will
happen.
Assuming you accurately tell it the current class, can you give an
example where super() doesn't refer to a superclass of the current class?
[...]
> On a tangent, is it just me, or is the super() documentation incorrect,
> or at least unclear? Quoting from the first two paragraphs:
Yes, it's a bit unclear, because it's a complex function for dealing with
a complicated situation. But if you have single inheritance, it's simple.
Anywhere you would write
class C(B):
def method(self, *args):
B.method(*args)
you can write
class C(B):
def method(self, *args):
super(C, self).method(*args)
and it will Just Work.
> super(type[, object-or-type])
>
> Return a proxy object that delegates method calls to a parent or
> sibling class of type.
I think that the bit about sibling class refers to super(type, type2)
calls, rather than the super(type, instance) calls which I've been
discussing. I've never needed, and don't understand, the two type version
of super(), so I can't comment on it.
--
Steven
Brian Victor
Yes, it would.
class A(object):
def test_mro(self):
print "In A"
class B(A):
def test_mro(self):
print "In B"
super(B, self).test_mro()
raise Exception()
class C(A):
def test_mro(self):
print "In C"
super(C, self).test_mro()
class D(C, B):
def test_mro(self):
print "In D"
super(D, self).test_mro()
D().test_mro()
Notice the exception being raised in B. This results in the following traceback:
Traceback (most recent call last):
File "mro.py", line 21, in <module>
D().test_mro()
File "mro.py", line 19, in test_mro
super(D, self).test_mro()
File "mro.py", line 14, in test_mro
super(C, self).test_mro()
File "mro.py", line 9, in test_mro
raise Exception()
Exception
Since the idea of super() as I understand it is to make sure every class
in an object's hierarchy gets its method called, there's really no way
to implement super() in a way that didn't involve a non-superclass being
called by some class's super() call.
--
Brian
Ian Kelly
<st...@remove-this-cybersource.com.au> wrote:
> On Fri, 30 Jul 2010 21:40:21 -0600, Ian Kelly wrote:
>
>> I have to chime in and agree that the name "super" is problematic. I'm
>> reading this thread with a sense of alarm because I apparently never
>> read the super() documentation too closely (why would I? "Oh, it just
>> accesses an attribute from a superclass. Moving on.") and have been
>> writing code for the past four years under the impression that super()
>> will always refer to a superclass of the current class.
>
> In Python 2.x, super() doesn't know what the current class is. You have
> to explicitly tell it. If you tell it a lie, surprising things will
> happen.
>
> Assuming you accurately tell it the current class, can you give an
> example where super() doesn't refer to a superclass of the current class?
Brian gave a good example, so I refer you to that.
>> On a tangent, is it just me, or is the super() documentation incorrect,
>> or at least unclear? Quoting from the first two paragraphs:
>
> Yes, it's a bit unclear, because it's a complex function for dealing with
> a complicated situation. But if you have single inheritance, it's simple.
> Anywhere you would write
>
> class C(B):
> def method(self, *args):
> B.method(*args)
>
> you can write
>
> class C(B):
> def method(self, *args):
> super(C, self).method(*args)
>
>
> and it will Just Work.
Until somebody else comes along and creates class E that inherits from
both C and D and also uses super(), and now suddenly the super() call
in C becomes equivalent to "D.method(*args)" instead for instances of
E, potentially with unexpected results. Or worse, E.method
incorrectly calls both C.method and D.method explicitly, and now
D.method gets invoked twice, once implicitly from C.method, and once
explicitly from E.method.
I realize that it is the responsibility of the person writing class E
to make sure they're working with class C correctly, but in order for
them to do that, ultimately every method in class C should be
documented as to whether it calls super() or not, and if so whether it
is designed only for single inheritance or with the MRO in mind. That
really shouldn't be necessary, and in my current view from a
maintenance perspective, best practice is to only use super() in the
multiple-inheritance scenarios it was specifically designed for
>> super(type[, object-or-type])
>>
>> Return a proxy object that delegates method calls to a parent or
>> sibling class of type.
>
> I think that the bit about sibling class refers to super(type, type2)
> calls, rather than the super(type, instance) calls which I've been
> discussing. I've never needed, and don't understand, the two type version
> of super(), so I can't comment on it.
I understand the two-type version works the same way, but is intended
for class methods, like so:
class A(object):
@classmethod
def f(cls, *args):
print "A.f"
class B(A):
@classmethod
def f(cls, *args):
print "B.f"
return super(B, cls).f(*args)
class C(A):
@classmethod
def f(cls, *args):
print "C.f"
return super(C, cls).f(*args)
class D(B, C):
@classmethod
def f(cls, *args):
print "D.f"
return super(D, cls).f(*args)
Cheers,
Ian
Gregory Ewing
> Assuming you accurately tell it the current class, can you give an
> example where super() doesn't refer to a superclass of the current class?
I think we're all confusing each other in this discussion
by not being clear enough about what we mean by the "current
class".
In a call super(C, self), there are two possible things it
could mean: the class C, or the type of self.
Obviously it will return some class in the mro of self,
but as Brian Victor's example demonstrates, it *doesn't*
necessarily return a class in the mro of C.
--
Greg
Gregory Ewing
> super(type[, object-or-type])
>
> ...
>
> The __mro__ attribute of the type lists the method resolution
> search order used by both getattr() and super(). The attribute is
> dynamic and can change whenever the inheritance hierarchy is updated.
That explanation does seem to be rather confusing. It would
certainly confuse *me* if I didn't already know what super()
did.
+1 on getting this part of the docs re-written to be several
times clearer.
--
Greg
Carl Banks
I was with you up to here.
It's *manifestly* necessary. One cannot inherit from a class reliably
without knowing how it's implemented. Knowing the interface is not
enough. This is not limited to multiple inheritance; it happens with
single inheritance as well, just less often. It's not limited to
Python, this is true of Java, C++, and every other OOP language I know
of.
Take list. Say you want a singly inherit from list to implement some
kind of observer that emits a signal every time an item is set. How
do you do it? You can't tell by looking at the interface. Do you
only have to override __setattr__? Or do you also have to override
other methods (insert, append, etc)? That depends on whether insert
et al. call __setattr__ or operate at a lower level. You can't tell
simply by looking at the interface. You have to have knowledge of the
internals of list.
It's the same problem with multiple inheritance, only exacerbated.
The issues with super is a new dimension, but it's the same problem:
you have to know what you're inheriting from.
This is a fundamental issue with all inheritance.
The problem, in my mind, is that people use inheritance for stuff that
doesn't need it.
class MyFoo(SomebodyElsesFoo):
def my_tacked_on_method(self):
self.call_this_method()
self.then_call_that_method()
Unnecessary and risky. Do this:
my_tacked_on_function(foo):
foo.call_this_method()
foo.then_call_that_method()
The only reason to inherit in a situation like this is if you need
polymorphism (you want to pass your MyFoo to a function that operates
on objects that define a my_tacked_on_method).
A lot Michele Simionato's objections to inheritance are based on the
idea that you don't necessarily know what you're inheriting from. I
say, if you don't know what you're inheriting from, you shouldn't be
inheriting. I say this as a strong proponent of inheritance and
multiple inheritance in particular. I believe it's the best way in
general for objects of different types to share large portions of
their behavior. But it has its place.
When you have a class you that don't anything about the implementation
of, that is NOT the place for inheritance. In that case you should
either use composition, or learn what it is you're inheriting from and
take responsibility if the owner changes it. Or just don't try to
share behavior.
My view of inheritance is a lot like the orthodox Catholic view of
sex. To Catholics, sex between married people is a wonderful, holy
thing. To me, inheriting from classes you know the implementation of
is a wonderful, holy thing. But, sex outside of marriage, and
inheriting from classes you don't know, is deadly sin and you will
burn in Hell.
> and in my current view from a
> maintenance perspective, best practice is to only use super() in the
> multiple-inheritance scenarios it was specifically designed for
I use super() exclusively for all types of inheritance, unless I'm
inheriting from a class I know uses old form (which is none of mine,
and I only rarely inherit from classes I don't own).
Carl Banks
Carl Banks
> When you have a class you that don't anything about the implementation
> of, that is NOT the place for inheritance.
And, just to be clear, if the class is explicity documented as being
subclassable and the documentation states the proper procedure for
doing so, that counts as "knowing about the implementation"--even if
you don't know the exact implementation, you do have sufficient
knowledge.
I just don't want someone following up saying, "Well I think it's ok
to inherit from a class you know nothing about if it's documented
properly." I agree, and that is knowing something about it.
Carl Banks
Steven D'Aprano
> Steven D'Aprano wrote:
>> On Sat, 31 Jul 2010 14:25:39 +1200, Gregory Ewing wrote:
>>
>>> Steven D'Aprano wrote:
>>>
>>>> A
>>>> / \
>>>> C B
>>>> \ /
>>>> D
>>>> / \
>>>> E F
>>>>
>>>> Yes, a super call might jog left from C to B, but only when being
>>>> called from one of the lower classes D-F. That's still an upwards
>>>> call relative to the originator, not sidewards.
>>>
>>> But it's not an upward call relative to the class mentioned in the
>>> super() call, which is why I say it's misleading.
>>
>> Which class would that be?
>>
>> I think I'm going to need an example that demonstrates what you mean,
>> because I can't make heads or tails of it. Are you suggesting that a
>> call to super(C, self).method() from within C might call
>> B.method(self)?
>
> Yes, it would.
[snip example]
Right, now I see what you mean. I don't have a problem with that
behaviour, it is the correct behaviour, and you are making the call from
D in the first place, so it *must* call B at some point.
If you initiate the call from C instead:
>>> C().test_mro()
In C
In A
then B is not in C's MRO, and does not get called. This is the case I was
referring to.
I admit it did surprise me the first time I saw the example you gave.
Hell, it *confused* me at first, until I realised that self inside the
method isn't necessarily an instance of C, but could be an instance of a
subclass of C (in this case, D), and then it was "Well duh, how obvious!"
That's no different from a classic-style call to a superclass:
# inside class C(B):
B.method(self, *args)
Inside B, method() gets called with an instance of C as self.
Bringing it back to super(), since super(C, self) gets a D instance as
argument, the MRO it looks at is D's MRO, and it all just works. I think
the docs could make that a little more clear, but then, given how non-
cooperative inheritance works exactly the same way, there's a good
argument for saying the docs don't need to be changed at all. If the
behaviour is confusing for super(), then it's confusing without super()
too.
See Guido's tutorial on cooperative multitasking for more detail:
http://www.python.org/download/releases/2.2.3/descrintro/#cooperation
> Since the idea of super() as I understand it is to make sure every class
> in an object's hierarchy gets its method called, there's really no way
> to implement super() in a way that didn't involve a non-superclass being
> called by some class's super() call.
You're reading the super() signature wrong.
super(C, self)
shouldn't be interpreted as "call C's superclasses from self". It means
"starting just after C in the MRO, call self.__class__'s superclasses".
super() never calls a non-superclass. If it did, it would be a huge bug,
and inheritance would break.
--
Steven
Gregory Ewing
> super(C, self)
>
> shouldn't be interpreted as "call C's superclasses from self". It means
> "starting just after C in the MRO, call self.__class__'s superclasses".
My contention is that nobody has any chance of guessing what
it does based on the name "super". Superness doesn't come into
it at all, except in the trivial sense that whatever class it
picks will be in the mro of the second argument, which doesn't
narrow it down much.
So the word "super" here is little better than meaningless noise.
If it were something like next_method(C, self) then one could
claim with some justification that it stands for "call the next
method found after C in the mro of self".
--
Greg
Michele Simionato
One "l" please! I am a man! ;-)
>
> But Michelle is wrong to conclude that the problem lies with the concept
> of *superclass*. The problem lies with the idea that there is ONE
> superclass. By dismissing the entire concept, he is throwing out the baby
> with the bathwater.
I am actually more radical than that. From
http://www.artima.com/weblogs/viewpost.jsp?thread=237121:
"""
In this series I have argued that super is tricky; I think nobody can
dispute that. However the existence of dark corners is not a
compelling argument against a language construct: after all, they are
rare and there is an easy solution to their obscurity, i.e.
documenting them. This is what I have being doing all along. On the
other hand, one may wonder if all super warts aren't hints of some
serious problem underlying. It may well be that the problem is not
with super, nor with cooperative methods: the problem may be with
multiple inheritance itself.
"""
Then I spend thousands of words in the "Mixin considered harmful"
series and in other articles arguing against
multiple inheritance and cooperative methods. They are just bad design
IMO. Of course we are in the range of opinions, this is a tricky
subject: many smart people agree with me and many others equally smart
disagree.
Still I believe that super is a red herring and that you should really
start thinking: what advantages did
multiple inheritance *really* bring into my code? Could have I done
without? And what would have happen?
These are the relevant question, not the exact meaning of super in
hairy hierarchies.
M. Simionato
Paul Rubin
> I am actually more radical than that. From
> http://www.artima.com/weblogs/viewpost.jsp?thread=237121:
When I look at that URL, I see a Java stack dump:
java.lang.RuntimeException: com.jivesoftware.forum.ForumThreadNotFoundException: ID -1 is not valid
at com.artima.jivecoupled.skins.weblogs.ViewPostPage.process(ViewPostPage.java:112)
...
Seems appropriate.
Jean-Michel Pichavant
JM
Steven D'Aprano
> On Jul 31, 5:08 am, Steven D'Aprano <st...@REMOVE-THIS-
> cybersource.com.au> wrote:
>> I have read Michelle Simionato's articles on super in Python.
>
> One "l" please! I am a man! ;-)
My apologies. You'd think I would know the difference between Michele and
Michelle :(
[snip]
> Still I believe that super is a red herring and that you should really
> start thinking: what advantages did
> multiple inheritance *really* bring into my code? Could have I done
> without? And what would have happen?
> These are the relevant question, not the exact meaning of super in hairy
> hierarchies.
Yes, these are very good points. It's always worth looking at
alternatives to subclassing in the first place. Far too often it's not
just the first idiom people think of, but the *only* idiom they think of.
--
Steven
Mark Lawrence
>
> JM
aka the colon. :)
Cheers.
Mark Lawrence
Mark Lawrence
> On Jul 31, 5:08 am, Steven D'Aprano<st...@REMOVE-THIS-
> cybersource.com.au> wrote:
>> I have read Michelle Simionato's articles on super in Python.
>
> One "l" please! I am a man! ;-)
>
>
> M. Simionato
Ethan Furman
I think the point is that when D initiates the super() chain, and C
calls super, B will then get its turn -- which has to seem arbitrary
from C's point of view.
~Ethan~
donn
> aka the colon. :)
Ha. This is a case of the colon being the appendix!
\d