super in Python 3 and variadic arguments

[Marco Buttu]

Given this class:

>>> class A:
... def afoo(*args):
... print(args)

in Python 3 we can write the following class:

>>> class B(A):
... def bfoo(*args):
... super(B, args[0]).afoo(*args[1:])
...
>>> B().bfoo(1, 2, 3)
(<__main__.B object at 0x7f5b3bde48d0>, 1, 2, 3)


without giving arguments to super, in this way:

>>> class B(A):
... def bfoo(self, *args):
... super().afoo(*args)
...
>>> B().bfoo(1, 2, 3)
(<__main__.B object at 0x7f5b3bdea0d0>, 1, 2, 3)

But it does not work in this case:

>>> class B(A):
... def bfoo(*args):
... super().afoo(*args[1:])
...
>>> B().bfoo(1, 2, 3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in bfoo
RuntimeError: super(): no arguments

How come?
--
Marco Buttu

[Ned Batchelder]

The no-args super() call inspects the calling environment to determine
the class and self. "self" is the first local name stored in
frame.f_code.co_localsplus, but *args doesn't put "args" into that entry
of the code object. Basically, super() is looking for the first regular
argument in the function.

--Ned.

[Marco Buttu]

On 10/09/2013 06:47 PM, Ned Batchelder wrote:

>> >>> class B(A):
>> ... def bfoo(*args):
>> ... super().afoo(*args[1:])
>> ...
>> >>> B().bfoo(1, 2, 3)
>> Traceback (most recent call last):
>> File "<stdin>", line 1, in <module>
>> File "<stdin>", line 3, in bfoo
>> RuntimeError: super(): no arguments
>>
>> How come?
>
> The no-args super() call inspects the calling environment to determine
> the class and self. "self" is the first local name stored in
> frame.f_code.co_localsplus, but *args doesn't put "args" into that entry
> of the code object

But is it a bug or the behavior we want? The first (implicit) argument
is stored as expected as the first one in the args tuple, and the args
tuple is inserted as expected in frame.f_locals:

>>> import inspect

>>> class B(A):
... def bfoo(*args):

... frame = inspect.currentframe()
... for obj, value in frame.f_locals.items():
... print(obj, value, sep=' --> ')
... # super().afoo(*args[1:])
...
>>> B().bfoo(1, 2, 3)
args --> (<__main__.B object at 0x7f28c960a590>, 1, 2, 3)
frame --> <frame object at 0x7f28cad4b240>

So, why does not super use it?

--
Marco Buttu

[Ned Batchelder]

I haven't seen the discussion that decided the behavior of super(), but
I'd guess that if you reported this as a bug, it would be closed as
wontfix, because: 1) the use case you describe isn't something people
actually write, 2) it would add to the complexity of super() to support
it, and 3) there's a simple way to write your code that does work:

class B(A):
def bfoo(self, *args):
super().afoo(*args)

(though it's a bit odd to call afoo from bfoo.)

Python has never claimed the kind of purity that makes everything work
in a totally simple consistent way. super() with no args is a kind of
hack to begin with. It involves a special case in the compiler (so that
using the name "super" as a function call will act as if you had
accessed the name "__class__" so that super can find it later), and
inspecting the stack frame during execution.

It's an interesting case of the Zen of Python. It violates one
("explicit is better than implicit"), but only because of another one
("practicality beats purity"). super(MyClass, self) in Python 2 is the
kind of brain-bender that so many people get wrong at first, that it's
helped plenty of people to do the arguments implicitly, even if there
are oddball edge cases that it doesn't seem to handle properly.

--Ned.

[Marco Buttu]

On 10/10/2013 01:04 PM, Ned Batchelder wrote:

> On 10/10/13 3:22 AM, Marco Buttu wrote:
>> >>> import inspect
>> >>> class B(A):
>> ... def bfoo(*args):
>> ... frame = inspect.currentframe()
>> ... for obj, value in frame.f_locals.items():
>> ... print(obj, value, sep=' --> ')
>> ... # super().afoo(*args[1:])
>> ...
>> >>> B().bfoo(1, 2, 3)
>> args --> (<__main__.B object at 0x7f28c960a590>, 1, 2, 3)
>> frame --> <frame object at 0x7f28cad4b240>
>>
>> So, why does not super use it?
>>
>

> Python has never claimed the kind of purity that makes everything work
> in a totally simple consistent way. super() with no args is a kind of
> hack to begin with. It involves a special case in the compiler (so that
> using the name "super" as a function call will act as if you had
> accessed the name "__class__" so that super can find it later), and
> inspecting the stack frame during execution.

It seems reasonable

> It's an interesting case of the Zen of Python. It violates one
> ("explicit is better than implicit"), but only because of another one
> ("practicality beats purity"). super(MyClass, self) in Python 2 is the
> kind of brain-bender that so many people get wrong at first, that it's
> helped plenty of people to do the arguments implicitly, even if there
> are oddball edge cases that it doesn't seem to handle properly.
>
> --Ned.

Thanks for the comprehensive answer ;)

--
Marco Buttu

[Steven D'Aprano]

On Thu, 10 Oct 2013 07:04:38 -0400, Ned Batchelder wrote:

> super() with no args is a kind of hack to begin with. It involves a
> special case in the compiler (so that using the name "super" as a
> function call will act as if you had accessed the name "__class__" so
> that super can find it later), and inspecting the stack frame during
> execution.

super() with no arguments is *completely* a hack[1], and one where GvR
has said "Never again!" if I remember correctly. I don't think he regrets
allowing the super compile-time magic, just that it really is magic and
he doesn't want to make a habit of it.

One of the side-effects of this being a hack is that this doesn't work:

class X(Y):
def method(self, arg):
f = super
f().method(arg)





[1] Which is not necessarily a bad thing!

--
Steven

[Ian Kelly]

On Thu, Oct 10, 2013 at 8:11 PM, Steven D'Aprano
<steve+comp....@pearwood.info> wrote:
> One of the side-effects of this being a hack is that this doesn't work:
>
> class X(Y):
> def method(self, arg):
> f = super
> f().method(arg)

Actually, that works just fine. The compiler sees that super is
accessed within the method and creates the closure necessary to make
it work. This does fail, however:

f = super

class X(Y):
def method(self, arg):

f().method(arg)

[Steven D'Aprano]

Ah, that's the one! Thanks for the correction.

I'll now go and write "I will always test my code snippets before
posting" on the blackboard one hundred times.



--
Steven

[Chris Angelico]

On Fri, Oct 11, 2013 at 2:00 PM, Steven D'Aprano
<steve+comp....@pearwood.info> wrote:
> I'll now go and write "I will always test my code snippets before
> posting" on the blackboard one hundred times.

print("I will always test my code snippets before posting\n"*100)

ChrisA

PS. Irony would be having a bug in that because I didn't test it. I
almost didn't, but remembered just before hitting Send.

[Marco Buttu]

On 10/11/2013 04:11 AM, Steven D'Aprano wrote:

> super() with no arguments is*completely* a hack[1], and one where GvR

> has said "Never again!" if I remember correctly. I don't think he regrets
> allowing the super compile-time magic, just that it really is magic and
> he doesn't want to make a habit of it.
>

...

> [1] Which is not necessarily a bad thing!

Thanks a lot for this anecdote :)

--
Marco Buttu

[Marco Buttu]

On 10/11/2013 04:33 AM, Ian Kelly wrote:

> On Thu, Oct 10, 2013 at 8:11 PM, Steven D'Aprano

>> >One of the side-effects of this being a hack is that this doesn't work:
>> >
>> >class X(Y):
>> > def method(self, arg):
>> > f = super
>> > f().method(arg)

> Actually, that works just fine. The compiler sees that super is
> accessed within the method and creates the closure necessary to make
> it work. This does fail, however:
>
> f = super
> class X(Y):
> def method(self, arg):
> f().method(arg)

Very interesting! Thanks :)
--
Marco Buttu