[Python-ideas] Thread.__init__ should call super()

[Ilya Kulakov]

Since one of the legit use-cases of using the Thread class is subclassing,

I think it's __init__ should call super() to support cooperative inheritance.

Or perhaps there is a good reason for not doing so?

Best Regards,

Ilya Kulakov

[Guido van Rossum]

You can subclass Thread just fine, you just can't have it in a multiple inheritance hierarchy except at the end of the MRO (before object). That shouldn't stop you from doing anything you want though -- you can define e.g.

class MyThread(Thread):

    def __init__(self, *args, **kwds):

        Thread.__init__(self, *args, **kwds)

        super(Thread, self).__init__(*args, **kwds)

and use this class instead of Thread everywhere. (You'll have to decide which arguments to pass on and which ones to ignore, but that's not specific to the issue of Thread.)

Of course you're probably better off not trying to be so clever.

_______________________________________________
Python-ideas mailing list
Python...@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

--

--Guido van Rossum (python.org/~guido)

[Steven D'Aprano]

On Fri, Oct 27, 2017 at 01:59:01PM -0700, Ilya Kulakov wrote:

> Since one of the legit use-cases of using the Thread class is subclassing,
> I think it's __init__ should call super() to support cooperative inheritance.
>
> Or perhaps there is a good reason for not doing so?

Are you talking about threading.Thread or some other Thread?

If you are talking about threading.Thread, its only superclass is
object, so why bother calling super().__init__?

To be successful, it would need to strip out all the parameters and just
call:

super().__init__()

with no args, as object.__init__() takes no parameters. And that does
nothing, so what's the point?

I'm afraid I don't see why you think that threading.Thread needs to call
super. Can you explain?


--
Steve

[Neil Girdhar]

On Friday, October 27, 2017 at 8:05:17 PM UTC-4, Steven D'Aprano wrote:

On Fri, Oct 27, 2017 at 01:59:01PM -0700, Ilya Kulakov wrote:

> Since one of the legit use-cases of using the Thread class is subclassing,
> I think it's __init__ should call super() to support cooperative inheritance.
>
> Or perhaps there is a good reason for not doing so?

Are you talking about threading.Thread or some other Thread?

If you are talking about threading.Thread, its only superclass is
object, so why bother calling super().__init__?

The way cooperative multiple inheritance works is that if someone defines

class SomeClass(Thread):

     def __init__(self, **kwargs):

           super().__init()

they expect this will initialize the base class Thread as desired.

Now, if they add another base class:

class SomeBase:

    def __init__(self, base_x):

        self.base_x = base_x

then they need to pass up the arguments:

class SomeClass(SomeBase, Thread):

     def __init__(self, **kwargs):

           super().__init(**kwargs)

Unfortunately, if the order of base classes is reversed, this no longer works because Thread doesn't call super:

class SomeClass(Thread, SomeBase):

     def __init__(self, **kwargs):

           super().__init(**kwargs)  # SomeBase is not initialized!

As things get more complicated it's not always possible to ensure that Thread is the last class in the inheritance, e.g., if there are two classes like Thread that don't call super.

[Neil Girdhar]

I meant:

class SomeBase:

    def __init__(self, base_x, **kwargs):

        super().__init__(**kwargs)

        self.base_x = base_x

[Neil Girdhar]

Out of curiosity, what is the benefit of not calling super from Thread.__init__?

[Steven D'Aprano]

On Sat, Oct 28, 2017 at 12:14:31AM -0700, Neil Girdhar wrote:
>
>
> On Friday, October 27, 2017 at 8:05:17 PM UTC-4, Steven D'Aprano wrote:
> >
> > On Fri, Oct 27, 2017 at 01:59:01PM -0700, Ilya Kulakov wrote:
> >
> > > Since one of the legit use-cases of using the Thread class is
> > subclassing,
> > > I think it's __init__ should call super() to support cooperative
> > inheritance.
> > >
> > > Or perhaps there is a good reason for not doing so?
> >
> > Are you talking about threading.Thread or some other Thread?
> >
> > If you are talking about threading.Thread, its only superclass is
> > object, so why bother calling super().__init__?
> >
>
> The way cooperative multiple inheritance works is that if someone defines

I didn't realise that Ilya was talking about *multiple* inheritance,
since he didn't use the word, only "cooperative". But since you
are talking about multiple inheritence:

> class SomeClass(Thread):
> def __init__(self, **kwargs):
> super().__init()
>
> they expect this will initialize the base class Thread as desired.

That won't work, since you misspelled __init__ and neglected to pass any
arguments :-) You need:

super().__init__(**kwargs)

otherwise Thread will not be initialised.

> Now, if they add another base class:
>
> class SomeBase:
> def __init__(self, base_x):
> self.base_x = base_x
>
> then they need to pass up the arguments:
>
> class SomeClass(SomeBase, Thread):
> def __init__(self, **kwargs):
> super().__init(**kwargs)

That's not going to work either, because you're passing arguments to
SomeBase that it doesn't understand: all the args that Thread expects.

And of course its going to doubly not work, since SomeBase fails to call
super, so Thread.__init__ still doesn't get called.

If you fix all those problems, you still have another problem: in
Thread, if you call

super().__init__(**kwargs)

then object.__init__ will fail, as I mentioned in my earlier post; but
if you call:

super().__init__()

then object is satisfied, but SomeBase.__init__ gets called with no
arguments. You can't satisfy both at the same time with a single call
to super.

> Unfortunately, if the order of base classes is reversed, this no longer
> works because Thread doesn't call super:
>
> class SomeClass(Thread, SomeBase):
> def __init__(self, **kwargs):
> super().__init(**kwargs) # SomeBase is not initialized!
>
> As things get more complicated it's not always possible to ensure that
> Thread is the last class in the inheritance, e.g., if there are two classes
> like Thread that don't call super.

You can't just add classes willy-nilly into the superclass list. That's
why it is called *cooperative* multiple inheritence: the superclasses
all have to be designed to work together. You *can't* have two classes
that don't call super -- and you must have one class just ahead of
object, to prevent object from receiving args it can't do anything with.

And that class might as well be Thread. At least, I can't think of any
reason why it shouldn't be Thread.

[Neil Girdhar]

On Sat, Oct 28, 2017 at 7:15 AM Steven D'Aprano <st...@pearwood.info> wrote:

On Sat, Oct 28, 2017 at 12:14:31AM -0700, Neil Girdhar wrote:
>
>
> On Friday, October 27, 2017 at 8:05:17 PM UTC-4, Steven D'Aprano wrote:
> >
> > On Fri, Oct 27, 2017 at 01:59:01PM -0700, Ilya Kulakov wrote:
> >
> > > Since one of the legit use-cases of using the Thread class is
> > subclassing,
> > > I think it's __init__ should call super() to support cooperative
> > inheritance.
> > >
> > > Or perhaps there is a good reason for not doing so?
> >
> > Are you talking about threading.Thread or some other Thread?
> >
> > If you are talking about threading.Thread, its only superclass is
> > object, so why bother calling super().__init__?
> >
>
> The way cooperative multiple inheritance works is that if someone defines

I didn't realise that Ilya was talking about *multiple* inheritance,
since he didn't use the word, only "cooperative". But since you
are talking about multiple inheritence:


> class SomeClass(Thread):
>      def __init__(self, **kwargs):
>            super().__init()
>
> they expect this will initialize the base class Thread as desired.

That won't work, since you misspelled __init__ and neglected to pass any
arguments :-) You need:

    super().__init__(**kwargs)

otherwise Thread will not be initialised.

(I corrected myself right after.) 

> Now, if they add another base class:
>
> class SomeBase:
>     def __init__(self, base_x):
>         self.base_x = base_x
>
> then they need to pass up the arguments:
>
> class SomeClass(SomeBase, Thread):
>      def __init__(self, **kwargs):
>            super().__init(**kwargs)

That's not going to work either, because you're passing arguments to
SomeBase that it doesn't understand: all the args that Thread expects.

That's totally fine.  That's how cooperative multiple inheritance works in Python.  SomeBase is supposed to pass along everything to its superclass init through kwargs just like I illustrated in my corrected code.

 

And of course its going to doubly not work, since SomeBase fails to call
super, so Thread.__init__ still doesn't get called.

If you fix all those problems, you still have another problem: in
Thread, if you call

    super().__init__(**kwargs)

then object.__init__ will fail, as I mentioned in my earlier post; but
if you call: 

    super().__init__()

then object is satisfied, but SomeBase.__init__ gets called with no
arguments. You can't satisfy both at the same time with a single call
to super.

No, this works fine.  The idea is that each class consumes the keyword arguments it wants and passes along the rest.  By the time you get to object, there are none left and object.__init__ doesn't complain.  If there are extra arguments, then object.__init__ raises.

 

> Unfortunately, if the order of base classes is reversed, this no longer
> works because Thread doesn't call super:
>
> class SomeClass(Thread, SomeBase):
>      def __init__(self, **kwargs):
>            super().__init(**kwargs)  # SomeBase is not initialized!
>
> As things get more complicated it's not always possible to ensure that
> Thread is the last class in the inheritance, e.g., if there are two classes
> like Thread that don't call super.

You can't just add classes willy-nilly into the superclass list. That's
why it is called *cooperative* multiple inheritence: the superclasses
all have to be designed to work together. You *can't* have two classes
that don't call super -- and you must have one class just ahead of
object, to prevent object from receiving args it can't do anything with. 

You don't need "one class just ahead of object".   Every class calls super passing along its arguments via kwargs.  And it's very hard when combining various mixins to ensure that a given order is maintained.

 

And that class might as well be Thread. At least, I can't think of any
reason why it shouldn't be Thread.

I'm sorry, but I don't agree with this.  Unfortunately, there are some various oversights in Python when it comes to cooperative multiple inheritance.  This is only one of them that I've also run into.

 

--
Steve
_______________________________________________
Python-ideas mailing list
Python...@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

--

---
You received this message because you are subscribed to a topic in the Google Groups "python-ideas" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/python-ideas/mgHYhQKAbdo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to python-ideas...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Antoine Pitrou]

On Fri, 27 Oct 2017 13:59:01 -0700
Ilya Kulakov <kulako...@gmail.com>
wrote:

> Since one of the legit use-cases of using the Thread class is subclassing,
> I think it's __init__ should call super() to support cooperative inheritance.

Not to derail this thread, but I find it much clearer to use the
functional form of the Thread class, i.e. to pass the `target` and
`args` constructor parameters.

Regards

Antoine.

[Ilya Kulakov]

Neil, thank you for doing much better job explaining the problem.

Generally, I'm cool with Python's standard library classes not calling super(), as many of them
are not designed for subclassing. But those which are should do that. E.g. take a look at more
recent asyncio's Protocol and Transport classes: they all properly call super().

One potential problem is that it will break existing code:

class X(Thread, SomethingElse):
def __init__(self):
Thread.__init__(self)
SomethingElse.__init__(self)

SomethingElse.__init__ will be called twice.
Is it a good reason for "old" classes to lag behind? I don't know.

Perhaps some mechanism (invisible to a user) can be designed to avoid that.
E.g. super() may leave a flag which should signal interpreter to "skip" all direct calls
of a function and warn about it (DeprecationWarning?).

Best Regards,
Ilya Kulakov