[Python-Dev] Enumeration items: `type(EnumClass.item) is EnumClass` ?

[Ethan Furman]

[creating new thread]

On 04/29/2013 01:30 AM, Steven D'Aprano wrote:
> On Sun, Apr 28, 2013 at 11:50:16PM -0700, Ethan Furman wrote:
>
>> In other words, currently:
>>
>> class Color(Enum):
>> red = 1
>> green = 2
>> blue = 3
>>
>> class MoreColor(Color):
>> cyan = 4
>> magenta = 5
>> yellow = 6
>> black = 7
>>
>> MoreColor.red is Color.red # True
>
> Correct.
>
>
>> But as soon as:
>>
>> type(Color.red) is Color # True
>> type(MoreColor.red) is MoreColor # True
>
> I don't believe this is correct. As I understand it, the proposal is the
> weaker guarantee:
>
> isinstance(Color.red, Color) # True, possibly using __instancecheck__


Words from Guido:

On 04/23/2013 08:11 AM, Guido van Rossum wrote:
> I gotta say, I'm with Antoine here. It's pretty natural (also coming
> from other languages) to assume that the class used to define the
> enums is also the type of the enum values. Certainly this is how it
> works in Java and C++, and I would say it's the same in Pascal and
> probably most other languages.
>

On 04/25/2013 02:54 PM, Guido van Rossum wrote:
> I don't know what's going on, but it feels like we had this same
> discussion a week ago, and I still disagree. Disregarding, the C[i]
> notation, I feel quite strongly that in the following example:
>
> class Color(Enum):
> red = 1
> white = 2
> blue = 3
> orange = 4
>
> the values Color.red etc. should be instances of Color. This is how
> things work in all other languages that I am aware of that let you
> define enums.

On 04/25/2013 03:19 PM, Guido van Rossum wrote:
> I suppose you were going to propose to use isinstance() overloading,
> but I honestly think that Color.red.__class__ should be the same
> object as Color.

On 04/25/2013 03:37 PM, Guido van Rossum wrote:
> TBH I had a hard time getting over the fact that even though the class
> said "a = 1", C.a is not the integer 1. But I did get over it.
> Hopefully you can get over *this* weirdness.

[and from the summary thread]

On 04/28/2013 01:02 PM, Guido van Rossum wrote:
> On Sun, Apr 28, 2013 at 12:32 PM, Ethan Furman wrote:
>>
>> - should enum items be of the type of the Enum class? (i.e. type(SPRING)
>> is Seasons)
>
> IMO Yes.
_______________________________________________
Python-Dev mailing list
Pytho...@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/dev-python%2Bgarchive-30976%40googlegroups.com

[Guido van Rossum]

Indeed, the "type(Color.red) is Color" claim was meant for the
situation where red is defined directly in Color, and I used type()
instead of isinstance() because Barry was proposing to overload
isinstance() to make this true without equating the classes. But for
the subclass case, I want MoreColor.red is Color.red and
isinstance(MoreColor.red, Color), but not isinstance(Color.red,
MoreColor). If you can't live with that, don't subclass enums.

> http://mail.python.org/mailman/options/python-dev/guido%40python.org



--
--Guido van Rossum (python.org/~guido)

[Ethan Furman]

On 04/29/2013 08:39 AM, Guido van Rossum wrote:
> Indeed, the "type(Color.red) is Color" claim was meant for the
> situation where red is defined directly in Color, and I used type()
> instead of isinstance() because Barry was proposing to overload
> isinstance() to make this true without equating the classes. But for
> the subclass case, I want MoreColor.red is Color.red and
> isinstance(MoreColor.red, Color), but not isinstance(Color.red,
> MoreColor). If you can't live with that, don't subclass enums.

So if I understand:

--> class Color(Enum):
... red = 1
... green = 2
... blue = 3

--> class MoreColor(Color):
... cyan = 4
... magenta = 5
... yellow = 6

--> type(MoreColor.red) is Color

--> type(MoreColor.red) is not MoreColor

In other words, while `red` is accessible in MoreColor, it's actually a Color instance?

--
~Ethan~

[Guido van Rossum]

On Mon, Apr 29, 2013 at 9:12 AM, Ethan Furman <et...@stoneleaf.us> wrote:
> On 04/29/2013 08:39 AM, Guido van Rossum wrote:
>>
>> Indeed, the "type(Color.red) is Color" claim was meant for the
>> situation where red is defined directly in Color, and I used type()
>> instead of isinstance() because Barry was proposing to overload
>> isinstance() to make this true without equating the classes. But for
>> the subclass case, I want MoreColor.red is Color.red and
>> isinstance(MoreColor.red, Color), but not isinstance(Color.red,
>> MoreColor). If you can't live with that, don't subclass enums.
>
>
> So if I understand:
>
> --> class Color(Enum):
> ... red = 1
> ... green = 2
> ... blue = 3
>
> --> class MoreColor(Color):
> ... cyan = 4
> ... magenta = 5
> ... yellow = 6
>
> --> type(MoreColor.red) is Color
>
> --> type(MoreColor.red) is not MoreColor
>
> In other words, while `red` is accessible in MoreColor, it's actually a
> Color instance?

Oh dear, this is actually a mess. I don't want MoreColor.red and
Color.red to be distinct objects, but then the isinstance() checks
will become confusing. If we don't override isinstance(), we'll get

not isinstance(Color.red, MoreColor)
isinstance(MoreColor.yellow, Color)

This would be pretty backwards.

I Ggoogled "enum subclassing" and found this StackOverflow article
explaining why you can't subclass enums in Java:
http://stackoverflow.com/questions/4604978/subclassing-an-enum which
refers to this more elaborate answer:
http://stackoverflow.com/questions/3427947/enumeration-inheritence-in-java/3428050#3428050

I think this conclusively shows that it's better to disallow
subclassing enums. (It also brings enum in line with bool, which is
also a "final" class.)

Adding Eli to the thread explicitly, because this needs to be
explained in the PEP.

--
--Guido van Rossum (python.org/~guido)

[Glenn Linderman]

The only thing that needs to be disallowed is defining additional named elements, for Liskov Substitution to not be violated.

So in an inheritance tree derived from Enum, it would be a requirement that all members of the enumeration be enumerated in one class.

It would be extremely nice if:

1) Enum could be subclassed to provide different, sharable, types of behaviors, then further subclassed to provide a number of distinct sets of values with those behaviors.

2) Enum could be subclassed to provide one set of values, and then further subclassed to provide a number a distinct sets of behaviors for those sets of values.

(I see a lot more benefit to #1, than #2, if a choice must be made.)

[Steven D'Aprano]

On 30/04/13 03:01, Guido van Rossum wrote:

> Oh dear, this is actually a mess. I don't want MoreColor.red and
> Color.red to be distinct objects, but then the isinstance() checks
> will become confusing. If we don't override isinstance(), we'll get
>
> not isinstance(Color.red, MoreColor)
> isinstance(MoreColor.yellow, Color)
>
> This would be pretty backwards.

Why is that backwards? MoreColor is a subclass of Color, so instances of MoreColor are instances of Color, but instances of Color are not instances of MoreColor. That's normal behaviour for subclasses. (All cats are mammals, but not all mammals are cats.)

The only "funny" thing about this is that enumeration values of a Enum are only instances of that Enum if they are declared inside that Enum, and as far as I'm concerned that's not especially funny at all. So:

Color declares red, so Color.red is a Color.

MoreColor does not declare red, it merely inherits it from Color, so MoreColor.red is Color.red which is not a MoreColor.


This leads to a simple rule:

Enum values are either instances of the enum they are defined in, or instances of the parent class they are inherited from.

If the user doesn't like that, they're free to not subclass Enums :-)

> I Ggoogled "enum subclassing" and found this StackOverflow article
> explaining why you can't subclass enums in Java:
> http://stackoverflow.com/questions/4604978/subclassing-an-enum which
> refers to this more elaborate answer:
> http://stackoverflow.com/questions/3427947/enumeration-inheritence-in-java/3428050#3428050

The first link says:

"And generally, a proper subclass is a specialization of its superclass."

and gives the Liskov Substitution Principle as the reason for prohibiting subclassing. LSP is a very useful principle, but it is not the only model for subclassing. That's why it's called a *principle* and not the Liskov Substitution *Law*.

(Yes, I stole that from Raymond Hettinger's talk on subclassing at PyCon.)

I think that the ability to extend enums with new ones, without duplicating code, is more important for enums than satisfying LSP. The common use-cases for enums do not lend itself to subtyping in the Liskov sense.

E.g. if I have a function that expects a Color red/blue/green enum, and I subclass it to give MoreColor yellow, I can't expect the function to suddenly recognise yellow. So I'm not going to use subclassing in this case, because it can't work.

But I will use subclassing to reuse values: if I have a function that expects red/blue/green/yellow, and red/blue/green are already defined in Color, I'll want to reuse them rather than duplicate them. Hence I will subclass Color specifically to extend it, not to specialise it.


The second link quotes:

"For the most part, extensibility of enums turns out to be a bad idea. It is confusing that elements of an extension type are instances of the base type and not vice versa."

Confusing for who, and why?

You're going to confuse *somebody* no matter what you do. Either:

- confuse people who try to subclass enums, and can't; or

- confuse people who try to subclass enums, and can, but then get confused by the result.





--
Steven

[Guido van Rossum]

You are too verbose. I have already said what I needed to say.

> http://mail.python.org/mailman/options/python-dev/guido%40python.org

--
--Guido van Rossum (python.org/~guido)

[Larry Hastings]

What's the problem with overriding the isinstance checks?  You mention it but seem to assume it's a bad idea.  That seems to me like it'd adequately solve that problem with an acceptable level of magic.


/arry

[MRAB]

On 29/04/2013 19:24, Steven D'Aprano wrote:
> On 30/04/13 03:01, Guido van Rossum wrote:
>
>> Oh dear, this is actually a mess. I don't want MoreColor.red and
>> Color.red to be distinct objects, but then the isinstance() checks
>> will become confusing. If we don't override isinstance(), we'll
>> get
>>
>> not isinstance(Color.red, MoreColor) isinstance(MoreColor.yellow,
>> Color)
>>
>> This would be pretty backwards.
>
> Why is that backwards? MoreColor is a subclass of Color, so instances
> of MoreColor are instances of Color, but instances of Color are not
> instances of MoreColor. That's normal behaviour for subclasses. (All
> cats are mammals, but not all mammals are cats.)
>

Let's say that Color is red, green, or blue.

Let's also say that MoreColor is superset of Color, in other words, any
of Color, plus cyan, magenta, or yellow.

Red is a Color and a MoreColor (member of Color and MoreColor).

Yellow is a MoreColor, but not a Color (member of MoreColor but not
Color).

That's the opposite of the rules of inheritance.

On the other hand, a function that expects MoreColor should also accept
Color, which is a subset.

>
> The second link quotes:
>
> "For the most part, extensibility of enums turns out to be a bad
> idea. It is confusing that elements of an extension type are
> instances of the base type and not vice versa."
>
> Confusing for who, and why?
>
> You're going to confuse *somebody* no matter what you do. Either:
>
> - confuse people who try to subclass enums, and can't; or
>
> - confuse people who try to subclass enums, and can, but then get
> confused by the result.
>

[Guido van Rossum]

On Mon, Apr 29, 2013 at 11:34 AM, Larry Hastings <la...@hastings.org> wrote:
> What's the problem with overriding the isinstance checks? You mention it
> but seem to assume it's a bad idea. That seems to me like it'd adequately
> solve that problem with an acceptable level of magic.

Depending on whether you are using isinstance() to check if an enum
value belongs in the set of acceptable enums, or to check it it makes
sense to call a certain method, you need isinstance() to behave
differently.

In particular, the default isinstance() would correctly state that
MoreColor.red is not a MoreColor instance, so any methods defined only
on MoreColor should not be called. OTOH it gives the wrong answer if
you are checking that Color.red is an acceptable MoreColor value. But
if you override isinstance() to give the right answer for the latter
(so isinstance(Color.red, MoreColor) is True), then you could
incorrectly conclude that it is safe to call a MoreColor method on
Color.red.

Please do read the StackOverflow links I gave:

http://stackoverflow.com/questions/4604978/subclassing-an-enum
http://stackoverflow.com/questions/3427947/enumeration-inheritence-in-java/3428050#3428050

[Serhiy Storchaka]

29.04.13 21:14, Glenn Linderman написав(ла):

> 1) Enum could be subclassed to provide different, sharable, types of
> behaviors, then further subclassed to provide a number of distinct sets
> of values with those behaviors.

You can use a multiclass inheritance for this.

> 2) Enum could be subclassed to provide one set of values, and then
> further subclassed to provide a number a distinct sets of behaviors for
> those sets of values.

How is it possible? You haven't any instance of subclass.

[Steven D'Aprano]

On 30/04/13 04:29, Guido van Rossum wrote:
> You are too verbose. I have already said what I needed to say.

Sorry about that, sometimes I do go on and on. Let me be more terse.

When it comes to enums, I believe that violating Liskov is a feature, not a bug.

Also, I understand that both Scala and Kotlin allow subclassing enums in exactly the way we're talking about.

--
Steven
_______________________________________________
Python-Dev mailing list
Pytho...@python.org
http://mail.python.org/mailman/listinfo/python-dev

Unsubscribe: http://mail.python.org/mailman/options/python-dev/dev-python%2Bgarchive-30976%40googlegroups.com

[Oscar Benjamin]

On 29 April 2013 20:04, Guido van Rossum <gu...@python.org> wrote:
> On Mon, Apr 29, 2013 at 11:34 AM, Larry Hastings <la...@hastings.org> wrote:
>> What's the problem with overriding the isinstance checks? You mention it
>> but seem to assume it's a bad idea. That seems to me like it'd adequately
>> solve that problem with an acceptable level of magic.
>
> Depending on whether you are using isinstance() to check if an enum
> value belongs in the set of acceptable enums, or to check it it makes
> sense to call a certain method, you need isinstance() to behave
> differently.

Would it not be better to avoid using isinstance() for this?

I would have expected membership testing to use "Red in Colors" rather
than "isinstance(Red, Color)" like in this stackoverflow question
about an enum class:
http://stackoverflow.com/questions/10445819/overriding-contains-method-for-a-class


Oscar

[Steven D'Aprano]

On 30/04/13 05:02, MRAB wrote:

>> Why is that backwards? MoreColor is a subclass of Color, so instances
>> of MoreColor are instances of Color, but instances of Color are not
>> instances of MoreColor. That's normal behaviour for subclasses. (All
>> cats are mammals, but not all mammals are cats.)
>>
> Let's say that Color is red, green, or blue.
>
> Let's also say that MoreColor is superset of Color, in other words, any
> of Color, plus cyan, magenta, or yellow.
>
> Red is a Color and a MoreColor (member of Color and MoreColor).
>
> Yellow is a MoreColor, but not a Color (member of MoreColor but not
> Color).
>
> That's the opposite of the rules of inheritance.

Membership != inheritance, you are conflating two independent concepts. I would expect that for membership testing, you should say "value in MoreColor", not isinstance(value, MoreColor).

flufl.enum has been in use for Mailman for many years, and I would like to hear Barry's opinion on this.



--
Steven

[Guido van Rossum]

On Mon, Apr 29, 2013 at 12:52 PM, Oscar Benjamin
<oscar.j....@gmail.com> wrote:
> Would it not be better to avoid using isinstance() for this?

In a recent thread I explained why using isinstance() is important.
It's how we check for every other type, and it would be awlward for
type-checking frameworks to have to special-case enums. (When we're
not using duck typing.)

--
--Guido van Rossum (python.org/~guido)

[MRAB]

On 29/04/2013 21:00, Steven D'Aprano wrote:
> On 30/04/13 05:02, MRAB wrote:
>
>>> Why is that backwards? MoreColor is a subclass of Color, so
>>> instances of MoreColor are instances of Color, but instances of
>>> Color are not instances of MoreColor. That's normal behaviour for
>>> subclasses. (All cats are mammals, but not all mammals are
>>> cats.)
>>>
>> Let's say that Color is red, green, or blue.
>>
>> Let's also say that MoreColor is superset of Color, in other words,
>> any of Color, plus cyan, magenta, or yellow.
>>
>> Red is a Color and a MoreColor (member of Color and MoreColor).
>>
>> Yellow is a MoreColor, but not a Color (member of MoreColor but
>> not Color).
>>
>> That's the opposite of the rules of inheritance.
>
> Membership != inheritance, you are conflating two independent
> concepts. I would expect that for membership testing, you should say
> "value in MoreColor", not isinstance(value, MoreColor).
>

I do know the difference, but it's not helpful that you're creating the
enum with a "class" statement!

> flufl.enum has been in use for Mailman for many years, and I would
> like to hear Barry's opinion on this.
>

[Barry Warsaw]

On Apr 30, 2013, at 06:00 AM, Steven D'Aprano wrote:

>flufl.enum has been in use for Mailman for many years, and I would like to
>hear Barry's opinion on this.

I'm not sure it matters now (I'm about a billion messages behind with little
hope of catching up), but FWIW, I have use cases for extending through
inheritance and haven't had any kind of semantic confusion. But then again, I
haven't particularly needed to do type checks or isinstance checks, and I
haven't needed to put methods on enums either.

-Barry