[Python-Dev] Enum -- last call for comments on 3.10 changes
103 views
Skip to first unread message
Ethan Furman
Jun 28, 2021, 4:00:22 AM6/28/21
to Python-Dev
I have spoken with Pablo (3.10 RM), and he agrees that a change to Enum str() in 3.10 and another in 3.11 is less than
ideal, so this new thread is to collect comments about Enum and it's str() and repr() and whether the changes take
effect in 3.10, 3.11, or both.
TL;DR -- sorry, there isn't one.
History:
--------
As Enum and IntEnum have started proliferating in the stdlib, some of those enumerations have changed the str() and
repr() of their members -- for example, in 3.8 re.RegexFlag had its repr() and str() changed to be module.name
(re.ASCII) for both instead of <enum_class.name: value> (<RegexFlag.ASCII: 256>) for the repr(); that change made sense
because all the members are exported to re's global name space, and they are accessed as `re.ASCII` in existing code.
While a good change (in my opinion and with which I had nothing to do), we now have stdlib enumerations with differing
str() and repr()s, which is a minor ding against recognizability and usability.
Current 3.10 Changes
--------------------
In an effort to standardize the stdlib enums, a new decorator was added: global_enum(). It can be called manually (for
example in re.RegexFlag), or automatically by Enum._convert_ (for example in ssl.VerifyFlags).
That changed was visually appealing, and I had users wanting that type of output for Enum in general, so after asking
for feedback on python-dev and python-ideas (and receiving little) I changed the basic str() and repr() of Enum to:
- str(): NAME
- repr(): enum.NAME
While working on some other bugs/changes in Enum.__format__ and serialization in general, I have come to believe that
IntEnum (and IntFlag) should be as near-perfect a drop-in replacement as possible for when they are used to replace
existing integer constants (which I believe is their most important use-case).
Reverting 3.10 Changes
----------------------
I have been increasingly unhappy with the general `Enum.__repr__` change, so while the stdlib global repr() change is
staying, Enum, IntEnum, etc., is going back to <enum_class.NAME: value>.
Proposed 3.10 Change (for 3.10.0 beta 4)
----------------------------------------
In order to improve the drop-in replacement use-case for IntEnum and IntFlag, I would like to change the str() for those
two to be simply the value of the member, as if it was still a plain integer and not part of an enumeration. At that
point the only obvious difference would be the repr(), which I think is the most important and useful change as that is
what (should) show up in log files, the REPL, etc.
This would also make IntEnum and IntFlag consistent with StrEnum as, for other reasons, StrEnum member str()s are the
values of the members rather than the names of the members. Note also that this change would not affect user created
enumerations that mixed in int on their own.
The Question
------------
With all the changes currently happening to the str()s and repr()s of the various enums, what are the good reasons to
not make this final change now, and instead have one more change in 3.11?
--
~Ethan~
Summary of Enum str()s and repr()s by category
| | repr() | str() | format()
| stdlib global | re.ASCII | re.ASCII | 256
| Enum | <Color.RED: 1> | Color.RED | RED
| IntEnum | <Color.RED: 1> | 1 | 1
| int, Enum | <Color.RED: 1> | Color.RED | 1 (will be RED in 3.12 due to back-compatibility
| | | constraints which should affect very few people as
| | | as they are probably using IntEnum instead of mixing
| | | in int themselves -- comments about this likelihood also
| | | appreciated)
_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/ZMC67QA2JVQJSWSFWRS6IM6ZX4EK277G/
Code of Conduct: http://python.org/psf/codeofconduct/
ideal, so this new thread is to collect comments about Enum and it's str() and repr() and whether the changes take
effect in 3.10, 3.11, or both.
TL;DR -- sorry, there isn't one.
History:
--------
As Enum and IntEnum have started proliferating in the stdlib, some of those enumerations have changed the str() and
repr() of their members -- for example, in 3.8 re.RegexFlag had its repr() and str() changed to be module.name
(re.ASCII) for both instead of <enum_class.name: value> (<RegexFlag.ASCII: 256>) for the repr(); that change made sense
because all the members are exported to re's global name space, and they are accessed as `re.ASCII` in existing code.
While a good change (in my opinion and with which I had nothing to do), we now have stdlib enumerations with differing
str() and repr()s, which is a minor ding against recognizability and usability.
Current 3.10 Changes
--------------------
In an effort to standardize the stdlib enums, a new decorator was added: global_enum(). It can be called manually (for
example in re.RegexFlag), or automatically by Enum._convert_ (for example in ssl.VerifyFlags).
That changed was visually appealing, and I had users wanting that type of output for Enum in general, so after asking
for feedback on python-dev and python-ideas (and receiving little) I changed the basic str() and repr() of Enum to:
- str(): NAME
- repr(): enum.NAME
While working on some other bugs/changes in Enum.__format__ and serialization in general, I have come to believe that
IntEnum (and IntFlag) should be as near-perfect a drop-in replacement as possible for when they are used to replace
existing integer constants (which I believe is their most important use-case).
Reverting 3.10 Changes
----------------------
I have been increasingly unhappy with the general `Enum.__repr__` change, so while the stdlib global repr() change is
staying, Enum, IntEnum, etc., is going back to <enum_class.NAME: value>.
Proposed 3.10 Change (for 3.10.0 beta 4)
----------------------------------------
In order to improve the drop-in replacement use-case for IntEnum and IntFlag, I would like to change the str() for those
two to be simply the value of the member, as if it was still a plain integer and not part of an enumeration. At that
point the only obvious difference would be the repr(), which I think is the most important and useful change as that is
what (should) show up in log files, the REPL, etc.
This would also make IntEnum and IntFlag consistent with StrEnum as, for other reasons, StrEnum member str()s are the
values of the members rather than the names of the members. Note also that this change would not affect user created
enumerations that mixed in int on their own.
The Question
------------
With all the changes currently happening to the str()s and repr()s of the various enums, what are the good reasons to
not make this final change now, and instead have one more change in 3.11?
--
~Ethan~
Summary of Enum str()s and repr()s by category
| | repr() | str() | format()
| stdlib global | re.ASCII | re.ASCII | 256
| Enum | <Color.RED: 1> | Color.RED | RED
| IntEnum | <Color.RED: 1> | 1 | 1
| int, Enum | <Color.RED: 1> | Color.RED | 1 (will be RED in 3.12 due to back-compatibility
| | | constraints which should affect very few people as
| | | as they are probably using IntEnum instead of mixing
| | | in int themselves -- comments about this likelihood also
| | | appreciated)
_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/ZMC67QA2JVQJSWSFWRS6IM6ZX4EK277G/
Code of Conduct: http://python.org/psf/codeofconduct/
Nick Coghlan
Jun 28, 2021, 10:00:52 AM6/28/21
to Ethan Furman, Python-Dev
On Mon, 28 Jun 2021, 6:02 pm Ethan Furman, <et...@stoneleaf.us> wrote:
I have spoken with Pablo (3.10 RM), and he agrees that a change to Enum str() in 3.10 and another in 3.11 is less than
ideal, so this new thread is to collect comments about Enum and it's str() and repr() and whether the changes take
effect in 3.10, 3.11, or both.
TL;DR -- sorry, there isn't one.
I'll have a go at one:
* Enum str() continuing as "module.NAME", while format() omits the module
* Enum repr() changing back to the historical behaviour, unless you opt in to the new behaviour with the global enum decorator: definite +1 here
* Making str() consistent with format() and the non-Enum base class for IntEnum et al (it is already consistent for StrEnum) rather than displaying the symbolic name
I know I said in the other thread that the last change was problematic, but I think you have a strong argument that the current behaviour breaks the "drop in replacement" promise of those concrete types.
So my vote would be to revert both the repr() and str() changes on IntEnum (et al) for 3.10, and then switch to the new str() approach for those classes in 3.11.
Cheers,
Nick.
Nick Coghlan
Jun 28, 2021, 10:11:28 AM6/28/21
to Ethan Furman, Python-Dev
On Mon, 28 Jun 2021, 11:54 pm Nick Coghlan, <ncog...@gmail.com> wrote:
So my vote would be to revert both the repr() and str() changes on IntEnum (et al) for 3.10, and then switch to the new str() approach for those classes in 3.11.
Sorry, I missed including the requested rationale:
* if the str() change is postponed to 3.11, then existing code will upgrade seamlessly to 3.10, and there is a full year for notification of the coming change to filter out. The cost is another year of the "drop in replacement" promise not being kept
* code that was already updated to cope with the module name disappearing from str presumably still handles it being present for compatibility with older versions, so shouldn't be hard to adapt to the prefix coming back
(My opinion would likely be different if we were earlier in the beta cycle).
Cheers,
Nick.
Ethan Furman
Jun 28, 2021, 10:43:31 AM6/28/21
to Python-Dev
On 6/28/21 6:54 AM, Nick Coghlan wrote:
> * Enum repr() changing back to the historical behaviour, unless you opt in to the
> new behaviour with the global enum decorator: definite +1 here
Question for Nick: this behavior is currently in place for stdlib enumerations, and has been since beta 0; are you
> * Enum repr() changing back to the historical behaviour, unless you opt in to the
> new behaviour with the global enum decorator: definite +1 here
saying this change should also be put off to 3.11 ?
--
~Ethan~
_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/AVNMYCH5PZ7CI6COY6NEZKSXO4VV56CA/
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Jelle Zijlstra
Jun 28, 2021, 11:03:02 AM6/28/21
to Ethan Furman, Python-Dev
Great!
Proposed 3.10 Change (for 3.10.0 beta 4)
----------------------------------------
In order to improve the drop-in replacement use-case for IntEnum and IntFlag, I would like to change the str() for those
two to be simply the value of the member, as if it was still a plain integer and not part of an enumeration. At that
point the only obvious difference would be the repr(), which I think is the most important and useful change as that is
what (should) show up in log files, the REPL, etc.
This would also make IntEnum and IntFlag consistent with StrEnum as, for other reasons, StrEnum member str()s are the
values of the members rather than the names of the members. Note also that this change would not affect user created
enumerations that mixed in int on their own.
The Question
------------
With all the changes currently happening to the str()s and repr()s of the various enums, what are the good reasons to
not make this final change now, and instead have one more change in 3.11?
Just to make myself less confused, the behavior of str(Color.RED) (where Color is an IntEnum) is:
- 3.9 and below: "Color.RED"
- Current 3.10 branch: "RED"
- Proposed change: "256"
I agree that we shouldn't change the behavior in 3.10 and again in 3.11, and also that the proposed behavior is the best long-term option.
If it's too late for completely new behavior in 3.10, we could instead revert 3.10 to use the 3.9 behavior and make the change only in 3.11.
--
~Ethan~
Summary of Enum str()s and repr()s by category
| | repr() | str() | format()
Somewhat unrelatedly, I don't like that str() and format() may return different results for enums; I think enums are the only type I commonly encounter for which this is the case. If possible, any change we make should move us closer to having both return the same result.
| stdlib global | re.ASCII | re.ASCII | 256
| Enum | <Color.RED: 1> | Color.RED | RED
| IntEnum | <Color.RED: 1> | 1 | 1
| int, Enum | <Color.RED: 1> | Color.RED | 1 (will be RED in 3.12 due to back-compatibility
| | | constraints which should affect very few people as
| | | as they are probably using IntEnum instead of mixing
| | | in int themselves -- comments about this likelihood also
| | | appreciated)
It does happen sometimes: https://grep.app/search?q=%28int%2C%20Enum%29&case=true&filter[lang][0]=Python .
Barry Warsaw
Jun 29, 2021, 12:57:31 PM6/29/21
to Ethan Furman, Python-Dev
The Steering Council discussed this topic at our meeting yesterday. We have some discomfort about the changes to Enum’s str and repr in Python 3.10, both in the specific changes and in the way the changes were decided on. No knock on Ethan or others who participated in those decisions, it’s just that to us, the changes cumulatively feel like they need a more formal, centralized discussion to understand all the issues in detail.
While we’re stopping short of requiring it, the Steering Council strongly suggests that for Python 3.10, the changes in Enum’s str and repr be reverted back to the Python 3.9 behavior, and that a PEP be written for Python 3.11.
Cheers,
-Barry (on behalf of the Steering Council)
While we’re stopping short of requiring it, the Steering Council strongly suggests that for Python 3.10, the changes in Enum’s str and repr be reverted back to the Python 3.9 behavior, and that a PEP be written for Python 3.11.
Cheers,
-Barry (on behalf of the Steering Council)
Nick Coghlan
Jun 30, 2021, 10:30:33 AM6/30/21
to Ethan Furman, Python-Dev
On Tue, 29 Jun 2021, 12:45 am Ethan Furman, <et...@stoneleaf.us> wrote:
On 6/28/21 6:54 AM, Nick Coghlan wrote:
> * Enum repr() changing back to the historical behaviour, unless you opt in to the
> new behaviour with the global enum decorator: definite +1 here
Question for Nick: this behavior is currently in place for stdlib enumerations, and has been since beta 0; are you
saying this change should also be put off to 3.11 ?
I'm not sure what you mean by "this change". I don't think the change to make repr() less informative for IntEnum instances should be done in any version.
I think repr() should continue to show both the symbolic name and the associated numeric value, as it did in 3.9 (I'm not worried about the exact spelling, though, I just don't want to have to find and look up a translation table when debugging).
Did I misunderstand the proposed change for the next beta?
Cheers,
Nick.
Ethan Furman
Jul 6, 2021, 10:01:50 PM7/6/21
to Python-Dev
On 6/29/21 9:50 AM, Barry Warsaw wrote:
> the Steering Council strongly suggests that for Python 3.10, the changes in Enum’s
> str and repr be reverted back to the Python 3.9 behavior, and that a PEP be written
> for Python 3.11.
I was unable to revert just the str and repr changes in the time available as many of them were integral to fixes and
> the Steering Council strongly suggests that for Python 3.10, the changes in Enum’s
> str and repr be reverted back to the Python 3.9 behavior, and that a PEP be written
> for Python 3.11.
improvements made to Flag. As a result the enum in 3.10 will be the same as 3.9 (complete module reversion).
A PEP will be forthcoming.
My apologies and sympathies to those that contributed to the 3.10 enum module -- look for those changes in 3.11.
--
~Ethan~
_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/LSTMFAPSPD3BGZ4D6HQFODXZVB3PLYKF/Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Miro Hrončok
Jul 11, 2021, 7:03:06 PM7/11/21
to pytho...@python.org
On 07. 07. 21 3:58, Ethan Furman wrote:
> On 6/29/21 9:50 AM, Barry Warsaw wrote:
>
> > the Steering Council strongly suggests that for Python 3.10, the changes in
> Enum’s
> > str and repr be reverted back to the Python 3.9 behavior, and that a PEP be
> written
> > for Python 3.11.
>
> I was unable to revert just the str and repr changes in the time available as
> many of them were integral to fixes and improvements made to Flag. As a result
> the enum in 3.10 will be the same as 3.9 (complete module reversion).
Hello Ethan.
> On 6/29/21 9:50 AM, Barry Warsaw wrote:
>
> > the Steering Council strongly suggests that for Python 3.10, the changes in
> Enum’s
> > str and repr be reverted back to the Python 3.9 behavior, and that a PEP be
> written
> > for Python 3.11.
>
> I was unable to revert just the str and repr changes in the time available as
> many of them were integral to fixes and improvements made to Flag. As a result
> the enum in 3.10 will be the same as 3.9 (complete module reversion).
I see the revert was merged with the "skip news" label. The changelog now
mentions the original change, but not the revert.
Is it possible to mention this retroactively in the 3.10.0b4 changelog, or is
that frozen now? I think it's important to document this for maintainers of
codebases that already adapted their code to the new behavior with an if Python
version >= 3.10 conditional.
Thanks for considering,
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Ethan Furman
Jul 12, 2021, 11:24:09 PM7/12/21
to pytho...@python.org, Miro Hrončok
On 7/11/21 4:00 PM, Miro Hrončok wrote:
> On 07. 07. 21 3:58, Ethan Furman wrote:
>> I was unable to revert just the str and repr changes in the time available as many of them were
>> integral to fixes and improvements made to Flag. As a result the enum in 3.10 will be the same
>> as 3.9 (complete module reversion).
>
> On 07. 07. 21 3:58, Ethan Furman wrote:
>> I was unable to revert just the str and repr changes in the time available as many of them were
>> integral to fixes and improvements made to Flag. As a result the enum in 3.10 will be the same
>> as 3.9 (complete module reversion).
>
> I see the revert was merged with the "skip news" label. The changelog now mentions the original
> change, but not the revert.
>
> Is it possible to mention this retroactively in the 3.10.0b4 changelog, or is that frozen now?
> I think it's important to document this for maintainers of codebases that already adapted their
> code to the new behavior with an if Python version >= 3.10 conditional.
Thanks for bringing that up, a new news entry has been added.
> change, but not the revert.
>
> Is it possible to mention this retroactively in the 3.10.0b4 changelog, or is that frozen now?
> I think it's important to document this for maintainers of codebases that already adapted their
> code to the new behavior with an if Python version >= 3.10 conditional.
--
~Ethan~
_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/RXFYYX74WVI5ALUNJUL645EANUVFNU6G/Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Miro Hrončok
Jul 13, 2021, 5:29:23 AM7/13/21
to Ethan Furman, pytho...@python.org
On 13. 07. 21 5:21, Ethan Furman wrote:
> On 7/11/21 4:00 PM, Miro Hrončok wrote:
> > On 07. 07. 21 3:58, Ethan Furman wrote:
>
> >> I was unable to revert just the str and repr changes in the time available
> as many of them were
> >> integral to fixes and improvements made to Flag. As a result the enum in
> 3.10 will be the same
> >> as 3.9 (complete module reversion).
> >
> > I see the revert was merged with the "skip news" label. The changelog now
> mentions the original
> > change, but not the revert.
> >
> > Is it possible to mention this retroactively in the 3.10.0b4 changelog, or
> is that frozen now?
> > I think it's important to document this for maintainers of codebases that
> already adapted their
> > code to the new behavior with an if Python version >= 3.10 conditional.
>
> Thanks for bringing that up, a new news entry has been added.
Thank you, Ethan.
> On 7/11/21 4:00 PM, Miro Hrončok wrote:
> > On 07. 07. 21 3:58, Ethan Furman wrote:
>
> >> I was unable to revert just the str and repr changes in the time available
> as many of them were
> >> integral to fixes and improvements made to Flag. As a result the enum in
> 3.10 will be the same
> >> as 3.9 (complete module reversion).
> >
> > I see the revert was merged with the "skip news" label. The changelog now
> mentions the original
> > change, but not the revert.
> >
> > Is it possible to mention this retroactively in the 3.10.0b4 changelog, or
> is that frozen now?
> > I think it's important to document this for maintainers of codebases that
> already adapted their
> > code to the new behavior with an if Python version >= 3.10 conditional.
>
> Thanks for bringing that up, a new news entry has been added.
_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/XNXJILZGHGSA3LAZSPP67GRNYDDUTCIZ/Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Reply all
Reply to author
Forward
0 new messages