[Python-Dev] PEP 677 (Callable Type Syntax): Rejection notice.

[Gregory P. Smith]

Thank you PEP authors for producing a very well written and laid out PEP. That made it easy to understand the proposal, rationale, and alternatives considered. We agree that the existing typing.Callable syntax, while capable, is less than ideal for humans. That said, we deliberated last week and ultimately decided to reject PEP-677 Callable Type Syntax.

Why? Several considerations led us here:

1. We feel we need to be cautious when introducing new syntax. Our new parser presents understandably exciting opportunities but we don’t want its existence to mean we add new syntax easily. A feature for use only in a fraction of type annotations, not something every Python user uses, did not feel like a strong enough reason to justify the complexity needed to parse this new syntax and provide meaningful error messages. Not only code complexity, humans are also parsers that must look forwards and backwards.

2. While the current Callable[x, y] syntax is not loved, it does work. This PEP isn’t enabling authors to express anything they cannot already. The PEP explicitly chose be conservative and not allow more syntax to express features going beyond what Callable supports. We applaud that decision, starting simple is good. But we can imagine a future where the syntax would desire to be expanded upon.

3. In line with past SC guidance, we acknowledge challenges when syntax desires do not align between typing and Python itself. Each time we add syntax solely for typing it shifts us further in the direction of typing being its own mini-language so we aim to tread lightly in what gets added here. Adopting PEP 677 would lock us into its syntax forever, potentially preventing other future syntax directions.

4. We did not like the visual and cognitive consequence of multiple `->` tokens in a def. Especially when code is not formatted nicely. Though we admit the correlation between Python typing and formatter users is high.

## Recommendation for future syntax enhancements:

When proposing a syntax change, low complexity is better. While not always possible, it’s ideal if it could still be described using our old <=3.8 parser. It is important to have in mind that adding syntax that only our modern PEG parser can handle could lead to greater cognitive load and external tooling implementation costs.

This should not be read as a ban on PEG-only syntax, we just think it should be used for broadly applicable features or else be relatively unintrusive.

Thanks,
The 3.11 Python Steering Council

[Shantanu Jain]

Hello!

Thanks for the decision, the points raised mostly make sense to me. However, I find myself and a few others are a little confused by point 2. I can read it as saying the following perhaps slightly contradictory things:

"It's good that PEP 677 is conservative and sticks to things Callable can do"

"PEP 677 isn't necessary, since Callable can do everything currently proposed"

"PEP 677 could be a slippery slope for further syntax expansions that can do things Callable cannot"

Would the case for new syntax have been stronger if it was proposing something Callable couldn't do? E.g., is the concern something like "The cost of new syntax is best paid by expanding the realm of what is expressible. While we see how PEP 677 could lead to such expansion in the future, the merits of future expansion are currently uncertain and the current proposal is too costly discussed on its own merits"?

Or is the concern forward compatibility in the eventuality of further syntax expansions? PEP 677 did discuss "extended syntax", which the proposed syntax would be forward compatible with. https://www.python.org/dev/peps/pep-0677/#extended-syntax-supporting-named-and-optional-arguments

Or something else entirely! Would appreciate any clarity :-)

Thank you!

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/NHCLHCU2XCWTBGF732WESMN42YYVKOXB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Gregory P. Smith]

On Thu, Feb 10, 2022 at 3:32 AM Shantanu Jain <haunts...@gmail.com> wrote:

Hello!

Thanks for the decision, the points raised mostly make sense to me. However, I find myself and a few others are a little confused by point 2. I can read it as saying the following perhaps slightly contradictory things:

"It's good that PEP 677 is conservative and sticks to things Callable can do"

"PEP 677 isn't necessary, since Callable can do everything currently proposed"

"PEP 677 could be a slippery slope for further syntax expansions that can do things Callable cannot"

Would the case for new syntax have been stronger if it was proposing something Callable couldn't do? E.g., is the concern something like "The cost of new syntax is best paid by expanding the realm of what is expressible. While we see how PEP 677 could lead to such expansion in the future, the merits of future expansion are currently uncertain and the current proposal is too costly discussed on its own merits"?

Or is the concern forward compatibility in the eventuality of further syntax expansions? PEP 677 did discuss "extended syntax", which the proposed syntax would be forward compatible with. https://www.python.org/dev/peps/pep-0677/#extended-syntax-supporting-named-and-optional-arguments

Or something else entirely! Would appreciate any clarity :-)

At least we're as consistent as the zen of python itself? ;)  We collectively struggled with how to word that one. Sorry if it is confusing.

From my individual perspective, agreeing with point 2 came down to a belief that if we accepted the simple syntax, a use would arise in the future where it'd be desirable to gain something more complicated. Even if only "3%" of callable annotations use those today. So we expected to wind up in future releases with a proposal to extend the 677 callable syntax to support parsing a more complete def. Good design in that it appears technically feasible, but perceived bad in that the decision then would be to go forward into an even more complicated, even harder to parse, world or to back off and go another direction entirely which would leave 677 callable shorthand as a partial expressive dead end. Think of it as attempting not to accumulate too many ways to do the same thing in the long term.

If the 677 proposed syntax had been expanded to include more def features instead of being conservative I think it would've been an even easier rejection for many of us as that would've triggered more "point 4" feelings of visual clutter and cognitive load.

-gps