[Python-Dev] Presenting PEP 695: Type Parameter Syntax

[Guido van Rossum]

After several rounds of debate on typing-sig, I'd like to request feedback on PEP 695: https://peps.python.org/pep-0695/

I am sponsoring this PEP, which was written by Eric Traut. The PEP attempts to solve the problem that defining generic classes, functions and type aliases currently is lacking dedicated syntax, instead using the cumbersome `T = TypeVar("T", ...)` notation to create global variables that serve as type variables.

As a personal historical note, I should mention that over 22 years ago I already pondered type parameters. In an old document that I saved I found the following code snippet:

```

def f<T> (a: T) -> T: ...

```

which is eerily close to the proposal in this PEP, except that the PEP uses square brackets:

```

def f[T](a: T) -> T: ...

```

It's been a long and circuitous road!

I am not quoting the entire PEP here, please follow the link: https://peps.python.org/pep-0695/

--

--Guido van Rossum (python.org/~guido)

[Petr Viktorin]

> https://peps.python.org/pep-0695/ <https://peps.python.org/pep-0695/>
>

A beautifully written PEP, thank you!
An extra thank you for clearly specifying compile-/run-time vs. type
checker behavior!


In “Type Parameter Declarations” it would be nice to better specify why
this example is an error:
```
class ClassA[__T, _ClassA__S]:
__T = 0 # OK
__S = 0 # Syntax Error (because mangled name is _ClassA__S)
```
It's specified later in the Compiler changes section (“An active type
variable symbol cannot be used for other purposes”), but it would be
nice to mention it up here too – perhaps replace the specific “Type
parameters for a generic function cannot overlap the name of a function
parameter.”


I'm not a fan of a third overload of `type`, after the “get type of”
function and “default metatype” class.
Would `typevar` not work?
(The addition of a soft keyword itself is a heavy change, though I'll
let grammar experts weigh in on that.)

I wonder if we should give some thought to other cases where a name is
repeated – for example, a hypothetical:
namedtuple Point = ("x", "y")
replacing:
Point = namedtuple("Point", ("x", "y"))
Is the proposed `type` potentially setting a precedent? A good one?


`TypeVar` taking `covariant`, `contravariant` and `autovariance` looks
inconsistent to an outsider. Why is it not `autovariant`?


The Rejected ideas mention “various syntactic options for specifying
type parameters that preceded def and class statements” rejected because
scoping is less clear and doesn't work well with decorators. I wonder if
decorator-like syntax itself was considered, e.g. something like:
```
@with type S
@with type T
@dec(Foo[S])
class ClassA: ...
```


And finally, I need to ask...
The reference implementation doesn't include documentation. Is there any
plan to document this feature outside this enhancement proposal?

If not, what needs to happen to get this documented?
_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/U3TBVMXBXDCLTW7AAF5RAYKUMHKOYPBW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Jelle Zijlstra]

That piece of syntax is for type *aliases*, not type *variables*, which are a different concept. Using "typevar" here would be quite confusing. We could use something like "alias" or "typealias", but I think "type" is the most intuitive term, and it matches other languages like TypeScript.

 

(The addition of a soft keyword itself is a heavy change, though I'll
let grammar experts weigh in on that.)

I wonder if we should give some thought to other cases where a name is
repeated – for example, a hypothetical:
     namedtuple Point = ("x", "y")
replacing:
     Point = namedtuple("Point", ("x", "y"))
Is the proposed `type` potentially setting a precedent? A good one?


`TypeVar` taking `covariant`, `contravariant` and `autovariance` looks
inconsistent to an outsider. Why is it not `autovariant`?


The Rejected ideas mention “various syntactic options for specifying
type parameters that preceded def and class statements” rejected because
scoping is less clear and doesn't work well with decorators. I wonder if
decorator-like syntax itself was considered, e.g. something like:
```
@with type S
@with type T
@dec(Foo[S])
class ClassA: ...
```

We did consider variations of that. Pure decorator syntax (like your "@dec" line) wouldn't allow us to get the scoping right, since decorators can't define new names. (Unless you use a walrus operator, but that wouldn't meet the goal of providing cleaner syntax.)

We also considered some ideas similar to your "@with type". It can work, but it feels more verbose than the current proposal, and it's not in line with what most other languages do.

 

And finally, I need to ask...
The reference implementation doesn't include documentation. Is there any
plan to document this feature outside this enhancement proposal?

If not, what needs to happen to get this documented?

I'm sure we'll provide detailed documentation if and when the PEP is accepted; full documentation seems a bit much to ask for in an early prototype.

[o.jacob...@gmail.com]

Hi, I like this PEP but I couldn't find the motivation for using angle brackets over square braces (brackets?). The survey in Appendix A is great but lacks any conclusions. From that survey alone I would assume that angle brackets would have been chosen over square braces, given that they are the most common option and appear in (afaik) the more popular languages in that list. I think the PEP should add a section about the choice of syntax in the rejected section, which can be expanded upon in Appendix A.

If you can't tell I'm in favor of angle brackets, I think the examples given in the PEP look a bit messy with so many parentheses and square braces in close proximity. Using angle brackets would make the distinction between typevars and function parameters clearer.

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/2LEHJKQGRHCHGQUFXUU3DTYKKDISNPFN/

[Guido van Rossum]

Yeah, we all would have liked angle brackets, but there would be problems with breaking lines between those. E.g.

def foo<

    T: str,

    S: int

> (arg1: T, arg2: S) -> tuple[T, S]:

    ...

cannot be parsed because the lexer doesn't treat angle brackets as matching pairs.

In addition, we already use square brackets for *using* generics (e.g. list[int]), and most surveyed languages use the same type of brackets in declarations and uses.

--

--Guido van Rossum (python.org/~guido)

Pronouns: he/him (why is my pronoun here?)

[Terry Reedy]

On 7/14/2022 6:16 PM, Guido van Rossum wrote:

> In addition, we already use square brackets for *using* generics (e.g.
> list[int]), and most surveyed languages use the same type of brackets in
> declarations and uses.

I do not yet use annotations, but knowing about 'list[int]', etc, I
could immediately read and understand the new examples.

> On Thu, Jul 14, 2022 at 1:10 PM <o.jacob...@gmail.com
> <mailto:o.jacob...@gmail.com>> wrote:
>
> Hi, I like this PEP but I couldn't find the motivation for using
> angle brackets over square braces (brackets?).

I presume you meant the reverse, for proposing [] over <>. I agree that
giving the motivation above (and the one deleted) would be good idea.

--
Terry Jan Reedy

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/JO6KUMZORJ5GCF5WWMNAMNYWBUUFFMGY/

[Paul Ganssle]

I actually really like some variation on `@with type S`, or some other variation that has a keyword, because that makes it much easier for someone newly encountering one of these syntax constructs to search to figure out what it does. If you didn't already know what the square brackets did, how would you try and find out? "what do square brackets mean in Python" would probably turn up a bunch of stuff about element access, and maybe something about type generic parameters.

By contrast, `@with type S` is kinda self-explanatory, and even if it's not, 'What does "with type" mean in Python' will almost certainly turn up meaningful results.

An additional benefit is that I find some of these examples to be a bit visually cluttered with all the syntax:

def func1[T](a: T) -> T: ...  # OK
class ClassA[S, T](Protocol): ... # OK

Which would look less cluttered with a prefix clause:

@with type T
def func1(a: T) -> T: ...  # OK

@with type S
@with type T

class ClassA(Protocol): ... # OK

Of the objections to this concept in the PEP, the most obvious one to me was that the scoping rules were less clear, but it is not entirely clear to me why the scope of the prefix clause couldn't be extended to include class / function decorators that follow the prefix clause; the choice of scoping seems like it was a defensible but mostly arbitrary one. I think as long as the new prefix clause is something that was syntactically forbidden prior to the introduction of PEP 695 (e.g. `@with type` or `[typevar: S]` or whatever), it will be relatively clear that this is not a normal decorator, and so "the scoping and time of execution doesn't match decorators" doesn't seem like a major concern to me relative to the benefits of using a more searchable syntax.

[Patrick Reader]

On 13/07/2022 14:14, o.jacob...@gmail.com wrote:
> Hi, I like this PEP but I couldn't find the motivation for using angle brackets over square braces (brackets?). The survey in Appendix A is great but lacks any conclusions. From that survey alone I would assume that angle brackets would have been chosen over square braces, given that they are the most common option and appear in (afaik) the more popular languages in that list. I think the PEP should add a section about the choice of syntax in the rejected section, which can be expanded upon in Appendix A.
>
> If you can't tell I'm in favor of angle brackets, I think the examples given in the PEP look a bit messy with so many parentheses and square braces in close proximity. Using angle brackets would make the distinction between typevars and function parameters clearer.

Another reason that square brackets should be preferred over angle
brackets is the difficulty in parsing:

list<list<int>>

is tokenised as list < list < int >> where the last token is a right
shift operator, so the parser has to know that sometimes >> is used when
two "angle bracket" groups are closed, instead of > >

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/CNIEGWWROWN4Y4IPVH72CDKCFO3OSHLY/

[Stephen J. Turnbull]

Paul Ganssle writes:

> If you didn't already know what the square brackets did, how would
> you try and find out?

First I'd look it up in Python Essential Reference (Hi, @dabeaz! it
won't be there, though ;-). Then I'd go to the Language Reference for
"def" and "class". And if that failed, then I'd go buy Barry Warsaw
lunch.

OK, not everybody has a personal relationship with senior core devs,
but is asking people to read the Language Reference really so bad?

> An additional benefit is that I find some of these examples to be a bit
> visually cluttered with all the syntax:
>
> def func1[T](a: T) -> T: ... # OK
> class ClassA[S, T](Protocol): ... # OK

Looks like the boomer version (square*) of C++ template variables. Of
course, people learn Python to escape from C++, so maybe that's not
persuasive.

* telling you how old I am without telling you how ooooold I am

> Which would look less cluttered with a prefix clause:
>
> @with type T def func1(a: T) -> T: ... # OK
> @with type S @with type T class ClassA(Protocol): ... # OK

For me, that's absolutely awful from a readability standpoint. Put
the "def" or "class" 10-20 characters in from the margin?

I guess "stacked" it's no less readable than any decorator, but I also
don't like overloading the well-defined decorator notation with magic.

@with type T
def func1(a: T) -> T: ... # OK

@with type S
@with type T
class ClassA(Protocol): ... # OK

A thought: would it be possible to actually make it a with statement?

with Typevar() as T:

def func1(a: T) -> T

Of course there might have to be magic in Typevar, but it would be far
more palatable to me than giving unary @ two kinds of magic.

IMO YMMV of course.

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/UTVCFIZ4CM3362JANAOBPEGJUTAWKSCI/