Pure python mode support for parameterized types

[Yun]

I'm trying to re-implement parts of my project using the new pure python style of cython and noticed that parameterized types compile differently between the decorator style and the cdef style. Can anyone let me know if I'm using it incorrectly? Thanks!

(I'm using python 3.10.4 and cython 3.0.0a10 in the examples.)

1) Main question

COMPILES: Using lower-case PEP-585 types with cdef

cdef list[str] list_return():

    return ["foo"]

DOESN'T COMPILE: "list[str] is not a type"

@cython.cfunc

def list_return() -> list[str]:

    return ["foo"]

Is there a reason why it doesn't work with the decorator style, or am I doing something wrong here? 

2) Side issue

From the doc: 

Support for the full range of annotations described by PEP-484 is not yet complete. Cython 3 currently understands the following features from the typing module:

• typed containers such as List[str], which is interpreted as list. The hint that the elements are of type str is currently ignored;

https://cython.readthedocs.io/en/latest/src/tutorial/pure.html?highlight=%22list%5Bstr%5D%22#typing-module

For some reason, typing.List has never worked for me, using either cpdefs or @cython.cfunc. For example:

DOESN'T COMPILE: 'List' is not a type identifier

from typing import List

cpdef List[str] list_return():

    return ["foo"]

This one isn't a huge deal since I'd rather use the PEP-585 style from (1) instead, but I figure I'd ask while I'm here; what is the correct way to use it?

3) Type alias support?

Potentially as a workaround for certain types, can I use aliased types (or explicit type alias PEP-613) inside a cdef function?

Is there some way to do something like:

list_type: TypeAlias = list[str]

@cython.cfunc

def list_return() -> list_type:

    return ["foo"]

4) Long term support?

In the long run, will more complex parameterized generic types be supported inside cdefs (and presumably also for @cython.cfunc)?

a) dict[str, list[int]] or dict[str, set[int]]

b) address: TypeAlias = dict[str, list[int]]

c) list[str | int]

Thanks for looking at all the questions; I'm trying to determine whether it's currently worth it to cythonize parts of my project using the newer style, so any ideas as to the kind of future support will very much be appreciated. 

[Stefan Behnel]

Hi,

thanks for bringing this up, and sorry for top-posting over your concrete
examples. Could you try the latest master branch? There were changes that
haven't been included in a release yet. Most of this should work now.

For type aliases, you can use "cython.typedef()".

Stefan



Yun schrieb am 22.04.22 um 03:57:

> I'm trying to re-implement parts of my project using the new pure python
> style of cython and noticed that parameterized types compile differently
> between the decorator style and the cdef style. Can anyone let me know if
> I'm using it incorrectly? Thanks!
> (I'm using python 3.10.4 and cython 3.0.0a10 in the examples.)
>

> *1) Main question*
> *COMPILES*: Using lower-case PEP-585 types with cdef

> cdef list[str] list_return():
> return ["foo"]
>

> *DOESN'T COMPILE*: "list[str] is not a type"

> @cython.cfunc
> def list_return() -> list[str]:
> return ["foo"]
>
> Is there a reason why it doesn't work with the decorator style, or am I
> doing something wrong here?
>

> *2) Side issue*

> From the doc:
> Support for the full range of annotations described by PEP-484 is not yet
> complete. Cython 3 currently understands the following features from
> the typing module:
>

> - typed containers such as List[str], which is interpreted as list. The

> hint that the elements are of type str is currently ignored;
>
> https://cython.readthedocs.io/en/latest/src/tutorial/pure.html?highlight=%22list%5Bstr%5D%22#typing-module
>
> For some reason, typing.List has never worked for me, using either cpdefs
> or @cython.cfunc. For example:
>

> *DOESN'T COMPILE:* 'List' is not a type identifier

> from typing import List
> cpdef List[str] list_return():
> return ["foo"]
>
> This one isn't a huge deal since I'd rather use the PEP-585 style from (1)
> instead, but I figure I'd ask while I'm here; what is the correct way to
> use it?
>

> *3) Type alias support?*

> Potentially as a workaround for certain types, can I use aliased types (or
> explicit type alias PEP-613) inside a cdef function?
>
> Is there some way to do something like:
>
> list_type: TypeAlias = list[str]
> @cython.cfunc
> def list_return() -> list_type:
> return ["foo"]
>

> *4) Long term support?*

[da-woods]

Hi,

To follow up in more detail:

>> *1) Main question*
>> *COMPILES*: Using lower-case PEP-585 types with cdef
>> cdef list[str] list_return():
>>      return ["foo"]

I'm actually surprised this works. I'm not sure what it'll deduce the
return type as

>>
>> *DOESN'T COMPILE*: "list[str] is not a type"
>> @cython.cfunc
>> def list_return() -> list[str]:
>>      return ["foo"]
>>
>> Is there a reason why it doesn't work with the decorator style, or am I
>> doing something wrong here?

We  haven't implemented subscripting list, tuple, dict, etc directly
yet. It probably isn't too hard but I wouldn't expect this to work on
the current branch.

>>
>> For some reason, typing.List has never worked for me, using either
>> cpdefs
>> or @cython.cfunc. For example:
>>
>> *DOESN'T COMPILE:* 'List' is not a type identifier
>> from typing import List
>> cpdef List[str] list_return():
>>      return ["foo"]

This should work with the current master build (but it hasn't yet made
it into a release). There's a bit of tidying still to (which I think is
why it hasn't made it into a release). The element type (`str`) isn't
currently used for anything (but no reason not to include it).

>>
>> list_type: TypeAlias = list[str]
>> @cython.cfunc
>> def list_return() -> list_type:
>>      return ["foo"]

We don't currently support this.

>>
>> *4) Long term support?*
>> In the long run, will more complex parameterized generic types be
>> supported
>> inside cdefs (and presumably also for @cython.cfunc)?
>>
>> a) dict[str, list[int]] or dict[str, set[int]]
>> b) address: TypeAlias = dict[str, list[int]]
>> c) list[str | int]

a) right now we don't do anything with the subscripted types (except for
tuples). `typing.Dict[str, typing.List[int]]` will work now (but is the
same as `dict` from Cython's point-of-view). We'd probably aim to
support the versions you've written (without the typing import) fairly
soon. I don't think we've decided what the long-term plan for using the
element types is...

b) Not sure. Probably...

c) Probably in future, but most likely only as `list`. We may well end
up using union types as fused types (but probably only in fairly limited
circumstances, like if the union type is used directly as a function
argument). We'd probably start with `typing.Union[...]` and then add the
newer `str | int` syntax later.

Hopefully that's helpful,

David

[Yun]

Thanks for the responses, they've helped quite a bit!

As a follow-up from the first example:

cdef list[str] list_return():

    return "foo"

               ^

Error: Cannot assign type 'unicode object' to 'list object[unicode object,ndim=1]'

Printing out the returned type from python also shows that it's a list:

def print_list():
    print(type(list_return())) # returns <class 'list'>

It seems like the list type is already being enforced (and not the str part of list[str], which is fine); is there potential for the @cython.cfunc style to get this same behavior?

Thanks again,

Yun