How to import numpy types in cython pure python mode

[Serhiy Yevtushenko]

Hi,

could someone clarify, whether there is a proper way to import numpy data types in pure python mode?

I'm trying to speed up some python code, and would like to keep it, if possible, in py file instead of .pyx in order to be able to use linters, doc and type checkers on python code.

I have found information, how one could use cython annotations for declaring types of local variables. However, my code is using as well numpy np.float64_t, np.uint_64 and np.ndarray variables. When one uses pyx file, they are getting imported using cimport numpy as np. But I was not able to find a proper way to import this type in pure python mode.

Could you please clarify, whether such functionality is present in cython and what is the proper way of using it in pure python mode?

Many thanks in advance

Serhiy Yevtushenko

[Stefan Behnel]

Hi,

Thanks for bringing this up. I think we should make the numpy module available as cython.cimports.numpy in Python code.

This should already work for compiled code (in Cython 3), but that change would allow the code to work also in Python, to a certain extent.

I didn't do so initially because it takes a while to import NumPy, and that shouldn't happen whenever importing the special cython module from Python. We should look into ways to keep the import lazy. Maybe with an import hook.

Can't say if there are any differences in the potentially compatible part of the API of both modules. If there's anything to improve (e.g. by defining alias names), we should do that. (As in, let NumPy do it on their side, since they maintain the C-API declarations for NumPy in Cython.

Stefan

[Casper van Elteren]

Not sure I understand this question. The types in question are available from python directly no? https://numpy.org/doc/stable/reference/arrays.scalars.html

```python

import numpy as np; print(np.int_, np.uintp) # etc
```

[David Menéndez Hurtado]

On Mon, 28 Jun 2021, 2:48 pm Stefan Behnel, <stef...@behnel.de> wrote:

Hi,

Am 28. Juni 2021 14:10:03 MESZ schrieb Serhiy Yevtushenko:
>could someone clarify, whether there is a proper way to import numpy
>data
>types in pure python mode?
>
>I'm trying to speed up some python code, and would like to keep it, if
>possible, in py file instead of .pyx in order to be able to use
>linters,
>doc and type checkers on python code.
>
>I have found information, how one could use cython annotations for
>declaring types of local variables. However, my code is using as well
>numpy
>np.float64_t, np.uint_64 and np.ndarray variables. When one uses pyx
>file,
>they are getting imported using cimport numpy as np. But I was not able
>to
>find a proper way to import this type in pure python mode.
>
>Could you please clarify, whether such functionality is present in
>cython
>and what is the proper way of using it in pure python mode?

Thanks for bringing this up. I think we should make the numpy module available as cython.cimports.numpy in Python code.

This should already work for compiled code (in Cython 3), but that change would allow the code to work also in Python, to a certain extent.

I didn't do so initially because it takes a while to import NumPy, and that shouldn't happen whenever importing the special cython module from Python. We should look into ways to keep the import lazy. Maybe with an import hook.

Pyforest does lazy imports through this code: https://github.com/8080labs/pyforest/blob/master/src/pyforest/_importable.py

[Serhiy Yevtushenko]

Hi

No. In cython there are two usage scenarios: 

using types in python code (which is covered by your example) and one, when one declares types of in cdef declarations.

The types in question are coming from cimport, and a used to generate C code.

so, in cython code it would look like:

cimport numpy as np

...

cdef np.uint64_t distance

When using pure python mode, it is possible to replace cdef with annotations, but one still have to declare (C-types) types of variables 

So, the previous declaration can look as:

@cython.locals(distance=np.uint64_t)

For C standard library types one could use:

from cython.cimports.libc import math

But it I try to use:

from cython.cimports.numpy import uint64_t

and use it in locals

@cython.locals(distance=uint64_t)

 then cython gives following compilation error:

"Not a type"

Therefore, I was not able to find the way to access variable C-Types for numpy in the pure python mode, which has given rise to the question in the thread.

Kind Regards

Serhiy

вторник, 29 июня 2021 г. в 14:50:02 UTC+2, casperva...@gmail.com:

[Casper van Elteren]

Apologies for not understanding but this example works for me

```python

# file pure.py

import numpy as np
def add_np_ints(x: np.int_, y: np.int_) -> np.int_:
    return x + y
```

```cython

#file pure.pxd
cimport numpy as np
cpdef np.int_ add_np_ints(np.uintp, np.uintp)
```

Or any other type I provided in the link would convert it in pure python mode.

What am I missing here? More infor can be found https://cython.readthedocs.io/en/latest/src/tutorial/pure.html

--

---
You received this message because you are subscribed to a topic in the Google Groups "cython-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cython-users/KXZti9XtR_w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cython-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/49d7c5d3-e987-473d-bd24-f9d3a4c941abn%40googlegroups.com.

--

• Checkout my website for contact info and current projects!
  

[Serhiy Yevtushenko]

Hi, Casper,

it's quite possible that I'm missing something, as I'm quite new to cython.

Could you please clarify following:

My attempt was concerntrated on using @locals annotation in python file in the pure python mode, which has not worked out due to :

- cimport not being available in pure python mode

- getting compiler error, when trying to import cython through cython.cimports.numpy

If I understand your suggestion, then you suggest the following:

1) In addition to python code in the pure python mode create additional .pxd file with type information

2) Performs @locals declaration and cimport there

And in this way then numpy types should be available and code should compile.

Is my understanding correct?

Thanks in Advance

Serhiy

ср, 30 черв. 2021 о 16:00 Casper van Elteren <casperva...@gmail.com> пише:

To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/CAOz1F_v88zHrMh5sN627bLtZD08%3DC9%2BUa9%2BiuaD56PU0mTLEEQ%40mail.gmail.com.

[Casper van Elteren]

Hi Serhiy,

Well the pxd file is one way to achieve it. The docs list several other ways of doing it (including type hinting, pxd files, decorators, decorators with pxd files etc).
The recommended way, however, is just using a native pyx file. I agree that the cython-mode needs some brushing up and is not really working well in my editor (emacs). It is on my backlog for a while now and not sure if I ever get around looking into this.
Anyhoozle, to answer your questions

1) yes pxd files can be used to write in how cython should interpret your python code

2) locals can be used either separate or in conjuction with this pxd file (see docs for more).

In your original code the cimport was  wrong (again see docs). Cimport keyword is not directly available in pure python mode; for that you would need a pyx file. You can, however, import numpy via the provided wrapper

`from cython.cimport import numpy as np` (as I did). From the docs it states

>The special cython.cimports package name gives access to cimports in code that uses Python syntax. Note that this does not mean that C libraries become available to Python code. It only means that you can tell Cython what cimports you want to use, without requiring special syntax. Running such code in plain Python will fail.

Hope this helps!