Best practices for importing pure python style modules

[Yun]

Hello again,

I'm back with some more questions about pure python mode, this time around importing and referencing @cython.cclass-style cython modules.

General idea I'm trying to understand:

If I want to use @cython.cclass in .py modules, what's the best practice (or examples I can look at) to properly reference & call them in both the python interpreted and compiled cython modes?

1. Are conditional imports needed everywhere they are referenced? ie,

if cython.compiled:

    from cython.cimports.cython_decorated_module import Foo

else:

    from cython_decorated_module import Foo

2. Is there a similar option for .py files like cimport_from_pyx

This option allows for compiling *.pyx without needing to define a corresponding *.pxd; is there a way to do this with @cython.cclass *.py files, or do I need to define a .pxd for them manually?

(And just to confirm, I definitely need a .pxd file that corresponds to any @cython.cfunc that I define?)

Thanks again,

Yun

[Stefan Behnel]

Hi,

Yun schrieb am 27.04.22 um 06:42:

> I'm back with some more questions about pure python mode, this time around
> importing and referencing @cython.cclass-style cython modules.
>

> *General idea I'm trying to understand:*

> If I want to use @cython.cclass in .py modules, what's the best practice
> (or examples I can look at) to properly reference & call them in both the
> python interpreted and compiled cython modes?
>

> *1. Are conditional imports needed everywhere they are referenced?* ie,
> if cython.compiled:
> from cython.cimports.cython_decorated_module <http://module1.py> import
> Foo
> else:
> from cython_decorated_module <http://module1.py> import Foo

Not sure what exactly you are asking here. If you want to use them from
within the same module, then you do not need anything else. If you mean to
use them (specifically, their Cython-/C-level features) from another
module, then you need to declare them in a .pxd file, so that other modules
can cimport from it. That cimport can either happen in the module's own
.pxd file (if you want to add one) or via a "cython.cimports.*" import in a
.py module.

We should make sure that this is properly explained in the documentation
about sharing declarations across modules.


> *2. Is there a similar option for .py files like cimport_from_pyx*

> This option allows for compiling *.pyx without needing to define a
> corresponding *.pxd; is there a way to do this with @cython.cclass *.py
> files, or do I need to define a .pxd for them manually?
> (And just to confirm, I definitely need a .pxd file that corresponds to any
> @cython.cfunc that I define?)

There is no "cimport_from_py" option, currently. Might be worth adding,
given that we do have "cimport_from_pyx". It'd be a bit more costly at
build time, given that it not only has to parse an entire .pyx (or .py)
file but also apply all type annotations. Parsing .pxd files tends to be
quicker since they contain little (or no) actual implementation code. But
I'm sure there are people out there who would be happy to pay a bit of
build time overhead for the additional comfort of not having to write .pxd
files.

Stefan

[Yun]

Thanks for taking the time to explain the how to reference cython modules from other ones. Here are some follow ups:

Re: Autogeneration of .pxd for cython-decorated.py

After some testing, it looks like "cimport_from_pyx" currently can already handle the cclasses and cfuncs from pure python mode: I just have to rename the file from cython-decorated.py to cython-decorated.pyx and it seems to go ahead and creates an implicit .pxd. 

Until something like "cimport_from_py" exists, should I just rename all my cython-decorated.py into cython-decorated.pyx at build time so I can get this behavior? 

Alternatively: .pxd with pure python style decorators

I tried to manually create a .pxd for cython-decorated.py, but I think the .pxd requires matching signatures, and doesn't seem to have support for the @cython decorators. Is there a way create .pxds with pure python style syntax to correspond to a cython-decorated.py?

Re: Is the conditional import the only way to get dual-functionality

If a cclass_a.py wants to call another cclass_b.py, and have it work in both the python interpreter and cythonized modes, is the conditional import&cimport the only way to get two modules to see each other in both modes?

My hope is that there's a more streamlined way than conditional importing because I'd have to put this on every module that wants to invoke another cythonized module in both modes, and the cimport isn't valid python and doesn't generally benefit from IDE helpers.

Thanks again for improving my understanding of Cython!

-yun