Declare global variables and class attributes in pure python mode

[Kevin Tewouda]

Hello everyone,

I'm learning cython and I want to use pure python mode to  speedup some of my projects. So, I wass trying different things and I discover that it is not that easy to declare and export global variables in the augmented pxd file despite the documentation that mentions the use of the cython.declare function.

Also I want to know if there is a way to specify class attribute in an augmented pxd file. According to this post on stackoverflow it is not really possible but I want to be sure so I ask it again here.

Here is my python file primes_py.py

from functools import partial

HOSTS = 3

def primes_python(nb_primes):
    p = []
    n = 2
    while len(p) < nb_primes:
        # Is n prime?
        for i in p:
            if n % i == 0:
                break

        # If no break occurred in the loop
        else:
            p.append(n)
        n += 1
    return p

Here is my pxd file primes_py.pxd

import cython

cython.declare(HOSTS=cython.int)

@cython.locals(p=cython.list, n=cython.int, i=cython.int)
cpdef list primes_python(int nb_primes)

my setup.py

from setuptools import setup, Extension
from Cython.Build import cythonize

prime_extension = Extension('primes_py', sources=['primes_py.py'], depends=['primes_py.pxd'])

setup(
    ext_modules=cythonize(['helloworld.pyx', 'primes_cy.py', prime_extension], language_level='3', annotate=True)
)

I'm using cython 0.29.12 and python 3.7.0

Also to resolve my issue with global variable, I try these different approaches without success:

cdef HOSTS
cdef int HOSTS
HOSTS = cython.declare(cython.int)

Every answer is welcome

Thanks

[Stefan Behnel]

Kevin Tewouda schrieb am 16.08.19 um 00:26:

> I'm learning cython and I want to use pure python mode to speedup some of
> my projects. So, I wass trying different things and I discover that it is
> not that easy to declare and export global variables in the augmented pxd
> file despite the documentation that mentions the use of the cython.declare

> <https://cython.readthedocs.io/en/latest/src/tutorial/pure.html#magic-attributes-within-the-pxd>
> function.

"cython.declare()" can be used in Cython modules, not in .pxd files. I
guess that could be clearer in the docs (PR welcome).

> Also I want to know if there is a way to specify class attribute in an
> augmented pxd file. According to this post

> <https://stackoverflow.com/questions/28749209/do-cython-extension-types-support-class-attributes> on

> stackoverflow it is not really possible but I want to be sure so I ask it
> again here.

Do it in the module code. Class attributes are more like code than declaration.

> Also to resolve my issue with global variable, I try these different
> approaches without success:
>
> cdef HOSTS
> cdef int HOSTS

These should work. Don't they? What error do you get?

Stefan

[Kevin Tewouda]

Hi Stefan, thank you for your reply

"cython.declare()" can be used in Cython modules, not in .pxd files. I 
guess that could be clearer in the docs (PR welcome). 

I suppose you mean cython or python files.. 

These should work. Don't they? What error do you get? 

When I declare

cdef int HOSTS

in a primes_py.pxd file, I expect that in another python module, I can do this

from primes_py import HOSTS

print(HOSTS)

but I have this error:

Traceback (most recent call last):
  File "C:/Users/rolla/.PyCharm2019.2/config/scratches/scratch.py", line 1, in <module>
    from primes_py import HOSTS
ImportError: cannot import name 'HOSTS' from 'primes_py' (D:\projets\python\cythoniser\primes_py.cp37-win32.pyd)

What do you think I'm doing wrong?

Best regards.

[Chris Barker]

On Mon, Aug 19, 2019 at 9:36 AM Kevin Tewouda <lewo...@gmail.com> wrote:

When I declare

cdef int HOSTS

in a primes_py.pxd file, I expect that in another python module, I can do this

from primes_py import HOSTS

print(HOSTS)

but I have this error:

Traceback (most recent call last):
  File "C:/Users/rolla/.PyCharm2019.2/config/scratches/scratch.py", line 1, in <module>
    from primes_py import HOSTS
ImportError: cannot import name 'HOSTS' from 'primes_py' (D:\projets\python\cythoniser\primes_py.cp37-win32.pyd)

What do you think I'm doing wrong?

the pxd file is used to tell *cython* what type the variable is. you still need to create the varibale (name) in primes.py:

HOSTS = 45 # or whatever value

-CHB

 

Best regards.

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/00ef89aa-8051-485e-b6e5-eed9f8592f51%40googlegroups.com.

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris....@noaa.gov

[Kevin Tewouda]

Hi Chris,

If you read my first message, this is already what I've done (it is set to 3) but it does not work.

Best regards.

Le lundi 19 août 2019 23:02:51 UTC+2, Chris Barker a écrit :

On Mon, Aug 19, 2019 at 9:36 AM Kevin Tewouda <lewo...@gmail.com> wrote:

When I declare

cdef int HOSTS

in a primes_py.pxd file, I expect that in another python module, I can do this

from primes_py import HOSTS

print(HOSTS)

but I have this error:

Traceback (most recent call last):
  File "C:/Users/rolla/.PyCharm2019.2/config/scratches/scratch.py", line 1, in <module>
    from primes_py import HOSTS
ImportError: cannot import name 'HOSTS' from 'primes_py' (D:\projets\python\cythoniser\primes_py.cp37-win32.pyd)

What do you think I'm doing wrong?

the pxd file is used to tell *cython* what type the variable is. you still need to create the varibale (name) in primes.py:

HOSTS = 45 # or whatever value

-CHB

 

Best regards.

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cython...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/00ef89aa-8051-485e-b6e5-eed9f8592f51%40googlegroups.com.

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris...@noaa.gov

[Kevin Tewouda]

Hi Chris,

If you read my first message, this is already what I've done.

Best regards.

To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/CALGmxELi8D7OBATueZg93DELyTLgC_jYS8asFa0j3ewE1Fw6Jg%40mail.gmail.com.

--

Tewouda T. R. Kevin

Ingénieur informatique options génie logiciel et réseaux informatiques à 3IL

Titulaire d'un diplôme post master en télécoms à Télécoms Paris Tech

Consultant informatique pour le compte d'Adneom

[Kevin Tewouda]

Hello everyone,

can somebody helps me to fix my issue related to variable export? As a reminder I define a variable in the.py file and its declaration in the.pxd file, you can read the first exchanges.

Best regards

Le jeudi 22 août 2019 06:28:24 UTC+2, Kevin Tewouda a écrit :

Hi Chris,

If you read my first message, this is already what I've done.

Best regards.

Le lun. 19 août 2019 à 23:02, 'Chris Barker' via cython-users <cython...@googlegroups.com> a écrit :

On Mon, Aug 19, 2019 at 9:36 AM Kevin Tewouda <lewo...@gmail.com> wrote:

When I declare

cdef int HOSTS

in a primes_py.pxd file, I expect that in another python module, I can do this

from primes_py import HOSTS

print(HOSTS)

but I have this error:

Traceback (most recent call last):
  File "C:/Users/rolla/.PyCharm2019.2/config/scratches/scratch.py", line 1, in <module>
    from primes_py import HOSTS
ImportError: cannot import name 'HOSTS' from 'primes_py' (D:\projets\python\cythoniser\primes_py.cp37-win32.pyd)

What do you think I'm doing wrong?

the pxd file is used to tell *cython* what type the variable is. you still need to create the varibale (name) in primes.py:

HOSTS = 45 # or whatever value

-CHB

 

Best regards.

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/00ef89aa-8051-485e-b6e5-eed9f8592f51%40googlegroups.com.

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris....@noaa.gov

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/CALGmxELi8D7OBATueZg93DELyTLgC_jYS8asFa0j3ewE1Fw6Jg%40mail.gmail.com.

[Peter Schay]

On Thursday, August 22, 2019 at 4:19:21 AM UTC-7, Kevin Tewouda wrote:

Hello everyone,

can somebody helps me to fix my issue related to variable export? As a reminder I define a variable in the.py file and its declaration in the.pxd file, you can read the first exchanges.

Hi,

Here's my understanding; hope this may help (and clarifications/corrections from the experts in this group would be appreciated)

When a python module imports M, the M.so file will take precedence over M.py.

When cython parses M.pxd and M.py to generate the C code for M.so, the declaration of variable x in M.pxd will take precedence over (and obscure) the assignment to x in M.py.

A cdef declaration in a .pxd says the symbol is meant to be invisible to python.

So... if you have "x=3" in M.py it will naturally be accessible from other python modules *unless* you have M.pxd which declares x to be cdef and then build the extension module M.so - in that case x will be C-only and invisible to python and the import will find the extension module before the pure python.

I am not sure you can declare a typed module-level global variable in Cython that will also be accessible from python.

Using cpdef instead of cdef does not seem to work (https://grokbase.com/t/gg/cython-users/12c65xdc9n/feature-request-allow-cpdef-global-variables).

Also if it can't be done in a .pxd file then won't work using a cython.declare in the .py either.

At least for ints, I think the cpdef enum syntax is supposed to give you the best of both worlds (typed access from cython, normal access from python) but again I'm not sure.

Regards,

Pete

[Kevin Tewouda]

Hi Peter,

thanky you for your answer. I came to the same conclusion about global variables. I think it can be an improvement to declare cpdef int .. in a pxd file.

And I confirm you that using Enum type is a good compromise.

Best regards

[Robert Bradshaw]

Thanks, Peter, you're spot on.

As for cpdef module globals, the difficulty is that one would need to
modify the getattr/setattr of the module object itself. There may also
be code (I think cpython has some) that assumes it can access/modify
module-level variables by modifying the underlying dict itself which
would (unsafely) bypass this.

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "cython-users" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/9ab9dd57-cdd8-4258-b545-4883facecf7e%40googlegroups.com.