How to correctly use pxd in combination with py-file?

[Martin Bammer]

In the first example I've got the following files:

test.py:

class A:

  B=1

  C=2

  D=5

test.pxd:

cdef class A:

  cdef int B

  cdef int C

  cdef int D

In the Python code I'm accessing members of class A with: A.B, A.C and A.D and this works.

But when I convert test.py with "cython -3 -a --cplus test.py" I get a cpp-file with:

B=1;

C=2;

D=5;

and gcc afterwards fails because B, C and D are not known. What I want is that the pxd file converts class A into a C-struct. Is this possible?

In the second example I'm using enums.

test.py:

from enum import Enum

class A(Enum):

    B=1

    C=2

    D=5

test.pxd:

cdef enum A:

    B, C, D

But in this example aready cython fails to convert test.py into test.cpp with the error message: 'A' redeclared.

How do I correctly convert Python enums into C-enums with pxd files?

An important condition is that I do not want to use pyx files, because I want to develop and directly execute and debug my Python files with the interpreter.

Greetings, Martin

[Martin Bammer]

Ok, I've found a solution myself. It is not very smart, but it works:

test.py:

class A:

    def init(self, B, C, D):

        self.B = B

        self.C = C

        self.D = D

a=A()

a.init(1,2,5)

test.pxd:

cdef class A:

    cdef int B, C, D

    cdef void init(self, int B, int C, int D)

cdef A a

The solution is to create an instance of class A and call an initialization function.

Please note that I'm not using __init__, because it can't be declared as a pure C-function, thus producing some overhead when called.

The problem I was facing in the example of the first post is that cython seems to have problems with static member variables.

Instead of declaring them as, for example  "int A_B = 1;", they are declared without class name prefix "B = 1;".

The drawback of this solution is that there exist no static member variables anymore, so that existing Python code like "A.B" has to be changed to "a.B".

Does anyone has a better solution?

Regards, Martin

[Chris Barker]

This is indeeed probably klkunkier than you need.

I'm not sure quite what you are trying to accomplish, but I think you want an "extension class":

http://docs.cython.org/src/userguide/extension_types.html

in particular, look for the __cinit_ method.

-CHB

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris....@noaa.gov

[Martin Bammer]

I know about the extension class. The problem is that I don't want to use pyx files, because then I would be facing the problems that I cannot directly use and execute the code with the Python interpreter. If I would use pyx files I would have to either always compile the pyx file to a pyd file before I can test and use the code or I would have to manage 2 files (py + pyx files). The combination of py and pxd files is for me the best because I can develop the code in pure Python and I can use it directly without the need to converting it into a pyd-file in every other project. With the pxd-file I can speedup the code. This solution is perfect. But unfortunately the pxd-files have several limitations. One of them ist that I cannot use __cinit__. If I use it in the py-file but not defining it in the pxd-file as a C-function, it is handled by Cython as a pure Python function. And in the pxd-file I'm not able to define it as a C-function. Cython does not allow to define special functions, like __init__ and __cinit__, in the pxd-files. So the solution I've posted in my second post is a more or less bad compromise solution.

Another problem I'm facing with Cython is that it cannot handle static member variables like Python does (see my first post). In Python I can define a class A with member variable B, initialized with value 1. I simply can access it with A.B. Unfortunately Cython cannot handle this, but I would need this in several projects. One of them is speeding up the libclang binding for Python. cindex.py is a pure Python layer. I would like to improve the speed using Cython. But several classes have several static variables defined.

I have no idea how complex it is implementing this feature in Cython, but it would be really great.

Regards, Martin

[Chris Barker]

On Mon, Jan 11, 2016 at 1:02 PM, Martin Bammer <mrb...@gmail.com> wrote:

I know about the extension class. The problem is that I don't want to use pyx files, because then I would be facing the problems that I cannot directly use and execute the code with the Python interpreter.

so you want pure python mode. I haven't used that, but I'm pretty sure you can do most of the same stuff.

 

Another problem I'm facing with Cython is that it cannot handle static member variables like Python does (see my first post). In Python I can define a class A with member variable B, initialized with value 1. I simply can access it with A.B. Unfortunately Cython cannot handle this, but I would need this in several projects.

this doesn't work in Cython???

class A(object):

    b = 1

    c = 2

    def __init__(self, .....):

        pass

a = A()

print a.b, a.c

Why not?

Or is that you can't figure out how to declare that A.b and A.c are particular static types?

in which case, try:

 

class A(object):
    b = cython.declare(cython.int, 1)

    c = cython.declare(cython.int, 2)

    def __init__(self, .....):
        pass

a = A()

print a.b, a.c

I haven't tried that, but the docs indicate it should work.

-CHB

 One of them is speeding up the libclang binding for Python. cindex.py is a pure Python layer. I would like to improve the speed using Cython. But several classes have several static variables defined.

I have no idea how complex it is implementing this feature in Cython, but it would be really great.

Regards, Martin

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Omer Katz]

I think you're looking for __cinit__(). See http://docs.cython.org/src/userguide/special_methods.html#initialisation-methods-cinit-and-init

[Martin Bammer]

so you want pure python mode. I haven't used that, but I'm pretty sure you can do most of the same stuff.

Exactly. The problem is that some important features are not usable in pure python mode.

 

Another problem I'm facing with Cython is that it cannot handle static member variables like Python does (see my first post). In Python I can define a class A with member variable B, initialized with value 1. I simply can access it with A.B. Unfortunately Cython cannot handle this, but I would need this in several projects.

this doesn't work in Cython???

class A(object):

    b = 1

    c = 2

    def __init__(self, .....):

        pass

a = A()

print a.b, a.c

Why not?

Compiling the created cpp-file with cython from the above example fails, because b and c are created as module global variables "b" and "c". So accessing "b" with "a.b" or "A.b" will fail with an "unknown identifier" error.
As I mentioned in my above posts static member variables are not handled correctly in cython. I'm not sure if this is a bug, or a just not implemented feature.
 

Or is that you can't figure out how to declare that A.b and A.c are particular static types?

in which case, try:

 

class A(object):
    b = cython.declare(cython.int, 1)

    c = cython.declare(cython.int, 2)

    def __init__(self, .....):
        pass

a = A()

print a.b, a.c

With the above example the cython compiler crashes with an AssertionError.
Traceback (most recent call last):
  File "setup.py", line 10, in <module>
    ext_modules = cythonize(Extension("test_module", sources = [ "test_module.py" ], language = "c++"), **ext_options ),
  File "C:\Python27\lib\site-packages\Cython\Build\Dependencies.py", line 877, in cythonize
    cythonize_one(*args)
  File "C:\Python27\lib\site-packages\Cython\Build\Dependencies.py", line 980, in cythonize_one
    result = compile([pyx_file], options)
  File "C:\Python27\lib\site-packages\Cython\Compiler\Main.py", line 684, in compile
    return compile_multiple(source, options)
  File "C:\Python27\lib\site-packages\Cython\Compiler\Main.py", line 662, in compile_multiple
    result = run_pipeline(source, options, context=context)
  File "C:\Python27\lib\site-packages\Cython\Compiler\Main.py", line 492, in run_pipeline
    err, enddata = Pipeline.run_pipeline(pipeline, source)
  File "C:\Python27\lib\site-packages\Cython\Compiler\Pipeline.py", line 365, in run_pipeline
    data = phase(data)
  File "C:\Python27\lib\site-packages\Cython\Compiler\Pipeline.py", line 53, in generate_pyx_code_stage
    module_node.process_implementation(options, result)
  File "C:\Python27\lib\site-packages\Cython\Compiler\ModuleNode.py", line 118, in process_implementation
    self.generate_c_code(env, options, result)
  File "C:\Python27\lib\site-packages\Cython\Compiler\ModuleNode.py", line 358, in generate_c_code
    self.generate_module_init_func(modules[:-1], env, globalstate['init_module'])
  File "C:\Python27\lib\site-packages\Cython\Compiler\ModuleNode.py", line 2186, in generate_module_init_func
    self.body.generate_execution_code(code)
  File "C:\Python27\lib\site-packages\Cython\Compiler\Nodes.py", line 442, in generate_execution_code
    stat.generate_execution_code(code)
  File "C:\Python27\lib\site-packages\Cython\Compiler\Nodes.py", line 442, in generate_execution_code
    stat.generate_execution_code(code)
  File "C:\Python27\lib\site-packages\Cython\Compiler\Nodes.py", line 4434, in generate_execution_code
    self.body.generate_execution_code(code)
  File "C:\Python27\lib\site-packages\Cython\Compiler\Nodes.py", line 442, in generate_execution_code
    stat.generate_execution_code(code)
  File "C:\Python27\lib\site-packages\Cython\Compiler\Nodes.py", line 4800, in generate_execution_code
    self.generate_assignment_code(code)
  File "C:\Python27\lib\site-packages\Cython\Compiler\Nodes.py", line 5091, in generate_assignment_code
    self.lhs.generate_assignment_code(self.rhs, code)
  File "C:\Python27\lib\site-packages\Cython\Compiler\ExprNodes.py", line 2132, in generate_assignment_code
    assert entry.type.is_pyobject, "Python global or builtin not a Python object"
AssertionError: Python global or builtin not a Python object


Regards, Martin

[Martin Bammer]

__cinit__ is not usable when using pure python mode and pxd files.

[Chris Barker]

On Mon, Jan 11, 2016 at 11:52 PM, Martin Bammer <mrb...@gmail.com> wrote:

this doesn't work in Cython???

class A(object):

    b = 1

    c = 2

    def __init__(self, .....):

        pass

a = A()

print a.b, a.c

Compiling the created cpp-file with cython from the above example fails, because b and c are created as module global variables "b" and "c". So accessing "b" with "a.b" or "A.b" will fail with an "unknown identifier" error.
As I mentioned in my above posts static member variables are not handled correctly in cython. I'm not sure if this is a bug, or a just not implemented feature.

sure looks like a big to me -- a goal of cython is to be able to correctly handle any python code -- and that is not in the least rare or unusual code....

OK -- I just tested it, and it works fine for me. either as *.py or *.pyx, run it thorugh cython, and the compiled module runs fine. Enclosed are the code and setup.py, compiled with:

./setup.py build_ext --inplace

Cython version 0.23.4 

How to statically type A.b and A.c in pure python mode is beyond, me, though...

-CHB

 

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Robert Bradshaw]

Static member variables work just fine, for def and cdef classes, but
they cannot be statically typed. (This would be nice to implement,
possibly it would have to be instance members that are initialized to
the shared default though.)

Unfortunately, it is correct that pure mode is not as complete as
"standard" cython code.