Cannot Convert pointer to Python Object after module split.

[Adam Martin]

The Problem:

Error compiling Cython file:

------------------------------------------------------------

...

cpdef Py_GetRemoteDevice(PyChannel Chan):

    ret_val = mod_one.PyRemoteDevice()

    cdef core.RemoteDevice  * tmp

    with nogil:

        tmp = core.GetRemoteDevice(Chan.__instance)

    ret_val.__set_ptr(tmp)

                        ^

------------------------------------------------------------

core/mod_two.pyx:11589:25: Cannot convert 'RemoteDevice *' to Python object

The Background:

I have a fairly large C library I'm wrapping with Cython. Initially I 

had everything in two files, core.pyx and core.pxd. However, the more

of the library that was wrapped the larger the core.c file generated.

2.5 million lines and counting. The compile time and memory usage became

cumbersome.

I decided to split things into multiple .pyx files. That's when things that used

to work stopped working.

Disclaimer:

Please keep in mind that I've greatly simplified things for this question.

The wrapped library is somewhat large and proprietary.

The Details:

core.pxd - contains all the necessary components from the C library

I'm wrapping. Also a simple Void * wrapper class declaration.

cdef class Void:

    cdef void *__void

    cdef __set_ptr(self, void *ptr)

    

cdef extern from "core.h":

    ctypedef unsigned char U8

    ctypedef unsigned int U32

    void OS_MemSet(U8 *dest, U8 byte, U32 len) nogil

cdef extern from "mod_one.h":

    cdef struct _RemoteDevice

    ctypedef _RemoteDevice RemoteDevice

    

cdef extern from "mod_two.h":

    cdef struct _Channel

    ctypedef _Channel Channel

core.pyx - not a whole lot is needed here, really it's just the 

implementation of the Void * wrapper.

cdef class Void:

    cdef __set_ptr(self, void *ptr):

        self.__void = ptr

mod_one.pxd - declares the wrapper class for the RemoteDevice C struct

cimport core

cdef class PyRemoteDevice:

    cdef core.RemoteDevice *__instance

    cdef __set_ptr(self, core.RemoteDevice *ptr)

mod_one.pyx - defines the RemoteDevice wrapper class. Note the 

__set_ptr function, which is cdef'd and takes a RemoteDevice *

from cpython.mem cimport PyMem_Malloc, PyMem_Free

import core

cimport core

cdef class PyRemoteDevice:

    def __cinit__(self):

        self.__instance = <core.RemoteDevice *>PyMem_Malloc(sizeof(core.RemoteDevice))

    def __init__(self):

        core.OS_MemSet(<core.U8 *>self.__instance, <core.U8>0, sizeof(core.RemoteDevice))

        

    cdef __set_ptr(self, core.RemoteDevice *ptr):

        self.__instance = ptr

        

    def __dealloc__(self):

        if self.__instance is not NULL:

            PyMem_Free(self.__instance)

            self.__instance = NULL

mod_two.pxd - declares the Channel wrapper class, which I'm only showing

here because it's used in the function that is causing me problems.

cimport core

cimport mod_one

cdef class PyChannel:

    cdef core.Channel *__instance

    cdef __set_ptr(self, core.Channel *ptr)

mod_two.pyx - defines the Channel wrapper class, and declares the

Py_GetRemoteDevice function, which wraps the GetRemoteDevice function 

from the C library.

from cpython.mem cimport PyMem_Malloc, PyMem_Free

import core

cimport core

import mod_one

cimport mod_one

cdef class PyChannel:

    def __cinit__(self):

        self.__instance = <core.Channel *>PyMem_Malloc(sizeof(core.Channel))

    def __init__(self):

        core.OS_MemSet(<core.U8 *>self.__instance, <core.U8>0, sizeof(core.Channel))

    cdef __set_ptr(self, core.Channel *ptr):

        self.__instance = ptr

        

    def __dealloc__(self):

        if self.__instance is not NULL:

            PyMem_Free(self.__instance)

            self.__instance = NULL

cpdef Py_GetRemoteDevice(PyChannel Chan):

    ret_val = mod_one.PyRemoteDevice()

    cdef core.RemoteDevice  * tmp

    with nogil:

        tmp = core.GetRemoteDevice(Chan.__instance)

    ret_val.__set_ptr(tmp)

    return ret_val

    

The problem I'm getting is that when Cythonizing mod_two, I get the 

following error:

Error compiling Cython file:

------------------------------------------------------------

...

cpdef Py_GetRemoteDevice(PyChannel Chan):

    ret_val = mod_one.PyRemoteDevice()

    cdef core.RemoteDevice  * tmp

    with nogil:

        tmp = core.GetRemoteDevice(Chan.__instance)

    ret_val.__set_ptr(tmp)

                        ^

------------------------------------------------------------

core/mod_two.pyx:11589:25: Cannot convert 'RemoteDevice *' to Python object

I'm a bit confused, because this used to work when I had everything in one

module. One difference I can see is that all my wrapper classes were wholly defined

in core.pyx because I wasn't needing to share them between modules. Now

I need to share them, so I've split them into .pyx and .pxd files.

Can anyone point me in the right direction? I've scoured the Cython docs and Google

but so far I haven't found anything that answers my question.

Thanks, and please let me know if any additional information is needed!

- Adam

[Robert Bradshaw]

Did you cimport mod_one? It doesn't seem to know the type of ret_val.

> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "cython-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cython-users...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Adam Martin]

I did cimport mod_one. However, you're right that it doesn't know the type of ret_val. It's because I haven't told it! I feel a bit silly... The line that reads

ret_val = mod_one.PyRemoteDevice()

Should read

cdef mod_one.PyRemoteDevice ret_val = mod_one.PyRemoteDevice()

Thanks for your help!