Using imported cdef class without gil in another cython module

[Oskar Maier]

Dear cython-users mailing list,

I've lately been fighting with trying to use one Cython class (cdef extension type) defined in a compiled module from another. Is this at all possible? And if yes, how?

Toy-example
###########

Files
=====

lib.pxd
-------
cdef class Cla:
    cdef int a
    
lib.pyx
-------
cdef class Cla:
    def __cinit__(self, int a):
        self.a = a
        
app.pxd
-------
cdef extern class lib.Cla:
    cdef int a

cdef class Test:
    cdef Cla obj
    cdef void test(self) nogil
    
app.pyx
-------
cdef class Test:
    def __cinit__(self):
        self.obj = Cla(2)
    
    cdef void test(self) nogil:
        cdef Cla obj = self.obj


Error on `cython app.pyx`
=========================
Error compiling Cython file:
------------------------------------------------------------
...
cdef class Test:
    def __cinit__(self):
        self.obj = Cla(2)
    
    cdef void test(self) nogil:
        cdef Cla obj = self.obj
            ^
------------------------------------------------------------

app.pyx:6:13: Assignment of Python object not allowed without gil

Error compiling Cython file:
------------------------------------------------------------
...
cdef class Test:
    def __cinit__(self):
        self.obj = Cla(2)
    
    cdef void test(self) nogil:
        ^
------------------------------------------------------------

app.pyx:5:9: Function declared nogil has Python locals or temporaries

Error compiling Cython file:
------------------------------------------------------------
...
cdef class Test:
    def __cinit__(self):
        self.obj = Cla(2)
    
    cdef void test(self) nogil:
        cdef Cla obj = self.obj
                          ^
------------------------------------------------------------

app.pyx:6:27: Creating temporary Python reference not allowed without gil



Is there some way use the cython (cdef, cpdef) parts of the class Cla without gil?  Maybe by declaring Cla public in lib.pxd and treating is as C-function?

Best,
Oskar

[Stefan Behnel]

Your problem is not the sharing, it's the "nogil". Read the error messages
above, they are quite clear about what's happening and why this cannot work
when the code does not own the GIL. One of the reasons why the GIL exists
is to protect the object reference counting, i.e. exactly the kind of
operations you do above.

Just remove the "nogil" declaration from the test() function. If you really
need to call it without holding the GIL, declare it as "with gil" instead,
so that it can acquire the GIL when being called. However, if you tell us a
bit more about your intention, we might be able to give better advice.

Stefan