C function calling back into my Cython function causes SIGABRT: Fatal Python error: PyThreadState_Get: no current thread

[jt]

I use ctypes to obtain & call a native C function. I am passing a Cython function to it as a callback, which it calls later on.

When it does, this is the result:

Fatal Python error: PyThreadState_Get: no current thread


Current thread 0x0000000004fbb740 (most recent call first):
  File "<string>", line 1 in <module>

(Followed by SIGABRT)

I can comment out the callback call in the native C code, and it won't happen, so it's definitely once the callback is actually called.

This is how I am passing the function callback when calling the C function:

    _scan_lib_ref.scan_folder.restype = ctypes.c_int
    _scan_lib_ref.scan_folder.argtypes = [
        ctypes.c_char_p, ctypes.POINTER(ctypes.c_char_p),
        ctypes.c_void_p,
    ]
   result_value = _scan_lib_ref.scan_folder(
        os.path.abspath(directory),
        ctypes.cast(int(<uintptr_t>&error_msg_buf_ptr),
                    ctypes.POINTER(ctypes.c_char_p)),
        ctypes.cast(int(<uintptr_t><void*>file_info_callback),
                    ctypes.c_void_p),
    )  # Indirectly causes the crash

And that's how the crashing function looks like:

cdef int32_t file_info_callback(
        char *path,
        int tag_count, char **tag_names, char **tag_values
        ):
    print("Hello!")
    return 1

("Hello" is never printed out)

And this is the signature of the native C function "scan_folder":

int scan_folder(const char *_path, char** error,
                void (*fileCallback)(const char *file,
                                     int tag_count,
                                     const char **tag_names,
                                     const char **tag_values));

Does anyone have a suggestion on what to investigate that could be possibly causing this? I don't use threads or anything (not on the C side either) so I'm not sure what's going on.

[Gregor Thalhammer]

Just guessing: your problems might be because of mixing ctypes and cython: ctypes releases the GIL when calling into external functions. cython cdef functions assume the GIL is acquired. Try declaring your cython callback function „with gil“, see 

https://cython.readthedocs.io/en/latest/src/userguide/external_C_code.html#acquiring-the-gil

best

Gregor

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/067e536a-8a19-4f8a-a42e-25999c23553a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[jt]

Ah thanks, good guess: that's it, I needed "with gil". Now it works fine, thanks so much!

[Sour Ce]

I'm getting this exact error when trying to compile my project with # cython: linetrace=True and/or # cython: profile=True

I don't use distutils or cythonize, so not exactly sure where to supply the added directive CYTHON_TRACE_NOGIL=1.

I tried adding # distutils: define_macros=CYTHON_TRACE_NOGIL=1 to the top of my project, as well as cython -X CYTHON_TRACE_NOGIL=1

The former doesn't fix the problem, and the second produces "Error in compiler directive: Unknown option: "CYTHON_TRACE"" for CYTHON_TRACE / CYTHON_TRACE_NOGIL.