pypy cffi callback crashes when using multithread

[Ming Hu]

I was using pypy with cffi callback to call my python function from my c library;

 it works fine when they are all in one thread, but it crashes when my c library create a
new thread and do the "callback" in new thread.

The error is "Segmentation fault: 11"

CPython works fine; and Pypy crashes.

My C library

#include <stdio.h>

int (*global_callbck)();

int (*global_thread_callback)();

#include <pthread.h>

int test_foo()

{

return 0;

}

int test_bar()

{

return 100;

}

void register_callback(int (*func)())

{

int res = func();

}

void *another_thread(void *arg)

{

printf("this is another thread\n");

global_thread_callback();

return NULL;

}

void register_thread_callback(int (*func)())

{

global_thread_callback = func;

pthread_t tid;

pthread_create(&tid, NULL, another_thread, NULL);

}

My python code

from cffi import FFI

ffi = FFI()

ffi.cdef("""

int test_foo();

int test_bar();

void register_callback(int (*func)());

void register_thread_callback(int (*func)());

""")

lib = ffi.dlopen("libtest.dylib")

def test_callback():

print 'call back OK'

return 0

# crashes here

def test_thread_callback():

print 'thread call back OK'

return 0

func = ffi.callback("int()", test_callback)

lib.register_callback(func)

thread_func = ffi.callback("int()", test_thread_callback)

lib.register_thread_callback(thread_func)

while True:

pass

[Armin Rigo]

Hi Ming Hu,

On Wed, Apr 24, 2013 at 10:58 AM, Ming Hu <hum...@gmail.com> wrote:
> I was using pypy with cffi callback to call my python function from my c
> library;
> it works fine when they are all in one thread, but it crashes when my c
> library create a
> new thread and do the "callback" in new thread.
> The error is "Segmentation fault: 11"

This is a known issue. Even on CPython it's partially undefined what
occurs, even though it seems to work in your case. But you cannot run
Python code in a thread not created by PyPy's careful logic. It's
probably possible to fix it inside PyPy but it's unclear the
performance impact this would have on all other CFFI callbacks.


A bientôt,

Armin.

[Ming Hu]

thanks. I think I have to find another way to solve my problem when using Pypy instead waiting for them to fix this.

Armin Rigo於 2013年4月27日星期六UTC+8下午4時16分54秒寫道：

[Armin Rigo]

Hi,

On Sat, Apr 27, 2013 at 5:36 PM, Ming Hu <hum...@gmail.com> wrote:
> thanks. I think I have to find another way to solve my problem when using
> Pypy instead waiting for them to fix this.

I'm just saying, I'm not sure it works correctly on top of CPython
too. It might work by luck only in the particular case you're using.


A bientôt,

Armin.

[Ming Hu]

Thanks, we've been using CPython with this kind of callbacks nearly a year, and it works fine for us.

Recently we want to try Pypy.

Since it does not work on Pypy, we've just implemented a Python thread and wait for the C thread to complete, and do the callback stuff,  so that the system emulate the callback function.

This emulation works on both Cpython and Pypy

Armin Rigo於 2013年4月28日星期日UTC+8上午2時53分37秒寫道：