cffi callback function pointer become invalid

[Yicong Huang]

Hi,

We embeded Pypy in C++ code, and used the callback function pointer to execute Python function.

And we noticed that, the callback function pointers might become invalid as the memory usage grows.

Here is the piece of test code:

   

    for(int i = 0; i < LOOP; i++){

       //read python code from file

       char* pyBuffer = readPyFile("test_Lower.py");

       pypy_execute_source_ptr(pyBuffer, &ptr);

       callers[i] = ptr.pyPtrs[0];

    }

    //check 1st function pointer and call python function

    caller = callers[0];

    ret = (*((char *(*)(char *))caller))("A");

    printf("%s\n", ret);

In the experiment, if the LOOP >150, caller becomes invalid, the callback to python function could not succed, and "Segmentation fault" error was reported.

For more complex Python function, the valid LOOP might be smaller.

I am wondering whether the error was caused by GC in Pypy that recycled the function pointers.

In python fill_api() code, I have used a gloabl list to append function pointer, but it did not help.

#global list

no_gc = []

def fill_api(ptr):

   api = ffi.cast("struct pypyAPI*", ptr)

   api.pyPtrs[0] = test_Lower

   no_gc.append(api)

   no_gc.append(test_Lower)

   return api

And are there any methods to prevent such errors?

[Armin Rigo]

Hi,

On 12 August 2015 at 04:38, Yicong Huang <hengh...@gmail.com> wrote:
> for(int i = 0; i < LOOP; i++){
> //read python code from file
> char* pyBuffer = readPyFile("test_Lower.py");
> pypy_execute_source_ptr(pyBuffer, &ptr);
> callers[i] = ptr.pyPtrs[0];
> }

This is the problem: pypy_execute_source_ptr() is supposed to be
called only once. By calling it multiple times, you override the
previously-defined names. That's admittedly a bug that should be
fixed in PyPy --- I'll fix it --- but the point is that you should do
a minor reorganization of your code and call it only once.


A bientôt,

Armin.