Defining callbacks at runtime

[Paul Nader]

Hi,

I have a c library function that takes a callback argument. I  define the python callback as extern "Python" which binds it to a function name on the python runtime side. This means that if I want to call the c function multiple times with different callback functions I need to define them all as extern "Python" in cdef() at compile time.

I would like to make the Python side generic so that users of it can define callback functions without having to recompile the library. Is there a way of doing this without defining the python callback functions in the cdef()?

Thanks

[Armin Rigo]

Hi!

On Wed, 15 Mar 2023 at 08:32, Paul Nader <paul....@gmail.com> wrote:
> I have a c library function that takes a callback argument. I define the python callback as extern "Python" which binds it to a function name on the python runtime side. This means that if I want to call the c function multiple times with different callback functions I need to define them all as extern "Python" in cdef() at compile time.
>
> I would like to make the Python side generic so that users of it can define callback functions without having to recompile the library. Is there a way of doing this without defining the python callback functions in the cdef()?

Yes and no: you can use the old-style callbacks
(https://cffi.readthedocs.io/en/latest/using.html#callbacks-old-style)
and if they work in your use case, then fine. Be sure to read the big
warnings, though.

That solution is not very clean and not fully supported everywhere.
The reason is that it's impossible to do what you describe in C,
either. The C solution is to write APIs in such a way that any
function that takes a callback as argument also takes a "void *" that
is passed back to the callback. If the C library that you're binding
to does not do that, then you're out of luck. If it does, then there
is a clean solution: call to the C library with always the same
"extern Python" function, but with different "void *" arguments. Use
"ffi.new_handle()" to build the "void *" out of a Python data
structure, for example the actual Python callable object, or any other
data.


A bientôt,

Armin

[Paul Nader]

Hi Armin,

Thanks for the quick response!

I changed them from @ffi.def_extern() to @ffi.callback(..) and it works!!!

Many thanks

Paul