Issue with ctypes and callback functions

[Bill Somerville]

Hi All,

Is this a good place to ask questions about the above?

Regards
Bill.

[eryk sun]

On Wed, Aug 3, 2016 at 9:38 AM, Bill Somerville
<BSome...@flexerasoftware.com> wrote:
>
> Is this a good place to ask questions about the above?

Discussing ctypes is fine here. There's also a ctypes-users list.

[Bill Somerville]

Thanks Eryk,

I am trying to handle Python callback functions in a SWIG interface, SWIG does not support callbacks where the target is a Python callable but I can do this with ctypes. My problem is that I need to have the callback receive a SWIG wrapped type as an argument by reference. I see that if I use a ctypes.Structure type then it works nicely but I have a huge and changing API to wrap and do not want to manually describe every C struct with ctypes.Structure when SWIG has already automatically created its own shadow classes. The SWIG struct wrapper is just a class wrapping a pointer so can be easily be passed just like ctypes.Structure can be. How do I tell ctypes that it needs to reconstitute the type from an address in the callback thunk it generates?

I have tried defining from_param() and self._as_parameter_ for the types which works for passing to a function but I don't need that as SWIG does that already. With callback arguments I get various errors depending on what methods/attributes I supply like:

Parsing argument 0
TypeError: cannot build parameter

At the point where the callback is being called.

To summarize:
++++++++++++++++++++++++++++++++++++
C API like:

typedef struct {
int member;
/* ... */
} my_type;
typedef int (*pfn) (my_type const *);
int fn (pfn);

Implementation could be:

#include "api.h"
int fn (pfn cb) {
my_type t = {42, /* ... */};
return cb (&t);
}

I want Python code like:

from ctypes import *
import api # SWIG extension module

def my_cb (t):
print ('in callback with:', t.member)
return 0

proto = CFUNCTYPE (c_int, my_type)
cb = proto (my_cb)
res = api.fn (cast (cb, c_void_p).value)
print ('fn returned:', res)

------------------------------------------------

I can extend the SWIG shadow class to do the creation of the prototype and thunk, the required cast above and any extra attributes or methods like _as_parameter_ and from_param() but I can't see any way of having the Python callback (my_cb) magically receive the SWIG wrapped 'my_type' struct.

TIA
Bill.

[eryk sun]

On Wed, Aug 3, 2016 at 1:21 PM, Bill Somerville
<BSome...@flexerasoftware.com> wrote:
> I can extend the SWIG shadow class to do the creation of the prototype and thunk, the
> required cast above and any extra attributes or methods like _as_parameter_ and
> from_param() but I can't see any way of having the Python callback (my_cb) magically
> receive the SWIG wrapped 'my_type' struct.

from_param is a hook method for a type that's set in a function
pointer's argtypes. It gets called to convert an argument when the
function pointer is called from Python. The return value can be a
ctypes instance, an object with a hard-coded conversion (e.g. a string
or integer), or an object that defines an _as_parameter_ attribute.

Only ctypes types are supported in callbacks, which unfortunately
isn't documented clearly. Specifically, the class dict needs to be an
extended C storage dict (i.e. StgDictObject), either to look up the
getfunc of a simple type or to ensure that instantiating a non-simple
type returns a ctypes instance (i.e. CDataObject) with a known size.
The relevant code in _CallPythonObject is as follows (when stripped of
declarations and error handling):

cnv = PySequence_GetItem(converters, i);
dict = PyType_stgdict(cnv);
if (dict && dict->getfunc && !_ctypes_simple_instance(cnv)) {
v = dict->getfunc(*pArgs, dict->size);
PyTuple_SET_ITEM(arglist, i, v);
} else if (dict) {
obj = (CDataObject *)PyObject_CallFunctionObjArgs(cnv, NULL);
memcpy(obj->b_ptr, *pArgs, dict->size);
PyTuple_SET_ITEM(arglist, i, (PyObject *)obj);
} else {
PyErr_SetString(PyExc_TypeError,
"cannot build parameter");

I don't have much experience with SWIG. Does it provide some means to
instantiate a wrapped type from an address? If it does, then you can
use a void pointer as the callback parameter.