[cython-users] returning an array from c

[prosswa]

Hello,


I've been trying to figure out how to call a c function that
returns an array that the c function creates. Here is a simple
example of the type of c function that I want to use:

double *make_arr(int length)
{
double *arr;
int iy;

arr = malloc(length*sizeof *arr);

if (arr == NULL) {
/* handle error
What is the best way to do this? */
}

for(iy=0; iy<=length; iy++) {
arr[iy]=(double)iy*20;
}

return arr;
}

Here is my cython code:
cdef extern from "clib_test.h":
double *make_arr(int length)

def call_make_arr(len):
return make_arr(len) # This is where I have trouble!

What is the best way to make this work? How can I free up the
memory once I'm finished with the array?


thanks

Robert

[Stefan Behnel]

prosswa, 30.04.2010 00:15:

Depends on what you do with it. If you want to return it to Python space (a
"def" function suggests that), you must either copy it into something that
Python code can deal with (e.g. a list of floats), or wrap it in an
extension class that guards the pointer, controls access to it, and frees
the memory in its __dealloc__ method. Depending on your use case, the first
(certainly simpler) approach might be useful, but in most cases, you will
want to do the latter for performance reasons, especially if you want to
pass the array back into C code later on.

Stefan

[RJVB]

Alternatively, link in numpy, and return your array via an ndarray
created using

[quote]
PyObject *PyArray_SimpleNewFromData(int nd, npy_intp* dims, int
typenum, void* data)¶

Sometimes, you want to wrap memory allocated elsewhere into an ndarray
object for downstream use. This routine makes it straightforward to do
that. The first three arguments are the same as in PyArray_SimpleNew,
the final argument is a pointer to a block of contiguous memory that
the ndarray should use as it’s data-buffer which will be interpreted
in C-style contiguous fashion. A new reference to an ndarray is
returned, but the ndarray will not own its data. When this ndarray is
deallocated, the pointer will not be freed.

You should ensure that the provided memory is not freed while the
returned array is in existence. The easiest way to handle this is if
data comes from another reference-counted Python object. The reference
count on this object should be increased after the pointer is passed
in, and the base member of the returned ndarray should point to the
Python object that owns the data. Then, when the ndarray is
deallocated, the base-member will be DECREF’d appropriately. If you
want the memory to be freed as soon as the ndarray is deallocated then
simply set the OWNDATA flag on the returned ndarray.
[/quote]

Something like

PyObject *Arr = PyArray_SimpleNewFromData( 1, &length, PyArray_DOUBLE,
arr );

You can get back at the array data in your C code using something like

PyObject *Arr; // parameter passed in from Python code
PyArrayObject *tmpArr; // intermediate variable
double *arr;

if( (tmpArr = (PyArrayObject*) PyArray_ContiguousFromObject(X,
PyArray_DOUBLE, 0,0)) ){
arr = (double*) PyArray_DATA(tmpArr);
}

For freeing this memory, I suppose you could either write your own
function that you export to Python, leaving it to the user to
deallocate the ndarrays your return, or you might try, after the call
to PyArray_SimpleNewFromData():

((PyArrayObject*)Arr)->flags |= NPY_OWNDATA

(I'd like to have confirmation of this interpretation myself!)

René