Passing Lists of Objects from Python to C

[Karlson]

Hi,

I am very new to Cython and fairly new to Python. I have a question:

I need to pass a list of objects (int, string, double) from a python
function to a C/C++ function. Is it possible and if so can you give
me an example to look at?

Regards,

Mikhail

[Christopher Barker]

Karlson wrote:
> I need to pass a list of objects (int, string, double) from a python
> function to a C/C++ function.

A little more detail is in order:

Is that list always (int, string, double), i.e. a C struct or function
parameters, or is it a list that could hold any of those?

-Chris

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris....@noaa.gov

[Mikhail Veygman]

I have a list of:

["A", "B", "C", "AA", "BB", "CC"] or
[1, 2, 3, 4, 5, 6] or
[1.0, 2.0, 3.0, 4.0]

I need to pass a list of these strings to an appropriate C/C++ function
that will take this for processing.
-
Regards,

Mikhail Veygman

[Stefan Behnel]

Mikhail Veygman wrote:

> Christopher Barker wrote:
>> Karlson wrote:
>>> I need to pass a list of objects (int, string, double) from a python
>>> function to a C/C++ function.
>>
>> A little more detail is in order:
>>
>> Is that list always (int, string, double), i.e. a C struct or function
>> parameters, or is it a list that could hold any of those?
>

> I have a list of:
>
> ["A", "B", "C", "AA", "BB", "CC"] or
> [1, 2, 3, 4, 5, 6] or
> [1.0, 2.0, 3.0, 4.0]
>
> I need to pass a list of these strings to an appropriate C/C++ function
> that will take this for processing.

Again, a little more detail would help. What's the signature of the C
function, and what C types can it accept as input? Or do you write the C
function yourself?

Stefan

[Mikhail Veygman]

C function I can write myself so it can vary.

I'd prefer: int *, double *, char ** for the respective lists.
-
Regards,

Mikhail Veygman

-----Original Message-----
From: Stefan Behnel <stef...@behnel.de>
Reply-to: cython...@googlegroups.com
To: cython...@googlegroups.com
Subject: [cython-users] Re: Passing Lists of Objects from Python to C

[Robert Bradshaw]

On Oct 5, 2009, at 10:13 AM, Mikhail Veygman wrote:

>
> C function I can write myself so it can vary.
>
> I'd prefer: int *, double *, char ** for the respective lists.

Yes, for example, if you have

int foo_c(int* data, size_t n)

Then you can do

def foo(L):
cdef size_t n = len(L)
cdef int* data = <int*>malloc(n * sizeof(int))
cdef int i
for i in range(n):
data[i] = L[i]
int res = foo_c(data, n)
free(data)
return res

A bit verbose, but that's because you need to malloc/free explicitly,
so it's difficult to know the "right" way to automatically convert a
list to an int*.

- Robert

[Stefan Behnel]

Mikhail Veygman wrote:

> Stefan Behnel wrote:
>> Mikhail Veygman wrote:
>>> Christopher Barker wrote:
>>>> Karlson wrote:
>>>>> I need to pass a list of objects (int, string, double) from a python
>>>>> function to a C/C++ function.
>>>> A little more detail is in order:
>>>>
>>>> Is that list always (int, string, double), i.e. a C struct or function
>>>> parameters, or is it a list that could hold any of those?
>>> I have a list of:
>>>
>>> ["A", "B", "C", "AA", "BB", "CC"] or
>>> [1, 2, 3, 4, 5, 6] or
>>> [1.0, 2.0, 3.0, 4.0]
>>>
>>> I need to pass a list of these strings to an appropriate C/C++ function
>>> that will take this for processing.
>>
>> Again, a little more detail would help. What's the signature of the C
>> function, and what C types can it accept as input? Or do you write the C
>> function yourself?
>

> C function I can write myself so it can vary.

In that case, I'd just pass the lists without further conversion.

> I'd prefer: int *, double *, char ** for the respective lists.

If you really need to do that conversion (in which case you should make
sure that the values really fit into an int, for example), you'll have to
allocate memory using malloc(), then copy the values over, pass them into
the C function and free the memory afterwards. Something like this might work:

cimport stdlib

def pass_list_on(ints):
cdef int i, value
cdef Py_ssize_t count = len(ints)
cdef int* mem = <int*>stdlib.malloc(sizeof(int) * count)
for i, value in enumerate(ints):
mem[i] = value
some_c_function(mem, count)
stdlib.free(mem)

Stefan

[Stefan Behnel]

Stefan Behnel wrote:
> cimport stdlib
>
> def pass_list_on(ints):
> cdef int i, value
> cdef Py_ssize_t count = len(ints)
> cdef int* mem = <int*>stdlib.malloc(sizeof(int) * count)
> for i, value in enumerate(ints):
> mem[i] = value
> some_c_function(mem, count)
> stdlib.free(mem)

Looks like Robert gave you more or less the same solution with the same
memory-leak bug in case of errors. :)

Better:

cdef int* mem = <int*>stdlib.malloc(sizeof(int) * count)

try:

for i, value in enumerate(ints):
mem[i] = value
some_c_function(mem, count)

finally:
stdlib.free(mem)

Stefan

[Mikhail Veygman]

Thank you All.
-
Regards,

Mikhail Veygman

-----Original Message-----
From: Stefan Behnel <stef...@behnel.de>
Reply-to: cython...@googlegroups.com
To: cython...@googlegroups.com
Subject: [cython-users] Re: Passing Lists of Objects from Python to C

[Christopher Barker]

Robert Bradshaw wrote:
> On Oct 5, 2009, at 10:13 AM, Mikhail Veygman wrote:
>>
>> C function I can write myself so it can vary.

Then write it in Cython...

> A bit verbose, but that's because you need to malloc/free explicitly, so
> it's difficult to know the "right" way to automatically convert a list
> to an int*.

Or use numpy -- it has all sorts of nifty code for converting sequences
of python types to numpy arrays, which use a C pointer to store the
data, so it does all that work for you. You can then work with the numpy
array directly in python or Cython, or grab the pointer and pass that
off to C.

Something like this:

import numpy as np
cimport numpy as np

def foo_int(L):
# L should be a sequence of objects that can be turned into ints
cdef np.ndarray[np.int_t, ndim=1]
arr = np.array(L, dtype=np.int)

# now you have a numpy array which you can work with directly
cdef int i
for i in range(arr.size):
do_something_with(arr[i])

Or you can grab the pointer, if you really want to pass it to C, but
I'll leave that as an exercise for the reader ;-)

Here is a good place to start:

http://wiki.cython.org/tutorials/numpy

I'd probably make use of numpy in your Python code anyway -- so maybe
you wouldn't have the lists to begin with!

If you really don't want the numpy dependency, you may be able to use
the std array.array class instead for this simple case.

[Robert Bradshaw]

On Oct 5, 2009, at 11:24 AM, Christopher Barker wrote:

> Robert Bradshaw wrote:
>> On Oct 5, 2009, at 10:13 AM, Mikhail Veygman wrote:
>>>
>>> C function I can write myself so it can vary.
>
> Then write it in Cython...
>
>> A bit verbose, but that's because you need to malloc/free
>> explicitly, so it's difficult to know the "right" way to
>> automatically convert a list to an int*.
>
> Or use numpy -- it has all sorts of nifty code for converting
> sequences of python types to numpy arrays, which use a C pointer to
> store the data, so it does all that work for you. You can then work
> with the numpy array directly in python or Cython, or grab the
> pointer and pass that off to C.

I would second both of these as very good suggestions.

- Robert