How to use arrays in Cython?

[Lars Buitinck]

Hi all,

I've just been trying to replace a dynamically growing Numpy array
with a cpython.array one to benefit from its resize_smart
capabilities, but I can't seem to figure out how it works. What I have
is a Numpy array X that is grown by calling resize(2 * X.size)
whenever it's full. I can replace it with an ordinary Python array,
calling its append member, but that doesn't get Cythonized properly.
Instead, I thought I could do


import array
cimport cpython.array as cparray

cdef cparray.array[np.int32_t] indices
indices = array.array("i")
# sanity check to make sure "i" is a 32-bit integer

indices.resize_smart(len(indices) + 1)


Which compiles fine, but when I try to run this, the interpreter
complains that 'array.array' object has no attribute 'resize_smart',
so apparently Cython "forgets" that indices is a cparray.array.

Can anyone help me out? I'm using Cython 0.17.

TIA,
Lars

--
Lars Buitinck
Scientific programmer, ILPS
University of Amsterdam

[Andreas van Cranenburgh]

On Saturday, September 15, 2012 2:15:27 PM UTC+2, Lars Buitinck wrote:

I've just been trying to replace a dynamically growing Numpy array
with a cpython.array one to benefit from its resize_smart
capabilities, but I can't seem to figure out how it works. What I have
is a Numpy array X that is grown by calling resize(2 * X.size)
whenever it's full. I can replace it with an ordinary Python array,
calling its append member, but that doesn't get Cythonized properly.

You should be able to do that, perhaps it is related to the remark about imports below.

Which compiles fine, but when I try to run this, the interpreter
complains that 'array.array' object has no attribute 'resize_smart',
so apparently Cython "forgets" that indices is a cparray.array.

resize and resize_smart are at the module level and should be called with an array instance as first argument.
Also, I changed both imports to 'array' because they are the same thing.
The following code runs without error for me:

import array
from cpython cimport array
cimport numpy as np

def main():
    cdef array.array[np.int32_t] indices = array.array("i")
    array.resize_smart(indices, len(indices) + 1)

main()

However, note that using its append and extend methods would trigger the same behavior internally.

Cheers from a UvA colleague

[Andreas van Cranenburgh]

For some more array usage examples in Cython, refer to the tests for the module:

https://github.com/cython/cython/blob/master/tests/run/pyarray.pyx

[Ian Bell]

On Mon, Sep 17, 2012 at 9:33 AM, Andreas van Cranenburgh <and...@unstable.nl> wrote:

For some more array usage examples in Cython, refer to the tests for the module:

https://github.com/cython/cython/blob/master/tests/run/pyarray.pyx

Adding a link to the pyarray tests to the docs would be great.  I (and I am sure quite a few other people) have also been looking for examples, and the ones in the tests seem pretty comprehensive and clear.

Ian

[Lars Buitinck]

2012/9/15 Andreas van Cranenburgh <and...@unstable.nl>:

>> I've just been trying to replace a dynamically growing Numpy array
>> with a cpython.array one to benefit from its resize_smart
>> capabilities, but I can't seem to figure out how it works. What I have
>> is a Numpy array X that is grown by calling resize(2 * X.size)
>> whenever it's full. I can replace it with an ordinary Python array,
>> calling its append member, but that doesn't get Cythonized properly.
>

> resize and resize_smart are at the module level and should be called with an
> array instance as first argument.

Just to let you know, I've finally got some time to get back to this
project and it works now. Thanks!

> However, note that using its append and extend methods would trigger the
> same behavior internally.

Yes, but a call to append shows up yellow in cython -a output, so I
want to avoid it. I've yet to do a full benchmark, but the HTML file
is looking ever whiter :)

[Joshua Landau]

On 17 September 2012 14:33, Andreas van Cranenburgh <and...@unstable.nl> wrote:

For some more array usage examples in Cython, refer to the tests for the module:

https://github.com/cython/cython/blob/master/tests/run/pyarray.pyx

Why does test_copy in this make the array ca if it never uses it? 

[Robert Bradshaw]

Looks like a typo. Fixed.

- Robert

[Andreas van Cranenburgh]

The difference between a and ca is that ca is typed (ca = 'c array').

The extra ca variable could be eliminated by directly specifying the type with the function argument.

[Joshua Landau]

Apologies, but I think you may be misinterpreting the situation.

The code was like this:

def test_copy(a):


    """
    >>> a = array.array('f', [1.0, 2.0, 3.0])


    >>> test_copy(a)
    array('f', [1.0, 2.0, 3.0])


    """
    cdef array.array ca = a


    cdef array.array b


    b = array.copy(a)


    assert a == b


    a[2] = 3.5


    assert b[2] != a[2]


    return b



Note b = array.copy(a), not .copy(ca).

This means that assigning to ca does nothing, as ca doesn't change a or b.

That was the confusion, and so I believe it was a typo as Robert Bradshaw said.

[Andreas van Cranenburgh]

I didn't mean to contradict that it was a typo, I was just suggesting a further simplification of the code.