Arithmetic on Cython Typed Memoryviews

[pythonOmetrist]

With the help of some generous folks, I obtained a ridiculous gain in speed using cython memory views: Bakeoff Part 1 Python vs Cython vs Cython Typed memory views: LDA by Gibbs Sampling

But: I cant seem to do simple things like add a value to an array of values stored in a memory view. I completely get it that that is not what a typed memoryview is supposed to do. But converting the memory view back to an np.array is slower than tortoises herding cats.

When i try to write a cdef function like:

cdef double[::1] _add(self,double[::1] arr,double val):
        cdef double[::1] newarr
        cdef int i, n
        #n = sizeof(arr)/sizeof(arr[0])
        newarr = np.empty(5)
        for i in xrange(n):
            newarr[i] = arr[i] + val
        return newarr

I get errors that say that the memoryviews are not contiguous."ValueError: Buffer and memoryview are not contiguous in the same dimension." Edit This actually does work if the memory view passed is not one thats has been sliced. But it adds 10 seconds to the process! 0.025 to 10+seconds....arrgh. And i may need three of these!

I am new to this stuff, but hoping there is some easy way to address this. otherwise, I have speedily reached the cells I want, but cant seem to do much with them :-).

I will be happy to include any methods to see how the calculation runs in terms of speed.

[Matěj Laitl]

There was an array expressions project ( https://github.com/cython/cython/pull/144 ) by Mark Florisson to do this, but it seems that plans to merge it to Cython master have been abandoned. More ready solution could be Ceygen ( https://pypi.python.org/pypi/Ceygen ), which should be rather fast, but will make your code a bit ugly.

        Matěj

[Chris Barker - NOAA Federal]

> On Thursday, April 18, 2013 2:08:37 PM UTC+2, pythonOmetrist wrote:

>> But converting the memory view back to
>> an np.array is slower than tortoises herding cats.

that should be a fast operation -- it requires building a ndarray, but
it should be abel to share the memory block. OK "fast" is relative, if
you're working with small arrays, then the overhead could be
comparatively large.

>> When i try to write a cdef function like:
>>
>> cdef double[::1] _add(self,double[::1] arr,double val):
>> cdef double[::1] newarr
>> cdef int i, n
>> #n = sizeof(arr)/sizeof(arr[0])
>> newarr = np.empty(5)
>> for i in xrange(n):
>> newarr[i] = arr[i] + val
>> return newarr

you've got a memoryview coming in and going out, so I'd avoid the
numpy overhead here (and size (5,) array is small as per the previous
discussion...) Also, I think [::1] is the same as [:] -- why not keep
it simple?

this should work, using a regular old C array:

# Memoryview on a C array
cdef double carr[5]
cdef double[:] newarr = carr

of course, that's statically defining the size, which is probably not
what you ultimately want.

you could malloc that c array, though there must be a way to create a
new empty memoryview (I'm still a bit fuzy on this....) Maybe a cython
array:

from cython.view cimport array as cvarray

cyarr = cvarray(shape=(n, ), itemsize=sizeof(double), format="d")
cdef double[:] newarr = cyarr

Anyone? What is the best way to create a new object that can get
returned as a memoryview?

HTH,
-Chris

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris....@noaa.gov