Den 30.07.2012 19:16, skrev Sturla Molden:

Here is an example of using memoryviews:

https://github.com/jakevdp/memview_benchmarks/blob/master/memview.pyx

The extra overhead here is just 2.2% compared to plain C pointer
arithmetics (using -O2) when a 1000 x 1000 array is passed to "compute
distances".

As we discussed in the same thread, doing this with np.ndarray is just
too slow:

https://github.com/jakevdp/memview_benchmarks/blob/master/cythonized_...

Anything but trivial functions will need slicing and function calls. You
can see here how memoryviews allows us to code in "Fortran style" with
arrays instead of using C style pointer arithmetics.

Pointer arithmetics is still not too bad in this simple case, but it can
be much more complicated (but why bother?):

https://github.com/jakevdp/memview_benchmarks/blob/master/pointer_ari...

So I suggest one of these idioms for calling C from Cython:

cdef double[:,::1] array = numpy_array
cdef Py_ssize_t m, n
m = array.shape[0]
n = array.shape[1]
foobar(m, n, &array[0,0])

cdef view.array array = <double[:,::1]> numpy_array
cdef Py_ssize_t m, n
m = array.shape[0]
n = array.shape[1]
foobar(m, n, <double*> array.data)

Sturla

The only difference is that first line: will they generate the same C
code?  -- why would one choose on or the other?

(there should be one obvious way to do it...)

I'll probably go update the wiki page, but to make sure I'm clear:

Using a view.array will build a Cython "memoryview", using the
extended buffer interface -- so you can pass in any Python object that
supports PEP 3118 buffers (does  anything other than numpy suport that
now??)

This is a lightweight way to both get a pointer to the data, and to
have a way, in Cython, to slice and dice the data.

Will this check for C-contiguous and raise an error if not?

-Chris

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Bar...@noaa.gov

On 6 August 2012 18:28, Chris Barker <chris.bar...@noaa.gov> wrote:

Not entirely. A cython.view.array is an array itself, that has memory
itself. In other words, it's a (lousy) replacement for numpy. It's
useful for users when they have memory allocated from somewhere and
want to turn it into a numpy array or memoryview. The array is more of
an intermediate thing, usually created implicitly by casting a
pointer: <double[:10:1, :10]> p creates a Fortran contiguous cython
array view on the memory of shape (10, 10).

Yes, buffers are always validated when they are assigned to by objects
at runtime. Assigning memoryviews to each other with wrong types is an
error, e.g.

cdef double[:, ::1] array = numpy_array # runtime check
cdef double[:, ::1] array2 = array # compile-time check

Den 06.08.2012 19:28, skrev Chris Barker:

The last line is also different ;-)

The first creates a typed memoryview of the NumPy array. That is, using
the NumPy array as an PEP 3118 buffer directly from Cython. No auxillary
Python object is created.

The second created an auxillary Cython array that shares memory with the
NumPy array. The Cython array is a Python object with a PEP 3118 buffer.
Unlike np.ndarray, it has a valid .data attribute that will not break.

If you dislike &array[0,0] syntax, pick the latter. If you don't want an
auxillary Python object, pick the former.

Typed memoryviews will give you convinient and fast data access and fast
array slicing in Cython. cython.view.array can e.g. be used if you need
to send a C pointer as a Python pep 3118 object to some Python routine.

Thus they are not exactly the same, but "almost".

Typical use cases:

double[:,:] or double[:,::1]:
   - You have a C pointer, you want fast "NumPy style" indexing and slicing.
   - You have a NumPy array, you want fast indexing and fast slicing.
   - You are considering to use Fortran because of its array syntax.

double[::1,:]:
   - You need to interface with Fortran code.
   - You are porting old Fortran code line-by-line.
   - You are working with OpenGL.

cython.view.array:
   - You have a C pointer and need a Python PEP 3118 object.
   - You want a safe replacement for malloc (one that is faster than NumPy).
   - You are porting old NumPy/Cython code that uses the .data attribute everywhere.

Also beware of this:

Common contra-indications for using double[::1,:] orcython.view.array:
   - You need to interface with BLAS. Use cblas instead.
   - You need to interface with LAPACK. Use LAPACKE instead.
   - You need to allocate WORK arrays for LAPACK. Use LAPACKE instead.

Sturla

On Mon, Aug 6, 2012 at 5:03 PM, Sturla Molden

OK -- the original start of this post was how best to essentially pass
though a numpy data buffer to C/C++ code (or Fortran, I suppose).

So it looks like:

cdef double[:,::1] array = numpy_array

is very lightweight, so may be a good way to go, and would support
other PEP 3118 buffer objects, so seems a good idea.

Should I change the recommendation in the Wiki?

http://wiki.cython.org/tutorials/NumpyPointerToC

-Chris

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Bar...@noaa.gov

Hi,

Python 2.6.6
GCC 3.4.5
Cython 0.15.1
XP 32

http://wiki.cython.org/tutorials/NumpyPointerToC

I tried building the extension using code but it seems multiply.pyx is
having some missing code.

@cython.boundscheck(False)
^
multiply.pyx:18:0: undeclared name not builtin: cython

This is because you are not importing cython. After adding "*cimport cython*",
next error is in generating *"multiply.c*" file.

multiply.o:multiply.c:(.text+0x5c3): undefined reference to
`_imp__c_multiply'

Any pointers? I am not much into cython lust started digging.

Regards

Prashant

I'm running Cython 0.16 -- that may  be the issue

I haven't tested this code with gcc on Windows -- are you successfully
compiling other stuff?

sorry about that -- I"ve updated the wiki -- I thought I cut&pasted
from a working version...

try the latest Cython -- then report back.

-Chris

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Bar...@noaa.gov

This is with cython 0.16

running build_ext
cythoning multiply.pyx to multiply.c
building 'multiply' extension
D:\MinGW\bin\gcc.exe -mno-cygwin -mdll -O -Wall
-IC:\Python26\lib\site-packages\numpy\core\include -IC:\Python26\include
-IC:\Python26\PC -c multiply.c -o build\temp.win32-2.6\Release\multiply.o
multiply.c: In function `__Pyx_GetBuffer':
multiply.c:4353: warning: unused variable `getbuffer_cobj'
multiply.c: In function `__Pyx_ReleaseBuffer':
multiply.c:4393: warning: unused variable `releasebuffer_cobj'
multiply.c: At top level:
C:/Python26/lib/site-packages/numpy/core/include/numpy/__multiarray_api.h:1 533:
warning: '_import_array' defined but not used
C:/Python26/lib/site-packages/numpy/core/include/numpy/__ufunc_api.h:227:
warning: '_import_umath' defined but not used
D:\MinGW\bin\gcc.exe -mno-cygwin -mdll -O -Wall
-IC:\Python26\lib\site-packages\numpy\core\include -IC:\Python26\include
-IC:\Python26\PC -c c_multiply.c -o
build\temp.win32-2.6\Release\c_multiply.o
c_multiply.c:22:2: warning: no newline at end of file
writing build\temp.win32-2.6\Release\multiply.def
D:\MinGW\bin\gcc.exe -mno-cygwin -shared -s
build\temp.win32-2.6\Release\multiply.o
build\temp.win32-2.6\Release\c_multiply.o
build\temp.win32-2.6\Release\multiply.def -LC:\Python26\libs
-LC:\Python26\PCbuild -lpython26 -lmsvcr90 -o
D:\Projects\cython\multiply.pyd
build\temp.win32-2.6\Release\multiply.o:multiply.c:(.text+0x6ff): undefined
reference to `_imp__c_multiply'
collect2: ld returned 1 exit status
error: command 'gcc' failed with exit status 1

Something weird is going on.

I've enclosed my copy of the code, sertup.py and test file, so we at
least make sure we're working with the same code.

Below is what I get on my Mac -- but it works OK for me on Windows, too:

Python 2.7 Cython 0.16, MSVC 2008 compiler

It looks like you're using MinGW, but I"d expect that to work -- what
else is different?

Python 3 maybe?

-Chris

## My results:

$ python setup.py build_ext --inplace
running build_ext
cythoning multiply.pyx to multiply.c
building 'multiply' extension
creating build
creating build/temp.macosx-10.3-fat-2.7
gcc-4.0 -fno-strict-aliasing -fno-common -dynamic -isysroot
/Developer/SDKs/MacOSX10.4u.sdk -arch ppc -arch i386 -g -O2 -DNDEBUG
-g -O3 -I/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-pack ages/numpy/core/include
-I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7
-c multiply.c -o build/temp.macosx-10.3-fat-2.7/multiply.o
gcc-4.0 -fno-strict-aliasing -fno-common -dynamic -isysroot
/Developer/SDKs/MacOSX10.4u.sdk -arch ppc -arch i386 -g -O2 -DNDEBUG
-g -O3 -I/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-pack ages/numpy/core/include
-I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7
-c c_multiply.c -o build/temp.macosx-10.3-fat-2.7/c_multiply.o
gcc-4.0 -bundle -undefined dynamic_lookup -arch ppc -arch i386
-isysroot /Developer/SDKs/MacOSX10.4u.sdk -isysroot
/Developer/SDKs/MacOSX10.4u.sdk -g
build/temp.macosx-10.3-fat-2.7/multiply.o
build/temp.macosx-10.3-fat-2.7/c_multiply.o -o
/Users/chris.barker/HAZMAT/GNOME-dev/GNOME-GIT/experiments/Cython-numpy_tes t/multiply.so
orrw-lm-1689049:Cython-numpy_test chris.barker$ py.test test_multiply.py
========================================================== test
session starts ===========================================================
platform darwin -- Python 2.7.3 -- pytest-2.2.3
collected 4 items

test_multiply.py ....

======================================================== 4 passed in
0.16 seconds ========================================================
orrw-lm-1689049:Cython-numpy_test chris.barker$

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Bar...@noaa.gov

Cython-numpy_test.zip 4K Download