Re: [cython-users] Using numpy functions inside prange

[Robert Bradshaw]

On Mon, Oct 22, 2012 at 3:02 PM, ATrindade <arlindo....@gmail.com> wrote:
> Hello all,
>
>
> I'm trying to compile the following simple cython code:
>
> import numpy as np
> cimport numpy as np
> from cython.parallel cimport prange,threadid
> from libc.stdio cimport printf
>
> cpdef np.ndarray[double, ndim=1] f(np.ndarray[double,ndim=1] x, double
> alpha):
> cdef np.ndarray[double, ndim=1] s=np.zeros(x.shape[0])
> cdef double tmp
> cdef int i
> cdef int id
>
> for i in prange(x.shape[0],nogil=True,num_threads=2):
> id=threadid()
>
> s[i]=np.sqrt(x[i])
> printf("%d\t %f\n",id,x[i])
> return s
>
> However when I do cython -a name.pyx I get the following error message:
>
> Error compiling Cython file:
> ------------------------------------------------------------
> ...
> cdef int id
>
> for i in prange(x.shape[0],nogil=True,num_threads=2):
> id=threadid()
> #with gil:
> s[i]=np.sqrt(x[i])
> ^
> ------------------------------------------------------------
>
> teste.pyx:15:23: Coercion from Python not allowed without the GIL
>
> Error compiling Cython file:
> ------------------------------------------------------------
> ...
> cdef int id
>
> for i in prange(x.shape[0],nogil=True,num_threads=2):
> id=threadid()
> #with gil:
> s[i]=np.sqrt(x[i])
> ^
> ------------------------------------------------------------
>
> teste.pyx:15:23: Calling gil-requiring function not allowed without gil
>
> Error compiling Cython file:
> ------------------------------------------------------------
> ...
> cdef int id
>
> for i in prange(x.shape[0],nogil=True,num_threads=2):
> id=threadid()
> #with gil:
> s[i]=np.sqrt(x[i])
> ^
> ------------------------------------------------------------
>
> teste.pyx:15:18: Accessing Python attribute not allowed without gil
>
> Error compiling Cython file:
> ------------------------------------------------------------
> ...
> cdef int id
>
> for i in prange(x.shape[0],nogil=True,num_threads=2):
> id=threadid()
> #with gil:
> s[i]=np.sqrt(x[i])
> ^
> ------------------------------------------------------------
>
> teste.pyx:15:18: Accessing Python global or builtin not allowed without gil
>
> Error compiling Cython file:
> ------------------------------------------------------------
> ...
> cdef int id
>
> for i in prange(x.shape[0],nogil=True,num_threads=2):
> id=threadid()
> #with gil:
> s[i]=np.sqrt(x[i])
> ^
> ------------------------------------------------------------
>
> teste.pyx:15:23: Constructing Python tuple not allowed without gil
>
> Error compiling Cython file:
> ------------------------------------------------------------
> ...
> cdef int id
>
> for i in prange(x.shape[0],nogil=True,num_threads=2):
> id=threadid()
> #with gil:
> s[i]=np.sqrt(x[i])
> ^
> ------------------------------------------------------------
>
> teste.pyx:15:25: Converting to Python object not allowed without gil
>
>
> I know that the gil must be released when one uses prange, However it seems
> to conflict with calling functions (in this case sqrt) available in numpy
> module.

Yep, np.sqrt requires the GIL. You have a "with gil" block in your
code, which should make it compile (but negate the parallelism
benefits)?

> Does anyone knows a way around this?

You have a 1-dimensional array, so using np.sqrt on each element is
going to be very inefficient. Invoke

from libc.math cimport sqrt

and use that one. (If your array was multi-dimensional, you'd have to
write out the loop itself, though Mark Florisson has some pretty cool
stuff in the pipeline for vectorized memory view arithmetic.)

- Robert