segfault in PyArray_FromAny from a memoryviewslice

[becker.nils]

hi,

i'm having trouble debugging cython code using memoryviews; basically, a loop calls the function given below thousands of times and then at some point the program segfaults. i tried using cygdb but as i am not very versed in that kind of thing i'm a bit lost. this probably just is a bug in my code but i have not been able to find it for some time now. there is a possibility that something in the memoryview implementation could be buggy.

any hints are appreciated!

n.

in case anyone cares to look at a backtrace, i am attaching the cython and c source of the corresponding module and a cygdb backtrace output coming from the commands

(gdb) backtrace full
(gdb) info registers
(gdb) x/16i $pc
(gdb) thread apply all backtrace
(gdb) quit

(i found this somewhere; not sure if it's complete info)

here is the cdef class method; the segfault happens in a call to np.asarray(self._xcur):

cdef class _ItoFastPath:
    """ABC to collect the fast path functions together"""
    # temporary variables for use inside methods.
    cdef:
        np.ndarray _xcur_a
        float_t[::1] _xcur
        float_t _delta
        AbsCond _abs
        int have_abs
        Incr _incr_c
        Reflect _refl


    def _prop_abs_noref_fast(_ItoFastPath self, 
                             float_t tfinal, 
                             float_t tinit, 
                             float_t[::1] xinit):
        cdef AbsCond abs = None
        cdef int have_abs = self.have_abs
        if have_abs:
            abs = self._abs
        cdef Incr incr = self._incr_c  # Gaussian only for now
        cdef float_t tcur = tinit
        self._xcur = xinit.copy()   # no overwriting!
        if not tfinal > tinit:
            return (tfinal, np.asarray(self._xcur))
        assert tfinal > tinit
        while tcur < tfinal - self._delta:
            incr.add_gauss(tcur, self._xcur, self._delta)
            if have_abs: 
                if abs.eval(tcur, self._xcur): # need for speed; shortcut 
                    raise Absorption(tcur, np.asarray(self._xcur))
            elif self.abs_cond(tcur, self._xcur):
                raise Absorption(tcur, np.asarray(self._xcur))
            tcur += self._delta
        incr.add_gauss(tcur, self._xcur, (tfinal - tcur))  # final incr
        if have_abs:
            if abs.eval(tcur, self._xcur):
                raise Absorption(tcur, np.asarray(self._xcur))
        elif self.abs_cond(tcur, self._xcur):
            raise Absorption(tcur, np.asarray(self._xcur))
        assert np.ndim(np.asarray(self._xcur)) == 1   # <--- this is where it segfaults. (see #20 in the backtrace)
        return (tfinal, np.asarray(self._xcur, dtype=np.float))

[becker.nils]

oh, and i am using cython 0.16, python 2.7.2, numpy 1.6.1

[mark florisson]

On 23 April 2012 15:58, becker.nils <becke...@gmx.net> wrote:
> hi,
>
> i'm having trouble debugging cython code using memoryviews; basically, a
> loop calls the function given below thousands of times and then at some
> point the program segfaults. i tried using cygdb but as i am not very versed
> in that kind of thing i'm a bit lost. this probably just is a bug in my code
> but i have not been able to find it for some time now. there is a
> possibility that something in the memoryview implementation could be buggy.

It's hard to say, as it is not very self-contained. You could start by
testing with normal buffers instead of memoryviews and see if it still
fails, in which case it will be nice to come up with a minimal working
example that fails so we can isolate the memoryview problem. You could
also use cy bt or cy bt -a to get a useful backtrace (or thread apply
all cy bt), gdb's backtrace command becomes somewhat useless as it is
then printing out all sorts of stuff you don't care about.

[becker.nils]

hi again,

i have now isolated what could have been the cause of the problems before. check this out:

--- test.pyx

import cython
cimport cython
import numpy as np
cimport numpy as np


def test1():
    print np.random.rand(3).dtype
    cdef np.float64_t[::1] aslice = np.random.rand(2)
    cdef float[::1] bslice = np.random.rand(2)   

def test2(nn):
    cdef double[::1] aslice = np.random.rand(2)
    cdef double[::1] bslice
    cdef int ii
    print np.asarray(aslice)
    for ii in range(nn):
        bslice = aslice.copy()
    print 'success.'


now in a console:

>from test import test1, test2

>test2(10)

runs through but

>test2(1000)

segfaults. this is on python 2.7.2, cython 0.16.


the other test function may or may not be related. when i call test1(), i get

ValueError: Buffer dtype mismatch, expected 'float' but got 'double'

when assigning to bslice. (aslice works.) i found that surprising since i though float and np.float are compatible.

n.

[mark florisson]

Yes, np.float is actually a np.float64, just like np.double :)

> n.
>
>

Thanks for the report, that's is very useful. It seems the initial
memoryview implementation had a reference count error. Can you try
again with the Cython from github?
https://github.com/cython/cython/commit/3d5bc03f1e7f4e3f34cec3131fe5cb0d0a177a18

[becker.nils]

Thanks for the report, that's is very useful. It seems the initial
memoryview implementation had a reference count error. Can you try
again with the Cython from github?
https://github.com/cython/cython/commit/3d5bc03f1e7f4e3f34cec3131fe5cb0d0a177a18

i did. everything seems to be working now. the test function as well as my original code that segfaulted before (posted (partially) at the beginning of the thread)

thanks!

n.