Extended numpy array declaration syntax and segfaults with Exceptions

[Matthew Brett]

Hi,

I have been getting some odd errors when using the nice extended array
declaration syntax.

I've put a self-contained example in the attached archive (cd
cython_np_syntax && make ext && python run_tests.py)

In brief - with the following file:

import numpy as np
cimport numpy as cnp

def error_testing():
cdef:
cnp.ndarray[cnp.float64_t] tmp = np.zeros((1,))
raise ValueError('An error')

- when running ``error_testing()`` - I always get:

run_tests.py:6: RuntimeWarning: tp_compare didn't return -1 or -2 for exception
error_testing()

the first time I run it. The second time I run it, the effect seems
to be sensitive to where I am running it (within ipython, in bare
python, within nosetests). For example ``python run_tests.py`` in
the attached archive:

from ndarray_test import error_testing

for i in range(2):
try:
error_testing()
except ValueError:
pass

- with my system - leads to a reliable segmenation fault. Running
``nosetests test_errors.py`` (same archive) - gives a variety of
different errors on different runs, e.g:

[mb312@blair-2 ~/tmp/cython_np_syntax]$ nosetests test_errors.py
/Users/mb312/usr/local/lib/python2.6/unittest.py:336: RuntimeWarning:
tp_compare didn't return -1 or -2 for exception
callableObj(*args, **kwargs)
.python(53201,0x7fff71048be0) malloc: *** error for object
0x102494830: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap
[mb312@blair-2 ~/tmp/cython_np_syntax]$ nosetests test_errors.py
/Users/mb312/usr/local/lib/python2.6/unittest.py:336: RuntimeWarning:
tp_compare didn't return -1 or -2 for exception
callableObj(*args, **kwargs)
.E
======================================================================
ERROR: Failure: ValueError (Buffer dtype mismatch, expected
'numpy.float64_t' but got end)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/mb312/usr/local/lib/python2.6/site-packages/nose-0.11.1-py2.6.egg/nose/loader.py",
line 224, in generate
for test in g():
File "/Users/mb312/tmp/cython_np_syntax/test_errors.py", line 11, in
test_testing
error_testing()
File "ndarray_test.pyx", line 7, in ndarray_test.error_testing
(ndarray_test.c:768)
cnp.ndarray[cnp.float64_t, ndim=1] tmp = np.zeros((1,))
ValueError: Buffer dtype mismatch, expected 'numpy.float64_t' but got end

----------------------------------------------------------------------
Ran 2 tests in 0.002s

FAILED (errors=1)

I don't get the error without the extended array declaration syntax -
for example:

def error_testing():
cdef:
cnp.ndarray tmp = np.zeros((1,))
raise ValueError('An error')

behaves as expected.

I've scanned the difference between the generated c with and without
the extended declaration syntax but nothing jumped out at me except my
feeling I should ask for help.

Is there anything else I can do to help debug the problem?

Thanks a lot,

Matthew

[Kurt Smith]

On Fri, Jun 25, 2010 at 2:41 PM, Matthew Brett <matthe...@gmail.com> wrote:
> Hi,
>
> I have been getting some odd errors when using the nice extended array
> declaration syntax.
>
> I've put a self-contained example in the attached archive (cd
> cython_np_syntax && make ext && python run_tests.py)

Thanks for the nice bug report. Unfortunately I'm unable to reproduce
it, maybe others will have more 'luck'.

I'm on cython 0.12.1, numpy 1.4.0, python 2.6.4. What versions of
cython, numpy & python are you using?

If you remove the 'raise' line does the code compile & run?

[Matthew Brett]

Hi,

>> I have been getting some odd errors when using the nice extended array
>> declaration syntax.
>>
>> I've put a self-contained example in the attached archive (cd
>> cython_np_syntax && make ext && python run_tests.py)
>
> Thanks for the nice bug report.  Unfortunately I'm unable to reproduce
> it, maybe others will have more 'luck'.
>
> I'm on cython 0.12.1, numpy 1.4.0, python 2.6.4. What versions of
> cython, numpy & python are you using?

Ah - yes - thanks - the internet is very rewarding. I have Cython
0.12.1, python 2.6.4, but numpy 2.0.0.dev8445. I see that reverting
to numpy 1.4.0 does indeed make the odd errors go away. So I suppose
this may be an incompatibility between Cython and relatively recent
numpy trunk.

> If you remove the 'raise' line does the code compile & run?

It does...

Thanks a lot,

Matthew

[Kurt Smith]

On Fri, Jun 25, 2010 at 3:45 PM, Matthew Brett <matthe...@gmail.com> wrote:

> Ah - yes - thanks - the internet is very rewarding.   I have Cython
> 0.12.1, python 2.6.4, but numpy 2.0.0.dev8445.  I see that reverting
> to numpy 1.4.0 does indeed make the odd errors go away.  So I suppose
> this may be an incompatibility between Cython and relatively recent
> numpy trunk.

This is worth keeping an eye on, thanks for the heads up.