Correct way to deal with Exceptions in C++ and Cython code

[John Tyree]

Hello everyone,

I have some classes in C++ which can throw exceptions. I have made pxd files for them and can use them successfully from Cython and Python. However, I noticed that an `assert` in a __delloc__ block was ignored as it was running. I looked at the generated code and saw that __dalloc__ is just another function, not a destructor or anything, so this was slightly confusing. After some shuffling around, I managed to cause python to segfault when exceptions were thrown. That made me even more confused. I found the one paragraph about Exceptions that's in the Cython wiki, but it was too incomplete to make heads or tails of it for my particular case. All of this has led to the following questions.

Given that both the C++ code *and* the Cython code wrapping it can each raise exceptions (via assert in Cython or whatever else), where is the correct place to tell Cython this?  In the *.pxd file I have the cdef cppclass ...: declarations marked with "except +" as is shown in the rectangle example on the wiki. Is it necessary, then, to also declare the Cython wrapper functions with except +?

If a Cython function is declared in the pxd file with except +, is it also required at the function definition site? For example if the foo.pxd contains  a cython cdef cppclass with a method bar and I declare it in a cdef cppclass block as "bar(int) except +", do I also need to define it with "cpdef bar(int) except +:" in the pyx file? What if it's just a regular "def" instead of "cpdef"? What if it's a "cdef" alone? If I put "except +" in all three places (class declaration, class definition, C++ class declaration), then python *usually* segfaults it seems, so I doubt that's the right way to do it.

What has been everyone's experience when working with Exceptions? Where do the annotations live?

-John

[Robert Bradshaw]

It is dangerous, as you have noticed, to let C++ exceptions propagate
through the Python call stack, including Cython code. Instead, you
should throw Python exceptions, and declare all your C++ library
functions (presumably in a cdef extern block in a pxd file) as except+
to translate them into Python exceptions at this boundary. It's not
safe to try to throw C++ exceptions from a Cython method.

If your C++ code is taking a Cython function as a callback and expects
*it* to throw exceptions, create a wrapper that catches a Python
exception and raises the appropriate C++ exception for your library to
handle. (See the Python/C API for how to do this, and also if there's
an "except return_value" that can be used to detect whether an
exception was raised.) This should be pretty rare.

- Robert

[John Tyree]

> --
>
> ---

Maybe I wasn't totally clear, or maybe I'm misunderstanding what you're saying.
I *am* throwing C++ exceptions from C++ code and trying to convert them with
Cython into Python exceptions.

I see segfaults if the cppclass declaration has except+ annotations AND the
cython wrapper class has them as well.

I have attached a small example.

python setup.py build_ext --inplace && python main.py

-John

[Robert Bradshaw]

A cdef class method should never have except+ annotations, it's a bug
to allow them. Take it out and your code should be fine.

- Robert

[John Tyree]

Ah I see, fair enough. Is this a trivial fix? Something a new person
interested in hacking on the compiler could attempt?

-John

[John Tyree]

Just so I am clear on this, do you mean to say that functions which are implemented in Cython and which throw Python exceptions (for example, from assert False) should still not be declared with except +?  I ask because I just noticed a lot of "AssertionError ignored..." flying by. When I added except + to the functions throwing AssertionErrors, they were correctly caught by Python.

- John

[John Tyree]

Forgot to include an example. This is a free function, not a class method.

 

cdef SizedArray[double]* to_SizedArray(np.ndarray v, cpp_string name) except +:

    assert v.dtype.type == np.float64, ("Types don't match! Got (%s) expected (%s)."

                                      % (v.dtype.type, np.float64))

    if not v.flags.c_contiguous:

        v = v.copy("C")

    return new SizedArray[double](<double *>np.PyArray_DATA(v), v.ndim, v.shape, name, True)

So in that function, the Assertion is explicitly ignored if I remove "except +".  Is it only for class methods that it's forbidden?

- John

[Robert Bradshaw]

Correct. They should be declared "except *" or "except [special
value][?]" or, if they return a Python object, without an except
clause at all.

http://docs.cython.org/src/userguide/language_basics.html#error-return-values

> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "cython-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cython-users...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Robert Bradshaw]

No, call that "except NULL"