Cython and CRTP classes

[Hannes Ovrén]

I am trying to wrap a set of C++ classes with Cython which are using the curiously recurring template pattern (CRTP) but I can't seem to get Cython to accept wrapping it.

The problem gets perhaps slightly more problematic since the classes are also templated over a type T, although I only want to use the double version in my Python wrapper.

I have made a simpler set of test classes to show what I want to do.

classes.h

#include <iostream>
#include <string>


template<typename T, class Derived>
class Base {
public:
    Base(T x) : x_(x) {
        std::cout << "Base::Base(" << x << ")" << std::endl;
    };
   
    Derived half() {
        Derived d(x_ / 2);
        return d;
    };
   
    virtual T calculate() = 0;
   
protected:
    T x_;
};


template<typename T>
class Foo : public Base<T, Foo<T>> {
public:
    Foo(T x) : Base<T, Foo<T>>(x) {
        std::cout << "Foo::Foo(" << x << ")" << std::endl;
    };
   
    T calculate() { return this->x_ * this->x_; }
};


template<typename T>
class Bar : public Base<T, Bar<T>> {
public:
    Bar(T x) : Base<T, Bar<T>>(x) {
        std::cout << "Bar::Bar(" << x << ")" << std::endl;
    };
   
    T calculate() { return this->x_; }
};

The reason for using CRTP is the Base::half() function since it needs to return the current (derived) class, and not the base class. A test C++ file using these classes is this:

#include "classes.h"
#include <iostream>


int main(int argc, char**argv) {
    std::cout << "Testing Foo squarer" << std::endl;
    Foo<double> foo(5);
    std::cout << "calculate: " << foo.calculate() << std::endl;    
    std::cout << "half calculate: " << foo.half().calculate() << std::endl;
   
    std::cout << "Testing Bar" << std::endl;
    Bar<double> bar(5);
    std::cout << "calculate: " << bar.calculate() << std::endl;    
    std::cout << "half calculate: " << bar.half().calculate() << std::endl;
}

I created a cython classes.pxd file to reflect my classes:

cdef extern from "classes.h":
    cdef cppclass Base[T, Derived]:
        Base(T)
        Derived half()
        T calculate()
   
    cdef cppclass Foo[T](Base[T, Foo[T]]):
        Foo(T)

    cdef cppclass Bar[T](Base[T, Bar[T]]):
        Bar(T)

and a pyclasses.pyx file for the wrapper classes. Note how I now want to specialize to T=double:

# distutils: language = c++


cimport classes


cdef class PyBase:
    cdef classes.Base* wrapped
   
    def calculate(self):
        return <double> self.wrapped.calculate()
   
    def __repr__(self):
        return '<{} {:f}>'.format(self.__class__.__name__, self.calculate())


cdef class PyFoo(PyBase):
   
    def __cinit__(self, double x):
        self.wrapped = <classes.Base*> new classes.Foo[double](x)
       
cdef class PyBar(PyBase):
   
    def __cinit__(self, double x):
        self.wrapped = <classes.Base*> new classes.Bar[double](x)

Unfortunately, I hit a problem trying to compile it:

Error compiling Cython file:
------------------------------------------------------------
...
    cdef cppclass Base[T, Derived]:
        Base(T)
        Derived half()
        T calculate()
   
    cdef cppclass Foo[T](Base[T, Foo[T]]):
                                    ^
------------------------------------------------------------

classes.pxd:8:37: Compiler crash in AnalyseDeclarationsTransform

File 'ModuleNode.py', line 116, in analyse_declarations: ModuleNode(classes.pxd:2:0,
    full_module_name = 'classes',
    is_pxd = True)
File 'Nodes.py', line 424, in analyse_declarations: StatListNode(classes.pxd:2:5)
File 'Nodes.py', line 480, in analyse_declarations: CDefExternNode(classes.pxd:2:5,
    include_file = 'classes.h')
File 'Nodes.py', line 424, in analyse_declarations: StatListNode(classes.pxd:3:4)
File 'Nodes.py', line 1438, in analyse_declarations: CppClassNode(classes.pxd:8:9,
    base_classes = [...]/1,
    in_pxd = True,
    name = 'Foo',
    templates = [...]/1,
    visibility = 'extern')
File 'Nodes.py', line 1117, in analyse: TemplatedTypeNode(classes.pxd:8:30)
File 'ExprNodes.py', line 3225, in analyse_as_type: IndexNode(classes.pxd:8:37,
    is_subscript = True,
    result_is_used = True,
    use_managed_ref = True)
File 'Nodes.py', line 1122, in analyse: TemplatedTypeNode(classes.pxd:8:37)

Compiler crash traceback from this point on:
  File "/home/hannes/miniconda/envs/rssim3/lib/python3.5/site-packages/Cython/Compiler/Nodes.py", line 1122, in analyse
    self.type = base_type.specialize_here(self.pos, template_types)
  File "/home/hannes/miniconda/envs/rssim3/lib/python3.5/site-packages/Cython/Compiler/PyrexTypes.py", line 3602, in specialize_here
    return self.specialize(dict(zip(self.templates, template_values)))
  File "/home/hannes/miniconda/envs/rssim3/lib/python3.5/site-packages/Cython/Compiler/PyrexTypes.py", line 3620, in specialize
    specialized.scope = self.scope.specialize(values, specialized)
AttributeError: 'NoneType' object has no attribute 'specialize'

Am I declaring the subclasses wrong in the PXD-file, or is Cython currently unable to handle this kind of CRTP-style self templating?

Any help regarding this is most welcome. Thanks!

Regards,

Hannes Ovrén 

[Robert Bradshaw]

On Thu, Sep 8, 2016 at 4:48 AM, Hannes Ovrén <kig...@gmail.com> wrote:
> I am trying to wrap a set of C++ classes with Cython which are using the
> curiously recurring template pattern (CRTP) but I can't seem to get Cython
> to accept wrapping it.
>

> Am I declaring the subclasses wrong in the PXD-file, or is Cython currently
> unable to handle this kind of CRTP-style self templating?
>
> Any help regarding this is most welcome. Thanks!

This was unsupported, fixed at
https://github.com/cython/cython/commit/1f98bcd70568049442e4fe8e513113032166257a
.

Note, however, you can't declare a pointer to the untemplated Base
type (there isn't such a type in C++, though you could give it a base
and put the calculate method there). Of course this is probably just
an oversimplification of the real code...

[Hannes Ovrén]

Hi!

Thanks for the quick reply, and fix!

How should the baseclass pointer be declared in Cython?

It should be able to accomodate both Foo<double> and Bar<double> subclasses.

I admit that I am not really sure how I would do this in C++ either.

Also, my subclasses have some methods of their own, which are not shared, which we can call

Foo::do_foo() and Bar::do_bar() that need to be exposed by the Python wrapper class.

I guess I can solve this by casting the base pointer like this:

class PyFoo(Base):
    ...

    def foo(self):
        return (<classes.Foo[double]*> self.wrapped).do_foo()

/Hannes

[Robert Bradshaw]

On Fri, Sep 9, 2016 at 6:27 AM, Hannes Ovrén <kig...@gmail.com> wrote:
> Hi!
>
> Thanks for the quick reply, and fix!
>
> How should the baseclass pointer be declared in Cython?
> It should be able to accomodate both Foo<double> and Bar<double> subclasses.
> I admit that I am not really sure how I would do this in C++ either.

The only way to do this is to have a superclass of Base that is not
templatized on Derived (in C++ or Cython).

> Also, my subclasses have some methods of their own, which are not shared,
> which we can call
> Foo::do_foo() and Bar::do_bar() that need to be exposed by the Python
> wrapper class.
> I guess I can solve this by casting the base pointer like this:
>
> class PyFoo(Base):
> ...
>
> def foo(self):
> return (<classes.Foo[double]*> self.wrapped).do_foo()

Yes, you have to do a cast. If it makes life easier, you could store a
copy to the specialized pointer in your subclass, e.g

class PyFoo(Base):
cdef Foo* foo_ptr
def __init__(self, ...):
super(PyFoo, self).__init__(...)
self.foo_ptr = <classes.Foo[double*]> self.wrapped
def foo(self):
return foo.do_foo()

> On Friday, September 9, 2016 at 9:48:19 AM UTC+2, Robert Bradshaw wrote:
>>
>> On Thu, Sep 8, 2016 at 4:48 AM, Hannes Ovrén <kig...@gmail.com> wrote:
>> > I am trying to wrap a set of C++ classes with Cython which are using the
>> > curiously recurring template pattern (CRTP) but I can't seem to get
>> > Cython
>> > to accept wrapping it.
>> >
>> > Am I declaring the subclasses wrong in the PXD-file, or is Cython
>> > currently
>> > unable to handle this kind of CRTP-style self templating?
>> >
>> > Any help regarding this is most welcome. Thanks!
>>
>> This was unsupported, fixed at
>>
>> https://github.com/cython/cython/commit/1f98bcd70568049442e4fe8e513113032166257a
>> .
>>
>> Note, however, you can't declare a pointer to the untemplated Base
>> type (there isn't such a type in C++, though you could give it a base
>> and put the calculate method there). Of course this is probably just
>> an oversimplification of the real code...
>

> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "cython-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cython-users...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.