Wrapping a complicated C++ library

[Pedro Camargo]

Hello,

             I am currently working on the next steps of a transportation modelling open source project, and one of the possible new undertakings is to wrap some pretty good C++ code to be used within our Python framework (which ties back to QGIS).

Before I start doing it, I would like to know if you can give me a pointer on how difficult it would be to wrap the code for the following repository:

https://github.com/olga-perederieieva/TAsK

I am guessing it is pretty difficult, since no wrapping of Boost Graph Libraries exist for Windows (Graph-tools does it for linux/mac), but I would really appreciate if somebody could take a quick look and give me an idea of effort involved on this.

Thanks in advance,

Pedro

[Ian Bell]

I know this is a cython mailing list, but have you considered other alternatives that may or may not make your life easier:

a) SWIG

b) pybind11

c) boost-python

I've been pretty happy with them for fairly complex API (especially SWIG and pybind11), where hand-building wrappers in cython becomes progressively more and more painful as the API becomes more complex.

YMMV though.

Ian

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kevin Thornton]

Ian's advice is sound--it comes down to how much "boiler plate" code you want to write/manage.

For modern C++, pybind11 is excellent.  A major difference b/w it and Cython is that the latter has automatic struct-to-dict conversion.  This feature is really handy for the user's of your package--functions can return things that "just work" in Python, and those same dicts can go right back into C/C++ functions expecting the struct type.

This feature is why we've stuck with Cython, and so we have to write some extra C++ code when move semantics, etc., are needed.

--Kevin

[Hai Nguyen]

hi

I am not familiar with that repo but I did the Cython stuff with several  C++ libraries before.

I think as long as you have a libtask.so file, a (or several) C++ header and an well defined API, you're good to go.

But it really depends on which C++ features you need to expose to Python.

Hai

[Chris Barker]

my  thoughts (having done this):

On Tue, Aug 16, 2016 at 11:34 PM, Ian Bell <ian.h...@gmail.com> wrote:

I know this is a cython mailing list, but have you considered other alternatives that may or may not make your life easier:

a) SWIG

SWIG is pretty cool, and makes the easy stuff really easy, But most of a big C++ ib won't be easy, so you need to write a lot of "glue" code to tell SWIG how to wrap your lib. And it gets really painful if you want to write python as part of the bindings (painful to debug anyway). And you end up with fairly "thick" wrappers -- SWIG-ified pointers, etc...

As an example, the wxPython project was built entirely with SWIG -- and that's a gib one. But when wanting to refactor it, Robin Dunn choose to go to SIP instead -- and hardly anyone knows more about SWIG than Robin.

(so maybe look at SIP :-) )

b) pybind11

c) boost-python

Pretty cool -- I hadn't seen pybind before, though it sounds similar in spirit to boost-python

The thing about boost-python, is that it seems to be designed to write extensions in C++, rather than bind, though, of course, your C++ extensions can call C++ -- so it should do a lot of the boiler plate for you.

However -- it depends a fair bit on what you want -- I tend to want a "pythonic" API -- so I need to write fairly "thick" wrappers -- exposing the functionality n a Pythonic way, using Python types as much as possible, etc, so the libs users can't tell that there is C++ code underneath.

In this case, Cyton shines -- you can write those thick wrapper is a very Python-like language -- work with Python types natively, etc -- really nice.

But you do end up with a lot of boilerplate to write, yes. As I said, I tend to want to write thick wrapper,s which require boilerplate anyway, so not so bad.

And aside from the boredom factor, you need to carefully work through everything to write tests, etc, so it's not all that much more work.

I'd take a look at XDress too -- automatically writes Cython wrappers around C++ code -- best of both worlds! It looked really promising -- but alas, doesn't look like it's being actively developed :-(

https://github.com/xdress/xdress

I think it comes down to how thick you want your wrappers to be.....

And/or how much of the API you need to expose -- sometimes there is a lot going on under the hood, but you only need Python to talk to a higher level setup-the-model-and-run API.

-CHB

https://github.com/olga-perederieieva/TAsK

I am guessing it is pretty difficult, since no wrapping of Boost Graph Libraries exist for Windows (Graph-tools does it for linux/mac), but I would really appreciate if somebody could take a quick look and give me an idea of effort involved on this.

Thanks in advance,

Pedro

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris....@noaa.gov

[Robert Bradshaw]

> In this case, Cython shines -- you can write those thick wrapper is a very

> Python-like language -- work with Python types natively, etc -- really nice.
>
> But you do end up with a lot of boilerplate to write, yes. As I said, I tend
> to want to write thick wrapper,s which require boilerplate anyway, so not so
> bad.

+1 to all of this.

Cython isn't a wrapper-generator. It's a tool that lets you invoke
C/C++as if from Python, which of course can be used to write wrappers
if desired (though often you want to simply use a library, not
necessarily wrap it).

> And aside from the boredom factor, you need to carefully work through
> everything to write tests, etc, so it's not all that much more work.
>
> I'd take a look at XDress too -- automatically writes Cython wrappers around
> C++ code -- best of both worlds! It looked really promising -- but alas,
> doesn't look like it's being actively developed :-(
>
> https://github.com/xdress/xdress

See also https://github.com/cython/cython/wiki/AutoPxd .
Unfortunately, there's no solid, actively developed solution for
automatically wrapping C++ in Python (with or without Cython)--all
require a fair amount of handholding/boilerplate for anything
non-trivial, though with Cython you can use this opportunity to craft
thick yet efficient bindings that span the impedance mismatch between
a typical C++ API and a natural, Pythonic one (which mismatch may be
part of the inherently difficulty in automatically wrapping C++).

> I think it comes down to how thick you want your wrappers to be.....
>
> And/or how much of the API you need to expose -- sometimes there is a lot
> going on under the hood, but you only need Python to talk to a higher level
> setup-the-model-and-run API.

And, looking at TAsK itself, this would likely be the case. The
primary user "API" for TAsK is to compile the binary and invoke it
with a set of text file inputs. Were I wrapping this library, I would
probably focus on a nice, Pythonic API for building up the
configuration (possibly with nice validation), and then run it by
spitting out the config as temporary files in the expected format,
invoking TAsK via subprocess, and the reading the results back in.

One could go a step further, essentially implementing
https://github.com/olga-perederieieva/TAsK/blob/master/src/main.cpp in
Cython (declaring just enough of the API do to so), but that may
involve changes to TAsK itself as the assumption that input and
outputs are via physical files seems pretty baked in. It's also
unclear what the benefit would be.

(If you're trying to wrap/expose the internal implementation details,
that's a different story and the more generic comments above are more
relevant).

- Robert