GSoC project idea: Ruby bindings to the CSymPy C++ symbolic library

114 views
Skip to first unread message

Ondřej Čertík

unread,
Mar 3, 2015, 7:01:33 PM3/3/15
to sciru...@googlegroups.com
Hi,

There is an interest in the Ruby scientific community to have a
computer algebra system (symbolic manipulation) in Ruby, e.g. I am
quoting an email exchange from this list (here is the link to the full
thread: https://groups.google.com/d/topic/sciruby-dev/iVRxZX4m7cU/discussion).

I am the original author of SymPy and recently we started developing a
very fast C++ library for symbolic manipulation called CSymPy
(https://github.com/sympy/csympy), the Python wrappers which provide
seamless interoperation with SymPy are optional (in fact they are off
by default), thus it can be used from any other language like Ruby. We
are also planning Julia wrappers.

The idea is that having a single code in C++, without any dependency
on Python (or Ruby) allows multiple communities to use the code and
contribute to it, and still use it from their favorite language. Which
in the long run is much better than having a pure Python version
(SymPy), and then having to completely rewrite it in Ruby, or Julia,
or any other language that comes in the future. And using a Python
library from other languages is possible (Julia has a great Python
bridge), but it is not ideal (you have to have Python installed,
suddenly you have to deal with lots of cross language issues, incur
another overhead, ...), it could in principle be used from C++ or C,
but again, there are a lot of low level technical issues. In other
words, a Python library is great if you are using it from Python, but
not so great if you want to use it from other languages. C++ on the
other hand, while not ideal, delivers speed, and it delivers easy
usage as a library from any language.

To answer few questions raised by Sam below: SymPy is used in Sage
together with Maxima, roughly for the same kind of tasks (for example
for integrals, the user can choose which library to use). SymPy should
be able to do what Maxima can in terms of features and speed. SymPy
has lots of additional modules for classical and quantum mechanics
etc. Sage uses Pynac, based on GiNaC, because it is much faster.
However, CSymPy is now much faster than Pynac, you can run the
benchmarks yourselves and see (run the "_sage" and the Python or C++
benchmarks of the same kind):

https://github.com/sympy/csympy/tree/master/benchmarks

E.g. on my computer right now:

certik@redhawk:~/repos/csympy/benchmarks(master)$ sage legendre1_sage.py
import...
done.
0 1
1 x
2 3/2*x^2 - 1/2
3 5/2*x^3 - 3/2*x
4 35/8*x^4 - 15/4*x^2 + 3/8
5 63/8*x^5 - 35/4*x^3 + 15/8*x
6 231/16*x^6 - 315/16*x^4 + 105/16*x^2 - 5/16
7 429/16*x^7 - 693/16*x^5 + 315/16*x^3 - 35/16*x
8 6435/128*x^8 - 3003/32*x^6 + 3465/64*x^4 - 315/32*x^2 + 35/128
9 12155/128*x^9 - 6435/32*x^7 + 9009/64*x^5 - 1155/32*x^3 + 315/128*x
Total time for legendre(500, x): 7.2570168972 s
certik@redhawk:~/repos/csympy/benchmarks(master)$ python legendre1.py
0 1
1 x
2 -1/2 + 3/2*x**2
3 5/2*x**3 + -3/2*x
4 3/8 + -15/4*x**2 + 35/8*x**4
5 63/8*x**5 + 15/8*x + -35/4*x**3
6 -5/16 + 105/16*x**2 + 231/16*x**6 + -315/16*x**4
7 -693/16*x**5 + 429/16*x**7 + 315/16*x**3 + -35/16*x
8 35/128 + -315/32*x**2 + -3003/32*x**6 + 3465/64*x**4 + 6435/128*x**8
9 12155/128*x**9 + -6435/32*x**7 + 315/128*x + -1155/32*x**3 + 9009/64*x**5
Total time for legendre(500, x): 0.983409166336 s


You can check that the symbolic code is equivalent, it just evaluates
the Rodrigues' formula (obviously this is not the fastest way to get
Legendre polynomials, it's just for benchmarking general symbolic
code).

The reason Pynac is slower is probably because they use Sage's GMP
functionality, while we don't call any Python from the C++ code (it's
pure C++), precisely for this reason, no need to worry about any
potential Python overhead, plus it allows us to use it from other
languages, such as Ruby or Julia.

Another option is to wrap GiNaC directly, but CSymPy also seems to be
a bit faster than GiNaC itself by now (you can run "_ginac" benchmarks
above), we designed the code so that one can play with various data
structures and really make sure the code is fast. Also I think the
code is simpler, but obviously I am biased. In terms of people, CSymPy
is pretty active, e.g. compare GiNaC and SymPy for the last year:

csympy$ git shortlog -ns --since="1 year ago"
336 Ondřej Čertík
271 Thilina Bandara Rathnayake
238 Sushant Hiray
109 Isuru Fernando
10 Peter Brady
9 Shivam Vats
4 Sumith
3 Thomas Hisch
1 Vinzent Steinberg

ginac$ git shortlog -ns --since="1 year ago"
6 Richard Kreckel
6 Vladimir V. Kisil
5 Alexei Sheplyakov
3 Stefan Weinzierl


or all time (CSymPy from 2012, GiNaC from 1999):

csympy$ git shortlog -ns
1039 Ondřej Čertík
338 Thilina Bandara Rathnayake
238 Sushant Hiray
109 Isuru Fernando
10 Peter Brady
9 Shivam Vats
5 Dale Lukas Peterson
4 Sumith
3 Thomas Hisch
1 Christopher Dembia
1 Julien Rioux
1 Vinzent Steinberg


ginac$ git shortlog -ns
669 Richard Kreckel
650 Christian Bauer
279 Jens Vollinga
201 Alexei Sheplyakov
66 Chris Dams
32 Alexander Frink
20 Vladimir V. Kisil
9 Stefan Weinzierl
7 Jan Rheinländer
7 Richard B. Kreckel
3 welzel
1 Matthias Dellweg
1 Paul Irofti

Another difference is that GiNaC uses a GPL license, while CSymPy uses
an MIT license (similar to SciRuby's license).

Compared to GiNaC, CSymPy is missing specialized polynomial
manipulation and series expansion (we are working on both now) and
pattern matching (we also plan to do it soon, hopefully as another
GSoC project).

As to the GSoC project, we would need your help to figure out the best
way to write the wrappers, e.g.:

* Is it better to have the wrappers in the same repository or separate?
* Which tool should be used to wrap it? I found some options here:
https://www.amberbit.com/blog/2014/6/12/calling-c-cpp-from-ruby/,
CSymPy has C wrappers as well, so one can wrap either C or C++
directly.

We'll test it on our Travis with each CSymPy commit to ensure the
wrappers continue working as we develop CSymPy further.

I was contemplating whether to post here or not, as it is a little
early, we'll be in much better position feature wise at the end of the
summer, but GSoC is a great opportunity, and I think we are mature
enough for this to be successful and immediately useful to the Ruby
community.

I don't know how many slots you'll get and if you'll be able to host
this GSoC project (and if there is a good student application for it),
but even if there won't be any GSoC projects in the end, I hope to
start collaboration on this. If there is anybody interested, please
let us know. Also please let us know if you think this might be a good
fit to your GSoC organization.

Ondrej

On Wednesday, January 2, 2013 at 5:41:55 PM UTC-7, sam.rawlins wrote:
>
>
>
> On Monday, December 31, 2012 7:22:21 PM UTC-7, steven.e...@me.com wrote:
>>
>> Ya, I think they can live separate happily but I've only just started thinking about this project. Just started writing code a few days ago. Will do an init commit next week sometime of what I have.
>>
>> I've also been using NMatrix and am quite impressed with it. I have some questions though that I'm writing down as I use it. In terms of performance I think you're right! I'm sure I could use lots of help once we get rolling. I'll post back when the repo is up.
>
>
> I've long wanted to create th equivalent of Sage (http://www.sagemath.org/) for Ruby, which is even more ambitious than a Ruby equivalent to SymPy (since SymPy is one of ~100 components of Sage). In any case, Sage uses Maxima (http://maxima.sourceforge.net/) as it's primary symbolic manipulation engine, rather than SymPy (I think... I could be wrong...). Here (http://www.sagemath.org/doc/tutorial/tour_algebra.html) are some examples of Sage's algebra and calculus capabilities. Maxima is written in Lisp and can run on CL, so I don't think Sage has any FFI to Maxima or anything like that; it calls out to it as a different process.
>
> Sage also uses Pynac [http://pynac.org/] for its symbolic manipulation. Pynac is a Python interfact to GiNaC (http://www.ginac.de/) and CLN. Unfortunately, GiNaC's last release was in Nov '11, but maybe it's just a slow-moving project.
>
> So I think the best bang for your buck is going to be using an existing fast, vetted symbolic engine, like GiNaC or Maxima. Not that it'll be a piece of cake, but you will inherit a lot of power and functionality and decades of maturity if you go with one of these, rather than something written from scratch in Ruby.
>
> Other stuff I found googling:
> * https://github.com/y-ich/maxima-on-web a Sinatra-built web app for interfacing with Maxima through a browser.
> * GlucSym is an unpublished Ruby-to-GiNaC bridge [http://www.mathweb.org/wiki/GlucSym] from 2008. Not holding my breath.
>
> In any case, good luck!
>>
>>
>> Thanks!
>> Steve
>>
>> On Dec 31, 2012, at 8:20 PM, Carlos Agarie <carlos...@gmail.com> wrote:
>>
>> Well, from what I read, SymPy is focused in symbolic algebra - so there's not really overlap between it and NMatrix, which is a *numerical* linear algebra library.
>>
>> And it'd be great to have it exclusively in Ruby. I'm not sure, but I don't think there's much need for performance in symbolic calculations.
>>
>> But first let's see what "symruby" wants to do. One thing at a time.
>>
>>
>> -----
>> Carlos Agarie
>>
>> Control engineering
>> Polytechnic School, University of São Paulo, Brazil
>> Computer engineering
>> Embry-Riddle Aeronautical University, USA
>>
>>
>> 2012/12/31 Eric MacAdie <emac...@gmail.com>
>>>
>>> sympy seems to have some linear algebra, which is already in SciRuby. Should there be some overlap? Or would it be good to have a library for LA that does not need ATLAS installed?
>>>
>>> - Eric MacAdie
>>>
>>>
>>> On Mon, Dec 31, 2012 at 1:42 PM, John Prince <jtpr...@gmail.com> wrote:
>>>>
>>>> I think some kind of clone of sympy functionality would be fantastic. I think it would be a very welcome contribution.
>>>
>>>
>>

John Woods

unread,
Mar 3, 2015, 8:46:51 PM3/3/15
to sciru...@googlegroups.com
Hi Ondřej,

I think we met at the Mentor Summit two years ago, possibly? Yes?

I believe this would be an excellent idea for SciRuby. It doesn't particularly matter which repository houses the Ruby gem, IMHO. I wrote a wrapper for FLANN, for example, which also has a Python API, and included that in the same repository as the C/C++/Python code — and it's been fine. The only caveat is that the Ruby code has to be compiled separately, though there may be a workaround of which I am unaware.

In answer to your other question, I've written Ruby–C/C++ wrappers in three ways — using Rice, using FFI, or by hand. I think FFI and by hand are both good options. By hand nearly always yields a more Ruby-like experience, but most of that can also be accomplished with some creative use of FFI. I've never tried Inline, but my guess is it lies somewhere between FFI and by-hand in terms of its utility.

Are you also volunteering to mentor, or is this something where we'd need to supply a mentor?

Thanks so much for reaching out!

John


--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Ondřej Čertík

unread,
Mar 3, 2015, 10:22:02 PM3/3/15
to sciru...@googlegroups.com
Hi John,

On Tue, Mar 3, 2015 at 6:46 PM, John Woods <john.o...@gmail.com> wrote:
> Hi Ondřej,
>
> I think we met at the Mentor Summit two years ago, possibly? Yes?

Ah yes, I was there. I was there also the last year.

>
> I believe this would be an excellent idea for SciRuby. It doesn't
> particularly matter which repository houses the Ruby gem, IMHO. I wrote a
> wrapper for FLANN, for example, which also has a Python API, and included
> that in the same repository as the C/C++/Python code — and it's been fine.
> The only caveat is that the Ruby code has to be compiled separately, though
> there may be a workaround of which I am unaware.

I see. So one would first install the library, and then go let's say into

wrappers/ruby

And install the wrappers. I think it's better if it is in the same
repository, as then we can simply keep it updated with any changes to
the C++ code.

>
> In answer to your other question, I've written Ruby–C/C++ wrappers in three
> ways — using Rice, using FFI, or by hand. I think FFI and by hand are both
> good options. By hand nearly always yields a more Ruby-like experience, but
> most of that can also be accomplished with some creative use of FFI. I've
> never tried Inline, but my guess is it lies somewhere between FFI and
> by-hand in terms of its utility.

I see. I think it's similar with Python. Writing by hand is not a bad
option, it is not terribly difficult. I actually use Cython, which
generates code similar to a hand written code, but otherwise looks
like Python.

>
> Are you also volunteering to mentor, or is this something where we'd need to
> supply a mentor?

Yes, I would be very happy to mentor it.

> Thanks so much for reaching out!

Thanks. I'll let our students know about this option so that they can
apply for it.

Thanks,
Ondrej

Carlos Agarie

unread,
Mar 4, 2015, 8:24:17 AM3/4/15
to sciru...@googlegroups.com
Hi Ondrej,

This seems like a very interesting project. Can you provide a description for it on SciRuby's project ideas page? You can find it in:

https://github.com/SciRuby/sciruby/wiki/Google-Summer-of-Code-2015-Ideas


-----
Carlos Agarie
Software Engineer
+55 11 97320-3878 | @carlos_agarie

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev...@googlegroups.com.

Ondřej Čertík

unread,
Mar 4, 2015, 11:21:07 AM3/4/15
to sciru...@googlegroups.com
Hi Carlos,

On Wed, Mar 4, 2015 at 6:23 AM, Carlos Agarie <carlos...@gmail.com> wrote:
> Hi Ondrej,
>
> This seems like a very interesting project. Can you provide a description
> for it on SciRuby's project ideas page? You can find it in:
>
> https://github.com/SciRuby/sciruby/wiki/Google-Summer-of-Code-2015-Ideas

Excellent, I've added it at the end:

https://github.com/SciRuby/sciruby/wiki/Google-Summer-of-Code-2015-Ideas#ruby-bindings-to-the-csympy-c-symbolic-library

Thanks,
Ondrej
Reply all
Reply to author
Forward
0 new messages