Stan Math Library: arXiv paper & standalone distro

[Bob Carpenter]

The Stan math lib paper is up on arXiv:

The Stan Math Library: Reverse-Mode Automatic
Differentiation in C++
Bob Carpenter, Matthew D. Hoffman, Marcus Brubaker,
Daniel Lee, Peter Li, Michael Betancourt

http://arxiv.org/abs/1509.07164

It explains how reverse-mode autodiff works to compute
gradients, then goes into gory detail of every aspect of
Stan's implementation from arena-based memory management at
the bottom to template metaprogramming for vectorization and
nested autodiff for the ODE solver on the top.

Let us know if you have comments, such as how we might
condense it into a 25 page paper or two for a journal.

Stan's math library is now a fully modular repository
of C++ code (with Boost and Eigen dependencies).
Like everything else, it's released under the BSD
in our GitHub organization:

https://github.com/stan-dev/math

This should make it much easier for people to use and contribute
to. The RStan devs may release a separate header package for the
Stan Math Library---we'll keep you posted.

- Bob

[Bob Carpenter]

> On Sep 30, 2015, at 5:58 AM, Ashley Ford <ashley...@gmail.com> wrote:
>
> An interesting paper,

Thanks.

> (I have only read the first third so far),

Sorry for my tendency to write in the long form!

> 2 questions.
> 1. Should an Rcpp interface to the library be reasonably straightforward ?

What do you want to do with it from R? The point of the library
is to let you take derivatives of templated C++ programs.

> 2. on my CentOS 6.7 system g++ is rather old
> g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-16)
> and although the simple example from the paper works, at least one test in make test-headers fails (its 206/866) error below.

Thanks for the careful reporting. We should've included <stdexcept> in
that file. No idea why the header tests are passing elsewhere --- I wonder if
it's a bug in the way they are configured externally with a driver program
including things it shouldn't. I created an issue, which I'm sure Daniel
will be able to sort out. Here's the tracker:

https://github.com/stan-dev/math/issues/186

I'll make sure to thank you in our acknowledgements, too.

> Can I assume if my programs compile OK with this compiler they are OK or do I need to get an alternative version of gcc ?

We've had issues on and off with g++ 4.4, which seems
to be somewhat of an outlier in terms of behavior.

I would update your GCC for two reasons: better behavior with
recent code, faster compilation, and generation of more efficient code.
Better yet, get a recent version of clang++ --- it compiles way faster
and generates code for Stan that's just as efficient at -O3 --- and we
rely heavily on compiler optimizations in Stan Math.

- Bob