ADIC: auto diff for c sources?
Sebastian Weber
I just found ADIC: http://www.mcs.anl.gov/research/projects/adic/
It is a pity that the license is not usable for Stan (maybe someone else has a look). But apart from that, the ADIC systems sounds amazing - you give it a piece of C source code and it generates you a c source code which gives you the partial derivatives. Having this, in particular, for the ODE integrator would be stunning.
Maybe there are similar systems out there already which have a better license?
Cheers,
Sebastian
Sebastian Weber
https://github.com/vgvassilev/clad
this thing is again not usable for Stan (LGPL) and only works with clang, but it reads very cool:
http://llvm.org/devmtg/2013-11/slides/Vassilev-Poster.pdf
Sebastian
Bob Carpenter
Theano (Python) does.
The problem is conditionals and branching. If we could get
rid of all of those, code generation is a no-brainer. But if
we'd done that, we woudn't have many of our matrix algorithms
and would've taken a lot longer to get ODEs out, etc.
We may still think about more in the future if we could
limit Stan's language's expressiveness.
- Bob
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
Bob Carpenter
templates do efficient forward mode, so you don't need to code
generate (or maybe I should say is that the template system gives
you all you need to code generate).
We need reverse mode for efficient gradients O(1) relative to
function evaluation rather than O(N) in forward mode.
Marcus Brubaker
Sebastian Weber
I am assuming here that doing code-generation will speed up the gradients considerably since the compiler can then take this code and make it optimized. Under this assumption: Is it thinkable to restrict the ODE system function in what is allowed and what not and do code generation for that? I mean, most ODE system functions are algebraically really simple and doing AD (which is great for its generality) is overkill.
I think even a much restricted set of operations which are allowed in a ODE system function would cover many cases. But of course, doing this "just" for ODEs... not sure.
Sebastian
PS: That this is a nobrainer to you... amazing. This looks like it would kill me.
On Tuesday, September 29, 2015 at 9:02:35 PM UTC+2, Bob Carpenter wrote:
> We've thought about code generation off and on. It's what
> Theano (Python) does.
>
> The problem is conditionals and branching. If we could get
> rid of all of those, code generation is a no-brainer. But if
> we'd done that, we woudn't have many of our matrix algorithms
> and would've taken a lot longer to get ODEs out, etc.
>
> We may still think about more in the future if we could
> limit Stan's language's expressiveness.
>
> - Bob
>
> > On Sep 29, 2015, at 3:40 AM,
> >
Sebastian Weber
Thanks - I am not an expert at all on this. So could LGPL code be distributed with stan? Or is redistribution not allowed, but a LGPL extension must be left in untouched tar files or whatever?
Sebastian
Bob Carpenter
not that it'd be easy to do! Basically, compile everything up
front that can be compiled. Code generation would follow the
way that autodiff works now. But we have lots of functions, so
it'd be a huge pain to get it going.
I don't want to make a restricted subsystem of things just
for ODEs. We have toyed with restricting all of Stan so we
could do this everywhere.
- Bob