Stan as a probabilistic programming language

[Dustin Tran]

I’m at MLSS over in Tuebingen (http://mlss.tuebingen.mpg.de/2015/), and the misconception that Stan’ isn’t “really” a probabilistic programming language is still quite prevalent.  Zoubin gave a talk yesterday mentioning a bunch of probabilistic programming languages, “Church, Venture, Anglican, Stochastic Python, Julia, …", and grouped Stan’s expressivity as in the line of JAGS and BUGS. I’ve also been talking with Frank and a few of his students here.

Three main beliefs seem to exist which lead to people believing this way: that Stan isn’t Turing complete, that it’s more like JAGS/BUGS than Church/Venture/Anglican, and that it can’t handle discrete parameters. The first one is wrong, the second “true” but irrelevant, and the third a problem with our current inference algorithms rather than the full generality of the Stan language. I put up this note in case anyone else may find this useful. It seems like if we just implement a generic MH/SMC that no one will actually use, we can more confidently say that Stan is in line with the purists (and still dominates them in practice).

Dustin

[Michael Betancourt]

Bob had a nice post on this last year, http://andrewgelman.com/2014/06/12/stan-turing-completeo-probabilistic-programming-language/.

Regarding the third point: we don’t do discrete parameters, but no one else can

fit _anything_ with reasonable speed, so where do you draw the line?

Really what we have to do is separate the language from the inference engine — the

languages (except for BUGS) are all Turing complete and can specify anything, and

the differences (and limitations) arise in the inference engines.  Frank/Vakesh/Zoubin/Noah

etc all want some magic inference engine that will fit every model super fast — we

acknowledge that such an engine doesn’t exist (yet) so we restrict out scope to the

models that we can fit magically fast (continuous sample spaces with HMC) in order

to provide a practical tool instead of just a theoretical one.  I think the relative size of

our user communities is testament enough to that!

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ben Goodrich]

On Thursday, July 16, 2015 at 2:28:17 AM UTC-4, Dustin Tran wrote:

Three main beliefs seem to exist which lead to people believing this way: that Stan isn’t Turing complete, that it’s more like JAGS/BUGS than Church/Venture/Anglican, and that it can’t handle discrete parameters. The first one is wrong, the second “true” but irrelevant, and the third a problem with our current inference algorithms rather than the full generality of the Stan language. I put up this note in case anyone else may find this useful. It seems like if we just implement a generic MH/SMC that no one will actually use, we can more confidently say that Stan is in line with the purists (and still dominates them in practice).

Peter Lee did a MH sampler but it wasn't exposed (at least not in RStan) because it wasn't useful, although even with that you couldn't declare discrete unknowns in the parameters block of a .stan program. But within the Stan language one can, in principle, marginalize over all the discrete unknowns and then draw from their posterior distribution in generated quantities, which should be satisfactory to the purists.

Ben

[Alp Kucukelbir]

i have a suggestion: once ADVI gets exposed to R and Python, i think we should craft a hacker news post. we can cite bob's excellent post, explain how HMC rocks and how ADVI scales. if the article succeeds, it should have a positive impact towards dissipating this myth about Stan not being a proper probabilistic programming framework.

[Bob Carpenter]

It's just semantics. We don't really do everything in
our programming language --- that is, we can't talk about
samplers, sampler combinations, computing expectations, etc.
On that grounds, I think it'd be fair to say at least
that Stan's a very limited kind of PPL compared to some
of the others.

Hakaru looks cool (Ken Shan and Rob Zinkov at Indiana), as does
the WebPPL demo (Noah Goodman). The latter's very cool in
that it compiles the language down to Javascript and uses
V8 to run it. Basically just what Andrew keeps asking us to
build for him:

language description:
http://dippl.org/chapters/02-webppl.html

runnable online version:
http://webppl.org

- Bob

> On Jul 17, 2015, at 4:15 AM, Alp Kucukelbir <akucu...@gmail.com> wrote:
>
> i have a suggestion: once ADVI gets exposed to R and Python, i think we should craft a hacker news post. we can cite bob's excellent post, explain how HMC rocks and how ADVI scales. if the article succeeds, it should have a positive impact towards dissipating this myth about Stan not being a proper probabilistic programming framework.
>