laplace for stan

[Andrew Gelman]

Hi all. I've been going thru BDA and I realize that in many of the examples (for instance, Gaussian processes), we recommend Laplace's approximation. This is what I've been calling the "lmer emulator". Anyway, I'm realizing how useful it will be to have this in Stan. The user will just have to write the model and indicate the parameters to be integrated over, and Stan will do the rest. This could really revolutionize a lot of statistical computing, I think.
A

[Michael Betancourt]

We just need parameter labeling, which I believe is already on the to-do list in preparation for lmer emulation and VB.

On Fri, Aug 9, 2013 at 5:16 PM, Andrew Gelman <gel...@stat.columbia.edu> wrote:

Hi all.  I've been going thru BDA and I realize that in many of the examples (for instance, Gaussian processes), we recommend Laplace's approximation.  This is what I've been calling the "lmer emulator".  Anyway, I'm realizing how useful it will be to have this in Stan.  The user will just have to write the model and indicate the parameters to be integrated over, and Stan will do the rest.  This could really revolutionize a lot of statistical computing, I think.

A

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Bob Carpenter]

Is labeling parameters more of a computational detail
or is the distinction part of the model?

If it's a computational detail, then I'd rather just list
the parameters in the call to the implementation. We could
use the names provided by the model to do everything we need,
I think. It might take another method or two in the model.

If it's something specific to the model, then I'd rather
put it in the model code. For this, the simplest approach would
be to introduce a "marginalized parameters" block along the same
line as the "parameters" block. This could also be used for
other nuisance parameters that users don't want to see reported
in the output, which is something we've also been thinking about.

So I guess I'm leaning toward (2). It'd be nice if the model
was a pure mathematical abstraction, but the reality is that it's
tied to implementation.

- Bob

> stan-dev+u...@googlegroups.com <mailto:stan-dev%2Bunsu...@googlegroups.com>.

[Michael Betancourt]

I like the idea of having multiple parameter blocks, but how would we generalize that to multiple clusters of parameters? For example, we have the marginalize/don't marginalize dichotomy for lmer, but a global/local dichotomy for VB (and perhaps more if the hierarchy is more than one layer deep). Is it easy enough to parse a generic "groupname_parameters" block and then concatenate them into a big parameter vector while still being able to retrieve them separately underneath the hood?

At one point you had talked about some means of tagging variables in the model description itself -- I thought that was still in the pipeline for eventual implementation.

[Bob Carpenter]

On 8/9/13 1:30 PM, Michael Betancourt wrote:
> I like the idea of having multiple parameter blocks, but how would we generalize that to multiple
> clusters of parameters? For example, we have the marginalize/don't marginalize dichotomy for lmer,
> but a global/local dichotomy for VB (and perhaps more if the hierarchy is more than one layer deep).

Good point. We can't use something hierarchical if it's
really a cross-cutting set of classifications (this is related
to the command-line discussion!).

> Is it easy enough to parse a generic "groupname_parameters" block and then concatenate them into a big
> parameter vector while still being able to retrieve them separately underneath the hood?

Yes, I think this would be pretty easy, but as you said, I
don't want to have groupname_marginalized and so on.

We can tag variables either on the inside or on the outside.
It depends what we need the target code to look like and I'm
still hazy on what that is for either the VB or the
max marginal cases. Maybe we can talk about it at the next
meeting?

> At one point you had talked about some means of tagging variables in the model description itself -- I thought that was still in the pipeline for eventual implementation.

We had talked about it. Andrew also wanted to tag statements
in the hopes of doing discrete models more efficiently.

- Bob

[Michael Betancourt]

> We can tag variables either on the inside or on the outside.
> It depends what we need the target code to look like and I'm
> still hazy on what that is for either the VB or the
> max marginal cases. Maybe we can talk about it at the next
> meeting?

Works for me.

>> At one point you had talked about some means of tagging variables in the model description itself -- I thought that was still in the pipeline for eventual implementation.
>
> We had talked about it. Andrew also wanted to tag statements
> in the hopes of doing discrete models more efficiently.

That would also admit prior/likelihood identification, but that seems much harder than just tagging the variables (an array of functions vs just an array of labels). Then again, I'm probably not the one who will implement it so that may not be entirely relevant. :-p