Variable Selection

[bart]

Hi,

I have a three-level, non-nested hierarchical model that I am interested in performing variable selection for. If I include interaction terms in my model, which I would like to do, the number of covariates becomes rather large, so I would like to find a  good way to perform selection. My only goal of this model is prediction, so there is no great need for simplicity, but I don't want to include everything for fear of overfitting. 

I am aware of the Laplace, Horseshoe, and Projective Predictive technique put forth by Vehtari, et al; I just wanted to check if there are any other ideas out there since most of the posts on variable selection are several years old at this point. 

This group has been a very valuable resource that I have referenced many times in the past. Thank you to everyone who contributes to this group, you have been extremely helpful!

bart

[Bob Carpenter]

We usually use hierarchical priors rather than variable selection
to deal with overfitting. Variable selection can be useful for
computation or in really large problems.

I don't think there's been anything else on variable selection
other than what Juho and Aki have been working on:

http://arxiv.org/abs/1510.04813

- Bob

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[bart]

Thanks, Bob!

Just wanted to make sure I completely understood your comment (sorry, I'm a little new to this). By hierarchical priors, are you referring to regularizers like the laplace prior and the horseshoe prior? I've implement both, and now am thinking of averaging my predictions from both of those as well as the unregularized model. Does this make sense?

[Bob Carpenter]

> On Nov 14, 2016, at 4:44 PM, bart <bart.bl...@gmail.com> wrote:
>
> Thanks, Bob!
>
> Just wanted to make sure I completely understood your comment (sorry, I'm a little new to this). By hierarchical priors, are you referring to regularizers like the laplace prior and the horseshoe prior? I've implement both, and now am thinking of averaging my predictions from both of those as well as the unregularized model. Does this make sense?

You can do that, but what you normally want to do in Bayesian modeling
is think about the generative process. Is the generative process a mixture
where some observations come from one model and one from the other for some
reason?

What kind of prior to use should depend on your data. What do you
know about the coefficients before you start?

And I mean a hierarchical model. You might want to check out this
simple case study if you aren't familiar with hierarchical models
or Bayesian modeling in general, there are two case studies that
should help:

http://mc-stan.org/documentation/case-studies.html

* A Primer on Bayesian Multilevel Modeling using PyStan

* Pooling with Hierarchical Models for Repeated Binary Trials

- Bob

[bart]

Thanks, Bob. Much appreciated.