Bayesian Model Selection and Averaging with winBUGS

[Matt Williamson]

Hi All,

I am running some hierarchical occupancy models in WinBUGS (via R2WinBUGS) and have what I hope is a fairly simple question. I would like to be able to compare the various models using an information criterion (likely DIC as this seems to be BUGS' default) and then generate model averaged predictors. I have done this using Richard McElreath's rethinking package (using compare and ensemble, respectively), but cannot seem to find a good way to do this when the models are BUGS objects.For what it's worth, I'm not running these via rethinking rstan as I have not found a way to fit this particular type of model in rstan. Any pointers or ideas for compare and ensemble analogs for BUGS objects? I'd greatly appreciate it...

Cheers,

M

Matt Williamson

Schwartz Lab

Graduate Group in Ecology

University of California, Davis

[Evan Eskew]

Hi Matt,

I don't have much experience working with WinBUGS, but is there anyway to easily extract posterior samples from your BUGS objects and get them in R? If you could get your samples and package them up as a list, you should be able to work with them in R using the rethinking package, no?

Best,

Evan

--
Check out our R resources at http://d-rug.github.io/
---
You received this message because you are subscribed to the Google Groups "Davis R Users' Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to davis-rug+...@googlegroups.com.
Visit this group at https://groups.google.com/group/davis-rug.
For more options, visit https://groups.google.com/d/optout.

[Jaime Ashander]

The package AICcmodavg 

 https://cran.r-project.org/web/packages/AICcmodavg/index.html

appears to produce model selection tables for  bugs and rjags (using (Q)AIC(c). Perhaps you could adapt that code to use DIC instead

- Jaime

[Matt Espe]

Hi Matt,

To unpack your question a bit:

- Is there a reason you are using WinBUGS rather than JAGS? I believe JAGS is currently under active development and will have better support, methods, etc. compared to WinBUGS. It seems the latest release of WinBUGS was 2007. They have very similar syntax, so you should be able to use JAGS for anything from WinBUGS. 

- There is a good chance you can run this model in Stan, but it might require some math. Have you looked at the manual and example models for Stan? There are lots of occupancy models listed under the Stan example models. 

- Model averaging and model comparison are a bit different goals. For model comparison, you should look at using WAIC rather than DIC in this context, as it has many important advantages over DIC in hierarchical models. Richard has a great discussion of this in his book. As far as implementation, there should be a way to generate WAIC using WinBUGS/JAGS, though you might need to output the likelihood to do it.

- As others have said, model averaging is not too difficult once you have posterior samples for each of the models and a weight for each. You should be able to easily generate posterior samples, but you need to be diligent and check that each model has fully converged. In my experience, WinBUGS/JAGS can easily get stuck in a region of the posterior and appear to have converged when it actually has not. Make sure you are fully checking all the diagnostics for each model. 

Hope that helps,

Matt

[Zack Steel]

Could try the dic.samples function if using the rjags package.

http://www.inside-r.org/packages/cran/rjags/docs/dic.samples