WAIC and/or LOO for JAGS supported MSOM

[Unai Ormazabal]

Dear Jurgen:

        First of all, congratulations on this excellent package. I have been working with it for the last year or so and I have learnt a lot about occupancy modelling and overall camera trapping data analysis. The issue I have is one that maybe goes beyond the scope of this group so if that's the case please feel free to tell me so and I will look for help elsewhere. I am currently making making a MSOM for a study with mesocarnivores and I am looking to find a way to choose the covariates that will produce the most parsimonious model. Working with Nimble I have found it very easy to do so (as it automatically calculates a WAIC value), and for this study it is possible for me to do it with Nimble because I only have continuous covariates. Nevertheless, on future projects I might have to use categorical covariates and as Nimble does not permit categorical covariates I am left with only the option of using JAGS. As far as I am concerned there is no way to obtain a WAIC or LOO estimate to infer how parsimonious each model is. I have found a scientific study by Vehtari et al. (2017) where they create the package loo to obtain PSIS-LOO and/or WAIC for this purpose, but it seems that the output I get when I fit the model (an mcmc.list) is not apt to be used with this package. I am still quite new to Bayesian inference and to be honest I don't have a clear picture of which step to take next. Is there any way of obtaining any measurement mentioned above to learn which covariates create the most parsimonious model? Is the loo package useful for that purpose? Could the diagnostics for the model, the Gelman-Rubin convergence statistic or the r-Hat values help select the most parsimonious model? I look forward to hearing from you.

             Kind regards:

                     Unai

[Juergen Niedballa]

Hello, 

thank you for the suggestion. Currently it is not possible to return WAIC or LOOIC for models in JAGS, since they don't return the log-likelihood. It is a good idea though and I'll consider it for a future update, but that won't happen soon (there are a few other things I'd like to add and implement first). 

Gelman-Rubin convergence statistics don't help identifying the most parsimonious model. They only tell you if the MCMC chains converged (settled on the same, stable parameter estimates). Likewise, Bayesian p-values don't tell you which model is the most parsimonious, but assess if the model adequately describes the data.

Sorry that what you request is not possible at the moment. Once it is implemented I'll add a post here. 

All the best,

Jürgen