Posterior predictive checks

[Matthijs Hollanders]

Hi,

I've read in a few threads in this list that it's possible to do posterior predictive checks after running the model, and that including the check within the model code and running MCMC to estimate the values is wasteful. However, I'm afraid I don't really know how to do posterior predictive checks after the model has run. Is it possible to get an explanation here, or could there be a blog post on the website of how to do it?

Kind regards,

Matt

[Chris Paciorek]

Hi Matthijs,

I'll give a brief overview of the steps that would generally be involved. Beyond that, I agree that having a detailed template/example would be helpful. We're in the middle of preparing for our virtual workshop next week, and this is a nice example for that, so I'm hoping to have something more concrete to share in coming days.

The following assumes that you want to simulate new datasets given the posterior samples of the model parameters. 

Basic steps:

1) run the MCMC, making sure to monitor nodes that are parents of the data nodes (this could be all stochastic+deterministic direct parents or all stochastic parents, basically whatever you need to be able to simulate the data nodes from the posterior predictive)

2) With the MCMC output (either as a matrix or as the mvSamples object of the compiled MCMC), loop through the saved samples and:

   a) Put the saved sample from a given iteration into the model

   b) Simulate into the data nodes (you'll need to specify 'includeData = TRUE' when calling 'simulate')

   c) Save the values of the simulated data nodes (for example, to a row of an output matrix)

3) carry out whatever assessment you want on the posterior predictive samples

Step 2 could be done (a) fully in R (including using an uncompiled model), (b) partly in R (using a compiled model, but otherwise done from R), or (c) fully in a compiled nimbleFunction that you create to carry out Steps 2a-2c.

-chris

--
You received this message because you are subscribed to the Google Groups "nimble-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nimble-users/61bf074f-f77f-43ff-83b6-8772e37d08e7n%40googlegroups.com.

[Matthijs Hollanders]

Hi Christopher,

Thanks for that response. I'll give it a go!

Matt

[Chris Paciorek]

hi Matthijs,

We now have a worked example of this available in slides from our workshop this week. Please see slides 24-28 of Module 9b:

https://github.com/nimble-training/nimble-virtual-2021/blob/main/content/9b_programming/nimblefunction_programming.html

We'll also try to work these up into an example at r-nimble.org/examples in coming weeks.

-chris

To view this discussion on the web visit https://groups.google.com/d/msgid/nimble-users/c19fe72e-8ae9-4efd-ab28-8d6cda7d21f3n%40googlegroups.com.

[Chris Paciorek]

And a note to close the loop here. There is now an example on the examples page on the website, directly accessible here.