Posterior predictive checks

42 views
Skip to first unread message

Matthijs Hollanders

unread,
May 14, 2021, 3:32:54 AMMay 14
to nimble-users
Hi,

I've read in a few threads in this list that it's possible to do posterior predictive checks after running the model, and that including the check within the model code and running MCMC to estimate the values is wasteful. However, I'm afraid I don't really know how to do posterior predictive checks after the model has run. Is it possible to get an explanation here, or could there be a blog post on the website of how to do it?

Kind regards,
Matt

Chris Paciorek

unread,
May 19, 2021, 4:32:32 PMMay 19
to Matthijs Hollanders, nimble-users
Hi Matthijs,

I'll give a brief overview of the steps that would generally be involved. Beyond that, I agree that having a detailed template/example would be helpful. We're in the middle of preparing for our virtual workshop next week, and this is a nice example for that, so I'm hoping to have something more concrete to share in coming days.

The following assumes that you want to simulate new datasets given the posterior samples of the model parameters. 

Basic steps:
1) run the MCMC, making sure to monitor nodes that are parents of the data nodes (this could be all stochastic+deterministic direct parents or all stochastic parents, basically whatever you need to be able to simulate the data nodes from the posterior predictive)
2) With the MCMC output (either as a matrix or as the mvSamples object of the compiled MCMC), loop through the saved samples and:
   a) Put the saved sample from a given iteration into the model
   b) Simulate into the data nodes (you'll need to specify 'includeData = TRUE' when calling 'simulate')
   c) Save the values of the simulated data nodes (for example, to a row of an output matrix)
3) carry out whatever assessment you want on the posterior predictive samples

Step 2 could be done (a) fully in R (including using an uncompiled model), (b) partly in R (using a compiled model, but otherwise done from R), or (c) fully in a compiled nimbleFunction that you create to carry out Steps 2a-2c.

-chris

--
You received this message because you are subscribed to the Google Groups "nimble-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nimble-users/61bf074f-f77f-43ff-83b6-8772e37d08e7n%40googlegroups.com.

Matthijs Hollanders

unread,
May 19, 2021, 11:00:11 PMMay 19
to nimble-users
Hi Christopher,

Thanks for that response. I'll give it a go!

Matt

Chris Paciorek

unread,
May 28, 2021, 4:13:20 PMMay 28
to Matthijs Hollanders, nimble-users
hi Matthijs,

We now have a worked example of this available in slides from our workshop this week. Please see slides 24-28 of Module 9b:


We'll also try to work these up into an example at r-nimble.org/examples in coming weeks.

-chris

Chris Paciorek

unread,
Jul 27, 2021, 3:23:39 PM (8 days ago) Jul 27
to Matthijs Hollanders, nimble-users
And a note to close the loop here. There is now an example on the examples page on the website, directly accessible here.
Reply all
Reply to author
Forward
0 new messages