How to use CPO?

[Christy Meredith]

Hello,
I have recently developed a CAR model using INLA. I want to report some type of measure of cross-validation, so I have requested the CPO output. However, I don't really understand what it is and how I can decompose these values to get some kind of measure of cross-validation error. Thank you for any suggestions!

[Håvard Rue]

The feature is described here,

http://www.r-inla.org/faq#TOC-How-can-I-compute-cross-validation-

if you plot an inla-object, like

plot(result)

then you'll see the CPO and PIT values

the CPO_i is \pi( y_i | y_-i )

the PIT_i is \pi( y_i^new <= y_i | y_-i )

and is Uniform(0,1) for the true model.

the adjusted PIT (for discrete responce only) is

PIT_i^adjusted = PIT_i - 1/2 * CPO_i

so the `=y_i' is counted half only.

hope this helps.

H

PS: use 'inla.cpo(..)'; see the FAQ.

--
Håvard Rue
Department of Mathematical Sciences
Norwegian University of Science and Technology
N-7491 Trondheim, Norway
Voice: +47-7359-3533 URL : http://www.math.ntnu.no/~hrue
Fax : +47-7359-3524 Email: havar...@math.ntnu.no

This message was created in a Microsoft-free computing environment.

[Christy Meredith]

Hello,
I estimated a sum(CPO) value for each of my models as a measure of fit. But the models with the highest sum(CPO) values are not necessarily the models with the lowest DIC. I am not sure what method I should use to evaluate competing models and why the results are not consistent for both indices. Is sum(CPO) a good measure of fit, or how can I use CPO to get a measure of fit for each model?

Thanks!

[Andrea Riebler]

Hi,

we use the logarithmic score calculated as -mean(log(inla.result$cpo)). However, you should also have a look at inla.result$failure to see whether all CPO values are reliable (see the FAQ section for details). According to a paper by Stone (1977), the cross-validated mean logarithmic score is asymptotically equivalent to the AIC, if the observations are independent. We used the logarithmic score  for disease mapping models including spatial effects, see for example Schrödle et al 2010. I think DIC and the logarithmic score need not to give exactly the same results. For  example, If you have many random effects in your model, DIC tends to underpenalize complex models, see Plummer, Biostatistics, 2008.

Best regards,

Andrea

[Christy Meredith]

That is exactly what I needed to know. Thank you so much for providing me with this information!!