Bayesian ridge regression and measurement error model in X

[Mindra Jaya]

Dear INLA Forum

Whether INLA can be used to solve the multicollinearity and measurement error simultaneously because to solve multicollinearity and measurement error need latent approach (i.e Bayesian ridge regression).  I have climate variables where the variables have high collinearity and there are indications of measurement errors. I read in Wang (2018), we can use Ridge regression approach to overcome the collinearity and model="mec" to solve the classical measurement error. However, those problems need latent approach. Is it possible to overcome those problems simultaneously?

Thank you very much.

Best

Mindra

 

[Jonathan Judge]

Mindra:

 

INLA’s default prior on all fixed effects is a diffuse Gaussian prior centered at 0.  So, provided your predictors have been centered at zero, which is a good idea regardless of modeling method, INLA effectively imposes some form of ridge regression on any model, at least in my opinion. 

 

The effectiveness of the ridge prior can depend on the spread component.  The default prior in INLA is on the precision, and is extremely diffuse.  Canonical ridge regression would set the prior on the precision to be 1 (identical to the variance, as its reciprocal).  If your predictors are scaled to the unit interval (0,1), which again is usually a good idea regardless, this is easy to apply across the board by just setting the default INLA prior on the precision of the fixed effects (control.fixed) to 1.  I would try that and see how it works. You can then monitor the marginal likelihoods to see how further adjustments to the spread (raising or lowering it) affects accuracy. There are other ways to more formally replicate ridge regression in INLA, but I’m not sure they are worth the additional hassle. Others may disagree.

 

To the extent you see the need to perform variable selection (forcing some variables to zero, rather than just minimizing their values), you may want to try running the model in Stan and selecting a regularized horseshoe prior.  Once you have selected only the variables you probably want, you can return to INLA to resume ridge regression optimization.

 

I will defer to others on the measurement error issue.  Measurement error, though, is just sampling predictors from a secondary distribution of the subject measurement with a typically-Gaussian spread of whatever the measurement error is.  I don’t recall whether that function is built in to INLA (I suspect it may be, or least there may be another function that approximates it), but perhaps you could set up a secondary likelihood within the INLA formula to replicate it if need be.

 

Best,

Jonathan

--
You received this message because you are subscribed to the Google Groups "R-inla discussion group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to r-inla-discussion...@googlegroups.com.
To post to this group, send email to r-inla-disc...@googlegroups.com.
Visit this group at https://groups.google.com/group/r-inla-discussion-group.
For more options, visit https://groups.google.com/d/optout.

 

[Helpdesk]

About the measurement error model, just is just another latent model
and can be combined with all other stuff. The main paper about this is

Muff, Riebler ++ in JRSSC, a few years ago

H

--
Håvard Rue
Helpdesk
he...@r-inla.org