Hello,
I've been finalizing my models and in particular, I have taken advantage of grouping.
My model is multi-leveled in that I have a number of groups of sensors, within each such group are five individual sensors. I used a model matrix, using the "z" model type (ala Gomez-Rubio). I then have an IID term for which part of the stream a sensor is in (main or side), and within this, I have a grouped IID term for wethere the sensor is upstream or downstream. Finally, I have a temporal component, which is day of sampling, and grouped in this term is hour of day. Both are rw2. The time of day is set as cyclic, which requires turning off "scale.model."
In essence, it appears as follows (Variables are in all caps)
formula<-sensor_reading ~ 0+f(SENSOR-NAME,model="iid",hyper=hypiid) + f(SUB_SENSOR#, model="z", Z=zSEN_NUM, hyper=hypiid) + f(HORIZONTAL_LOCATION, model="iid", hyper=hypiid, group=UP/DOWNSTREAM, control.group=list(model="iid",hyper=hypiid, scale.model=TRUE)) + f(SAMPLING_DAY, model=rw2, cyclic=FALSE, scale.model=TRUE, values=seq(0,last_day), hyper=hyperprec, diagonal=1e-4, constr=FALSE, group=HOUR_OF_DAY, control.group=list(model=rw2, cyclic=TRUE, hyper=hyperprec, scale.model=FALSE, adjust.for.con.comp=TRUE))
I guess I'm looking for some insights about what the ramifications of not scaling the latent effects would be and to ask if there is any plan to allow group effects to bhave scaled latent effects and be cyclic.
My model outputs look excellent, the data fits exceptionally well with the linear predictions (a very linear diagonal with slope=1), and the residuals follow a horizontal line at 0 well. My DIC is surprisingly low, and my CPO and PIT both look good. Yet, when I generated a validation set by leaving out 1/5th of my data (one randomly selected sensor per sensor group), changing them to NA, and then allowing INLA to compute the NAs, only about 1/5th of this validation data is within the 95% CI. I fear overfitting. Could this be related to not scaling the hour of day term?
I have run about 20-30 different variations of this model, and grouping hour within sampling day provides considerably better fits than an interaction effect, or treating them as separate effects. I am wondering if this excellent fit is the result of my not scaling that effect.
Any insight would be helpful.
Thanks!