k-fold cross-validation CVvalue?

36 views

Skip to first unread message

Jay Gedir

unread,

Sep 5, 2021, 7:19:18 PM9/5/21

to nimble-users

Hey All,

When running a k-fold cross-validation (i.e., runCrossValidate), not sure I understand what the CVvalue represents. The documentation states that it measures “the model's ability to predict new data – smaller relative values indicate better model performance”. This suggests to me that it can only be used in a relative sense (e.g., like model selection – like what Andrew was doing with his N-mixture models [27Aug20 post]). When I have used k-fold cross-validation in other analyses (e.g., glmer), for each model I would calculate a stand-alone value of prediction success – e.g., rho = 0.95 indicates that the model correctly predicts new data 95% of the time. Can a prediction like this be derived from the NIMBLE GLMM output and/or CVvalue?