Clarifying blavaan warning about effective # parameters exceeding sample statistics

[Aleija R.-R.]

Hi all,

I have a longitudinal model (model 1, # parameters = 117) that has been getting the following error and output when attempting to set pD = "loo": 

1.        “blavaan WARNING: The effective number of parameters exceeds the number of sample statistics (covariances, etc.), so fit index calculations may lead to uninterpretable results.”

Posterior summary statistics and highest posterior density (HPD)
 90% credible intervals for devm-based fit indices:

               EAP Median   MAP    SD    V5    V6
BRMSEA         NaN     NA    NA    NA    NA    NA
BGammaHat    0.976  0.976 0.976 0.000 0.976 0.976
adjBGammaHat 8.026  8.023 8.016 0.055 7.934 8.115
BMc          0.885  0.885 0.885 0.001 0.884 0.887
BCFI         0.956  0.956 0.956 0.000 0.956 0.957
BTLI         8.158  8.155 8.146 0.058 8.062 8.251
BNFI         0.956  0.956 0.956 0.000 0.956 0.957

My understanding is that this is because the effective number of parameters (in this case p_loo) is greater than the total # of potential variances, covariances, etc (i.e., p(p+1)/2, where p = # of observed variables). For model 1, there are 20 observed variables (18 + 2 time-invariant covariates), making p(p+1)/2 = 210, and p_loo = 230 (per fitMeasures()). 

However, I have another longitudinal model (model 2, # parameters = 125) that does NOT generate the above warning despite the total # of potential variances/covariances/etc being less than p_loo. For context, model 2 has 21 observed variables (18 + 3 time-invariant covariates) and p_loo = 238. However, model 2 generates interpretable (though somewhat contradictory) fit indices using pD = "loo" (see below). 

Posterior summary statistics and highest posterior density (HPD)
 90% credible intervals for devm-based fit indices:

               EAP Median   MAP    SD lower upper
BRMSEA       0.143  0.143 0.143 0.001 0.142 0.144
BGammaHat    0.974  0.974 0.974 0.000 0.974 0.974
adjBGammaHat 0.521  0.521 0.522 0.003 0.516 0.527
BMc          0.870  0.870 0.870 0.001 0.869 0.871
BCFI         0.950  0.950 0.950 0.000 0.950 0.951
BTLI         0.460  0.461 0.460 0.004 0.454 0.467
BNFI         0.950  0.950 0.950 0.000 0.950 0.951

I looked at the source code for blavFitIndices and could not find the warning, so I couldn't determine exactly what "under the hood" leads to it. So my question is a two-parter:

1. Is p(p+1)/2 the formula that blavaan uses to determine the "number of sample statistics (covariances, etc.)"? 

2. If p(p+1)/2 is the formula used to determine whether that warning pops up, why do two models with p_loo > sample statistics have different behaviors when using blavFitIndices()?

Thanks!

[Ed Merkle]

For models with meanstructure = TRUE, you need to add an extra p for the mean parameters. I would guess that this is leading to what you observe, though it is close for model 1!

Ed

[Aleija R.-R.]

Hi Ed,

Thank you, that's good to know! And since blavaan employs a "full information" likelihood when there are missing data, I'm guessing meanstructure defaults to true for all blavaan fits with missing data?

[Ed Merkle]

Yes, and you should also be able to check for your specific model using a command like blavInspect(fit, "meanstructure"). Then you will see TRUE or FALSE depending on whether means were included.