Measurement Invariance using WLSMV

[Kris Nenezic]

Dear Yves Rosseel/Terrence Jorgensen, 

I hope this message finds you well! I am a rising 3rd year student doing undergraduate Maths and Statistics. 

I am using CFA (and also EFA) to confirm the factor structure of a 9-item scale (which are binary items yes/no). Naturally I used the WLSMV estimator and as you very well document, the best fit indices to communicate in this case are the "robust" ones following the work of Savalei (2021) which criticized using "scaled" fit indices since they overestimate the fit (since they use CFI + RMSEA formulae of ML blindly and also wrong fit function). 

Up to here I had no problems. However if we want to then test for measurement invariance using measEq.syntax, no robust fit indices are available when constraining the thresholds or thresholds + loadings. The polychoric matrices happen to be positive definite without an eigenvalue close to 0 so I assume that there is no issue with cML but that the functions are not equipped to compute robust fit indices for measurement invariance purposes? Since you do say in the documentation "robust fit indices are to be used whenever available" so I am assuming they aren't always available. 

I read one of the posts you replied on about this topic (Apr 3) and your suggestion is to move away from delta CFI and to: 

1) use lavTestLRT() 

2) follow by a discussion on lavResiduals() + lavTestScore() as kind of like a "more detailed diagnostic to why there is significantly worse fit"? (which wouldnt be explained by delta CFI)? 

Could you just confirm your stance on all this (i.e. what you suggest as best practice)? Could you maybe briefly explain why you consider lavTestLRT() superior in this case even though it uses the standard chi-square value rather than the robust (I assume because difference of std chi-square values is a valid statistic, and if we implement a correction then we know it is approx. chi-square distributed)? i.e. do you think the SB correction is superior enough to be the currently best-way of how to tackle measurement invariance? 

Thanks a lot for all the work you put into lavaan and all the questions - it is inspirational how up-to-date and helpful it is. 

Best regards, 

Kristijan Nenezic  

[Adrian Garcia Selva]

Dear Kristijan,

I assume that the outputs from your CFA models with thresholds and loadings constraints show “NA” values for the robust fit indices. If this is the case, it is likely that the cause of the problem lies in the same issue that Terrence Jorgensen addressed in a related discussion (https://stats.stackexchange.com/questions/655190/robust-fit-indices-not-calculating-when-constraining-intercepts).

The baseline or independence model generated by default freely estimates the thresholds across all groups. Consequently, when you impose equality constraints on the thresholds—or on both thresholds and loadings—in your target invariance model, this target model is no longer strictly nested within the default baseline model. Therefore, the software does not calculate these robust incremental fit indices.

As pointed by Terrence in the StackExchange post, a possibility to resolve this issue and obtain the robust fit indices would be to specify a custom baseline model that incorporates the same threshold equality constraints as your target measurement invariance model. Operationally, you can accomplish this by first utilizing the lav_partable_independence() function, which generates the foundational parameter table for the standard independence model. Next, you should add the necessary cross-group equality constraints specifically on the threshold parameters. The final step would be to fit the custom baseline model to your data and subsequently pass this fitted model object directly to the  baseline_model=   argument within the  fitMeasures()   function.

However, upon attempting to generate a brief reproducible example to illustrate this workaround for this post, I discovered that lavaan (version 0.7-2) currently fails to calculate the indices using the  fitMeasures(baseline_model=)  and instead returns the following error: "robust fit indices are currently not available for categorical data in combination with a non-default baseline model; returning NA." I seem to recall that I used this custom baseline method in previous analyses, which leads me to suspect that a recent update may have altered this specific functionality. Given this unexpected behavior, I wonder if Yves Rosseel or Terrence Jorgensen might be able to shed some light on this matter, clarifying whether this is an intended limitation or if there is a newly recommended procedure to obtain these robust fit indices under such measurement invariance conditions.

On the other hand, it is not recommended to use differences in fit indices (i.e., the CFI delta) to assess the presence of measurement invariance in categorical models, since their conventional cutoff values were derived almost exclusively from simulations conducted with continuous data. Consequently, the likelihood ratio test remains the most valid option for testing the presence of invariance.

Best regards,

Adrián García