Comparing constrained model and general model using measurement invariance analyses

598 views
Skip to first unread message

Torres

unread,
Mar 5, 2018, 4:08:50 AM3/5/18
to lavaan
Hello lavaan group,

I am conducting a cross-lagged panel model using lavaan. My general model is acceptable fit. Now I want to test if the model is the same across gender (constrained model). I was suggested to test the difference in chi-square between the general model and the constrained model. But I am still wondering whether I should test the model of configural using measurement invariance analysis, in which the group is "Gender". The model is as following:
modelconfig <- cfa(mymodel, data = mydata, group = 'Gender', estimator = 'MLR', missing = 'FIML') 
So can anyone show me if the modelconfig is suitable to test if the general model is the same across gender?

Another question I would like to consult you is that when I use command measurementInvariance(mymodel, data = mydata, group = 'Gender', estimator = 'MLR', missing = 'FIML'), I am always receiving a warning message as following: Error in if (cd < 0) { : missing value where TRUE/FALSE needed
But when I used the command to test configural model, the warning message disappeared. I can not solve the problem. Thus, can anyone please help me to solve it?

Thank you very much for considering my request,

Best,

Torres


Terrence Jorgensen

unread,
Mar 5, 2018, 4:57:56 AM3/5/18
to lavaan
I want to test if the model is the same across gender (constrained model). I was suggested to test the difference in chi-square between the general model and the constrained model. But I am still wondering whether I should test the model of configural using measurement invariance analysis, in which the group is "Gender".

The test of model fit is not specifically a test of whether that model fits equally across groups.  See this article about using a permutation test of configural invariance:


The permutation test is automated for lavaan models in the semTools package.  See the ?permuteMeasEq help page.

Error in if (cd < 0) { : missing value where TRUE/FALSE needed

This looks like the scaling factor might be missing.  When you fit the model yourself (without the measurementInvariance() function), did you notice missing values in the Robust column of the summary() output?

But when I used the command to test configural model, the warning message disappeared. 

When you used which command?  Just the cfa() function?

If the scaling factor cannot be computed for some reason, then you won't be able to calculate robust test statistics to compare models.  But permutation tests could be a solution, because they do not require the robust test statistic to have nominal Type I error rates:


Terrence D. Jorgensen
Postdoctoral Researcher, Methods and Statistics
Research Institute for Child Development and Education, the University of Amsterdam

Torres

unread,
Mar 5, 2018, 7:23:45 AM3/5/18
to lavaan




Vào 10:57:56 UTC+1 Thứ Hai, ngày 05 tháng 3 năm 2018, Terrence Jorgensen đã viết:
I want to test if the model is the same across gender (constrained model). I was suggested to test the difference in chi-square between the general model and the constrained model. But I am still wondering whether I should test the model of configural using measurement invariance analysis, in which the group is "Gender".

The test of model fit is not specifically a test of whether that model fits equally across groups.  See this article about using a permutation test of configural invariance:

Thank you Terrence D. Jorgensen for your response. I found the permutation test to be interesting but I may be I would turn to the traditional approach which is about the test of chi-square difference because the test seem more straight forward to me so far. Can you please suggest me the function which can be used to test the significant in chi-square? 


The permutation test is automated for lavaan models in the semTools package.  See the ?permuteMeasEq help page.

Error in if (cd < 0) { : missing value where TRUE/FALSE needed

This looks like the scaling factor might be missing.  When you fit the model yourself (without the measurementInvariance() function), did you notice missing values in the Robust column of the summary() output?
when fitting the model, I found no strange in the Robust column of the summary () output

But when I used the command to test configural model, the warning message disappeared. 

When you used which command?  Just the cfa() function?

If the scaling factor cannot be computed for some reason, then you won't be able to calculate robust test statistics to compare models.  But permutation tests could be a solution, because they do not require the robust test statistic to have nominal Type I error rates:

I used the cfa () function, more specifically configmodel <- cfa(modelconfig, data = mydata, group = 'Gender', estimator = 'MLR', missing = 'FIML')  

Terrence Jorgensen

unread,
Mar 7, 2018, 6:30:47 AM3/7/18
to lavaan
Can you please suggest me the function which can be used to test the significant in chi-square? 

The paper I linked to has example syntax in the online supplementary materials:

Torres

unread,
Mar 7, 2018, 5:50:28 PM3/7/18
to lavaan
Thank you Terrence D. Jorgensen :)

Vào 12:30:47 UTC+1 Thứ Tư, ngày 07 tháng 3 năm 2018, Terrence Jorgensen đã viết:

David Kai-Chun Yang

unread,
Apr 11, 2019, 9:15:53 AM4/11/19
to lavaan
Dear all

I have a related question regarding the application of permuteMeasEq for SB scaled chi-square statistics in MGCFA estimated by ULSM.

As ULSM is my estimator, the original chi-square statistics were incorrect and for nest model comparisons, the difference in chi-square statistics needed to be robustified.

My question is: when I use permuteMeasEq to evaluate the difference in chi-square statistics in MGCFA, which version of the difference in chi-square statistics should I target?

For the results of measurementInvariance, only "chisq" and "chisq.scaled" are available and there is no robustified "chisq" for permutation tests.

I am not sure if permutation tests of "chisq" or "chisq.scaled" is a reasonable method to evaluate measurement invariance as that for AFIs (e.g. robust CFI, robust RMSEA)?

The R codes in the supplementary materials provided by Terrence ( http://dx.doi.org/10.1037/met0000152.supp ) use both "chisq" and "chisq.scaled" as targets of permutation tests.
Since I did not have the full-text of that paper, I have no idea how this issue was dealt with by the authors.

Thanks a lot!

Best,
KC

Torres於 2018年3月5日星期一 UTC+8下午5時08分50秒寫道:

Terrence Jorgensen

unread,
Apr 12, 2019, 4:51:26 AM4/12/19
to lavaan
My question is: when I use permuteMeasEq to evaluate the difference in chi-square statistics in MGCFA, which version of the difference in chi-square statistics should I target?

The whole point of the permutation test is that it doesn't matter.  In this paper with categorical data, we permuted the naïve chi-squared statistic, which resulted in more nominal Type I error rates than the analytical robust chi-squared


I am not sure if permutation tests of "chisq" or "chisq.scaled" is a reasonable method to evaluate measurement invariance as that for AFIs (e.g. robust CFI, robust RMSEA)?

The R codes in the supplementary materials provided by Terrence ( http://dx.doi.org/10.1037/met0000152.supp ) use both "chisq" and "chisq.scaled" as targets of permutation tests.
Since I did not have the full-text of that paper, I have no idea how this issue was dealt with by the authors.

Results showed that permutation of both naïve and robust chi-squared statistics, as well as any fit index, provides the same results (with only minor deviations due to sampling error).  So within a particular sample, the p values might not all match across different fit measures, but they all have nominal error rates if the exchangeability assumption is met (i.e., groups have distributions with similar shapes, not necessarily normal).  
 
I attached the accepted version of the paper (preprint).
Kite.et.al.accepted.pdf
Jorgensen.et.al.2017-accepted.PermuteMeasEq.pdf

David Yang

unread,
Apr 12, 2019, 9:25:16 AM4/12/19
to lav...@googlegroups.com
Dear Terrence,

Thanks for your kind reply and very helpful answers to my questions.

I also appreciated the full-text of the nice papers from you and your colleagues and I will check the related details in them.

Have a nice weekend.

Best,
Kai-Chun (David) 



Terrence Jorgensen <tjorge...@gmail.com> 於 2019年4月12日 週五 下午4:51寫道:
--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To post to this group, send email to lav...@googlegroups.com.
Visit this group at https://groups.google.com/group/lavaan.
For more options, visit https://groups.google.com/d/optout.


--
Kai-Chun Yang MD, PhD

Department of Psychiatry
Taipei Veterans General Hospital, Taipei, Taiwan

Reply all
Reply to author
Forward
0 new messages