Measurement Invariance

[Tomás Arriaza]

I don't know what I am doing wrong. I am trying to test for measurement invariance for first time with lavaan, but I get two warnings that are worrying me.

I would really appreciate any feedback of my script. I want to be able to replicate Mplus analysis (where I didn't have problems) or at least explain differences in specifications.

Thank you very much in advance!

My results:

Scaled Chi-Squared Difference Test (method = “satorra.2000”) lavaan NOTE: The “Chisq” column contains standard test statistics, not the robust test that should be reported per model. A robust difference test is a function of two standard (not robust) statistics. Df AIC BIC Chisq Chisq diff Df diff Pr(>Chisq) model1.metric 1154 4168.1 model1.configural 1295 4036.1 -92.527 141 1 model1.scalar 1295 4036.1 0.000 0 Warning messages: 1: In lavTestLRT(model1.configural, model1.metric, model1.scalar, scaled.shifted = TRUE) : lavaan WARNING: Some restricted models fit better than less restricted models; either these models are not nested, or the less restricted model failed to reach a global optimum. Smallest difference = -132.008261570369 2: In lavTestLRT(model1.configural, model1.metric, model1.scalar, scaled.shifted = TRUE) : lavaan WARNING: some models have the same degrees of freedom

My script:

syntax <- "

Part1 =~ ics1_1 + ics1_2 + ics1_3 + ics1_4 + ics1_5
Part2 =~ ics2_1 + ics2_2 + ics2_3 + ics2_4_r + ics2_5 + ics2_6_r + ics2_7_r + ics2_8 + ics2_9 + ics2_10_r
Part3 =~ ics3_1 + ics3_2 + ics3_3 + ics3_4_r + ics3_5 +  ics3_6_r + ics3_7_r + ics3_8 + ics3_9 + ics3_10_r
"

model1.configural <- cfa(syntax,
              data = db2,
              ordered = c("ics1_1","ics1_2","ics1_3","ics1_4","ics1_5",
                          "ics2_1","ics2_2","ics2_3","ics2_4_r","ics2_5",
                          "ics2_6_r","ics2_7_r","ics2_8","ics2_9","ics2_10_r",
                          "ics3_1","ics3_2","ics3_3","ics3_4_r","ics3_5",
                          "ics3_6_r","ics3_7_r","ics3_8","ics3_9","ics3_10_r"),
              estimator = "WLSMV", missing = "pairwise",mimic = "Mplus",
              group = "consumo")

model1.metric <- cfa(syntax,
                     data = db2,
                     ordered = c("ics1_1","ics1_2","ics1_3","ics1_4","ics1_5",
                                 "ics2_1","ics2_2","ics2_3","ics2_4_r","ics2_5",
                                 "ics2_6_r","ics2_7_r","ics2_8","ics2_9","ics2_10_r",
                                 "ics3_1","ics3_2","ics3_3","ics3_4_r","ics3_5",
                                 "ics3_6_r","ics3_7_r","ics3_8","ics3_9","ics3_10_r"),
                     estimator = "WLSMV", missing = "pairwise",mimic = "Mplus",
                     group = "consumo", group.equal = c("loadings"))

model1.scalar <- cfa(syntax,
                     data = db2,
                     ordered = c("ics1_1","ics1_2","ics1_3","ics1_4","ics1_5",
                                 "ics2_1","ics2_2","ics2_3","ics2_4_r","ics2_5",
                                 "ics2_6_r","ics2_7_r","ics2_8","ics2_9","ics2_10_r",
                                 "ics3_1","ics3_2","ics3_3","ics3_4_r","ics3_5",
                                 "ics3_6_r","ics3_7_r","ics3_8","ics3_9","ics3_10_r"),
                     estimator = "WLSMV", missing = "pairwise",mimic = "Mplus",
                     group = "consumo", group.equal = c("loadings", "thresholds"))

model1.strict  <- cfa(syntax,
                     data = db2,
                     ordered = c("ics1_1","ics1_2","ics1_3","ics1_4","ics1_5",
                                 "ics2_1","ics2_2","ics2_3","ics2_4_r","ics2_5",
                                 "ics2_6_r","ics2_7_r","ics2_8","ics2_9","ics2_10_r",
                                 "ics3_1","ics3_2","ics3_3","ics3_4_r","ics3_5",
                                 "ics3_6_r","ics3_7_r","ics3_8","ics3_9","ics3_10_r"),
                     estimator = "WLSMV", missing = "pairwise",mimic = "Mplus",
                     group = "consumo", group.equal = c("loadings", "thresholds","residuals"))

lavTestLRT(model1.configural,model1.metric,model1.scalar,
           scaled.shifted = TRUE)

[Jošt Bartol]

Hi, Tomas,

Perhaps lavaan does not understand what you mean by "thresholds". See here https://lavaan.ugent.be/tutorial/groups.html, and maybe try "intercepts" instead.

Best,

Jošt

V V sre., 3. jul. 2024 ob 04:27 je oseba Tomás Arriaza <tom.a...@gmail.com> napisala:

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/c559650e-640a-4a95-bdd2-ac9715dc55c7n%40googlegroups.com.

[Tomás Arriaza]

Thank you. I tried that but the results and warnings are the same.

[Yago Luksevicius de Moraes]

`lavaan` do understand "thresholds", but the problem starts in the configural-metric difference. The configural model has more degrees of freedom than the metric one, but it should be the other way around.

I see no obvious problem in your syntax, but I've never tried to run a MG-CFA mimicking Mplus. If you disable this parameter, is the error solved? If not, can you paste here the output of the configural and metric models?

Best,

Yago

[Tomás Arriaza]

As you noticed, the problem was the mimic argument. Using mimic, lavaan estimated the same chi square as Mplus for the overall model, but estimate a different chi square for the configural model and generate this problems with the invariance measurement (different number of parameters, chi square and shift parameter correction).

I disable the parameter and the second warning was solved. It's working with some differences in comparison to Mplus due to the different shift parameter correction. I separately run

lavTestLRT(model1.configural,model1.metric,
           scaled.shifted = TRUE)

lavTestLRT(model1.metric,model1.scalar,
           scaled.shifted = TRUE)

and warning is generated by the second comparison (metric vs scalar). This is the output after disable the mimic argument.

Scaled Chi-Squared Difference Test (method = “satorra.2000”) lavaan NOTE: The “Chisq” column contains standard test statistics, not the robust test that should be reported per model. A robust difference test is a function of two standard (not robust) statistics. Df AIC BIC Chisq Chisq diff Df diff Pr(>Chisq) model1.configural 1088 3821.5 model1.metric 1154 4168.1 135.996 66 9.019e-07 *** model1.scalar 1295 4036.1 -92.527 141 1 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Warning message: In lavTestLRT(model1.configural, model1.metric, model1.scalar, scaled.shifted = TRUE) : lavaan WARNING: Some restricted models fit better than less restricted models; either these models are not nested, or the less restricted model failed to reach a global optimum. Smallest difference =

-132.008261570369

Here the output of configural, metric model and scalar:

> model1.configural lavaan 0.6.17 ended normally after 120 iterations Estimator DWLS Optimization method NLMINB Number of model parameters 412 Number of observations per group: 3 236 1 111 2 157 4 112 Number of missing patterns per group: 3 19 1 12 2 15 4 22 Model Test User Model: Standard Scaled Test Statistic 3821.513 2842.847 Degrees of freedom 1088 1088 P-value (Chi-square) 0.000 0.000 Scaling correction factor 1.762 Shift parameter 674.448 simple second-order correction Test statistic for each group: 3 1277.276 983.143 1 675.387 504.759 2 1023.952 752.907 4 844.897 602.037

> model1.metric lavaan 0.6.17 ended normally after 91 iterations Estimator DWLS Optimization method NLMINB Number of model parameters 412 Number of equality constraints 66 Number of observations per group: 3 236 1 111 2 157 4 112 Number of missing patterns per group: 3 19 1 12 2 15 4 22 Model Test User Model: Standard Scaled Test Statistic 4168.123 2799.462 Degrees of freedom 1154 1154 P-value (Chi-square) 0.000 0.000 Scaling correction factor 1.999 Shift parameter 714.484 simple second-order correction Test statistic for each group: 3 1343.141 945.597 1 766.473 512.151 2 1116.437 740.564 4 942.072 601.149

> model1.scalar lavaan 0.6.17 ended normally after 128 iterations Estimator DWLS Optimization method NLMINB Number of model parameters 496 Number of equality constraints 291 Number of observations per group: 3 236 1 111 2 157 4 112 Number of missing patterns per group: 3 19 1 12 2 15 4 22 Model Test User Model: Standard Scaled Test Statistic 4036.115 3024.422 Degrees of freedom 1295 1295 P-value (Chi-square) 0.000 0.000 Scaling correction factor 1.816 Shift parameter 801.810 simple second-order correction Test statistic for each group: 3 1320.139 1034.163 1 738.150 550.967 2 1064.803 790.724 4 913.023 648.568

[Jošt Bartol]

Hi Tomas and Yago,

Thank you for clarifying that.

As the warning says, it seems that the scalar model fits better (based on chi-square) than the metric model. It is indeed a bit strange that a more restricted model has lower chi-square (although perhaps not impossible). Maybe you check the scalar model to make sure there is nothing weird going on? Also, I noticed that in your initial post the configural and scalar models have the same chi-square, which is weird (that's also why I initially thought that the command "thresholds" does not work). What could be the reason for same chi-square between configural and scalar models?

Also notice that for group 2, the non-scaled chi square is lower in the scalar model (1064.803) than in the metric model (1116.437) while in all other cases (also scaled chi-square) it is higher. Maybe something is going on with group 2?

Best I got, hope it helps.

Best, Jošt

V V sre., 3. jul. 2024 ob 18:26 je oseba Tomás Arriaza <tom.a...@gmail.com> napisala:

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/dfa1a19b-846b-4828-9f55-69565cb4c220n%40googlegroups.com.

[Tomás Arriaza]

I found something very weird. The number of free parameters ignoring constraints is 412 for configural and metric models, but 496 for scalar. As far as I know, it should be equal for the three of them.

[Terrence Jorgensen]

Loadings are not comparable (thus, equivalence is not testable) across groups when their scales are not linked across groups.  Latent-response scales can be linked by equating thresholds.  Read the Wu & Estabrook (2016) reference on the ?semTools::measEq.syntax help page, and see the help-page examples.  You can also try searching past posts in this forum about measurement invariance with categorical (binary or ordinal) indicators to read more.

Terrence D. Jorgensen    (he, him, his)
Assistant Professor, Methods and Statistics
Research Institute for Child Development and Education, the University of Amsterdam
http://www.uva.nl/profile/t.d.jorgensen