Measurement Invariance [lavaan WARNING: scaling factor is negative]

[DVC]

Hello,

I'm doing a measurement and structural invariance test and would like to get some feedback. Specifically, I'm getting an error message as I arrive at the strict invariance part.

full_model <- '

#first-order latent variables

ER =~ Q3c + Q3f  

EI =~ Q3h  + Q3j + Q3i

EP =~ Q3k + Q3l + Q3m + Q3n  

PR =~ Q12c + Q12d  + Q12a 

PI =~ Q12f + Q12g + Q12h 

PP =~ Q12i + Q12j + Q12k + Q12l 

 

MR =~ Q13d + Q13e  + Q13a + Q13c

MI =~ Q13f + Q13g + Q13h

MP =~ Q13i + Q13j + Q13k + Q13l 

IN =~ Q8f + Q8e + Q8g + Q8j + Q8b

FO =~ Q8o + Q8n + Q8m + Q8l 

EA =~ Q4a + Q4b + Q4c + Q4d + Q4e

#second-order latent variables

EID =~  ER + EI + EP

PID =~ PR + PI + PP

MID =~ MR + MI + MP 

AGEN =~ IN + FO

#regression 

EID ~ AGEN + PID + MID

PID ~ AGEN + MID

MID ~ AGEN

EA ~ AGEN + EID 

'

####Model 1: configural invariance (structure):

fit1 <- sem(full_model, data = study, std.lv = TRUE,  estimator = "MLM", group = "a")

####Model 2: weak invariance (equal loadings):

fit2 <- sem(full_model, data = study, std.lv = TRUE, estimator = "MLM",group = "a", group.equal=c("loadings"))

#[Model 1 versus model 2]

anova(fit1, fit2, method = "satorra.bentler.2010") #not significant

####Model 3: strong invariance (equal loadings + intercepts):

fit3 <- sem(full_model, data = study, std.lv = TRUE, estimator = "MLM",group = "a", group.equal=c("loadings", "intercepts"))

summary(fit3, fit.measures = T, standardized = T, modindices = F, rsq = T)

#[Model 2 versus model 3]

anova(fit2, fit3, method = "satorra.bentler.2010") #significant

#[Model 1 versus model 3]

anova(fit1, fit3, method = "satorra.bentler.2010") #significant

####Model 3: strong invariance (equal loadings + intercepts) with intercepts freely estimated:

fit3.2 <- sem(full_model, data = study, std.lv = TRUE, estimator = "MLM",group = "a", group.equal=c("loadings", "intercepts") , group.partial = c("Q12d ~1", "Q12a~1",  "Q12f ~1", "Q3m~1", "Q3n~1", "Q4a~1"))

#[Model 2 versus model 3.2]

anova(fit2, fit3.2, method = "satorra.bentler.2010") #not significant

#[Model 1 versus model 3]

anova(fit1, fit3.2, method = "satorra.bentler.2010") #not significant

####Model 4: strict invarinace (equal loadings + intercepts + means) 

fit4 <- sem(full_model, data = study, std.lv = TRUE, estimator = "MLM", group = "a", group.equal=c("loadings", "intercepts", "residuals"), group.partial =  c("Q12d ~1", "Q12a~1",  "Q12f ~1", "Q3m~1", "Q3n~1", "Q4a~1"))

anova(fit3.2, fit4, method = "satorra.bentler.2001") --> Here is where I get the error message "lavaan WARNING: scaling factor is negative"

I can't seem to compute the chi-square difference. Any suggestions on what could be happening? 

Thank you in advance.

[Terrence Jorgensen]

I'm getting an error message as I arrive at the strict invariance part.

No you aren't.  You are getting a warning message.  Errors mean something went wrong, preventing the function from completing its task.  A warning means something might have gone wrong.

 "lavaan WARNING: scaling factor is negative"

See my response here:

https://groups.google.com/d/msg/lavaan/S7-ROFyrh-w/lhhewe22AQAJ

Terrence D. Jorgensen

Assistant Professor, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

http://www.uva.nl/profile/t.d.jorgensen

[DVC]

Thank you for responding. I think my question was not the correct question to ask. 

While I am getting the warning message "scaling factor is negative" my real issue is that the "chi-sq diff",  "df diff" and "Pr(>Chisq)" are not calculating when comparing 

weak with strict

anova(fit3.2, fit4, method = "satorra.bentler.2001")

or strong with strict 

anova(fit2, fit4, method = "satorra.bentler.2001")

but these the "chi-sq diff",  "df diff" and "Pr(>Chisq)" do calculate when comparing configural with strict

anova(fit1, fit4, method = "satorra.bentler.2001").

I'm not sure what I am doing wrong. OR am I misunderstanding something about measurement invariance? 

Thank you in advance

[Terrence Jorgensen]

I think my question was not the correct question to ask. 

No, it was indeed the cause of the difference tests not being calculated.  If the scaling factor is negative, then the scaled chi-squared will be negative, but the chi-squared distribution is strictly nonnegative, so it would be impossible to provide a p value.  That is why the strictly-positive version of the robust correction ("satorra.bentler.2010") was developed.

but these the "chi-sq diff",  "df diff" and "Pr(>Chisq)" do calculate when comparing configural with strict

The negative scaling factor depends on the pair of models being compared, not a single model.

[Dina Verdín]

I tried running the following, changing “satorra.bentler.2001” to “satorra.bentler.2010”

anova(fit3.2, fit4, method = "satorra.bentler.2010")

and
anova(fit2, fit4, method = "satorra.bentler.2010")

 

But I’m still getting the negative scaling factor, based on your previous message I’m led to believe that fit3.2 (strong) vs fit 4 (strict) and fit2 (weak) vs fit4 (strict) cannot be compared but fit 1 (configural) vs fit4 (strict) can be compared.

 

I’m inclined to believe that this is how measurement invariance works, but I’m not sure. In the past, I’ve stopped after achieving strong invariance.

 

--
You received this message because you are subscribed to a topic in the Google Groups "lavaan" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lavaan/m9gqzB7mlrw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lavaan+un...@googlegroups.com.
To post to this group, send email to lav...@googlegroups.com.
Visit this group at https://groups.google.com/group/lavaan.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/caebb734-cd9e-480e-8d45-b0d78ba52410%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Terrence Jorgensen]

I tried running the following, changing “satorra.bentler.2001” to “satorra.bentler.2010”

But I’m still getting the negative scaling factor

Try installing the development version of lavaan in a fresh R session, then restart and try again, in case this is a bug that was already resolved.

install.packages("lavaan", repos = "http://www.da.ugent.be", type = "source")

If you still get a negative test even when requesting the strictly positive test, please provide a minimal script and data to reproduce the issue, so the problem can be tracked down.

[Dina Verdín]

Hello,

I figured out the issue. 

There seems to be a problem using the estimator = "MLM" when using the anova function to determine model differences, by switching to estimator = "ML" I can evaluate strict invariance

Thank you for helping me work through this.

--
You received this message because you are subscribed to a topic in the Google Groups "lavaan" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lavaan/m9gqzB7mlrw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lavaan+un...@googlegroups.com.
To post to this group, send email to lav...@googlegroups.com.
Visit this group at https://groups.google.com/group/lavaan.

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/43c41403-2892-485f-bcaf-4cc133ae4cbd%40googlegroups.com.

[Terrence Jorgensen]

There seems to be a problem using the estimator = "MLM" when using the anova function to determine model differences, by switching to estimator = "ML" I can evaluate strict invariance

Why did you set MLM to begin with?  That is a robust estimator.  If your data are not normally distributed, then setting ML will not provide robust SEs and test statistics.

[Dina Verdín]

Yes, my data is not normally distributed so I tend to use MLM when conducting cfa and sem.

It sounds like switching to ML solved one problem but created another. 

Do you have suggestions on running a measurement invariance test when my data is not normally distributed? 

I typically use anova () to compare the models.

Thank you.

--
You received this message because you are subscribed to a topic in the Google Groups "lavaan" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lavaan/m9gqzB7mlrw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lavaan+un...@googlegroups.com.
To post to this group, send email to lav...@googlegroups.com.
Visit this group at https://groups.google.com/group/lavaan.

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/d4e6baa5-e60d-4260-bf6d-3f0f4d2cf5c2%40googlegroups.com.

[car...@web.de]

Use lavTestLRT: https://www.google.com/amp/s/rdrr.io/a/cran/lavaan/man/lavTestLRT.html

Am 23.05.19, 22:11, "Dina Verdín" <dver...@gmail.com> schrieb:

You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.

To post to this group, send email to lav...@googlegroups.com.
Visit this group at https://groups.google.com/group/lavaan.

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/CAB%2BZXPce9b_EGCUtKdjXexJiX96j3%2Bf2Bm9w9iLg26e8B8u8QA%40mail.gmail.com.