Skip to first unread message

Nov 12, 2018, 8:06:50 AM11/12/18

to lavaan

Hello,

I try to find the modification indices in another way since "lavTestScore" does not work for certain relationships (outcome~CESD1 for exemple).

In order to test the method, I created a first model:

'dep=~CESD3 + CESD6 + CESD9 + CESD10 + CESD14 + CESD17 + CESD18

pos=~CESD4 + CESD8 + CESD12 + CESD16

som=~CESD1 + CESD2 + CESD5 + CESD7 + CESD11 + CESD13 + CESD20

int=~CESD15 + CESD19

depression=~dep+pos+som+int

depression~age + sex + educ2 + educ3

outcome1~depression + age + sex + educ2 + educ3

outcome2~depression + age + sex + educ2 + educ3

outcome3~depression + age + sex + educ2 + educ3

outcome4~depression + age + sex + educ2 + educ3

outcome1~~outcome2

outcome1~~outcome3

outcome1~~outcome4

outcome2~~outcome3

outcome2~~outcome4

outcome3~~outcome4

'

fit1 <- cfa(model1, data = bdd, estimator = "WLSMV",

ordered=c("CESD1","CESD2","CESD3","CESD4","CESD5","CESD6","CESD7","CESD8",

"CESD9","CESD10","CESD11","CESD12","CESD13","CESD14","CESD15","CESD16",

"CESD17","CESD18","CESD19","CESD20","outcome1","outcome2",

"outcome3","outcome4"))

My model 2 is the same with only* "outcome~dep" *in addition and I compute:

fit2 <- cfa(model2, data = bdd, estimator = "WLSMV",

ordered=c("CESD1","CESD2","CESD3","CESD4","CESD5","CESD6","CESD7","CESD8",

"CESD9","CESD10","CESD11","CESD12","CESD13","CESD14","CESD15","CESD16",

"CESD17","CESD18","CESD19","CESD20","outcome1","outcome2",

"outcome3","outcome4")).

However, when I try "lavTestLRT (fit, fit2)", I can never find the same p-value and chisq diff that I get with "lavTestScore". I tried several methods but it's different every time. Is there any way to get exactly the same results as with "lavTestScore" ? Is it necessary to add "std.lv=T"?

Thank you for your feedback,

Emmanuel

Nov 12, 2018, 8:59:23 AM11/12/18

to lavaan

I try to find the modification indices in another way ... However, when I try "lavTestLRT (fit, fit2)", I can never find the same p-value and chisq diff that I get with "lavTestScore".

Because they are only asymptotically equivalent tests. Modification indices are score tests; a LRT is not a "modification index". You can read about the equivalence of score, LR, and Wald tests in this nice illustration:

Buse, A. (1982). The likelihood ratio, Wald, and Lagrange multiplier tests: An expository note. *The American Statistician, 36*(3a), 153-157. https://doi.org/10.1080/00031305.1982.10482817

Is there any way to get exactly the same results as with "lavTestScore" ?

No, and you shouldn't want it to be. The LRT is better because it uses the actual information about how both models fit. The score test only approximates what the LRT would be from information in the restricted model. Because it is almost certainly misspecified, score tests (modification indices) are notoriously biased, so we can't really trust their *p* values, although the "big means bad" heuristic is fairly pervasive.

Terrence D. Jorgensen

Assistant Professor, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

Nov 15, 2018, 10:34:35 AM11/15/18

to lavaan

Ok Terence. Thank you again.

I found this text about modification indices but but I'm asking if there's not a mistake:

"This type
of chi-square difference test, known as a **likelihood ratio test, is the type of
modification index** used in Lisrel, Mplus, and Amos. EQS provides two slightly different types of modification
indices. EQS uses similar asymptotically equivalent tests, called the Lagrange multiplier
for adding parameters to the model and Wald tests for eliminating parameters from
the model. A similar test called the Score test is occasionally mentioned."

And if I use LRT, which method would be the "best" (I use WLSMV if it matters)?

Emmanuel

Nov 23, 2018, 10:23:57 AM11/23/18

to lavaan

I found this text about modification indices but but I'm asking if there's not a mistake:

"This type of chi-square difference test, known as a

likelihood ratio test, is the type of modification indexused in Lisrel, Mplus, and Amos. EQS provides two slightly different types of modification indices. EQS uses similar asymptotically equivalent tests, called the Lagrange multiplier for adding parameters to the model and Wald tests for eliminating parameters from the model. A similar test called the Score test is occasionally mentioned."

What is this text from? I have never heard anyone refer to Wald tests as modification indices. Score tests are Lagrange multipliers, and they are called modification indices only in the SEM literature.

Wald and score tests are both asymptotic approximations of the LRT, but the LRT is the gold standard.

And if I use LRT, which method would be the "best" (I use WLSMV if it matters)?

You can still calculate these tests using (robust) DWLS estimation. Again, LRT is the gold standard. You can read this short article for a description of how the 3 tests are different ways of testing the same information:

Nov 28, 2018, 8:20:42 AM11/28/18

to lavaan

Thank you Terence. When I said "which method would be the "best", I was referring to methods like "satorra.2000", etc.

Emmanuel

Nov 29, 2018, 6:09:01 AM11/29/18

to lavaan

When I said "which method would be the "best", I was referring to methods like "satorra.2000", etc.

Satorra (2000) is used for DWLS because the associated test statistic is scaled AND shifted (mean- and variance-adjusted: "WLSMV"). For robust MLE of continuous data, simply a scaled test statistic (mean-adjusted: "MLM") tends to yield error rates close enough to nominal, so Satorra & Bentler (2001) works (or their 2010 alternative, when their 2001 version occasionally yields a negative test statistic).

Reply all

Reply to author

Forward

0 new messages

Search

Clear search

Close search

Google apps

Main menu