modification indices with lavTestLRT

[Emmanuel W]

Hello,

I try to find the modification indices in another way since "lavTestScore" does not work for certain relationships (outcome~CESD1 for exemple).

In order to test the method, I created a first model:

model1 <-
'dep=~CESD3 + CESD6 + CESD9 + CESD10 + CESD14 + CESD17 + CESD18
pos=~CESD4 + CESD8 + CESD12 + CESD16
som=~CESD1 + CESD2 + CESD5 + CESD7 + CESD11 + CESD13 + CESD20
int=~CESD15 + CESD19
depression=~dep+pos+som+int
depression~age + sex + educ2 + educ3

outcome1~depression + age + sex + educ2 + educ3
outcome2~depression + age + sex + educ2 + educ3
outcome3~depression + age + sex + educ2 + educ3
outcome4~depression + age + sex + educ2 + educ3

outcome1~~outcome2
outcome1~~outcome3
outcome1~~outcome4
outcome2~~outcome3
outcome2~~outcome4
outcome3~~outcome4
'

fit1 <- cfa(model1, data = bdd, estimator = "WLSMV",
ordered=c("CESD1","CESD2","CESD3","CESD4","CESD5","CESD6","CESD7","CESD8",
"CESD9","CESD10","CESD11","CESD12","CESD13","CESD14","CESD15","CESD16",
"CESD17","CESD18","CESD19","CESD20","outcome1","outcome2",
"outcome3","outcome4"))

My model 2 is the same with only "outcome~dep" in addition and I compute:

fit2 <- cfa(model2, data = bdd, estimator = "WLSMV",
ordered=c("CESD1","CESD2","CESD3","CESD4","CESD5","CESD6","CESD7","CESD8",
"CESD9","CESD10","CESD11","CESD12","CESD13","CESD14","CESD15","CESD16",
"CESD17","CESD18","CESD19","CESD20","outcome1","outcome2",
"outcome3","outcome4")).

However, when I try "lavTestLRT (fit, fit2)", I can never find the same p-value and chisq diff that I get with "lavTestScore". I tried several methods but it's different every time. Is there any way to get exactly the same results as with "lavTestScore" ? Is it necessary to add "std.lv=T"?

Thank you for your feedback,

Emmanuel

[Terrence Jorgensen]

I try to find the modification indices in another way ... However, when I try "lavTestLRT (fit, fit2)", I can never find the same p-value and chisq diff that I get with "lavTestScore".

Because they are only asymptotically equivalent tests.  Modification indices are score tests; a LRT is not a "modification index".  You can read about the equivalence of score, LR, and Wald tests in this nice illustration:

Buse, A. (1982). The likelihood ratio, Wald, and Lagrange multiplier tests: An expository note. The American Statistician, 36(3a), 153-157.  https://doi.org/10.1080/00031305.1982.10482817

Is there any way to get exactly the same results as with "lavTestScore" ? 

No, and you shouldn't want it to be.  The LRT is better because it uses the actual information about how both models fit.  The score test only approximates what the LRT would be from information in the restricted model.  Because it is almost certainly misspecified, score tests (modification indices) are notoriously biased, so we can't really trust their p values, although the "big means bad" heuristic is fairly pervasive.

Terrence D. Jorgensen

Assistant Professor, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

http://www.uva.nl/profile/t.d.jorgensen

[Emmanuel W]

Ok Terence. Thank you again.

I found this text about modification indices but but I'm asking if there's not a mistake:

"This type of chi-square difference test, known as a likelihood ratio test, is the type of modification index used in Lisrel, Mplus, and Amos. EQS provides two slightly different types of modification indices. EQS uses similar asymptotically equivalent tests, called the Lagrange multiplier for adding parameters to the model and Wald tests for eliminating parameters from the model. A similar test called the Score test is occasionally mentioned."

And if I use LRT, which method would be the "best" (I use WLSMV if it matters)?

Emmanuel

[Terrence Jorgensen]

I found this text about modification indices but but I'm asking if there's not a mistake:

"This type of chi-square difference test, known as a likelihood ratio test, is the type of modification index used in Lisrel, Mplus, and Amos. EQS provides two slightly different types of modification indices. EQS uses similar asymptotically equivalent tests, called the Lagrange multiplier for adding parameters to the model and Wald tests for eliminating parameters from the model. A similar test called the Score test is occasionally mentioned."

What is this text from?  I have never heard anyone refer to Wald tests as modification indices.  Score tests are Lagrange multipliers, and they are called modification indices only in the SEM literature.

Wald and score tests are both asymptotic approximations of the LRT, but the LRT is the gold standard.

And if I use LRT, which method would be the "best" (I use WLSMV if it matters)?

You can still calculate these tests using (robust) DWLS estimation.  Again, LRT is the gold standard.  You can read this short article for a description of how the 3 tests are different ways of testing the same information:

https://doi.org/10.1080/00031305.1982.10482817

[Emmanuel W]

Thank you Terence. When I said "which method would be the "best", I was referring to methods like "satorra.2000", etc.

Emmanuel

[Terrence Jorgensen]

When I said "which method would be the "best", I was referring to methods like "satorra.2000", etc.

Satorra (2000) is used for DWLS because the associated test statistic is scaled AND shifted (mean- and variance-adjusted: "WLSMV").  For robust MLE of continuous data, simply a scaled test statistic (mean-adjusted: "MLM") tends to yield error rates close enough to nominal, so Satorra & Bentler (2001) works (or their 2010 alternative, when their 2001 version occasionally yields a negative test statistic).