Satorra-Bentler 2010 error?

108 views
Skip to first unread message

Hynek Cígler

unread,
Aug 16, 2018, 6:05:34 PM8/16/18
to lavaan
Hello, I'm using lavTestLRT to compare model. I has been always using Satorra-Bentler 2010 method , but I have strong feeling its function results of the function has been changed in last months. I'm strongly afraid that the test statistic has an inverse value.

An example: I have two nested models, one vs. two dimensional catCFA. Their fit is terrible, but it does not matter in that case:
cfa1: "x2(27) = 133.29, p < .001, TLI = 0.859, RMSEA = 0.134 (95% CI = [0.112; 0.158]), SRMR = 0.084"
cfa2: "x2(25) = 79.13, p < .001, TLI = 0.922, RMSEA = 0.1 (95% CI = [0.075; 0.125]), SRMR = 0.064"

And now all the three methods for LRT:

> lavTestLRT(cfa1, cfa2, method = "satorra.2000")
Scaled Chi Square Difference Test (method = "satorra.2000")

     Df AIC BIC  Chisq Chisq diff Df diff Pr(>Chisq)    
cfa2 25         53.314                                  
cfa1 27         93.555     41.276       2  1.089e-09 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


> lavTestLRT(cfa1, cfa2, method = "satorra.bentler.2001")
Scaled Chi Square Difference Test (method = "satorra.bentler.2001")

     Df AIC BIC  Chisq Chisq diff Df diff Pr(>Chisq)    
cfa2 25         53.314                                  
cfa1 27         93.555     44.415       2  2.267e-10 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


> lavTestLRT(cfa1, cfa2, method = "satorra.bentler.2010")
Scaled Chi Square Difference Test (method = "satorra.bentler.2010")

     Df AIC BIC  Chisq Chisq diff Df diff Pr(>Chisq)
cfa2 25         53.314                              
cfa1 27         93.555    -46.193       2          1

Terrence Jorgensen

unread,
Aug 18, 2018, 10:27:27 AM8/18/18
to lavaan
To track down the possible bug, can you share your R script and enough data to reproduce the problem?

Terrence D. Jorgensen
Postdoctoral Researcher, Methods and Statistics
Research Institute for Child Development and Education, the University of Amsterdam

Hynek Cígler

unread,
Aug 23, 2018, 5:04:46 AM8/23/18
to lavaan
Dear Terrence,
I am sorry for the delay, I did not receive notice about your response. Data and script is here: https://is.muni.cz/de/hynek.cigler/satorra-bentler-2010.zip . Unfortunately, I used the chi2 difference test for ordinal CFAs, which could be problematic. So I included also interval CFAs to the script and result is still the same. (Models have terrible fit, but it should not bias the results of lavTestLRT function).
Best, Hynek



Dne sobota 18. srpna 2018 16:27:27 UTC+2 Terrence Jorgensen napsal(a):

Terrence Jorgensen

unread,
Aug 25, 2018, 8:10:48 AM8/25/18
to lavaan
I used the chi2 difference test for ordinal CFAs, which could be problematic.

I was under the impression that the Satorra-Bentler corrections were only for continuous nonnormal data.  Actually, the Satorra (2000) correction was also developed for continuous data, but I think the pertinent difference is that it can apply to mean- and variance-adjusted statistics (e.g., MLMV, WLSMV), as opposed to the Satorra-Bentler corrections applying to only mean-adjusted statistics (e.g., MLM, MLR, WLSM).  

Hynek Cígler

unread,
Aug 28, 2018, 6:24:56 AM8/28/18
to lavaan
Yes, so I said it could be problematic and I therefore re-run the analyses using MLR estimator. The chi-square difference using Satorra-Bentler (2010) is still negative (compared to e.g. Satorra, 2000, or Satorra-Bentler, 2001). The problem is not therefore based on the WLSMV estimator, but definitely in the way how the Satorra-Bentler (2010) scaled chi2 difference is computed. 

Dne sobota 25. srpna 2018 14:10:48 UTC+2 Terrence Jorgensen napsal(a):

Terrence Jorgensen

unread,
Aug 28, 2018, 6:50:28 AM8/28/18
to lavaan
Yes, so I said it could be problematic and I therefore re-run the analyses using MLR estimator. The chi-square difference using Satorra-Bentler (2010) is still negative (compared to e.g. Satorra, 2000, or Satorra-Bentler, 2001). The problem is not therefore based on the WLSMV estimator, but definitely in the way how the Satorra-Bentler (2010) scaled chi2 difference is computed. 

Thanks for sending data and syntax.  Actually, it does not return an error when using MLR, and it returns a positive test statistic (5.4147).  In the script you sent, you forgot to add "b" to the model names when checking the test under MLR.

lavTestLRT(cfa1b, cfa2b, method = "satorra.2000") ## OK
lavTestLRT
(cfa1b, cfa2b, method = "satorra.bentler.2001") ## OK
lavTestLRT
(cfa1, cfa2, method = "satorra.bentler.2010") ## produces strange result (negative x2 statistic)
lavTestLRT
(cfa1b, cfa2b, method = "satorra.bentler.2010") ## produces positive x2 statistic

Hynek Cígler

unread,
Aug 29, 2018, 2:47:04 AM8/29/18
to lavaan
Shame on me. Thank you for help and patience! :-)

Dne úterý 28. srpna 2018 12:50:28 UTC+2 Terrence Jorgensen napsal(a):
Reply all
Reply to author
Forward
0 new messages