Scaling factor when comparing two models using anova

[Torres]

Hi everyone, 

I am comparing two models in which the first model is a two-factor model and the second model is a second-order factor including one higher order factor with two lower order factors. After comparing the two models using anova() I received a warning message following: scaling factor is negative. More details about the output are following:

> anova(modelfit1,modelfit2)

Scaled Chi Square Difference Test (method = "satorra.bentler.2001")

                    Df   AIC   BIC  Chisq Chisq diff Df diff Pr(>Chisq)

modelfit2      42 15705 15846 147.77                              

modelfit1      43 15703 15840 147.77                  1           

Warning message:

In lav_test_diff_SatorraBentler2001(mods[[m]], mods[[m + 1]]) :

  lavaan WARNING: scaling factor is negative

Unfortunately I am not sure if it is valid to conclude that the two models are significant difference. Can anyone help me explain why I received the warning message? 

Any help will be appreciated,

Thank you,

Torres

[Terrence Jorgensen]

Unfortunately I am not sure if it is valid to conclude that the two models are significant difference. Can anyone help me explain why I received the warning message? 

That can happen in the original formulation of the scaling factor, so those authors published an alternative method that guarantees a positive scaling factor.  To use that method, request method = "satorra.bentler.2010"

Terrence D. Jorgensen

Postdoctoral Researcher, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

UvA web page: http://www.uva.nl/profile/t.d.jorgensen

[Torres]

Thank you Terrence. I tried anova(modelfit1,modelfit2,method = "satorra.bentler.2010") but I still received the same warning message. Therefore, I nested model 1 in model 2 in which I set no covariance within model 1 (orthorgonal = T). Then the warning message disappeared and I was able to compare the two model. Results are following:

Scaled Chi Square Difference Test (method = "satorra.bentler.2001")

                              Df    AIC    BIC     Chisq     Chisq diff   Df diff Pr(>Chisq)

fitmodel2                42  15705 15846 147.77                              

fitmodel1                44  15836 15969 282.70     95.858       2  < 2.2e-16

                                     

fitmodel2

fitmodel2

Can you please tell me whether the results obtained from the comparison are still valid after the adjustment for the model 1?

Kind regards,

Torres

 

Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Vào 16:09:25 UTC+2 Chủ Nhật, ngày 24 tháng 6 năm 2018, Terrence Jorgensen đã viết:

[Terrence Jorgensen]

Can you please tell me whether the results obtained from the comparison are still valid after the adjustment for the model 1?

I'm not sure I understand what you are asking.  You got results of a model comparison, which are valid for testing the hypothesis that those two models have equivalent fit.  I don't see how fixing the factor correlation to zero represents the hypothesis you wanted to test in your original post (first-order CFA vs. higher-order factor).  Of course, you can't test that hypothesis with only 2 indicators anyway, since the models would be statistically equivalent (the factor correlation would simply be reparameterized as higher-order factor variance or loadings).