CFI contradicting TLI

[Simon R]

Hello lavaan community,

I estimated a relativley large ordered SEM in lavaan, the model converged and I got the follwing output:

lavaan 0.6-8 ended normally after 136 iterations

  Estimator                                               DWLS
  Optimization method                           NLMINB
  Number of model parameters            140           
  Number of observations                      255532
                                                     
Model Test User Model:
                                                Standard      Robust
  Test Statistic                      748400.496  313640.880
  Degrees of freedom                  1027        1027
  P-value (Chi-square)                  0.000       0.000
  Scaling correction factor                            2.391
  Shift parameter                                            610.829
       simple second-order correction                              

Model Test Baseline Model:
  Test statistic                            519728.865  380978.572
  Degrees of freedom                            78         78
  P-value                                               0.000       0.000
  Scaling correction factor                                  1.364

User Model versus Baseline Model:
  Comparative Fit Index (CFI)                  0.000       0.179
  Tucker-Lewis Index (TLI)                       0.891       0.938                      
  Robust Comparative Fit Index (CFI)                          NA
  Robust Tucker-Lewis Index (TLI)                               NA

Root Mean Square Error of Approximation:

  RMSEA                                                               0.053       0.035
  90 Percent confidence interval - lower         0.053       0.034
  90 Percent confidence interval - upper         0.053       0.035
  P-value RMSEA <= 0.05                                   0.000       1.000
                                                                 
  Robust RMSEA                                                                NA
  90 Percent confidence interval - lower                        NA
  90 Percent confidence interval - upper                        NA

Standardized Root Mean Square Residual:
  SRMR                                                                    0.031       0.031

Does anyone have an explanation for me why the CFI ( 0.179 ) is so much lower than the TLI (0.938 )?

Many thanks in advance for answering my question.

Simon

[Shu Fai Cheung]

What is the nature of the model? A multiple group model? A multilevel model?

The df of the user model (1027) is much larger than the df of the baseline model (78).

-- Shu Fai

[Simon R]

Thank you for your fast reply.  No, it was neither a multipe group nor a multilevel model. The nature of the model looks more or less like this

model <- '

  # measurement model

    Lat_y <- x1 + x2

    Lat_A =~ x3 + x4 +x5

    Lat_B =~  x6+ x7   

    Lat_C =~ x8 + x9

  # regressions

     Lat_y ~ Lat_A + Lat_B + Lat_C + x10 + … + x30

'

Many of the explanatory variables are categorical, but not ordered. x1 and x2 are measured on a Likert scale and the command to estimate the model was

sem.model <- sem(model, verbose = TRUE, data = df,
                                   ordered = c("x1", "x2"), estimator = "WLSMV")

Thank you for your help.

[Jošt Bartol]

Hi,

the CFI and TLI simply disagree because the way they are computed (see here https://davidakenny.net/cm/fit.htm).

The question I would have, however, is how come the degrees of freedom for the null (baseline) model are so much lower (df baseline = 78) than the degrees of freedom for your (user) model (df user = 1027). As much as I am aware, the null should have more df than the user-specified model... Maybe you can provide more information about that.

Hope it helps,

Jošt

[Shu Fai Cheung]

Sorry that I could not be of much help. I tried to simulate what you did but got some warnings. Did you get any warning message?

As commented by another member, the df of the model you fitted is much larger than that of the baseline model. This is new to me because the baseline model usually is the independence model. The fitted model should not have larger df than the baseline model. Maybe something's wrong with the fitted model, the baseline model, or both?

-- Shu Fai

[Simon R]

Thanks anyway for dealing with my question. I get the warning that the smallest eigenvalue is close to zero.

I calculated the CFI according to Savalei (2021)

https://doi.org/10.1080/00273171.2020.1717922

and the new proposed CFI is quite similar to the calculated TLI. I will look more into the df of my models. If I find a solution for my problem, I will post it. Thank you all.

Simon

[Christian Arnold]

Well, I don't know if that has anything in common with the problem. But it seems striking to me that Lat_y is "measured" formatively. The "<-" operator does a few things in the background. Maybe there is a connection to your problem?

---

Von: lav...@googlegroups.com <lav...@googlegroups.com> im Auftrag von Shu Fai Cheung <shufai...@gmail.com>
Gesendet: Mittwoch, 23. Februar 2022, 03:56
An: lavaan
Betreff: Re: CFI contradicting TLI

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/397bac9d-a979-46e0-9ede-1a7fc54643e4n%40googlegroups.com.

[Simon R]

The "<-" operator is a typo in my post. Sorry for the confusion. In the model in R it is "=~". Thank you.

[Edward Rigdon]

The CFI is not the issue. The issue is that the target model has more DF (1027) than the null model (78). That is absurd. The 78 DF for the beeline model implies that the model has 13 observed variables. There is no way for the target model to have 1,027 DF, even if you imposed some bizarre constraints on your ordinal variables--even if you had, say, 10 response categories per observed variable. Something is fundamentally wrong.

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/AM0PR04MB595553DDFFC7AF9F93D289ABFC3D9%40AM0PR04MB5955.eurprd04.prod.outlook.com.

[Simon R]

Okay, thank you for your response. I tried now to estimate the following models

model <- '

  # measurement model

    Lat_A =~ x3 + x4 +x5

    Lat_B =~  x6+ x7   

    Lat_C =~ x8 + x9

  # regressions

    x1 ~ Lat_A + Lat_B + Lat_C + x10 + … + x30

'

sem.model <- sem(model, verbose = TRUE, data = df,)

sem.model.ord <- sem(model, verbose = TRUE, data = df,
                                   ordered = c("x1"), estimator = "WLSMV")

The dependent variable x1 is measured on a Likert scale (0-10). In the sem.model lavaan takes the variable x1 as a continous variable and in the sem.model.ord as an ordinal variable.

# sem.model

Model Test User Model:                             

  Test statistic                               692080.4321

  Degrees of freedom                                   936

  P-value (Chi-square)                              0.0000

Model Test Baseline Model:

  Test statistic                             65839895.1257

  Degrees of freedom                                  1038

  P-value                                                      0.0000

The degrees of freedom make sense for me, but for the same model, only switching from ML to DWLS as the dependent variable is a ordinal, I get the following Output for sem.model.ord

Model Test User Model:

                                                Standard      Robust

  Test Statistic                              692980.910         284216.250

  Degrees of freedom                                 936         936

  P-value (Chi-square)                             0.000       0.000

  Scaling correction factor                                    2.443

  Shift parameter                                                   559.676

       simple second-order correction                               

Model Test Baseline Model:

  Test statistic                            432020.897        318618.317

  Degrees of freedom                                66          66

  P-value                                                    0.000       0.000

  Scaling correction factor                                  1.356

The degrees of are very differnt for the two models. I will look more into the degrees of freedom... thank you.

Simon