Question about absence of robust fit statistics in cfa.mi() output

Bryan Stiles

unread,

Mar 19, 2025, 9:58:34 PM3/19/25

to lavaan

Hi all,

I know this is a frequently raised question, but I think it touches on a different issue. Following another thread with Dr. Jorgensen, I recently updated my lavaan and lavaan.mi packages and noticed when running cfa.mi() that the output no longer is listing Robust versions of the CFI, TLI, and RMSEA. Here's an example from a CFA model of mine:

> summary(cfa2out, standardized = TRUE, rsquare=TRUE, fit.measures=TRUE, test = "D2",
+ pool.robust = TRUE)
OLDlavaan.mi object based on 20 imputed data sets.
See class?OLDlavaan.mi help page for available methods.

Convergence information:
The model converged on 20 imputed data sets

Rubin's (1987) rules were used to pool point and SE estimates across 20 imputed data sets, and to calculate degrees of freedom for each parameter's t test and CI.

Model Test User Model:

Standard Scaled
Test statistic 103.339 127.092
Degrees of freedom 53 53
P-value 0.000 0.000

Model Test Baseline Model:

Test statistic 757.618 689.170
Degrees of freedom 66 66
P-value 0.000 0.000

User Model versus Baseline Model:

Comparative Fit Index (CFI) 0.927 0.881
Tucker-Lewis Index (TLI) 0.909 0.852

Root Mean Square Error of Approximation:

RMSEA 0.075 0.091
Confidence interval - lower 0.053 0.071
Confidence interval - upper 0.096 0.111
P-value H_0: RMSEA <= 0.05 0.031 0.001

I was previously seeing a "Robust Comparative Fit Index" and "Robust Tucker-Lewis Index," and similar variant for the RMSEA under their (ostensibly) non-robust counterparts. Was this deprecated in the recent updates? If running a WLSMV-estimated CFA, should one just now interpret the values under the scaled column? I'm just wondering if this is aligned with the studies that have been referenced here on the lavaan group when there are questions about interpreting the robust statistics whenever provided (e.g., https://www.tandfonline.com/doi/full/10.1080/00273171.2018.1455142 and https://www.tandfonline.com/doi/full/10.1080/00273171.2020.1717922).

Thanks,

Bryan

Terrence Jorgensen

unread,

Mar 20, 2025, 7:25:11 AM3/20/25

to lavaan

I recently updated my lavaan and lavaan.mi packages

But you are still using the semTools function, which never provided them in this scenario:

> summary(cfa2out, standardized = TRUE, rsquare=TRUE, fit.measures=TRUE, test = "D2",
+ pool.robust = TRUE)
OLDlavaan.mi object based on 20 imputed data sets.
See class?OLDlavaan.mi help page for available methods.

If you use both in a session, make sure you load semTools first, then load lavaan.mi so it masks the deprecated semTools functions.

Terrence D. Jorgensen (he, him, his)

Assistant Professor, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

http://www.uva.nl/profile/t.d.jorgensen

Bryan Stiles

unread,

Mar 23, 2025, 1:19:42 PM3/23/25

to lavaan

Thank you. I re-ran this with lavaan.mi and am returning the following output, which I believe is using the most current function.

Having this now, I want to check my understanding of the fit indices based on what I have read on the lavaan forum. Recognizing I am running a WLSMV-estimated CFA:

1) Whenever a robust variant of the chi-square, CFI, TLI, RMSEA, etc. is returned (e.g., "cfi.robust," "tli.robust"), these should be interpreted instead of the ".scaled" variant, correct?

2) Regarding specifying "pool.robust = TRUE/FALSE," I believe you had written elsewhere on the forum (https://groups.google.com/g/lavaan/c/Zp4-xlbcfbI) that "semTools currently applies the naïve formulas using the robust chi-squareds, but a truly appropriate method should probably pool the naïve statistics, then apply a robust adjustment afterward (e.g., using some kind of pooled scaling factor, and shift factor if applicable)." I'm not sure if that comment applies here, but if it does, would it be ideal keeping the default in fitmeasures(), pool.robust = FALSE (displayed below), which I believe provides this adjustment?

Thank you,

Bryan

:

> cfa1out <- cfa.mi(cfa1, data=imp2final, ordered=c("SCS_1rev", "SCS_2", "SCS_3", "SCS_4rev", "SCS_5", "SCS_6",
+ "SCS_7", "SCS_8rev", "SCS_9rev", "SCS_10", "SCS_11rev", "SCS_12rev"),
+ estimator = "WLSMV")
Found more than one class "lavaan.mi" in cache; using the first, from namespace 'lavaan.mi'
Also defined by ‘semTools’
> summary(cfa1out, standardized = TRUE)
lavaan.mi object fit to 20 imputed data sets using:
- lavaan (0.6-19)
- lavaan.mi (0.1-0)
See class?lavaan.mi help page for available methods.

Convergence information:
The model converged on 20 imputed data sets.
Standard errors were available for all imputations.

Estimator DWLS
Optimization method NLMINB
Number of model parameters 60

Number of observations 250

Model Test User Model:

Standard Scaled

Test statistic 247.462 326.892
Degrees of freedom 54 54
P-value 0.000 0.000
Average scaling correction factor 0.794
Average shift parameter 15.266
simple second-order correction
Pooling method D2
Pooled statistic “standard”
“scaled.shifted” correction applied AFTER pooling

Parameter Estimates:

Parameterization Delta
Standard errors Robust.sem
Information Expected
Information saturated (h1) model Unstructured

Pooled across imputations Rubin's (1987) rules
Augment within-imputation variance Scale by average RIV
Wald test for pooled parameters t(df) distribution

Pooled t statistics with df >= 1000 are displayed with
df = Inf(inity) to save space. Although the t distribution
with large df closely approximates a standard normal
distribution, exact df for reporting these t tests can be
obtained from parameterEstimates.mi()

Latent Variables:
Estimate Std.Err t-value df P(>|t|) Std.lv Std.all
scs =~
SCS_1rev 0.629 0.038 16.573 Inf 0.000 0.629 0.629
SCS_2 0.635 0.038 16.811 Inf 0.000 0.635 0.635
SCS_3 0.667 0.037 18.005 Inf 0.000 0.667 0.667
SCS_4rev 0.653 0.038 16.969 Inf 0.000 0.653 0.653
SCS_5 0.380 0.042 9.041 Inf 0.000 0.380 0.380
SCS_6 0.626 0.038 16.259 Inf 0.000 0.626 0.626
SCS_7 0.617 0.038 16.083 Inf 0.000 0.617 0.617
SCS_8rev 0.621 0.038 16.312 Inf 0.000 0.621 0.621
SCS_9rev 0.664 0.038 17.242 Inf 0.000 0.664 0.664
SCS_10 0.456 0.043 10.660 Inf 0.000 0.456 0.456
SCS_11rev 0.755 0.028 26.719 Inf 0.000 0.755 0.755
SCS_12rev 0.672 0.035 19.283 Inf 0.000 0.672 0.672

Thresholds:
Estimate Std.Err t-value df P(>|t|) Std.lv Std.all
SCS_1rev|t1 -1.340 0.112 -11.973 Inf 0.000 -1.340 -1.340
SCS_1rev|t2 -0.517 0.084 -6.184 Inf 0.000 -0.517 -0.517
SCS_1rev|t3 0.293 0.081 3.628 Inf 0.000 0.293 0.293
SCS_1rev|t4 1.009 0.096 10.486 Inf 0.000 1.009 1.009
SCS_2|t1 -2.042 0.182 -11.222 Inf 0.000 -2.042 -2.042
SCS_2|t2 -1.066 0.098 -10.829 Inf 0.000 -1.066 -1.066
SCS_2|t3 -0.034 0.080 -0.428 Inf 0.668 -0.034 -0.034
SCS_2|t4 0.930 0.093 9.948 Inf 0.000 0.930 0.930
SCS_3|t1 -2.409 0.258 -9.331 Inf 0.000 -2.409 -2.409
SCS_3|t2 -1.461 0.120 -12.209 Inf 0.000 -1.461 -1.461
SCS_3|t3 -0.479 0.083 -5.769 Inf 0.000 -0.479 -0.479
SCS_3|t4 0.656 0.086 7.611 Inf 0.000 0.656 0.656
SCS_4rev|t1 -1.433 0.118 -12.169 Inf 0.000 -1.433 -1.433
SCS_4rev|t2 -0.559 0.084 -6.634 Inf 0.000 -0.559 -0.559
SCS_4rev|t3 0.274 0.081 3.396 Inf 0.001 0.274 0.274
SCS_4rev|t4 0.915 0.093 9.842 Inf 0.000 0.915 0.915
SCS_5|t1 -1.852 0.156 -11.891 Inf 0.000 -1.852 -1.852
SCS_5|t2 -1.192 0.104 -11.463 Inf 0.000 -1.192 -1.192
SCS_5|t3 -0.185 0.080 -2.310 Inf 0.021 -0.185 -0.185
SCS_5|t4 0.981 0.095 10.299 Inf 0.000 0.981 0.981
SCS_6|t1 -1.136 0.101 -11.202 Inf 0.000 -1.136 -1.136
SCS_6|t2 -0.295 0.081 -3.647 Inf 0.000 -0.295 -0.295
SCS_6|t3 0.502 0.083 6.017 Inf 0.000 0.502 0.502
SCS_6|t4 1.353 0.113 12.009 Inf 0.000 1.353 1.353
SCS_7|t1 -1.911 0.163 -11.717 Inf 0.000 -1.911 -1.911
SCS_7|t2 -1.216 0.105 -11.562 Inf 0.000 -1.216 -1.216
SCS_7|t3 -0.192 0.080 -2.392 Inf 0.017 -0.192 -0.192
SCS_7|t4 0.827 0.090 9.158 Inf 0.000 0.827 0.827
SCS_8rev|t1 -1.216 0.105 -11.562 Inf 0.000 -1.216 -1.216
SCS_8rev|t2 -0.212 0.080 -2.643 Inf 0.008 -0.212 -0.212
SCS_8rev|t3 0.490 0.083 5.893 Inf 0.000 0.490 0.490
SCS_8rev|t4 1.237 0.106 11.646 Inf 0.000 1.237 1.237
SCS_9rev|t1 -1.098 0.100 -11.009 Inf 0.000 -1.098 -1.098
SCS_9rev|t2 -0.233 0.080 -2.894 Inf 0.004 -0.233 -0.233
SCS_9rev|t3 0.305 0.081 3.773 Inf 0.000 0.305 0.305
SCS_9rev|t4 0.994 0.096 10.391 Inf 0.000 0.994 0.994
SCS_10|t1 -1.461 0.120 -12.209 Inf 0.000 -1.461 -1.461
SCS_10|t2 -0.656 0.086 -7.611 Inf 0.000 -0.656 -0.656
SCS_10|t3 0.316 0.081 3.898 Inf 0.000 0.316 0.316
SCS_10|t4 1.379 0.114 12.068 Inf 0.000 1.379 1.379
SCS_11rev|t1 -1.329 0.111 -11.944 Inf 0.000 -1.329 -1.329
SCS_11rev|t2 -0.316 0.081 -3.898 Inf 0.000 -0.316 -0.316
SCS_11rev|t3 0.468 0.083 5.645 Inf 0.000 0.468 0.468
SCS_11rev|t4 1.136 0.101 11.202 Inf 0.000 1.136 1.136
SCS_12rev|t1 -1.751 0.144 -12.120 Inf 0.000 -1.751 -1.751
SCS_12rev|t2 -0.668 0.086 -7.732 Inf 0.000 -0.668 -0.668
SCS_12rev|t3 0.181 0.080 2.266 Inf 0.023 0.181 0.181
SCS_12rev|t4 0.978 0.095 10.283 Inf 0.000 0.978 0.978

Variances:
Estimate Std.Err t-value df P(>|t|) Std.lv Std.all
scs 1.000 1.000 1.000
.SCS_1rev 0.604 0.604 0.604
.SCS_2 0.596 0.596 0.596
.SCS_3 0.555 0.555 0.555
.SCS_4rev 0.574 0.574 0.574
.SCS_5 0.855 0.855 0.855
.SCS_6 0.609 0.609 0.609
.SCS_7 0.619 0.619 0.619
.SCS_8rev 0.614 0.614 0.614
.SCS_9rev 0.559 0.559 0.559
.SCS_10 0.792 0.792 0.792
.SCS_11rev 0.430 0.430 0.430
.SCS_12rev 0.548 0.548 0.548

> fitmeasures(cfa1out)
"D3" and "D4" only available using maximum likelihood estimation. Changed to pool.method = "D2".
Found more than one class "lavaan.mi" in cache; using the first, from namespace 'lavaan.mi'
Also defined by ‘semTools’

Test statistic(s) pooled using the D2 pooling method.
Pooled statistic: “standard” (pool.robust=FALSE)
Method to robustify pooled statistic: “scaled.shifted”

npar fmin chisq
60.000 0.500 247.462
df pvalue chisq.scaled
54.000 0.000 326.892
df.scaled pvalue.scaled chisq.scaling.factor
54.000 0.000 0.794
baseline.chisq baseline.df baseline.pvalue
4405.546 66.000 0.000
baseline.chisq.scaled baseline.df.scaled baseline.pvalue.scaled
2150.401 66.000 0.000
baseline.chisq.scaling.factor cfi tli
2.082 0.955 0.946
cfi.scaled tli.scaled cfi.robust
0.869 0.840 0.780
tli.robust nnfi rfi
0.732 0.946 0.931
nfi pnfi ifi
0.944 0.772 0.956
rni nnfi.scaled rfi.scaled
0.955 0.840 0.814
nfi.scaled pnfi.scaled ifi.scaled
0.848 0.694 0.870
rni.scaled nnfi.robust rni.robust
0.869 0.732 0.780
rmsea rmsea.ci.lower rmsea.ci.upper
0.120 0.105 0.135
rmsea.ci.level rmsea.pvalue rmsea.close.h0
0.900 0.000 0.050
rmsea.notclose.pvalue rmsea.notclose.h0 rmsea.scaled
1.000 0.080 0.142
rmsea.ci.lower.scaled rmsea.ci.upper.scaled rmsea.pvalue.scaled
0.128 0.158 0.000
rmsea.notclose.pvalue.scaled rmsea.robust rmsea.ci.lower.robust
1.000 0.137 0.123
rmsea.ci.upper.robust rmsea.pvalue.robust rmsea.notclose.pvalue.robust
0.151 0.000 1.000
rmr rmr_nomean srmr
0.067 0.085 0.085
srmr_bentler srmr_bentler_nomean crmr
0.067 0.085 0.070
crmr_nomean srmr_mplus srmr_mplus_nomean
0.092 NA NA
cn_05 cn_01 gfi
73.602 82.573 0.970
agfi pgfi mfi
0.937 0.460 0.678
wrmr
1.473

Terrence Jorgensen

unread,

Mar 24, 2025, 5:19:10 AM3/24/25

to lavaan

1) Whenever a robust variant of the chi-square, CFI, TLI, RMSEA, etc. is returned (e.g., "cfi.robust," "tli.robust"), these should be interpreted instead of the ".scaled" variant, correct?

Yes.

2) Regarding specifying "pool.robust = TRUE/FALSE," I believe you had written elsewhere on the forum (https://groups.google.com/g/lavaan/c/Zp4-xlbcfbI)

That post is from 2017, before lavaan was updated to allow pool.robust=FALSE for this scenario.

would it be ideal keeping the default in fitmeasures(), pool.robust = FALSE (displayed below), which I believe provides this adjustment?

Yes, that is the better method.

Bryan Stiles

unread,

Mar 25, 2025, 9:22:56 PM3/25/25

to lavaan

Thank you for your assistance!

Combasco

unread,

Jun 2, 2025, 6:31:50 AM6/2/25

to lavaan

Hey all,

I'd like to press a bit further on this point, in order to be able to justify these decisions I'm also making in my thesis.

1) Whenever a robust variant of the chi-square, CFI, TLI, RMSEA, etc. is returned (e.g., "cfi.robust," "tli.robust"), these should be interpreted instead of the ".scaled" variant, correct?

Yes.

Why?
I'm trying to understand what actually happens with indices like cfi.robust, tli.robust, etc. when using WLSMV estimation and how it's different from cfi.scaled, tli.scaled etc. From what I understand, cfi.scaled is computed using the scaled chi-square statistic (which itself is based on the pooled naive chi-square, adjusted using the D2 test) in the normal formula for CFI. In that sense, it's more of an approximation than a truly robust version of the fit index. So here's my question: what's the actual difference between cfi.scaled and cfi.robust? I was under the impression that robust versions of the CFI, TLI, RMSEA, etc., don't yet exist for WLSMV.

Would appreciate any clarification!

Tim

Op woensdag 26 maart 2025 om 02:22:56 UTC+1 schreef bryans...@gmail.com:

Terrence Jorgensen

unread,

Jun 12, 2025, 7:15:58 AM6/12/25

to lavaan

1) Whenever a robust variant of the chi-square, CFI, TLI, RMSEA, etc. is returned (e.g., "cfi.robust," "tli.robust"), these should be interpreted instead of the ".scaled" variant, correct?

Yes.
Why?

http://dx.doi.org/10.1080/00273171.2012.715252

http://dx.doi.org/10.1080/00273171.2014.933697

Reply all

Reply to author

Forward