Robust fit indices not available for threshold model

96 views

Skip to first unread message

blairb...@gmail.com

unread,

Sep 30, 2024, 6:21:33 PM9/30/24

to lavaan

Hello all,

I am investigating measurement invariance in lavaan. The scale has 12 items and 4 factors. The items are on a 5-point likert scale, so I used the ordered command and am following the Wu & Estabrook (2016) method. To explain some of the syntax, I wanted to freely estimate the loadings of all items (versus fixing the first item in each factor).

The configural model produced normal output with no errors. However, when evaluating threshold invariance, I am getting NAs for the robust fit indices. No warnings or errors were produced. Can anyone point me to why the robust fit indices would not compute?

Syntax and relevant output below:

#model
ies3<-'
upe=~ NA*ies3_1 + ies3_2 + ies3_3
epr=~ NA*ies3_4 + ies3_5 + ies3_6
rhsc=~ NA*ies3_7 + ies3_8 + ies3_9
bfcc=~ NA*ies3_10 + ies3_11 + ies3_12
upe ~~ epr + rhsc + bfcc
epr ~~ rhsc + bfcc
rhsc ~~ bfcc
upe ~~ 1*upe
epr ~~ 1*epr
rhsc ~~ 1*rhsc
bfcc ~~ 1*bfcc
'
items<-c("ies3_1", "ies3_2", "ies3_3", "ies3_4", "ies3_5", "ies3_6", "ies3_7", "ies3_8", "ies3_9", "ies3_10", "ies3_11", "ies3_12")

#configural
config<-measEq.syntax(configural.model = ies3, data = ies3data, group="hfss_FI", group.equal="configural", parameterization="delta", ID.fac="std.lv", ID.cat="Wu.Estabrook.2016",
ordered=items)
model.config <- as.character(config)
fit.config<-cfa(model.config, data=ies3data, group="hfss_FI", parameterization="delta", ordered=items)
summary(fit.config, fit.measures=T)

###output

lavaan 0.6-19 ended normally after 35 iterations

Estimator DWLS
Optimization method NLMINB
Number of model parameters 132

Number of observations per group: Used Total
1 276 303
0 686 718

Model Test User Model:
Standard Scaled
Test Statistic 218.778 402.833
Degrees of freedom 96 96
P-value (Chi-square) 0.000 0.000
Scaling correction factor 0.585
Shift parameter 28.644
simple second-order correction
Test statistic for each group:
1 96.052 96.052
0 306.781 306.781

Model Test Baseline Model:

Test statistic 63271.685 26585.675
Degrees of freedom 132 132
P-value 0.000 0.000
Scaling correction factor 2.387

User Model versus Baseline Model:

Comparative Fit Index (CFI) 0.998 0.988
Tucker-Lewis Index (TLI) 0.997 0.984

Robust Comparative Fit Index (CFI) 0.962
Robust Tucker-Lewis Index (TLI) 0.948

Root Mean Square Error of Approximation:

RMSEA 0.052 0.082
90 Percent confidence interval - lower 0.043 0.073
90 Percent confidence interval - upper 0.061 0.090
P-value H_0: RMSEA <= 0.050 0.371 0.000
P-value H_0: RMSEA >= 0.080 0.000 0.635

Robust RMSEA 0.083
90 Percent confidence interval - lower 0.073
90 Percent confidence interval - upper 0.094
P-value H_0: Robust RMSEA <= 0.050 0.000
P-value H_0: Robust RMSEA >= 0.080 0.709

Standardized Root Mean Square Residual:

SRMR 0.048 0.048

#threshold
thresh<-measEq.syntax(configural.model = ies3, data = ies3data, group="hfss_FI", group.equal="thresholds", parameterization="delta", ID.fac="std.lv", ID.cat="Wu.Estabrook.2016",
ordered=items)
model.thresh<-as.character(thresh)
fit.thresh<-cfa(model.thresh, data=ies3data, group="hfss_FI", ordered = items)
summary(fit.thresh, fit.measures=T)

###threshold output

lavaan 0.6-19 ended normally after 81 iterations

Estimator DWLS
Optimization method NLMINB
Number of model parameters 156
Number of equality constraints 48

Number of observations per group: Used Total
1 276 303
0 686 718

Model Test User Model:
Standard Scaled
Test Statistic 229.941 411.621
Degrees of freedom 120 120
P-value (Chi-square) 0.000 0.000
Scaling correction factor 0.612
Shift parameter 35.713
simple second-order correction
Test statistic for each group:
1 106.799 106.799
0 304.822 304.822

Model Test Baseline Model:

Test statistic 63271.685 26585.675
Degrees of freedom 132 132
P-value 0.000 0.000
Scaling correction factor 2.387

User Model versus Baseline Model:

Comparative Fit Index (CFI) 0.998 0.989
Tucker-Lewis Index (TLI) 0.998 0.988

Robust Comparative Fit Index (CFI) NA
Robust Tucker-Lewis Index (TLI) NA

Root Mean Square Error of Approximation:

RMSEA 0.044 0.071
90 Percent confidence interval - lower 0.035 0.064
90 Percent confidence interval - upper 0.052 0.079
P-value H_0: RMSEA <= 0.050 0.887 0.000
P-value H_0: RMSEA >= 0.080 0.000 0.027

Robust RMSEA NA
90 Percent confidence interval - lower NA
90 Percent confidence interval - upper NA
P-value H_0: Robust RMSEA <= 0.050 NA
P-value H_0: Robust RMSEA >= 0.080 NA

Standardized Root Mean Square Residual:

SRMR 0.048 0.048

Ingo Man

unread,

Oct 1, 2024, 8:40:34 AM10/1/24

to lavaan

Hello, your syntax seems right to me. Only, in my opinion you do not need to specify

upe ~~ 1*upe
epr ~~ 1*epr
rhsc ~~ 1*rhsc
bfcc ~~ 1*bfcc

since the command ID.fac="std.lv" says to scale the factors via variance of lvs to 1.

Try to control the syntax generated by semTools:

## print lavaan syntax to the Console

cat(as.character(thresh))

## print a summary of model features

summary(thresh)

Everything as is should be?

Maybe it has something to do with the standard estimator used? First, I would estimate the model without the ordered-commands via MLR and then try to goe through the invariance steps. Is this working?
Maybe then, try WLSMV or some other appropriate estimator. Still no robust fit measures?

Kindly,

Marcus

Terrence Jorgensen

unread,

Oct 2, 2024, 8:21:49 AM10/2/24

to lavaan

This was cross-posted on CV, where I posted an answer.

https://stats.stackexchange.com/questions/655190/robust-fit-indices-not-calculating-when-constraining-intercepts

Terrence D. Jorgensen (he, him, his)
Assistant Professor, Methods and Statistics
Research Institute for Child Development and Education, the University of Amsterdam
http://www.uva.nl/profile/t.d.jorgensen

Reply all

Reply to author

Forward

0 new messages