Fit Configural and Fit Threshold Purpose in Invariance Testing

[Brandon Bretl]

I'm conducting invariance testing between 4 groups on 8 factors, one factor at a time. When I run the syntax, the "fit.config" and "fit.thresh" are always the same and I get an error message indicating that some models "have the same degrees of freedom." I understand configural, metric, and scalar invariance testing to a certain degree, but I'm not sure what the fit.config/fit.thresh is supposed to accomplish. Below is the first part of the code I'm using and output. 

So what is the purpose of the fit.config and fit.thresh comparison?

Thanks,
Brandon

#Invariance testing
syntax.config <- measEq.syntax(configural.model = mft_fact, data=mft_dat, parameterization="theta",
                               ID.fac="std.lv", ID.cat = "Wu.Estabrook.2016", group = g1)
cat(as.character(syntax.config))
summary(syntax.config)
## threshold invariance
syntax.thresh <- measEq.syntax(configural.model = mft_fact, data = mft_dat,
                               parameterization = "theta",
                               ID.fac = "std.lv", ID.cat = "Wu.Estabrook.2016",
                               group = g1, group.equal = "thresholds",
                               long.equal = "thresholds")
## notice that constraining 4 thresholds allows intercepts and residual
## variances to be freely estimated in all but the first group & occasion
cat(as.character(syntax.thresh))
## print a summary of model features
summary(syntax.thresh)
## Fit a model to the data either in a subsequent step:
mod.config <- as.character(syntax.config)
fit.config <- cfa(mod.config, data = mft_dat, group = g1,
                  parameterization = "theta")
## or in a single step:
fit.thresh <- measEq.syntax(configural.model = mft_fact, data = mft_dat,
                            parameterization = "theta",
                            ID.fac = "std.lv", ID.cat = "Wu.Estabrook.2016",
                            group = g1, group.equal = "thresholds",
                            long.equal = "thresholds", return.fit = TRUE)
## compare their fit to test threshold invariance
anova(fit.config, fit.thresh)

...

> compareFit(fit.config, fit.thresh, fit.metric, fit.scalar)

################### Nested Model Comparison #########################

Chi-Squared Difference Test

           Df   AIC   BIC   Chisq Chisq diff Df diff Pr(>Chisq)    

fit.config 32 13887 14256  58.579                                  

fit.thresh 32 13887 14256  58.579      0.000       0               

fit.metric 47 13881 14177  82.684     24.105      15    0.06334 .  

fit.scalar 62 13908 14131 139.644     56.960      15  8.336e-07 ***

---

Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

####################### Model Fit Indices ###########################

              chisq df pvalue   cfi   tli        aic        bic rmsea  srmr

fit.config  58.579† 32   .003 .983† .968  13886.745  14256.156  .059  .028†

fit.thresh  58.579† 32   .003 .983† .968  13886.745  14256.156  .059  .028†

fit.metric  82.684  47   .001 .977  .971† 13880.850† 14177.351  .056† .051 

fit.scalar 139.644  62   .000 .951  .952  13907.810  14131.401† .072  .062 

################## Differences in Fit Indices #######################

                        df    cfi    tli    aic     bic  rmsea  srmr

fit.thresh - fit.config  0  0.000  0.000  0.000   0.000  0.000 0.000

fit.metric - fit.thresh 15 -0.006  0.003 -5.895 -78.805 -0.003 0.023

fit.scalar - fit.metric 15 -0.027 -0.019 26.960 -45.950  0.016 0.011

Warning message:

In (function (object, ..., method = "default", A.method = "delta",  :

  lavaan WARNING: some models have the same degrees of freedom

[Edward Rigdon]

Brandon--

     You need to read Wu and Estabrook (2016) and decide for yourself. Yes, DF for these two models are equal. In the absence of equal thresholds, lavaan imposes other constraints to make the model identified. When thresholds are constrained equal, lavaan does not impose those additional constraints. Wu and Estabrook basically argue that "thresholds equal" *is* the configural or baseline equality model for measurement invariance with ordinal observed variables. The paper is heavy going, but only because you leave the domain of simple answers to face the actual complexity.

https://link.springer.com/article/10.1007/s11336-016-9506-0  

     Also, Terry has said he is working on a tutorial to help people dealing with the complexities of ordinal variable SEM. I'm sure ot will be awesome when available (probably should be a "Tutorial" paper in Psych Methods).

--Ed Rigdon

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/d4e65cf2-e7b1-426e-8158-55c98f63eef3%40googlegroups.com.

[Terrence Jorgensen]

When I run the syntax, the "fit.config" and "fit.thresh" are always the same and I get an error message indicating that some models "have the same degrees of freedom." 

Then it sounds like your indicators have 3 categories, in which case equivalent thresholds is statistically equivalent to any minimally identified configural model (because you exchange 1 threshold to estimate the item's intercept, and a second threshold to estimate the item's residual variance, but no more thresholds are available to buy degrees of freedom).  The comments in the syntax you posted look like mine from the ?measEq.syntax help page, for an application to 5-category data.  Notice the comments below those examples warn about 3- and 2-category data (final example is binary, skipping steps that cannot be differentiated).

Terrence D. Jorgensen

Assistant Professor, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

http://www.uva.nl/profile/t.d.jorgensen

[Edward Rigdon]

Even when I think I've got it...

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/d7c76838-a78d-4ed0-9a27-c1cdabb1e134%40googlegroups.com.

[Brandon Bretl]

Thanks, Terrence. I went back and read the measEq.syntax, but I'm not sure what you mean by "categories." In my model, I have 1 factor, 6 indicators, and each indicator has 5 response options (Likert 1-5). I am not testing repeated measures or longitudinal data. 

cpa <- '
cpa =~ Q1_2 + Q1_8 + Q2_2 + Q3_2 + Q4_2 + Q5_2
'

I ran the syntax again from the measEq.syntax and I still get this:

Chi-Squared Difference Test

           Df   AIC   BIC   Chisq Chisq diff Df diff Pr(>Chisq)    

fit.config 36 15477 15827  46.739                                  

fit.thresh 36 15477 15827  46.739      0.000       0               

fit.metric 51 15473 15750  72.645     25.905      15  0.0390291 *  

fit.scalar 66 15486 15690 115.471     42.826      15  0.0001675 ***

[Terrence Jorgensen]

I'm not sure what you mean by "categories." ... each indicator has 5 response options

Those are categories.  Did you look at your configural model's summary() to see if you even estimated any thresholds?  You are not declaring any variables as ordered=, in which case lavaan won't know they are ordered unless they are stored as ordered factors in your data.frame.

http://lavaan.ugent.be/tutorial/cat.html