semTools compareFit() question

[Aaron]

The new measEq.syntax() function is great! It greatly simplifies measurement invariance testing. I have been having a problem with the compareFit() function though. I am testing long invariance of categorical items, so I am constraining thresholds, then loadings, then intercepts, then means, and finally residuals using Wu.Estabrook.2016 for identification, similar to the example. I ran everything in a loop and put the results in a list, like the example as well. When I run compareFit on the full list, I get this error:

Error in svd(X) : a dimension is zero

If I use the anova() function with pairwise comparisons, it gives me the same error if I include the configural model, but when comparing two of the constrained models. If I try to use compareFit() with pairwise comparisons, though, I get a different error:

Error in data.frame(Df = Df, AIC = aic, BIC = bic, Chisq = STAT, `Chisq diff` = STAT.delta,  : 

  row names contain missing values

With the anova() output, AIC and BIC values aren't given so I wonder if that is part of the problem in the pairwise comparisons. Any other ideas why I might be having these errors?

(as an additional side - I know semTools calculates additional fit indices as well. Is it possible to output change in Gamma hat as well? I've had a journal editor request those two before and have always had to calculate them after the fact).

Thanks,

Aaron

[Terrence Jorgensen]

AIC and BIC values aren't given

Because you have categorical data and do not use ML estimation.

so I wonder if that is part of the problem in the pairwise comparisons.

No, the help page example works fine, and that uses categorical data.

Any other ideas why I might be having these errors?

Do you have binary data?  If so, you need to go directly from configural to simultaneously constraining thresholds, loadings, and intercepts.

Also, check that you have the development version installed (0.5-1.908), which may have fixed this error already.

sessionInfo() # check version
# if necessary:
devtools::install_github("simsem/semTools/semTools")

(as an additional side - I know semTools calculates additional fit indices as well. Is it possible to output change in Gamma hat as well? I've had a journal editor request those two before and have always had to calculate them after the fact).

I don't recommend relying on changes in any fit index:

https://doi.org/10.1037/met0000152

https://doi.org/10.1080/10705511.2017.1421467

But compareFit() currently only calls fitMeasures().  I'll make a note to add an argument for moreFitIndices() like the permuteMeasEq() function has.

Terrence D. Jorgensen

Assistant Professor, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

http://www.uva.nl/profile/t.d.jorgensen

[Aaron]

Thanks Terrence,

I was able to install the development version and that addresses the second error - the row names problem when not including the configural model - but I am still getting the svd(X) error when I include the configural model. Any thoughts about that one?

And I agree about fit indices--none are perfect--but people still expect them. It would be great if a future version could request moreFitIndices()in addition to fitMeasures()

Thanks,

Aaron

[Terrence Jorgensen]

I am still getting the svd(X) error when I include the configural model. Any thoughts about that one?

Did you pay attention to this question?  

Do you have binary data?  If so, you need to go directly from configural to simultaneously constraining thresholds, loadings, and intercepts.

It is the only situation I can think of that results in that error, which (if you have binary items) results from comparing statistically equivalent models (constraining only thresholds merely releases constraints on intercepts, so the df and fit are the same).

[Aaron Kaat]

Sorry, I did see that question but didn't respond to it. I have 1 binary item but the rest are 3-point ordinal.

--
You received this message because you are subscribed to a topic in the Google Groups "lavaan" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lavaan/EoGEZCv_ZrU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lavaan+un...@googlegroups.com.
To post to this group, send email to lav...@googlegroups.com.
Visit this group at https://groups.google.com/group/lavaan.
For more options, visit https://groups.google.com/d/optout.

[Terrence Jorgensen]

Sorry, I did see that question but didn't respond to it. I have 1 binary item but the rest are 3-point ordinal. 

OK, I think 3-category data lead to the same equivalence issue as binary data.  Think of thresholds as the "measurement model" parameters for the underlying latent item-responses (LIR).  If you only have 1 threshold, you can only estimate it if you assume the LIR is a z-score (M=0, SD=1).  But if fix the threshold to 0, you can trade that degree of freedom for another piece of information:

• If you estimate the mean of the LIR, it will be the same magnitude but opposite sign as the threshold estimate was
• If you fix the mean and threshold to different arbitrary values (e.g., 0 and 1, or -2 and +2), the estimated variance or the LIR will be whatever it needs to be so that the observed proportion of the sample in the first category will be below your first threshold

If you have 2 thresholds, again you can estimate both if you fix the LIR's M=0 and SD=1.  But if you fix the thresholds to arbitrary values (e.g., 0 and 1, which is what LISREL does), you can trade those degrees of freedom to estimate the M and SD of the LIR instead.  They are all statistically equivalent models because they yield the same model-implied proportions in each response category.

Measurement equivalence works the same way, except instead of fixing thresholds to arbitrary values, you simply set the second group's thresholds to be the same as the first group's, and you use those df to buy estimates of the second group's LIR's M and SD (actually, intercept and residual variance in a CFA).  So using the default Wu & Estabrook guidelines, constraining item thresholds to equality frees the corresponding item intercepts and (residual) variances, resulting in a model that is statistically equivalent to the default configural model in which the thresholds were freely estimated on the condition that intercepts and residual variances were equal.  

So the solution is to start with constrained thresholds -- use that as your configural model.  Then add constraints on loadings, then intercepts, ...

Now that I have a break from teaching, I am hoping to resolve some default behaviors in measEq.syntax(), although I'm not sure there is anything I can really do to prevent issues like this other than adding checks that issue warnings when people have binary/ternary data.  At a minimum, I will add binary and ternary examples to the help-page examples, which users will hopefully notice when it applies to their data.

[Terrence Jorgensen]

It would be great if a future version could request moreFitIndices()in addition to fitMeasures()

This feature is now available in version 0.5-4.911's compareFit() by setting moreIndices=TRUE.  Selecting particular indices to print is controlled with summary(..., fit.measures=), shown in the updated help-page examples.

devtools::install_github("simsem/semTools/semTools")