How to test if there is significant configural measurement invariance using lavaan group syntax?

[Brent Caldwell]

Dear Lavaan group

I am trying to do measurement invariance on a cfa of a psychological battery.  measurementInvarianceCat worked well 

for all but one grouping (Gender) so I thought I would try using the equivalent syntax in Lavaan for multiple group cfa.

My question is how to test the significance of the configural model, ie how to test if the two groups have similar factor structure and pattern of item to factor pattern?  

I thought that my code was too ugly so I've used generic illustrative code below (I have hideously long vector names to help me remember what on earth they are) 

I know that the anova can be used to test the significance of the increasing restrictions, for example 

cfa.no.group <- cfa(model,data)

cfa.config <- cfa(model,data,group="group")

cfa.loadings <- cfa(model,data,group="group",group.equal="loadings", parameterization="theta")

cfa.loadings.thresholds <- cfa(model,data,group="group",group.equal=c("loadings","thresholds"), parameterization="theta")

cfa.loadings.thresholds.residuals <- cfa(model,data,group="group",group.equal=c("loadings","thresholds","residuals"), parameterization="theta")

I can see very easily how to test the above models compared to the confirg model, using anova, but to what does one compare the config model to 

see if adding the configural constraints had a significant effect?  I tried anova(cfa.no.group , cfa.config) but got an error message:

Error in lav_test_diff_af_h1(m1 = m1, m0 = m0) : 

  lavaan ERROR: unconstrained parameter set is not the same in m0 and m1

  

I have a large sample size so I'm worried about focussing on the chi squared, and so I thought I'd manually calculate the difference in cfi between

cfa.no.group and cfa.config, which was 0.012, which is greater than 0.01 which from memory Hu & Bentler would say indicates a significant deprecation in goodness of fit, so there might not be configural invariance

Thank you very much for your advice.

P.S. if you are interested in what happened to group="Gender" with measurementInvarianceCat, it produced an error message that it had not converged. 

I was curious about this, so I ran two cfa, one in which I subset the data to male data, and one in which I subset to female.  Both the male and female cfa

converged well, and as far as I could see they had very similar goodness of fit indices.  

Thank you very much

Brent

[Brent Caldwell]

I should add that I'm using lavaan version 0.5.20

Thanks for your help 

[Terrence Jorgensen]

I can see very easily how to test the above models compared to the confirg model, using anova, but to what does one compare the config model to see if adding the configural constraints had a significant effect?  I tried anova(cfa.no.group , cfa.config)

That is not "adding configural constraints" -- the no-groups model is essentially your most restricted multiple-group model, in the sense that you assume all parameters are equivalent across groups.  So cfa.no.group is actually nested within cfa.config, not the other way around.  (But to get that chi-squared difference using anova(), you would have to fit cfa.no.group as a multigroup model where all parameters are constrained to equality.)

The less-constrained model to which the configural model is compared is the saturated model, and that chi-squared difference is the chi-squared for the configural model (because chi-squared for the saturated model = 0, due to df = 0).  In other words, the conventional test for the configural model is a test of absolute fit, not a model comparison.  See Cheung & Rensvold (2002) or Meade et al. (2008) for details.

I have a large sample size so I'm worried about focussing on the chi squared, and so I thought I'd manually calculate the difference in cfi between cfa.no.group and cfa.config, which was 0.012, which is greater than 0.01 which from memory Hu & Bentler would say indicates a significant deprecation in goodness of fit, so there might not be configural invariance

Fair enough, but Hu & Bentler (1998, 1999) investigated using fit indices for absolute fit, not model comparisons in general nor measurement invariance specifically.  The Delta-CFI < .01 cutoff was suggested by Cheung & Rensvold (2002, see link above), and Meade et al. (2008) provided a stricter cutoff (.002) for greater power.  Neither of these yields consistent Type I error rates across a variety of models and sample sizes, though. 

But again, that cutoff has nothing to do with testing configural invariance.  That is a question of whether the least-constrained multigroup model fits well in an absolute sense.  If you fear chi-squared is too sensitive, you can use CFI as an indicator of approximate fit, using traditional guidelines (CFI > .90 indicates adequate fit, > .95 is good fit).  Hu & Bentler tentatively recommended a cutoff of .95, based on continuous data and one type of CFA model, so take that with a packet of salt.

P.S. if you are interested in what happened to group="Gender" with measurementInvarianceCat, it produced an error message that it had not converged. 

You got this message from the configural model?  If it was from a model with constraints, then fitting the model to one group at a time isn't analogous to the multigroup model.  

The models you are fitting above (using the group.equal argument) are not the models fitted by the measurementInvarianceCat() function.  To read about the models it does fit, see this page:

http://sunthud.com/catInvariance.html

Reading those example syntax files and adapting them for your gender model should help you narrow does which model is failing to converge.

Terry

[Brent Caldwell]

Thank you so much for your help!  The thing that didn’t converge was:

measurementInvarianceCat(model,  data=data, group = "Male ", parameterization="theta",estimator="wlsmv",

                ordered=c(" . . . all the names of the items in the data, each of them separated by a comma”))

 

I forgot to say in my post that the psychological battery that the data are from, contains 37 items and 11 primary factors, and 4 correlated residual error covariances and there are 1357 observations per item (the same size is 1357).  I had a look at the link to Sunthud’s website, but he says that the code assumes no measurement error correlations, and code is of a style that is so different from what I’m used to, that I don’t understand it.  I’m trained in medicine not statistics, and have picked up R through the kind help of R experts who have helped me to analyse data from the clinical trials that I have done.  Sometimes the gaps in what I know about R make me feel so stupid! 

 

Thank you so much!

Best wishes

Brent

--
You received this message because you are subscribed to a topic in the Google Groups "lavaan" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lavaan/FQ5EWmclbjI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lavaan+un...@googlegroups.com.
To post to this group, send email to lav...@googlegroups.com.
Visit this group at https://groups.google.com/group/lavaan.
For more options, visit https://groups.google.com/d/optout.

[Brent Caldwell]

Hi, thank you so much for your advice, I've attached my data and my code (I've shortened the original vector names), I'd be so grateful to anyone who could tell me why measurementInvarianceCat or how I can use the lavaan syntax to investigate  the cfa with group=Male.bis4"  in it.   does not converge, even though the separate cfa's for males and females separately fit fine, and have very similar goodness of fit indices.  The attached code takes a while to run, sometimes the measurementInvarianceCat commands take 15 minutes each on my computer which has cpu of 3.6GHz I think.

[Terrence Jorgensen]

Hi, thank you so much for your advice, I've attached my data and my code (I've shortened the original vector names), I'd be so grateful to anyone who could tell me why measurementInvarianceCat or how I can use the lavaan syntax to investigate  the cfa with group=Male.bis4"  in it.   does not converge, even though the separate cfa's for males and females separately fit fine, and have very similar goodness of fit indices.  The attached code takes a while to run, sometimes the measurementInvarianceCat commands take 15 minutes each on my computer which has cpu of 3.6GHz I think.

I would certainly expect a model with 37 ordinal indicators to run slowly, but I think this is compounded by the fact that your indicators all have 7 categories.  This is why it is so helpful to provide the data and script that cause a problem.  Your models run fine when you treat the data as continuous, although you should still use a robust estimator (MLR) to adjust for non-normality.  With 5 or more categories, you can expect comparable results whether you treat data as ordered or continuous (read this article for details), but the latter method tends to cause fewer problems like the ones you encountered.  Now, the only warnings are that there are 2 missing values of gender (you can safely ignore that), and that the latent covariance matrix is not positive definite.  The cause of this appears to be redundancy among at least 2 factors (correlations approaching 1, sometimes exceeding 1 -- see my comments at the bottom of the attachment).

I cleaned up your code to make it more legible, in case you ever need to post more syntax in the future.  There are also many tricks to making it shorter and easier to read, which I hope are helpful to you as well (e.g., using paste() instead of typing out 37 variable names, read.csv() already imports a data.frame so you don't have to make another data.frame out of it).

Reading through the lavaan tutorial pages also might help you learn how to read Sunthud's example syntax files:

http://lavaan.ugent.be/tutorial/groups.html

Good luck,

Terry

[Brent Caldwell]

Wow thank you so much Terrence!  You have been so kind to me!  I have tried what you suggested and it all works, and it all makes sense!  It looks like in the whole sample (male and female combined) there was a very strong correlation between Loss of Control and Tolerance of 0.859, which is not strong enough to produce non positive definite matrix of latent variables.  This correlation became all the more noticeable when I used measurementInvarianceCat to compare males to females, because among females, the correlation between these variable was even higher, and the size of the correlated became out of bounds at 1.002 for females as calculated from (cor.means <- inspect(MI.gender[["fit.means"]], "cor.lv")), which explains why the correlation matrix of latent variables was not positive definite for females when they were being compared to males.  I now definitely have my answer: there is not even configural measurement invariance for gender with the scale among New Zealand smokers.  A study failed to replicate the factor structure among Spanish speaking Latino smokers in Texas due to  high intercorrelations (>.90)  among the Loss of Control, Craving, Tolerance, and Cue Exposure subscales, and another USA study also found high correlations between factors.  

Thank you so much for you help and especially for taking the time to teach me about statistics in general and writing more beautiful R code in particular.  This is vital to the veracity of the results because, as Keats noted, 

'Beauty is truth, truth beauty,—that is all
Ye know on earth, and all ye need to know.'

Thank you so much!

Best wishes

Brent 

On Monday, 25 January 2016 19:12:02 UTC+13, Brent Caldwell wrote:

[Terrence Jorgensen]

Thank you so much for you help and especially for taking the time to teach me about statistics in general and writing more beautiful R code in particular.  This is vital to the veracity of the results because, as Keats noted, 

'Beauty is truth, truth beauty,—that is all
Ye know on earth, and all ye need to know.'

Agreed :-)  Good luck in your research,

Terry

[Brent Caldwell]

Dear Terrance

I apologise for continuing to ask more questions.  Am I correct in thinking that because there were acceptable goodness of fit measures, when I fitted to the model data subset so that it only contained data from females, that this meant that data from females fit the model?  And because there was not even configural measurement invariance for gender, this means that data from females does not fit the model when variables like indicator-to-latent factor pattern are constrained to be the same as males, and this probably means that females must have a different indicator to latent factor loading pattern?  So I thought I would try to calculate factor scores for females from the cfa that was fitted to only female data, which was:

fit.model1.female <- cfa(model1,ordered=paste0("v",1:37),data=subset(vrawdataframe,Male.bis4==0))

because it wouldn’t be right to get scores for females by subsetting scores from fit.model1 <- cfa(model1,ordered=paste0("v",1:37),data=vrawdataframe) because that model assumes that there are equal indicator to factor loading pattern, equal loadings, equal thresholds, etc between genders which there isn’t.  So I tried to calculate the scores with predict(fit.model1.female) but all the scores were NA as can be seen from

summary(predict(fit.model1.female))

What have I don’t wrong? How can there be no scores in this fit, when it is possible to calculate goodness-of-fit measures from it summary(fit.model1.female,fit.measures=T)

Thank you so much for your patience with me!

Best wishes

Brent

 

From: lav...@googlegroups.com [mailto:lav...@googlegroups.com] On Behalf Of Terrence Jorgensen
Sent: Wednesday, 27 January 2016 9:33 p.m.
To: lavaan <lav...@googlegroups.com>
Subject: Re: How to test if there is significant configural measurement invariance using lavaan group syntax?

 

Thank you so much for you help and especially for taking the time to teach me about statistics in general and writing more beautiful R code in particular.  This is vital to the veracity of the results because, as Keats noted, 

--