Multigroup CFA / Measurement invariance with ordinal indicators

[René Mõttus]

Hi All,

I would like to test for measurement invariance across groups with latent traits defined by ordered-categorical items. However, I cannot fit the multiple group models without constraining item thresholds equal in all groups (mean equality constraints don't do the trick). This is a problem in that a measurement invariance test should also involve models with freely estimated loadings & thresholds.

I copy a working sample code below.

Thanks in advance!

René Mõttus

data = read.table("http://ubuntuone.com/0skmfm74wqiCOq5b4wJTec", sep="\t", header=T)

for(a in 1:3) data[,a] = ordered(data[,a])

require(lavaan)

model = 'f1 =~ rand.var.1  + rand.var.2 + rand.var.3'

fit = cfa(model, data=data, group="grupp", group.equal="means") # configural invariance model -- does not converge 

  # it only converges when I set "thresholds" equal but I need a measurement invariance test all the way long

fit = cfa(model, data=data, group="grupp", group.equal="thresholds") # converges but I need simpler models too

summary(fit, fit.measures=T)

[yrosseel]

On 12/19/2012 03:44 PM, Ren� M�ttus wrote:
> Hi All,
>
> I would like to test for measurement invariance across groups with
> latent traits defined by ordered-categorical items. However, I cannot
> fit the multiple group models without constraining item thresholds equal
> in all groups (mean equality constraints don't do the trick). This is a
> problem in that a measurement invariance test should also involve models
> with freely estimated loadings & thresholds.

If you don't hold the thresholds fixed across groups, you need to set
the 'scaling parameters' to one in all groups:

model = ' f1 =~ rand.var.1 + rand.var.2 + rand.var.3

rand.var.1 ~*~ c(1,1)*rand.var.1
rand.var.2 ~*~ c(1,1)*rand.var.2
rand.var.3 ~*~ c(1,1)*rand.var.3
'
fit = cfa(model, data=data, group="grupp")

To read more about these 'scaling' parameters in multiple groups and
categorical data, see:

Muth�n, B. & Christoffersson, A. (1981). Simultaneous factor analysis of
dichotomous variables in several groups. Psychometrika, 46, 407-419.


I guess, lavaan users got used to the idea that lavaan should take care
of this automagically. And perhaps, I will consider it... for 0.5-12.

Yves.

[René Mõttus]

Hi Yves,

Many thanks for that. That worked.

So the MI test goes as usual?

fit.config = cfa(model, data=data, group="grupp", group.equal=c("means"))

fit.metric = cfa(model, data=data, group="grupp", group.equal=c("means", "loadings"))

fit.scalar = cfa(model, data=data, group="grupp", group.equal=c("loadings", "thresholds"))

fit.strict = cfa(model, data=data, group="grupp", group.equal=c("loadings", "thresholds", "residuals"))

René

[yrosseel]

This is just one of many possibilities. Plenty of literature out there
about MI and categorical data.

Yves.

[René Mõttus]

Thanks --- René

On Wednesday, December 19, 2012 8:32:55 PM UTC, Yves Rosseel wrote:

[robin ghertner]

Reviving an old thread here.

I'm testing measurement invariance of a 5-factor model with all ordinal indicators. As Rene, I'm following the Vandenberg framework for configural->strict invariance. The problem I'm running into is constraining the residual variances, using group.equal="residuals". When I do so, the model doesn't change at all - all fit indices are identical to the configural model (i.e. no constraints).

I understand that the residual variances are dependent on other parameters in the model, which suggests to me that perhaps the constraint is meaningless since these are not freely estimated parameters. I haven't yet found the literature that discusses this - the one paper I did find discussed equivalence of error variance, but doesn't go further.:

Temme, D. (2006). Assessing measurement invariance of ordinal indicators in cross-national research. In International advertising and communication (pp. 455-472). DUV.

http://link.springer.com/chapter/10.1007%2F3-8350-5702-2_24

Any thoughts? Something I'm missing?

Robin

[yrosseel]

On 09/13/2013 03:22 PM, robin ghertner wrote:
> Reviving an old thread here.
>
> I'm testing measurement invariance of a 5-factor model with all ordinal
> indicators. As Rene, I'm following the Vandenberg framework for
> configural->strict invariance. The problem I'm running into is
> constraining the residual variances, using group.equal="residuals". When
> I do so, the model doesn't change at all - all fit indices are identical
> to the configural model (i.e. no constraints).

The 'delta' parameterization currently used by lavaan when data are
categorical does not estimate the residuals (they are a function of
other model parameters). Hence, group.equal="residuals" has not effect.
(Perhaps a warning would be in order).

But you can constrain the 'delta' scaling factors to be equal across
groups, although you will have to do it manually. There is no
group.equal argument for it now.

If x1, x2, x3, ... are indicators of a latent variable, you could write
something like:

x1 ~*~ c(d1,d1,d1)*x1
x2 ~*~ c(d2,d2,d2)*x2
x2 ~*~ c(d3,d3,d3)*x2
...

In this example, I have assumed three groups. By setting the labels
equal across groups, the delta factors will be constrained to be equal.

Yves.

[robin ghertner]

Thanks Yves, this works fine.

[Sokolov Boris]

Hi, Yves

I tried to check measurement invariance for a model that included one latent variable defined by three continuous indicators and three latent variables, each defined by three categorical indicators. I revealed that by default Lavaan estimated a configural model with all thresholds fixed to be equal across groups. 

However, if I understand the concept of MI correctly, the configural model should have group- specific estimates for each parameter, exluding only parameters restricted to achieve identification. So, to test invariance for a model that include both contionuous and categorical indicators, one should consequently fit four models and then compare them by some kind of a LR test.

These four models are:

1) Configural model: all loadings, intercepts and thresholds are group-specific.

2) Model with equal loadings

3) Model with equal loadings and intercepts

4) Model with equal loadings, intercepts, and thresholds

I managed to fit the latter three models by fixing (and freeing) relevant parameters manually in Lavaan. To achieve identification, all latent variances were fixed to 1 in all groups, and scaling factors for each categorical indicator were set to 1 in the first group (Lavaan did that internally). However, I was not able to estimate the configural model. 

I started with the model in which all loadings, intercepts and factors were group-specific, excluding only the first loadings for each latent variable, which were set to 1 for each group to achieve identification.

So the model syntax looked like

' Autonomy  =~ c(1, 1, 1, 1)*Independence + c(l21, l22, l23, l24)*Imagination + ...

...

Homosexuality ~ c(i11, i12, i13, i14)*1 

...

Independence |c(t11, t12, t13, t14)*t1

...

'

Again, all latent variances were fixed to 1 in all groups, and scaling factors for each categorical indicator were set to 1 in the first group by Lavaan automatically. Nevertheless, the model probably was not identified, and standard errors and scaled fit statistics were not computed

I then additionally tried to fix all scaling factors in each group to 1, as was recommended in this thread above:

'...

 Independence ~*~ c(1,1,1,1)*Independence

...

'

 But the model was not converged.

Is it possible at all to estimate the configural model for my case in lavaan? If it possible, how I can do it?

Many thanks in advance,

Boris

понедельник, 16 сентября 2013 г., 9:43:42 UTC+3 пользователь yrosseel написал:

[Kathrin Hahn]

Hi Terry,

the link you posted (http://www.myweb.ttu.edu/spornpra/catInvariance.html)  isn´t working anymore, but I think I could really use that information. Do you have it still online somewhere?

Thanks, 

Kathrin

Am Freitag, 16. Oktober 2015 10:55:00 UTC+2 schrieb Terrence Jorgensen:

... the first loadings for each latent variable were set to 1 for each group to achieve identification... all latent variances were fixed to 1 in all groups

You don't have to do both, just one or the other.

Testing invariance with categorical indicators is a bit more nuanced because of the unidentified scale of the latent continuous variable underlying each categorical indicator, each of which measures the latent common factor.  These nuances are not identical between binary and ordinal indicators.  Here is a web page that provides details and references, as well as R scripts with examples using lavaan.

http://www.myweb.ttu.edu/spornpra/catInvariance.html

Terry

[Alex Schoemann]

Hi Kathrin,

The page has moved to this location: http://sunthud.com/catInvariance.html

[cla...@googlemail.com]

Hi Terry, Hi Alexander,

Reviving the issue again, since I am also investigating MI with ordered-categorical indicators in a 5-factor model across 4 groups (running a mulit-group CFA, N=13000), and secondly testing longitudinal MI (4 occassions, N=8000) using the same instrument, 5 factors each 5 indicators, 3 point likert ordinal scale. I am a beginner in lavaan and I hope this isnt to much of a simple question. I am currently looking into fixing thresholds and the link above could be helpful to me.

I have been reading up a lot at Millsap & Yun-Tein (2004), Liu et al. (2016), Brown (2015), Kline (2023), Little, Laursen, Card (2012). I decided to follow the marker-indicator approach to scale the latent variable for model identification. Do I still need to fix thresholds and how do I do so in the case of multi-group CFA versus longitudinal CFA? Is it correct to apply the marker indicator approach to both (multi-group and longitudinal MI) or do I have to change strategy of model identification?

thanks a lot in advance

Best

Claudia

[Timothy Wong]

Hey Claudia,

I had similar issues to you some time ago (showing measurement invariance in longitudinal models with categorical indicators).

Model identification in these cases can get tricky, but luckily there is the measEq.syntax() function in semTools.

The function implements the comments of Wu and Estabrook (2016), which explains the problems with identifying measurement invariance models (with categorical indicators) using Millsap and Yun-Tein's (2004) method.

The technical details are beyond me, but my general understanding of Wu and Estabrook's (2016) claim is that the constraints used by Millsap and Yun-Tein (2004) to identify measurement invariance models are overly restrictive. In the simpler case of a longitudinal model with no groups (i.e. multiple occasions, one group), one way to identify a configural model according to MYT (2004) is to follow these steps:

1. For all groups set ν^(g) [the vector of item intercepts at all occassions] = 0 and a reference loading to 1 for each factor;
2. In the first group set κ⁽¹⁾ [the vector of common factor means at one occasion] = 0 and diag(Σ⁽¹⁾) [the variance-covariance matrix of items] = I;
3. Set invariant one threshold per item and a second threshold in the reference item of each factor.

Ideally, the "marker" loading which you choose to constrain to 1 (in step 1) should not change the parameter estimates. However, as Wu and Estabrook found, this choice actually matters and can lead to drastically different outcomes, when using the configural model in subsequent models (e.g., when progressing from configural model to loadings model, by setting loadings to be equivalent across time points). The same can be said when progressing from testing a thresholds-invariant model (which involves choosing a threshold to constrain across occassions / groups), to testing a thresholds-and-loadings-invariant model.

Details aside, measEq.syntax()implements Wu and Estabrook's solution to the above problem by default, and should be flexible enough for your use case. For instance:

• long.equal - allows you to set the level of longitudinal invariance to test for (i.e. thresholds invariance, loadings AND thresholds invariance, etc etc.)
• group.equal - allows you to set equivalence between groups
  

[cla...@googlemail.com]

Thank you, this was very helpful and indeed solved my problem while learning some more insights into sem and mi.

Best,

Claudia

[Frog]

Does anyone have an updated link to the link Terrence provided us with? none of the links listed in this thread are working. 

F. G.

[Terrence Jorgensen]

Does anyone have an updated link to the link Terrence provided us with? none of the links listed in this thread are working. 

You don't want those examples anyway, they are invalid (as this 2016 paper showed).  I'd recommend reading that (it is not easy, but skim the heavy math and pay attention to their conclusions) and looking at the semTools::measEq.syntax() examples to see how it's done.

Terrence D. Jorgensen    (he, him, his)
Assistant Professor, Methods and Statistics
Research Institute for Child Development and Education, the University of Amsterdam
http://www.uva.nl/profile/t.d.jorgensen