Measurement invariance with ordinal indicators

[Rebecca Taylor]

Hi,

I have followed a few threads on here that have been really helpful:

https://groups.google.com/g/lavaan/c/YG_8uJffKQQ/m/HVlYtVsMBQAJ

https://groups.google.com/g/lavaan/c/7sglArLoIno/m/3tyM3Ju6CwAJ

and have made much use of your slides here:

https://www.uva.nl/binaries/content/documents/personalpages/j/o/t.d.jorgensen/en/tab-three/tab-three/cpitem%5B2%5D/asset?1556027804259,

thank you, but just need some further clarification on lavaan default constraints for assessing measurement invariance with ordinal indicators. In particular, I have one ordinal indicator that is the sole indicator of a latent, and another which sits alongside two continuous indicators; I believe I need to deal with these differently and that is where I come unstuck.

My model is as follows:

mod <- " 

f1 =~ 1*ordx1

f2 =~ 1* contx2

f3 =~ 1*contx3

f4 =~ 1*contx4

f5 =~ contx5a + contx5b + ordx5c

f6 =~ 1* contx6

#  error variances of continuous single indicators predetermined using reliability

contx2 ~~ ev2*contx2 

contx3 ~~ ev2*contx3 

contx4 ~~ ev2*contx4 

contx6 ~~ ev2*contx6

# choosing to allow identification of  single ordinal ordx1 by fixing indicator variance at 0 and latent (residual) variance as 1:

ordx1 ~~ 0*ordx1

f1 ~~ 1*f1   "

I fit this using 

fit <- cfa(mod, data, estimator="WLSMV", std.lv=T)

Question 1: the single ordinal, ordx1, has 4 categories (3 thresholds). The intercept of this is fixed at 0  - do I need to fix the intercept at all? What default call to cfa does/undoes this? I had assumed this was because a numerical intercept of a categorical variable made no theoretical sense, until I begin testing for metric invariance and found that I needed to free the intercepts when constraining thresholds to equivalence across groups. 

And so to invariance: I check in the following order as advised in the threads above and by Wu & Esterbrook:

  configural   ->   thresholds   ->   thresholds and loadings (metric)   ->   thresholds, loadings and intercepts (scalar)

and constrain thresholds and free intercepts by adding the following:

          " ordx1 | c(oa1, oa1)*t1 + c(ob1, ob1)*t2 + c(oc1, oc1)*t3

           ordx5c | c(qa1, qa1)*t1 + c(qb1, qb1)*t2

           ordx1 ~ c(NA1,NA2)*1   

           ordx5c | c(NA3,NA4)*1    "

Question 2: should I free the intercepts of the two ordinal variables as above in the model syntax or is there an argument to the cfa function? The other manifest variable intercepts are already free and I want the latent means to remain constrained to 0, I think. 

Having done this, constraining loadings (only the three on f5) to fully test metric invariance is fine. Moving on to scalar invariance, is it sufficient now to use "group.equal="intercepts" in the cfa call? From your slides:

" Free identification constraints on common factor means, except for 1 reference category; set v=0 in reference category for identification, so impose equivalence by fixing v=0 in all contexts"

I'm not sure what this means, but by simply using group.equal="intercepts", I estimate a negative variance in ordx1 in one group (0 in the other) - I'm pretty sure this is user error! 

Question 3: in order to add constraints on intercepts, should I have released some others? Should I treat the single indicator (ordx1) differently to that with other continuous indicators (ord5c)?

I'd be really grateful for any further insight you can offer, many thanks.

Rebecca

[Terrence Jorgensen]

# choosing to allow identification of  single ordinal ordx1 by fixing indicator variance at 0 and latent (residual) variance as 1:

ordx1 ~~ 0*ordx1

f1 ~~ 1*f1   "

I fit this using 

fit <- cfa(mod, data, estimator="WLSMV", std.lv=T)

Setting std.lv=TRUE will free your factor loadings.  For the single ordinal indicator, you need to fix that loading to 1.

Question 1: the single ordinal, ordx1, has 4 categories (3 thresholds). The intercept of this is fixed at 0  - do I need to fix the intercept at all?

It is already fixed to 0 by default.

 

What default call to cfa does/undoes this?

If you want to free it, you must write the syntax ('variable ~ 1'), but that requires fixing at least 1 threshold (e.g., to zero) in place of fixing the intercept.

I had assumed this was because a numerical intercept of a categorical variable made no theoretical sense, until I begin testing for metric invariance and found that I needed to free the intercepts when constraining thresholds to equivalence across groups. 

It is not the mean/intercept of the ordinal variable, but of its latent response, assumed to be normally distributed. 

And so to invariance

Invariance across what?  There is no group= argument in your cfa() call, but the syntax below seems to indicate you have 2 groups.

  

I constrain thresholds and free intercepts by adding the following:

          " ordx1 | c(oa1, oa1)*t1 + c(ob1, ob1)*t2 + c(oc1, oc1)*t3

           ordx5c | c(qa1, qa1)*t1 + c(qb1, qb1)*t2

           ordx1 ~ c(NA1,NA2)*1   

           ordx5c | c(NA3,NA4)*1    "

You cannot free both intercepts by equating estimated thresholds.  The location of the latent response is still not identified in Group 1, unless you fix a threshold to a particular value in Group 1.  Or you could simply leave the intercepts fixed to 0 in Group 1, but free them in Group 2.  The semTools::measEq.syntax() function does the latter.

Question 2: should I free the intercepts of the two ordinal variables as above in the model syntax or is there an argument to the cfa function?

Use syntax.

The other manifest variable intercepts are already free and I want the latent means to remain constrained to 0, I think. 

It is arbitrary, especially for single-indicator factors.  When you constrain a loading across groups, you free that group's latent variance in its place.  Same with trading an intercept for a latent mean.  You cannot test invariance of single-indicator constructs.

Moving on to scalar invariance, is it sufficient now to use "group.equal="intercepts" in the cfa call?

Use syntax to fit the model you want to fit.  For categorical indicators, don't try to equate a freely estimated value to a fixed one; just fix the parameter to the same value in both groups. 

From your slides:

" Free identification constraints on common factor means, except for 1 reference category; set v=0 in reference category for identification, so impose equivalence by fixing v=0 in all contexts"

I'm not sure what this means, but by simply using group.equal="intercepts"

You would never only constrain intercepts.  Only constrain intercepts for items whose loadings are constrained equal.

I estimate a negative variance in ordx1 in one group (0 in the other) - I'm pretty sure this is user error! 

This is a single-indicator factor; check the identification issue I mentioned above (fix its loading to 1).  But again, you can't test invariance of anything for this single indicator, except its thresholds.  The loadings are equated anyway, by virtue of fixing them to 1 in both groups for identification.  You can constrain its thresholds across groups and free Group 2's factor variance to allow for that difference (if the H0 of equal thresholds is not rejected).  Since it is a single indicator factor (whose residual variance == 0), the factor actually represents the latent response variable.

Question 3: in order to add constraints on intercepts, should I have released some others? Should I treat the single indicator (ordx1) differently to that with other continuous indicators (ord5c)?

Think of the latent response variables (LRVs) as latent factors (first-order factors, which are indicators of higher-order common factors), and their measurement parameters are the thresholds that link them to the observed ordinal responses.  

https://doi.org/10.1080/10705510701758406

Start by fixing their locations (intercepts) and scales (residual variances) to 0 and 1, respectively, in the configural model.  

• Equating thresholds across groups allows you to free intercepts and residual variances of LRVs in Group 2.  
• From there, invariance of loadings and intercepts proceeds as normal (because LRVs are continuous indicators). 

But you should only be concerned with invariance of loadings and intercepts for the multiple-indicator factors.  Invariance can only be assumed (not tested) for single-indicators factors because when you equate a loading (or intercept), the same difference is reflected instead by freeing the factor variance (or mean) in Group 2.

Terrence D. Jorgensen

Assistant Professor, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

http://www.uva.nl/profile/t.d.jorgensen

[Rebecca Taylor]

Terrence,

Thanks so much for your reply, it really helps with my understanding of what is going on. To answer your points:

Setting std.lv=TRUE will free your factor loadings.  For the single ordinal indicator, you need to fix that loading to 1.

Yes indeed, I did this above using f1 =~ 1*ordx1, and f1 =~ c(1,1)*ordx1 in the multigroup models.

It is not the mean/intercept of the ordinal variable, but of its latent response, assumed to be normally distributed. 

The link you posted clarifies this thanks - I had thought that the latent response was the f1 above, but it is actually the underlying y_i*, that acts like a middle stage between ordx1 (y_i) and f1? 

And so to invariance

Invariance across what?  There is no group= argument in your cfa() call, but the syntax below seems to indicate you have 2 groups.

Yes, I did not post the cfa call for the invariance models but do include group="sex" (among other groups to be tested) for this example.

You cannot free both intercepts by equating estimated thresholds.  The location of the latent response is still not identified in Group 1, unless you fix a threshold to a particular value in Group 1.  Or you could simply leave the intercepts fixed to 0 in Group 1, but free them in Group 2.

OK, thanks, I had not appreciated this. I have amended this part in the code for the threshold model using:

         " ordx1 | c(oa1, oa1)*t1 + c(ob1, ob1)*t2 + c(oc1, oc1)*t3

           ordx5c | c(qa1, qa1)*t1 + c(qb1, qb1)*t2                                       # equating thresholds

           ordx1 ~ c(0,NA1)*1   

           ordx5c | c(0,NA2)*1       "                                                                   # fixing intercepts to 0 in gp1 and free in gp2

                                                 

Is this OK? Is it correct to free either an intercept or a residual variance? I understand that this only applies to the threshold model, and I would not test the single indicator ordx1 (or any of the continuous single indicator factors) any further than this. The model appears OK (chisquare is good, residuals are small, variances are positive, regression coefficients are sensible) but do get a "vcov is not positive-definite...smallest eigenvalue is close to 0" warning - could this be related to misspecification? On this basis I accept that the thresholds are invariant to sex.

As you say, I can only test loading invariance on the factor with multiple indicators as all single indicator loadings are constrained to 1 out of necessity. This is easy to do and again the model allows me to accept full metric invariance (thresholds and loadings invariant to sex, where they are allowed to vary at all).

You would never only constrain intercepts.  Only constrain intercepts for items whose loadings are constrained equal.

The group.equal="intercepts" call was in addition to the constraints on thresholds and loadings already specified in the syntax, so the question related to whether I needed to constrain all intercepts in the model or only some - thanks for your clarification that again I would only constrain intercepts on the one multiple-indicator factor. I can do this is the syntax with:

         "contx5a ~ c(i1,i1)*1

          contx5b ~ c(i2,i2)*1

          ordx5c ~ c(i3,i3)*1  "

is this that same as using the group.equal call, given that all other intercepts are set by default of being single indicators? (or being specified as ordx1 ~ c(0,NA1)*1, in the case of the single ordinal indicator). Also, as I now constrain the ord5xc variance to equality, do I need to free the factor variance by removing std.lv=T?

Your help is invaluable, thanks so much

Rebecca

[Terrence Jorgensen]

Is it correct to free either an intercept or a residual variance?

Technically, you could free either, as long as the intercept and a single threshold are different values.  But if there are multiple thresholds, I don't think that detail matters.

 

I understand that this only applies to the threshold model, and I would not test the single indicator ordx1 (or any of the continuous single indicator factors) any further than this

Depending one what your structural model is and what hypotheses you want to test, you might still want to impose constraints on the measurement parameters to free the latent ones, even though the specifications are statistically equivalent.  For example, if you are regressing f1 on something (or something on f1), then its scale will determine the value of the slope.  If f1's (residual) variable is arbitrarily equal across groups by being fixed to 1, then the slopes aren't really comparable across groups (so constraining them to equality would not be a valid test of their equivalence).  So although you can't test invariance for single-indicator factors, you might still want to impose the constraints that represent those assumptions.

 

a "vcov is not positive-definite...smallest eigenvalue is close to 0" warning - could this be related to misspecification?

I do not think that warning is a function of misspecification.  It is more likely to be related to identification.  But if you have no other signals of lack of identification, then you can probably ignore the warning.

 

         "contx5a ~ c(i1,i1)*1

          contx5b ~ c(i2,i2)*1

          ordx5c ~ c(i3,i3)*1  "

is this that same as using the group.equal call, given that all other intercepts are set by default of being single indicators? (or being specified as ordx1 ~ c(0,NA1)*1, in the case of the single ordinal indicator).

I would use syntax to set Group 2's intercepts to 0, rather than constraining a free estimate to equal a fixed parameter's value (not sure that works).

 

Also, as I now constrain the ord5xc variance to equality, do I need to free the factor variance by removing std.lv=T?

Use syntax to set Group 2's variance free (NA) while keeping Group 1's fixed for identification.  The group.equal= shortcut is really only useful in the simplest of situations.  Model syntax is the way to specify the model you want to fit.