Basic questions regarding measurement invariance and regression analyses

[Siu Kit Yeung]

Dear all,

I am very new to MI so these questions may seem basic. 

I conducted cross-regional (United Kingdom versus Hong Kong China) measurement invariance tests w Lavaan. I have the following questions:

1) For the DV (which only consists of 3 items, about behavioral intentions), w MGCFA, I found evidence for metric invariance but not scalar invariance. Is it worthwhile to run measurement invariance for a scale with only 3 items? Any papers regarding this issue?

2) Given that we found no support for scale invariance, is it okay to run multiple regressions that includes message framing as IV (gain versus nonloss framing), region*message framing as an interaction term (region as a potential moderator), plus another covariate, with behavioral intentions as the DV (in which there is support for metric invariance but not scalar invariance) I know the means in behavioral intentions between regions cannot be compared well, so I won't interpret the region term of the regression. But is it methodologically sound to test the interaction (region*message framing)? 

3) I also included individual difference moderators (e.g. regulatory focus). Again, I found support for metric invariance but not scalar invariance with a regulatory focus scale (Vriend et al., 2022). Let's say I run another regression with message framing as IV, region as another term (I won't interpret the group differences), a covariate, regulatory focus, and interaction term of message framing*regulatory focus, with the same DV (behavioral intentions). Is it okay to combine both the UK and HK samples in the regression analyses while focusing on testing the individual difference moderation (message framing*regulatory focus)? Or should I separate HK and UK samples in running such analyses?

Will appreciate if there are academic papers/references so that I can read more, thank you!

Regards,

Kit

[Terrence Jorgensen]

I found evidence for metric invariance but not scalar invariance

Then you can compare latent covariance-structure parameters (e.g., (co)variances, regression slopes) across groups.

If you can establish partial scalar invariance, latent means/intercepts can be comparable across groups.  But with only 3 indicators, it is difficult to be confident about any particular 2-indicator solution being "true".

https://doi.org/10.1007/s11336-014-9408-y

Is it worthwhile to run measurement invariance for a scale with only 3 items?

Yes.

 

Any papers regarding this issue?

There are numerous articles you can search on Google Scholar about measurement invariance.

 

2) Given that we found no support for scale invariance, is it okay to run multiple regressions

Yes. Regression slopes are comparable without equating intercepts.  But since you have an interaction term:

• simple effects (i.e., effect of a focal predictor, when the moderator = 0) are only comparable if the moderator's location (mean) is linked across groups
• the interaction effect is only comparable across groups if both region and message have a linked location across groups

If region and message are both observed variables, then this should be fine, as long as you don't center them at different values per group (e.g., group-mean centering).  If you recenter them at all, just use the same value (e.g., the grand mean, or the minimum value across groups).

If region or message are common factors, then the latent location needs to be linked by establishing (at least partial) scalar invariance.  And there would be other complications as well.

3) ... Is it okay to combine both the UK and HK samples in the regression analyses while focusing on testing the individual difference moderation (message framing*regulatory focus)? Or should I separate HK and UK samples in running such analyses?

I think it would be more straight-forward to model all these interactions in a normal regression analysis.  But analyzing composites instead of latent variables implies strict invariance.  You already don't have scalar invariance, so I'm not sure what straight-forward advice I can give you for such a complicated model.

Good luck,

Terrence D. Jorgensen    (he, him, his)
Assistant Professor, Methods and Statistics
Research Institute for Child Development and Education, the University of Amsterdam
http://www.uva.nl/profile/t.d.jorgensen