measurement invariance for ordinal variables

[Mirtes Garcia Pereira]

Hello, I am testing for a measurement invariance using multigroup CFA and lavaan. I copied below the script I used to run configural, metric and scalar invariance using WLSMV estimator.  Each variable data was obtained using a likert scale (range 0 to 4).

PCL.model1001 <- ' factorb =~ PCL1 + PCL2 + PCL3 + PCL4 + PCL5
factorc =~ PCL6 + PCL7
factord =~ PCL8 + PCL9 + PCL10 + PCL11 + PCL12 + PCL13 + PCL14
factore =~ PCL15 + PCL16 + PCL17 + PCL18 + PCL19 + PCL20 '

# model 1: configural invariance age_group
fit1 <- cfa(PCL.model1001, data = pcl_ti_data_amostra1001,  estimator = "WLSMV", ordered = TRUE, group = "age_group")
# model 2: metric (weak) invariance age_group
fit2 <- cfa(PCL.model1001, data=pcl_ti_data_amostra1001, estimator="WLSMV", ordered = TRUE, group="age_group",group.equal="loadings")
# model 3: scalar (strong) invariance age_group
fit3 <- cfa(PCL.model1001, data=pcl_ti_data_amostra1001, estimator="WLSMV", ordered = TRUE, group="age_group", group.equal=c("loadings", "thresholds"))

My question is, if I change the estimator to MLR and ordered=FALSE, to test scalar invariance  should I change  group.equal=c("loadings", "thresholds"))  to group.equal=c("loadings", "intercepts")) as data  will be considered as continuous?

Thanks in advance for any help.

Mirtes Garcia Pereira
Coordenadora do Programa de Pós-graduação
em Ciências Biomédicas (Fisiologia e Farmacologia)
Laboratório de Neurofisiologia do Comportamento
Departamento de Fisiologia e Farmacologia
Universidade Federal Fluminense

Rio de Janeiro - Brasil

[Terrence Jorgensen]

configural, metric and scalar invariance using WLSMV estimator

If you rely on the latent-response assumption of the threshold model, loadings are not comparable (not on the same metric) until AFTER you constrain thresholds to equality.  See references on the ?semTools::measEq.syntax help page, and see Examples there.

if I change the estimator to MLR and ordered=FALSE, to test scalar invariance  should I change  group.equal=c("loadings", "thresholds"))  to group.equal=c("loadings", "intercepts")) as data  will be considered as continuous?

Yes, but even with WLSMV (threshold model for categorical indicators), you can test equivalence of loadings then intercepts, AFTER first testing equivalence of thresholds.  I am aware there is plenty of faulty literature out there that perpetuates the misconception that thresholds are the ordinal equivalent of intercepts, but that is misinformation. 

Terrence D. Jorgensen

Assistant Professor, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

http://www.uva.nl/profile/t.d.jorgensen

[Mirtes Garcia Pereira]

Thanks so much for your feedback. So your advice is that if I keep with the wlsmv estimator I should use the Wu and Estabrook procedure and use   measEq.syntax ? 

Otherwise, if I decide to use the MLR estimator, can I follow the script below using lavaan?

# model 1: configural invariance age_group

fit1 <- cfa(PCL.model1001, data = pcl_ti_data_amostra1001,  estimator = "MLR",  group = "age_group")

# model 2: metric (weak) invariance age_group

fit2 <- cfa(PCL.model1001, data=pcl_ti_data_amostra1001, estimator="MLR",  group="age_group",group.equal="loadings")

# model 3: scalar (strong) invariance age_group

fit3 <- cfa(PCL.model1001, data=pcl_ti_data_amostra1001, estimator="MLR",  group="age_group", group.equal=c("loadings", "thresholds"))

Mirtes Garcia Pereira
Coordenadora do Programa de Pós-graduação
em Ciências Biomédicas (Fisiologia e Farmacologia)
Laboratório de Neurofisiologia do Comportamento
Departamento de Fisiologia e Farmacologia
Universidade Federal Fluminense

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/85d87c1f-7abc-452a-801e-8a6bc85db43cn%40googlegroups.com.

[Terrence Jorgensen]

your advice is that if I keep with the wlsmv estimator I should use the Wu and Estabrook procedure and use   measEq.syntax ? 

Otherwise, if I decide to use the MLR estimator, can I follow the script below using lavaan?

Yes, except:

# model 3: scalar (strong) invariance age_group

fit3 <- cfa(PCL.model1001, data=pcl_ti_data_amostra1001, estimator="MLR",  group="age_group", group.equal=c("loadings", "thresholds"))

If you use MLR (treating the data as continuous), there are no thresholds, so constrain intercepts after loadings to test scalar invariance.

[Mirtes Garcia Pereira]

If you use MLR (treating the data as continuous), there are no thresholds, so constrain intercepts after loadings to test scalar invariance.

 For sure,  I copied from the previous script using wlsmv and forgot to change the thresholds by intercepts. 

Thanks a lot.

Mirtes Garcia Pereira
Coordenadora do Programa de Pós-graduação
em Ciências Biomédicas (Fisiologia e Farmacologia)
Laboratório de Neurofisiologia do Comportamento
Departamento de Fisiologia e Farmacologia
Universidade Federal Fluminense

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/ebe9ea7c-5bba-4df0-b2a3-cb582f684e13n%40googlegroups.com.

[Mirtes Garcia Pereira]

Hello Terrence, I ran the scalar invariance testing in another model and scalar invariance was not supported . To identify constraints that might be released   I ran lavTestScore including the argument cumulative=TRUE to identify the highest statistics value. After that, I used partable to find parameter labels but I am not certain If I am selecting the correct constraint to release based on the results. I am copying an example with a selection of parts of the table.

#specify the model
TIS.model1001 <- ' factora =~ TI2 + TI3 + TI4 + TI5 + TI6 + TI8 '
# model 1: configural invariance TRAUMA_INDEX
fit1_trauma <- cfa(TIS.model1001, data = pcl_ti_data_amostra1001,  estimator = "MLR", group = "TRAUMA_INDEX")
# model 2: metric (weak) invariance TRAUMA_INDEX
fit2_trauma <- cfa(TIS.model1001, data=pcl_ti_data_amostra1001, estimator="MLR",  group="TRAUMA_INDEX",group.equal="loadings")
# model 3: scalar (strong) invariance TRAUMA_INDEX
fit3_trauma <- cfa(TIS.model1001, data=pcl_ti_data_amostra1001, estimator="MLR",  group="TRAUMA_INDEX", group.equal=c("loadings", "intercepts"))

lavTestScore(fit3_trauma, cumulative = TRUE)

univariate score tests:

     lhs op   rhs     X2 df p.value
1   .p2. == .p22.  0.066  1   0.797
2   .p3. == .p23.  0.678  1   0.410
3   .p4. == .p24.  0.970  1   0.325
4   .p5. == .p25.  0.332  1   0.564
5   .p6. == .p26.  2.042  1   0.153
6  .p14. == .p34.  8.657  1   0.003
7  .p15. == .p35.  3.589  1   0.058
8  .p16. == .p36.  9.023  1   0.003
9  .p17. == .p37.  0.488  1   0.485
10 .p18. == .p38. 18.499  1   0.000

...

cumulative score tests:
     lhs op   rhs     X2 df p.value
10 .p18. == .p38. 18.499  1       0
22 .p19. == .p59. 30.278  2       0
20 .p17. == .p57. 42.846  3       0

> partable(fit3_trauma)
     id     lhs op     rhs user block group free ustart exo label plabel start   est    se

36   36     TI4 ~1            0     2     2   33     NA   0 .p16.  .p36. 2.577 2.064 0.128
37   37     TI5 ~1            0     2     2   34     NA   0 .p17.  .p37. 2.046 1.811 0.110
38   38     TI6 ~1            0     2     2   35     NA   0 .p18.  .p38. 3.163 3.438 0.115
39   39     TI8 ~1            0     2     2   36     NA   0 .p19.  .p39. 2.092 1.773 0.125

So , I tested for partial invariance after releasing  "TI6 ~ 1":

#partial scalar invariance model
fit4_trauma <- cfa(TIS.model1001, data=pcl_ti_data_amostra1001, estimator="MLR",  group="TRAUMA_INDEX", group.equal=c("loadings", "intercepts"), group.partial = c("TI6 ~ 1" )).

 

As scalar and metric invariance models were still different , I released the second constraint  in the cumulative score tests list:

cumulative score tests:
     lhs op   rhs     X2 df p.value
10 .p18. == .p38. 18.499  1       0
22 .p19. == .p59. 30.278  2       0

> partable(fit3_trauma)
     id     lhs op     rhs user block group free ustart exo label plabel start   est    se

...

50   50     TI5 ~~     TI5    0     3     3   46     NA   0        .p50. 1.947 2.244 0.239
51   51     TI6 ~~     TI6    0     3     3   47     NA   0        .p51. 2.297 2.541 0.225
52   52     TI8 ~~     TI8    0     3     3   48     NA   0        .p52. 1.927 1.909 0.244
53   53 factora ~~ factora    0     3     3   49     NA   0        .p53. 0.050 3.076 0.269
54   54     TI2 ~1            0     3     3   50     NA   0 .p14.  .p54. 2.298 1.954 0.135
55   55     TI3 ~1            0     3     3   51     NA   0 .p15.  .p55. 2.580 2.252 0.149
56   56     TI4 ~1            0     3     3   52     NA   0 .p16.  .p56. 2.302 2.064 0.128
57   57     TI5 ~1            0     3     3   53     NA   0 .p17.  .p57. 1.706 1.811 0.110
58   58     TI6 ~1            0     3     3   54     NA   0 .p18.  .p58. 3.478 3.438 0.115
59   59     TI8 ~1            0     3     3   55     NA   0 .p19.  .p59. 1.686 1.773 0.125
60   60 factora ~1            0     3     3   56     NA   0        .p60. 0.000 0.171 0.178
61   61 factora =~     TI2    1     4     4    0      1   0        .p61. 1.000 1.000 0.000

#partial scalar invariance model

fit4_trauma <- cfa(TIS.model1001, data=pcl_ti_data_amostra1001, estimator="MLR",  group="TRAUMA_INDEX", group.equal=c("loadings", "intercepts"), group.partial = c("TI6 ~ 1", "TI8 ~ 1"  )).

My questions are:

1- Is this an acceptable way of identifying which constraints should be released?

2- The constraints to be released were identified correctly  after checking for their labels in partable?

3- Is the partial scalar variance model correct?

Thanks in advance

 

Mirtes Garcia Pereira
Coordenadora do Programa de Pós-graduação
em Ciências Biomédicas (Fisiologia e Farmacologia)
Laboratório de Neurofisiologia do Comportamento
Departamento de Fisiologia e Farmacologia
Universidade Federal Fluminense

Em sáb., 2 de abr. de 2022 às 03:45, Terrence Jorgensen <tjorge...@gmail.com> escreveu:

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/ebe9ea7c-5bba-4df0-b2a3-cb582f684e13n%40googlegroups.com.

[Terrence Jorgensen]

1- Is this an acceptable way of identifying which constraints should be released?

2- The constraints to be released were identified correctly  after checking for their labels in partable?

Seems right to me.  But if scalar invariance still fails, then you have to recursively apply this method.  That's where the trouble starts because score tests are biased when the alternative model is still not correct (i.e., when there is more than 1 item with DIF, the tests for DIF-free items will have inflated Type I error rates).  

Something else you could do is check how much the latent-mean estimate changes in group 2 between full- and partial-invariance models.  That quantifies the impact of the DIF on your estimate of interest (see Oberski et al., 2014 for discussion).

3- Is the partial scalar variance model correct?

No one can know, but your syntax follows logically from the output, as far as I can tell.

[cla...@googlemail.com]

Hi Terrence, 

Thanks for the insights. I have a follow-up question. 

I am also working with ordered-categorical data in MI testing (three response categories). I came across a lot of papers that tested scalar invariance as invariance of thresholds - surprisingly. In Millsap 2011, he describes these steps however not elaborating why constraining thresholds instead of intercepts. I haven't found a mathematical explanation (also I am "just" a applied psych researcher). Could you point me towards work that would support my strategy to test the following MI sequence: factor patterns (configural), loadings (metric), thresholds (threshold), intercepts (scalar) and unique variances (residual MI)?  Or rather, how come that you also disagree with the notion that thresholds are the intercept-equivalent in ordered data?

This is especially important to my case, since I only have three response categories and therefore the configural and threshold model have the same degrees of freedom - ergo they have the same fit in this case.

Thank you in ad advance

Claudia

[Terrence Jorgensen]

Could you point me towards work that would support my strategy to test the following MI sequence: factor patterns (configural), loadings (metric), thresholds (threshold), intercepts (scalar) and unique variances (residual MI)? 

No, thresholds must be equated before any parameters that depend on the latent-response scale.

 

Or rather, how come that you also disagree with the notion that thresholds are the intercept-equivalent in ordered data?

https://doi.org/10.1007/s11336-016-9506-0 

This is especially important to my case, since I only have three response categories and therefore the configural and threshold model have the same degrees of freedom - ergo they have the same fit in this case.

Correct, with 3 categories, you only have 2 thresholds per item, which you can trade for 2 latent-response parameters (intercept and residual/marginal variance).  So you can't test threshold invariance, only assume it.  Therefore, the first test you conduct is to compare any configural model to one that constraints both loadings and thresholds (without constraining intercepts or variances).