Constructing a partial invariance model

[Andrej]

Hi all,

I'm new to CFA and lavan in particular. I try to conduct multi-group CFA on my collected data to test for measurement invariance with respect to two age groups. Data are composed of eight five-point Likert items (ss1, ss2, ..., ss8). The total score is denoted as sss.

Using various tutorials I built the following CFA model (I extended the baseline model with error covariance between items 2 and 4):

model_2 <- 'sss =~ ss1 + ss2 + ss3 + ss4 + ss5 + ss6 + ss7 + ss8
            ss2 ~~ ss4'

Then I started with measurement invariance modeling.

configural <- cfa(model_2, data = bla1, estimator = "DWLS", group = "age",
                  ordered = c("ss1", "ss2", "ss3", "ss4", "ss5", "ss6", "ss7", "ss8"),
                  parameterization = "theta")
weak <- cfa(model_2, data = bla1, estimator = "DWLS", group = "age", group.equal = "loadings",
            ordered = c("ss1", "ss2", "ss3", "ss4", "ss5", "ss6", "ss7", "ss8"),

            parameterization = "theta")

anova(configural, weak)
> anova(configural, weak)
Chi Square Difference Test

           Df AIC BIC  Chisq Chisq diff Df diff Pr(>Chisq)
configural 38         52.574                             
weak       45         62.189     9.6155       7     0.2114

The chi-square difference was not significant, so I confirmed weak invariance. However, when I tested for strong invariance:

strong <- cfa(model_2, data = bla1, estimator = "DWLS", group = "age", group.equal = c("loadings", "intercepts"),
              ordered = c("ss1", "ss2", "ss3", "ss4", "ss5", "ss6", "ss7", "ss8"), parameterization = "theta")
anova(strong, weak)
Chi Square Difference Test

       Df AIC BIC   Chisq Chisq diff Df diff Pr(>Chisq)   
weak   45          62.189                                 
strong 68         156.586     94.397      23  1.285e-10 ***

the chi-square difference was significant and strong invariance can not be confirmed. Next, I tried to establish partial strong
invariance. Using the commands:

lavTestScore(strong)
parTable(strong)

the following modifications were suggested:

ss2 | t2
ss2 | t1
sss =~ ss5
ss2 | t3
sss =~ ss2
ss5 | t2
ss5 | t1
sss =~ ss4

My questions:
- What does constraint "ss2 | t2" mean?

- What does constraint "sss =~ ss5" mean?

- Is it correct to specify partial strong model as:

strong_partial <- cfa(HS.model_2, data = bla1, estimator = "DWLS", group = "age", group.equal = c("loadings", "intercepts"),

                      ordered = c("ss1", "ss2", "ss3", "ss4", "ss5", "ss6", "ss7", "ss8"),

                      parameterization = "theta",

                      group.partial = c("ss2 | t2", "ss2 | t1", "sss =~ ss5", "ss2 | t3", "sss =~ ss2", "ss5 | t2", "ss5 | t1", "sss =~ ss4"))

Thanks in advance any answer.

Best, Andrej

[Terrence Jorgensen]

The total score is denoted as sss.

That is the name of your common factor, not a "total score", which would be a linear composite of your indicators (i.e., formative measurement).  In CFA, it is the indicators that are linear composites of the latent common and unique factors. 

strong <- cfa(model_2, data = bla1, estimator = "DWLS", group = "age", group.equal = c("loadings", "intercepts")

You have categorical indicators, so the intercepts are not estimated (unless you free them in the syntax).   Instead, you should constrain thresholds.  If you look at your summary() output, you will see that lavaan noticed this mistake and silently corrects it by constraining the thresholds instead of intercepts of categorical indicators.

Also, your anova() output does not indicate that a robust test statistic was provided, only the naïve difference between the 2 model's chi-squared statistics.  Does it not provide a "Scaled Chi Square Difference Test (method = "satorra.2000")" when you compare models fitted with DWLS to ordered indicators?

the following modifications were suggested:

Using what criterion?  Were you looking at the $uni output, using a particular alpha level?

ss2 | t2
ss2 | t1
sss =~ ss5
ss2 | t3
sss =~ ss2
ss5 | t2
ss5 | t1
sss =~ ss4

These are parameters, not constraints.  The lavTestScore() function provides tests of constraints.  Are you saying the univariate tests of the equality constraints on these parameters were all significant?  I would be wary of these test statistics, because they are not robust (as the warning printed by lavTestScore() tells you).

- What does constraint "ss2 | t2" mean?

It is the second threshold of the variable "ss2".

 

- What does constraint "sss =~ ss5" mean?

It is a factor loading.  

- Is it correct to specify partial strong model as:

strong_partial <- cfa(HS.model_2, data = bla1, estimator = "DWLS", group = "age", group.equal = c("loadings", "intercepts"),

                      ordered = c("ss1", "ss2", "ss3", "ss4", "ss5", "ss6", "ss7", "ss8"),

                      parameterization = "theta",

                      group.partial = c("ss2 | t2", "ss2 | t1", "sss =~ ss5", "ss2 | t3", "sss =~ ss2", "ss5 | t2", "ss5 | t1", "sss =~ ss4"))

That is how the group.partial argument works, but your anova() output (assuming it is valid) indicated the null hypothesis of weak invariance can't be rejected, so I don't see why you would consider freeing constraints on loadings. Again, the tests provided by lavTestScore() are not robust, so they do not conform to what your would expect to find from anova() if you compared models with and without those constraints.  And if you were looking at univariate tests, those are only tests of whether each individual constraint should be released, assuming all other constraints remained.  For a simultaneous test of releasing multiple constraints, try adding the argument cumulative = TRUE.  But again, the tests are not robust, so you should probably manually fit models with one constraint released at a time, until the partial strong model fits as well as the weak model.  I would also recommend treating thresholds as a set -- since multiple thresholds of item ss2 seem to (possibly) differ, you could try releasing all of that item's thresholds to see if that alone makes your model fit as well as the weak model.

Terrence D. Jorgensen

Postdoctoral Researcher, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

UvA web page: http://www.uva.nl/profile/t.d.jorgensen

[Andrej Kastrin]

Terrence,

thanks for your answer. It helps me a lot.

Best, Andrej

On 17. 07. 2018 12:39, Terrence Jorgensen wrote:
> The total score isdenoted as sss.

>
>
> That is the name of your common factor, not a "total score", which would
> be a linear composite of your indicators (i.e., formative measurement).
> In CFA, it is the indicators that are linear composites of the latent
> common and unique factors.
>
> strong <- cfa(model_2, data = bla1, estimator = "DWLS", group =
> "age", group.equal = c("loadings", "intercepts")
>
>
> You have categorical indicators, so the intercepts are not estimated
> (unless you free them in the syntax).   Instead, you should constrain

> thresholds.  If you look at yoursummary() output, you will see that

> lavaan noticed this mistake and silently corrects it by constraining the
> thresholds instead of intercepts of categorical indicators.
>

> Also, youranova() output does not indicate that a robust test statistic

> was provided, only the naïve difference between the 2 model's
> chi-squared statistics.  Does it not provide a "Scaled Chi Square
> Difference Test (method = "satorra.2000")" when you compare models
> fitted with DWLS to ordered indicators?
>
> the following modifications were suggested:
>
>

> Using what criterion?  Were you looking at the$uni output, using a

> particular alpha level?
>
> ss2 | t2
> ss2 | t1
> sss =~ ss5
> ss2 | t3
> sss =~ ss2
> ss5 | t2
> ss5 | t1
> sss =~ ss4
>
>

> These are parameters, not constraints.  ThelavTestScore() function

> provides tests of constraints.  Are you saying the univariate tests of
> the equality constraints on these parameters were all significant?  I
> would be wary of these test statistics, because they are not robust (as

> the warning printed bylavTestScore() tells you).

>
> - What does constraint "ss2 | t2" mean?
>
>
> It is the second threshold of the variable "ss2".
>
> - What does constraint "sss =~ ss5" mean?
>
>
> It is a factor loading.
>
> - Is it correct to specify partial strong model as:
>
> strong_partial <- cfa(HS.model_2, data = bla1, estimator = "DWLS",
> group = "age", group.equal = c("loadings", "intercepts"),
>                       ordered = c("ss1", "ss2", "ss3", "ss4",
> "ss5", "ss6", "ss7", "ss8"),
>                       parameterization = "theta",
>                       group.partial = c("ss2 | t2", "ss2 | t1",
> "sss =~ ss5", "ss2 | t3", "sss =~ ss2", "ss5 | t2", "ss5 | t1", "sss
> =~ ss4"))
>
>

> That is how the group.partial argument works, but youranova() output

> (assuming it is valid) indicated the null hypothesis of weak invariance
> can't be rejected, so I don't see why you would consider freeing
> constraints on loadings. Again, the tests provided by lavTestScore() are
> not robust, so they do not conform to what your would expect to find

> fromanova() if you compared models with and without those constraints.

> And if you were looking at univariate tests, those are only tests of
> whether each individual constraint should be released, assuming all
> other constraints remained.  For a simultaneous test of releasing
> multiple constraints, try adding the argument cumulative = TRUE.  But
> again, the tests are not robust, so you should probably manually fit
> models with one constraint released at a time, until the partial strong
> model fits as well as the weak model.  I would also recommend treating
> thresholds as a set -- since multiple thresholds of item ss2 seem to
> (possibly) differ, you could try releasing all of that item's thresholds
> to see if that alone makes your model fit as well as the weak model.
>
> Terrence D. Jorgensen
> Postdoctoral Researcher, Methods and Statistics
> Research Institute for Child Development and Education, the University
> of Amsterdam
> UvA web page: http://www.uva.nl/profile/t.d.jorgensen
>

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "lavaan" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/lavaan/x6Hd7-0mpzU/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> lavaan+un...@googlegroups.com
> <mailto:lavaan+un...@googlegroups.com>.
> To post to this group, send email to lav...@googlegroups.com
> <mailto:lav...@googlegroups.com>.
> Visit this group at https://groups.google.com/group/lavaan.
> For more options, visit https://groups.google.com/d/optout.