Longitudinal invariance test with two factors

1,354 views
Skip to first unread message

Ashenafi

unread,
Jul 4, 2015, 2:38:30 AM7/4/15
to lav...@googlegroups.com
Dear Lavaan users,

I am interested to conduct longitudinal invariance test using two factors/scales on a categorical data. I learned longInvariance () can do the function but it works only in one scale. Would you mind suggesting the convenient way to do it in Lavaan? If possible,  with syntax example.

Kind regards,
Ashenafi   

Terrence Jorgensen

unread,
Jul 4, 2015, 9:12:09 AM7/4/15
to lav...@googlegroups.com

I learned longInvariance () can do the function but it works only in one scale.

It also does not work for categorical indicators.  You just need to write out the model syntax yourself.

Would you mind suggesting the convenient way to do it in Lavaan? If possible,  with syntax example.

There are some examples at this web page:


I think the examples are all for multiple-group models, but the same principles apply for testing longitudinal invariance (just think of Time 1 and 2 as Group 1 and 2).

Terry

Ashenafi

unread,
Jul 4, 2015, 2:10:11 PM7/4/15
to lav...@googlegroups.com
Hi Terry,

Thank you so much. This will definitely solve my problem. 
Ashenafi

Ulrich Schroeders

unread,
Jul 6, 2015, 6:28:06 AM7/6/15
to lav...@googlegroups.com
Dear Yves,

a follow-up question: We don't want to use MGCFA with categorial data in a longitudinal setting, but to estimate the all model parameters simultaneously (wide instead of long format), as recommended by Little, Preacher, Card, & Selig (2007) DOI: 10.1177/0165025407077757 , but this is for continous data. The main reason for this approach is that we want to allow the residuals to correlate over time.
Our problem: Because in categorical data thresholds and factor loadings have to be varied in tandem, we encounter some problem ( lavaan WARNING: could not compute standard errors! ) for the configual invariance testing. For this first step in invariance testing we allowed factor loadings AND thresholds to be freely estimated and, additionally, the residual variances were set to 1 (for identification purpose). However, we are not sure how to deal with the mean structure: Estimating all means of the latent variables over time was resulting in the above mentioned (computational) issue. Fixing them to 0 at all measuremen time points, somehow solves the problem of SE computation.
Are there other means to estimate SEs with WLMSV estimator? We also wonder why this could be a computational problem.

Thanks in advance for your help, keep up your great work!
Ulrich + Ashenafi

Ulrich Schroeders

unread,
Jul 6, 2015, 7:56:56 AM7/6/15
to lav...@googlegroups.com
It seems we found the answer: http://www.statmodel.com/discussion/messages/23/94.html?1396478598
That is, fixing the means at all time point to zero since we allow the thresholds to be different.

Kind regards, thx,
Ulrich + Ashenafi

Oscar Kjell

unread,
Dec 25, 2016, 3:36:00 AM12/25/16
to lavaan
Hi Lavaan users,
I've unknowingly tried to use the longinvariance() function for a 2-factor model with interval data. However the link above, that should explain how to carry out the task without longinvariance, is broken. Could someone please advice me on how to do this without longinvariance?

I've asked this question on stackoverflow as well; if someone perhaps wants to answer there or here. http://stackoverflow.com/questions/41099149/lavaan-longitudinal-invariance-cfa-with-a-2-factor-model-in-r 

Any help is much appreciated,
Kind Regards,
Oscar

Ulrich Schroeders

unread,
Dec 27, 2016, 2:09:30 PM12/27/16
to lavaan
Dear Oscar,

this thread was originally about categorical data and longitudinal MI.
In case you (simply) want MI in a multi-group case with groups equal to time points (i.e, without residual correlations between indicators across time points) and continuous data, use the measurementInvariance function of the semTools package.

Kind regards
Ulrich

kma...@aol.com

unread,
Dec 27, 2016, 10:47:21 PM12/27/16
to lavaan
Oscar,
From the longinvariance() help file:

>This function currently works with only one scale.

https://cran.r-project.org/web/packages/semTools/semTools.pdf

You can specify the model and constraints manually.

Keith
------------------------
Keith A. Markus
John Jay College of Criminal Justice, CUNY
http://jjcweb.jjay.cuny.edu/kmarkus
Frontiers of Test Validity Theory: Measurement, Causation and Meaning.
http://www.routledge.com/books/details/9781841692203/


JR

unread,
Apr 23, 2018, 6:33:47 AM4/23/18
to lavaan
Is there any source you could suggest, that describes the basics one needs to do this manually? I'm really stuck on the matter, as I have eight factors and two time points and have no idea how to compare the models.

I have written down the two models. Now, before when I wanted to compare models I just used chi square difference test:

cbind(CFA01=inspect(fit01, 'fit.measures'), CFA03=inspect(fit03, 'fit.measures'))
anova(fit01, fit03) 

would that give me configural invariance? 

Terrence Jorgensen

unread,
Apr 24, 2018, 8:59:14 AM4/24/18
to lavaan
Is there any source you could suggest, that describes the basics one needs to do this manually?

I'm working on a paper demonstrating invariance tests for categorical indicators across both groups and occasions.  It's not yet in shape enough to share, but if you email me directly (follow the link under my signature), I can make a note to share it with you when it is ready.

Now, before when I wanted to compare models I just used chi square difference test:
anova(fit01, fit03) 

That's right.

would that give me configural invariance? 

No, configural invariance is not tested using model comparison.  Comparing a model that constrains measurement parameters to equality to a model that does not (i.e., the configural model) tests the null hypothesis that those constraints hold. 

Configural invariance is typically tested by looking at that model's chi-squared test, not a chi-squared difference test.  But that conflates configural invariance with all other ways that the configural model is constrained (i.e., why it has degrees of freedom).  You can use a permutation test to test the null hypothesis of configural invariance, irrespective of whether the model fits perfectly.



Terrence D. Jorgensen
Postdoctoral Researcher, Methods and Statistics
Research Institute for Child Development and Education, the University of Amsterdam

Ulrich Schroeders

unread,
Apr 24, 2018, 10:51:06 AM4/24/18
to lavaan
If you are looking for longitudinal measurement invariance testing in a MGCFA context with categorical data:
Edossa, A. K., Schroeders, U., Weinert, S., & Artelt, C. (2018). The development of emotional and behavioral self-regulation and their effects on academic achievement in childhood. International Journal of Behavioral Development, 42, 192–202. https://doi.org/10.1177/0165025416687412
Yeah, I know shameless self-citing... but Table 1 summarizes the necessary constraints.

In the course of the revision another more technical paper appeared:
Liu, Y., Millsap, R. E., West, S. G., Tein, J.-Y., Tanaka, R., & Grimm, K. J. (2016). Testing measurement invariance in longitudinal data with ordered-categorical measures. Psychological Methods. doi:10.1037/met0000075
Even though the factor loading invariance step doesn't make sense - in my understanding - since factor loadings and intercepts have to be varied in tandem with categorical data.

Hope this helps, kind regards
Ulrich

fotin...@gmail.com

unread,
Aug 3, 2018, 3:38:41 PM8/3/18
to lavaan
Hi Ulrich,

Your paper has been very useful.  I am still struggling with syntax though.  Does that mean if I want to apply all the different constraints mentioned in Table 1 I will need to list all the syntax out and include Time 0 Time 1 etc even if I have one scale?

I still cannot find a good example of longInvariance where all the syntax is listed out for two different scales, as mentioned above the link is broken.  Is there an update on this front as to a good basic resource for this now?  Possibly an example for the paper you have referenced? :-)

Thanks!

Ulrich Schroeders

unread,
Aug 4, 2018, 3:22:46 AM8/4/18
to lavaan
Hi,

I wanted to do this for a long time; here U go:

This is the syntax we used in the paper, i.e., the syntax to test for configural, strong, and strict longitudinal MI with categorical data.

Hope this helps, kind regards
Ulrich

Terrence Jorgensen

unread,
Aug 26, 2018, 6:08:40 PM8/26/18
to lavaan
Anyone still paying attention to this thread might be interested to know there is a new function in the development version, which can help with testing invariance of multiple repeatedly measured factors.  

Jessica Fritz

unread,
Sep 5, 2018, 7:30:44 AM9/5/18
to lavaan
Hi lavaan-ers,

I have a longitudinal CFA (two time points = two factors) and a couple of categorical items. To ease the explanation of my question, I simulated some data (as attached; please note the data does not fit the model well, but should be sufficient for demonstrative purposes), namely three categorical items for each of the two time points.

I now would like to 'replicate' the invariance models (see below) I would get if I would use the longInvariance() package and (naively) would treat my variables as normal.

```{r}
library("lavaan")
library("semTools")

#1: specify model
Model_c <- 'Factor_t1 =~ x1t1 + x2t1 + x3t1
           Factor_t2 =~ x1t2 + x2t2 + x3t2'

#2: Create list of variables
var1 <- c("x1t1", "x2t1", "x3t1")
var2 <- c("x1t2", "x2t2", "x3t2")
constrainedVar <- list(var1, var2)

#3: fit the models
fit_c <- longInvariance(Model_c, varList = constrainedVar, auto = "all", constrainAuto = TRUE, data =
                                  mydata,  estimator="MLR", strict = TRUE, std.lv = TRUE)

#4: results
#1: configural model
results_c1 <- summary(fit_c$fit.configural, standardized=TRUE, fit.measures=TRUE, rsquare=TRUE, modindices=TRUE)
#fixed (both latent factor variances)

#2:fixed loadings model
results_c1 <- summary(fit_c$fit.loadings, standardized=TRUE, fit.measures=TRUE, rsquare=TRUE, modindices=TRUE)
#fixed (first (not both!) latent factor variance(s), loadings)

#3: fixed loadings and intercepts model
results_c1 <- summary(fit_c$fit.intercepts, standardized=TRUE, fit.measures=TRUE, rsquare=TRUE, modindices=TRUE)
#fixed (first (not both!) latent factor variance(s), loadings, intercepts)

#4: fixed loadings intercept and item variances model
results_c1 <- summary(fit_c$fit.residuals, standardized=TRUE, fit.measures=TRUE, rsquare=TRUE, modindices=TRUE)
#fixed (first (not both!) latent factor variance(s), loadings, intercepts, item variances)
```

As I'd rather not want to treat my variables as normal but as categorical ordered (ordinal), and would like to apply autocorrelations and equating item variances for the two time points, I cannot use the measurementInvarianceCat()package. I thus tried to write the necessary code manually to 'replicate' the above results in the categorical way. In short, I get a problem with equating the item variances for the two time points, which I will shed light on in a bit more detail below the code.


```{r}
#1: configural model
Model1 <- 'Factor_t1 =~ NA*x1t1 + x2t1 + x3t1
           Factor_t2 =~ NA*x1t2 + x2t2 + x3t2
Factor_t1 ~~ 1*Factor_t1
Factor_t2 ~~ 1*Factor_t2
x1t1 ~~ x1t2
x2t1 ~~ x2t2
x3t1 ~~ x3t2'
fit1 <- cfa(Model1, data = mydata,  estimator="WLSMV", ordered=c("x1t1", "x2t1", "x3t1",
                                                                 "x1t2", "x2t2", "x3t2"))
results1 <- summary(fit1, fit.measures=TRUE, rsquare=TRUE, standardized=TRUE)
#fixed (both latent factor variances)


#2: fixed loadings model
Model2 <- 'Factor_t1 =~ NA*x1t1 + aa*x1t1 + bb*x2t1 + cc*x3t1
           Factor_t2 =~ NA*x1t2 + aa*x1t2 + bb*x2t2 + cc*x3t2
Factor_t1 ~~ 1*Factor_t1
x1t1 ~~ x1t2
x2t1 ~~ x2t2
x3t1 ~~ x3t2'
fit2 <- cfa(Model2, data = mydata,  estimator="WLSMV", ordered=c("x1t1", "x2t1", "x3t1",
                                                                 "x1t2", "x2t2", "x3t2"))
results2 <- summary(fit2, fit.measures=TRUE, rsquare=TRUE, standardized=TRUE)
#fixed (first (not both!) latent factor variance(s), loadings)


#3: fixed loadings and thresholds model
Model3 <- 'Factor_t1 =~ NA*x1t1 + aa*x1t1 + bb*x2t1 + cc*x3t1
           Factor_t2 =~ NA*x1t2 + aa*x1t2 + bb*x2t2 + cc*x3t2
Factor_t1 ~~ 1*Factor_t1
x1t1 ~~ x1t2
x2t1 ~~ x2t2
x3t1 ~~ x3t2
x1t1 | c(a)*t1 + c(b)*t2 + c(c)*t3
x1t2 | c(a)*t1 + c(b)*t2 + c(c)*t3
x2t1 | c(d)*t1 + c(e)*t2 + c(f)*t3
x2t2 | c(d)*t1 + c(e)*t2 + c(f)*t3
x3t1 | c(i)*t1 + c(j)*t2 + c(k)*t3
x3t2 | c(i)*t1 + c(j)*t2 + c(k)*t3'
fit3 <- cfa(Model3, data = mydata,  estimator="WLSMV", ordered=c("x1t1", "x2t1", "x3t1",
                                                                 "x1t2", "x2t2", "x3t2"))
results3 <- summary(fit3, fit.measures=TRUE, rsquare=TRUE, standardized=TRUE)
#fixed (first (not both!) latent factor variance(s), loadings, thresholds)


#4: fixed loadings thresholds and item variances model --> fix factor variance
#lavaan recognizes the variance modifiers (dd,ee,ff) but does not equate them, why?
Model4 <- 'Factor_t1 =~ NA*x1t1 + aa*x1t1 + bb*x2t1 + cc*x3t1
           Factor_t2 =~ NA*x1t2 + aa*x1t2 + bb*x2t2 + cc*x3t2
Factor_t1 ~~ 1*Factor_t1
x1t1 ~~ x1t2
x2t1 ~~ x2t2
x3t1 ~~ x3t2
x1t1 | c(a)*t1 + c(b)*t2 + c(c)*t3
x1t2 | c(a)*t1 + c(b)*t2 + c(c)*t3
x2t1 | c(d)*t1 + c(e)*t2 + c(f)*t3
x2t2 | c(d)*t1 + c(e)*t2 + c(f)*t3
x3t1 | c(i)*t1 + c(j)*t2 + c(k)*t3
x3t2 | c(i)*t1 + c(j)*t2 + c(k)*t3
x1t1 ~~ dd*x1t1
x1t2 ~~ dd*x1t2
x2t1 ~~ ee*x2t1
x2t2 ~~ ee*x2t2
x3t1 ~~ ff*x3t1
x3t2 ~~ ff*x3t2'
fit4 <- cfa(Model4, data = mydata,  estimator="WLSMV", ordered=c("x1t1", "x2t1", "x3t1",
                                                                 "x1t2", "x2t2", "x3t2"))
results4 <- summary(fit4, fit.measures=TRUE, rsquare=TRUE, standardized=TRUE)
#fixed (first (not both!) latent factor variance(s), loadings, thresholds, --> not item variance why?)


#4B: fixed loadings thresholds and item variances model --> fix item variance
#lavaan recognizes the variance modifiers (dd,ee,ff) but does not equate them, why?
Model4B <- 'Factor_t1 =~ aa*x1t1 + bb*x2t1 + cc*x3t1
            Factor_t2 =~ aa*x1t2 + bb*x2t2 + cc*x3t2
x1t1 ~~ x1t2
x2t1 ~~ x2t2
x3t1 ~~ x3t2
x1t1 | c(a)*t1 + c(b)*t2 + c(c)*t3
x1t2 | c(a)*t1 + c(b)*t2 + c(c)*t3
x2t1 | c(d)*t1 + c(e)*t2 + c(f)*t3
x2t2 | c(d)*t1 + c(e)*t2 + c(f)*t3
x3t1 | c(i)*t1 + c(j)*t2 + c(k)*t3
x3t2 | c(i)*t1 + c(j)*t2 + c(k)*t3
x1t1 ~~ dd*x1t1
x1t2 ~~ dd*x1t2
x2t1 ~~ ee*x2t1
x2t2 ~~ ee*x2t2
x3t1 ~~ ff*x3t1
x3t2 ~~ ff*x3t2'
fit4B <- cfa(Model4B, data = mydata,  estimator="WLSMV", ordered=c("x1t1", "x2t1", "x3t1",
                                                                   "x1t2", "x2t2", "x3t2"))
results4B <- summary(fit4B, fit.measures=TRUE, rsquare=TRUE, standardized=TRUE)
#fixed (both first item variances, loadings, thresholds, --> not item variance why?)


#4C: fixed loadings thresholds and item variances model  --> fix both factor variances
#when fixing two factors (which I actually think may be redundant as the modifier "aa" for the first items loadings needs to be estimated only once), I do get the fixed variances for the modifiers (dd,ee,ff).
Model4C <- 'Factor_t1 =~ NA*x1t1 + aa*x1t1 + bb*x2t1 + cc*x3t1
           Factor_t2 =~ NA*x1t2 + aa*x1t2 + bb*x2t2 + cc*x3t2
Factor_t1 ~~ 1*Factor_t1
Factor_t2 ~~ 1*Factor_t2
x1t1 ~~ x1t2
x2t1 ~~ x2t2
x3t1 ~~ x3t2
x1t1 | c(a)*t1 + c(b)*t2 + c(c)*t3
x1t2 | c(a)*t1 + c(b)*t2 + c(c)*t3
x2t1 | c(d)*t1 + c(e)*t2 + c(f)*t3
x2t2 | c(d)*t1 + c(e)*t2 + c(f)*t3
x3t1 | c(i)*t1 + c(j)*t2 + c(k)*t3
x3t2 | c(i)*t1 + c(j)*t2 + c(k)*t3
x1t1 ~~ dd*x1t1
x1t2 ~~ dd*x1t2
x2t1 ~~ ee*x2t1
x2t2 ~~ ee*x2t2
x3t1 ~~ ff*x3t1
x3t2 ~~ ff*x3t2'
fit4C <- cfa(Model4C, data = mydata,  estimator="WLSMV", ordered=c("x1t1", "x2t1", "x3t1",
                                                                 "x1t2", "x2t2", "x3t2"))
results4C <- summary(fit4C, fit.measures=TRUE, rsquare=TRUE, standardized=TRUE)
#fixed (both (!) latent factor variances, loadings, thresholds, item variances, R-squares)
```

Lavaan does seem to recognize my variance modifiers (dd,ee,ff), as it correctly prints them in the results output, yet, even though they are recognized, they are not equal for the two time points.

My question now is, whether anyone knows or can advice me on how I can specify model 4 (either A, B or C) correctly, so that the item variances in those models belonging to the identifiers (dd,ee,ff) will have the same variance per identifier (without having to fix both factor variances to 1, as I did in model 4C)?

Is that possible at all with ordinal data?

Thank you very much for helping me,

Jes
mydata_simulated_2018.09Sep.05.csv

ulrich.s...@gmail.com

unread,
Sep 5, 2018, 7:39:46 AM9/5/18
to lav...@googlegroups.com
Did you try parametrization=theta and std.lv=T? Keep a look at the factor loading constraints, however.

Kind regards
Ulrich

--
You received this message because you are subscribed to a topic in the Google Groups "lavaan" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lavaan/nfdatPgLLhc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lavaan+un...@googlegroups.com.
To post to this group, send email to lav...@googlegroups.com.
Visit this group at https://groups.google.com/group/lavaan.
For more options, visit https://groups.google.com/d/optout.

Jessica Fritz

unread,
Sep 5, 2018, 7:53:09 AM9/5/18
to lav...@googlegroups.com
Dear Ulrich,
 
Brilliant! parametrization=theta does seem to do the trick.
 
Very many thanks!
All the best,
 
Jes
 
Gesendet: Mittwoch, 05. September 2018 um 12:39 Uhr
Von: "ulrich.s...@googlemail.com" <ulrich.s...@gmail.com>
An: lav...@googlegroups.com
Betreff: Re: Longitudinal invariance test with two factors
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages