Longitudinal CFA with dichotomously scored items

83 views
Skip to first unread message

Len Kali

unread,
Sep 8, 2020, 8:41:34 AM9/8/20
to lavaan
I am working with longitudinal data with 3 measurement occasions & 14 dichotomously scored items per measurement occasion. I would be very grateful for guidance regarding longitudinal CFA and doing separate CFAs for each measurement occasion.

Non-longitudinal CFA:

1) My non-longitudinal CFA questions pertain to output from the below code. (Time 1 is used as an example, but Times 2 and 3 follow the same structure.) Is my code correct for how to treat dichotomously scored items? (I'm wondering if my next questions are just because of an error in my code.)

T1.CFA <- '

T1 =~

   X1.T1 +

   X2.T1 +

   X3.T1 +

   X4.T1 +

   X5.T1 +

   X6.T1 +

   X7.T1 +

   X8.T1 +

   X9.T1 +

   X10.T1 +

   X11.T1 +

   X12.T1 +

   X13.T1 +

   X14.T1

   '


fitT1.CFA <- cfa(T1.CFA, data=data, 

                             ordered=c(   "X1.T1" ,

                                          "X2.T1" ,

                                          "X3.T1" ,

                                          "X4.T1" ,

                                          "X5.T1" ,

                                          "X6.T1" ,

                                          "X7.T1" ,

                                          "X8.T1" ,

                                          "X9.T1" ,

                                          "X10.T1" ,

                                          "X11.T1" ,

                                          "X12.T1" ,

                                          "X13.T1" ,

                                          "X14.T1"))       

summary(fitT1.CFA, fit.measures=TRUE, standardized=TRUE)


2) Under the Variances heading in the output, there are Estimates but no Standard Errors. What does that mean?

3) What does it mean the below warning message disappeared when I added meanstructure=TRUE, std.lv=TRUE?

1: In lav_model_estimate(lavmodel = lavmodel, lavpartable = lavpartable,  :

  lavaan WARNING: the optimizer warns that a solution has NOT been found!


Longitudinal CFA: 

4) When trying to do longitudinal CFA to test for measurement invariance across time, I got these warning messages:

1: In lav_samplestats_step2(UNI = FIT, wt = wt, ov.names = ov.names, : lavaan WARNING: correlation between variables X4.T2 and X3.T2 is (nearly) 1.0 

2: In lav_model_vcov(lavmodel = lavmodel, lavsamplestats = lavsamplestats, : lavaan WARNING: The variance-covariance matrix of the estimated parameters (vcov) does not appear to be positive definite! The smallest eigenvalue (= -8.874240e-4) is smaller than zero. This may be a symptom that the model is not identified.

It only warns that neighboring items at the same measurement occasion are highly correlated in the longitudinal CFA, not in the individual CFA for that measurement occasion. This makes me wonder if it's due to programming errors on my part. I'm also wondering if warning 2 (non positive-definite variance-covariance matrix) has to do with the large amount of parameters it's estimating with 3 measurement occasions * 14 items each? Could this be a numerical integration problem? Warning 1 was absent in the individual CFA for T2, which makes me wonder if both longitudinal CFA warning messages were a symptom of the same thing, and if that issue is my code or perhaps this being too much for the integrator.

Here is my approximate code for the longitudinal CFAs -- approximate, because I'm still actively tinkering with it in response to warning messages. I'm adapting from the helpful code provided at https://www.rdocumentation.org/packages/semTools/versions/0.5-2/topics/measEq.syntax. 

I will be grateful for any advice on what I should be doing differently!

L.cfa <- '

T1 =~

   X1.T1 +

   X2.T1 +

   X3.T1 +

   X4.T1 +

   X5.T1 +

   X6.T1 +

   X7.T1 +

   X8.T1 +

   X9.T1 +

   X10.T1 +

   X11.T1 +

   X12.T1 +

   X13.T1 +

   X14.T1

 

T2 =~

   X1.T2 +

   X2.T2 +

   X3.T2 +

   X4.T2 +

   X5.T2 +

   X6.T2 +

   X7.T2 +

   X8.T2 +

   X9.T2 +

   X10.T2 +

   X11.T2 +

   X12.T2 +

   X13.T2 +

   X14.T2

  

T3 =~   

   X1.T3 +

   X2.T3 +

   X3.T3 +

   X4.T3 +

   X5.T3 +

   X6.T3 +

   X7.T3 +

   X8.T3 +

   X9.T3 +

   X10.T3 +

   X11.T3 +

   X12.T3 +

   X13.T3 +

   X14.T3  '

 

longFacNames <- list(X = c("T1", "T2", "T3"))

 

# test for longitudinal measurement invariance with help from semTools

 

test.seq <- list(strong = c("thresholds","loadings","intercepts"),

                 means = "means",

                 strict = "residuals")

meq.list <- list()

for (i in 0:length(test.seq)) {

  if (i == 0L) {

    meq.label <- "configural"

    group.equal <- ""

    long.equal <- ""

  } else {

    meq.label <- test.seq[i]

    group.equal <- test.seq[1:i]

    long.equal <- test.seq[1:i]

  }

  meq.list[[meq.label]] <- measEq.syntax(configural.model = L.cfa,

                                         data = data,

                                         

                                         ordered=c(   "X1.T1" ,

                                                      "X2.T1" ,

                                                      "X3.T1" ,

                                                      "X4.T1" ,

                                                      "X5.T1" ,

                                                      "X6.T1" ,

                                                      "X7.T1" ,

                                                      "X8.T1" ,

                                                      "X9.T1" ,

                                                      "X10.T1" ,

                                                      "X11.T1" ,

                                                      "X12.T1" ,

                                                      "X13.T1" ,

                                                      "X14.T1" ,

                                                      "X1.T2" ,

                                                      "X2.T2" ,

                                                      "X3.T2" ,

                                                      "X4.T2" ,

                                                      "X5.T2" ,

                                                      "X6.T2" ,

                                                      "X7.T2" ,

                                                      "X8.T2" ,

                                                      "X9.T2" ,

                                                      "X10.T2" ,

                                                      "X11.T2" ,

                                                      "X12.T2" ,

                                                      "X13.T2" ,

                                                      "X14.T2" ,

                                                      "X1.T3" ,

                                                      "X2.T3" ,

                                                      "X3.T3" ,

                                                      "X4.T3" ,

                                                      "X5.T3" ,

                                                      "X6.T3" ,

                                                      "X7.T3" ,

                                                      "X8.T3" ,

                                                      "X9.T3" ,

                                                      "X10.T3" ,

                                                      "X11.T3" ,

                                                      "X12.T3" ,

                                                      "X13.T3" ,

                                                      "X14.T3"),

                                                     

                                         parameterization = "theta",

                                         #ID.fac = "std.lv",

                                         ID.cat = "Mplus",

                                         group = NULL,

                                         #group.equal = NULL,

                                         longFacNames = longFacNames,

                                         long.equal = long.equal,

                                         return.fit = TRUE)

}

 

compareFit(meq.list)


I will be VERY grateful for any advice, thank you!

Terrence Jorgensen

unread,
Sep 11, 2020, 4:26:29 AM9/11/20
to lavaan
1) Is my code correct for how to treat dichotomously scored items? 

Looks fine

2) Under the Variances heading in the output, there are Estimates but no Standard Errors. What does that mean?

It means those values are fixed.  In fact, you are using the default parameterization = "delta", so the variances are not even model parameter (well, they are implied by the latent scales, factor loadings, and factor variance).  Here are two informative tutorials, and the Mplus documentation of the parameterizations (which lavaan also uses):




3) What does it mean the below warning message disappeared when I added meanstructure=TRUE, std.lv=TRUE?

  lavaan WARNING: the optimizer warns that a solution has NOT been found!


Not sure why that would only happen when setting the first factor loading to 1.  What does your summary() output look like when you set std.lv=TRUE?  Are the loadings all positive?

It should not be necessary to set meanstructure=TRUE.  That is the default (and necessary) when you have any ordered= outcomes.
 
4) When trying to do longitudinal CFA to test for measurement invariance across time, I got these warning messages:

1: In lav_samplestats_step2(UNI = FIT, wt = wt, ov.names = ov.names, : lavaan WARNING: correlation between variables X4.T2 and X3.T2 is (nearly) 1.0 

2: In lav_model_vcov(lavmodel = lavmodel, lavsamplestats = lavsamplestats, : lavaan WARNING: The variance-covariance matrix of the estimated parameters (vcov) does not appear to be positive definite! The smallest eigenvalue (= -8.874240e-4) is smaller than zero. This may be a symptom that the model is not identified.

It only warns that neighboring items at the same measurement occasion are highly correlated in the longitudinal CFA, not in the individual CFA for that measurement occasion. This makes me wonder if it's due to programming errors on my part.

That doesn't seem plausible, but you can check the estimated polychoric correlations from your longitudinal model and compare them to the estimates from your Time-2 model.

lavInspect(fitT2.CFA, "sampstat")
lavInspect
(meq.list[[1]], "sampstat")


I'm also wondering if warning 2 (non positive-definite variance-covariance matrix) has to do with the large amount of parameters it's estimating with 3 measurement occasions * 14 items each?

Given the size of your model, if your N < 500 or 1000 (depending on the asymmetry of your thresholds), you could expect some instability, and probably a somewhat inflated chi-squared statistic.



Could this be a numerical integration problem? 

lavaan is not using numerical integration.  By default, it uses DWLS estimation.

Warning 1 was absent in the individual CFA for T2, which makes me wonder if both longitudinal CFA warning messages were a symptom of the same thing

The fact that they appear together makes sense. If 2 variables are so highly correlated, it would make sense that their parameters are probably also have highly correlated sampling distributions, leading to the negative eigenvalue.

I will be grateful for any advice on what I should be doing differently!

On the help page you reference, I generally advised not to use return.fit=TRUE.  If you are having trouble with estimation, you should be fitting models one at a time so that you know exactly where the problems occur.

Terrence D. Jorgensen
Assistant Professor, Methods and Statistics
Research Institute for Child Development and Education, the University of Amsterdam

Len Kali

unread,
Sep 11, 2020, 10:41:12 AM9/11/20
to lav...@googlegroups.com
THANK YOU so much for your incredibly helpful, thorough, and clear responses and explanations, Dr. Jorgensen!!! I am extremely grateful for you sharing your time, wisdom, and guidance. Also, I cannot thank you enough for measEq.syntax, which has been immensely helpful!

 1) Re:

Not sure why that would only happen when setting the first factor loading to 1.  What does your summary() output look like when you set std.lv=TRUE?  Are the loadings all positive?

Yes, all loadings are positive. Under the Estimate column they are somewhat higher without std.lv=TRUE, and the Std.lv and Std.all columns look identical. When std.lv=TRUE, all 3 columns look identical.


 2) Thank you so much for your advice to check the estimated polychoric correlations from the longitudinal model and compare them to the estimates from the Time-2 model. Further, thank you so much for providing such helpful code to run diagnostics! For the Time 2 non-longitudinal CFA, lavInspect(fitT2.CFA, "sampstat"), under$cov, the relation is .960 between the two items implicated in the longitudinal CFA's warning message of a correlation of nearly 1.

For the longitudinal CFA, lavInspect(meq.list[[1]], "sampstat"), the relation was .991.

In both, there are other inter-item correlations of .9, though those two items appear to have the highest correlation. How high is too high? I’m considering item parceling but am unsure of the implications of doing that at just one measurement occasion when testing for longitudinal measurement invariance.


 3) With no missing data treatment, the sample size is under 300 at each measurement occasion and is 117 for the longitudinal CFA. Is this too small for a model with 3 measurement occasions and 14 items per measurement occasion?


I would be very grateful for thoughts and advice. 

Terrence Jorgensen

unread,
Sep 14, 2020, 9:33:09 AM9/14/20
to lavaan

How high is too high?

I'd say they both are, but clearly lavaan only issues the message when the polychoric correlation is closer to 1 than 0.96
 

I’m considering item parceling but am unsure of the implications of doing that at just one measurement occasion when testing for longitudinal measurement invariance.

 If you do it, you should do it at all occasions.  Alternatively, you could simply choose one or the other as an indicator, discarding the other (again, at all occasions), but I would only do that if the two items were nearly identical at all occasions.  You can check 2-way contingency tables for that pair at each occasion to see how (non)redundant the variables are.

table(data$X4.T1, data$X3.T1)
table
(data$X4.T2, data$X3.T2)
table
(data$X4.T3, data$X3.T3)

 3) With no missing data treatment, the sample size is under 300 at each measurement occasion and is 117 for the longitudinal CFA.


That is why your polychorics differ between the full and T-2 models.  They are being fit to different data.
 

Is this too small for a model with 3 measurement occasions and 14 items per measurement occasion?

Certainly yes.  You are estimating 129 parameters in the full configural model, which is > N.  Even if you set missing="pairwise", you would have very few observations per estimated parameter.  Consider using estimator="PML", which I believe has better small-sample performance, but it is still not ideal to have so little data to estimate so large a model.

Reply all
Reply to author
Forward
0 new messages