semTools Reliability with Longitudinal Bifactor Question

[Allie Choate]

Hello all,

I had a question regarding the calculations behind omega hierarchical using the 'reliability' command in semTools.

I have a bifactor model with a general and two specific factors being estimated across 7 time points, so I have autoregressive paths connecting each respective factor over time (e.g., general factor at first time point predicting itself at the next time point). My confusion is that when I calculate McDonald's OmegaH using the reliability function for this model, the estimates for 'omega3' in the output don't match my hand calculations. However, if I estimate the bifactor model cross-sectionally at each time point, the estimates from semTools do match my hand calculations.

I'm assuming then that the autoregressives are impacting my reliability estimates somehow, but I'm not sure how exactly this is playing out...The estimate for omegaH that semTools gives me for my general factor at the first time point is ~.655, but my hand calculations show that the value should be around ~.58.

If anyone has any thoughts on what could be happening, that would be very much appreciated! Ideally, I was planning on using the estimates from the model with all time-points modeled simultaneously, but now I'm wondering if it would be more appropriate to use the cross-sectional estimates instead? Again, any insight or comments would be very much appreciated!

Thanks!

[Terrence Jorgensen]

Have you tried changing your latent (auto)regressions to covariances, and doing your hand calculations with those estimates?  I assume semTools::reliability() estimates will not change because it uses the model-implied covariance matrix of latent variables, as it should be.  When you have latent regressions, then the latent (co)variances are not model parameters; instead, Psi is the residual covariance matrix (so only unconditional (co)variances for exogenous latent variables).  Treating the residual (co)variances like total (co)variances could be messing up your hand calculations.

Also, lavaan/semTools can't possibly know your model is longitudinal, so it will only calculate the reliability of the "total" composite by assuming you will sum ALL items in your model (i.e., across all time points, which again, lavaan/semTools can't recognize are different time points).  So separate models for each occasion is probably what you want, if you are interested in the "total" (actual) composite reliability rather than the abstract fantasy reliability of a particular construct that has other latent factors somehow partialed out.

Terrence D. Jorgensen

Assistant Professor, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

http://www.uva.nl/profile/t.d.jorgensen

[Allie Choate]

Hi Dr. Jorgensen,

Thank you for your detailed reply! This is very helpful and would make sense then as to why my hand calculations aren't lining up.

However, I re-ran the bifactor model with switching the latent auto-regressions to covariances, and the reliability estimates between the model with the covariances and the model with the regressions are still fairly different, especially for the 'omega3' estimates (omega2 is a little closer, but still varies). Not quite sure why this is changing so much, but seems like it may be best to estimate the models cross-sectionally... at least for reliability purposes anyways.

[Terrence Jorgensen]

I re-ran the bifactor model with switching the latent auto-regressions to covariances, and the reliability estimates between the model with the covariances and the model with the regressions are still fairly different

If your autoregressive model did not have a saturated latent structure (e.g., if you only estimated first-order autoregressions), then the model-implied latent covariance matrix will be different.  You can compare the models' chi-squared and df to see if they are equivalent.  If not, then you can't expect reliability estimates to be identical.

[Allie Choate]

Ah, makes sense!

Thanks so much again for your help!

--
You received this message because you are subscribed to a topic in the Google Groups "lavaan" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lavaan/1M4Hb64Rbx0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/af2f1ed4-22e6-4648-a893-af519d533876%40googlegroups.com.

--

Allie Choate

Doctoral Student of Clinical Psychology

University of South Florida

Office: PCD 4144B