Multi-Level CFA

[stephenba...@gmail.com]

Hi,

I am currently attempting to learn multi-level reliability for scales.

I want to try and replicate the study [1] where the author suggests to implement the following steps. The aim is to see at what level of measurement an analysis is most effective. In this paper the auther aims to prove that people should analysis the Data in GLOBE not on individual level but on societal level. He then goes about proving this showing the loadings on societal level are far higher than on individual level

• "Step 1: perform a conventional confirmatory factor analysis on the sample total covariance"
• "Step 2: estimate between-group level variation"
• "Step 3: perform a factor analysis on the sample pooled-within covariance matrix"
• "Step 4: perform a factor analysis on the sample between-group covariance matrix"
• "Step 5: perform multilevel confirmatory factor analysis"

Im relatively new to SEM and CFA and like using the lavaan package very much. It appears the authors of this paper used MPlus

Is it possible to have this workflow in lavaan using R? I see in page two of the manual [2] that lavaan provides "support for hierarchical/multilevel datasets (multilevel cfa, multilevel sem)". I have not been able to fin any tutorial on it

I'm aware of the other packages XXm [3] and openMX [4] but wanted to see if lavaan supports this type of analysis before looking further afield and learning a new syntax .

References

https://www.statmodel.com/download/Dyer%202005-Leadership%20Quarterly-Multilevel%20FA.pdf [1]
http://lavaan.ugent.be/tutorial/before.html [2]
http://xxm.times.uh.edu/ [3]
http://openmx.ssri.psu.edu/education [4]

Thanks
Steve

[Terrence Jorgensen]

I am currently attempting to learn multi-level reliability for scales.

That is a complex topic.  Consider adding this to your reading list:

Geldhof, J. G., Preacher, K. J., & Zyphur, M. J. (2014). Reliability estimation in a multilevel confirmatory factor analysis framework. Psychological Methods, 19(1), 72–91. doi:10.1037/a0032138

I want to try and replicate the study [1] where the author suggests to implement the following steps. The aim is to see at what level of measurement an analysis is most effective. In this paper the auther aims to prove that people should analysis the Data in GLOBE not on individual level but on societal level. He then goes about proving this showing the loadings on societal level are far higher than on individual level

By definition, that must be the case if loadings are approximately equal across clusters (metric invariance across clusters implies equal loadings at the between and within levels).  And if strong measurement invariance holds exactly, that implies zero residual variance at the between level (so standardized factor loadings will be close to 1 when strong invariance holds approximately).

Jak, S., Oort, F. J., & Dolan, C. V. (2013). A test for cluster bias: Detecting violations of measurement invariance across clusters in multilevel data. Structural Equation Modeling, 20(2), 265–282. doi:10.1080/10705511.2013.769392

So higher standardized loadings at the between level don't necessarily imply that the construct has more meaning or is "measured better" at that level.  The reliability you should be interested in is the reliability at the level of measurement about which you have other research questions.

• "Step 1: perform a conventional confirmatory factor analysis on the sample total covariance"
• "Step 2: estimate between-group level variation"
• "Step 3: perform a factor analysis on the sample pooled-within covariance matrix"
• "Step 4: perform a factor analysis on the sample between-group covariance matrix"
• "Step 5: perform multilevel confirmatory factor analysis"

Im relatively new to SEM and CFA and like using the lavaan package very much. It appears the authors of this paper used MPlus

Is it possible to have this workflow in lavaan using R? I see in page two of the manual [2] that lavaan provides "support for hierarchical/multilevel datasets (multilevel cfa, multilevel sem)". I have not been able to fin any tutorial on it

The material you quoted is a bullet point under the text of what is "currently NOT available in lavaan".  But multilevel support is on its way.  It will not be implemented the Mplus way, though, but the GLLAMM way.   Regardless of whether you can use the same workflow, that 12-year-old advice is not necessarily the best to follow.  MSEM is a rapidly developing field, and we are still learning about the full implications of things like constraints across clusters and across levels of measurement.  Here are some more recent readings:

Stapleton, L. M., Yang, J. S., & Hancock, G. R. (2016). Construct meaning in multilevel settings. Journal of Educational and Behavioral Statistics, 41(5), 481–520. doi:10.3102/1076998616646200

Tay, L., Woo, S. E., & Vermunt, J. K. (2014). A conceptual and methodological framework for psychometric isomorphism validation of multilevel construct measures. Organizational Research Methods, 17(1), 77–106. doi:10.1177/1094428113517008

Terrence D. Jorgensen

Postdoctoral Researcher, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

UvA web page: http://www.uva.nl/profile/t.d.jorgensen