Multilevel path analysis - Comparision between lavaan and Mplus
Kauyer Lor
Dear Lavaan Group,
I'm a graduate student conducting a multilevel path analysis in lavaan and Mplus. When comparing the results between the two programs, the results are similar but not identical. It would be greatly appreciated if you had any input on whether the differences:
(a) are likely to be simply the typical/ignorable differences between different software or
(b) might represent unintended model discrepancies or other issues in our code.
We have searched in this discussion group and found several questions involving comparisons with Mplus but couldn’t find a direct answer to this question related to “how much” of a difference in results should be a concern, particularly for multilevel path analysis.
Quick background: This is from a daily diary study; we are interested in within-person relationships; we used person-mean centering; the model has 4 predictors, 3 mediators, and 2 outcomes; we used ML estimation for both; and we used LISTWISE = ON in Mplus (our understanding is this is what lavaan uses with multilevel analyses).
We have checked that the datasets are the same and our review of the two sets of results indicates many similarities (e.g., for both: df = 7, baseline model df = 37, free parameters = 50, number of observations = 856, number of clusters = 160).
Below are a few quick examples. More detailed input and side-by-side results are attached for easier readability. Thank you in advance for any insights!
LAVAAN OUTPUT:
MPLUS OUTPUT:
Estimator ML
Model Fit Test Statistic 0.223
Degrees of freedom 7
P-value (Chi-square) 1.000
Model test baseline model:
Minimum Function Test Statistic 571.971
Degrees of freedom 37
P-value 0.000
Chi-Square Test of Model Fit
Value 0.001
Degrees of Freedom 7
P-Value 1.0000
Chi-Square Test of Model Fit for the Baseline Model
Value 568.052
Degrees of Freedom 37
P-Value 0.0000
Level 1 [within]:
Regressions:
Estimate Std.Err z-value P(>|z|)
intmot ~
autonomy 0.348 0.035 9.815 0.000
taskvar 0.030 0.029 1.054 0.292
inter -0.087 0.028 -3.114 0.002
jobcomp -0.107 0.030 -3.601 0.000
odidist1 ~
autonomy -0.038 0.109 -0.346 0.729
taskvar 0.073 0.088 0.824 0.410
inter -0.162 0.086 -1.882 0.060
jobcomp -0.120 0.091 -1.318 0.188
odedist1 ~
autonomy -0.043 0.099 -0.433 0.665
taskvar 0.266 0.080 3.310 0.001
inter 0.132 0.079 1.685 0.092
jobcomp 0.240 0.083 2.884 0.004
Estimate S.E. Est./S.E. P-Value
Within Level
INTMOT ON
AUTONOMY 0.348 0.035 9.794 0.000
TASKVAR 0.030 0.029 1.028 0.304
INTER -0.092 0.028 -3.265 0.001
JOBCOMP -0.104 0.030 -3.499 0.000
ODIDIST1 ON
AUTONOMY -0.045 0.108 -0.412 0.681
TASKVAR 0.066 0.088 0.748 0.455
INTER -0.165 0.086 -1.923 0.054
JOBCOMP -0.118 0.091 -1.292 0.196
ODEDIST1 ON
AUTONOMY -0.038 0.099 -0.382 0.702
TASKVAR 0.264 0.080 3.301 0.001
INTER 0.127 0.078 1.626 0.104
JOBCOMP 0.238 0.083 2.861 0.004
Level 2 [NewID]:
Covariances:
Estimate Std.Err z-value P(>|z|)
.odgoals1 ~~
.agoals2 132.208 36.693 3.603 0.000
Intercepts:
Estimate Std.Err z-value P(>|z|)
.odgoals1 83.883 1.209 69.355 0.000
.agoals2 55.268 2.291 24.128 0.000
Variances:
Estimate Std.Err z-value P(>|z|)
.odgoals1 166.178 25.982 6.396 0.000
.agoals2 572.785 90.266 6.345 0.000
Between Level
ODGOALS1 WITH
AGOALS2 131.897 36.916 3.573 0.000
Means
ODGOALS1 83.916 1.211 69.288 0.000
AGOALS2 55.348 2.300 24.061 0.000
Variances
ODGOALS1 166.795 26.093 6.392 0.000
AGOALS2 579.439 91.236 6.351 0.000
Thanks,
Kauyer
Terrence Jorgensen
When comparing the results between the two programs, the results are similar but not identical.
- accelerated EM (EMA; the default)
- regular EM
- directly maximizing the observed-data log-likelihood (ODLL)
- https://www.statmodel.com/HTML_UG/chapter16V8.htm