Skip to first unread message
Manolo Cabran
Aug 14, 2018, 2:48:55 AM8/14/18
to lavaan
Hi,
this is a copy of what I posted on StackOverflow, for which, as of today 14 August 2018, I have not received any answer.
I am attending a class on Confirmatory Factor Analysis `CFA' with `MPlus`; however, I am very keen in learning how to do it with `R-lavaan`. I have run a model with `MPLUS` and the same model with `R lavaan`. The model has three latent variables and ten indicators. All parameters are set as free. The model estimation terminated normally in both `R` and `MPlus`. The `R` code, the `MPlus` input file and the `.dat` files are available in this GitHub repository.
When I look at the outputs, everything looks the same. In most cases, the values are the same; when they are not, they differ to the second or third decimal. This is true for:
- Number of groups
- Number of observations
- Number of dependent, independent and latent variables
- Number of missing data patterns
- Number of Free Parameters
- Loglikelihood
- Information Criteria (Chi-Square Test of Model Fit,
RMSEA, CFI/TLI, Chi-Square Test of Model Fit for the Baseline Model,
and SRMR)
- All model results (e.g. latent and independent variables,
both unstandardized and fully standardized)
- R-SQUARE
One important difference is the estimator: in `MPlus` is `ML`, in `R` is `FIML`. I first started with `ML` in both, but in `R` I got all estimates different from `MPlus`. I guess this is due to the different way `R` and `MPlus` handle `NAs`.
However, when I look at the modification indexes, `MPlus` has many more indexes than `R` and the values are significantly different. If I fix the minimum value of the modification indexes in both `R` and MPlus at 10, in `R` I get 8 indexes, in MPlus I get 29.
This is what I get from MPlus:
ON/BY Statements
IPEQOPT ON TRACO /
TRACO BY IPEQOPT 25.739 -0.337 -0.210 -0.226
IMPENV ON TRACO /
TRACO BY IMPENV 33.457 0.446 0.278 0.295
IMPENV ON ATTI /
ATTI BY IMPENV 24.846 -0.203 -0.126 -0.133
ON Statements
TRACO ON IPEQOPT 22.308 -0.131 -0.210 -0.195
TRACO ON IMPENV 19.577 0.168 0.270 0.254
ATTI ON IMPENV 19.133 -0.117 -0.189 -0.178
UNIV ON IPEQOPT 16.226 0.133 0.292 0.272
IMPTRAD ON IMSMETN 10.895 -0.112 -0.112 -0.070
IMPTRAD ON IPUDRST 12.675 -0.106 -0.106 -0.081
IMPTRAD ON IMPENV 31.976 0.176 0.176 0.133
IPEQOPT ON IMPTRAD 22.316 -0.089 -0.089 -0.120
IPEQOPT ON IPBHPRP 15.872 -0.098 -0.098 -0.119
IPUDRST ON IMPTRAD 15.505 -0.076 -0.076 -0.100
IPUDRST ON IMDFETN 10.021 0.090 0.090 0.077
IPUDRST ON IMPENV 16.234 -0.197 -0.197 -0.194
IMPENV ON IMPTRAD 68.801 0.161 0.161 0.214
IMPENV ON IMSMETN 15.721 -0.114 -0.114 -0.095
IMPENV ON IMDFETN 23.385 -0.144 -0.144 -0.124
IMPENV ON IMPCNTR 12.216 -0.091 -0.091 -0.083
IMPENV ON IPUDRST 16.231 -0.170 -0.170 -0.172
WITH Statements
IPEQOPT WITH TRACO 22.309 -0.086 -0.138 -0.170
IPEQOPT WITH UNIV 16.228 0.088 0.193 0.237
IPEQOPT WITH IMPTRAD 12.171 -0.080 -0.080 -0.094
IPUDRST WITH IPMODST 11.775 0.085 0.085 0.091
IPUDRST WITH IMPTRAD 16.337 -0.094 -0.094 -0.110
IMPENV WITH TRACO 19.576 0.098 0.157 0.206
IMPENV WITH ATTI 19.135 -0.068 -0.110 -0.144
IMPENV WITH IMPTRAD 52.625 0.166 0.166 0.209
IMPENV WITH IPUDRST 16.230 -0.114 -0.114 -0.183
And this is what I get from `R`:
modificationindices(cfa4, minimum.value = 10)
> modificationindices(cfa4, minimum.value = 10)
lhs op rhs mi epc sepc.lv sepc.all sepc.nox
43 TraCo =~ ipeqopt 25.742 -0.337 -0.210 -0.226 -0.226
45 TraCo =~ impenv 33.458 0.446 0.278 0.295 0.295
52 Atti =~ impenv 24.847 -0.203 -0.126 -0.133 -0.133
67 ipmodst ~~ ipudrst 11.776 0.085 0.085 0.091 0.091
74 imptrad ~~ ipeqopt 12.171 -0.080 -0.080 -0.094 -0.094
75 imptrad ~~ ipudrst 16.336 -0.094 -0.094 -0.110 -0.110
76 imptrad ~~ impenv 52.626 0.166 0.166 0.209 0.209
104 ipudrst ~~ impenv 16.227 -0.114 -0.114 -0.183 -0.183
If I lower the minimum value in `R` to 4, I have 21 indexes
modificationindices(cfa4, minimum.value = 4)
> modificationindices(cfa4, minimum.value = 4)
lhs op rhs mi epc sepc.lv sepc.all sepc.nox
43 TraCo =~ ipeqopt 25.742 -0.337 -0.210 -0.226 -0.226
45 TraCo =~ impenv 33.458 0.446 0.278 0.295 0.295
47 Atti =~ imptrad 6.279 -0.114 -0.071 -0.056 -0.056
49 Atti =~ ipbhprp 4.088 0.085 0.052 0.046 0.046
50 Atti =~ ipeqopt 6.187 0.094 0.058 0.062 0.062
51 Atti =~ ipudrst 9.056 0.117 0.072 0.076 0.076
52 Atti =~ impenv 24.847 -0.203 -0.126 -0.133 -0.133
57 Univ =~ imsmetn 7.344 -0.094 -0.043 -0.054 -0.054
59 Univ =~ impcntr 4.496 0.084 0.038 0.045 0.045
65 ipmodst ~~ impcntr 7.733 0.046 0.046 0.071 0.071
67 ipmodst ~~ ipudrst 11.776 0.085 0.085 0.091 0.091
71 imptrad ~~ imsmetn 5.735 -0.032 -0.032 -0.063 -0.063
74 imptrad ~~ ipeqopt 12.171 -0.080 -0.080 -0.094 -0.094
75 imptrad ~~ ipudrst 16.336 -0.094 -0.094 -0.110 -0.110
76 imptrad ~~ impenv 52.626 0.166 0.166 0.209 0.209
86 ipbhprp ~~ impcntr 6.967 -0.034 -0.034 -0.079 -0.079
87 ipbhprp ~~ ipeqopt 4.281 -0.041 -0.041 -0.067 -0.067
97 imdfetn ~~ ipudrst 5.444 0.023 0.023 0.114 0.114
99 impcntr ~~ ipeqopt 4.448 0.025 0.025 0.055 0.055
102 ipeqopt ~~ ipudrst 8.411 0.068 0.068 0.102 0.102
104 ipudrst ~~ impenv 16.227 -0.114 -0.114 -0.183 -0.183
Of these 21, only three values are the same (to the second/third decimal) in both `R-lavaan` and MPlus outputs, and these are the ON/BY STATEMENTS in MPlus. Of the 7 that are in common in both `R` and MPlus, only one `IPUDRST ON IMPENV` has the same value (again, to the second/third decimal), the other six `TRACO ON IPEQOPT, TRACO ON IMPENV, ATTI ON IMPENV, IMPTRAD ON IMSMETN, IMPTRAD ON IPUDRST, IMPTRAD ON IMPENV` are completely off. The remaining are not even in common.
Any suggestion on what could be actually be wrong in my code? Can the two algorithms estimating modification indexes be so different in `R` and in `MPlus` to give such output? What else?
Thanks
Manolo
*Update 11 August 2018*
I encountered the same problem when testing for invariance. When I tested for invariances in `R`, I got the same results as in `MPlus`. In both `R` and `MPlus`, the modification indexes for configural and metric invariance are under 10, so they are not reported. However, for scalar invariance, I have ten modification indexes in `MPlus` that are above the value of ten, while in `R` I do not have any modification index.
Terrence Jorgensen
Aug 14, 2018, 6:37:23 AM8/14/18
to lavaan
One important difference is the estimator: in `MPlus` is `ML`, in `R` is `FIML`.
You are confusing the estimator with the method for handling missing data. And you mean `lavaan`, which is a specific software package that is written for the R environment. There are other SEM packages for R, including `sem`, `OpenMx`, and `nlsem`.
I first started with `ML` in both, but in `R` I got all estimates different from `MPlus`. I guess this is due to the different way `R` and `MPlus` handle `NAs`.
Mplus uses FIML to handle missing values (for continuous data only) by default, whereas in lavaan you need to set missing = "fiml"
`MPlus` has many more indexes than `R`
I think lavaan checks whether a latent variable is exogenous or endogenous, and it only calculates modification indices that would make sense within the context of the fitted model (i.e., it will not suggest a factor should be regressed on another variable if it is exogenous). Mplus suggests lots of extra parameters that make no sense in a CFA, like regressing a factor on an indicator of another factor.
and the values are significantly different... Can the two algorithms estimating modification indexes be so different in `R` and in `MPlus` to give such output?
The algorithm for calculating Lagrange multipliers (modification indices are a special case of a 1-df statistics) should be the same, but the optimizers are different. Mplus uses their own, and lavaan uses ones available in R (e.g., ?nlminb). Lagrange multipliers are calculated from the first and second derivatives of the parameter estimates, so they can differ to the degree that the different optimizers find different solutions.
I encountered the same problem when testing for invariance. When I tested for invariances in `R`, I got the same results as in `MPlus`. In both `R` and `MPlus`, the modification indexes for configural and metric invariance are under 10, so they are not reported. However, for scalar invariance, I have ten modification indexes in `MPlus` that are above the value of ten, while in `R` I do not have any modification index.
Mplus provides a 1-df modification index for each equality-constrained parameter, which are collectively redundant because a single equality constraint on 2 parameters only results in 1 df. lavaan does not provide modification indices for any estimated parameters, only for fixed parameters. But the lavTestScore() function provides a way to test whether equality constraints should be released.
Terrence D. Jorgensen
Postdoctoral Researcher, Methods and Statistics
Research Institute for Child Development and Education, the University of Amsterdam
UvA web page: http://www.uva.nl/profile/t.d.jorgensen
Reply all
Reply to author
Forward
0 new messages