Why do I get a different SRMR when I use estimator = 'MLR' vs estimator = 'ML' ?

[Amonet]

I was always under the impression that the results of the SRMR are unchanged whether I use MLR for robust estimates, or ML for unadjusted estimates. Now I noticed that the SRMR differs for the two options. Of perhaps important note: I am using lavaan version 0.5-23-1097 (i.e. last version of 0.5), because I experienced discrepancies between earlier results obtained via version 0.5 vs. results obtained via version 0.6+. 

Largely oversimplified, the SRMR can be seen as the average difference between the observed covariances (i.e. the data) and the model implied covariances. Shouldn't this number, the SRMR, be the same whether I use MLR or ML? Or does it also use the scaling parameter to adjust the SRMR? 

Thanks,

Amonet 

[Terrence Jorgensen]

I was always under the impression that the results of the SRMR are unchanged whether I use MLR for robust estimates, or ML for unadjusted estimates.

The point estimates should be identical, so SRMR should not be affected.  However, MLR used to require the option meanstructure = TRUE, which is no longer the case in version 0.6-2 (not sure about 0.6-1).  So in previous versions, if you did not set meanstructure = TRUE (or it was not trigged by default, e.g., for a multigroup model or setting missing = "fiml"), then SRMR would include mean residuals in the MLR model but not the ML model.  That could be a reason for seeing differences.  There are actually a couple ways to calculate SRMR, and you can see the range of them (with and without including mean residuals) from the fitMeasures() output.

You can also check lavInspect(fit, "options") output to see if the other options are the same when switching to MLR (should change the se and test options)

Now I noticed that the SRMR differs for the two options. Of perhaps important note: I am using lavaan version 0.5-23-1097 (i.e. last version of 0.5), because I experienced discrepancies between earlier results obtained via version 0.5 vs. results obtained via version 0.6+. 

I am using the development version (0.6-2) and the ?cfa help-page example produces the same SRMRs with ML and MLR.  

example(cfa)
fitm <- update(fit, meanstructure = TRUE)
fitr <- update(fitm, estimator = "mlr")
fitMeasures(fitm)
fitMeasures(fitr)

If you don't (with the same other options set), could you provide a reproducible example?

Terrence D. Jorgensen

Postdoctoral Researcher, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

UvA web page: http://www.uva.nl/profile/t.d.jorgensen

 

[Amonet]

Hi Terrence, 

Thanks once more for helping me. 

You are right, it was because of meanstructure = TRUE when I used MLR, which was not on with ML as I did not specify this. 

I also checked it with the development version, there it worked properly as well as you suggested.

Thanks

Amonet 

[Amonet]

Hi Terrence,

Do you know where I can find the formula's behind the different ways of calculating the SRMR, and preferably for the longitudinal case? I am particularly interested in how the formula changes when the means are incorporated in the calculation. 

As far as I know, the following formula is used for the SRMR without the mean vector: 

where s_ij = observed covariances, sigma_ij = the model implied covariances, s_ii and s_jj are the observed standard deviations for the i'th and j'th manifest variable, and k = the number of manifest variables. 

(got this formula from: https://stats.stackexchange.com/questions/337670/formula-for-standardized-root-mean-square-residual-srmr-in-longitudinal-latent/341719?noredirect=1#comment666817_341719 )

 

Thanks,

Amonet

[Terrence Jorgensen]

Do you know where I can find the formula's behind the different ways of calculating the SRMR, and preferably for the longitudinal case?

Longitudinal has nothing to do with it.  The formula is applied to all the observed and model-implied covariance matrices (in all groups, if applicable) of observed variables, regardless of whether any of those happen to be repeated measures.

I am particularly interested in how the formula changes when the means are incorporated in the calculation.

Instead of averaging only the residuals from the observed and model-implied covariance matrices, the residuals between observed and model-implied mean vectors are also summed (and the sum is divided by the number of all residuals, resulting in a mean).  A variable's mean (residual) can be stanrdardized merely by dividing by the variable's SD.

As far as I know, the following formula is used for the SRMR without the mean vector: 

where s_ij = observed covariances, sigma_ij = the model implied covariances, s_ii and s_jj are the observed standard deviations for the i'th and j'th manifest variable, and k = the number of manifest variables. 

(got this formula from: https://stats.stackexchange.com/questions/337670/formula-for-standardized-root-mean-square-residual-srmr-in-longitudinal-latent/341719?noredirect=1#comment666817_341719 )

Yes, this is the formula applied by Jöreskog & Sorbom in the LISREL program, and by Peter Bentler is the EQS program.  You can find that formula in those software manuals (e.g., formula 14.9 of the EQS manual) or in papers such as Hu & Bentler (1998, Table 1).  As the formula indicates, these correlation residuals are calculated by standardizing the covariance residuals, which are the raw differences between observed and predicted sample moments.  So I call this the "subtract, then standardize" formula.

Ken Bollen's (1989) Structural Equations with Latent Variables, often called the "SEM bible", defines SRMR using correlation residuals instead of covariance residuals.  The observed and model-implied covariance matrices are each standardized, and SRMR is calculated as the average difference between correlations, so "standardize, then subtract".  

Although both formulas yield equivalent residuals in most situations, Bollen's formula ignores misspecification in (residual) variances.  Because each matrix is standardized, the diagonal of both correlation matrices will always be 1, resulting in correlations residuals == 0 by definition.  Such situations occur when homogeneity of (residual) variances is imposed across groups or repeated measures, such as latent curve models or strictly invariant CFAs.  Interestingly, Muthén's (2018, see formulas 2 & 3, which include mean structure) Mplus package uses the "standardize, then subtract" method for off-diagonal correlation residuals and the mean vector, but "subtract, then standardize" for the variances. 

[Amonet]

Thank you for the great help!