fit measures in sem, generally

[Dr. Hans Hansen]

Dear lavaan-users,

today i have a more general question. It goes beyond lavaan, but since i feel that R users are keen to understand what happens under the hood of an analysis, i think it's suitable to be posted here.

So, what sense do you make out of the variety of fit indizes? I know the various "rules of thumb", like RMSEA should be less than 0.08 or 0.05, but are there any chances to figure out, where the model is misspecified and to what extend the misspecification matters? And: how much do you care about fit indizes in your analysis?

Looking forward to hear your thoughts,
Hans

[Sunthud Pornprasertmanit]

Hi Dr. Hans,

I think fit indices are one way to evaluate model fit. I know that many groups of researchers are trying to improve the way that fit indices are currently used or find the alternative methods for model fit evaluation. One example is Saris, Satorra, and van der Weld (2009) approach to find a model misspecification by modification indices and power to detect nontrivial misfits in modification indices. This is very interesting approach and the miPowerFit function is available in the semTools package. Addition fit indices are available in the moreFitIndices function in that package too.

For my personal perspective on fit indices, I think that it reflects the belief that hypothesized model do not replicate the model underlying the data and fit indices are designed to capture how well this hypothesized model approximates the relations among variables in the data. Regardless of good or bad model fit, I think we should look at the fit at parameter levels because a good model fit does not mean that the hypothesized model is corrected. Like philosophy of science, we should gather information of model fit from various sources to justify whether the hypothesized model should be the one.

Sunthud

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To post to this group, send email to lav...@googlegroups.com.
Visit this group at http://groups.google.com/group/lavaan?hl=en.
 
 

[Dr. Hans Hansen]

Thanks for your reply. However, today i experimented with different estimation methods and found out that these influence fit measures more than i thought of. I feel that i have really a hard time comparing all these parameters from different models, since their behaviour seems unpredictable to me.

Whats the best book which goes into the depth of fit-indizes and estimation methods but suitable for a non-mathematician?

Best, Hans

[yrosseel]

Hm. I'm not sure what the 'best' book is for this, but there is a
discussion of fit indices in Kline's book (although AFAIR not related to
estimators). And of course Bollen's 1989 book still contains a lot of
useful information.

Kline B (2011). Principles and Practice of Structural Equation Modeling.
The Guilford Press. (third edition)

Bollen KA (1989). Structural Equations with Latent Variables. John Wiley
& Sons.

Yves.

[Dr. Hans Hansen]

Thanks. I have the Kline book, the Bollen is rather expensive. I'll have a look at the library. Thanks, Hans

[Sunthud Pornprasertmanit]

I found this article is very useful. It goes each fit index in details:

West, S. G., Taylor, A. B., & Wu, W. (2012). Model fit and model selection in structural equation modeling. In R. H. Hoyle (Ed.), Handbook of Structural Equation Modeling. New York: Guilford.

Sunthud

On Sat, Nov 24, 2012 at 9:49 AM, Dr. Hans Hansen <bea...@gmx.de> wrote:

Thanks. I have the Kline book, the Bollen is rather expensive. I'll have a look at the library. Thanks, Hans

[yrosseel]

On 11/24/2012 05:20 PM, Sunthud Pornprasertmanit wrote:
> I found this article is very useful. It goes each fit index in details:
>

> West, S. G., Taylor, A. B., &**Wu, W. (2012). Model fit and model

> selection in structural equation modeling. In R. H. Hoyle (Ed.),

> /Handbook of Structural Equation Modeling./ New York: Guilford.

While we are on the topic of fit indices: the dev version has expanded
the list of available fit measures. See below. Any fit measures that
should be added?

Yves.


library(lavaan)
example(cfa)
fitMeasures(fit)
fmin chisq df pvalue
0.142 85.306 24.000 0.000
baseline.chisq baseline.df baseline.pvalue cfi
918.852 36.000 0.000 0.931
tli nnfi rfi nfi
0.896 0.896 0.861 0.907
pnfi ifi rni logl
0.605 0.931 0.931 -3737.745
unrestricted.logl npar aic bic
-3695.092 21.000 7517.490 7595.339
ntotal bic2 rmsea rmsea.ci.lower
301.000 7528.739 0.092 0.071
rmsea.ci.upper rmsea.pvalue rmr rmr_nomean
0.114 0.001 0.082 0.082
srmr srmr_nomean cn_05 cn_01
0.065 0.065 129.490 152.654
gfi agfi pgfi mfi
0.943 0.894 0.503 0.903
ecvi
0.423

or in a more verbose (and unfinished) from:

> lavaan:::print.fit.measures(fitMeasures(fit))
Model test baseline model:

Minimum Function Test Statistic 918.852
Degrees of freedom 36
P-value 0.000

Full model versus baseline model:

Comparative Fit Index (CFI) 0.931
Tucker-Lewis Index (TLI) 0.896
Bentler-Bonett Non-normed Fit Index (NNFI) 0.896
Bentler-Bonett Normed Fit Index (NFI) 0.907
Parsimony Normed Fit Index (PNFI) 0.605
Bollen's Relative Fit Index (RFI) 0.861
Bollen's Incremental Fit Index (IFI) 0.931
Relative Noncentrality Index (RNI) 0.931

Loglikelihood and Information Criteria:

Loglikelihood user model (H0) -3737.745
Loglikelihood unrestricted model (H1) -3695.092

Number of free parameters 21
Akaike (AIC) 7517.490
Bayesian (BIC) 7595.339
Sample-size adjusted Bayesian (BIC) 7528.739

Root Mean Square Error of Approximation:

RMSEA 0.092
90 Percent Confidence Interval 0.071 0.114
P-value RMSEA <= 0.05 0.001

Standardized Root Mean Square Residual:

RMR 0.082
RMR (No Mean) 0.082
SRMR 0.065
SRMR (No Mean) 0.065

Other Fit Indices:

Hoelter Critical N (CN) alpha=0.05 129.490
Hoelter Critical N (CN) alpha=0.01 152.654

Goodness of Fit Index (GFI) 0.943
Adjusted Goodness of Fit Index (AGFI) 0.894
Parsimony Goodness of Fit Index (PGFI) 0.503

McDonald Fit Index (MFI) 0.903

Expected Cross-Validation Index (ECVI) 0.423

[Christopher M. Conway]

Hu and Bentler 1999 is the gold standard on this, based on extensive simulation. Basically, you want CFI, SRMR, and RMSEA. If I recall correctly, they don't really address the information theory based measures like AIC.

Hu, L., and Bentler, P. M. 1999. “Cutoff Criteria for Fit Indexes in Covariance Structure Analysis: Conventional Criteria Versus New Alternatives,” Structural Equation Modeling (6:1), pp. 1.

[Ruben Arslan]

I recently needed rmsea of the baseline model to discuss a poor incremental fit index, but Sunthud already put it in semTools.

Best, Ruben

--
Ruben

[yrosseel]

On 11/25/2012 12:22 PM, Ruben Arslan wrote:
> I recently needed rmsea of the baseline model to discuss a poor
> incremental fit index, but Sunthud already put it in semTools.

That reminds me I should make the internal function
'independence.model.fit' public:

example(cfa)
NullModel <- lavaan:::independence.model.fit(fit)
fitMeasures(NullModel)

so you can get all fit measures for the NullModel.

Yves.

[Ruben Arslan]

a better solution even.

[Sunthud Pornprasertmanit]

I think the addition of WRMR (categorical-indicator version of SRMR provided in Mplus) would be nice. Usually I do not use it but I think there are many people I know using it.

As an unrelated note, the moreFitIndices function in semTools have plenty of the obsolete and/or unpopular fit indices. Please feel free move some of them into the lavaan package.

Sunthud

[yrosseel]

On 11/25/2012 05:58 PM, Sunthud Pornprasertmanit wrote:
> I think the addition of WRMR (categorical-indicator version of SRMR
> provided in Mplus) would be nice. Usually I do not use it but I think
> there are many people I know using it.

Indeed. Good suggestion.

> As an unrelated note, the moreFitIndices function in semTools have
> plenty of the obsolete and/or unpopular fit indices. Please feel free
> move some of them into the lavaan package.

I'll have a look a them...

Yves.

[yrosseel]

On 11/25/2012 05:58 PM, Sunthud Pornprasertmanit wrote:

> I think the addition of WRMR (categorical-indicator version of SRMR
> provided in Mplus) would be nice.

The WRMR is now (dev lavaan 0.5-16) added to the output of
fitMeasures(). If mimic="Mplus", it will also been shown in the
summary(*, fit.measures=TRUE) output.

Yves.