The reported fit measure "bic2"

[James Grace]

I have been unable to find documentation to explain which of the several alternative version of the Bayesian Information Criterion is being reported by lavaan under the label "bic2". Can anyone help me with this?

Thanks,

Jim Grace

[Terrence Jorgensen]

bic2 is the sample-size adjusted BIC that Mplus reports, yet they never seem to recommend its use if you look at their discussion boards.  The formula simply replaces the actual sample size with 

(n* = (n + 2) / 24)

Terrence D. Jorgensen

Assistant Professor, Methods and Statistics

Research Institute for Child Development and Education, the University of Amsterdam

http://www.uva.nl/profile/t.d.jorgensen

 

[Jarrett Byrnes]

Is there a reference for this metric?

On Jan 2, 2020, at 8:28 AM, Terrence Jorgensen <tjorge...@gmail.com> wrote:



--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/1ba1315a-bb16-4ea5-a351-091b7deb2430%40googlegroups.com.

[Terrence Jorgensen]

Is there a reference for this metric?

I can't find one anywhere, not even in Bollen et al.'s (2014) review of many BICs: https://doi.org/10.1080/10705511.2014.856691

Bengt Muthén mentioned there wasn't much theoretical support for it in a message, so I'm not sure why they report it (and I'm sure Yves merely feels obligated to report it because many users expect to see what Mplus provides): http://www.statmodel.com/discussion/messages/11/10259.html?1503359571

[James Grace]

There is actually a thorough treatment of it in. Lin et al. 2017. Selecting path models in SEM: A comparison of model selection criteria. Structural Equation Modeling 24: 855-869

There it is listed as ABIC. This is an excellent review and simulation study, best I have seen yet. Very complicated topic with an "it depends" answer to which one(s) are 'best'. Frustratingly, AICc is left out (a standard in biology), but CAIC is included.

Thanks to you both for the help. I guessed it was the sample-size adjusted one, but needed to be certain.

Jim

[Terrence Jorgensen]

Frustratingly, AICc is left out (a standard in biology)

Probably because it was developed for univariate (generalized) linear models, not multivariate models, so not really applicable to SEM.  Still would have been nice to see its empirical performance.

Thanks for the citation!

[James Grace]

Terrence, I expect you are correct, because AICc is known to be undefined in multivariate models (and many models used in biology are multivariate). Dave Anderson (of Burnham and Anderson) has claimed in his "little book" that AICc is useful for multivariate models even though it is not proper. 

As a result of the influence of B&A in biology, biologists commonly use AICc in SEM studies and the AICcModavg package creates model comparison tables based on AICc using lavaan model objects. For this reason alone I would be very curious to see how it performs relative to AIC and CAIC in manifest (path) models like those studied in Lin et al., as well as in CFA models like Bollen et al. examined. 

--
You received this message because you are subscribed to a topic in the Google Groups "lavaan" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lavaan/RGkAbn7ixlQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/59204a8e-36b3-4a75-97e0-a9df9fcebc8a%40googlegroups.com.

[Terrence Jorgensen]

Is there a reference for this metric?

Finally found a reference for sample-size adjusted BIC:

Sclove, S. L. (1987). Application of model-selection criteria to some problems in multivariate analysis. Psychometrika, 52, 333–343. https://doi.org/10.1007/BF02294360

 

Cited in a new publication:  

Lai, K. (2020). Using Information Criteria Under Missing Data: Full Information Maximum Likelihood Versus Two-Stage Estimation. Structural Equation Modeling, 1–14. https://doi.org/10.1080/10705511.2020.1780925

[Joseph Bonito]

It's worth nothing that David Kenny recommends SABIC for GAPIM (group actor-partner independence models) model comparisons.  See Kenny and Garcia (2012).  And Terrance, would it make sense to relabel "bic2" as "sabic" in lavaan?  

Joe

[Terrence Jorgensen]

would it make sense to relabel "bic2" as "sabic" in lavaan?  

That seems like a more informative label, but years of precedent might require keeping what other programs depend on.  Perhaps a more thorough documentation of the list of available fitMeasures() is in order.

[Joseph Bonito]

Agree on both counts.

Joe