The reported fit measure "bic2"

235 views
Skip to first unread message

James Grace

unread,
Jan 1, 2020, 9:37:46 PM1/1/20
to lavaan
I have been unable to find documentation to explain which of the several alternative version of the Bayesian Information Criterion is being reported by lavaan under the label "bic2". Can anyone help me with this?

Thanks,
Jim Grace


Terrence Jorgensen

unread,
Jan 2, 2020, 8:28:13 AM1/2/20
to lavaan
bic2 is the sample-size adjusted BIC that Mplus reports, yet they never seem to recommend its use if you look at their discussion boards.  The formula simply replaces the actual sample size with 

(n* = (n + 2) / 24)

Terrence D. Jorgensen
Assistant Professor, Methods and Statistics
Research Institute for Child Development and Education, the University of Amsterdam
 

Jarrett Byrnes

unread,
Jan 2, 2020, 8:29:17 AM1/2/20
to lav...@googlegroups.com
Is there a reference for this metric?

On Jan 2, 2020, at 8:28 AM, Terrence Jorgensen <tjorge...@gmail.com> wrote:


--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/1ba1315a-bb16-4ea5-a351-091b7deb2430%40googlegroups.com.

Terrence Jorgensen

unread,
Jan 2, 2020, 8:46:29 AM1/2/20
to lavaan
Is there a reference for this metric?

I can't find one anywhere, not even in Bollen et al.'s (2014) review of many BICs: https://doi.org/10.1080/10705511.2014.856691

Bengt Muthén mentioned there wasn't much theoretical support for it in a message, so I'm not sure why they report it (and I'm sure Yves merely feels obligated to report it because many users expect to see what Mplus provides): http://www.statmodel.com/discussion/messages/11/10259.html?1503359571

James Grace

unread,
Jan 2, 2020, 12:36:48 PM1/2/20
to lavaan
There is actually a thorough treatment of it in. Lin et al. 2017. Selecting path models in SEM: A comparison of model selection criteria. Structural Equation Modeling 24: 855-869
There it is listed as ABIC. This is an excellent review and simulation study, best I have seen yet. Very complicated topic with an "it depends" answer to which one(s) are 'best'. Frustratingly, AICc is left out (a standard in biology), but CAIC is included.

Thanks to you both for the help. I guessed it was the sample-size adjusted one, but needed to be certain.
Jim

Terrence Jorgensen

unread,
Jan 3, 2020, 11:12:20 AM1/3/20
to lavaan
Frustratingly, AICc is left out (a standard in biology)

Probably because it was developed for univariate (generalized) linear models, not multivariate models, so not really applicable to SEM.  Still would have been nice to see its empirical performance.

Thanks for the citation!

James Grace

unread,
Jan 3, 2020, 1:06:55 PM1/3/20
to lav...@googlegroups.com
Terrence, I expect you are correct, because AICc is known to be undefined in multivariate models (and many models used in biology are multivariate). Dave Anderson (of Burnham and Anderson) has claimed in his "little book" that AICc is useful for multivariate models even though it is not proper. 

As a result of the influence of B&A in biology, biologists commonly use AICc in SEM studies and the AICcModavg package creates model comparison tables based on AICc using lavaan model objects. For this reason alone I would be very curious to see how it performs relative to AIC and CAIC in manifest (path) models like those studied in Lin et al., as well as in CFA models like Bollen et al. examined. 

--
You received this message because you are subscribed to a topic in the Google Groups "lavaan" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lavaan/RGkAbn7ixlQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lavaan+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lavaan/59204a8e-36b3-4a75-97e0-a9df9fcebc8a%40googlegroups.com.

Terrence Jorgensen

unread,
Aug 17, 2020, 4:30:11 AM8/17/20
to lavaan
Is there a reference for this metric?

Finally found a reference for sample-size adjusted BIC:

Sclove, S. L. (1987). Application of model-selection criteria to some problems in multivariate analysis. Psychometrika, 52, 333–343. https://doi.org/10.1007/BF02294360
 
Cited in a new publication:  

Lai, K. (2020). Using Information Criteria Under Missing Data: Full Information Maximum Likelihood Versus Two-Stage Estimation. Structural Equation Modeling, 1–14. https://doi.org/10.1080/10705511.2020.1780925

Joseph Bonito

unread,
Aug 17, 2020, 12:18:16 PM8/17/20
to lavaan
It's worth nothing that David Kenny recommends SABIC for GAPIM (group actor-partner independence models) model comparisons.  See Kenny and Garcia (2012).  And Terrance, would it make sense to relabel "bic2" as "sabic" in lavaan?  

Joe

Terrence Jorgensen

unread,
Aug 17, 2020, 4:35:20 PM8/17/20
to lavaan
would it make sense to relabel "bic2" as "sabic" in lavaan?  

That seems like a more informative label, but years of precedent might require keeping what other programs depend on.  Perhaps a more thorough documentation of the list of available fitMeasures() is in order.

Joseph Bonito

unread,
Aug 20, 2020, 5:09:19 PM8/20/20
to lavaan
Agree on both counts.

Joe
Reply all
Reply to author
Forward
0 new messages