AIC model selection with lavaan

[Sandpiper]

Hello.

I'm trying to test the fit of path models which have differing number of paths between 2 and 4. When I use the usual approach of significant testing, the p-values end up being '0' in all case. I suspect that it's from somewhat large sample size (N=~3000) that I'm working with. So I'm trying to use Information Theoretic approach comparing AIC values. I'm using 'aictab.lavaan' function to get AIC, AICc estimates. However, the resulting AICc values are something that I've never seen before.. I know that the absolute value of AIC means nothing. But in this case, the delta-AIC values in a model set varies from 0 to 77000... which is weird. Thus I'm not convinced that this is the right way to go. Is there anyone who's comparing path models using AIC who can give me some insights on this issue?

Thank you very much for your attention and help in advance!

[Seongho Bae]

Hi, Sandpiper,

Can you upload reproducible file here if that isn't sensitive data?

--

Seongho Bae

2015년 3월 13일 금요일 오전 4시 31분 56초 UTC+9, Sandpiper 님의 말:

[yrosseel]

On 03/12/2015 08:31 PM, Sandpiper wrote:
> using 'aictab.lavaan'

This is not a function from the lavaan package. If you request the
regular AIC values by using

AIC(fit)

do the results match the output of aictab.lavaan?

Yves.

[Sandpiper]

Yes, the AICc values are the same using the two commends.. 

[Jarrett Byrnes]

Hello! I wrote the aictab.lavaan function (if you’re using what I think you are using) - while there may be some differences in absolute value, this is due to calculation of the value of AIC. It’s only different by an additive constant, however. 

What package did you get it from? Or website? If there’s a problem, I should fix it!

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To post to this group, send email to lav...@googlegroups.com.
Visit this group at http://groups.google.com/group/lavaan.
For more options, visit https://groups.google.com/d/optout.

[Sandpiper]

Hello! Thank you for your attention. I should first say thank you for writing this script! I got the attached script from a friend who attended a SEM workshop in Boston. I hope it's ok for me to use it.. Values come out practically the same, only differ at 0.01. My questions is with this huge delta-AICc values that I've not seen with any other dataset or papers... It is very likely from my own fault of using it wrong. I attached part of my dataset (lavaan_sample.csv) and simple code (lavaan_sample.r) as well. My dataset is no where close to meet the multinormality, and I was hoping to escape that issue with using AIC model selection instead... Is this acceptable approach? Or should I not use SEM as a whole?

[yrosseel]

On 03/16/2015 11:29 PM, Sandpiper wrote:
> My questions
> is with this huge delta-AICc values that I've not seen with any other
> dataset or papers...

They seem fine to me.

The huge values are due to the fact that you are seeing the AIC values
based on the 'joint' distribution of all variables. Perhaps you are more
used to see the AIC values based on the Y variables only (ignoring X)?

Yves.

[Jarrett Byrnes]

Well, remember that the multivariate normality violation is still going to impact your likelihood. A few ideas:

1) A big delta-AIC is definitely possible. This is due to the difference between big multi-variate hypotheses, rather than a hypothesis about a single univariate model. Those differences in fit add up, as it were.

2) If you’re seriously worried about multivariate non-normality, even after transformation, if you have an observed variable only model, why not use equation-level estimation techniques (see Jim Grace & Bill Shipley’s work on the topic and Jon Lefcheck’s piecewiseSEM package - which your friend from the workshop should be quite familiar with!) You can use glms there instead of linear models, which might be more appropriate for the error distributions you are working with. You can use this to also calculate an AIC a la Shipley’s recent Ecology paper.

<AICc.lavaan.fnc><lavaan_sample.r><lavaan_sample.csv>

[Sandpiper]

Thank you so much for the very helpful advice! After receiving your comments, I used piecewise SEM and I think it works beautifully.. Now I have sensible results. I found an easy explanation on the piecewise SEM in the following blog as well. 

http://jonlefcheck.net/2014/07/06/piecewise-structural-equation-modeling-in-ecological-research/

[MARIO GARRIDO ESCUDERO]

Dear Jarret,

I want to recover this issue from years ago.

I use your script aictab.lavaan to ordered 3 path analyses (no latent variables) according to the AICc and compute the wi of the three competing models. Thanks, it is really helpful.

I have some questions:

1. Can I use bictab have to order my path analyses according to BIC? I checked and the BIC values from bictab and from BIC in lavaan are the same, so I guess so, am I right? also the Weights are usable for path analyses?

2. AICc values from aictab.lavaan and from aictab are different, which looks logic for me. In additio, aictab.lavaan give me AICc results, is there something I can get AIC values and not the corrected AICc version for models in lavaan?

Thanks!

El martes, 24 de marzo de 2015, 16:56:09 (UTC+1), Jarrett Byrnes escribió:

Well, remember that the multivariate normality violation is still going to impact your likelihood. A few ideas:

1) A big delta-AIC is definitely possible. This is due to the difference between big multi-variate hypotheses, rather than a hypothesis about a single univariate model. Those differences in fit add up, as it were.

2) If you’re seriously worried about multivariate non-normality, even after transformation, if you have an observed variable only model, why not use equation-level estimation techniques (see Jim Grace & Bill Shipley’s work on the topic and Jon Lefcheck’s piecewiseSEM package - which your friend from the workshop should be quite familiar with!) You can use glms there instead of linear models, which might be more appropriate for the error distributions you are working with. You can use this to also calculate an AIC a la Shipley’s recent Ecology paper.

On Mar 16, 2015, at 6:29 PM, Sandpiper <eunb...@gmail.com> wrote:

Hello! Thank you for your attention. I should first say thank you for writing this script! I got the attached script from a friend who attended a SEM workshop in Boston. I hope it's ok for me to use it.. Values come out practically the same, only differ at 0.01. My questions is with this huge delta-AICc values that I've not seen with any other dataset or papers... It is very likely from my own fault of using it wrong. I attached part of my dataset (lavaan_sample.csv) and simple code (lavaan_sample.r) as well. My dataset is no where close to meet the multinormality, and I was hoping to escape that issue with using AIC model selection instead... Is this acceptable approach? Or should I not use SEM as a whole?

On Monday, March 16, 2015 at 4:53:56 PM UTC-5, Jarrett Byrnes wrote:

Hello! I wrote the aictab.lavaan function (if you’re using what I think you are using) - while there may be some differences in absolute value, this is due to calculation of the value of AIC. It’s only different by an additive constant, however. 

What package did you get it from? Or website? If there’s a problem, I should fix it!

On Mar 16, 2015, at 5:49 PM, Sandpiper <eunbi...@gmail.com> wrote:

Yes, the AICc values are the same using the two commends.. 

On Friday, March 13, 2015 at 8:33:09 AM UTC-5, yrosseel wrote:

On 03/12/2015 08:31 PM, Sandpiper wrote:
> using 'aictab.lavaan'

This is not a function from the lavaan package. If you request the
regular AIC values by using

AIC(fit)

do the results match the output of aictab.lavaan?

Yves.

--
You received this message because you are subscribed to the Google Groups "lavaan" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lavaan+un...@googlegroups.com.
To post to this group, send email to lav...@googlegroups.com.
Visit this group at http://groups.google.com/group/lavaan.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "lavaan" group.

To unsubscribe from this group and stop receiving emails from it, send an email to lav...@googlegroups.com.

[MARIO GARRIDO ESCUDERO]

Hi, 

my apologies, now I understand the code, for any reason I did not catch previously.

Answering myself, if helps anyone: with aictab.lavaan can be computed AIC, CAIC, QAIC and cQAIC 

To do analagous computation and tables for BIC in lavaan, a script must be done, probably based on useBIC function. 

Thanks