General questions of AIC

[Camille Voisin]

Hi,

I am a new user of dadi, and I have really general questions about AIC comparison.

I tried to find on the forum the answer, but not exactly what I am looking for. 

1/Can I compare the AIC of 2 island models if these 2 models are computed with different directions but same populations (I mean pop1 = a, pop2 = b and then pop1 = b, pop2 = a)?

2/Is there an AIC value that completely rejects all models (more than 1000 for example) or I keep the best one whatever the value is?

3/Is there a minimum AIC difference between 2 models to say that one is better than the other (like more than 1, 10, 100…)?

Thanks a lot for your return.

Best,

Camille

[Ryan Gutenkunst]

Hello Camille,

1) If the data is the same, then likelihoods will be comparable.

2) This is a general AIC question, without a good answer. Note that dadi computes a composite likelihood if the data are linked, which will bias AIC analyses toward more complex models. In principle, there are corrections for this, but I’ve never seen them used.

3) In the composite case, I cannot offer any guidance here.

In general, I’m skeptical of AIC as a model decision criterion.

Best,
Ryan

> --
> You received this message because you are subscribed to the Google Groups "dadi-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to dadi-user+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/dadi-user/c5fce200-747d-4924-9880-30f8a4246f08n%40googlegroups.com.

[Camille Voisin]

Hello Ryan,

Thanks for your answer, it’s really helpful.

You said : “In general, I’m skeptical of AIC as a model decision criterion. “
In my case I have AIC and log-likelihood very close to each other. 

So, the best way is to perform a likelihood ratio test between these two models?

Best,

Camille

[Ryan Gutenkunst]

Hello Camille,

For nested models, I much prefer the Likelihood Ratio Test. This is primarily because I understand it better. With a likelihood ratio test, there is a real null hypothesis that one can calculate power and false positives, etc. With AIC, it’s much more ambiguous what “correct” performance is.

Best,
Ryan

> To view this discussion on the web visit https://groups.google.com/d/msgid/dadi-user/2c2c65dc-3759-48b1-9121-bd3a738c52dcn%40googlegroups.com.

[Camille Voisin]

Hello Ryan,

Thanks for you advice, 

I will try this method.

Best,

Camille