How to estimate the composite likelihood AIC (CLAIC) with DaDi?

[Diego V]

I'm trying to compare some non-nested models I fit with DaDi.

I came across a method called "GADMA" (https://academic.oup.com/gigascience/article/9/3/giaa005/5768731) that applies Genetic Algorithms to find the best demographic model using DaDi (or other engines), and the CL-AIC as the criterion. I'm attaching the relevant part of the paper.

I was wondering if you could give me any advice on how to implement it in DaDi (I tried checking the code of GADMA but couldn't find the relevant section).

I think i've been able to get the GIM and the FIM. I think I've also understood decently well how the bootstrapping is implemented in DaDi. I have also checked the underlying code to perform a LRT and to get the confidence intervals if that will be of any help.

Thank you for all the work you put into running this forum!

Best,

Diego

[Diego V]

I messed up the screenshot, sorry. Here is the full relevant section

[Ryan Gutenkunst]

Hello Diego,

This isn’t directly supported in dadi. But if you want to do some implementation…

In Godambe.py, the GIM_uncert function calculates the J and H matrices as intermediate steps (returned from the get_godambe function). So you could do extract the results from there and then do the math to calculate CLAIC.

(When we wrote Coffman (2016), we considered implementing CLAIC, but decided against it. I generally not a fan of AIC analysis, even though it’s so common, because it’s unclear what the null expectation is to validate the calculation.)

Best,
Ryan

> --
> You received this message because you are subscribed to the Google Groups "dadi-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to dadi-user+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/dadi-user/74ca834a-4bed-4066-9778-5b5f012ce78bn%40googlegroups.com.
> <Screenshot 2023-02-28 at 12-58-34 GADMA Genetic algorithm for inferring demographic history of multiple populations from allele frequency spectrum data.png>

[Diego V]

Hi Ryan,

Thanks a lot for the information, I'll give it a try.

On another note, I have a final question regarding the estimation of confidence intervals. I've used two approaches:

•  With dadi.Godambe.GIM_uncert(... log=False) to estimate the std. dev. and then used it to calculate the 95% CI.

• With  dadi.Godambe.GIM_uncert(... log=True) to estimate the std. dev. of the logs of the parameters. I then calculated the 95%CI in the log scale and exponentiated them to get estimate the 95%CI of the parameters.

Which approach is the preferred one? Is the same approach always preferred, or does it depend on the kind of parameter? For instance, parameters with range [0, 1] (e.g. the fraction in a population split) vs parameters with much wider ranges, such as population size.

Thank you once again for your help.

Best,

Diego

[Ryan Gutenkunst]

Hello Diego,

I don’t think there’s a strict rule, since both are based on an uncontrolled quadratic approximation. In the limit that uncertainties are small, both approaches should converge to each other. I tend to use the log scale when the uncertainties are large enough to admit non sensible values (< 0 for many parameters for example), since the log scale excludes zero. With a parameter bounded by [0,1], neither the normal or lognormal approximation will be good for cases with large uncertainty.

Does that help?

Best,
Ryan

> To view this discussion on the web visit https://groups.google.com/d/msgid/dadi-user/e0a932af-6289-4356-b24f-c150c4806d7cn%40googlegroups.com.