model selection by using tracer likelihood score

[Victor Lundström]

Hi everyone,

I'm working my way through the steps found in this book chapter to produce a time-calibrated tree

so that I can compare divergence events to palaeoclimatic events recorded in my study area (western norway):

Gjesfjeld_Jordan2019_Chapter_ContributionsOfBayesianPhyloge.pdf (rug.nl)

The data for the study can be found here: GitHub - erikgjes/Bayes_Phylogenies: Data and code associated with book chapter on Bayesian Phylogenies in Archaeology

On page 171, the authors present the results from 6 (out of 16) different models by assessing

the marginal likelihood score. Wanting to try something similar for my analysis, I decided

to replicate their top 6 models. However, since I can't get the path sampler app to work for some reason, I decided to simply rely on the likelihood score that is printed in the trace file (mean likelihood). 

Also, I produced a total of eight models: Model 1-4: Coalescent bayesian skyline with a strict clock and where the gamma category count starts with 1 model#1, 2 for model#2 and so forth. 

Model 5-8: Coalescent bayesian skyline with a relaxed exponential clock. The procedure for 

gamma category count was done in the same way as for models 1 to 4.

From this procedure, and according to the author's reasoning (the highest likelihood score = best fit), my best model should be:

Gamma category counts: 2

Clock: Relaxed Exponential 

Prior: Coalescent bayesian Skyline

Would this be a viable way weeding out the best fit model? I should also mention

that this way of testing is biased to some extent since I haven't yet figured out what other

priors I should try that allows me to use tip dates. CBS is the only one so far that works for me.

Best,
Victor

[Remco Bouckaert]

Hi Victor,

Life would be so much nicer if that comparison would be valid, but it isn’t. It is always possible to get model with a higher likelihood by adding more parameters, and direct likelihood comparisons do not take this in account.

By going from strict to relaxed clock, more parameters are required, so you have to take these in account when comparing models. Marginal likelihood estimates through path sampling or nested sampling are the recommended ways to do so.

Cheers,

Remco

On 18/01/2024, at 1:38 AM, Victor Lundström <lundstr...@gmail.com> wrote:

Hi everyone,

I'm working my way through the steps found in this book chapter to produce a time-calibrated tree

so that I can compare divergence events to palaeoclimatic events recorded in my study area (western norway):

Gjesfjeld_Jordan2019_Chapter_ContributionsOfBayesianPhyloge.pdf (rug.nl)

The data for the study can be found here: GitHub - erikgjes/Bayes_Phylogenies: Data and code associated with book chapter on Bayesian Phylogenies in Archaeology

On page 171, the authors present the results from 6 (out of 16) different models by assessing

the marginal likelihood score. Wanting to try something similar for my analysis, I decided

to replicate their top 6 models. However, since I can't get the path sampler app to work for some reason, I decided to simply rely on the likelihood score that is printed in the trace file (mean likelihood). 

Also, I produced a total of eight models: Model 1-4: Coalescent bayesian skyline with a strict clock and where the gamma category count starts with 1 model#1, 2 for model#2 and so forth. 

Model 5-8: Coalescent bayesian skyline with a relaxed exponential clock. The procedure for 

gamma category count was done in the same way as for models 1 to 4.

From this procedure, and according to the author's reasoning (the highest likelihood score = best fit), my best model should be:

Gamma category counts: 2

Clock: Relaxed Exponential 

Prior: Coalescent bayesian Skyline

<image.png>

Would this be a viable way weeding out the best fit model? I should also mention

that this way of testing is biased to some extent since I haven't yet figured out what other

priors I should try that allows me to use tip dates. CBS is the only one so far that works for me.

Best,
Victor

--
You received this message because you are subscribed to the Google Groups "beast-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beast-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beast-users/CAJwYDjHjvam49xfvu_b7aRZWv%3DTFH8b%2BdcN2f6a2EoCqGx3KmQ%40mail.gmail.com.

[Victor Lundström]

Hi Remco,

Oh no! Ah well, I was afraid my results had been achieved a little too easy.

So how do I go about doing this in path sampler then.
Whenever I try to follow this tutorial: BEAST 2

I get this message, and nothing happens.

Thanks for responding Remco!

Best,
Victor

To view this discussion on the web visit https://groups.google.com/d/msgid/beast-users/0AB8896F-3BCE-4196-9E5A-C23B38024FBB%40gmail.com.

[Remco Bouckaert]

Hi Victor,

Perhaps the analysis may be running in the background, but the screen is not updated. You might confirm this by seeing how much CPU is used: only if no process is using significant CPU there may be something wrong.

For large analyses, it is probably more efficient to prepare the XML (see BFD* tutorial) and run from a terminal.

Cheers,

Remco

On 19/01/2024, at 9:05 AM, Victor Lundström <lundstr...@gmail.com> wrote:

Hi Remco,

Oh no! Ah well, I was afraid my results had been achieved a little too easy.

So how do I go about doing this in path sampler then.
Whenever I try to follow this tutorial: BEAST 2

I get this message, and nothing happens.

To view this discussion on the web visit https://groups.google.com/d/msgid/beast-users/CAJwYDjHFO6ksK7BB0TCcQb2AAN9_MktPz7%2BQcfuug%2BoHcA5uVQ%40mail.gmail.com.

[Victor Lundström]

This is the prompt:
"   java.lang.NoSuchMethodException: sa.beauti.SampledNodeDataWalkerInputEditor.<init>(beastfx.app.inputeditor.BeautiDoc)

java.lang.NullPointerException: Cannot invoke "Object.getClass()" because "<local3>" is null

at modelselection.inference.PathSampler.run(Unknown Source)

at beastfx.app.tools.Application.createDialog(Unknown Source)

at beastfx.app.util.Console$2.run(Unknown Source)

at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$10(Unknown Source)

at java.base/java.security.AccessController.doPrivileged(Unknown Source)

at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$11(Unknown Source)

at javafx.graphics/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(Unknown Source)

at javafx.graphics/com.sun.glass.ui.win.WinApplication._runLoop(Native Method)

at javafx.graphics/com.sun.glass.ui.win.WinApplication.lambda$runLoop$3(Unknown Source)

at java.base/java.lang.Thread.run(Unknown Source)

Path Sampler done in 0 seconds. Close window to finish application."

And here it is together with my CPU activity (or absence of)

Best,
Victor

On Mon, Jan 22, 2024 at 10:27 PM Victor Lundström <lundstr...@gmail.com> wrote:

Even if the prompt says ”path sampler finished in 0 seconds”?

Thanks again for responding!

/Victor

To view this discussion on the web visit https://groups.google.com/d/msgid/beast-users/E0AA7F94-AB6B-4923-88C8-05228D67CBE2%40gmail.com.

[Victor Lundström]

Even if the prompt says ”path sampler finished in 0 seconds”?

Thanks again for responding!

/Victor

To view this discussion on the web visit https://groups.google.com/d/msgid/beast-users/E0AA7F94-AB6B-4923-88C8-05228D67CBE2%40gmail.com.