model selection by using tracer likelihood score

399 views
Skip to first unread message

Victor Lundström

unread,
Jan 17, 2024, 1:31:56 PMJan 17
to beast...@googlegroups.com
Hi everyone,

I'm working my way through the steps found in this book chapter to produce a time-calibrated tree
so that I can compare divergence events to palaeoclimatic events recorded in my study area (western norway):

On page 171, the authors present the results from 6 (out of 16) different models by assessing
the marginal likelihood score. Wanting to try something similar for my analysis, I decided
to replicate their top 6 models. However, since I can't get the path sampler app to work for some reason, I decided to simply rely on the likelihood score that is printed in the trace file (mean likelihood). 

Also, I produced a total of eight models: Model 1-4: Coalescent bayesian skyline with a strict clock and where the gamma category count starts with 1 model#1, 2 for model#2 and so forth. 
Model 5-8: Coalescent bayesian skyline with a relaxed exponential clock. The procedure for 
gamma category count was done in the same way as for models 1 to 4.

From this procedure, and according to the author's reasoning (the highest likelihood score = best fit), my best model should be:

Gamma category counts: 2
Clock: Relaxed Exponential 
Prior: Coalescent bayesian Skyline

image.png

Would this be a viable way weeding out the best fit model? I should also mention
that this way of testing is biased to some extent since I haven't yet figured out what other
priors I should try that allows me to use tip dates. CBS is the only one so far that works for me.

Best,
Victor





Remco Bouckaert

unread,
Jan 18, 2024, 12:19:37 PMJan 18
to beast...@googlegroups.com
Hi Victor,

Life would be so much nicer if that comparison would be valid, but it isn’t. It is always possible to get model with a higher likelihood by adding more parameters, and direct likelihood comparisons do not take this in account.

By going from strict to relaxed clock, more parameters are required, so you have to take these in account when comparing models. Marginal likelihood estimates through path sampling or nested sampling are the recommended ways to do so.

Cheers,

Remco

On 18/01/2024, at 1:38 AM, Victor Lundström <lundstr...@gmail.com> wrote:

Hi everyone,

I'm working my way through the steps found in this book chapter to produce a time-calibrated tree
so that I can compare divergence events to palaeoclimatic events recorded in my study area (western norway):

On page 171, the authors present the results from 6 (out of 16) different models by assessing
the marginal likelihood score. Wanting to try something similar for my analysis, I decided
to replicate their top 6 models. However, since I can't get the path sampler app to work for some reason, I decided to simply rely on the likelihood score that is printed in the trace file (mean likelihood). 

Also, I produced a total of eight models: Model 1-4: Coalescent bayesian skyline with a strict clock and where the gamma category count starts with 1 model#1, 2 for model#2 and so forth. 
Model 5-8: Coalescent bayesian skyline with a relaxed exponential clock. The procedure for 
gamma category count was done in the same way as for models 1 to 4.

From this procedure, and according to the author's reasoning (the highest likelihood score = best fit), my best model should be:

Gamma category counts: 2
Clock: Relaxed Exponential 
Prior: Coalescent bayesian Skyline

<image.png>

Would this be a viable way weeding out the best fit model? I should also mention
that this way of testing is biased to some extent since I haven't yet figured out what other
priors I should try that allows me to use tip dates. CBS is the only one so far that works for me.

Best,
Victor






--
You received this message because you are subscribed to the Google Groups "beast-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beast-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beast-users/CAJwYDjHjvam49xfvu_b7aRZWv%3DTFH8b%2BdcN2f6a2EoCqGx3KmQ%40mail.gmail.com.

Victor Lundström

unread,
Jan 18, 2024, 3:14:44 PMJan 18
to beast...@googlegroups.com
Hi Remco,

Oh no! Ah well, I was afraid my results had been achieved a little too easy.
So how do I go about doing this in path sampler then.
Whenever I try to follow this tutorial: BEAST 2

I get this message, and nothing happens.
image.png

Thanks for responding Remco!

Best,
Victor


Remco Bouckaert

unread,
Jan 22, 2024, 4:25:54 PMJan 22
to beast...@googlegroups.com
Hi Victor,

Perhaps the analysis may be running in the background, but the screen is not updated. You might confirm this by seeing how much CPU is used: only if no process is using significant CPU there may be something wrong.

For large analyses, it is probably more efficient to prepare the XML (see BFD* tutorial) and run from a terminal.

Cheers,

Remco

On 19/01/2024, at 9:05 AM, Victor Lundström <lundstr...@gmail.com> wrote:

Hi Remco,

Oh no! Ah well, I was afraid my results had been achieved a little too easy.
So how do I go about doing this in path sampler then.
Whenever I try to follow this tutorial: BEAST 2

I get this message, and nothing happens.

Victor Lundström

unread,
Jan 23, 2024, 6:50:44 PMJan 23
to beast...@googlegroups.com
This is the prompt:
"   java.lang.NoSuchMethodException: sa.beauti.SampledNodeDataWalkerInputEditor.<init>(beastfx.app.inputeditor.BeautiDoc)
java.lang.NullPointerException: Cannot invoke "Object.getClass()" because "<local3>" is null
at modelselection.inference.PathSampler.run(Unknown Source)
at beastfx.app.tools.Application.createDialog(Unknown Source)
at beastfx.app.util.Console$2.run(Unknown Source)
at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$10(Unknown Source)
at java.base/java.security.AccessController.doPrivileged(Unknown Source)
at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$11(Unknown Source)
at javafx.graphics/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(Unknown Source)
at javafx.graphics/com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
at javafx.graphics/com.sun.glass.ui.win.WinApplication.lambda$runLoop$3(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Path Sampler done in 0 seconds. Close window to finish application."

And here it is together with my CPU activity (or absence of)
image.png

Best,
Victor

On Mon, Jan 22, 2024 at 10:27 PM Victor Lundström <lundstr...@gmail.com> wrote:
Even if the prompt says ”path sampler finished in 0 seconds”?

Thanks again for responding!

/Victor

Victor Lundström

unread,
Jan 23, 2024, 6:50:44 PMJan 23
to beast...@googlegroups.com
Even if the prompt says ”path sampler finished in 0 seconds”?

Thanks again for responding!

/Victor
Reply all
Reply to author
Forward
0 new messages