continue an interrupted analysis

[chr...@gmail.com]

Hey everyone

I am kind of new in this area so my questions may sound naive. Is there a way to recover a session in case you accidentally close the running BEAST window? Also, how correct it is to combine the log files from two independent runs in order to get the desired number of generations? In other words, if I already had 8 million generations before I closed the window, can I run a new analysis of 2 million generations and use logcombiner to combine the 2 runs?

Thank you in advance

Chrysa

[Alexei Drummond]

Dear Chrysa,

In BEAST 2.0 you can easily resume an accidentally closed BEAST run (or a completed BEAST run) by simply running BEAST again with the same XML and selecting the "resume" option in the dialog box (or with the -resume option on command line). This is possible because BEAST 2 automatically writes and keeps up-to-date a special .state file that stores the latest state in the MCMC run at all times. 

In BEAST 1.x there is no easy way to resume a crashed or completed run.

Cheers

Alexei

[chr...@gmail.com]

Thanx Alexei

I will give it a try :D

[historus]

Hi,

sorry for bringing up this topic after almost 2 years but today my BEAST run was interrupted as we had a power outage. 2 month of calculation (at about 75%) seem to be lost. So I'd like to ask what "no easy way" means exactly. Is there a way to continue this run?

Any help is appreciated.

Thanks,

Matt

[Anton]

Hello Matt,
Have you found a way to continue the run? I hope so, because I'm in exactly the same trouble.
I'm using Beast 1.8. I've searched all FAQs and forums, but have not found a solution for <2.0 Beast versions. There is a how-to (http://beast.bio.ed.ac.uk/Tutorial-4) on injecting user-specified starting trees in xml, but the trees in Beast tree log are not in Newick format. I'm very upset to lose a huge amount of work because of this issue.

Thank you,
Anton

[Andrew Rambaut]

BEAST 1.8 doesn’t have a way of restarting a run but if you have done 75% of the chain length you intended it to run for, there may be enough post burn-in samples to get a good estimate.

Look at the output you have so far and estimate what the burnin is. If it is a modest fraction of the length of the chain you have run, then that is fine. You would then probably be better off

running a second independent chain and combining the samples rather than restarting the existing chain. 

You can use the last tree in your existing chain as a starting tree for the new run (and set all the parameters’ starting values to the values at the end of the run). This i not quite a restart

(because it is not possible to re-assign certain values) but will mean your new chain will start close to where the last chain left off.

Andrew

--
You received this message because you are subscribed to the Google Groups "beast-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beast-users...@googlegroups.com.
To post to this group, send email to beast...@googlegroups.com.
Visit this group at http://groups.google.com/group/beast-users.
For more options, visit https://groups.google.com/d/optout.

Andrew Rambaut 

Institute for Evolutionary Biology | Centre for Infection, Immunity & Evolution 

Ashworth Laboratories, University of Edinburgh, Edinburgh, EH9 3FL, UK

contact – a.ra...@ed.ac.uk | http://tree.bio.ed.ac.uk | tel +44 131 6508624

[Anton]

Dear Andrew,

Many thanks for your prompt response! I understand the options available as well as a general inability to resume the exact chain given the latest available values from the previous run.

Unfortunately, in my case the burnin fraction is still huge. So I’d like to proceed with XML editing option as you suggested, because I have a very long chain and suspect that its interruption/failure may happen again.

I can re-assign most of the parameters’ values in XML (taken from the previous .log file), but what I can’t still accomplish is to convert the last tree from the interrupted .trees log to newick format (i.e., for <newick> XML tag). The resulting trees are large enough given the number of sequences at hand (>1500), so the conversion can’t even be done manually. Could you please advise some tools for this?

 

I’m attaching the last logged tree of the interrupted chain so you could see what it looks like. Currently I just get the following XML parsing error when trying to reference this tree inside <treeModel> element.

 

<<Parsing error - poorly formed XML (possibly not an XML file):

The reference to entity "rate" must end with the ';' delimiter.>>

Thank you,
Anton

[Steven Myers]

I'm doing a SNAPP analysis in BEAST 2.0. My path sampling was interrupted at step 72 of 80 steps. I tried to resume the run and it's just overwriting steps (i.e. starting at step 0 again).

[Steven Myers]

I'd like to know if there's any way to keep the steps I've already run, and continue from the step that I was up to.

Also, at the end of the analysis I'll need to estimate the marginal likelihood, but will the results from all steps show on the screen at the end of analysis if I only analysed the steps that remained? Is there some other way to estimate the ML if I have the results from a series of steps? I found this https://groups.google.com/forum/#!topic/beast-users/rJdnQo6EHA0 but it seems to be for linux -- I'm on a windows machine.

[Remco Bouckaert]

Hi Steven,

Essentially, what needs to be done is run BEAST on the beast.xml files in each of the step folders. In the parent of the step folder (the one containing step0, step1, step2, etc.) there is one of more batch file to run all these beast.xml files. (It tends to be more efficient to start with the end state of the previous step, so the state file gets copied from one to the next step folder and BEAST runs resuming from the previous step’s state.) You can edit those files, remove the steps already done and then start these batch files, but you can also just run BEAST on each of the remaining beast.xml files, but perhaps you need longer runs if you don’t resume from the other step’s state.

To calculate the marginal likelihood, you can run the path sample analyser. To start it, open BEAUti, select the File/Launch Apps menu and select PathSampleAnalyser from the list of apps that are shown in the dialog box. A menu pops up that asks for the step directory, and some other information.

Cheers,

Remco

--
You received this message because you are subscribed to the Google Groups "beast-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beast-users...@googlegroups.com.
To post to this group, send email to beast...@googlegroups.com.

Visit this group at https://groups.google.com/group/beast-users.

[Steven Myers]

thanks Remco

[Ángela Cano]

Dear Andrew,

Five years have passed since you replied to this question. I'm sorry to come back to it, but I would like to follow your advice regarding using the last sampled tree and the parameters' values. I have managed to use the last tree as starting tree, but I don't see how to set all the parameters’ starting values. Could you please elaborate?

I tried to find a tutorial showing how to do it and explored the BEAUTi options, as well as the XML file, but I still don't see how.

Many thanks in advance. 

On Thursday, 30 July 2015 15:16:07 UTC+1, rambaut wrote:

BEAST 1.8 doesn’t have a way of restarting a run but if you have done 75% of the chain length you intended it to run for, there may be enough post burn-in samples to get a good estimate.

Look at the output you have so far and estimate what the burnin is. If it is a modest fraction of the length of the chain you have run, then that is fine. You would then probably be better off

running a second independent chain and combining the samples rather than restarting the existing chain. 

You can use the last tree in your existing chain as a starting tree for the new run (and set all the parameters’ starting values to the values at the end of the run). This i not quite a restart

(because it is not possible to re-assign certain values) but will mean your new chain will start close to where the last chain left off.

Andrew

To unsubscribe from this group and stop receiving emails from it, send an email to beast...@googlegroups.com.

To post to this group, send email to beast...@googlegroups.com.
Visit this group at http://groups.google.com/group/beast-users.
For more options, visit https://groups.google.com/d/optout.