ESS values dropped in resuming BEAST runs

[Taehyung Kwon]

Dear all

I ran BEAST for 1 billion chain (1st run). Since I wanted my run to be converged, I extended another 1 billion chain to my run using -resume option (2nd run).

And I found out that ESS values had dropped when second BEAST run restarted.

1) I guess this came from different random seed in first and second run, and I may suppose to specify the same seed at the very beginning of BEAST runs. Is this right?

2) If BEAST is sampling posteriors from MCMC, is it possible for me to still use my extended run of 2 billion chain as a legit result?

Always thanks for your time.

Sincerely,

Taehyung

[Tim Vaughan]

Dear Taehyung,

It's entirely possible for ESS estimates to drop as the analysis
proceeds. This can happen when, for instance, the chain is exploring
a local maximum and then much later transitions to another local
maximum. In response to your questions:

1) You don't need to use the same seed to resume the analysis.
2) You should be able to treat your 2 billion step resumed chain as
you would the result of a single contiguous run.

Before drawing any conclusions though, definitely run this analysis
multiple times and compare the results to ensure you have convergence.

Tim

> --
> You received this message because you are subscribed to the Google Groups "beast-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to beast-users...@googlegroups.com.
> To post to this group, send email to beast...@googlegroups.com.
> Visit this group at https://groups.google.com/group/beast-users.
> For more options, visit https://groups.google.com/d/optout.

[Taehyung Kwon]

Thanks Tim.

I may need to clarify my question.

I understood that ESS can be dropped as the chain runs, but what I found in screen.log file is this:

...

99980000        -48967.05498357639      74.0    -48275.68123027148      -691.3737533049077

99990000        -49006.80237608028      74.0    -48275.5845364936       -731.2178395866824

100000000       -48931.247326313984     74.0    -48275.31171173237      -655.9356145816155    => Where the first run ended

100010000       -48950.91159859125      N       -48280.27610626433      -670.6354923269174      => As I resumed the chain, samples seem to be reset

100020000       -49069.63635713353      2.0     -48261.194649976795     -808.4417071567361

100030000       -48959.68254615577      3.0     -48276.29008820391      -683.3924579518573

...

I was wondering if I can count this as a single continuous run.

Taehyung

[Tim Vaughan]

Hi Taehyung,

Don't worry about the ESS in the screen log: this is computed solely
based on what has been produced in the current BEAST run (as opposed
to chain). This is why there's an "N" in the ESS column here: there's
only been a single iteration in the second run: not enough to compute
an ESS. It's just there to provide a rough indication for how the
particular run is going, and is not an indication of the ESS for the
full chain.

The important thing is that the chain in run 2 starts with the exact
same state (tree, parameter values, etc) as the one that run 1
finished with. If you're worried about this you can pay attention to
the starting likelihood quoted by BEAST when run 2 begins and check
that this is exactly the same as the final log likelihood in the log
file from the previous run.

Hope this makes sense,

Tim

[Taehyung Kwon]

So whether screen log seems disconnected or not, I presume that trace log file still gets 2 billion chains.

Thank you so much for detailed explanation.

Have a great day!

SIncerely,

Taehyung