How does Tracer computes the ESS and why does it differ from tracecomp

[Guillaume L]

Hi everyone,

I could not find a documentation sourcing the way ESS are computed in Tracer.

I ran Phylobayes and I noticed that Tracer ESS estimates differ from tracecomp. A third method based on the R function 'coda::effectiveSize' differs from the two others.

Note that prior to feeding the Phylobayes trace to Tracer, I removed the columns 2 and 3 using:

    cut -f1,4- ${tree}.pb.trace | sed 's/^#//' > ${tree}.pb.tracer

What causes these differences? I am doing it wrong or is it expected by different underlying formulas?

Thanks in advance,

Guillaume

[Jordan Douglas]

Hi Guillaume,

The code for calculating the ESS in beast and tracer can be found here

https://github.com/CompEvol/beast2/blob/master/src/beast/base/inference/util/ESS.java#L153

As for if/how the equation differs from Phylobayes or the R coda package, I could not say, but I wouldn't be surprised if the equations are different. Also note that the beast ESS calculator has a MAX_LAG=2000 constant, which is there to make the calculation more efficient.  

Hope this helps,

Jordan