more on low ess for prior and posterior

sara

unread,

Mar 1, 2010, 7:51:44 AM3/1/10

to beast-users

Hi,

I am trying EBSP analyses for several datasets, each one with 5 genes
(one mt and 4 nuc).
Even with 100M generation runs, my ess's on prior and posterior are a
bit low (between 90 and 160). The rest of the parameters seem ok,
including rates and gene.treesLK, only some of the demographic
parameters are also low (not extremelly, usually between 100 and 200).
The Ne curves seem ok (fitting what I would expect on the different
species)

On a previous thread here, low ess's were attributed to
overparametrization basically and either the use of HKY (instead of
GTR), or the use of uniform/lognormal (instead of jeffreys) prior was
advised, BUT I am already using HKY (which is actually the model
selected by modeltest) to all my genes, so I am wondering if this is
the same problem (overparametrization) and how can I work it out?

May it be a problem with the model type (on tree prior)? I am using
the linear. Should I use the stepwise or exponential? (maybe using
this preliminary runs to have an idea of the prior I could use on the
nuber of steps??)

Any advice would be really appreciated,
and thanks again,

sara

unread,

Mar 1, 2010, 9:01:33 AM3/1/10

to beast-users

maybe I can be a bit more specific. the low ess's are usually on:
demographic.populationMean
some populationSizes (varying from a few to almost all, generally
skewed towards the first ones: popSize1, 2, ...)
some demographic.Indicators

Maybe what is missing is an informative prior on the populationMean...
what do other people think?

pepster

unread,

Mar 4, 2010, 4:13:24 PM3/4/10

to beast-users

You can try and set the population mean to a fixed value (use the
median from you run), and see if that makes a difference. Post the
result here ...

-Joseph

sara

unread,

Mar 8, 2010, 8:36:55 AM3/8/10

to beast-users

thanks, I'll do that.
meanwhile, a question that I posted before but has not been answered:
how can I get the estimate of the tmrca (and hpd intervals) of the
group for which I am performing ebsp to?

In previous (BSP) versions there as a rootheight parameter that I do
not find now... or am I missing something?

thanks,

sara

unread,

Mar 12, 2010, 1:30:27 PM3/12/10

to beast-users

Hi,
I did not find how to set it to a fixed value. The options on the
prior on the pop mean are 1/X, exponential, Laplace, Normal, ...

I tried using the 1/x (wich is the default) but giving it already the
mean value from my previous runs as the "initial value", but nothing
changed, i.e., ess's still low on prior, posterios and many of the
popsizes and indicator parameters still.

How should I work this around?? with a normal or uniform very tight
interval around my "best" estimate??

any insight really welcome,

On 4 Mar, 21:13, pepster <jhe...@gmail.com> wrote:

Andrew Rambaut

unread,

Mar 14, 2010, 4:43:12 AM3/14/10

to spr...@gmail.com, beast-users

To fix any parameter to a value, you need to turn off any operators on that parameter in the Operators table (the check boxes in the first column). Then the parameter won't change from its initial value. It doesn't matter what prior they have so I suggest leaving it as uniform.

Andrew

> --
> You received this message because you are subscribed to the Google Groups "beast-users" group.
> To post to this group, send email to beast...@googlegroups.com.
> To unsubscribe from this group, send email to beast-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/beast-users?hl=en.
>
>

___________________________________________________________________
Andrew Rambaut
Institute of Evolutionary Biology University of Edinburgh
Ashworth Laboratories Edinburgh EH9 3JT
EMAIL - a.ra...@ed.ac.uk TEL - +44 131 6508624

--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

sara

unread,

Apr 22, 2010, 1:00:16 PM4/22/10

to beast-users

Hi,
to try to increase the ess's on demographic.popMean (and other
parameters, see my email above please), I tried to set the popMean as
suggested to a fixed value, by speciiying its value (used the one from
previous runs) on the priors table and unchecking the
"demographic.popMean" from the operators table.

However, results are quite strange: nothing changes concerning ess's
on prior/posterior/demographic.popMean/popSizes and indicators (all
low) and the demographic.popMean value instead of remaining identical
to the prior (1.757/2.412 - 2 datasets) turned to 7.198 and 71 (mean),
respectively.

So I guess I effectivelly did not turn off the operators... and wonder
how to do it, or what could be the problem.

xml parameter_prior and operator portions pasted below.

any suggestion?

thanks!

#### PRIOR###
<coalescentLikelihood id="coalescent">
<model>
<variableDemographic idref="demographic"/>
</model>

</coalescentLikelihood>
<sumStatistic id="demographic.populationSizeChanges"
elementwise="true">
<parameter idref="demographic.indicators"/>
</sumStatistic>
<exponentialDistributionModel id="demographic.populationMeanDist">
<mean>

<parameter id="demographic.populationMean" value="2.412"/>
</mean>
</exponentialDistributionModel>

###OPERATORS###


<operators id="operators">
<scaleOperator scaleFactor="0.75" weight="0.1">
<parameter idref="Pastsem_citb.kappa"/>
</scaleOperator>
<deltaExchange delta="0.01" weight="0.1">
<parameter idref="Pastsem_citb.frequencies"/>
</deltaExchange>
<scaleOperator scaleFactor="0.75" weight="0.1">
<parameter idref="Pastsem_mc1r.kappa"/>
</scaleOperator>
<deltaExchange delta="0.01" weight="0.1">
<parameter idref="Pastsem_mc1r.frequencies"/>
</deltaExchange>
<scaleOperator scaleFactor="0.75" weight="0.1">
<parameter idref="Pastsem_pdc.kappa"/>
</scaleOperator>
<deltaExchange delta="0.01" weight="0.1">
<parameter idref="Pastsem_pdc.frequencies"/>
</deltaExchange>
<scaleOperator scaleFactor="0.75" weight="0.1">
<parameter idref="Pastsem_rag2.kappa"/>
</scaleOperator>
<deltaExchange delta="0.01" weight="0.1">
<parameter idref="Pastsem_rag2.frequencies"/>
</deltaExchange>
<scaleOperator scaleFactor="0.75" weight="0.1">
<parameter idref="Pastsem_reln.kappa"/>
</scaleOperator>
<deltaExchange delta="0.01" weight="0.1">
<parameter idref="Pastsem_reln.frequencies"/>
</deltaExchange>
<scaleOperator scaleFactor="0.75" weight="3">
<parameter idref="Pastsem_mc1r.clock.rate"/>
</scaleOperator>
<scaleOperator scaleFactor="0.75" weight="3">
<parameter idref="Pastsem_pdc.clock.rate"/>
</scaleOperator>
<scaleOperator scaleFactor="0.75" weight="3">
<parameter idref="Pastsem_rag2.clock.rate"/>
</scaleOperator>
<scaleOperator scaleFactor="0.75" weight="3">
<parameter idref="Pastsem_reln.clock.rate"/>
</scaleOperator>
<upDownOperator scaleFactor="0.75" weight="30">
<up>
<parameter idref="Pastsem_mc1r.clock.rate"/>
<parameter idref="Pastsem_pdc.clock.rate"/>
<parameter idref="Pastsem_rag2.clock.rate"/>
<parameter idref="Pastsem_reln.clock.rate"/>
</up>
<down>
<parameter idref="demographic.populationMean"/>
<parameter idref="demographic.popSize"/>
<parameter idref="Pastsem_citb.treeModel.allInternalNodeHeights"/>
<parameter idref="Pastsem_mc1r.treeModel.allInternalNodeHeights"/>
<parameter idref="Pastsem_pdc.treeModel.allInternalNodeHeights"/>
<parameter idref="Pastsem_rag2.treeModel.allInternalNodeHeights"/>
<parameter idref="Pastsem_reln.treeModel.allInternalNodeHeights"/>
</down>
</upDownOperator>
<subtreeSlide size="0.03" gaussian="true" weight="15">
<treeModel idref="Pastsem_citb.treeModel"/>
</subtreeSlide>
<narrowExchange weight="15">
<treeModel idref="Pastsem_citb.treeModel"/>
</narrowExchange>
<wideExchange weight="3">
<treeModel idref="Pastsem_citb.treeModel"/>
</wideExchange>
<wilsonBalding weight="3">
<treeModel idref="Pastsem_citb.treeModel"/>
</wilsonBalding>
<scaleOperator scaleFactor="0.75" weight="3">
<parameter idref="Pastsem_citb.treeModel.rootHeight"/>
</scaleOperator>
<uniformOperator weight="30">
<parameter idref="Pastsem_citb.treeModel.internalNodeHeights"/>
</uniformOperator>
<subtreeSlide size="8.499999999999999E-5" gaussian="true"
weight="15">
<treeModel idref="Pastsem_mc1r.treeModel"/>
</subtreeSlide>
<narrowExchange weight="15">
<treeModel idref="Pastsem_mc1r.treeModel"/>
</narrowExchange>
<wideExchange weight="3">
<treeModel idref="Pastsem_mc1r.treeModel"/>
</wideExchange>
<wilsonBalding weight="3">
<treeModel idref="Pastsem_mc1r.treeModel"/>
</wilsonBalding>
<scaleOperator scaleFactor="0.75" weight="3">
<parameter idref="Pastsem_mc1r.treeModel.rootHeight"/>
</scaleOperator>
<uniformOperator weight="30">
<parameter idref="Pastsem_mc1r.treeModel.internalNodeHeights"/>
</uniformOperator>
<subtreeSlide size="2.0E-5" gaussian="true" weight="15">
<treeModel idref="Pastsem_pdc.treeModel"/>
</subtreeSlide>
<narrowExchange weight="15">
<treeModel idref="Pastsem_pdc.treeModel"/>
</narrowExchange>
<wideExchange weight="3">
<treeModel idref="Pastsem_pdc.treeModel"/>
</wideExchange>
<wilsonBalding weight="3">
<treeModel idref="Pastsem_pdc.treeModel"/>
</wilsonBalding>
<scaleOperator scaleFactor="0.75" weight="3">
<parameter idref="Pastsem_pdc.treeModel.rootHeight"/>
</scaleOperator>
<uniformOperator weight="30">
<parameter idref="Pastsem_pdc.treeModel.internalNodeHeights"/>
</uniformOperator>
<subtreeSlide size="4.4E-5" gaussian="true" weight="15">
<treeModel idref="Pastsem_rag2.treeModel"/>
</subtreeSlide>
<narrowExchange weight="15">
<treeModel idref="Pastsem_rag2.treeModel"/>
</narrowExchange>
<wideExchange weight="3">
<treeModel idref="Pastsem_rag2.treeModel"/>
</wideExchange>
<wilsonBalding weight="3">
<treeModel idref="Pastsem_rag2.treeModel"/>
</wilsonBalding>
<scaleOperator scaleFactor="0.75" weight="3">
<parameter idref="Pastsem_rag2.treeModel.rootHeight"/>
</scaleOperator>
<uniformOperator weight="30">
<parameter idref="Pastsem_rag2.treeModel.internalNodeHeights"/>
</uniformOperator>
<subtreeSlide size="4.5E-4" gaussian="true" weight="15">
<treeModel idref="Pastsem_reln.treeModel"/>
</subtreeSlide>
<narrowExchange weight="15">
<treeModel idref="Pastsem_reln.treeModel"/>
</narrowExchange>
<wideExchange weight="3">
<treeModel idref="Pastsem_reln.treeModel"/>
</wideExchange>
<wilsonBalding weight="3">
<treeModel idref="Pastsem_reln.treeModel"/>
</wilsonBalding>
<scaleOperator scaleFactor="0.75" weight="3">
<parameter idref="Pastsem_reln.treeModel.rootHeight"/>
</scaleOperator>
<uniformOperator weight="30">
<parameter idref="Pastsem_reln.treeModel.internalNodeHeights"/>
</uniformOperator>
<sampleNonActiveOperator weight="15">
<distribution>
<parameter idref="demographic.populationMeanDist"/>
</distribution>
<data>
<parameter idref="demographic.popSize"/>
</data>
<indicators>
<parameter idref="demographic.indicators"/>
</indicators>
</sampleNonActiveOperator>
<bitFlipOperator weight="30">
<parameter idref="demographic.indicators"/>
</bitFlipOperator>
<scaleOperator scaleFactor="0.5" weight="6">
<parameter idref="demographic.popSize"/>
<indicators pickoneprob="1.0">
<parameter idref="demographic.indicators"/>
</indicators>
</scaleOperator>
<upDownOperator scaleFactor="0.75" weight="3">
<up>
</up>
<down>
<parameter idref="Pastsem_citb.treeModel.allInternalNodeHeights"/>
</down>
</upDownOperator>
<upDownOperator scaleFactor="0.75" weight="3">
<up>
<parameter idref="Pastsem_mc1r.clock.rate"/>
</up>
<down>
<parameter idref="Pastsem_mc1r.treeModel.allInternalNodeHeights"/>
</down>
</upDownOperator>
<upDownOperator scaleFactor="0.75" weight="3">
<up>
<parameter idref="Pastsem_pdc.clock.rate"/>
</up>
<down>
<parameter idref="Pastsem_pdc.treeModel.allInternalNodeHeights"/>
</down>
</upDownOperator>
<upDownOperator scaleFactor="0.75" weight="3">
<up>
<parameter idref="Pastsem_rag2.clock.rate"/>
</up>
<down>
<parameter idref="Pastsem_rag2.treeModel.allInternalNodeHeights"/>
</down>
</upDownOperator>
<upDownOperator scaleFactor="0.75" weight="3">
<up>
<parameter idref="Pastsem_reln.clock.rate"/>
</up>
<down>
<parameter idref="Pastsem_reln.treeModel.allInternalNodeHeights"/>
</down>
</upDownOperator>
</operators>

On 14 Mar, 09:43, Andrew Rambaut <a.ramb...@ed.ac.uk> wrote:
> To fix any parameter to a value, you need to turn off any operators on that parameter in the Operators table (the check boxes in the first column). Then the parameter won't change from its initial value. It doesn't matter whatpriorthey have so I suggest leaving it as uniform.

>
> Andrew
>
> On 13 Mar 2010, at 05:30, sara wrote:
>
>
>
> > Hi,
> > I did not find how to set it to a fixed value. The options on the

> >prioron the pop mean are 1/X, exponential, Laplace, Normal, ...

>
> > I tried using the 1/x (wich is the default) but giving it already the
> > mean value from my previous runs as the "initial value", but nothing

> > changed, i.e.,ess'sstilllowonprior, posterios and many of the

> > popsizes and indicator parameters still.
>
> > How should I work this around?? with a normal or uniform very tight
> > interval around my "best" estimate??
>
> > any insight really welcome,
>
> > On 4 Mar, 21:13, pepster <jhe...@gmail.com> wrote:
> >> You can try and set the population mean to a fixed value (use the
> >> median from you run), and see if that makes a difference. Post the
> >> result here ...
>
> >> -Joseph
>
> >> On Mar 2, 3:01 am, sara <spro...@gmail.com> wrote:
>

> >>> maybe I can be a bit more specific. thelowess'sare usually on:

> >>> demographic.populationMean
> >>> some populationSizes (varying from a few to almost all, generally
> >>> skewed towards the first ones: popSize1, 2, ...)
> >>> some demographic.Indicators
>

> >>> Maybe what is missing is an informativeprioron the populationMean...

> >>> what do other people think?
>
> >>> On 1 Mar, 12:51, sara <spro...@gmail.com> wrote:
>
> >>>> Hi,
>
> >>>> I am trying EBSP analyses for several datasets, each one with 5 genes
> >>>> (one mt and 4 nuc).
> >>>> Even with 100M generation runs, myess'sonpriorandposteriorare a

> >>>> bitlow(between 90 and 160). The rest of the parameters seem ok,

> >>>> including rates and gene.treesLK, only some of the demographic

> >>>> parameters are alsolow(not extremelly, usually between 100 and 200).

> >>>> The Ne curves seem ok (fitting what I would expect on the different
> >>>> species)
>

> >>>> On a previous thread here,lowess'swere attributed to

> >>>> overparametrization basically and either the use of HKY (instead of
> >>>> GTR), or the use of uniform/lognormal (instead of jeffreys)priorwas
> >>>> advised, BUT I am already using HKY (which is actually the model
> >>>> selected by modeltest) to all my genes, so I am wondering if this is
> >>>> the same problem (overparametrization) and how can I work it out?
>

> >>>> May it be a problem with the model type (on treeprior)? I am using

> >>>> the linear. Should I use the stepwise or exponential? (maybe using

> >>>> this preliminary runs to have an idea of thepriorI could use on the

> >>>> nuber of steps??)
>
> >>>> Any advice would be really appreciated,
> >>>> and thanks again,
>
> > --
> > You received this message because you are subscribed to the Google Groups "beast-users" group.
> > To post to this group, send email to beast...@googlegroups.com.
> > To unsubscribe from this group, send email to beast-users...@googlegroups.com.

> > For more options, visit this group athttp://groups.google.com/group/beast-users?hl=en.

>
> ___________________________________________________________________
> Andrew Rambaut
> Institute of Evolutionary Biology University of Edinburgh
> Ashworth Laboratories Edinburgh EH9 3JT

> EMAIL - a.ramb...@ed.ac.uk TEL - +44 131 6508624

>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.

pepster

unread,

Apr 22, 2010, 6:57:13 PM4/22/10

to beast-users

The mean is still changed by the up-down operator. You should remove
that line.

-Joseph

> <upDownOperator scaleFactor="0.75" weight="30">
> <up>
> <parameter idref="Pastsem_mc1r.clock.rate"/>
> <parameter idref="Pastsem_pdc.clock.rate"/>
> <parameter idref="Pastsem_rag2.clock.rate"/>
> <parameter idref="Pastsem_reln.clock.rate"/>
> </up>
> <down>

> <parameter idref="demographic.populationMean"/> <======= ****THIS LINE***

> <parameter idref="demographic.popSize"/>
> <parameter idref="Pastsem_citb.treeModel.allInternalNodeHeights"/>
> <parameter idref="Pastsem_mc1r.treeModel.allInternalNodeHeights"/>
> <parameter idref="Pastsem_pdc.treeModel.allInternalNodeHeights"/>
> <parameter idref="Pastsem_rag2.treeModel.allInternalNodeHeights"/>
> <parameter idref="Pastsem_reln.treeModel.allInternalNodeHeights"/>
> </down>
> </upDownOperator>

On Apr 23, 5:00 am, sara <spro...@gmail.com> wrote:
> Hi,
> to try to increase the ess's on demographic.popMean (and other
> parameters, see my email above please), I tried to set the popMean as
> suggested to a fixed value, by speciiying its value (used the one from
> previous runs) on the priors table and unchecking the
> "demographic.popMean" from the operators table.
>
> However, results are quite strange: nothing changes concerning ess's
> on prior/posterior/demographic.popMean/popSizes and indicators (all
> low) and the demographic.popMean value instead of remaining identical
> to the prior (1.757/2.412 - 2 datasets) turned to 7.198 and 71 (mean),
> respectively.
>
> So I guess I effectivelly did not turn off the operators... and wonder
> how to do it, or what could be the problem.
>
> xml parameter_prior and operator portions pasted below.
>
> any suggestion?
>
> thanks!

Reply all

Reply to author

Forward