ValueError: Tree is not ultrametric in sumtree

[Bastien Boussau]

Hi all,

I've been trying to use sumtree to summarize a sample of trees of my own making. These are ultrametric trees, and I'm interested in the topology as well as the uncertainty on node ages. Unfortunately, sumtree complains that these trees are not ultrametric, even though I consider they look ultrametric enough in Figtree (I attach a sample of 10 trees in case someone wants to have a look/make a test).
So I guess it's all a matter of threshold. Is there a way to set a threshold and say "if the deviation from a perfectly ultrametric tree is < threshold, still consider it as an ultrametric tree"? That would be very convenient for me I think.

Thanks!

Bastien.


PS: the command I used and its output:
sumtrees.py -b 0 --ultrametric --support-as-labels --output=consensus.tre2 outputTreesOMPAll.txt10First
======================================================================
SumTrees - Phylogenetic Tree Split Support Summarization
Version 3.3.1 (May 05 2011)
By Jeet Sukumaran and Mark T. Holder
(using the DendroPy Phylogenetic Computing Library Version 3.12.0)
======================================================================

Output file already exists: "consensus.tre2"
Overwrite (y/N)? y
SumTrees: Running in serial mode.
SumTrees: 1 source(s) to be processed.
SumTrees: Processing 1 of 1: 'outputTreesOMPAll.txt10First'
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.6/bin/sumtrees.py", line 5, in <module>
    pkg_resources.run_script('DendroPy==3.12.0', 'sumtrees.py')
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/setuptools-0.6c12dev_r88846-py2.6.egg/pkg_resources.py", line 489, in run_script
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/setuptools-0.6c12dev_r88846-py2.6.egg/pkg_resources.py", line 1214, in run_script
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/DendroPy-3.12.0-py2.6.egg/EGG-INFO/scripts/sumtrees.py", line 1049, in <module>
   
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/DendroPy-3.12.0-py2.6.egg/EGG-INFO/scripts/sumtrees.py", line 722, in main_cli
   
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/DendroPy-3.12.0-py2.6.egg/EGG-INFO/scripts/sumtrees.py", line 297, in process_sources_serial
   
  File "build/bdist.macosx-10.3-fat/egg/dendropy/treesplit.py", line 580, in count_splits_on_tree
  File "build/bdist.macosx-10.3-fat/egg/dendropy/dataobject/tree.py", line 1589, in calc_node_ages
ValueError: Tree is not ultrametric


I'm using the latest version of Dendropy I believe.

I put below the list of 10 trees that is causing problem, attaching it did not work.

((((((ANASP:0.342857,ANAVT:0.342857):0.685714,NOSP7:1.02857):0.742857,TRIEI:1.77143):0.0571429,((((CYAA5:0.857143,CYAP8:0.857143):0.571429,(CYAP7:1.37143,GLVIO1:1.37143):0.0571429):0.228571,SYNY3:1.65714):0.0571429,SYNP2:1.71429):0.114286):0.0571429,((THEEB:1.48571,CYAP4:1.48571):0.0571429,ACAM1:1.54286):0.342857):0.114286,(((SYNR3:1.31429,(((PROMM:0.228571,PROM3:0.228571):0.971429,((PRMAR1:0.8,SYNJB:0.8):0.171429,((PROM1:0.114286,PROMT:0.114286):0.8,((PROM9:0.514286,(PROM2:0.457143,(PROM0:0.4,PROMS:0.4):0.0571429):0.0571429):0.0571429,(PROM5:0.285714,PROMP:0.285714):0.285714):0.342857):0.0571429):0.228571):0.0571429,((SYNPW:0.628571,SYNS3:0.628571):0.457143,(SYNPX:0.742857,(SYNS9:0.685714,SYNSC:0.685714):0.0571429):0.342857):0.171429):0.0571429):0.285714,(SYNE7:0.0571429,SYNP6:0.0571429):1.54286):0.342857,((SYNJA:0.171429,PROM4:0.171429):0.971429,MICAN:1.14286):0.8):0.0571429);
((((((ANASP:0.342857,ANAVT:0.342857):0.685714,NOSP7:1.02857):0.742857,TRIEI:1.77143):0.0571429,((((CYAA5:0.857143,CYAP8:0.857143):0.571429,(CYAP7:1.37143,GLVIO1:1.37143):0.0571429):0.228571,SYNY3:1.65714):0.0571429,SYNP2:1.71429):0.114286):0.0571429,((THEEB:1.48571,CYAP4:1.48571):0.0571429,ACAM1:1.54286):0.342857):0.114286,(((SYNR3:1.31429,(((PROMM:0.228571,PROM3:0.228571):0.971429,((PRMAR1:0.8,SYNJB:0.8):0.171429,((PROM1:0.114286,PROMT:0.114286):0.8,((PROM9:0.514286,(PROM2:0.457143,(PROM0:0.4,PROMS:0.4):0.0571429):0.0571429):0.0571429,(PROM5:0.285714,PROMP:0.285714):0.285714):0.342857):0.0571429):0.228571):0.0571429,((SYNPW:0.628571,SYNS3:0.628571):0.457143,(SYNPX:0.742857,(SYNS9:0.685714,SYNSC:0.685714):0.0571429):0.342857):0.171429):0.0571429):0.285714,(SYNE7:0.0571429,SYNP6:0.0571429):1.54286):0.342857,((SYNJA:0.171429,PROM4:0.171429):0.971429,MICAN:1.14286):0.8):0.0571429);
((((((ANASP:0.342857,ANAVT:0.342857):0.685714,NOSP7:1.02857):0.742857,TRIEI:1.77143):0.0571429,((((CYAA5:0.857143,CYAP8:0.857143):0.571429,(CYAP7:1.37143,GLVIO1:1.37143):0.0571429):0.228571,SYNY3:1.65714):0.0571429,SYNP2:1.71429):0.114286):0.0571429,((THEEB:1.48571,CYAP4:1.48571):0.0571429,ACAM1:1.54286):0.342857):0.114286,(((SYNR3:1.31429,(((PROMM:0.228571,PROM3:0.228571):0.971429,((PRMAR1:0.8,SYNJB:0.8):0.171429,((PROM1:0.114286,PROMT:0.114286):0.8,((PROM9:0.514286,(PROM2:0.457143,(PROM0:0.4,PROMS:0.4):0.0571429):0.0571429):0.0571429,(PROM5:0.285714,PROMP:0.285714):0.285714):0.342857):0.0571429):0.228571):0.0571429,((SYNPW:0.628571,SYNS3:0.628571):0.457143,(SYNPX:0.742857,(SYNS9:0.685714,SYNSC:0.685714):0.0571429):0.342857):0.171429):0.0571429):0.285714,(SYNE7:0.0571429,SYNP6:0.0571429):1.54286):0.342857,((SYNJA:0.171429,PROM4:0.171429):0.971429,MICAN:1.14286):0.8):0.0571429);
((((((ANASP:0.342857,ANAVT:0.342857):0.685714,NOSP7:1.02857):0.742857,TRIEI:1.77143):0.0571429,((((CYAA5:0.857143,CYAP8:0.857143):0.571429,(CYAP7:1.37143,GLVIO1:1.37143):0.0571429):0.228571,SYNY3:1.65714):0.0571429,SYNP2:1.71429):0.114286):0.0571429,((THEEB:1.48571,CYAP4:1.48571):0.0571429,ACAM1:1.54286):0.342857):0.114286,(((SYNR3:1.31429,(((PROMM:0.228571,PROM3:0.228571):0.971429,((PRMAR1:0.8,SYNJB:0.8):0.171429,((PROM1:0.114286,PROMT:0.114286):0.8,((PROM9:0.514286,(PROM2:0.457143,(PROM0:0.4,PROMS:0.4):0.0571429):0.0571429):0.0571429,(PROM5:0.285714,PROMP:0.285714):0.285714):0.342857):0.0571429):0.228571):0.0571429,((SYNPW:0.628571,SYNS3:0.628571):0.457143,(SYNPX:0.742857,(SYNS9:0.685714,SYNSC:0.685714):0.0571429):0.342857):0.171429):0.0571429):0.285714,(SYNE7:0.0571429,SYNP6:0.0571429):1.54286):0.342857,((SYNJA:0.171429,PROM4:0.171429):0.971429,MICAN:1.14286):0.8):0.0571429);
((((((ANASP:0.342857,ANAVT:0.342857):0.685714,NOSP7:1.02857):0.742857,TRIEI:1.77143):0.0571429,((((CYAA5:0.857143,CYAP8:0.857143):0.571429,(CYAP7:1.37143,GLVIO1:1.37143):0.0571429):0.228571,SYNY3:1.65714):0.0571429,SYNP2:1.71429):0.114286):0.0571429,((THEEB:1.48571,CYAP4:1.48571):0.0571429,ACAM1:1.54286):0.342857):0.114286,(((SYNR3:1.31429,(((PROMM:0.228571,PROM3:0.228571):0.971429,((PRMAR1:0.8,SYNJB:0.8):0.171429,((PROM1:0.114286,PROMT:0.114286):0.8,((PROM9:0.514286,(PROM2:0.457143,(PROM0:0.4,PROMS:0.4):0.0571429):0.0571429):0.0571429,(PROM5:0.285714,PROMP:0.285714):0.285714):0.342857):0.0571429):0.228571):0.0571429,((SYNPW:0.628571,SYNS3:0.628571):0.457143,(SYNPX:0.742857,(SYNS9:0.685714,SYNSC:0.685714):0.0571429):0.342857):0.171429):0.0571429):0.285714,(SYNE7:0.0571429,SYNP6:0.0571429):1.54286):0.342857,((SYNJA:0.171429,PROM4:0.171429):0.971429,MICAN:1.14286):0.8):0.0571429);
((((((ANASP:0.342857,ANAVT:0.342857):0.685714,NOSP7:1.02857):0.742857,TRIEI:1.77143):0.0571429,((((CYAA5:0.857143,CYAP8:0.857143):0.571429,(CYAP7:1.37143,GLVIO1:1.37143):0.0571429):0.228571,SYNY3:1.65714):0.0571429,SYNP2:1.71429):0.114286):0.0571429,((THEEB:1.48571,CYAP4:1.48571):0.0571429,ACAM1:1.54286):0.342857):0.114286,(((SYNR3:1.31429,(((PROMM:0.228571,PROM3:0.228571):0.971429,((PRMAR1:0.8,SYNJB:0.8):0.171429,((PROM1:0.114286,PROMT:0.114286):0.8,((PROM9:0.514286,(PROM2:0.457143,(PROM0:0.4,PROMS:0.4):0.0571429):0.0571429):0.0571429,(PROM5:0.285714,PROMP:0.285714):0.285714):0.342857):0.0571429):0.228571):0.0571429,((SYNPW:0.628571,SYNS3:0.628571):0.457143,(SYNPX:0.742857,(SYNS9:0.685714,SYNSC:0.685714):0.0571429):0.342857):0.171429):0.0571429):0.285714,(SYNE7:0.0571429,SYNP6:0.0571429):1.54286):0.342857,((SYNJA:0.171429,PROM4:0.171429):0.971429,MICAN:1.14286):0.8):0.0571429);
((((((ANASP:0.342857,ANAVT:0.342857):0.685714,NOSP7:1.02857):0.742857,TRIEI:1.77143):0.0571429,((((CYAA5:0.857143,CYAP8:0.857143):0.571429,(CYAP7:1.37143,GLVIO1:1.37143):0.0571429):0.228571,SYNY3:1.65714):0.0571429,SYNP2:1.71429):0.114286):0.0571429,((THEEB:1.48571,CYAP4:1.48571):0.0571429,ACAM1:1.54286):0.342857):0.114286,(((SYNR3:1.31429,(((PROMM:0.228571,PROM3:0.228571):0.971429,((PRMAR1:0.8,SYNJB:0.8):0.171429,((PROM1:0.114286,PROMT:0.114286):0.8,((PROM9:0.514286,(PROM2:0.457143,(PROM0:0.4,PROMS:0.4):0.0571429):0.0571429):0.0571429,(PROM5:0.285714,PROMP:0.285714):0.285714):0.342857):0.0571429):0.228571):0.0571429,((SYNPW:0.628571,SYNS3:0.628571):0.457143,(SYNPX:0.742857,(SYNS9:0.685714,SYNSC:0.685714):0.0571429):0.342857):0.171429):0.0571429):0.285714,(SYNE7:0.0571429,SYNP6:0.0571429):1.54286):0.342857,((SYNJA:0.171429,PROM4:0.171429):0.971429,MICAN:1.14286):0.8):0.0571429);
((((((ANASP:0.342857,ANAVT:0.342857):0.685714,NOSP7:1.02857):0.742857,TRIEI:1.77143):0.0571429,((((CYAA5:0.857143,CYAP8:0.857143):0.571429,(CYAP7:1.37143,GLVIO1:1.37143):0.0571429):0.228571,SYNY3:1.65714):0.0571429,SYNP2:1.71429):0.114286):0.0571429,((THEEB:1.48571,CYAP4:1.48571):0.0571429,ACAM1:1.54286):0.342857):0.114286,(((SYNR3:1.31429,(((PROMM:0.228571,PROM3:0.228571):0.971429,((PRMAR1:0.8,SYNJB:0.8):0.171429,((PROM1:0.114286,PROMT:0.114286):0.8,((PROM9:0.514286,(PROM2:0.457143,(PROM0:0.4,PROMS:0.4):0.0571429):0.0571429):0.0571429,(PROM5:0.285714,PROMP:0.285714):0.285714):0.342857):0.0571429):0.228571):0.0571429,((SYNPW:0.628571,SYNS3:0.628571):0.457143,(SYNPX:0.742857,(SYNS9:0.685714,SYNSC:0.685714):0.0571429):0.342857):0.171429):0.0571429):0.285714,(SYNE7:0.0571429,SYNP6:0.0571429):1.54286):0.342857,((SYNJA:0.171429,PROM4:0.171429):0.971429,MICAN:1.14286):0.8):0.0571429);
((((((ANASP:0.342857,ANAVT:0.342857):0.685714,NOSP7:1.02857):0.742857,TRIEI:1.77143):0.0571429,((((CYAA5:0.857143,CYAP8:0.857143):0.571429,(CYAP7:1.37143,GLVIO1:1.37143):0.0571429):0.228571,SYNY3:1.65714):0.0571429,SYNP2:1.71429):0.114286):0.0571429,((THEEB:1.48571,CYAP4:1.48571):0.0571429,ACAM1:1.54286):0.342857):0.114286,(((SYNR3:1.31429,(((PROMM:0.228571,PROM3:0.228571):0.971429,((PRMAR1:0.8,SYNJB:0.8):0.171429,((PROM1:0.114286,PROMT:0.114286):0.8,((PROM9:0.514286,(PROM2:0.457143,(PROM0:0.4,PROMS:0.4):0.0571429):0.0571429):0.0571429,(PROM5:0.285714,PROMP:0.285714):0.285714):0.342857):0.0571429):0.228571):0.0571429,((SYNPW:0.628571,SYNS3:0.628571):0.457143,(SYNPX:0.742857,(SYNS9:0.685714,SYNSC:0.685714):0.0571429):0.342857):0.171429):0.0571429):0.285714,(SYNE7:0.0571429,SYNP6:0.0571429):1.54286):0.342857,((SYNJA:0.171429,PROM4:0.171429):0.971429,MICAN:1.14286):0.8):0.0571429);
((((((ANASP:0.342857,ANAVT:0.342857):0.685714,NOSP7:1.02857):0.742857,TRIEI:1.77143):0.0571429,((((CYAA5:0.857143,CYAP8:0.857143):0.571429,(CYAP7:1.37143,GLVIO1:1.37143):0.0571429):0.228571,SYNY3:1.65714):0.0571429,SYNP2:1.71429):0.114286):0.0571429,((THEEB:1.48571,CYAP4:1.48571):0.0571429,ACAM1:1.54286):0.342857):0.114286,(((SYNR3:1.31429,(((PROMM:0.228571,PROM3:0.228571):0.971429,((PRMAR1:0.8,SYNJB:0.8):0.171429,((PROM1:0.114286,PROMT:0.114286):0.8,((PROM9:0.514286,(PROM2:0.457143,(PROM0:0.4,PROMS:0.4):0.0571429):0.0571429):0.0571429,(PROM5:0.285714,PROMP:0.285714):0.285714):0.342857):0.0571429):0.228571):0.0571429,((SYNPW:0.628571,SYNS3:0.628571):0.457143,(SYNPX:0.742857,(SYNS9:0.685714,SYNSC:0.685714):0.0571429):0.342857):0.171429):0.0571429):0.285714,(SYNE7:0.0571429,SYNP6:0.0571429):1.54286):0.342857,((SYNJA:0.171429,PROM4:0.171429):0.971429,MICAN:1.14286):0.8):0.0571429);

[Jeet Sukumaran]

Hi Bastien,


The DendroPy machinery underlying sumtrees actually does allow for
relaxing of this constraint through specification of the maximum
precision. But the sumtrees application in the public release does not
support setting this option.

If you download the latest developmental snapshot from GitHub, you
will find that the sumtrees script there supports an "--ultrametricity-
precision" option which will let you customize this. Setting it to a
large enough number should work. It is experimental, so let me know if
there are any problems.

[Bastien Boussau]

Hi Jeet,

Thanks for your reply! Unfortunately I have not been able to make it work.
I downloaded the latest version using this command:
git clone git://github.com/jeetsukumaran/DendroPy.git

but the sumtrees script here does not seem to have the "-ultrametricity-precision" option.

Indeed if I do:
grep -d recurse "ultrametricity" *

I do not get this option (although grep does find the word "ultrametricity" in various files as shown below:
grep -d recurse "ultrametricity" *
dendropy/coalescent.py:def node_waiting_time_pairs(tree, check_ultrametricity_prec=0.0000001):
dendropy/coalescent.py:    tree.calc_node_ages(check_prec=check_ultrametricity_prec)
dendropy/coalescent.py:def extract_coalescent_frames(tree, check_ultrametricity_prec=0.0000001):
dendropy/coalescent.py:    nwti = node_waiting_time_pairs(tree, check_ultrametricity_prec=check_ultrametricity_prec)
dendropy/coalescent.py:def log_probability_of_coalescent_tree(tree, haploid_pop_size, check_ultrametricity_prec=0.0000001):
dendropy/dataobject/tree.py:        ultrametricity. If `check_prec` is negative or False, then this check
dendropy/dataobject/tree.py:                    # note: no enforcement of ultrametricity!
dendropy/reconcile.py:            ultrametricity_check_prec=False,
dendropy/reconcile.py:            ``ultrametricity_check_prec``
dendropy/reconcile.py:                ultrametricity. Otherwise this is the threshold within which
dendropy/reconcile.py:        self.ultrametricity_check_prec = ultrametricity_check_prec
dendropy/reconcile.py:            self.calc_node_ages(check_prec=self.ultrametricity_check_prec)
dendropy/reconcile.py:            contained_tree.calc_node_ages(check_prec=self.ultrametricity_check_prec)
dendropy/reconcile.py:            contained_tree.calc_node_ages(check_prec=self.ultrametricity_check_prec)

I guess I can alter sumtrees.py so that it does not check that the trees are ultrametric, that may be the simplest? Otherwise maybe I did not download the development version but instead the stable version through git...
Bastien.

[Jeet Sukumaran]

Sorry Bastien, forgot to push to the GitHub repo. Correct revision is
ecca3c1.

Should point out that, currently, setting the precision to 0 ignores
precision checks altogether. This possibly violates the principle of
minimum surprise, or not. Have not decided yet, and may change it in
the final release. So, to ensure your pipeline keeps working, if you
want to skip the precision checks just set it to a large number (e.g.,
1e10).

> ...
>
> read more »

[Bastien Boussau]

Thanks Jeet, it works!

Bastien.