best ML tree vs cons tree
Giorgio Matassi
Dear All,
What to do if the topologies of best ML (bipartitionsBranchLabels) and MRE trees differ?
Are rapid-BS values and those on the MRE tree comparable (eg in terms of significance; ie >75%)?
Best,
Giorgio
Alexandros Stamatakis
> What to do if the topologies of best ML (bipartitionsBranchLabels) and
> MRE trees differ?
more data in terms of genes, i.e., make the alignment longer ...
> Are rapid-BS values and those on the MRE tree comparable (eg in terms of
> significance; ie >75%)?
built from the rapid BS replicates, then the values are directly
comparable and in principle for rapid BS you can assume the typical BS
thresholds ...
alexis
>
> Best,
>
> Giorgio
>
> --
> You received this message because you are subscribed to the Google
> Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to raxml+un...@googlegroups.com
> <mailto:raxml+un...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.
--
Alexandros (Alexis) Stamatakis
Research Group Leader, Heidelberg Institute for Theoretical Studies
Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology
Adjunct Professor, Dept. of Ecology and Evolutionary Biology, University
of Arizona at Tucson
www.exelixis-lab.org
Grimm
in case you don't have the money to add the number of genes require to resolve this (which can be quite a lot, depending on the reason for ML failing to optimise a best-known tree which is not exactly the topology preferred in the bootstrap samples), you may see what are the topological alternatives competing with each other.
Here's what you do:
Rather than summing up the bootstrap replicate sample using a majority-rule consensus tree (which generally has little information), you sum them up using a consensus network.
Then you can see which alternatives compete with the low supported branches in your tree.
You either read in the RAxML bootstrap sample file (RAxML_bootstrap...) in SplitsTree: start SplitsTree -> open the RAxML_bootstrap... file -> Choose Consensus Network from the Menu -> And "COUNT" option in the popping-up window, as threshold you can choose whatever you want (if you have quite diffuse signals, don't go too low, because the number of competing splits with low frequencies in the bootstrap samples usually increases very much)
Or, if you a familiar, with R, you can do the whole lot with the new functions in the PHANGORN library.
Using the result you can then depict the alternative topologies to that in the RAxML's best-known tree, and either using RAxMLs in-built SH test or the AU-test you can check if those topologies are really substantially worse than RAxML's best known.
Cheers, G.
Romina Batista
Romina Batista
Giorgio Matassi
First of
all, please accept, Alexis, Grim and Romina, my most sincere apologies for not
following my own post.
The reason is that I completely forgot about it. It looks as if it is about one-year-old (gosh!).
I had
forgotten it to the point that I am currently discussing the very same issue (on
less naïve
grounds compared to my first question) with Colleagues in Lyon and Vienna.
The point
is still what to choose between best-ML tree and consensus tree. But also how
to construct
the consensus tree; and I am not convinced that using the bootstrapped
trees is really the best option.
I have been comparing best-ML tree and the corresponding consensus tree with different distance
metrics coupled to the usual statistics (AU, ELW; etc). In my test
data set (a difficult one, on purpose)
the differences are relevant (frightening?).
If interested to pursue this issue, we could open a new forum-topic.
Thanks for your time and suggestions.
Sorry again.
Best,
Giorgio
Grimm
The point is still what to choose between best-ML tree and consensus tree.
It depends on what you want:
If you want a tree with a topology that best fits your data, you choose the best-ML tree. To find the most optimal tree a single run may not be enough, see e.g. Alexi's somewhat undercited paper on how-to-set-up a ML analysis with RAxML
Using RAxML to Infer Phylogenies
http://onlinelibrary.wiley.com/doi/10.1002/0471250953.bi0614s51/abstract;jsessionid=7FCC8285AA7649F7AC1490341632A84F.f04t02?userIsAuthenticated=false&deniedAccessCustomisedMessage=
If you are interested in how strong the support for (potentially competing) (taxon) bipartitions is, and why some branches have high and others low boostrap support, you can look at the bootstrap support network, the consensus network of the bootstrap pseudoreplicates. This is what David Morrison called "explanatory data analysis", and it's crucial when you work with non-trivial data (although it is not very common to do it; I did for over 10 years, and it brought me about 75% misery and 25% encouraging comments from the reviewers' side)
To transfer information between support methods and trees, network functionality has been implemented in the Phangorn library for R (the final paper is now out: http://dx.doi.org/10.1111/2041-210X.12760)
Re: consensus trees, my opinion is that they are pretty pointless these days. I don't know a single case (real-world or simulated data) for using a consensus tree. Clear data sets will result in a clear tree, if this fails, then the data is not clear and usually has non-treelike signal, so one would not search for just one tree.
Consensus trees were helpful in the dawn of phylogenetics, when we only had parsimony (because distances were banned as "phenetic") to get something that looks like a tree out of some or thousands of equally parsimonious trees; they had a renaissance with Bayesian inference, because it is (or was) the most handy way to sum-up the 10,000 or more post-burn saved topologies. However, having worked a lot with complex data, I cannot recommend to use a majority rule (or other) consensus tree even for this purpose. I would either take a 0.95 (or even 1.0, i.e. strict consensus)-cut-off tree, but preferably use the consensus network also here ("PP network"). In case the number of leaves/split patterns skyrock (as in Rominas problem) so that it's mechanically impossible to calculate the network with current implementations, I would map the PP simply on the ML-optimised tree.
But also how to construct the consensus tree; and I am not convinced that using the bootstrapped trees is really the best option.
See above. Why do you want to make a consensus tree at all? If you need alternative topologies, rather infer a set of trees, e.g. using different sets of data or taxon sets (e.g. with and without rogues, excluding taxa with a lot of missing data etc), and test at which point they become statistically worse than the main, comprehensive ML tree.
I have been comparing best-ML tree and the corresponding consensus tree with different distance metrics coupled to the usual statistics (AU, ELW; etc). In my test data set (a difficult one, on purpose)
the differences are relevant (frightening?).
For me "difficult data" means complex signals. So you have a tree with low supported branches, and data that aspect-wise prefers competing (alternative) topologies? Then it's really not frightenting, but to be expected. The less tree-like the signal in the primary data, the higher are the chances that the optimised ML-tree conflicts with best-supported alternatives in the bootstrap sample.
The effect may be even stronger (depending on the data used) when ML-tree is compared to the Bayesian-preferred topology. Just a relative recent example regarding ambiguous signal (one of the last cases I had to work with). Su et al (2015, Taxon 64:491–506) published a Santales tree based on up-to-seven genes. The matrix is a swiss cheese, the tree includes several extremely long-branches. Nevertheless, many backbone branches were well supported, and the authors note that there were little differences between the ML-optimised tree and the Bayesian consensus tree. So the case seems to be unproblematic. However, the branches we had to discuss for our paper (on the pollen of one of the Santalaes families, Grímsson et al., 2017, http://dx.doi.org/10.1080/00173134.2016.1261939) had quite low BS (<50) but high PP (>0.9). In course of the battle with one reviewer, I ended up writing an essay (attached, became File S6) about the signal in that seven gene matrix for the group we were interested in, pointing out that their high PP but low BS were due to the fact that only one of the seven genes, the best-sampled and most variable, actually supported the proximal branches in the preferred tree, and the other where either neutral or strongly opposing it.
So their 2015 ML tree may have been the indeed the best tree regarding the complete data, any other tree would have been in too stark conflict with the signal in the one dominating gene. The PP >> BS because Bayesian inference optimises (like the ML tree) towards the topology that best explains all the data. On the other hand, the BS pseudoreplicates captured the conflicting signal from nuclear vs. plastid genes, and slowest vs. fastest evolving plastid genes. Using a series of ML inferences and bootstrap support (consensus) networks, it's relatively simple to sort this out
Cheers, G
Romina Batista
Dear all,
I am very grateful for this pleasant discussion. Grim, thanks for such detailed explanation and for share very nice papers. Specially the one about Phangorn library in R. It is a very good example to support the idea (personal at least!) how important is an alternative topology as a consensus network.
“(although it is not very common to do it; I did for over 10 years, and it brought me about 75% misery and 25% encouraging comments from the reviewers' side)”
- I hope this scenario will change with genomic data sets getting so popular, and with the insights these kind of data can recover. Personal thoughts from someone who is reviewing own manuscript, using these kind of approaches.
- I managed to construct a SuperNetwork, using a filtered data set (‘only’ 335 loci). As Grim mentioned it can be impossible try to calculate with so many leaves (my case n=113, and thousands of loci).
- Giorgio, one very nice approach, I am also using, to complement the SuperNetwork overview (the one I have has a reticulated scenario of a rapid radiation event!) is the internode certainty, IC (using RaxML, see Manual X. Computing TC and IC values, pg. 49-53). As Grim mentioned “If you are interested in how strong the support for (potentially competing) (taxon) bipartitions is, and why some branches have high and others low boostrap support” you can then calculate the IC score, by calculating the frequency of each bipartition in a “reference tree” (let’s say the bestML tree). The method is very well explained here: https://academic.oup.com/mbe/article-lookup/doi/10.1093/molbev/msw040. I think it is a very helpful output!
Thanks a lot for your attention and help,
All the best, Romina B.Giorgio Matassi
Hi,
>The
point is still what to choose between best-ML tree and consensus tree.
>>It depends on what you want:
Grimm: OK, I got the point and fully agree.
Yet, a
related issue is how to construct the consensus tree (see below)
> Why do you want to make a consensus tree at all?
"explanatory data analysis" for example, as you rightly suggest.
> you can look at the bootstrap support network, the consensus network of the bootstrap >pseudoreplicates.
Will definitely try this out.
And I
recommend to the followers of this post, both beginners and confirmed in
phylogenetics, to read the excellent paper by Grimm and colleagues, http://dx.doi.org/10.1111/2041-210X.12760)
>This is what David Morrison called "explanatory data analysis", and it's crucial when you work >with non-trivial data
Absolutely!
> I did for over 10 years,
Well then, if you are willing to help, you are most welcome.
The phylogeny I am working on is one of the most difficult
ever; guess what? Hox genes!
Warning: it is likely to bring more misery.
> So you have a tree with low supported branches, and data that aspect-wise
prefers competing >(alternative) topologies? Then it's really not
frightenting, but to be expected.
I’ve used frightening in a somewhat different meaning. Example: bestML and corresponding contree (my data set) are “frighteningly” different (normalized RF of about 0.4). Same to be said for equivalent trees according to AU (my data set). Topology differs; interpretation too.
>The less tree-like the signal in the primary data, the higher are the
chances that the optimised
>ML-tree conflicts with best-supported
alternatives in the bootstrap sample.
Sure!
>Giorgio,
one very nice approach, I am also using, to complement the SuperNetwork
overview (the
>one I have has a reticulated scenario of a rapid radiation
event!) is the internode certainty, IC
>(using RaxML, see Manual X. Computing TC and IC values, pg. 49-53).
Romina: Yes, but “One potential drawback when applying the IC … is that their values may not be representative when small numbers of characters or gene trees are used. … our measures are likely most informative when applied to large amounts of data (e.g., hundreds of characters or dozens of genes or hundreds of bootstrap replicates).” (Salichos et al 2016 MBE 31:1261)
And this is exactly my case (single
gene, few characters); anyhow I’d given it a try already,
but am unsure how to interpret the results (as expected)
*********
I know you guys have been through all this for a
few years, yet, given the number of
followers of this post, the issue seems to be of interest still.
Many issues are at stake here. Will mention just one of them.
We all know that we have to deal with another crucial
issue: the starting-tree bias.
There are four options I know of for start-tree
choice (implemented in different
programs): BioNJ, MP, random, bestML.
As Grimm reminds us “The bootstrap replicates
can be used either to compute consensus
trees (contree) of various flavors or
to draw confidence values onto a reference tree, e.g.,
the best-scoring ML tree.”
Suppose we still want to
construct a contree. There are two ways of computing it: from bootstrap trees
or from bestMLtrees from independent runs. (independent runs; another crucial
issue by the way).
What are we to expect comparing the two?
Intuitively, I prefer the
second option (looks like a better tree-space search procedure to me).
What I
would do is: use all four start-tree types and X independent runs each.
I think I remember seeing a similar suggestion in some old RAxML doc (but may be mistaken).
Does this make sense to you?
All the best,
Giorgio
On Monday, May 30, 2016 at 4:05:56 PM UTC+2, Giorgio Matassi wrote:
Grimm
"Yet, a related issue is how to construct the consensus tree (see below)....0.4 RF distances" -- you could try using the Adams consensus tree approach. In constrast to other consensus tree approaches, the Adams consensus allows identifying the leaves/subtrees that are behind the topology ambiguity. It's probably the best trade-off between the strict consensus tree and the maj-rule.
The RF distances can be high because of many branches slightly differening or few branches strongly differing. But the most comprehensive approach is the consensus network as it simply contains all/more frequent trees/branches (depending on the setting)
"Well
then, if you are willing to help, you are most welcome." -- Thanks, but I would need to be paid in cash and per hour. I'm not on taxpayers' subsidaries anymore.
"And this is exactly my case (single gene, few characters)" -- If the likelihood surface is very flat because there is only little discriminating signal in the matrix including the risk that some sequences may be ancestral (composition-wise or in reality) to other sequences in the matrix, a better option may be to retreat to parsimony-based network methods designed for populations genetics such as the Median-Joining or Statistical Parsimony. Median-joining has occassionally been used at the genus level or above.
Good luck, anyway.
G
Alexandros Stamatakis
Just a quick comment from my side as I am on paternity leave:
>
> What I would do is: use all four start-tree types and X independent runs
> each.
tree (fro the same distance measure evidently).
Othwerise, I would agree, x random starting trees, x parsimony starting
trees, assess their differences and then maybe do more tree inferences.
> I think I remember seeing a similar suggestion in some old RAxML doc
> (but may be mistaken).
Alexis
>
> Does this make sense to you?
>
> All the best,
>
> Giorgio
>
>
>
> On Monday, May 30, 2016 at 4:05:56 PM UTC+2, Giorgio Matassi wrote:
>
> Dear All,
>
> What to do if the topologies of best ML (bipartitionsBranchLabels)
> and MRE trees differ?
>
>
> Are rapid-BS values and those on the MRE tree comparable (eg in
> terms of significance; ie >75%)?
>
> Best,
>
> Giorgio
>