Tree topology p-values

[Bikash Shrestha]

I used IQ-tree for topology incongruence test and got the following result:

The p-value for all of the test except ELW, bp-RELL is greater than 0.05. If I focus particularly at the p-value of AU test, it's higher than 0.05 so I can not reject the null hypothesis that all trees are equally good explanation of data. 

My very basic question (perhaps stupid) is that why there are two p-values for each test? Since it's comparison between two topologies, shouldn't there be one p-value for each test indicating if it's significantly different or not? 

Could you please help me with an answer?

Thank you!

[Heiko Schmidt]

Dear Bikash Shrestha,

Thank you for your interest in IQ-Tree and testing trees.

> I used IQ-tree for topology incongruence test and got the following result:
>

> The p-value for all of the test except ELW, bp-RELL

In fact, ELW and bp-RELL do not provide p-values.

Here likelihoods are used in ELW to compute weights (which add to to 1.0 over all trees) or the bootstrap proportions in bp-RELL how often each tree is best for one of many RELL-bootstrap samples (which add to to 1.0 over all trees). The ones with high weights/proportions are assumed to be the best supported by the data, and the confidence set is build by collecting trees with decreasing weights/proportions until their cumulative sum hits 0.95.

> is greater than 0.05. If I focus particularly at the p-value of AU test, it's higher than 0.05 so I can not reject the null hypothesis that all trees are equally good explanation of data.

…based on their likelihood.

> My very basic question (perhaps stupid) is that why there are two p-values for each test? Since it's comparison between two topologies, shouldn't there be one p-value for each test indicating if it's significantly different or not?

There IS a p-value telling you for each tree in the tested treeset wether that tree is significantly worse/different/etc (depending on the test’s Null hypothesis of the respective test).

If all p-values are above the chosen cut-off value (typically 0.05), this means that no tree can be discarded. Or the other way around none of the trees can be preferred over the other, even if one has a higher likelihood.

It is of course not the aim of the test to find a tree (or some) which significantly worse, but to tell you IF there are any worse trees (i.e. not meeting the Null hypothesis for most test) and if so which ones.

It is, however, not very meaningful to do most of the tests with just two trees. SH and AU are tests designed operate on sets of tree. And SH actually requires that THE maximum likelihood tree is among the tested trees.

So it is very important to also keep the Null hypotheses tested and the requirements in mind, because they differ between the tests:

(a) The Kishino-Hasegawa test (KH) was designed to test whether likelihood are significantly different, without any knowledge which of the trees is the better one. Thus, the Null (H0) and alternative hypothesis (HA) are:
- H0: The two trees are equally supported by the likelihood, i.e. the expected difference between the likelihoods is 0
- HA: The two trees are not supported equally, i.e. the expected difference between the likelihoods is not 0

This means it just tests for difference, i.e. it is a 2-sided test.

However, since often KH was (and is) used to test whether the likelihood of a tree is significantly worse than that of the best tree found. In this case the 2-sided approach is not a valid option to use. The only way to use KH in such a situation is to implement it as a 1-sided test, which is the one implemented in IQ-TREE:

(b) The single sided Kishino-Hasegawa test (KH) has the following the Null (H0) and alternative hypothesis (HA):
- H0: The two trees are equally supported by the likelihood, i.e. the expected difference between the likelihoods is 0
- HA: The 2nd tree has a worse support, i.e. the expected difference between the likelihoods (L(ML)-L(2)) is larger than 0.

Note: This does not test whether the assumption 0 difference of the Null hypothesis is in anyway correct.

(c) The Shimodaira-Hasegawa test (SH) is a proper test for sets of trees with more than 2 trees, and has the following the Null (H0) and alternative hypothesis (HA):
- H0: All tested trees Tx of the treeset (including the ML tree!) are equally good explanations of the data.
- HA: Some or all trees Tx of the treeset are not equally good explanations of the data.

Note:
- THE maximum likelihood tree is required among the tested trees. If it is not, the test tells you whether and which trees are significantly worse, but that does not mean that the best tree in the set is any close to being any good or correct.
- A problem of this test is that number of trees selected in the SH test as not being significantly worse is strongly correlated with the number of trees in the treeset tested. That means, the more tree topologies are included in the test, the more trees are accepted. This is attributed to a selection bias.
- Furthermore, in the case that you test just 2 trees with SH it basically does perform a KH test.

(d) The approximately unbiased test (AU) tests the following:
H0(Ta): the expected likelihood value of tree Ta is larger or equal to the expected likelihood values for all trees Tx in the treeset.
HA(Ta): the expected likelihood value of tree Ta is smaller…

Note: If many of the best trees in your treeset have almost equal likelihoods, you may gain an over-confidence into the wrong tree(s).

For further information I suggest reading the chapter “Testing Tree Topologies” from “The Phylogenetic Handbook” (Lemey et al. 2009). It aims at being more comprehensible than the original papers.

> Could you please help me with an answer?

I hope the above helped a little.

Best wishes,
Heiko


-----------------------------------------------------------------------------
Heiko Schmidt
Center for Integrative Bioinformatics Vienna (CIBIV)
University of Vienna / Max F. Perutz Laboratories (MFPL)
Campus Vienna Biocenter 5 (VBC5)
A-1030 Vienna, Austria
-----------------------------------------------------------------------------