Approximately Unbiased test returns p-value of 1.0 while other tests are significant

[Dan Proud]

I want to use the Approximately Unbiased test to compare tree topologies. I want to compare the Maximum Likelihood tree to multiple other trees that have constraints on the monophyly of the taxa of interest.

I obtained four trees using likelihood methods, and used those as the input in IQ-TREE to run the AU test. I'm actually most interested in comparing trees #2 and #3 (alternative phylogenetic hypotheses) to tree #1 (the ML tree) shown below. Tree #4 is a tree in which I included monophyletic constraints that I knew would produce a tree that was much worse than the ML tree. So the fourth tree is expected to differ significantly.

I'm noticing a strange result with respect to the behavior of the AU test. I will share the results here, and if someone wishes to look at the input data files I'm happy to share them by email.

USER TREES

----------

See belo509raxml_8loci_r1.phy.trees for trees with branch lengths.

Tree      logL    deltaL  bp-RELL    p-KH     p-SH    c-ELW     p-AU

-------------------------------------------------------------------------

  1   -42271.306   0.000  0.5485 + 0.5937 + 1.0000 + 0.5457 + 0.5115 + 

  2   -42272.563   1.257  0.4140 + 0.4063 + 0.7919 + 0.4154 + 0.4794 + 

  3   -42296.487  25.181  0.0375 - 0.0546 + 0.3303 + 0.0389 - 0.4705 + 

  4   -42736.708 465.401  0.0000 - 0.0000 - 0.0000 - 0.0000 - 1.0000 + 

deltaL  : logL difference from the maximal logl in the set.

bp-RELL : bootstrap proportion using RELL method (Kishino et al. 1990).

p-KH    : p-value of one sided Kishino-Hasegawa test (1989).

p-SH    : p-value of Shimodaira-Hasegawa test (2000).

c-ELW   : Expected Likelihood Weight (Strimmer & Rambaut 2002).

p-AU    : p-value of approximately unbiased (AU) test (Shimodaira, 2002).

Plus signs denote the 95% confidence sets.

Minus signs denote significant exclusion.

All tests performed 10000 resamplings using the RELL method.

So just to be sure that I understand this correctly, the AU test compares each of the user trees to the user tree that has the greatest logL. In other words, I am making three comparisons: T2-T1, T3-T1, and T4-T1. Is that correct?

Then I would interpret these results to say that T4 is significantly worse than T1 based on all tests except p-AU. For that test the p-value is 1.000. This seems strange to me. So I'm writing to ask whether this is an error in the way that I'm running the analysis, or interpreting the results, or whether there is a problem with the calculation of p-AU?

I have compared the results from IQ-TREE v1.6.6 on XSEDE to those obtained using the IQ-TREE web server v1.6.6. Using both platforms, I am experiencing the same problem. I tried to run the same dataset using v1.6.2 on XSEDE, however, it completes the tree inference analysis, and when it arrives at the user trees it gives me an error in the log file: "Tree 1ERROR: #leaves and taxa_set do not match!" (even though the same exact files do work in 1.6.6).

Thank you in advance for your help!

[Minh Bui]

Dear Dan,

The AU test implementation in IQ-TREE is currently suspended due to some fundamental problem, which I couldn’t clarify  with Shimodaira, CONSEL author. It happens when bp-RELL is zero (see discussion: https://groups.google.com/forum/#!topic/iqtree/Dw4rl-SYxqY). So pls use CONSEL instead.

Regarding:

"Tree 1ERROR: #leaves and taxa_set do not match!" (even though the same exact files do work in 1.6.6).

Yes this problem was fixed in v1.6.6

Cheers

Minh

--
You received this message because you are subscribed to the Google Groups "IQ-TREE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to iqtree+un...@googlegroups.com.
To post to this group, send email to iqt...@googlegroups.com.
Visit this group at https://groups.google.com/group/iqtree.
For more options, visit https://groups.google.com/d/optout.

[Dan Proud]

Thank you for the clarification, Minh.

[Ales Bucek]

Hi,
I run into the same problem with iqtree 1.6.8 where, as I understood, the problem with estimation of p-values of AU test was fixed. Topology test was run as follows:

iqtree-1.6.8-Linux/bin/iqtree -s ${WorkingDir}/${InputAlignment} -nt ${SLURM_CPUS_PER_TASK} -spp ${WorkingDir}/${PartitioningScheme} -z ${WorkingDir}/TreeGraph_trees.tre -te ${WorkingDir}/Fin24.treefile -zb 10000 -zw -au -pre ${SLURM_JOB_NAME}

Tree      logL    deltaL  bp-RELL    p-KH     p-SH    p-WKH    p-WSH       c-ELW       p-AU
-------------------------------------------------------------------------------------------
  1 -5641283.195  414.44       0 -      0 -      0 -      0 -      0 -  1.5e-112 -        1 +
  2 -5641334.838  466.08       0 -      0 -      0 -      0 -      0 - 2.95e-120 -        1 +
  3 -5641324.895  456.14       0 -      0 -      0 -      0 -      0 - 8.73e-118 -        1 +
  4 -5641352.307  483.55       0 -      0 -      0 -      0 -      0 - 1.62e-124 -        1 +
  5 -5641387.955   519.2       0 -      0 -      0 -      0 -      0 - 1.88e-143 -        1 +
  6 -5640916.227  47.473  0.0065 - 0.0082 -  0.307 + 0.0082 - 0.0278 -   0.00659 -    0.458 +
  7 -5641312.867  444.11       0 -      0 -      0 -      0 -      0 - 2.15e-129 -        1 +
  8 -5641383.439  514.69       0 -      0 -      0 -      0 -      0 - 4.77e-152 -        1 +
  9 -5640868.754       0   0.994 +  0.992 +      1 +  0.992 +      1 +     0.993 +    0.542 +
 10 -5641393.774  525.02       0 -      0 -      0 -      0 -      0 - 8.57e-152 -        1 +

I'd be happy for any suggestions. I can share more details on inputs/outputs.

Best,

Ales

[Minh Bui]

[Minh Bui]

Hi Ales,

This problem should have been fixed, not sure why…. Can you please send me the dataset via personal email? 

Thanks,

Minh

On 17 Dec 2018, at 4:48 pm, Ales Bucek <bucek...@gmail.com> wrote:

[Karen Meusemann]

Dear Minh,

can you pls quickly check whether this was also an issue in earlier versions then (below 1.6.6 - 1.6.5, 1.6.4 and 1.5.5?) I need to know this since I am writing together a  MS and when IO did the AU tests, it was a while ago - I need to check whether I have to repeat everything or not...

Many thanks, see also other question on the Au test and quartet mapping pls!

Many thanks Karen

[Minh Bui]

Hi Karen again, Yes this problem with v1.6.8 was fixed in v1.6.9, see the release note: http://www.iqtree.org/news/:

• IMPORTANT: Incorrect AU test p-values with partition models, introduced in v1.6.8 (reported by Ales Bucek).

That means, it does not affect any other version.

Minh