SegFault when computing Robinson-Foulds distances

[Chris Johns]

Hi there,

I'm trying to compute Robinson-Foulds distances between one large tree and many smaller trees (507 gene trees) via command:

raxml -f R -m GTRCAT -t 507_nuc_b100 -z alltrees.txt -n RF -T 2

I receive the following segmentation fault that terminates the run. This happens locally and with the MPI version on cluster:

Parsing reference tree 507_nuc_b100_fastest.treefile

allocated 11524 bytes

The reference tree has 40 tips

Found 507 trees in File allgenetrees

Small tree 0 has 35 tips and 32 bipartitions

Relative RF tree 0: 0.843750

Segmentation fault (11)

 Any ideas?

Thank you!

Chris

[Alexandros Stamatakis]

Hi Chris,

first time this is reported for the RF distance calculation.

Could you please send me the input files?

Thank you,

Alexis

On 23.06.2017 02:20, Chris Johns wrote:
> Hi there,
>
> I'm trying to compute Robinson-Foulds distances between one large tree
> and many smaller trees (507 gene trees) via command:
>

> *raxml -f R -m GTRCAT -t 507_nuc_b100 -z alltrees.txt -n RF -T 2*
>
> *
> *

> I receive the following segmentation fault that terminates the run. This
> happens locally and with the MPI version on cluster:
>

> *Parsing reference tree 507_nuc_b100_fastest.treefile
> **
> **allocated 11524 bytes
> **
> **The reference tree has 40 tips
> **
> **Found 507 trees in File allgenetrees
> **
> **Small tree 0 has 35 tips and 32 bipartitions
> **
> **Relative RF tree 0: 0.843750
> **
> **Segmentation fault (11)*

>
>
> Any ideas?
>
> Thank you!
> Chris

> *
> *
>
> --
> You received this message because you are subscribed to the Google
> Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to raxml+un...@googlegroups.com
> <mailto:raxml+un...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

--
Alexandros (Alexis) Stamatakis

Research Group Leader, Heidelberg Institute for Theoretical Studies
Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology
Adjunct Professor, Dept. of Ecology and Evolutionary Biology, University
of Arizona at Tucson

www.exelixis-lab.org

[Alexandros Stamatakis]

it seems though that the -t option is unnecessary as RAxML will only
compute RF distances among all trees in the file provided by -z,

alexis

On 23.06.2017 02:20, Chris Johns wrote:

> Hi there,
>
> I'm trying to compute Robinson-Foulds distances between one large tree
> and many smaller trees (507 gene trees) via command:
>

> *raxml -f R -m GTRCAT -t 507_nuc_b100 -z alltrees.txt -n RF -T 2*
>
> *
> *

> I receive the following segmentation fault that terminates the run. This
> happens locally and with the MPI version on cluster:
>

> *Parsing reference tree 507_nuc_b100_fastest.treefile
> **
> **allocated 11524 bytes
> **
> **The reference tree has 40 tips
> **
> **Found 507 trees in File allgenetrees
> **
> **Small tree 0 has 35 tips and 32 bipartitions
> **
> **Relative RF tree 0: 0.843750
> **
> **Segmentation fault (11)*
>
>

> Any ideas?
>
> Thank you!
> Chris

[Chris Johns]

Emailed! Thank you Alexis. 

[Alexandros Stamatakis]

dear chris,

thanks for sending the data, raxml does provide an error message that
explains why the RF calc fails:

ERROR: Cannot find tree species: Cameraria_ohridella.DP14051
The species names in the input tree and alignment file may not match,
please check!
raxmlHPC-AVX: treeIO.c:1409: treeReadLen: Assertion `0' failed.

So the reason why this is faling is that the gene trees apparently have
different sizes and distinct taxon subsets. The RAxMl RF calculation
option requires you ta provide trees with exactly the same size and
taxon set as input.

Alexis

> > an email to raxml+un...@googlegroups.com <javascript:>
> > <mailto:raxml+un...@googlegroups.com <javascript:>>.

> > For more options, visit https://groups.google.com/d/optout

> <https://groups.google.com/d/optout>.

>
> --
> Alexandros (Alexis) Stamatakis
>
> Research Group Leader, Heidelberg Institute for Theoretical Studies
> Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology
> Adjunct Professor, Dept. of Ecology and Evolutionary Biology,
> University
> of Arizona at Tucson
>

> www.exelixis-lab.org <http://www.exelixis-lab.org>

[Chris Johns]

Hi Alexis,

Thanks for the reply! I did not see that extra error info on my end. I checked my last run for RF that was successful and there were gene trees of varying sizes there. For example, from the output of the successful RF calculation: 

Parsing reference tree alltrees.suptree

The reference tree has 62 tips

Found 503 trees in File allgenetrees

Small tree 0 has 51 tips and 48 bipartitions

Relative RF tree 0: 0.750000

Small tree 100 has 57 tips and 54 bipartitions

Relative RF tree 100: 0.629630

Small tree 200 has 53 tips and 50 bipartitions

Relative RF tree 200: 0.800000

Small tree 300 has 53 tips and 50 bipartitions

Relative RF tree 300: 0.800000

Small tree 400 has 44 tips and 41 bipartitions

Relative RF tree 400: 0.585366

Small tree 500 has 57 tips and 54 bipartitions

Relative RF tree 500: 0.722222

Number of small trees skipped: 0

So I'm a bit confused. Does RAxML allow varying gene tree sizes? 

Thanks!

Chris

[Alexandros Stamatakis]

Dear Chris,

Could you please send me the command line and input files that generated
this output?

Thanks,

Alexis

On 28.06.2017 16:07, Chris Johns wrote:
> Hi Alexis,
>
> Thanks for the reply! I did not see that extra error info on my end. I
> checked my last run for RF that was successful and there were gene trees
> of varying sizes there. For example, from the output of the successful
> RF calculation:
>

> *Parsing reference tree alltrees.suptree*
> *The reference tree has 62 tips*
> *
> *
> *Found 503 trees in File allgenetrees*
> *
> *
> *Small tree 0 has 51 tips and 48 bipartitions*
> *Relative RF tree 0: 0.750000*
> *
> *
> *Small tree 100 has 57 tips and 54 bipartitions*
> *Relative RF tree 100: 0.629630*
> *
> *
> *Small tree 200 has 53 tips and 50 bipartitions*
> *Relative RF tree 200: 0.800000*
> *
> *
> *Small tree 300 has 53 tips and 50 bipartitions*
> *Relative RF tree 300: 0.800000*
> *
> *
> *Small tree 400 has 44 tips and 41 bipartitions*
> *Relative RF tree 400: 0.585366*
> *
> *
> *Small tree 500 has 57 tips and 54 bipartitions*
> *Relative RF tree 500: 0.722222*
> *
> *
> *Number of small trees skipped: 0*

www.exelixis-lab.org

[Chris A. Johns]

Hi Alexis,

Files are attached.

Thank you!

Chris

--
Chris A. Johns
Department of Biology
McGuire Center for Lepidoptera and Biodiversity
Florida Museum of Natural History
University of Florida
www.chrisajohns.com | johns....@gmail.com

To unsubscribe from this group and stop receiving emails from it, send an email to raxml+unsubscribe@googlegroups.com <mailto:raxml+unsubscribe@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.

--
Alexandros (Alexis) Stamatakis

Research Group Leader, Heidelberg Institute for Theoretical Studies
Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology

www.exelixis-lab.org

--
You received this message because you are subscribed to a topic in the Google Groups "raxml" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/raxml/olb8O3R3RmI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to raxml+unsubscribe@googlegroups.com.

[Alexandros Stamatakis]

Dear Chris,

Thanks, now I see what is happening.

You are using the -f R option and not the standard -f r option.
This is also why this does not exit with an error, unlike what I expected.

As specified in the on-line help -f R does the following:

"-f R": compute all pairwise Robinson-Foulds (RF) distances between a
large reference tree passed via "-t" and many smaller trees
(that must have a subset of the taxa of the large tree) passed
via "-z".
This option is intended for checking the plausibility of very
large phylogenies that can not be inspected
visually any more.

The details of this algorithm are described here:

https://sco.h-its.org/exelixis/pubs/DavidDaoBachelorThesis.pdf

To be able to compute RF distances between trees where one tree has a
subset of the leaves of another tree, some additional computations are
required. The -f r option only calculates the RF between trees with
exactly identical sets of taxa.

In any case, at first I thought that you were using -f r, but you are
using -f R which I didn't notice.

So, if you are using -f R, the tree you pass via -t must contain all
taxa that are contained in the gene trees passed via -z.

In other words, all trees passed via -z must contain a subset of the
taxa of the tree passed via -t. This does not seem to be the case here,
and hence the error you get.

Hope this helps,

Alexis

On 04.07.2017 13:46, Chris A. Johns wrote:
> Hi Alexis,
>
> Files are attached.
>
> Thank you!
> Chris
>
>
> --
> Chris A. Johns
> Department of Biology
> McGuire Center for Lepidoptera and Biodiversity
> Florida Museum of Natural History
> University of Florida

> www.chrisajohns.com <http://www.chrisajohns.com> |
> johns....@gmail.com <mailto:johns....@gmail.com>

>
> On Tue, Jul 4, 2017 at 5:50 AM, Alexandros Stamatakis
> <alexandros...@gmail.com

> <mailto:raxml%2Bun...@googlegroups.com> <javascript:>
> > > <mailto:raxml+un...@googlegroups.com
> <mailto:raxml%2Bun...@googlegroups.com> <javascript:>>.

> <mailto:raxml%2Bun...@googlegroups.com> <javascript:>
> > <mailto:raxml+un...@googlegroups.com
> <mailto:raxml%2Bun...@googlegroups.com> <javascript:>>.

> > For more options, visit
> https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>
> <https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>>.
>
> -- Alexandros (Alexis) Stamatakis
>
> Research Group Leader, Heidelberg Institute for Theoretical
> Studies
> Full Professor, Dept. of Informatics, Karlsruhe Institute
> of Technology
> Adjunct Professor, Dept. of Ecology and Evolutionary Biology,
> University
> of Arizona at Tucson
>
> www.exelixis-lab.org <http://www.exelixis-lab.org>
> <http://www.exelixis-lab.org>
>
> --
> You received this message because you are subscribed to the
> Google Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from
> it, send an email to raxml+un...@googlegroups.com

> <mailto:raxml%2Bunsu...@googlegroups.com>
> <mailto:raxml+un...@googlegroups.com
> <mailto:raxml%2Bunsu...@googlegroups.com>>.

> For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
>
>
> --
> Alexandros (Alexis) Stamatakis
>
> Research Group Leader, Heidelberg Institute for Theoretical Studies
> Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology
>

> www.exelixis-lab.org <http://www.exelixis-lab.org>
>
> --

> You received this message because you are subscribed to a topic in
> the Google Groups "raxml" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/raxml/olb8O3R3RmI/unsubscribe

> <https://groups.google.com/d/topic/raxml/olb8O3R3RmI/unsubscribe>.

> To unsubscribe from this group and all its topics, send an email to

> raxml+un...@googlegroups.com
> <mailto:raxml%2Bunsu...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
>
>
> --

> You received this message because you are subscribed to the Google
> Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to raxml+un...@googlegroups.com

> <mailto:raxml+un...@googlegroups.com>.

[Chris A. Johns]

Thank you Alexis this helps a lot! Your time is much appreciated. 

--
Chris A. Johns
Department of Biology
McGuire Center for Lepidoptera and Biodiversity
Florida Museum of Natural History
University of Florida

www.chrisajohns.com | johns....@gmail.com

        it, send an email to raxml+unsubscribe@googlegroups.com
        <mailto:raxml%2Bunsubscribe@googlegroups.com>
        <mailto:raxml+unsubscribe@googlegroups.com
        <mailto:raxml%2Bunsubscribe@googlegroups.com>>.

        For more options, visit https://groups.google.com/d/optout
        <https://groups.google.com/d/optout>.


    --     Alexandros (Alexis) Stamatakis

    Research Group Leader, Heidelberg Institute for Theoretical Studies
    Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology

    www.exelixis-lab.org <http://www.exelixis-lab.org>

    --     You received this message because you are subscribed to a topic in
    the Google Groups "raxml" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/topic/raxml/olb8O3R3RmI/unsubscribe
    <https://groups.google.com/d/topic/raxml/olb8O3R3RmI/unsubscribe>.

    To unsubscribe from this group and all its topics, send an email to

    raxml+unsubscribe@googlegroups.com
    <mailto:raxml%2Bunsubscribe@googlegroups.com>.

    For more options, visit https://groups.google.com/d/optout
    <https://groups.google.com/d/optout>.


--

You received this message because you are subscribed to the Google Groups "raxml" group.

To unsubscribe from this group and stop receiving emails from it, send an email to raxml+unsubscribe@googlegroups.com <mailto:raxml+unsubscribe@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.

--
Alexandros (Alexis) Stamatakis

Research Group Leader, Heidelberg Institute for Theoretical Studies
Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology

www.exelixis-lab.org

--
You received this message because you are subscribed to a topic in the Google Groups "raxml" group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/raxml/olb8O3R3RmI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to raxml+unsubscribe@googlegroups.com.

[Jacek Kominek]

Hi, I wanted to resurrect this issue, since I am encountering a similar problem in my dataset. 

I am using "-f R" to run a comparison between a species tree inferred from a concatenated superalignment and ~1500 individual gene trees. What happens is that RAxML (the latest github version) segfaults if one of the tree provided in the file passed via "-z" contains ALL of the taxa contained in the tree passed via "-t". I know that the "-f R" option is intended to test the trees with fewer taxa than in the "reference tree", but it definitely shouldn't segfault in this case, especially since I tested and it can calculate the RF distance between the offending tree and reference just fine when run using "-f r". Moreover, the immediate output is not very helpful, because in my case it broke off at tree #953. Since it stopped mid-line, I figured it might be due to lag in printing from the stdout buffer, and it took me some further testing to figure out that the offending tree was actually #1034 - once removed, all calculations worked like a charm.

Since this can be worked around by just splitting the dataset into 2 pieces and running them separately, it is not a gamebreaking bug by any means, but it's a weird an unclear situation nonetheless, and I believe it is a relatively easy fix - looking around it seems to be in the plausibilityChecker function in bipartitionList.c, since it's a segfault, my blind guess would be that one of the malloc's probably asks for too little of memory to cover the complete tree. Hope that helps, I can send some sample data, if necessary.

        it, send an email to raxml+un...@googlegroups.com
        <mailto:raxml%2Bu...@googlegroups.com>
        <mailto:raxml+un...@googlegroups.com
        <mailto:raxml%2Bu...@googlegroups.com>>.

        For more options, visit https://groups.google.com/d/optout
        <https://groups.google.com/d/optout>.


    --     Alexandros (Alexis) Stamatakis

    Research Group Leader, Heidelberg Institute for Theoretical Studies
    Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology

    www.exelixis-lab.org <http://www.exelixis-lab.org>

    --     You received this message because you are subscribed to a topic in
    the Google Groups "raxml" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/topic/raxml/olb8O3R3RmI/unsubscribe
    <https://groups.google.com/d/topic/raxml/olb8O3R3RmI/unsubscribe>.

    To unsubscribe from this group and all its topics, send an email to

    raxml+un...@googlegroups.com
    <mailto:raxml%2Bu...@googlegroups.com>.

    For more options, visit https://groups.google.com/d/optout
    <https://groups.google.com/d/optout>.


--

You received this message because you are subscribed to the Google Groups "raxml" group.

To unsubscribe from this group and stop receiving emails from it, send an email to raxml+un...@googlegroups.com <mailto:raxml+un...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.

--
Alexandros (Alexis) Stamatakis

Research Group Leader, Heidelberg Institute for Theoretical Studies
Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology

www.exelixis-lab.org

--
You received this message because you are subscribed to a topic in the Google Groups "raxml" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/raxml/olb8O3R3RmI/unsubscribe.

To unsubscribe from this group and all its topics, send an email to raxml+un...@googlegroups.com.

[Alexandros Stamatakis]

Thanks a million Jacek,

I'll put that on my list, unfortunately there is a huge backlog.

Hope you are well,

Alexis

> www.chrisajohns.com <http://www.chrisajohns.com> |
> johns....@gmail.com <javascript:>

> <http://www.chrisajohns.com> | johns....@gmail.com
> <javascript:> <mailto:johns....@gmail.com <javascript:>>

>
>
> On Tue, Jul 4, 2017 at 5:50 AM, Alexandros Stamatakis

> <alexandros...@gmail.com <javascript:>

> <javascript:>
>         <mailto:raxml%2Bu...@googlegroups.com <javascript:>>
>         <mailto:raxml+un...@googlegroups.com <javascript:>
>         <mailto:raxml%2Bu...@googlegroups.com <javascript:>>>.

>         For more options, visit
> https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>
>         <https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>>.
>
>
>     --     Alexandros (Alexis) Stamatakis
>
>     Research Group Leader, Heidelberg Institute for
> Theoretical Studies
>     Full Professor, Dept. of Informatics, Karlsruhe
> Institute of Technology
>

> www.exelixis-lab.org <http://www.exelixis-lab.org>
> <http://www.exelixis-lab.org>
>
>     --     You received this message because you are

> subscribed to a topic in
>     the Google Groups "raxml" group.
>     To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/raxml/olb8O3R3RmI/unsubscribe
> <https://groups.google.com/d/topic/raxml/olb8O3R3RmI/unsubscribe>
>
> <https://groups.google.com/d/topic/raxml/olb8O3R3RmI/unsubscribe
> <https://groups.google.com/d/topic/raxml/olb8O3R3RmI/unsubscribe>>.
>     To unsubscribe from this group and all its topics, send
> an email to

> raxml+un...@googlegroups.com <javascript:>
>     <mailto:raxml%2Bu...@googlegroups.com <javascript:>>.

>     For more options, visit
> https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>
>     <https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>>.
>
>
> --

> You received this message because you are subscribed to the
> Google Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails
> from it, send an email to raxml+un...@googlegroups.com

> <javascript:> <mailto:raxml+un...@googlegroups.com

> <javascript:>>.
> For more options, visit https://groups.google.com/d/optout

> <https://groups.google.com/d/optout>.
>
>
> --
> Alexandros (Alexis) Stamatakis
>
> Research Group Leader, Heidelberg Institute for Theoretical Studies
> Full Professor, Dept. of Informatics, Karlsruhe Institute of
> Technology
>
> www.exelixis-lab.org <http://www.exelixis-lab.org>
>
> --
> You received this message because you are subscribed to a topic
> in the Google Groups "raxml" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/raxml/olb8O3R3RmI/unsubscribe
> <https://groups.google.com/d/topic/raxml/olb8O3R3RmI/unsubscribe>.
> To unsubscribe from this group and all its topics, send an email

> to raxml+un...@googlegroups.com <javascript:>.

> For more options, visit https://groups.google.com/d/optout

[Jacek Kominek]

I'm good, thanks Alexis, and no problem, always happy to help.