"-­f k" mode and more than 128 partitions - segmentation fault

[Artem Pankin]

Hi, 

When I tried running RAxML 8.2.11 in the "-f k" mode with > 128 partitions it returned the following statement. 

"You are trying to use 353 partitioned models for an individual per-gene branch length estimate.

Currently only 128 are allowed to improve efficiency.

In order to change this please replace the line "#define NUM_BRANCHES   128" in file "axml.h" 

by "#define NUM_BRANCHES   353" and then re-compile RAxML." 

I did that and the recompiled binary now exits with segmentation fault. Other analyses with the recompiled binary are running fine.

Here is the command.

raxmlHPC -f k -s test.fa -q test.part -M -m GTRGAMMA -t RAxML_bestTree.TEST -n TEST1

Are there any further options to troubleshoot?

Thanks

Artem

[Alexey Kozlov]

Hi Artem,

could you please post full RAxML output for this run, before the segfault?

This will help us identify the problem.

Best,
Alexey

On 14.05.2018 16:01, Artem Pankin wrote:
> Hi,
>
> When I tried running RAxML 8.2.11 in the "-f k" mode with > 128 partitions it returned the following statement.
>

> /"You are trying to use 353 partitioned models for an individual per-gene branch length estimate./
> /Currently only 128 are allowed to improve efficiency./
> /
> /
> /In order to change this please replace the line "#define NUM_BRANCHES   128" in file "axml.h" /
> /by "#define NUM_BRANCHES   353" and then re-compile RAxML." /

>
> I did that and the recompiled binary now exits with segmentation fault. Other analyses with the recompiled binary are
> running fine.
>
> Here is the command.
>
> raxmlHPC -f k -s test.fa -q test.part -M -m GTRGAMMA -t RAxML_bestTree.TEST -n TEST1
>
> Are there any further options to troubleshoot?
>
> Thanks
>
> Artem
>

> --
> You received this message because you are subscribed to the Google Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to raxml+un...@googlegroups.com
> <mailto:raxml+un...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

[Artem Pankin]

Hi Alexey,

The quote regarding "128 partitions" is the full output. The recompiled binary with adjusted NUM_BRANCHES returns only a single line "Segmentation fault" with the mentioned command.  

Cheers

Artem

[Alexey Kozlov]

this is weird, and what about RAxML_info.TEST1 file?

> <javascript:>
> > <mailto:raxml+un...@googlegroups.com <javascript:>>.
> > For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>.

[Alexandros Stamatakis]

did you do rm *.o before re-compiling?

alexis

--
Alexandros (Alexis) Stamatakis

Research Group Leader, Heidelberg Institute for Theoretical Studies
Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology

www.exelixis-lab.org

[Artem Pankin]

I did + makefile executes "rm -f *.o raxmlHPC" as a first command.

Artem

[Artem Pankin]

neither of the two runs (128 partitions error and segfault) created *info* or any other files.

[Alexey Kozlov]

Hi Artem,

ok I see, could you please send your input files to my e-mail?


Best,
Alexey

[Artem Pankin]

Alexey,

I've just dropped you the files. 

I tried running re-compiled binary on another machine and it didn't immediately exit with segfault but gave the following error.

#######

RAxML was called as follows:

raxmlHPC -f k -s test.phy -q test.part -M -m GTRGAMMA -t RAxML_bestTree.TEST -n

*** Error in `/biodata/dep_coupland/grp_korff/artem/raxml_git/standard-RAxML-8.2.11/raxmlHPC': double free or corruption (!prev): 0x00007f971be87a40 ***

Aborted

[Alexey Kozlov]

Hi Artem,

thanks for the files, they helped me to identify the problem:

Branch length stealing - as well as a couple of other RAxML commands - implicitly disables alignment pattern
compression, which in turn does not work with non-contiguous partitions (e.g. p1=1-10,16-20 p2=11-15).

As a quick workaround, I'd suggest either:

1. Re-sort your alignment columns such that every partition is a contiguous segment (e.g. p1=1-15 p2=16-20)
You can trick RAxML into doing this for you by e.g. adding a dummy gap-only sequence to the alignment and running "-f c"
command.

2. Hack the code by commenting out line 2298 in axml.c:

--- a/axml.c
+++ b/axml.c
@@ -2298,7 +2298,7 @@ static void sitesort(rawdata *rdta, cruncheddata *cdta, tree *tr, analdef *adef)
index[0] = -1;


- if(adef->compressPatterns)
+ // if(adef->compressPatterns)

I'm not sure this solution is 100% bullet-proof, but it seems to work for me (maybe Alexis can comment on this since he
is more familiar with the code).

Please note, that you should use vertorized and parallelized version of RAxML (e.g. raxmlHPC-PTHREADS-AVX2) on such a
big dataset.

Finally, since you alignment contains very few undetermined characters (2%), branch length stealing is unlikely to
change the original tree.

Hope this helps,
Alexey

[Artem Pankin]

Hi Alexey,

Commenting out that line of the code worked, great. Thank you for such a prompt help. 

For the full-scale analysis, I'm using multithreaded SSE3 version. I've got other datasets with lots of missing data -> curious about the performance of the branch length stealing mode on those.

Thanks again.

Artem

[Alexey Kozlov]

OK I see, thanks for the clarification, have fun :)

Alexey

[Alexandros Stamatakis]

Hi Artem and Alexey,

> 2. Hack the code by commenting out line 2298 in axml.c:
>
> --- a/axml.c
> +++ b/axml.c
> @@ -2298,7 +2298,7 @@ static void sitesort(rawdata *rdta, cruncheddata
> *cdta, tree *tr, analdef *adef)
>    index[0] = -1;
>
>
> -  if(adef->compressPatterns)
> + //  if(adef->compressPatterns)
>
> I'm not sure this solution is 100% bullet-proof, but it seems to work
> for me (maybe Alexis can comment on this since he is more familiar with
> the code).

That should work. There is actually a stand alone version of branch
length stealing of which I am more confident that it works correctly:

https://github.com/ddarriba/ForeSeqs

Alexis