Issue with raxml-ng speed

[Camille Delavaux]

Good afternoon,

I'm writing because I feel that after switching to raxml-ng my jobs are taking a much longer time. If you see any obvious things in my run that looks like it might slow things down, I would really appreciate it. Here is what I'm calling (with ~3500 seqs in the allignment):

raxml-ng --tree-constraint pruned.bac120_r89_mth_munged.newick --msa bact120_ssu_rep-seqs_clean_aligned.fasta --model GTR+FO+G --search --opt-branches on --opt-model on --tree rand{10} --threads 12 --prefix result --seed 2

Many thanks,

Camille

[Camille Delavaux]

After 18 hours, I am still on the first tree:

[18:23:42 -3135790.994895] SLOW spr round 209 (radius: 5)

[Alexey Kozlov]

Hi Camille,

your constraint tree might be a problem, please see recent discussion here:

https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/raxml/Qi3aD-yf3Jo/oxAwb-7cDAAJ

Also, please attach full raxml-ng log when reporting problems/bugs, this makes it much easier to
understand what's going on...

Best,
Alexey

> on --tree rand{10} --threads 12--prefix result --seed 2
>
>
> Many thanks,
>
>
> Camille
>
> --
> You received this message because you are subscribed to the Google Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
> raxml+un...@googlegroups.com <mailto:raxml+un...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/raxml/75432b59-2313-4db9-81c6-630923dafe0c%40googlegroups.com
> <https://groups.google.com/d/msgid/raxml/75432b59-2313-4db9-81c6-630923dafe0c%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Camille Delavaux]

Hi Alexey, Thank you,

Here is the log file so far. Going from 2 to 12 threads has definitely sped up the process, but still it seems like it will take days-weeks to complete. Any suggestions are appreciated!

Thanks again,

Camille

On Friday, November 22, 2019 at 7:39:19 AM UTC-6, Alexey Kozlov wrote:

Hi Camille,

your constraint tree might be a problem, please see recent discussion here:

https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/raxml/Qi3aD-yf3Jo/oxAwb-7cDAAJ

Also, please attach full raxml-ng log when reporting problems/bugs, this makes it much easier to
understand what's going on...

Best,
Alexey

On 21.11.19 22:56, Camille Delavaux wrote:
> After 18 hours, I am still on the first tree:
>
> [18:23:42 -3135790.994895] SLOW spr round 209 (radius: 5)
>
>
> Am Donnerstag, 21. November 2019 15:54:36 UTC-6 schrieb Camille Delavaux:
>
>     Good afternoon,
>
>     I'm writing because I feel that after switching to raxml-ng my jobs are taking a much longer
>     time. If you see any obvious things in my run that looks like it might slow things down, I would
>     really appreciate it. Here is what I'm calling (with ~3500 seqs in the allignment):
>
>     raxml-ng --tree-constraint pruned.bac120_r89_mth_munged.newick --msa
>     bact120_ssu_rep-seqs_clean_aligned.fasta --model GTR+FO+G --search --opt-branches on --opt-model
>     on --tree rand{10} --threads 12--prefix result --seed 2
>
>
>     Many thanks,
>
>
>     Camille
>
> --
> You received this message because you are subscribed to the Google Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to

> ra...@googlegroups.com <mailto:ra...@googlegroups.com>.

[Alexey Kozlov]

Hi Camille,

if you have more than 12 cores available, you can run multiple tree searches in parallel as
described here:

https://github.com/amkozlov/raxml-ng/wiki/Parallelization#coarse-grained-parallelization-for-short-alignments

Best,
Alexey

> > ra...@googlegroups.com <javascript:> <mailto:ra...@googlegroups.com <javascript:>>.

> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/raxml/75432b59-2313-4db9-81c6-630923dafe0c%40googlegroups.com
> <https://groups.google.com/d/msgid/raxml/75432b59-2313-4db9-81c6-630923dafe0c%40googlegroups.com>
> >

> <https://groups.google.com/d/msgid/raxml/75432b59-2313-4db9-81c6-630923dafe0c%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/raxml/75432b59-2313-4db9-81c6-630923dafe0c%40googlegroups.com?utm_medium=email&utm_source=footer>>.

>
>
> --
> You received this message because you are subscribed to the Google Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to

> raxml+un...@googlegroups.com <mailto:raxml+un...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/raxml/ad4cdda4-2062-4aec-af6c-6f70d023a608%40googlegroups.com
> <https://groups.google.com/d/msgid/raxml/ad4cdda4-2062-4aec-af6c-6f70d023a608%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Camille Delavaux]

Thank you,

Otherwise, it doesn't seem like there will be any obvious way to speed it up in the current set up?

Camille

> ra...@googlegroups.com <mailto:ra...@googlegroups.com>.

[Alexey Kozlov]

Well as has been discussed here a dozen of times, this type of data is generally problematic:

you have ~4.5000 sequences with just ~13.000 patterns, and alignment contains 95% (!) missing data
-> the signal is very weak.

Maybe GTR+G+I or GTR+R model would fit the data a bit better, you can use eg. ModelTest-NG to test it:

https://academic.oup.com/mbe/advance-article/doi/10.1093/molbev/msz189/5552155

Best,
Alexey

> https://groups.google.com/d/msgid/raxml/ad4cdda4-2062-4aec-af6c-6f70d023a608%40googlegroups.com
> <https://groups.google.com/d/msgid/raxml/ad4cdda4-2062-4aec-af6c-6f70d023a608%40googlegroups.com>
> >
> <https://groups.google.com/d/msgid/raxml/ad4cdda4-2062-4aec-af6c-6f70d023a608%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/raxml/ad4cdda4-2062-4aec-af6c-6f70d023a608%40googlegroups.com?utm_medium=email&utm_source=footer>>.

>
>
> --
> You received this message because you are subscribed to the Google Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to

> raxml+un...@googlegroups.com <mailto:raxml+un...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/raxml/30a1a3b6-3b57-48d7-924d-cd025afc2ad5%40googlegroups.com
> <https://groups.google.com/d/msgid/raxml/30a1a3b6-3b57-48d7-924d-cd025afc2ad5%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Heather Rose Kates]

Hi Camille, just chiming in from my own errors (only relevant if this turns out to be unrelated to your data, I've not analyzed a similar alignment): If you're running this as a job on a computing cluster- is there any chance you're specifying a different number of --threads in your raxml-ng command as you have specified in your resource requests? I did this by accident and the jobs did run but the calculations were about 20 times slower (than when --cpus-per-task and --threads matched). I was doing coarse grained parallelization so I had quite a few replicates to confirm the effect of this error ( : 

Heather Rose