run time for topology-constrained analysis

83 views
Skip to first unread message

Pascal Title

unread,
Oct 24, 2019, 2:21:53 PM10/24/19
to raxml
Hi,

I am running raxml-ng on a large sparse supermatrix. This supermatrix contains 7708 taxa and 8 partitions. I find that on average a single raxml-ng run from a single starting tree, using 16 threads, takes 1.5 weeks to complete.

I am also running a parallel analysis with the same dataset, but where I am including a constraint tree with 1075 taxa (but otherwise everything else is identical). This run has been going for over 25 days.

Is it normal for a constrained analysis to take so much longer than an unconstrained run? Or is this an indication of a problem?

thanks!

-Pascal

Alexey Kozlov

unread,
Oct 24, 2019, 4:55:12 PM10/24/19
to ra...@googlegroups.com
Hi Pascal,

typically, constrained tree search runs faster, although the opposite is also possible - at least in
theory.

Could you please post both log files?

Best,
Alexey
> --
> You received this message because you are subscribed to the Google Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
> raxml+un...@googlegroups.com <mailto:raxml+un...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/raxml/a90fa301-2493-4dcf-ab1b-cd4f13b26b85%40googlegroups.com
> <https://groups.google.com/d/msgid/raxml/a90fa301-2493-4dcf-ab1b-cd4f13b26b85%40googlegroups.com?utm_medium=email&utm_source=footer>.

Pascal Title

unread,
Oct 24, 2019, 4:59:53 PM10/24/19
to raxml
Log files attached.

thanks!
-Pascal

On Thursday, October 24, 2019 at 4:55:12 PM UTC-4, Alexey Kozlov wrote:
Hi Pascal,

typically, constrained tree search runs faster, although the opposite is also possible - at least in
theory.

Could you please post both log files?

Best,
Alexey

On 24.10.19 20:21, Pascal Title wrote:
> Hi,
>
> I am running raxml-ng on a large sparse supermatrix. This supermatrix contains 7708 taxa and 8
> partitions. I find that on average a single raxml-ng run from a single starting tree, using 16
> threads, takes 1.5 weeks to complete.
>
> I am also running a parallel analysis with the same dataset, but where I am including a constraint
> tree with 1075 taxa (but otherwise everything else is identical). This run has been going for over
> 25 days.
>
> Is it normal for a constrained analysis to take so much longer than an unconstrained run? Or is this
> an indication of a problem?
>
> thanks!
>
> -Pascal
>
> --
> You received this message because you are subscribed to the Google Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
constrained.txt
unconstrained.txt

Alexey Kozlov

unread,
Oct 24, 2019, 6:48:21 PM10/24/19
to ra...@googlegroups.com
Hi Pascal,

thanks for the files!

What I can see so far:

- using 32 threads seems to be slower than 16 threads, your CPU probably has 16 physical cores?

- constrained tree search takes much longer to converge (158+ SLOW rounds vs. just 33 for
unconstrained), this could happen unfortunately if the constraint tree "contradicts" signal in the
alignment...

Best,
Alexey
> > ra...@googlegroups.com <javascript:> <mailto:ra...@googlegroups.com <javascript:>>.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/raxml/a90fa301-2493-4dcf-ab1b-cd4f13b26b85%40googlegroups.com
> <https://groups.google.com/d/msgid/raxml/a90fa301-2493-4dcf-ab1b-cd4f13b26b85%40googlegroups.com>
> >
> <https://groups.google.com/d/msgid/raxml/a90fa301-2493-4dcf-ab1b-cd4f13b26b85%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/raxml/a90fa301-2493-4dcf-ab1b-cd4f13b26b85%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
>
> --
> You received this message because you are subscribed to the Google Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
> raxml+un...@googlegroups.com <mailto:raxml+un...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/raxml/5d02c68c-b919-41b1-b47d-e2c58aed5283%40googlegroups.com
> <https://groups.google.com/d/msgid/raxml/5d02c68c-b919-41b1-b47d-e2c58aed5283%40googlegroups.com?utm_medium=email&utm_source=footer>.

Pascal Title

unread,
Oct 25, 2019, 11:51:52 AM10/25/19
to raxml
The constrained run is on a 32-core (64 thread) machine.

So perhaps this is due to conflicting signal...

Pascal Title

unread,
Oct 30, 2019, 11:31:59 AM10/30/19
to raxml
I conducted another test. For a 1400 taxon subset of my sequence matrix, I ran the raxml-ng analysis with a constraint tree, and also ran the same analysis in standard RAxML, both with GTRGAMMA.

raxml-ng took 30 hours and reached a likelihood of -1446721.34
standard raxml took 7 hours and reached a likelihood of -1467902.82

It appears that standard raxml is just much more efficient when it comes to a constrained analysis. Is that expected? But raxml-ng appears to reach a better likelihood.
Do you have any recommendations?

thanks!
-Pascal

Alexey Kozlov

unread,
Oct 31, 2019, 10:01:05 AM10/31/19
to ra...@googlegroups.com
Hi Pascal,

if raxml8 finishes faster but returns a worse tree (the loglh difference is quite large), I wouldn't
call it "much more efficient" :)

Still it would be interesting to see raxml8 outputs. Maybe the difference is due to distinct
models/settings...

Best,
Alexey
> <https://groups.google.com/d/msgid/raxml/5d02c68c-b919-41b1-b47d-e2c58aed5283%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/raxml/5d02c68c-b919-41b1-b47d-e2c58aed5283%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
>
> --
> You received this message because you are subscribed to the Google Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
> raxml+un...@googlegroups.com <mailto:raxml+un...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/raxml/54ed3387-e744-477b-abaf-cb1f05fb99ec%40googlegroups.com
> <https://groups.google.com/d/msgid/raxml/54ed3387-e744-477b-abaf-cb1f05fb99ec%40googlegroups.com?utm_medium=email&utm_source=footer>.

Pascal Title

unread,
Oct 31, 2019, 10:11:57 AM10/31/19
to raxml
Hi Alexey,

The command used for standard RAXML was:
raxmlHPC-PTHREADS-AVX -s gekko.phy -m GTRCAT -q concatenated.partitions.raxml.partitions -o Sphenodon_punctatus -g gekko.tre -T 10 -p 12345 -n gekkoTest

and the command used for raxml-ng was:
raxml-ng --msa gekko.phy --model concatenated.partitions --prefix gekko --tree-constraint gekko.tre --threads 16 --seed 123 --tree rand{1}
... with the model specified as GTR+G

I've also attached the info file from the standard raxml run.

Thanks for your help in trying to better understand this!

-Pascal
RAxML_info.gekkoTest

Alexey Kozlov

unread,
Oct 31, 2019, 10:28:08 AM10/31/19
to ra...@googlegroups.com
Hi Pascal,

thanks for the info, now it's clear why raxml8 was faster: you used GTRCAT model which is
significantly cheaper to compute than GTR+G.

The equivalent command line for raxml8 would be:

> raxmlHPC-PTHREADS-AVX -s gekko.phy -m GTRGAMMAX -q concatenated.partitions.raxml.partitions -o
> Sphenodon_punctatus-g gekko.tre -T 10-p 12345-n gekkoTest -d

Best,
Alexey

On 31.10.19 15:11, Pascal Title wrote:
> Hi Alexey,
>
> The command used for standard RAXML was:
> |
> raxmlHPC-PTHREADS-AVX -s gekko.phy -m GTRCAT -q concatenated.partitions.raxml.partitions -o
> Sphenodon_punctatus-g gekko.tre -T 10-p 12345-n gekkoTest
> |
>
> and the command used for raxml-ng was:
> |
> raxml-ng --msa gekko.phy --model concatenated.partitions --prefix gekko --tree-constraint gekko.tre
> --threads 16--seed 123--tree rand{1}
> https://groups.google.com/d/msgid/raxml/54ed3387-e744-477b-abaf-cb1f05fb99ec%40googlegroups.com
> <https://groups.google.com/d/msgid/raxml/54ed3387-e744-477b-abaf-cb1f05fb99ec%40googlegroups.com>
> >
> <https://groups.google.com/d/msgid/raxml/54ed3387-e744-477b-abaf-cb1f05fb99ec%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/raxml/54ed3387-e744-477b-abaf-cb1f05fb99ec%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
>
> --
> You received this message because you are subscribed to the Google Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
> raxml+un...@googlegroups.com <mailto:raxml+un...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/raxml/7e0ab0cb-3ee2-4a2d-9d76-057fb9051600%40googlegroups.com
> <https://groups.google.com/d/msgid/raxml/7e0ab0cb-3ee2-4a2d-9d76-057fb9051600%40googlegroups.com?utm_medium=email&utm_source=footer>.

Pascal Title

unread,
Oct 31, 2019, 10:34:34 AM10/31/19
to raxml
Sorry, I was accidentally grabbed the wrong test.

I did in fact use the GTRGAMMA model with raxml8:
raxmlHPC-PTHREADS-AVX -s gekko.phy -m GTRGAMMA -q concatenated.partitions.raxml.partitions -o Sphenodon_punctatus -g gekko.tre -T 10 -p 12345 -n gekkoTest

Correct output file attached.

RAxML_info.gekkoTest

Alexey Kozlov

unread,
Oct 31, 2019, 11:03:21 AM10/31/19
to ra...@googlegroups.com
ok, then it seems like raxml8 search just converged faster, but to a worse tree...

On 31.10.19 15:34, Pascal Title wrote:
> Sorry, I was accidentally grabbed the wrong test.
>
> I did in fact use the GTRGAMMA model with raxml8:
> |
> raxmlHPC-PTHREADS-AVX -s gekko.phy -m GTRGAMMA -q concatenated.partitions.raxml.partitions -o
> https://groups.google.com/d/msgid/raxml/7e0ab0cb-3ee2-4a2d-9d76-057fb9051600%40googlegroups.com
> <https://groups.google.com/d/msgid/raxml/7e0ab0cb-3ee2-4a2d-9d76-057fb9051600%40googlegroups.com>
> >
> <https://groups.google.com/d/msgid/raxml/7e0ab0cb-3ee2-4a2d-9d76-057fb9051600%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/raxml/7e0ab0cb-3ee2-4a2d-9d76-057fb9051600%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
>
> --
> You received this message because you are subscribed to the Google Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
> raxml+un...@googlegroups.com <mailto:raxml+un...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/raxml/805f9d83-3441-4d07-8427-da1a0f16ef81%40googlegroups.com
> <https://groups.google.com/d/msgid/raxml/805f9d83-3441-4d07-8427-da1a0f16ef81%40googlegroups.com?utm_medium=email&utm_source=footer>.
Reply all
Reply to author
Forward
0 new messages