best tree computation on HPC

[Pieter]

Dear,

 

In our lab, we conduct research that requires us to construct large trees for various virus pathogens.

In order to do this, we constructed a workflow that was based on the Bootstrapping section (Step 4) of the step-by-step documentation manual (https://sco.h-its.org/exelixis/web/software/raxml/hands_on.html).

 

We aim to construct trees with 1000 bootstraps, so to optimally use our HPC infrastructure, we run each bootstrap separately, using:

raxmlHPC-SSE3 -x $seed -p $trees -# 1

Where $seed is an element of a set of random integers and $trees is an element of another set of random integers.

 

To determine the best tree on the other hand, we run the best tree search as an MPI coordinated process:

mpirun raxmlHPC-MPI -f a -x 12356 -p 123 -# 20

However, when reading the explanation in the step-by-step manual, this process runs 20 distinct tree searches, and export the tree with the largest likelihood.

 

Therefore, we were wondering whether it would be possible to run these 20 searches as separate processes, export 20 distinct trees with their likelihood, such that we can select the best tree manually. This way of computing would enable us to have more control of our HPCs job scheduling system.

 

Would the following approach be sound to achieve this purpose:

- create 2 sets of random integers of size 20 s1 and s2

- run 20 instances of raxmlHPC-SSE3 -x $seed1 -p $seed2 -# 1, where $seed1 is an element popped from the above set s1 and $seed2 is an element popped from the above set s2 

- from the 20 computed trees, select the one with the highest likelihood

 

Thanks in advance and kind regards,

Pieter

[Alexandros Stamatakis]

Hi Pieter,

> In our lab, we conduct research that requires us to construct large
> trees for various virus pathogens.
>
> In order to do this, we constructed a workflow that was based on the
> Bootstrapping section (Step 4) of the step-by-step documentation manual
> (https://sco.h-its.org/exelixis/web/software/raxml/hands_on.html).
>
> We aim to construct trees with 1000 bootstraps, so to optimally use our
> HPC infrastructure, we run each bootstrap separately, using:
>
> raxmlHPC-SSE3 -x $seed -p $trees -# 1
>
> Where $seed is an element of a set of random integers and $trees is an
> element of another set of random integers.

Why don't you use the so-called bootstopping option (see manual) of the
code? This might help to avoid running unecessary replicates.

Also, why don't you use the AVX version of the code?

In this case it would be preferrable to use the standard bootstrap, -b
for which there also exists a coarse-grained MPI parallalization.

> To determine the best tree on the other hand, we run the best tree
> search as an MPI coordinated process:
>
> mpirun raxmlHPC-MPI -f a -x 12356 -p 123 -# 20
>
> However, when reading the explanation in the step-by-step manual, this
> process runs 20 distinct tree searches, and export the tree with the
> largest likelihood.
>
> Therefore, we were wondering whether it would be possible to run these
> 20 searches as separate processes, export 20 distinct trees with their
> likelihood, such that we can select the best tree manually. This way of
> computing would enable us to have more control of our HPCs job
> scheduling system.

Yes, that's feasible, only that a final optimization step on the best
scoring of these 20 trees (which does not improve the likelihood a lot
though) would not be executed. Also the above command will also execute
bootstraps. It should rather read:

mpirun raxmlHPC-MPI -p 123 -# 20

which will generate 20 trees in parallel.

> Would the following approach be sound to achieve this purpose:
>
> - create 2 sets of random integers of size 20 s1 and s2
>
> - run 20 instances of raxmlHPC-SSE3 -x $seed1 -p $seed2 -# 1, where
> $seed1 is an element popped from the above set s1 and $seed2 is an
> element popped from the above set s2

You should omit -x as this will generate bootstrap trees.

> - from the 20 computed trees, select the one with the highest likelihood

Yes, but be careful with -x which generates bootstrap trees.

Also, the simplest way is to do as many BS replicates as required with
the -b option (using the bootstopping criterion to check if you have
already done enough bootstraps) and then do 20 independent ML searches.

Finally, for the sake of efficiency I would strongly recommend switching
to the RAxML re-design RAxML-NG:

https://github.com/amkozlov/raxml-ng

Alexis

>
> Thanks in advance and kind regards,
>
> Pieter
>

> --
> You received this message because you are subscribed to the Google
> Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to raxml+un...@googlegroups.com
> <mailto:raxml+un...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

--
Alexandros (Alexis) Stamatakis

Research Group Leader, Heidelberg Institute for Theoretical Studies
Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology

www.exelixis-lab.org

[Pieter Libin]

Hi Alexis,

Thanks for your quick reply!

On Thu, Aug 10, 2017 at 5:29 PM, Alexandros Stamatakis <alexandros...@gmail.com> wrote:

Hi Pieter,

In our lab, we conduct research that requires us to construct large trees for various virus pathogens.

In order to do this, we constructed a workflow that was based on the Bootstrapping section (Step 4) of the step-by-step documentation manual (https://sco.h-its.org/exelixis/web/software/raxml/hands_on.html).

We aim to construct trees with 1000 bootstraps, so to optimally use our HPC infrastructure, we run each bootstrap separately, using:

raxmlHPC-SSE3 -x $seed -p $trees -# 1

Where $seed is an element of a set of random integers and $trees is an element of another set of random integers.

Why don't you use the so-called bootstopping option (see manual) of the code? This might help to avoid running unecessary replicates.

Computing the bootstraps is not a bottle neck at the moment, but I look into your suggestion.

 

Also, why don't you use the AVX version of the code?

Good point, historical reasons I think, we will rebuild with AVX support.

 

In this case it would be preferrable to use the standard bootstrap, -b for which there also exists a coarse-grained MPI parallalization.

I'll look into the suggestion, but running all bootstraps as separate processes is quite efficient with respect to the acquisition of resources on our HPC (i.e., single HPC nodes are easier to acquire than clusters of nodes). 

To determine the best tree on the other hand, we run the best tree search as an MPI coordinated process:

mpirun raxmlHPC-MPI -f a -x 12356 -p 123 -# 20

However, when reading the explanation in the step-by-step manual, this process runs 20 distinct tree searches, and export the tree with the largest likelihood.

Therefore, we were wondering whether it would be possible to run these 20 searches as separate processes, export 20 distinct trees with their likelihood, such that we can select the best tree manually. This way of computing would enable us to have more control of our HPCs job scheduling system.

Yes, that's feasible, only that a final optimization step on the best scoring of these 20 trees (which does not improve the likelihood a lot though) would not be executed.

OK, is it possible to do this extra optimisation step, after the 20 trees are generated? By cat-ing the trees together to start this search?

If not, is there another way to obtain a similar quality of likelihood upon computing these 20 trees separately?

 

Also the above command will also execute bootstraps. It should rather read:

mpirun raxmlHPC-MPI  -p 123 -# 20

which will generate 20 trees in parallel.

Would the following approach be sound to achieve this purpose:

- create 2 sets of random integers of size 20 s1 and s2

- run 20 instances of raxmlHPC-SSE3 -x $seed1 -p $seed2 -# 1, where $seed1 is an element popped from the above set s1 and $seed2 is an element popped from the above set s2

You should omit -x as this will generate bootstrap trees.

- from the 20 computed trees, select the one with the highest likelihood

Yes, but be careful with -x which generates bootstrap trees.

OK.

 

Also, the simplest way is to do as many BS replicates as required with the -b option (using the bootstopping criterion to check if you have already done enough bootstraps) and then do 20 independent ML searches.

Finally, for the sake of efficiency I would strongly recommend switching to the RAxML re-design RAxML-NG:

https://github.com/amkozlov/raxml-ng

OK, I saw that this software is in Beta phase, is it robust enough to use the results in publications? 

 

Alexis

Kind regards,

Pieter 

Thanks in advance and kind regards,

Pieter

--
You received this message because you are subscribed to the Google Groups "raxml" group.

To unsubscribe from this group and stop receiving emails from it, send an email to raxml+unsubscribe@googlegroups.com <mailto:raxml+unsubscribe@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.

--
Alexandros (Alexis) Stamatakis

Research Group Leader, Heidelberg Institute for Theoretical Studies
Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology

www.exelixis-lab.org

--
You received this message because you are subscribed to a topic in the Google Groups "raxml" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/raxml/UXXkkV_Am0o/unsubscribe.
To unsubscribe from this group and all its topics, send an email to raxml+unsubscribe@googlegroups.com.

[Alexandros Stamatakis]

Hi Pieter,

> Why don't you use the so-called bootstopping option (see manual) of
> the code? This might help to avoid running unecessary replicates.
>
>
> Computing the bootstraps is not a bottle neck at the moment, but I look
> into your suggestion.

It will reduce the CO^2 footprint ;-)

> Also, why don't you use the AVX version of the code?
>
>
> Good point, historical reasons I think, we will rebuild with AVX support.

:-) there's also AVX2 support by the way.

> In this case it would be preferrable to use the standard bootstrap,
> -b for which there also exists a coarse-grained MPI parallalization.
>
>
> I'll look into the suggestion, but running all bootstraps as separate
> processes is quite efficient with respect to the acquisition of
> resources on our HPC (i.e., single HPC nodes are easier to acquire than
> clusters of nodes).

I see, that's very cluster specific evidently, but -b will do a more
thorough tree search, so better do less replicates with bootstopping and
more thorough searches.

> To determine the best tree on the other hand, we run the best
> tree search as an MPI coordinated process:
>
> mpirun raxmlHPC-MPI -f a -x 12356 -p 123 -# 20
>
> However, when reading the explanation in the step-by-step
> manual, this process runs 20 distinct tree searches, and export
> the tree with the largest likelihood.
>
> Therefore, we were wondering whether it would be possible to run
> these 20 searches as separate processes, export 20 distinct
> trees with their likelihood, such that we can select the best
> tree manually. This way of computing would enable us to have
> more control of our HPCs job scheduling system.
>
>
> Yes, that's feasible, only that a final optimization step on the
> best scoring of these 20 trees (which does not improve the
> likelihood a lot though) would not be executed.
>
>
> OK, is it possible to do this extra optimisation step, after the 20
> trees are generated? By cat-ing the trees together to start this search?

No, but the impact of this little extra optimization is not huge anyway.
We also sometimes don't use it when we analyze empirical datasets.

> If not, is there another way to obtain a similar quality of likelihood
> upon computing these 20 trees separately?

You could run the SH-like test with -f J on the best scoring tree:

1. This might improve the topology slightly
2. and yield so called SH-like support values that are an alternative to
the bootstrap.

> Also the above command will also execute bootstraps. It should
> rather read:
>
> mpirun raxmlHPC-MPI -p 123 -# 20
>
> which will generate 20 trees in parallel.
>
> Would the following approach be sound to achieve this purpose:
>
> - create 2 sets of random integers of size 20 s1 and s2
>
> - run 20 instances of raxmlHPC-SSE3 -x $seed1 -p $seed2 -# 1,
> where $seed1 is an element popped from the above set s1 and
> $seed2 is an element popped from the above set s2
>
>
> You should omit -x as this will generate bootstrap trees.
>
> - from the 20 computed trees, select the one with the highest
> likelihood
>
>
> Yes, but be careful with -x which generates bootstrap trees.
>
>
> OK.
>
>
> Also, the simplest way is to do as many BS replicates as required
> with the -b option (using the bootstopping criterion to check if you
> have already done enough bootstraps) and then do 20 independent ML
> searches.
>
> Finally, for the sake of efficiency I would strongly recommend
> switching to the RAxML re-design RAxML-NG:
>
> https://github.com/amkozlov/raxml-ng
> <https://github.com/amkozlov/raxml-ng>
>
>
> OK, I saw that this software is in Beta phase, is it robust enough to
> use the results in publications?

Yes, it's already been tested quite a lot.

Alexis

>
>
>
> Alexis
>
>
> Kind regards,
>
> Pieter
>
>
>
> Thanks in advance and kind regards,
>
> Pieter
>
> --
> You received this message because you are subscribed to the
> Google Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from

> it, send an email to raxml+un...@googlegroups.com
> <mailto:raxml%2Bunsu...@googlegroups.com>
> <mailto:raxml+un...@googlegroups.com
> <mailto:raxml%2Bunsu...@googlegroups.com>>.

> For more options, visit https://groups.google.com/d/optout

> <https://groups.google.com/d/optout>.

>
>
> --
> Alexandros (Alexis) Stamatakis
>
> Research Group Leader, Heidelberg Institute for Theoretical Studies
> Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology
>

> www.exelixis-lab.org <http://www.exelixis-lab.org>

>
>
> --
> You received this message because you are subscribed to a topic in
> the Google Groups "raxml" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/raxml/UXXkkV_Am0o/unsubscribe

> <https://groups.google.com/d/topic/raxml/UXXkkV_Am0o/unsubscribe>.

> To unsubscribe from this group and all its topics, send an email to

> raxml+un...@googlegroups.com
> <mailto:raxml%2Bunsu...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout

> <https://groups.google.com/d/optout>.

>
>
> --
> You received this message because you are subscribed to the Google
> Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to raxml+un...@googlegroups.com
> <mailto:raxml+un...@googlegroups.com>.

[Fabricia Nascimento]

Hello!

I'm trying to do the same with my data and run some of my analysis in a HTC, and I did started using the RAxML-NG. 

However, I started running some preliminary analysis using raxml-ng and something is really bugging me.

My alignment has 44 sequences and 1,304 bp. An ML in my Desktop with this alignment runs fairly quick (OS X Yosemite version 10.10.5).

When I use raxml-ng (v.0.4.1, I downloaded it on 22nd August) to estimate a ML tree and 100 bootstrap replicates [raxml-ng -all -msa alignment.fasta -model K80+G --tree pars{10} --bs-trees 100 --threads 2], I have a tree with bootstrap mapped to it that seems fine. For example, for a specific clade the bootstrap support is 100 (which I assume is 100%?, is that correct to assume?). 

However, when I use raxml-ng to estimate the tree with 1,000 replicates [raxml-ng -all -msa alignment.fasta -model K80+G --tree pars{10} --bs-trees 1000 --threads 2], my tree looks weird. For the same clade that had 100 support for the 100 bootstrap replicate, I have a support of 23, and all the other supports are also very low.

So, I mapped the results of raxml-ng (using the raxml-ng best ML tree and the 1000 bootstrap replicates), but using raxml (not raxml-ng) [raxml -n map -s masked_sn_lanl2016_prAln3_B_FFN.fasta -f b -m GTRGAMMA -t RAxML_bestTree.tre -z RAxML_bootstrap], and the tree looks fine and the support also fine. By the way, how can I map bootstrap support to a best ML tree using raxml-ng? Is that possible?

Do you have any idea why I am getting such results? Am I doing something incorrect? After having these results I started to get inclined in using raxml instead of raxml-ng to estimate my trees in a HTC for publication, but I still would like to know what I am doing that is, maybe, not correct.

Thanks!

Fabricia.

>         <mailto:raxml%2Bu...@googlegroups.com>
>         <mailto:raxml+un...@googlegroups.com
>         <mailto:raxml%2Bu...@googlegroups.com>>.

>         For more options, visit https://groups.google.com/d/optout
>         <https://groups.google.com/d/optout>.
>
>
>     --
>     Alexandros (Alexis) Stamatakis
>
>     Research Group Leader, Heidelberg Institute for Theoretical Studies
>     Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology
>
>     www.exelixis-lab.org <http://www.exelixis-lab.org>
>
>
>     --
>     You received this message because you are subscribed to a topic in
>     the Google Groups "raxml" group.
>     To unsubscribe from this topic, visit
>     https://groups.google.com/d/topic/raxml/UXXkkV_Am0o/unsubscribe
>     <https://groups.google.com/d/topic/raxml/UXXkkV_Am0o/unsubscribe>.
>     To unsubscribe from this group and all its topics, send an email to
>     raxml+un...@googlegroups.com

>     <mailto:raxml%2Bu...@googlegroups.com>.

[Alexey Kozlov]

Hi Fabricia,

this sounds weird indeed, could you please send me your alignment and all raxml-ng output files for both runs to my
personal e-mail address?

> When I use raxml-ng (v.0.4.1, I downloaded it on 22nd August) to estimate a ML tree and 100 bootstrap replicates

> [/raxml-ng -all -msa alignment.fasta -model K80+G --tree pars{10} --bs-trees 100 --threads 2/], I have a tree with

> bootstrap mapped to it that seems fine. For example, for a specific clade the bootstrap support is 100 (which I assume
> is 100%?, is that correct to assume?).

yes, this is correct.

> So, I mapped the results of raxml-ng (using the raxml-ng best ML tree and the 1000 bootstrap replicates), but using

> raxml (not raxml-ng) [/raxml -n map -s masked_sn_lanl2016_prAln3_B_FFN.fasta -f b -m GTRGAMMA -t RAxML_bestTree.tre -z
> RAxML_bootstrap/], and the tree looks fine and the support also fine. *By the way, how can I map bootstrap support to a
> best ML tree using raxml-ng? Is that possible?*

Not yet, but I'll add this feature soon.

Best,
Alexey

> > it, send an email to raxml+un...@googlegroups.com <javascript:>
> > <mailto:raxml%2Bu...@googlegroups.com <javascript:>>
> > <mailto:raxml+un...@googlegroups.com <javascript:>
> > <mailto:raxml%2Bu...@googlegroups.com <javascript:>>>.

> > For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>
> > <https://groups.google.com/d/optout <https://groups.google.com/d/optout>>.
> >
> >
> > --
> > Alexandros (Alexis) Stamatakis
> >
> > Research Group Leader, Heidelberg Institute for Theoretical Studies
> > Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology
> >

> > www.exelixis-lab.org <http://www.exelixis-lab.org> <http://www.exelixis-lab.org>

> >
> >
> > --
> > You received this message because you are subscribed to a topic in
> > the Google Groups "raxml" group.
> > To unsubscribe from this topic, visit
> > https://groups.google.com/d/topic/raxml/UXXkkV_Am0o/unsubscribe
> <https://groups.google.com/d/topic/raxml/UXXkkV_Am0o/unsubscribe>
> > <https://groups.google.com/d/topic/raxml/UXXkkV_Am0o/unsubscribe
> <https://groups.google.com/d/topic/raxml/UXXkkV_Am0o/unsubscribe>>.
> > To unsubscribe from this group and all its topics, send an email to

> > raxml+un...@googlegroups.com <javascript:>
> > <mailto:raxml%2Bu...@googlegroups.com <javascript:>>.

> > For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>

> > <https://groups.google.com/d/optout <https://groups.google.com/d/optout>>.
> >
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "raxml" group.
> > To unsubscribe from this group and stop receiving emails from it, send

> > an email to raxml+un...@googlegroups.com <javascript:>
> > <mailto:raxml+un...@googlegroups.com <javascript:>>.

> > For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>.
>
> --
> Alexandros (Alexis) Stamatakis
>
> Research Group Leader, Heidelberg Institute for Theoretical Studies
> Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology
>
> www.exelixis-lab.org <http://www.exelixis-lab.org>
>
> --

[Alexey Kozlov]

Dear Fabricia,

this issue was fixed in the latest raxml-ng release available here:

https://github.com/amkozlov/raxml-ng/releases/tag/0.5.0

I also added support mapping option (analogous to '-f b' in old RAxML), which can be uses as follows:

./raxml-ng --support --tree bestML.tree --bs-trees bootstraps.newick

Thanks again for spotting this bug! I will write a separate google group post on this, since it could affect other
raxml-ng users as well.

Best,
Alexey

On 02.09.2017 14:12, 'Fabricia Nascimento' via raxml wrote:
> Hello!
>
> I'm trying to do the same with my data and run some of my analysis in a HTC, and I did started using the RAxML-NG.
> However, I started running some preliminary analysis using raxml-ng and something is really bugging me.
>
> My alignment has 44 sequences and 1,304 bp. An ML in my Desktop with this alignment runs fairly quick (OS X Yosemite
> version 10.10.5).
>
> When I use raxml-ng (v.0.4.1, I downloaded it on 22nd August) to estimate a ML tree and 100 bootstrap replicates

> [/raxml-ng -all -msa alignment.fasta -model K80+G --tree pars{10} --bs-trees 100 --threads 2/], I have a tree with

> bootstrap mapped to it that seems fine. For example, for a specific clade the bootstrap support is 100 (which I assume
> is 100%?, is that correct to assume?).
>

> However, when I use raxml-ng to estimate the tree with 1,000 replicates [/raxml-ng -all -msa alignment.fasta -model
> K80+G --tree pars{10} --bs-trees 1000 --threads 2/], my tree looks weird. For the same clade that had 100 support for

> the 100 bootstrap replicate, I have a support of 23, and all the other supports are also very low.
>
> So, I mapped the results of raxml-ng (using the raxml-ng best ML tree and the 1000 bootstrap replicates), but using

> raxml (not raxml-ng) [/raxml -n map -s masked_sn_lanl2016_prAln3_B_FFN.fasta -f b -m GTRGAMMA -t RAxML_bestTree.tre -z
> RAxML_bootstrap/], and the tree looks fine and the support also fine. *By the way, how can I map bootstrap support to a
> best ML tree using raxml-ng? Is that possible?*
> *
> *

> > it, send an email to raxml+un...@googlegroups.com <javascript:>
> > <mailto:raxml%2Bu...@googlegroups.com <javascript:>>
> > <mailto:raxml+un...@googlegroups.com <javascript:>

> > <mailto:raxml%2Bu...@googlegroups.com <javascript:>>>.

> > For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>
> > <https://groups.google.com/d/optout <https://groups.google.com/d/optout>>.
> >
> >
> > --
> > Alexandros (Alexis) Stamatakis
> >
> > Research Group Leader, Heidelberg Institute for Theoretical Studies
> > Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology
> >

> > www.exelixis-lab.org <http://www.exelixis-lab.org> <http://www.exelixis-lab.org>

> >
> >
> > --
> > You received this message because you are subscribed to a topic in
> > the Google Groups "raxml" group.
> > To unsubscribe from this topic, visit
> > https://groups.google.com/d/topic/raxml/UXXkkV_Am0o/unsubscribe
> <https://groups.google.com/d/topic/raxml/UXXkkV_Am0o/unsubscribe>
> > <https://groups.google.com/d/topic/raxml/UXXkkV_Am0o/unsubscribe
> <https://groups.google.com/d/topic/raxml/UXXkkV_Am0o/unsubscribe>>.
> > To unsubscribe from this group and all its topics, send an email to

> > raxml+un...@googlegroups.com <javascript:>
> > <mailto:raxml%2Bu...@googlegroups.com <javascript:>>.

> > For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>

> > <https://groups.google.com/d/optout <https://groups.google.com/d/optout>>.
> >
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "raxml" group.
> > To unsubscribe from this group and stop receiving emails from it, send

> > an email to raxml+un...@googlegroups.com <javascript:>
> > <mailto:raxml+un...@googlegroups.com <javascript:>>.

> > For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>.
>
> --
> Alexandros (Alexis) Stamatakis
>
> Research Group Leader, Heidelberg Institute for Theoretical Studies
> Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology
>
> www.exelixis-lab.org <http://www.exelixis-lab.org>
>
> --

[Fabricia Nascimento]

Thanks Alexey! I'm glad that I could help you.

I will use the new version.

Cheers,

Fabricia.