no bestTree file

[Emily W.]

Hi,
Based on the instructions in "The RAxML v8.2.X Manual" I am trying to run multiple inferences on my dataset - the instructions say:
"If you have a cluster available you would specify:
raxmlHPC­MPI -­f d ­-m GTRCAT ­-p 12345 ­-s ex_al ­-# 100 ­-n MultipleOriginal"


The command I submitted is below. It finished running, and I see the RAxML_log, RAxML_parsimonyTree, and RAxML_result files, but not a "RAxML_bestTree" file. Does that have something to do with using GTRGAMMA instead of GTRCAT, is it an error, or is this not the correct command?

raxmlHPC-MPI -f d -m GTRGAMMA -p 12345 -s L123_phylo41.phy -# 100 -n r100


Thanks,
Emily

[Alexey Kozlov]

Hi Emily,

it seems like your analysis hasn't finished, e.g RAxML exited with an error or your job was killed because it reached
the runtime limit. Please check job output file as well as RAxML_info file (and post them here if still unsure).

Best,
Alexey

> --
> You received this message because you are subscribed to the Google Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to raxml+un...@googlegroups.com
> <mailto:raxml+un...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

[Emily W.]

HI Alexey,

I actually had the same problem with a shorter run that I did trying random starting trees (raxmlHPC-MPI -f d -m GTRGAMMA -d -p 12345 -s L123_phylo41.phy -# 10 -n randST) - there was no bestTrees file written out. The .err files for both runs are empty. I have attached the info and out files for the larger run here - I don't see any errors or exiting of the program. I double checked my cluster hours, and I still have plenty left (we don't have a runtime limit), so that is not the problem either.

Is there a way to summarize the output files and find the best Tree?

Thanks,

Emily

[Alexandros Stamatakis]

Dear Emily,

The best tree file is automatically written if you use -f a, apart from
that please consult the RAxML manual at
http://sco.h-its.org/exelixis/resource/download/NewManual.pdf especially
pages 53 and following.

There's also a web-based step by step tutorial here:

http://sco.h-its.org/exelixis/web/software/raxml/hands_on.html

Alexis

> send an email to raxml+un...@googlegroups.com <javascript:>
> > <mailto:raxml+un...@googlegroups.com <javascript:>>.

> > For more options, visit https://groups.google.com/d/optout

> <https://groups.google.com/d/optout>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to raxml+un...@googlegroups.com
> <mailto:raxml+un...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

--
Alexandros (Alexis) Stamatakis

Research Group Leader, Heidelberg Institute for Theoretical Studies
Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology
Adjunct Professor, Dept. of Ecology and Evolutionary Biology, University
of Arizona at Tucson

www.exelixis-lab.org

[Emily W.]

Dear Alexis, 

Thanks for the response. I have been referring to both of the resources that you suggest - I was trying to run an analysis like the one described under the heading "Finding the Best-Known Likelihood tree (BKL)" (p 56-57). Later, under the "Obtaining Confidence Values" (p 57) you say that it makes the most sense to draw the bootstrap values onto the best-scoring ML tree, but it's not clear how to find that tree - should I just look through the results manually?

-Emily

[Alexandros Stamatakis]

On 18.04.2017 21:38, Emily W. wrote:
> Dear Alexis,
>
> Thanks for the response. I have been referring to both of the resources
> that you suggest - I was trying to run an analysis like the one
> described under the heading "Finding the Best-Known Likelihood tree
> (BKL)" (p 56-57). Later, under the "Obtaining Confidence Values" (p 57)
> you say that it makes the most sense to draw the bootstrap values onto
> the best-scoring ML tree, but it's not clear how to find that tree -
> should I just look through the results manually?

either manually or via a script that checks the RAxML_info file you need
to identify the tree with the best likelihood score, that's the best
scoring tree,

alexis

> Alexandros (Alexis) Stamatakis
>
> Research Group Leader, Heidelberg Institute for Theoretical Studies
> Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology
> Adjunct Professor, Dept. of Ecology and Evolutionary Biology,
> University
> of Arizona at Tucson
>

> www.exelixis-lab.org <http://www.exelixis-lab.org>

[Emily W.]

Okay, thanks for clarifying. I was hoping there was a built in way to do it but I will just do it manually.

Emily

[Alexandros Stamatakis]

there is a built in way:

e.g., calling:

./raxmlHPC-AVX -s dna.phy.dat -m GTRGAMMA -N 10 -n TE1

will do 10 tree searches and in the very end output lines like this to
the terminal:

Starting final GAMMA-based thorough Optimization on tree 5 likelihood
-100.241385 ....

Final GAMMA-based Score of best tree -100.241385

Program execution info written to
/home/stamatak/Desktop/GIT/raxml-hpc/standard-RAxML/RAxML_info.TE2
Best-scoring ML tree written to:
/home/stamatak/Desktop/GIT/raxml-hpc/standard-RAxML/RAxML_bestTree.TE2

Overall execution time: 0.775348 secs or 0.000215 hours or 0.000009 days



So maybe as Alexey suggested your analysis has not run through yet?

Alexis

[Alexey Kozlov]

Hi Emily and Alexis,

I checked the output files and I think I know what went wrong with this run:

@Emily: you're right that your run has finished successfully. However, you ran RAxML-MPI with just 1 process, which is
not the intended usage of MPI version. You should either:

- configure MPI properly to start multiple processes (please consult you cluster docs, it's usually specified in the
submission script and/or by prepending the RAxML invocation with "mpirun -n <numberOfProcesses>")

- use PTHREADS version instead (and specify the desired number of threads with -T)

- if you really don't need parallelization (usually you do), please use the sequential version, e.g. raxmlHPC-AVX

In any case, please consider using SSE/AVX version of RAxML since those are much faster.

With either of the above options, you should automatically obtain the best-scoring tree in the end.

@Alexis: technically there seems to be a bug leading to this behavior in case we have only 1 MPI rank, I guess RAxML was
just never tested in this configuration since it's not sensible...

Best,
Alexey

[Emily W.]

Okay, that does make sense - it didn't seem like it was running in parallel properly. I did ask for 8 cores in the submission script, but I wouldn't be entirely surprised if there is a problem with the installation - I think I read something about that causing issues with it actually executing the job in parallel on here before. I will ask my cluster admins to take a look.

As for using other versions - we currently have 4 versions available on our cluster - can you suggest which one would be better?

raxmlHPC-HYBRID

raxmlHPC-HYBRID-SSE3

raxmlHPC-MPI

raxmlHPC-MPI-SSE3

Emily

[Alexey Kozlov]

Hi Emily,

> Okay, that does make sense - it didn't seem like it was running in parallel properly. I did ask for 8 cores in the
> submission script, but I wouldn't be entirely surprised if there is a problem with the installation - I think I read
> something about that causing issues with it actually executing the job in parallel on here before. I will ask my cluster
> admins to take a look.

right

> As for using other versions - we currently have 4 versions available on our cluster - can you suggest which one would be
> better?
> raxmlHPC-HYBRID
> raxmlHPC-HYBRID-SSE3
> raxmlHPC-MPI
> raxmlHPC-MPI-SSE3

raxmlHPC-MPI-SSE3 or raxmlHPC-HYBRID-SSE3 (please search this group for trade-offs of the hybrid version, it was
discussed a couple of times).

Best,
Alexey

[Alexandros Stamatakis]

Hi Alexey,

> @Alexis: technically there seems to be a bug leading to this behavior in
> case we have only 1 MPI rank, I guess RAxML was just never tested in
> this configuration since it's not sensible...

Well, it was never supposed to be run with just one MPI rank, I think I
was aware of this but didn't bother fixing it.

Alexis

[Alexey Kozlov]

Hi Alexis,

> Well, it was never supposed to be run with just one MPI rank, I think I was aware of this but didn't bother fixing it.

OK I see... Maybe we should print a warning message then?

Best,
Alexey

[Alexandros Stamatakis]

On 19.04.2017 13:19, Alexey Kozlov wrote:
> Hi Alexis,
>
>> Well, it was never supposed to be run with just one MPI rank, I think
>> I was aware of this but didn't bother fixing it.
>
> OK I see... Maybe we should print a warning message then?

well maybe, but on the other hand there's RAxML-NG now, so better print
it there maybe?

alexis