Issue using RaxML-ng to make an ExaBayes consensus tree

[Daryl Lam]

Greetings Alexis,

I'm having an issue making a consensus tree in ExaBayes.  The output tree files represent about 1.6 terabytes of data in total.
On a blog post you stated that RAxML-ng had multiple thread option that can be used to create a consensus tree.
I ran "raxml-ng --consense MRE --tree ExaBayes_topologies* --prefix consMRE" and got the following error:
"Reading input trees from file: ExaBayes_topologies.run-0.myRun
ERROR: ERROR reading tree file (LIBPLL-111): syntax error, unexpected STRING, expecting OPAR. (line 1 column 0-6)".
Any assistance would be appreciated.
Thanks in advance,

Daryl

[Alexandros Stamatakis]

Dear Daryl,

Thanks, could you maybe send us a small sample of the trees (to a
personal email if you prefer) such that we can check?

Alexis

> --
> You received this message because you are subscribed to the Google
> Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to raxml+un...@googlegroups.com
> <mailto:raxml+un...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/raxml/6e4536b2-a595-4665-86b1-47fe5d88c9den%40googlegroups.com
> <https://groups.google.com/d/msgid/raxml/6e4536b2-a595-4665-86b1-47fe5d88c9den%40googlegroups.com?utm_medium=email&utm_source=footer>.

--
Alexandros (Alexis) Stamatakis

Research Group Leader, Heidelberg Institute for Theoretical Studies
Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology
Affiliated Scientist, Evolutionary Genetics and Paleogenomics (EGP) lab,
Institute of Molecular Biology and Biotechnology, Foundation for
Research and Technology Hellas

www.exelixis-lab.org

[Tobias Senoner]

Hi Alexis,

As we discussed in PM, we are having exactly the same issue. We are using RAxML-NG to consense trees from ExaBayes, each topology file is around 100MB. The error message is exactly the same, just in case here it is:

Analysis options:
  run mode: Build consensus tree (MRE)
  start tree(s): user
  random seed: 1700754185
  SIMD kernels: AVX2
  parallelization: PTHREADS (40 threads), thread pinning: OFF
Reading input trees from file: ExaBayes_topologies.run-0.3FTx_resubmission_DSD_v2

ERROR: ERROR reading tree file (LIBPLL-111): syntax error, unexpected STRING, expecting OPAR. (line 1 column 0-6)

Thanks in advance,
Tobias Senoner

[Alexandros Stamatakis]

Could you send us one of these tree files to debug it?

Alexis

> <https://groups.google.com/d/msgid/raxml/6e4536b2-a595-4665-86b1-47fe5d88c9den%40googlegroups.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/raxml/6e4536b2-a595-4665-86b1-47fe5d88c9den%40googlegroups.com?utm_medium=email&utm_source=footer>>.

>
> --
> Alexandros (Alexis) Stamatakis
>
> Research Group Leader, Heidelberg Institute for Theoretical Studies
> Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology
> Affiliated Scientist, Evolutionary Genetics and Paleogenomics (EGP)
> lab,
> Institute of Molecular Biology and Biotechnology, Foundation for
> Research and Technology Hellas
>

> www.exelixis-lab.org <http://www.exelixis-lab.org>

>
> --
> You received this message because you are subscribed to the Google
> Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to raxml+un...@googlegroups.com
> <mailto:raxml+un...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/raxml/0bff8430-a9a4-4c31-a727-6ec0bbf0a491n%40googlegroups.com <https://groups.google.com/d/msgid/raxml/0bff8430-a9a4-4c31-a727-6ec0bbf0a491n%40googlegroups.com?utm_medium=email&utm_source=footer>.

--
Alexandros (Alexis) Stamatakis

ERA Chair, Institute of Computer Science, Foundation for Research and
Technology - Hellas

Research Group Leader, Heidelberg Institute for Theoretical Studies
Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology

www.biocomp.gr (Crete lab)
www.exelixis-lab.org (Heidelberg lab)

[Tobias Senoner]

Hi Alexis,

Thank you for offering to debug this.

I have uploaded the tree files at the following link: https://rostlab.org/public/senoner/ExaBayes_RAxML/

Please let me know if you can find the issue and how to consensus the trees using RAxML-ng.

Best regards,

Tobias

[Oleksiy Kozlov]

Hi Tobias,
-
the problem is simple: ExaBayes output is in NEXUS format, but RAxML-NG and RAxML do only support
NEWICK tree file as input.

So you would need to convert these files to Newick first.

However, I'd rather recommend to simply use "consense" command of ExaBayes.
It is not parallelized, but on my machine, it took ~10 minutes per file to build MRE consensus.
So in total, you'd have to wait for 2-3 hours, which is still acceptable I guess.

Hope this helps,
Oleksiy

> <https://groups.google.com/d/msgid/raxml/6e4536b2-a595-4665-86b1-47fe5d88c9den%40googlegroups.com> <https://groups.google.com/d/msgid/raxml/6e4536b2-a595-4665-86b1-47fe5d88c9den%40googlegroups.com <https://groups.google.com/d/msgid/raxml/6e4536b2-a595-4665-86b1-47fe5d88c9den%40googlegroups.com>>
> > >
> >
> <https://groups.google.com/d/msgid/raxml/6e4536b2-a595-4665-86b1-47fe5d88c9den%40googlegroups.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/raxml/6e4536b2-a595-4665-86b1-47fe5d88c9den%40googlegroups.com?utm_medium=email&utm_source=footer> <https://groups.google.com/d/msgid/raxml/6e4536b2-a595-4665-86b1-47fe5d88c9den%40googlegroups.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/raxml/6e4536b2-a595-4665-86b1-47fe5d88c9den%40googlegroups.com?utm_medium=email&utm_source=footer>>>.

> >
> > --
> > Alexandros (Alexis) Stamatakis
> >
> > Research Group Leader, Heidelberg Institute for Theoretical Studies
> > Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology
> > Affiliated Scientist, Evolutionary Genetics and Paleogenomics (EGP)
> > lab,
> > Institute of Molecular Biology and Biotechnology, Foundation for
> > Research and Technology Hellas
> >

> > www.exelixis-lab.org <http://www.exelixis-lab.org> <http://www.exelixis-lab.org

> <http://www.exelixis-lab.org>>
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "raxml" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> > an email to raxml+un...@googlegroups.com
> > <mailto:raxml+un...@googlegroups.com>.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/raxml/0bff8430-a9a4-4c31-a727-6ec0bbf0a491n%40googlegroups.com

> <https://groups.google.com/d/msgid/raxml/0bff8430-a9a4-4c31-a727-6ec0bbf0a491n%40googlegroups.com> <https://groups.google.com/d/msgid/raxml/0bff8430-a9a4-4c31-a727-6ec0bbf0a491n%40googlegroups.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/raxml/0bff8430-a9a4-4c31-a727-6ec0bbf0a491n%40googlegroups.com?utm_medium=email&utm_source=footer>>.

>
> --
> Alexandros (Alexis) Stamatakis
>
> ERA Chair, Institute of Computer Science, Foundation for Research and
> Technology - Hellas
> Research Group Leader, Heidelberg Institute for Theoretical Studies
> Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology
>

> www.biocomp.gr <http://www.biocomp.gr> (Crete lab)
> www.exelixis-lab.org <http://www.exelixis-lab.org> (Heidelberg lab)

>
> --
> You received this message because you are subscribed to the Google Groups "raxml" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
> raxml+un...@googlegroups.com <mailto:raxml+un...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/raxml/7bca17ae-2cd7-4053-80b6-276a1dbcc162n%40googlegroups.com
> <https://groups.google.com/d/msgid/raxml/7bca17ae-2cd7-4053-80b6-276a1dbcc162n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Oleksiy Kozlov]

Quick update: if you still manage to convert NEXUS to NEWICK, you'll get quite an impressive speedup
with RAxML-NG/RAxML (few seconds vs. 10 minutes).

Unfortunately, I couldn't find any easy-to-use tool for this conversion. I just extracted topologies
using grep/sed like this:

$ grep "tree gen" ExaBayes_topologies.run-0.3FTx_resubmission_DSD_v2 | sed 's/.*U\] //' > exab.nw

This works, but requires an additional post-processing step to replace taxon numbers with original
taxon names from NEXUS. Should be relatively straight-forward to write a small Python script for
this task.

[Grimm]

Re: " Unfortunately, I couldn't find any easy-to-use tool for this conversion. " – I always used Mesquite for that, you open the NEXUS tree file (no matter which NEXUS subformat, it handles them all) and you can export a tree as "phylip"-formatted, which is NEWICK-format.

PS Mesquite pretty much can do all the file transformations (tree, alignments), only limitation is, it cannot well handle very large matrices (I guess because it graphically views them instantly)

[Tobias Senoner]

Hi Oleksiy,

Thank you for your response and for the solution.

The tree I mentioned was just a basic example. I have several other trees that have not converged even after seven days. Therefore, I plan to implement your suggested method of initially converting the tree to the NEWICK format, followed by RAxML-NG, and finally renaming the taxon numbers with a Python script.

Best,
Tobias