parsing file aln.part

[gang liu]

Hi, Colleagues,

I am running the exabayes in a computer cluster. When I submitted the job, but failed. According to the error report, please see below:

The program was called as follows: 

exabayes -f mafft-nexus-internal-trimmed-gblocks-clean-75p.phylip -n run1 -q aln.part -s 26069 -c config.nexus -R 4 

================================================================

You provided an alignment file in phylip format. Trying to parse it...

Error: parsing file aln.part failed. 

Please double-check, whether each site is assigned a partition. 

Side note: the notation for choosing every n-th (e.g., triplet) 

character has changed from '\3' to '/3'.[holy2a02207:58238] *** Process received signal ***

[holy2a02207:58238] Signal: Aborted (6)

[holy2a02207:58238] Signal code:  (-6)

[holy2a02207:58238] [ 0] /usr/lib64/libpthread.so.0(+0xf5e0)[0x2b428c3c95e0]

[holy2a02207:58238] [ 1] /usr/lib64/libc.so.6(gsignal+0x37)[0x2b428c60f1f7]

[holy2a02207:58238] [ 2] /usr/lib64/libc.so.6(abort+0x148)[0x2b428c6108e8]

[holy2a02207:58238] [ 3] /usr/lib64/libc.so.6(+0x2e266)[0x2b428c608266]

[holy2a02207:58238] [ 4] /usr/lib64/libc.so.6(+0x2e312)[0x2b428c608312]

[holy2a02207:58238] [ 5] exabayes[0x4638f1]

[holy2a02207:58238] [ 6] exabayes[0x43461a]

[holy2a02207:58238] [ 7] exabayes[0x435ea9]

[holy2a02207:58238] [ 8] exabayes[0x412613]

[holy2a02207:58238] [ 9] exabayes[0x41380d]

[holy2a02207:58238] [10] exabayes[0x40f6ec]

[holy2a02207:58238] [11] /usr/lib64/libc.so.6(__libc_start_main+0xf5)[0x2b428c5fbc05]

[holy2a02207:58238] [12] exabayes[0x41115f]

[holy2a02207:58238] *** End of error message ***

--------------------------------------------------------------------------

mpirun noticed that process rank 0 with PID 0 on node holy2a02207 exited on signal 6 (Aborted).

--------------------------------------------------------------------------

Could you help to take a look and tell me how I can address such issue? 

Thank you very much. 

Best,

Gang

[Alexandros Stamatakis]

Can you please share your aln.part file with us.

Apparently there is an error with the partition file format.

Alexis

> --
> You received this message because you are subscribed to the Google
> Groups "ExaBayes" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to exabayes+u...@googlegroups.com
> <mailto:exabayes+u...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/exabayes/cf4c59f6-5874-4f1f-ae9b-3a5b6f0cb9a5%40googlegroups.com
> <https://groups.google.com/d/msgid/exabayes/cf4c59f6-5874-4f1f-ae9b-3a5b6f0cb9a5%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

--
Alexandros (Alexis) Stamatakis

Research Group Leader, Heidelberg Institute for Theoretical Studies
Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology

www.exelixis-lab.org

[gang liu]

Yes, thank you, Alexis. Here attached are the aln.part and config.nexus files. I try to use the below sbatch job:

#!/bin/bash

#SBATCH -J mpitest

#SBATCH -o mpitest.out

#SBATCH -e mpitest.err

#SBATCH -p general

#SBATCH -n 8

#SBATCH -t 0-12:00

#SBATCH --mem-per-cpu=40000

source new-modules.sh  

module load jdk/1.8.0_45-fasrc01

module load gcc/7.1.0-fasrc01 openmpi/2.1.0-fasrc02 exabayes/1.5-fasrc02

mpirun -np 8 exabayes -f mafft-nexus-internal-trimmed-gblocks-clean-75p.phylip -n run1 -q /n/scratchlfs/lius_lab/lius/uce-tutorial/taxon-sets/all/mafft-nexus-internal-trimmed-gblocks-clean-75p-exbayes/aln.part -s 26069 -c /n/scratchlfs/lius_lab/lius/uce-tutorial/taxon-sets/all/mafft-nexus-internal-trimmed-gblocks-clean-75p-exbayes/config.nexus -R 4 

The job does seem running for 12 hours, but only the file ExaBayes_info.run1, which inside reads like below:

You provided an alignment file in phylip format. Trying to parse it...

================================================================

Then after 12 hours, the job stopped because of the timeout. So do you think my command is wrong or the time is not enough? Thank you very much.

Best wishes,

Gang

> an email to exab...@googlegroups.com
> <mailto:exab...@googlegroups.com>.

[Alexandros Stamatakis]

The partition file looks okay, but only contains one partition, did you
try to run it without a partition file?

The delay might be due to the fact that you are not using enough CPU
cores, so basically you should use one core per 1000 MSA sites and per
chain, that makes roughly 500 cores per chain, so if you run 4 MCMCMCM
chains you could use 2000 cores for this.

Alexis

> > an email to exab...@googlegroups.com <javascript:>
> > <mailto:exab...@googlegroups.com <javascript:>>.

> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/exabayes/cf4c59f6-5874-4f1f-ae9b-3a5b6f0cb9a5%40googlegroups.com
> <https://groups.google.com/d/msgid/exabayes/cf4c59f6-5874-4f1f-ae9b-3a5b6f0cb9a5%40googlegroups.com>
>
> >

> <https://groups.google.com/d/msgid/exabayes/cf4c59f6-5874-4f1f-ae9b-3a5b6f0cb9a5%40googlegroups.com?utm_medium=email&utm_source=footer

> <https://groups.google.com/d/msgid/exabayes/cf4c59f6-5874-4f1f-ae9b-3a5b6f0cb9a5%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> > For more options, visit https://groups.google.com/d/optout

> <https://groups.google.com/d/optout>.

>
> --
> Alexandros (Alexis) Stamatakis
>
> Research Group Leader, Heidelberg Institute for Theoretical Studies
> Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology
>

> www.exelixis-lab.org <http://www.exelixis-lab.org>

>
> --
> You received this message because you are subscribed to the Google
> Groups "ExaBayes" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to exabayes+u...@googlegroups.com
> <mailto:exabayes+u...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/exabayes/2ddc1142-4326-44a6-beab-912b2c886b3e%40googlegroups.com
> <https://groups.google.com/d/msgid/exabayes/2ddc1142-4326-44a6-beab-912b2c886b3e%40googlegroups.com?utm_medium=email&utm_source=footer>.

[gang liu]

Dear Alexis, Thank you very much for your kind look. I checked the PHYLIP file used, and based on the head of PHYLIP, it read 4 407584. However , in the aln.part file, I used DNA, p1=1-499929, does it mean it should be 1-407584?

In addition, I felt a little confused about the preparation of the partition file. I want to run with a partition file, cause it will save time. So for my case, can I use  like below? Is there some rules for generating the aln.part file? Thank you very much

DNA, gene1=1-10000
DNA, gene2-codonPos1=10001-15000\3
DNA, gene2-codonPos2=10002-15000\3
DNA, gene2-codonPos3=10003-15000\3
DNA, composit=15001-20000,20001-407584 

> an email to exab...@googlegroups.com
> <mailto:exab...@googlegroups.com>.

[Alexandros Stamatakis]

On 17.05.19 07:14, gang liu wrote:
> Dear Alexis, Thank you very much for your kind look. I checked the
> PHYLIP file used, and based on the head of PHYLIP, it read 4 407584.
> However , in the aln.part file, I used DNA, p1=1-499929, does it mean it
> should be 1-407584?

yes

>
> In addition, I felt a little confused about the preparation of the
> partition file. I want to run with a partition file, cause it will save
> time.

I won't save time, partitioned analyses take longer than unpartitioned
ones ...

> So for my case, can I use  like below? Is there some rules for
> generating the aln.part file? Thank you very much
>
> DNA, gene1=1-10000
> DNA, gene2-codonPos1=10001-15000\3
> DNA, gene2-codonPos2=10002-15000\3
> DNA, gene2-codonPos3=10003-15000\3
> DNA, composit=15001-20000,20001-407584

That looks about right, be careful with \ versus / I don't remember
which one is used in ExaBayes partition files,

Alexis

> https://groups.google.com/d/msgid/exabayes/2ddc1142-4326-44a6-beab-912b2c886b3e%40googlegroups.com
> <https://groups.google.com/d/msgid/exabayes/2ddc1142-4326-44a6-beab-912b2c886b3e%40googlegroups.com>
>
> >
> <https://groups.google.com/d/msgid/exabayes/2ddc1142-4326-44a6-beab-912b2c886b3e%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/exabayes/2ddc1142-4326-44a6-beab-912b2c886b3e%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>

> > For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
>
> --
> Alexandros (Alexis) Stamatakis
>
> Research Group Leader, Heidelberg Institute for Theoretical Studies
> Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology
>
> www.exelixis-lab.org <http://www.exelixis-lab.org>
>
> --
> You received this message because you are subscribed to the Google
> Groups "ExaBayes" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to exabayes+u...@googlegroups.com
> <mailto:exabayes+u...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/exabayes/4ef31d24-ccf5-4bee-af59-93ac6418c588%40googlegroups.com
> <https://groups.google.com/d/msgid/exabayes/4ef31d24-ccf5-4bee-af59-93ac6418c588%40googlegroups.com?utm_medium=email&utm_source=footer>.

[gang liu]

Hi Alexis,

Based on your suggestions, it works fine. Thank you. The aln.part is the cause.

Best,

Gang

> an email to exab...@googlegroups.com
> <mailto:exab...@googlegroups.com>.

[mmi...@sdsc.edu]

Hi I wanted to note, vis a vis the / vs \ issue that the example file that distributes with exabayes fails on our machines.

The correct format for the file in the distribution should be this:

DNA, gene1-c1=1-1000/3
DNA, gene1-c2=2-1000/3
DNA, gene1-c3=3-1000/3
DNA, gene2=1001-1269

The one distributed with the code fails:

DNA, gene1-c1=1-1000\3 

DNA, gene1-c2=2-1000\3 

DNA, gene1-c3=3-1000\3 

DNA, gene2=1001-1269

[Alexandros Stamatakis]

Thanks a lot Mark,

I will try to get in touch with Andre such that he fixes it.

Alexis

> <http://www.exelixis-lab.org <http://www.exelixis-lab.org>>

> <http://www.exelixis-lab.org <http://www.exelixis-lab.org>>

> >
> > --
> > You received this message because you are subscribed to the
> Google
> > Groups "ExaBayes" group.
> > To unsubscribe from this group and stop receiving emails from
> it, send
>
> > an email to exab...@googlegroups.com
> > <mailto:exab...@googlegroups.com>.
>
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/exabayes/4ef31d24-ccf5-4bee-af59-93ac6418c588%40googlegroups.com <https://groups.google.com/d/msgid/exabayes/4ef31d24-ccf5-4bee-af59-93ac6418c588%40googlegroups.com>
> >

> <https://groups.google.com/d/msgid/exabayes/4ef31d24-ccf5-4bee-af59-93ac6418c588%40googlegroups.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/exabayes/4ef31d24-ccf5-4bee-af59-93ac6418c588%40googlegroups.com?utm_medium=email&utm_source=footer>>.

> > For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
>
> --
> Alexandros (Alexis) Stamatakis
>
> Research Group Leader, Heidelberg Institute for Theoretical Studies
> Full Professor, Dept. of Informatics, Karlsruhe Institute of
> Technology
>
> www.exelixis-lab.org <http://www.exelixis-lab.org>
>
> --
> You received this message because you are subscribed to the Google
> Groups "ExaBayes" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to exabayes+u...@googlegroups.com
> <mailto:exabayes+u...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/exabayes/0557b33a-c463-4191-9615-30180f9d1075n%40googlegroups.com <https://groups.google.com/d/msgid/exabayes/0557b33a-c463-4191-9615-30180f9d1075n%40googlegroups.com?utm_medium=email&utm_source=footer>.

--
Alexandros (Alexis) Stamatakis

ERA Chair, Institute of Computer Science, Foundation for Research and
Technology - Hellas

Research Group Leader, Heidelberg Institute for Theoretical Studies
Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology

www.biocomp.gr (Crete lab)
www.exelixis-lab.org (Heidelberg lab)