RAxML running forever with no progress for some replicates of a bootstrapped run

[siavash mirarab]

Hi Alexis, 

I am running the latest version of RAxML (retrieved from git repository on September 25) with bootstrapping on some 800 alignments, some of them very gappy. RAxML runs fine on most of these alignments, but for some of these (about 30 of them) it runs for some number of replicates, and then stops making any progress. It runs forever but fails to finish that one replicate. If I run with a different seed number, it usually runs fine again for some number of replicates, but eventually gets stuck again (other replicates finishes in about a minute). This problem could possibly have something to do with high levels of gapiness, though I am not sure, because on some of the gappiest alignments it actually runs fine, but on some of them it does not.  

I am attaching one particular alignment where I had the problem. To reproduce, run the following command:

raxmlHPC-PTHREADS-SSE3-git-Sept25 -m GTRCAT -n ml -s FNA2AA.7511.input.FNA2AA.aligned.phylip -N 107 -b 200107107 -T 4 -p 110107

It works fine for the first 8 replicates (less than a minute each on my machine), but then gets stuck, and fails to make any progress from there on, even after running for a couple of hours.

Regards

Siavash 

[Fidel Botero]

Hi Siavash,

Looking at your command line I see you're indicating -T 4. Do you actually have 4 different CPUs on your machine? A bad setting of the T parameter will produce an important decrease in performance (see Manual of RAxML). Have you tried with a smaller value for T ? Keep in mind it can not be <2 or the PHTHREADS version will produce an error).

Hope you can solve it !

Best,

Fidel

2012/9/26 siavash mirarab <smir...@gmail.com>

--
Fidel Botero
Biologiste
Doctorant Université Montpellier 2
Équipe Phylogénie et Évolution Moléculaires
Institut des Sciences de l'Évolution de Montpellier - ISE-M (UMR 5554 UM2-CNRS)
04 67 14 92 00

[siavash mirarab]

Yes, I have 16 cpus, and the same issue happens with one cpu too. Also, note that if that was the case, all replicates would slow down, but here a certain number of replicates run quickly with no problem (less than a minute), but then on one replicate it gets stuck and never finishes (even after hours of running). This has nothing to do with the slowness of computer/cpu/infrastructure/etc., because it always gets stuck on the same replicate. It seems raxml is either unable to reach a local optimum and keeps running (maybe due to some weird properties of the alignment) or is stuck in an endless loop. 

Regards

Siavash

[Andre J. Aberer]

Hi Siavash,

maybe try the run with -f j and explicitly check out the alignment
replicates...I'd expect, you'll see what's wrong with it then.

HTH,
Andre

--
Best regards,
Andre J. Aberer

M.Sc. (Bioinformatics)
Scientific Computing Group

Heidelberg Institute for Theoretical Studies (HITS gGmbH)
Schloss-Wolfsbrunnenweg 35
D-69118 Heidelberg

Tel.: +49 6221 533 264
Fax: +49 6221 533 298
WWW: http://www.exelixis-lab.org
http://www.h-its.org/english/research/sco/index.php

Amtgericht Mannheim / HRB 337446
Managing Directors: Dr. h.c. Dr.-Ing. E.h. Klaus Tschira, Prof. Dr.-Ing. Andreas Reuter

[Alexandros Stamatakis]

Hi Siavash,

There is no need to CC Andre or me when you are posting to the google
group, we will get the message twice.

Please send me the input files, but I can't promise that I'll be able to
debug this any time soon.

In the meantime you can try running with GTRGAMMA to see if the same
thing happens.

Alexis

--
Alexandros (Alexis) Stamatakis

Research Group Leader, Heidelberg Institute for Theoretical Studies
(HITS gGmbH)
Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology
Adjunct Professor, Dept. of Ecology and Evolutionary Biology, University
of Arizona at Tucson

Schloss-Wolfsbrunnenweg 35
D-69118 Heidelberg
Germany

Tel: +49 1511 7496080 (Mobile)
+49 6221 533240 (Office)
Fax: +49 6221 533298
Skype: stamatak
Email: Alexandros...@h-its.org

WWW: http://www.exelixis-lab.org
http://www.h-its.org/english/research/sco/index.php

Amtsgericht Mannheim/HRB 337446
Managing Directors:
Dr. h.c. Dr.-Ing. E.h. Klaus Tschira/Prof. Dr.-Ing. Andreas Reuter

[bli]

On Wednesday, September 26, 2012 4:41:29 PM UTC+1, siavash mirarab wrote:

I am running the latest version of RAxML (retrieved from git repository on September 25) with bootstrapping on some 800 alignments, some of them very gappy. RAxML runs fine on most of these alignments, but for some of these (about 30 of them) it runs for some number of replicates, and then stops making any progress. It runs forever but fails to finish that one replicate. If I run with a different seed number, it usually runs fine again for some number of replicates, but eventually gets stuck again (other replicates finishes in about a minute). This problem could possibly have something to do with high levels of gapiness, though I am not sure, because on some of the gappiest alignments it actually runs fine, but on some of them it does not.  

For the record, this looks quite similar to the problem I reported here:
https://groups.google.com/forum/?fromgroups=#!topic/raxml/brc3_rOOuFc

And I suspect this could be related to a lack of informative characters in some of the bootstrapped replicates, because for me the problem happens with datasets where a high number of characters were recoded using IUPAC ambiguity codes.

Blaise