[maker-devel] Contigs failing - could not make datastore directory

[Felix Bemm]

Hi,

I have a problem with Contigs failing like this:

examining contents of the fasta file and run log
ERROR: could not make datastore directory
--> rank=11, hostname=r5n04
ERROR: Failed while examining contents of the fasta file and run log
ERROR: Chunk failed at level:0, tier_type:0
FAILED CONTIG:MA_gen_v8.05_c31675

There is enough space available on both the storage device and the tmp
device that is being used. The datastore log files looks like this:

MA_gen_v8.05_c1149 FAILED
MA_gen_v8.05_c1153 FAILED
MA_gen_v8.05_c1154 FAILED
MA_gen_v8.05_c1155 FAILED
MA_gen_v8.05_c1156 FAILED

Since there is no datastore for this contigs, I can not provide any more
informationen. Approx. 1/3 of the contigs were annotated, after that
maker started to fail. It kept running for a while and finishing some of
the contigs in the second or third try like this:

MA_gen_v8.05_c4443 FAILED
MA_gen_v8.05_c4443 FAILED
MA_gen_v8.05_c4443 MA_gen_v8.05M_datastore/A7/9B/MA_gen_v8.05_c4443/ STARTED
MA_gen_v8.05_c4443 MA_gen_v8.05M_datastore/A7/9B/MA_gen_v8.05_c4443/
FINISHED

Any ideas what can cause this problems?

Best regards
Felix

--
Felix Bemm
Department of Bioinformatics
University of Würzburg, Germany
Tel: +49 931 - 31 83696
Fax: +49 931 - 31 84552
felix...@uni-wuerzburg.de

_______________________________________________
maker-devel mailing list
maker...@box290.bluehost.com
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

[Carson Holt]

Let me know what you find. With some failures it's hard to identify the
problem especially with long running processes, which is why I made MAKER
restartable, so you can at least get completion by just launching again.

--Carson

[Carson Holt]

Hi Felix,

There are other potential issues as well, for example a file quota limit
set by your administrator or full local /tmp which will be unique to each
node and cannot be queried directly from the root node on a cluster. On a
cluster one node may also not have the NFS location mounted. Or it could
be system related, for example on ibrix NFS mounted systems you can get a
weird filesystem issue with ghost files and directories when using MAKER
that can neither be created nor deleted. Try running the failed contigs
in a different directory (even if it's on the same disk). Also what
version of MAKER are you using?

--Carson



On 10/4/13 3:10 AM, "Felix Bemm" <felix...@uni-wuerzburg.de> wrote:

[Felix Bemm]

Hi Carson,

there is definitely no quota neither a full local tmp device. I changed
to another tmp device, which is unfortunately residing on nfs but now it
seems to work. I am using the latest version of maker (2.28). The
complete job is running on a single machine with 32 mpi threads.

Bests
Felix

[Carson Holt]

You can also try the 2.29 beta if you want (on the website). Also when
MAKER exits, it tries to delete anything left behind in temporary folders,
so is it possible that it is creating space on exit after filling up /tmp?

Given that the issue seems to go away when you switch to another
directory, it suggests that this might be the kind of thing that is
specific to something about the current state of your your system.

Thanks,
Carson

[Felix Bemm]

Do you have a changelog for 2.29b?

On 07.10.2013 16:33, Carson Holt wrote:
> You can also try the 2.29 beta if you want (on the website). Also when
> MAKER exits, it tries to delete anything left behind in temporary folders,
> so is it possible that it is creating space on exit after filling up /tmp?

I will try to monitor that!

> Given that the issue seems to go away when you switch to another
> directory, it suggests that this might be the kind of thing that is
> specific to something about the current state of your your system.

I agree with that. Interestingly it happens with the only directory
where I would not expect that.