AbySS hangs at reading files

[amit upadhyay]

Hello,

I have two fastq files (paired-end library) about 25GB each. ABySS was stuck at Reading these files for about 18 hours. Is this because of the large size of the file or incorrect compilation?

Thank you,
Amit.

[Rick Westerman]

By "stuck" do you mean that no CPU time was being used or that the CPUs were working but no visible output was being generated?  If the latter then this is normal behavior.  The solution, if you have multiple CPUs and fast disk I/O, is to split up the files into to multiple smaller ones before giving them to ABySS.   If the problem is the former one -- no CPU time being used -- or a different problem entirely then please give more details.

[amit upadhyay]

Hello,

Thank you for the reply.

I tried splitting up the input files for paired end reads. I ran the command as follows:

abyss-pe np=16  name=P_logei k=64 lib="libA libB" libA=" pa_1.fq  pb_1.fq pc_1.fq pd_1.fq pe_1.fq pf_1.fq pg_1.fq ph_1.fq" libB="pa_2.fq  pb_2.fq pc_2.fq pd_2.fq pe_2.fq pf_2.fq pg_2.fq ph_2.fq"

The log file shows that it starts reading all files and then it seems to finish reading just one and nothing happens for 20 hours. Is this the right way to run abyss for split input files?

The output in the log file is as follows:

/opt/sgi/mpt/mpt-2.04/bin/mpirun -np 16 ABYSS-P -k64 -q3   --coverage-hist=coverage.hist -s P_logei-bubbles.fa  -o P_logei-1.fa pa_1.fq pb_1.fq pc_1.fq pd_1.fq pe_1.fq pf_1.fq pg_1.fq ph_1.fq pa_2.fq pb_2.fq pc_2.fq pd_2.fq pe_2.fq pf_2.fq pg_2.fq ph_2.fq

MPI: Module xpmem is not loaded.  MPI memory mapping features (single copy transfers, one-sided communication and SHMEM functions) are disabled. XPMEM must be enabled on your system to use these features.

ABySS 1.5.1

ABYSS-P -k64 -q3 --coverage-hist=coverage.hist -s P_logei-bubbles.fa -o P_logei-1.fa pa_1.fq pb_1.fq pc_1.fq pd_1.fq pe_1.fq pf_1.fq pg_1.fq ph_1.fq pa_2.fq pb_2.fq pc_2.fq pd_2.fq pe_2.fq pf_2.fq pg_2.fq ph_2.fq

Running on 16 processors

0: Running on host harpoon3

4: Running on host harpoon3

1: Running on host harpoon3

3: Running on host harpoon3

5: Running on host harpoon3

6: Running on host harpoon3

7: Running on host harpoon3

14: Running on host harpoon3

2: Running on host harpoon3

8: Running on host harpoon3

9: Running on host harpoon3

10: Running on host harpoon3

11: Running on host harpoon3

12: Running on host harpoon3

13: Running on host harpoon3

15: Running on host harpoon3

0: Reading `pa_1.fq'...

3: Reading `pd_1.fq'...

6: Reading `pg_1.fq'...

2: Reading `pc_1.fq'...

4: Reading `pe_1.fq'...

5: Reading `pf_1.fq'...

7: Reading `ph_1.fq'...

9: Reading `pb_2.fq'...

11: Reading `pd_2.fq'...

1: Reading `pb_1.fq'...

10: Reading `pc_2.fq'...

12: Reading `pe_2.fq'...

13: Reading `pf_2.fq'...

8: Reading `pa_2.fq'...

14: Reading `pg_2.fq'...

15: Reading `ph_2.fq'...

`pa_2.fq': discarded 24912 reads shorter than 64 bases

`pa_2.fq': discarded 32 reads containing non-ACGT characters

Thank you,

Amit.

--
You received this message because you are subscribed to a topic in the Google Groups "ABySS" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/abyss-users/LqNja8oWALI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to abyss-users...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rick Westerman]

Your command line is incorrect.  You need to put pairs next to each other in their own library.   The documentation has the following example.

---

abyss-pe k=64 name=ecoli lib='pe200 pe500' \
    pe200='pe200_1.fa pe200_2.fa' pe500='pe500_1.fa pe500_2.fa' \
    se='se1.fa se2.fa'

------------

In your case you would do:

abyss-pe np=16  name=P_logei k=64 lib="libA libB libC libD" libA=" pa_1.fq pa_2.fq" libB=" pb_1.fq pb_2.fq" libC="pc_1.fq pc_2.fq" etc.

On Sunday, July 6, 2014 7:49:12 PM UTC-4, amit upadhyay wrote:

Hello,

Thank you for the reply.

I tried splitting up the input files for paired end reads. I ran the command as follows:

abyss-pe np=16  name=P_logei k=64 lib="libA libB" libA=" pa_1.fq  pb_1.fq pc_1.fq pd_1.fq pe_1.fq pf_1.fq pg_1.fq ph_1.fq" libB="pa_2.fq pa pb_2.fq pc_2.fq pd_2.fq pe_2.fq pf_2.fq pg_2.fq ph_2.fq"