split_library fastq error

[Lekha Menon]

Hi,

I am a newbie to Illumina data. I have been trying to do split_libraries_fastq.py and also multiple_split_library commands and got the following error:

(qiime1) ubuntu@ubuntu-vm:~$ multiple_split_libraries_fastq.py -i join_reads -o split_library --demultiplexing_method sampleid_by_file

Traceback (most recent call last):

File "/opt/miniconda3/envs/qiime1/bin/multiple_split_libraries_fastq.py", line 4, in <module>

__import__('pkg_resources').run_script('qiime==1.9.1', 'multiple_split_libraries_fastq.py')

File "/opt/miniconda3/envs/qiime1/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/__init__.py", line 744, in run_script

File "/opt/miniconda3/envs/qiime1/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/__init__.py", line 1499, in run_script

File "/opt/miniconda3/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/multiple_split_libraries_fastq.py", line 219, in <module>

main()

File "/opt/miniconda3/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/multiple_split_libraries_fastq.py", line 216, in main

close_logger_on_success=True)

File "/opt/miniconda3/envs/qiime1/lib/python2.7/site-packages/qiime/workflow/util.py", line 122, in call_commands_serially

raise WorkflowError(msg)

qiime.workflow.util.WorkflowError:

*** ERROR RAISED DURING STEP: split_libraries_fastq.py

Command run was:

split_libraries_fastq.py -i /home/ubuntu/join_reads/T0_join.fastq,/home/ubuntu/join_reads/25_2.join.fastq,/home/ubuntu/join_reads/18_2_join.fastq,/home/ubuntu/join_reads/18_6_join.fastq,/home/ubuntu/join_reads/25_4_join.fastq,/home/ubuntu/join_reads/30_4_join.fastq,/home/ubuntu/join_reads/18_4_join.fastq,/home/ubuntu/join_reads/25_6_join.fastq,/home/ubuntu/join_reads/30_2_join.fastq,/home/ubuntu/join_reads/30_6_join.fastq --sample_ids T0,25,18,18,25,30,18,25,30,30 -o /home/ubuntu/split_library --barcode_type 'not-barcoded'

Command returned exit status: 1

Stdout:

Stderr

Traceback (most recent call last):

File "/opt/miniconda3/envs/qiime1/bin/split_libraries_fastq.py", line 4, in <module>

__import__('pkg_resources').run_script('qiime==1.9.1', 'split_libraries_fastq.py')

File "/opt/miniconda3/envs/qiime1/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/__init__.py", line 744, in run_script

File "/opt/miniconda3/envs/qiime1/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/__init__.py", line 1499, in run_script

File "/opt/miniconda3/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/split_libraries_fastq.py", line 365, in <module>

main()

File "/opt/miniconda3/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/split_libraries_fastq.py", line 344, in main

for fasta_header, sequence, quality, seq_id in seq_generator:

File "/opt/miniconda3/envs/qiime1/lib/python2.7/site-packages/qiime/split_libraries_fastq.py", line 239, in process_fastq_single_end_read_file_no_barcode

phred_offset=phred_offset):

File "/opt/miniconda3/envs/qiime1/lib/python2.7/site-packages/qiime/split_libraries_fastq.py", line 317, in process_fastq_single_end_read_file

parse_fastq(fastq_read_f, strict=False, phred_offset=phred_offset)):

File "/opt/miniconda3/envs/qiime1/lib/python2.7/site-packages/skbio/parse/sequences/fastq.py", line 147, in parse_fastq

raise FastqParseError("Incomplete FASTQ record found at end "

skbio.parse.sequences._exception.FastqParseError: Incomplete FASTQ record found at end of file

The files of the fastq are in a directory called join_reads. I tried attaching the file which was zipped but it gave me an error. Hence could not attach the file. 

Your help is much appreciated.

Thanks,

Lekha

[Yoshiki Vázquez Baeza]

Hello,

The problem seems to be due to a malformed FASTQ file, my recommendation is that you look over the files that multiple_split_libraries_fastq.py is processing and you check if any of these look incorrect to you. I would suggest using the tail UNIX command. Remember the format should be:

@sequenceidentifier

ACAAAAAAAAAAA .... sequence

+

!@#$$% ...... quality

If a record is incomplete then you can either trace when the file became corrupted or delete that last record.

Thanks!

Yoshiki.

[Lekha Menon]

Thank you. That is really helpful. We corrected it and worked.