Open picking returning error: possibly due to problem in joining

[mpbr...@tamu.edu]

I am working with some Illumina data that has been run before without being joined. (Workflow: Multiple split libraries>truncate reverse primer>closed and open picking> core diversity analysis)

To process it again with joining, I moved my data to a different computer and therefore a different Virtual Box (but still using QIIME 1.9). We learned soon after that this data, which was processed by a different sequencing center than our usual, was given to us multiplexed and without barcodes or sample ids. With that in mind, we developed a new workflow: join paired ends (we've tried both fastqjoin and SeqPrep) > convert fastaqual to fastq > add qiime labels > open/closed picking >core diversity analysis. 

No matter what I do, or which workflow I use on this new computer, the open picking returns an error saying there's an empty fasta file being passed to it. I have validated my mapping file and the demultiplexed fasta file. No other scripts return an error.

I have both fecal and skin samples and I am running their 16S data separately. The fecal sequences are considerably smaller fastq files than the skin sequences, but this had never caused any problems until I moved to this new computer.

Is it worth uninstalling and reinstalling the Virtual Box? Are there other joining methods I could try?

[Colin Brislawn]

Hello there,

Can you post the exact qiime script you ran and also the error it throws? This will us troubleshoot your issue.

Also, qiime now comes with script to process separate files. Check these out!

http://qiime.org/scripts/multiple_join_paired_ends.html

http://qiime.org/scripts/multiple_split_libraries_fastq.html

Colin

[mpbr...@tamu.edu]

qiime@qiime-190-virtual-box:~/Desktop/Florida_Dogs$ pick_open_reference_otus.py -i id_added_fecal/combined_seqs.fna -o open_picking_fecal -p canine_16S_params.txt
Traceback (most recent call last):
  File "/usr/local/bin/pick_open_reference_otus.py", line 453, in <module>
    main()
  File "/usr/local/bin/pick_open_reference_otus.py", line 432, in main
    minimum_failure_threshold=minimum_failure_threshold)
  File "/usr/local/lib/python2.7/dist-packages/qiime/workflow/pick_open_reference_otus.py", line 1071, in pick_subsampled_open_reference_otus
    status_update_callback=status_update_callback)
  File "/usr/local/lib/python2.7/dist-packages/qiime/workflow/pick_open_reference_otus.py", line 327, in align_and_tree
    close_logger_on_success=close_logger_on_success)
  File "/usr/local/lib/python2.7/dist-packages/qiime/workflow/util.py", line 122, in call_commands_serially
    raise WorkflowError(msg)
qiime.workflow.util.WorkflowError: 
*** ERROR RAISED DURING STEP: Filter alignment
Command run was:
 filter_alignment.py -o open_picking_fecal/pynast_aligned_seqs -i open_picking_fecal/pynast_aligned_seqs/rep_set_aligned.fasta 
Command returned exit status: 1
Stdout:
Stderr
Traceback (most recent call last):
  File "/usr/local/bin/filter_alignment.py", line 155, in <module>
    main()
  File "/usr/local/bin/filter_alignment.py", line 108, in main
    raise ValueError("An empty fasta file was provided. "
ValueError: An empty fasta file was provided. Did the alignment complete sucessfully? Did PyNAST discard all sequences due to too-stringent minimum length or minimum percent ID settings?

The same thing happened when I ran multiple_split_libraries_fastq.py to demultiplex and ran the open picking. I used multiple_join_paired_ends.py to join my sequences. 

[Colin Brislawn]

Thanks for sending all that information to me.

Well it looks like this script does not like your fasta files. Let's see if something is strange about them.

Can you run these commands for me and post the results?

cd id_added_fecal

ls -alh

head combined_seqs.fna

Thanks!

Colin

[mpbr...@tamu.edu]

Sorry it took me so long! I ran this on a Mac using macqiime instead of on the Virtual Box I usually use.

MacQIIME dhcp-0693:~ $ cd ~/Desktop/id_added_fecal
MacQIIME dhcp-0693:id_added_fecal $ ls -alh
total 824
drwxrwxrwx   3 mbranco  staff   102B Nov 12 16:27 .
drwx------+ 10 mbranco  staff   340B Nov 20 14:20 ..
-rwxrwxrwx   1 mbranco  staff   411K Nov 12 09:43 combined_seqs.fna
MacQIIME dhcp-0693:id_added_fecal $ head combined_seqs.fna
>C3F_10 M00784:219:000000000-AFJUJ:1:1101:26333:9392
GTAGGTGAACCTGCAGAAGGATCATTAGTGAAAGCAAGAGGCCTGCCATACGGACGCGCTACTCGCGCAGTCTCTGGCGCCTCAACCCTTTACACAAACATCCAAACCCGTGTGCACCTGTAGTCTGTTGGGCTTGAGCAATCGAGCCGGGCAGCTACGCAAATTTAAACCAAACTCGTATGGTTGTATGGACGTGTAAATCGTTGGACCGTAACTGGCCAGCAAACAAACAATACAACTTTCGACAACG
>C3F_11 M00784:219:000000000-AFJUJ:1:1101:26359:12665
AGCAAGCATCTCATTTTGTGCATATACCTGGTCTTTCGTATTCTGGCGTGAAGTCGCCGACTGAATGCCAGCAATCTCTTTTTGAGTCTCATTTTGCATCTCGGCAATCTCTTTCTGATTGTCCAGTTGCATTTTAGTAAGCTCTTTTTGATTCTCAAATCCGGCGTCAACCATACCAGCAGAGGAAGCATCAGCACCAGCACGCTCCCAAGCATTAAGCTCAGGAAATGCAGCAGCAAGATAATCACGAGTATCCTTTCCTTTATCAGCGGCAGACTTGCCACCAAGTCCAACCAAATCAAGCAACTTATCAGAAACGGCAGAAGTGCCAGCCTGCAACGTACCTTCAAGAAGTCCTTTACCAGCTTTAGCCATAGCACCAGAAACAAAACTAGGGACGGCCTCATCAGGGTTAG
>C3F_12 M00784:219:000000000-AFJUJ:1:1101:12024:15171
GTAGGTGAACCTGCAGAAGGATCATTAGTGAAAGCAAGAGGCCTGCCATACGGACGCGCTACTCGCGCAGTCTCTGGCGCCTCAACCCTTTACACAAACATCCAAACCCGTGTGCACCTGTAGTCTGGTGGGCTTGAGCAATCGAGCCGGGCAGCTACGCAAATTTAAACCAAACTCGTGTGGTTGTATGAACGTGTAAATCGTTGGACCGTAACTGGCCAGCAAACAAACAATACAACTTTCGACAACG
>C3F_13 M00784:219:000000000-AFJUJ:1:1102:3013:9885
GCTTTTTCACGTTCTGGTTGGTTGTGGCCTGTTGATGCTAAAGGTGAGCCGCTTAAAGCTACCAGTTATATGGCTGTTGGTTTCTATGTGGCTAAATACGTTAACAAAAAGTCAGATATGGACCTTGCTGCTAAAGGTCTAGGAGCTAAAGAATGGAACAACTCACTAAAAACCAAGCTGTCGCTACTTCCCAAGAAGCTGTTCAGAATCAGAATGAGCCGCAACTTCGGGATGAAAATGCTCACAATGACAAATCTGTCCACGGAGTGCTTAAGCCAACTTACCAAGCTGGGTGACGACGCGACGCCGTTCAACCAGATATTGAAGCAGAACGCAAAAAGAGAGATGAGATTGAGGCTGGGAAAAGTTACTGTAGCCGACGTTTTGGCGGCGCAACCTGTGACGACAAATCTGCTCAAAT
>C3F_14 M00784:219:000000000-AFJUJ:1:1103:21784:8845
GTAGGTGAACCTGCAGAAGGATCATTAGTGAAAGCAAGAGGCCTGCCATACGGACGCGCTACTCGCGCAGTCTCTGGCGCCTCAACCCTTTACACAAACATCCAAACCCGTGTGCACCTGTAGTCTGTTGGGCTTGAGCAATCGAGCCGGGCAGCTACGCAAATTTAAACCAAACTCGTATGGTTGTATGGACGTGTAAATCGTTGGACCGTAACTGGCCAGCAAACAAACAATACAACTTTCGACAACG
MacQIIME dhcp-0693:id_added_fecal $ 

[Colin Brislawn]

Hi there,

Oh your fasta file looks ok.

Take a look through your output folder called open_picking_fecal. Based on that error, I'm guessing that script got most of the way through, then failed. Also check the pynast_aligned_seqs folder, because that was the last command to finish before the scrip threw this error.

Colin