Problems “chimera checking sequences” pipeline using usearch61 algorithm

[Matías Di Paola]

Due to lack of memory on my personal computer I have run the latest AMI version (ami-19118ff72) of qiime in the Amazon EC2.

I want to perform the “chimera checking sequences” with usearch61 algorithm.

After installing the correct version of usearch61 in the server, copying the seqs.fna coming from the split_library_output script and downloading the reference database gg_13_5.fasta I ran:

identify_chimeric_seqs.py -i seqs.fna -m usearch61 -o usearch_checked_chimeras/ -r gg_13_5.fasta

For my surprise there was only two files:

seqs.fna_consensus_with_abundance.uc

seqs.fna_smallmem_clustered.log

No “.txt” extension file, so I continue with the pipeline using the “.uc”:

filter_fasta.py -f seqs.fna -o seqs_chimeras_filtered.fna -s usearch_checked_chimeras/ seqs.fna_consensus_with_abundance.uc -n

the output file was:

seqs_chimeras_filtered.fna

when I try to run the last step of the pipeline:

pick_otus.py -m usearch61 -i seqs_chimeras_filtered.fna -o usearch61_picked_otus/

I get the following error:

cogent.app.util.ApplicationError: Error running usearch61. Possible causes are unsupported version (current supported version is usearch v6.1.544) is installed or improperly formatted input file was provided

why there is not a “.txt” file in the identify_chimeric_seqs.py output? could I use.uc file to continue with the pipeline o I have to switch to uclust algorithm to pick the OTUs? I am not sure what the .uc kind of file is, I find searching in the tutorial that is an intermediate uclust(.uc) files.

I attached 20 lines of  the input files and the log file.

Thanks in advanced for the help

Matias

[Daniel McDonald]

Hi Matías,

Can you please send the output from the following commands?

print_qiime_config.py -t

which usearch

I'm a bit confused as well about the lack of the .txt file. 

The .uc file is the standard output file from usearch. However, the identify_chimeric_seqs.py script should be outputting a .txt file as I understand it. 

Best,

Daniel

[Matías Di Paola]

Hi Daniel,

I already finish my session in AMazon EC2, but I used the last AMI published.

ubuntu@ip-172-31-1-162:~$ print_qiime_config.py -t

System information

==================

         Platform: linux2

   Python version: 2.7.3 (default, Aug  1 2012, 05:14:39)  [GCC 4.6.3]

Python executable: /usr/bin/python

QIIME default reference information

===================================

For details on what files are used as QIIME's default references, see here:

 https://github.com/biocore/qiime-default-reference/releases/tag/0.1.2

Dependency versions

===================

          QIIME library version: 1.9.1

           QIIME script version: 1.9.1

qiime-default-reference version: 0.1.2

                  NumPy version: 1.9.2

                  SciPy version: 0.15.1

                 pandas version: 0.16.1

             matplotlib version: 1.4.3

            biom-format version: 2.1.4

                   h5py version: 2.5.0 (HDF5 version: 1.8.4)

                   qcli version: 0.1.1

                   pyqi version: 0.3.2

             scikit-bio version: 0.2.3

                 PyNAST version: 1.2.2

                Emperor version: 0.9.51

                burrito version: 0.9.1

       burrito-fillings version: 0.1.1

              sortmerna version: SortMeRNA version 2.0, 29/11/2014

              sumaclust version: SUMACLUST Version 1.0.00

                  swarm version: Swarm 1.2.19 [May 26 2015 15:28:37]

                          gdata: Installed.

QIIME config values

===================

For definitions of these settings and to learn how to configure QIIME, see here:

 http://qiime.org/install/qiime_config.html

 http://qiime.org/tutorials/parallel_qiime.html

                     blastmat_dir: /qiime_software/blast-2.2.22-release/data

      pick_otus_reference_seqs_fp: /usr/local/lib/python2.7/dist-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta

                         sc_queue: all.q

      topiaryexplorer_project_dir: None

     pynast_template_alignment_fp: /usr/local/lib/python2.7/dist-packages/qiime_default_reference/gg_13_8_otus/rep_set_aligned/85_otus.pynast.fasta

                  cluster_jobs_fp: start_parallel_jobs.py

pynast_template_alignment_blastdb: None

assign_taxonomy_reference_seqs_fp: /usr/local/lib/python2.7/dist-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta

                     torque_queue: friendlyq

                    jobs_to_start: 1

                       slurm_time: None

            denoiser_min_per_core: 50

assign_taxonomy_id_to_taxonomy_fp: /usr/local/lib/python2.7/dist-packages/qiime_default_reference/gg_13_8_otus/taxonomy/97_otu_taxonomy.txt

                         temp_dir: /home/ubuntu/temp/

                     slurm_memory: None

                      slurm_queue: None

                      blastall_fp: /qiime_software/blast-2.2.22-release/bin/blastall

                 seconds_to_sleep: 1

QIIME base install test results

===============================

.........

----------------------------------------------------------------------

Ran 9 tests in 0.580s

OK

ubuntu@ip-172-31-1-162:~$ which usearch

/// No results

I download and copied usearch6.1.544_i86linux32 to de amazon server.

and then:

sudo mv usearch* /usr/local/bin/usearch61

sudo chmod a+x /usr/local/bin/usearch61

ubuntu@ip-172-31-1-162:~$ usearch61 

(C) Copyright 2010-12 Robert C. Edgar, all rights reserved.

http://drive5.com/usearch

Licensed to: matias...@qb.fcen.uba.ar

For documentation, please visit:

  http://drive5.com/usearch

I hope this helps

Cheers

[Daniel McDonald]

Okay, thanks. 

So I'm not sure why identify_chimeric_sequences.py isn't dumping out the expected .txt file. I suspect the reason that pick_otus.py is dying is because filter_fasta.py (I don't think...) knows how handle a .uc file properly. 

Inquiring more internally on this. Will follow up when I know more.

Best,

Daniel

[Matías Di Paola]

Thanks Daniel,

By the way I hadn 't denoise the raw data (coming from 454) I just mention it because I am new in this kind of analysis an I dont know if it has something to do with this error.

Matias 

[Daniel McDonald]

That shouldn't be an issue

Best,

Daniel

[Daniel McDonald]

Hi Matías,

I asked around -- our best guess is that identify_chimeric_sequences.py did not complete properly and terminated prematurely. Possible to rerun and verify?

Best,

Daniel

[Matías Di Paola]

Hi Daniel,

I will try again in a local server. I let you know.

Regards,
Matias