Groups keyboard shortcuts have been updated
Dismiss
See shortcuts

No module named burrito.util - Macqiime

165 views
Skip to first unread message

Manon Duret

unread,
Oct 17, 2016, 9:26:39 AM10/17/16
to qiime...@googlegroups.com
Hello,

I am trying to use pick_open_reference_otus.py however I keep getting the same error message related to assign_taxonomy.py.
I therefore tried to run the QIIME test file:

python test_assign_taxonomy.py

And this is the error message I am getting:

qiime_test mtd1e14$ python test_assign_taxonomy.py

Traceback (most recent call last):

  File "test_assign_taxonomy.py", line 25, in <module>

    from burrito.util import ApplicationError

ImportError: No module named burrito.util


Is there a problem with the installation of Python my Mac?

Many thanks for your help,

Manon

Colin Brislawn

unread,
Oct 17, 2016, 1:27:27 PM10/17/16
to Qiime 1 Forum
Hello Manon,

Thanks for getting in touch with us! 

burrito.util is one of the python packages qiime uses to call other pieces of software. How did you go about installing qiime? The current miniconda based method should install burrito well on both Linux and OSX. 

Colin

Manon Duret

unread,
Oct 18, 2016, 5:29:45 AM10/18/16
to Qiime 1 Forum
Hi Colin,

Many thanks for your answer!

I installed, and successfully run QIIME, over a year ago. I tried to run it again recently without success.

I just installed miniconda and created a virtual environment in which I am running assign_taxonomy.py test again. I actually have a list of errors - I think I will try to uninstall and re-install MacQIIME completely and keep you updated with my progress.

Thanks!!

Manon

Manon Duret

unread,
Oct 18, 2016, 8:55:39 AM10/18/16
to Qiime 1 Forum
Hi Colin,

I tried to re-install MacQIIME, but now this is the message I get when trying to run test_assign_taxonomy.py:

-bash: mtd1e14$: command not found


Both MacQIIME and qiime1 environment give me the same results.

Could it be linked with my version of Python?

Thank you!

Manon

Colin Brislawn

unread,
Oct 18, 2016, 10:42:14 AM10/18/16
to Qiime 1 Forum
Hello Manon,

Thanks for trying this out for us. 

The qiime install tutorial has you run conda create -n qiime1 python=2.7; which will give you Python 2.7, the right version for qiime. 

Could you run these commands for me and post the results:
conda info -e
source activate qiime1
print_qiime_config.py -t

Thanks!
Colin

Manon Duret

unread,
Oct 18, 2016, 10:58:50 AM10/18/16
to Qiime 1 Forum

Hi Colin,


Here are the results:


$ conda info -e

# conda environments:

#

qiime1                   /Users/mtd1e14/miniconda2/envs/qiime1

root                  *  /Users/mtd1e14/miniconda2





And:


$ print_qiime_config.py -t


System information

==================

         Platform: darwin

   Python version: 2.7.12 |Continuum Analytics, Inc.| (default, Jul  2 2016, 17:43:17)  [GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)]

Python executable: /Users/mtd1e14/miniconda2/envs/qiime1/bin/python


QIIME default reference information

===================================

For details on what files are used as QIIME's default references, see here:

 https://github.com/biocore/qiime-default-reference/releases/tag/0.1.3


Dependency versions

===================

          QIIME library version: 1.9.1

           QIIME script version: 1.9.1

qiime-default-reference version: 0.1.3

                  NumPy version: 1.10.4

                  SciPy version: 0.17.1

                 pandas version: 0.18.1

             matplotlib version: 1.4.3

            biom-format version: 2.1.5

                   h5py version: 2.6.0 (HDF5 version: 1.8.16)

                   qcli version: 0.1.1

                   pyqi version: 0.3.2

             scikit-bio version: 0.2.3

                 PyNAST version: 1.2.2

                Emperor version: 0.9.51

                burrito version: 0.9.1

       burrito-fillings version: 0.1.1

              sortmerna version: SortMeRNA version 2.0, 29/11/2014

              sumaclust version: Not installed.

                  swarm version: Swarm 1.2.19 [Mar  5 2016 17:12:19]

                          gdata: Installed.


QIIME config values

===================

For definitions of these settings and to learn how to configure QIIME, see here:

 http://qiime.org/install/qiime_config.html

 http://qiime.org/tutorials/parallel_qiime.html


                     blastmat_dir: None

      pick_otus_reference_seqs_fp: /Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta

                         sc_queue: all.q

      topiaryexplorer_project_dir: None

     pynast_template_alignment_fp: /Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set_aligned/85_otus.pynast.fasta

                  cluster_jobs_fp: start_parallel_jobs.py

pynast_template_alignment_blastdb: None

assign_taxonomy_reference_seqs_fp: /Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta

                     torque_queue: friendlyq

                    jobs_to_start: 1

                       slurm_time: None

            denoiser_min_per_core: 50

assign_taxonomy_id_to_taxonomy_fp: /Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/taxonomy/97_otu_taxonomy.txt

                         temp_dir: /var/folders/7n/6nf2h4kn33135gkg3hhns__w0000gp/T/

                     slurm_memory: None

                      slurm_queue: None

                      blastall_fp: blastall

                 seconds_to_sleep: 1


QIIME base install test results

===============================

.........

----------------------------------------------------------------------

Ran 9 tests in 0.037s


OK



Thank you!


Manon

Colin Brislawn

unread,
Oct 18, 2016, 11:05:09 AM10/18/16
to Qiime 1 Forum
Oh hello Manon,

Thanks for the prompt reply. Your qiime install looks good to me. Using the -t flag along with print_qiime_config.py now the standard way to test your qiime scripts, so may that's why test_assign_taxonomy.py is not working. 

What qiime scripts are you hoping to run on your data? Perhaps we can test for errors on your actual data set (or maybe it will just work well!)

Let us know what you find,
Colin

Manon Duret

unread,
Oct 18, 2016, 11:13:16 AM10/18/16
to Qiime 1 Forum
Hi Colin,

And thank you for yours!

I just tried running the test files for assign taxonomy, and here is the result I got:

$ test_assign_taxonomy.py

-bash: test_assign_taxonomy.py: command not found


I tried to run it in the correct folder ("qiime_tests")... shouldn't it work?

I would like to run pick_open_reference_otus.py using the PR2 database. Here is the script I would like to run:

mtd1e14$ pick_open_reference_otus.py -i seqs.fna -r pr2_gb203_version_4.3.fasta -o results/ -p params_p.txt --suppress_align_and_tree


Using the attached parameter file.


Thank you!!


Manon

params_p.txt

Colin Brislawn

unread,
Oct 18, 2016, 12:03:03 PM10/18/16
to Qiime 1 Forum
Hello Manon,

Like I said, I'm familear with qiime tests using 'print_qiime_config.py -t'. That -t flag will run tests.

Thanks for posting your pick OTUs script. Have you tried running it? What happens?
Colin

Manon Duret

unread,
Oct 19, 2016, 5:05:19 AM10/19/16
to qiime...@googlegroups.com
Hi Colin,

I tried to run the line in the qiime1 environment and this is the message error I obtained:

Traceback (most recent call last):

  File "/Users/mtd1e14/miniconda2/envs/qiime1/bin/pick_open_reference_otus.py", line 4, in <module>

    __import__('pkg_resources').run_script('qiime==1.9.1', 'pick_open_reference_otus.py')

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/__init__.py", line 744, in run_script

    

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/__init__.py", line 1499, in run_script

    

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/pick_open_reference_otus.py", line 453, in <module>

    main()

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/pick_open_reference_otus.py", line 432, in main

    minimum_failure_threshold=minimum_failure_threshold)

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime/workflow/pick_open_reference_otus.py", line 1030, in pick_subsampled_open_reference_otus

    status_update_callback=status_update_callback)

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime/workflow/pick_open_reference_otus.py", line 232, in assign_tax

    close_logger_on_success=close_logger_on_success)

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime/workflow/util.py", line 122, in call_commands_serially

    raise WorkflowError(msg)

qiime.workflow.util.WorkflowError: 


*** ERROR RAISED DURING STEP: Assign taxonomy

Command run was:

 assign_taxonomy.py -o results//blast_assigned_taxonomy -i results//rep_set.fna --reference_seqs_fp pr2_gb203_version_4.3.fasta --id_to_taxonomy_fp pr2_gb203_version_4.3.taxo --assignment_method blast

Command returned exit status: 1

Stdout:


Stderr

Traceback (most recent call last):

  File "/Users/mtd1e14/miniconda2/envs/qiime1/bin/assign_taxonomy.py", line 4, in <module>

    __import__('pkg_resources').run_script('qiime==1.9.1', 'assign_taxonomy.py')

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/__init__.py", line 744, in run_script

    

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/__init__.py", line 1499, in run_script

    

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/assign_taxonomy.py", line 417, in <module>

    main()

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/assign_taxonomy.py", line 394, in main

    log_path=log_path)

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime/assign_taxonomy.py", line 500, in __call__

    abspath(reference_seqs_path), output_dir=blast_db_dir)

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/burrito_fillings-0.1.1-py2.7.egg/bfillings/formatdb.py", line 116, in build_blast_db_from_fasta_path

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/burrito-0.9.1-py2.7.egg/burrito/util.py", line 201, in __init__

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/burrito-0.9.1-py2.7.egg/burrito/util.py", line 468, in _error_on_missing_application

burrito.util.ApplicationNotFoundError: Cannot find formatdb. Is it installed? Is it in your path?



And this is the error I get when running it under MacQIIME:

bash: mtd1e14$: command not found


Thank you for your help!

Manon

Colin Brislawn

unread,
Oct 19, 2016, 11:37:00 AM10/19/16
to Qiime 1 Forum
Hello Manon,

Ah, OK. If you want to use blast for taxonomy assignment, you need to install it. Because you are using a miniconda environment, this is pretty easy! Inside your qiime1 environment, run:
conda install blast-legacy
Then run the script again.

Let me know how that works for you,
Colin

Jamie Morton

unread,
Oct 19, 2016, 12:24:33 PM10/19/16
to Qiime 1 Forum
Yeah, that does sound like a installation error.  What does

print_qiime_config.py -t

return?  That would confirm if burrito is installed correctly.

Manon Duret

unread,
Oct 20, 2016, 4:42:15 AM10/20/16
to Qiime 1 Forum
@ Colin:

Hi Colin,

I tried to run the code again using with uclust, and here are the errors I obtained:

$ pick_open_reference_otus.py -i seqs.fna -r pr2_gb203_version_4.3.fasta -o results/ -p params_p.txt --suppress_align_and_tree

Traceback (most recent call last):

  File "/Users/mtd1e14/miniconda2/envs/qiime1/bin/pick_open_reference_otus.py", line 4, in <module>

    __import__('pkg_resources').run_script('qiime==1.9.1', 'pick_open_reference_otus.py')

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/__init__.py", line 744, in run_script

    

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/__init__.py", line 1499, in run_script

    

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/pick_open_reference_otus.py", line 453, in <module>

    main()

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/pick_open_reference_otus.py", line 432, in main

    minimum_failure_threshold=minimum_failure_threshold)

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime/workflow/pick_open_reference_otus.py", line 1030, in pick_subsampled_open_reference_otus

    status_update_callback=status_update_callback)

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime/workflow/pick_open_reference_otus.py", line 232, in assign_tax

    close_logger_on_success=close_logger_on_success)

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime/workflow/util.py", line 122, in call_commands_serially

    raise WorkflowError(msg)

qiime.workflow.util.WorkflowError: 


*** ERROR RAISED DURING STEP: Assign taxonomy

Command run was:

 assign_taxonomy.py -o results//uclust_assigned_taxonomy -i results//rep_set.fna --reference_seqs_fp pr2_gb203_version_4.3.fasta --id_to_taxonomy_fp pr2_gb203_version_4.3.taxo --assignment_method uclust

Command returned exit status: 1

Stdout:


Stderr

Traceback (most recent call last):

  File "/Users/mtd1e14/miniconda2/envs/qiime1/bin/assign_taxonomy.py", line 4, in <module>

    __import__('pkg_resources').run_script('qiime==1.9.1', 'assign_taxonomy.py')

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/__init__.py", line 744, in run_script

    

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/__init__.py", line 1499, in run_script

    

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/assign_taxonomy.py", line 417, in <module>

    main()

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/assign_taxonomy.py", line 386, in main

    taxon_assigner = taxon_assigner_constructor(params)

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime/assign_taxonomy.py", line 1234, in __init__

    self.id_to_taxonomy = self._parse_id_to_taxonomy_file(id_to_taxonomy_f)

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime/assign_taxonomy.py", line 117, in _parse_id_to_taxonomy_file

    identifier, taxonomy = map(strip, line.split('\t'))

ValueError: too many values to unpack





I also ran conda install blast-legacy, and this is what I obtained:

Fetching package metadata .......

Solving package specifications: .

PackageNotFoundError: Package not found: '' Package missing in current osx-64 channels: 

  - blast-legacy


You can search for packages on anaconda.org with


    anaconda search -t conda blast-legacy


You may need to install the anaconda-client command line client with


    conda install anaconda-client



I installed blast 2.5.0 yesterday - but perhaps I have not done it correctly?

Manon Duret

unread,
Oct 20, 2016, 4:45:02 AM10/20/16
to Qiime 1 Forum
@ Jamie:

Hi Jamie,

This is the results for the print_qiime_config.py -t command ran in qiime1 environment:

Ran 9 tests in 0.039s


OK


I assume it looks fine, correct?


Thank you very much for your help!!


Manon

Jamie Morton

unread,
Oct 20, 2016, 11:57:10 AM10/20/16
to Qiime 1 Forum
Ok, I somehow missed the previous conversions when responding to this ...

To sanity check to see if you have blast installed, you can run

which formatdb

And see if that returns a file path.

Also, can you paste the exact command that you ran test_assign_taxonomy.py?
Looks like you may be accidentally copying your username. 

Manon Duret

unread,
Oct 21, 2016, 4:58:23 AM10/21/16
to Qiime 1 Forum
@ Jamie

Hi Jamie,

I tried to run the command which formatdb within the qiime1 environment (qiime.org/install/install.html), within macqiime environment and outside both, but none of them returned a file path. Do I need blast to run pick_open_reference_otus.py using uclust to assign taxonomy?

When running the test assign_taxonomy command, I call the directory qiime_test, and type the command:

test_assign_taxonomy.py


And this is the return:

-bash: test_assign_taxonomy.py: command not found


However, when running the test as Colin suggested with the command, within qiime1 or macqiime environment:

print_qiime_config.py -t


The command works and my return in qiime1 is:

Ran 9 tests in 0.044s


OK






And the return within macqiime is:

System information

==================

         Platform: darwin

   Python version: 2.7.10 |Anaconda 2.2.0 (x86_64)| (default, May 28 2015, 17:04:42)  [GCC 4.2.1 (Apple Inc. build 5577)]

Python executable: /macqiime/anaconda/bin/python


QIIME default reference information

===================================

For details on what files are used as QIIME's default references, see here:


Dependency versions

===================

          QIIME library version: 1.9.1

           QIIME script version: 1.9.1

qiime-default-reference version: 0.1.2

                  NumPy version: 1.9.2

                  SciPy version: 0.15.1

                 pandas version: 0.16.1

             matplotlib version: 1.4.3

            biom-format version: 2.1.4

                   h5py version: 2.4.0 (HDF5 version: 1.8.14)

                   qcli version: 0.1.1

                   pyqi version: 0.3.2

             scikit-bio version: 0.2.3

                 PyNAST version: 1.2.2

                Emperor version: 0.9.51

                burrito version: 0.9.1

       burrito-fillings version: 0.1.1

              sortmerna version: SortMeRNA version 2.0, 29/11/2014

              sumaclust version: SUMACLUST Version 1.0.00

                  swarm version: Swarm 1.2.19 [Jun  2 2015 14:40:16]

                          gdata: Installed.


QIIME config values

===================

For definitions of these settings and to learn how to configure QIIME, see here:

 http://qiime.org/install/qiime_config.html

 http://qiime.org/tutorials/parallel_qiime.html


                     blastmat_dir: None

      pick_otus_reference_seqs_fp: /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta

                         sc_queue: all.q

      topiaryexplorer_project_dir: None

     pynast_template_alignment_fp: /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set_aligned/85_otus.pynast.fasta

                  cluster_jobs_fp: start_parallel_jobs.py

pynast_template_alignment_blastdb: None

assign_taxonomy_reference_seqs_fp: /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta

                     torque_queue: friendlyq

                    jobs_to_start: 1

                       slurm_time: None

            denoiser_min_per_core: 50

assign_taxonomy_id_to_taxonomy_fp: /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/taxonomy/97_otu_taxonomy.txt

                         temp_dir: /tmp/

                     slurm_memory: None

                      slurm_queue: None

                      blastall_fp: blastall

                 seconds_to_sleep: 60


QIIME base install test results

===============================

.........

----------------------------------------------------------------------

Ran 9 tests in 0.036s


OK


I don't understand what is going on! Can you see what is the issue here?


Thank you for your help. I am starting to feel a bit desperate!


All the best,


Manon

Colin Brislawn

unread,
Oct 21, 2016, 12:56:32 PM10/21/16
to qiime...@googlegroups.com
Hello Manon,

Thanks for running 'which formatdb' for us. Because it did not return a path when run inside your qiime environment, we know that it's not installed. To install it, first enter your qiime1 environment then install it using conda:
source activate qiime1
conda install -c bioconda blast-legacy=2.2.22
Let me know what that command returns. 

You mentioned: 

"I installed blast 2.5.0 yesterday - but perhaps I have not done it correctly?"

Qiime does not need BLAST+ 2.5.0. Qiime needs blast-legacy, v2.2.22. This is the version that should be installed by running that conda command. 


Thanks for troubleshooting this with us. It may not feel like it, but we are making progress! 

Colin

Manon Duret

unread,
Oct 24, 2016, 4:56:10 AM10/24/16
to Qiime 1 Forum
@ Colin

Hi Colin!

Thanks for sticking with me!

This is the return I get from conda install -c bioconda blast-legacy=2.2.22:

Fetching package metadata .........

Solving package specifications: .

PackageNotFoundError: Package not found: '' Package missing in current osx-64 channels: 

  - blast-legacy 2.2.22*


You can search for packages on anaconda.org with


    anaconda search -t conda blast-legacy


You may need to install the anaconda-client command line client with


    conda install anaconda-client


As I couldn't download blast-legacy, v2.2.22 (I didn't find a link that was functioning), I used the following commands conda install anaconda-client, followed by anaconda search -t conda blast-legacy which returned:

Using Anaconda API: https://api.anaconda.org

Run 'anaconda show <USER/PACKAGE>' to get more details:

Packages:

     Name                      |  Version | Package Types   | Platforms      

     ------------------------- |   ------ | --------------- | ---------------

     bioconda/blast-legacy     |   2.2.22 | conda           | linux-64       

                                          : The Basic Local Alignment Search Tool (BLAST) finds regions of local similarity between sequences.

     biocore/blast-legacy      |   2.2.22 | conda           | linux-64, osx-64

                                          : BLAST finds regions of similarity between biological sequences.

     zechxu/blast-legacy       |   2.2.22 | conda           | linux-64, osx-64

                                          : BLAST finds regions of similarity between biological sequences.

Found 3 packages


I chose biocore/blast-legacy and installed it with the command conda install --channel https://conda.anaconda.org/biocore blast-legacy.


Now the command which formatdb returns:

/Users/mtd1e14/miniconda2/envs/qiime1/bin/formatdb


Hurray!


I am now running pick_open_reference_otus.py -i seqs.fna -r pr2_gb203_version_4.3.fasta -o results/ -p params_p.txt --suppress_align_and_tree using the same files I previously used (and using blast as the assignment method. I will keep you updated on the progress.


Many thanks!!


Manon



Manon Duret

unread,
Oct 24, 2016, 5:43:32 AM10/24/16
to Qiime 1 Forum
Hi,

This is the return:

Traceback (most recent call last):

  File "/Users/mtd1e14/miniconda2/envs/qiime1/bin/pick_open_reference_otus.py", line 4, in <module>

    __import__('pkg_resources').run_script('qiime==1.9.1', 'pick_open_reference_otus.py')

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/__init__.py", line 744, in run_script

    

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/__init__.py", line 1499, in run_script

    

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/pick_open_reference_otus.py", line 453, in <module>

    main()

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/pick_open_reference_otus.py", line 432, in main

    minimum_failure_threshold=minimum_failure_threshold)

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime/workflow/pick_open_reference_otus.py", line 1030, in pick_subsampled_open_reference_otus

    status_update_callback=status_update_callback)

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime/workflow/pick_open_reference_otus.py", line 232, in assign_tax

    close_logger_on_success=close_logger_on_success)

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime/workflow/util.py", line 122, in call_commands_serially

    raise WorkflowError(msg)

qiime.workflow.util.WorkflowError: 


*** ERROR RAISED DURING STEP: Assign taxonomy

Command run was:

 assign_taxonomy.py -o results//blast_assigned_taxonomy -i results//rep_set.fna --reference_seqs_fp pr2_gb203_version_4.3.fasta --id_to_taxonomy_fp pr2_gb203_version_4.3.taxo --assignment_method blast

Command returned exit status: 1

Stdout:


Stderr

Traceback (most recent call last):

  File "/Users/mtd1e14/miniconda2/envs/qiime1/bin/assign_taxonomy.py", line 4, in <module>

    __import__('pkg_resources').run_script('qiime==1.9.1', 'assign_taxonomy.py')

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/__init__.py", line 744, in run_script

    

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/__init__.py", line 1499, in run_script

    

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/assign_taxonomy.py", line 417, in <module>

    main()

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/assign_taxonomy.py", line 394, in main

    log_path=log_path)

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime/assign_taxonomy.py", line 505, in __call__

    open(self.Params['id_to_taxonomy_filepath'], 'U'))

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime/assign_taxonomy.py", line 117, in _parse_id_to_taxonomy_file

    identifier, taxonomy = map(strip, line.split('\t'))

ValueError: too many values to unpack


I obtained a biom table (otu_table_mc2.biom) which doesn't contain any taxonomy information. I join the .txt file I obtained with the command biom convert -i results/otu_table_mc2.biom -o results/otu_table.txt --to-tsv --header-key taxonomy if you want to have a look.

I also attached the blast assignment log file (rep_set_tax_assignments.log).


We are so close now! Thank you so much!!! 


Manon

otu_table.txt
rep_set_tax_assignments.log

Colin Brislawn

unread,
Oct 24, 2016, 2:17:59 PM10/24/16
to Qiime 1 Forum
Hello Manon,

Oh great! I'm glad you found the -c biocore channel and that blast-legacy is now working for you. I appreciate your troubleshooting all throughout this process. 

I'm not sure why assign taxonomy is now acting funny... I searched for that error on the forums, and it looks like it comes up when folks try to use custom databases and taxonomy files. Perhaps these are formatted strangely?

I'm not very familiar with the qiime taxonomy scripts, but I can forward to a developer who is, if we get stuck. 

Thanks!
Colin

Manon Duret

unread,
Oct 25, 2016, 10:51:32 AM10/25/16
to Qiime 1 Forum
Hi Colin!

Thank you SO MUCH for your help!

Indeed: it seems that the PR2 database (https://figshare.com/articles/PR2_rRNA_gene_database/3803709) need some formatting in order to be used in QIIME - the taxonomy file has 7 levels, instead of the 6 levels accepted for the input file for assign_taxonomy.py...

Cheers!

Manon

Manon Duret

unread,
Nov 3, 2016, 11:53:24 AM11/3/16
to Qiime 1 Forum

Hi Colin,


Sorry for the delay, I was on holiday :)


I have modified my .taxo file so it only has 7 levels. This is now the issue I am having with the command:

(qiime1) 241-192:0_TARA_ocean mtd1e14$ assign_taxonomy.py -i results_uclust/rep_set.fna -t inputs/pr2_mod.taxo -r inputs/pr2.fasta -m blast


/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/matplotlib/__init__.py:1011: UserWarning: Duplicate key in file "/Users/mtd1e14/.matplotlib/matplotlibrc", line #2

  (fname, cnt))

Traceback (most recent call last):

  File "/Users/mtd1e14/miniconda2/envs/qiime1/bin/assign_taxonomy.py", line 4, in <module>

    __import__('pkg_resources').run_script('qiime==1.9.1', 'assign_taxonomy.py')

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/__init__.py", line 744, in run_script

    

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/__init__.py", line 1499, in run_script

    

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/assign_taxonomy.py", line 417, in <module>

    main()

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime-1.9.1-py2.7.egg-info/scripts/assign_taxonomy.py", line 394, in main

    log_path=log_path)

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime/assign_taxonomy.py", line 537, in __call__

    current_seqs, blast_db, id_to_taxonomy_map))

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime/assign_taxonomy.py", line 586, in _seqs_to_taxonomy

    blast_hits = self._get_blast_hits(blast_db, seqs)

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/qiime/assign_taxonomy.py", line 636, in _get_blast_hits

    add_seq_names=False)

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/burrito_fillings-0.1.1-py2.7.egg/bfillings/blast.py", line 669, in blast_seqs

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/burrito_fillings-0.1.1-py2.7.egg/bfillings/blast.py", line 406, in __init__

  File "/Users/mtd1e14/miniconda2/envs/qiime1/lib/python2.7/site-packages/burrito_fillings-0.1.1-py2.7.egg/bfillings/blast.py", line 171, in __init__

RuntimeError: BLAST cannot run if the BLASTMAT environment variable is not set.


Usually, the BLASTMAT environment variable points to the NCBI data directory,

which contains matrices like PAM30 and PAM70, etc.


Alternatively, you may create a .ncbirc file to define these variables.


From help file:


2) Create a .ncbirc file. In order for Standalone BLAST to operate, you

have will need to have a .ncbirc file that contains the following lines:


[NCBI]

Data="path/data/"


Where "path/data/" is the path to the location of the Standalone BLAST

"data" subdirectory. For Example:


Data=/root/blast/data


The data subdirectory should automatically appear in the directory where

the downloaded file was extracted. Please note that in many cases it may

be necessary to delimit the entire path including the machine name and

or the net work you are located on. Your systems administrator can help

you if you do not know the entire path to the data subdirectory.


Make sure that your .ncbirc file is either in the directory that you

call the Standalone BLAST program from or in your root directory.


Which I don't really understand as we previously made sure that blast was installed in the qiime1 environment... Have I done something obviously wrong?


Thank you!!


Manon

Colin Brislawn

unread,
Nov 3, 2016, 12:18:50 PM11/3/16
to Qiime 1 Forum
Hello Manon,

Looks like that's and error with blast. Perhaps blast-legacy is not installed correctly? What do these commands return when you run them?
which blast
echo $BLASTMAT
ls -al ~/.

I've not used blast for taxonomy assignment, so may need to reach out to another developer.

Colin

Manon Duret

unread,
Nov 4, 2016, 10:01:38 AM11/4/16
to Qiime 1 Forum
Hi Colin,

Indeed it looks like blast is not properly installed...
which blast and echo $BLASTMAT returned nothing.

This is the return for ls -al ~/. :

total 60200

drwxr-xr-x@ 57 mtd1e14  staff      1938  3 Nov 15:59 .

drwxr-xr-x   6 root     admin       204 22 Oct  2014 ..

-rw-------   1 mtd1e14  staff         3  6 Dec  2014 .CFUserTextEncoding

-rw-r--r--@  1 mtd1e14  staff     24580  3 Nov 15:46 .DS_Store

drwxr-xr-x   3 mtd1e14  staff       102 29 Oct  2015 .R

-rw-r--r--   1 mtd1e14  staff  30690571 11 May 16:18 .RData

-rw-r--r--   1 mtd1e14  staff         0 11 Mar  2016 .Rapp.history

-rw-r--r--   1 mtd1e14  staff     22132 18 Oct 16:49 .Rhistory

drwx------   2 mtd1e14  staff        68  4 Nov 13:42 .Trash

-rw-------   1 mtd1e14  staff       647 25 Aug 01:55 .Xauthority

drwxr-xr-x   4 mtd1e14  staff       136 27 Oct  2014 .adobe

drwxr-x---   4 mtd1e14  staff       136  2 Dec  2015 .android

drwxr-xr-x   8 mtd1e14  staff       272 27 Apr  2016 .arb_prop

drwxr-xr-x   2 mtd1e14  staff        68 27 Apr  2016 .arb_pts

drwxr-xr-x   4 mtd1e14  staff       136 27 Apr  2016 .arb_tmp

-rw-------   1 mtd1e14  staff     23823  3 Nov 15:53 .bash_history

-rw-r--r--   1 mtd1e14  staff       176 25 Oct 09:57 .bash_profile

-rw-r--r--   1 mtd1e14  staff        86 25 Oct 09:57 .bash_profile-miniconda2.bak

-rw-r--r--   1 mtd1e14  staff        90 18 Oct 10:02 .bash_profile.macports-saved_2016-10-24_at_13:35:50

drwx------   5 mtd1e14  staff       170 19 May 12:40 .cache

drwxr-xr-x   3 mtd1e14  staff       102 18 Oct 10:11 .conda

drwxr-xr-x   5 mtd1e14  staff       170 19 May 14:36 .config

drwxr-xr-x   2 mtd1e14  staff        68 18 Oct 10:02 .continuum

drwx------   3 mtd1e14  staff       102 28 Oct  2014 .cups

drwx------  11 mtd1e14  staff       374 13 Oct 13:59 .dropbox

drwx------   2 mtd1e14  staff        68 19 May 12:41 .gnome2

drwx------   3 mtd1e14  staff       102 27 Oct  2014 .iprint

-rw-------   1 mtd1e14  staff        35  8 Dec  2015 .lesshst

drwxr-xr-x   3 mtd1e14  staff       102 19 May 12:40 .local

drwxr-xr-x   3 mtd1e14  staff       102 21 Dec  2014 .matlab

drwxr-xr-x   5 mtd1e14  staff       170  3 Nov 15:48 .matplotlib

-rw-r--r--   1 mtd1e14  staff      1643 17 May 15:28 .odv4_macx64

drwxr-xr-x   8 mtd1e14  staff       272 13 May 09:29 .oracle_jre_usage

-rw-r--r--   1 mtd1e14  staff      1561 27 Apr  2016 .profile

-rw-r--r--   1 mtd1e14  staff       243 26 Apr  2016 .profile.macports-saved_2016-04-27_at_10:34:02

-rw-r--r--   1 mtd1e14  staff       617 27 Apr  2016 .profile.macports-saved_2016-04-27_at_10:42:17

-rw-r--r--   1 mtd1e14  staff       991 27 Apr  2016 .profile.macports-saved_2016-04-27_at_11:19:33

-rw-------   1 mtd1e14  staff      1024 21 Oct 10:26 .rnd

drwxr-xr-x  16 mtd1e14  staff       544 21 Oct 15:07 .rstudio-desktop

drwx------   3 mtd1e14  staff       102  7 Mar  2016 .ssh

drwxr-xr-x   6 mtd1e14  staff       204  8 Dec  2015 .subversion

drwxr-xr-x   3 mtd1e14  staff       102 13 May 09:29 .swt

lrwxr-xr-x   1 mtd1e14  staff        63 27 Apr  2016 ARB_last_session.tgz -> /Users/mtd1e14/.arb_prop/logs/session.20160427_112255.45386.tgz

drwxr-xr-x   2 mtd1e14  staff        68 22 Oct  2014 Applications

drwxr-xr-x  23 mtd1e14  staff       782 22 Jan  2016 CytoscapeConfiguration

drwx------+ 12 mtd1e14  staff       408  4 Nov 13:45 Desktop

drwxr-xr-x+ 11 mtd1e14  staff       374 20 Sep 15:05 Documents

drwx------+  7 mtd1e14  staff       238  4 Nov 13:42 Downloads

drwx------@  8 mtd1e14  staff       272  4 Nov 09:23 Dropbox

drwx------@ 59 mtd1e14  staff      2006 30 Jun 10:47 Library

drwx------+  4 mtd1e14  staff       136  6 Feb  2015 Movies

drwx------+  5 mtd1e14  staff       170 23 Apr  2016 Music

drwx------+  6 mtd1e14  staff       204 26 Sep 17:10 Pictures

drwxr-xr-x+  4 mtd1e14  staff       136 18 Nov  2015 Public

drwxr-xr-x   5 mtd1e14  staff       170  6 Mar  2016 igv

drwxr-xr-x  12 mtd1e14  staff       408 25 Oct 11:14 miniconda2

drwxr-xr-x  94 mtd1e14  staff      3196 18 Oct 13:50 qiime_tests


I will try to install blast again and try with uclust, if none works I will start a new chat for blast issues.


Thank you so much for your precious help!


Manon

Colin Brislawn

unread,
Nov 4, 2016, 10:59:57 AM11/4/16
to Qiime 1 Forum
Hello Manon,

Thanks for posting that for me. Sorry that blast is being strange. The UCLUST taxonomy assigner works well, and should 'just work' as part of the default qiime package.

Let us know if you have more questions,
Colin

Reply all
Reply to author
Forward
0 new messages