Error Problem assigning taxonomy for Fungal ITS using blast

Jay T

unread,

Apr 25, 2017, 10:20:14 PM4/25/17

to Qiime 1 Forum

System information

==================

Platform: darwin

Python version: 2.7.10 |Anaconda 2.2.0 (x86_64)| (default, May 28 2015, 17:04:42) [GCC 4.2.1 (Apple Inc. build 5577)]

Python executable: /macqiime/anaconda/bin/python

QIIME default reference information

===================================

For details on what files are used as QIIME's default references, see here:

https://github.com/biocore/qiime-default-reference/releases/tag/0.1.2

Dependency versions

===================

QIIME library version: 1.9.1

QIIME script version: 1.9.1

qiime-default-reference version: 0.1.2

NumPy version: 1.11.1

SciPy version: 0.15.1

pandas version: 0.16.1

matplotlib version: 1.4.3

biom-format version: 2.1.4

h5py version: 2.6.0 (HDF5 version: 1.8.13)

qcli version: 0.1.1

pyqi version: 0.3.2

scikit-bio version: 0.2.3

PyNAST version: 1.2.2

Emperor version: 0.9.51

burrito version: 0.9.1

burrito-fillings version: 0.1.1

sortmerna version: SortMeRNA version 2.0, 29/11/2014

sumaclust version: SUMACLUST Version 1.0.00

swarm version: Swarm 1.2.19 [Jun 2 2015 14:40:16]

gdata: Installed.

QIIME config values

===================

For definitions of these settings and to learn how to configure QIIME, see here:

http://qiime.org/install/qiime_config.html

http://qiime.org/tutorials/parallel_qiime.html

blastmat_dir: None

pick_otus_reference_seqs_fp: /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta

sc_queue: all.q

topiaryexplorer_project_dir: None

pynast_template_alignment_fp: /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set_aligned/85_otus.pynast.fasta

cluster_jobs_fp: start_parallel_jobs.py

pynast_template_alignment_blastdb: None

assign_taxonomy_reference_seqs_fp: /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta

torque_queue: friendlyq

jobs_to_start: 1

slurm_time: None

denoiser_min_per_core: 50

assign_taxonomy_id_to_taxonomy_fp: /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/taxonomy/97_otu_taxonomy.txt

temp_dir: /tmp/

slurm_memory: None

slurm_queue: None

blastall_fp: blastall

seconds_to_sleep: 60

MacQIIME Todds-Mac-Pro:Bela_ITS2 $

*** ERROR RAISED DURING STEP: Assign taxonomy

Command run was:

parallel_assign_taxonomy_blast.py -i OTUs/rep_set.fna -o OTUs/blast_assigned_taxonomy -T --jobs_to_start 8 --reference_seqs_fp sh_refs_qiime_ver7_dynamic_s_01.08.2015.fasta --id_to_taxonomy_fp sh_taxonomy_qiime_ver7_dynamic_s_01.08.2015.txt

Command returned exit status: 1

Stdout:

Stderr

Traceback (most recent call last):

File "/macqiime/anaconda/bin/parallel_assign_taxonomy_blast.py", line 117, in <module>

main()

File "/macqiime/anaconda/bin/parallel_assign_taxonomy_blast.py", line 113, in main

suppress_submit_jobs=opts.suppress_submit_jobs)

File "/macqiime/anaconda/lib/python2.7/site-packages/qiime/parallel/util.py", line 133, in __call__

self._precommand_initiation(input_fp, output_dir, working_dir, params)

File "/macqiime/anaconda/lib/python2.7/site-packages/qiime/parallel/assign_taxonomy.py", line 141, in _precommand_initiation

params['reference_seqs_fp'], output_dir=working_dir)

File "/macqiime/anaconda/lib/python2.7/site-packages/bfillings/formatdb.py", line 121, in build_blast_db_from_fasta_path

app_result = fdb(fasta_path)

File "/macqiime/anaconda/lib/python2.7/site-packages/burrito/util.py", line 285, in __call__

'StdErr:\n%s\n' % open(errfile).read())

burrito.util.ApplicationError: Unacceptable application exit status: 3

Command:

cd "/Users/twpierson/Downloads/Bela_ITS2/OTUs/blast_assigned_taxonomy/BTA_oQYuz_/"; formatdb -l "sh_refs_qiime_ver7_dynamic_s_01.08.2015.fasta.log" -o T -n "sh_refs_qiime_ver7_dynamic_s_01.08.2015.fasta" -i "sh_refs_qiime_ver7_dynamic_s_01.08.2015.fasta" -p F > "/dev/null" 2> "/dev/null"

StdOut:

StdErr:

MacQIIME Todds-Mac-Pro:Bela_ITS2 $

Jai Ram Rideout

unread,

Apr 26, 2017, 6:39:19 PM4/26/17

to Qiime 1 Forum

Hello,

Do you have the blastall and formatdb commands installed? MacQIIME doesn't include those; here are instructions for installing BLAST in OS X. The link on that page to download BLAST is dead; try this one instead.

After installing BLAST, run print_qiime_config.py -tf to confirm that the installation was successful. If that all looks good, try rerunning your parallel_assign_taxonomy_blast.py command.

Best,

Jai

Jay T

unread,

Apr 27, 2017, 5:12:01 PM4/27/17

to Qiime 1 Forum

I can confirm blastall is installed.

Interesting in pick_open_reference_otus.py, the workflow seems to break when it gets to this step but when I run it separately....it seems to work okay.

*** ERROR RAISED DURING STEP: Assign taxonomy

Command run was:

parallel_assign_taxonomy_blast.py -i Vsearch_picked_ITS_OTUs/rep_set.fna -o Vsearch_picked_ITS_OTUs/blast_assigned_taxonomy -T --jobs_to_start 6 --reference_seqs_fp sh_refs_97_dev.fasta --id_to_taxonomy_fp sh_taxonomy_97_dev.txt

Command returned exit status: 1

Stdout:

Stderr

Traceback (most recent call last):

File "/macqiime/anaconda/bin/parallel_assign_taxonomy_blast.py", line 117, in <module>

main()

File "/macqiime/anaconda/bin/parallel_assign_taxonomy_blast.py", line 113, in main

suppress_submit_jobs=opts.suppress_submit_jobs)

File "/macqiime/anaconda/lib/python2.7/site-packages/qiime/parallel/util.py", line 133, in __call__

self._precommand_initiation(input_fp, output_dir, working_dir, params)

File "/macqiime/anaconda/lib/python2.7/site-packages/qiime/parallel/assign_taxonomy.py", line 141, in _precommand_initiation

params['reference_seqs_fp'], output_dir=working_dir)

File "/macqiime/anaconda/lib/python2.7/site-packages/bfillings/formatdb.py", line 121, in build_blast_db_from_fasta_path

app_result = fdb(fasta_path)

File "/macqiime/anaconda/lib/python2.7/site-packages/burrito/util.py", line 285, in __call__

'StdErr:\n%s\n' % open(errfile).read())

burrito.util.ApplicationError: Unacceptable application exit status: 3

Command:

cd "/Users/twpierson/Downloads/Bela_ITS2/Vsearch_picked_ITS_OTUs/blast_assigned_taxonomy/BTA_mDEN5_/"; formatdb -l "sh_refs_97_dev.fasta.log" -o T -n "sh_refs_97_dev.fasta" -i "sh_refs_97_dev.fasta" -p F > "/dev/null" 2> "/dev/null"

StdOut:

StdErr:

I ran this outside of pick open reference and it ran fine:

parallel_assign_taxonomy_blast.py -i Vsearch_picked_ITS_OTUs/new_refseqs.fna -o blast_assigned_taxonomy/

Jay T

unread,

Apr 27, 2017, 6:42:20 PM4/27/17

to Qiime 1 Forum

# qiime_config

# WARNING: DO NOT EDIT OR DELETE Qiime/qiime_config

# To overwrite defaults, copy this file to $HOME/.qiime_config or a full path

# specified by $QIIME_CONFIG_FP and edit that copy of the file.

# This version was made by Jeff on 20150224 and stored at /macqiime/configs/qiime_config.txt

# cluster_jobs_fp /macqiime/bin/start_parallel_jobs.py

blastmat_dir

blastall_fp blastall

jobs_to_start 1

seconds_to_sleep 60

denoiser_min_per_core 50

topiaryexplorer_project_dir

torque_queue friendlyq

slurm_queue

slurm_memory

temp_dir /tmp/

Jai Ram Rideout

unread,

Apr 28, 2017, 8:40:49 PM4/28/17

to Qiime 1 Forum

Hi Jay,

Do you have the formatdb command installed as well?

The command you're running outside of pick_open_reference_otus.py isn't the same command being run within that script (you'll notice several flags are missing). Can you try running the exact command that's failing?

Please also provide the output from running print_qiime_config.py -tf

Best,

Jai

Jay T

unread,

Apr 28, 2017, 10:06:07 PM4/28/17

to Qiime 1 Forum

MacQIIME Todds-Mac-Pro:Bela_ITS2 $ print_qiime_config.py -tf

RDP Classifier version (if installed): rdp_classifier-2.2.jar

Java version (if installed): 1.8.0_121

QIIME config values

===================

For definitions of these settings and to learn how to configure QIIME, see here:

http://qiime.org/install/qiime_config.html

http://qiime.org/tutorials/parallel_qiime.html

blastmat_dir: None

pick_otus_reference_seqs_fp: /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta

sc_queue: all.q

topiaryexplorer_project_dir: None

pynast_template_alignment_fp: /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set_aligned/85_otus.pynast.fasta

cluster_jobs_fp: start_parallel_jobs.py

pynast_template_alignment_blastdb: None

assign_taxonomy_reference_seqs_fp: /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta

torque_queue: friendlyq

jobs_to_start: 1

slurm_time: None

denoiser_min_per_core: 50

assign_taxonomy_id_to_taxonomy_fp: /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/taxonomy/97_otu_taxonomy.txt

temp_dir: /tmp/

slurm_memory: None

slurm_queue: None

blastall_fp: blastall

seconds_to_sleep: 60

QIIME full install test results

===============================

...F.......................

======================================================================

FAIL: test_ampliconnoise_install (__main__.QIIMEDependencyFull)

AmpliconNoise install looks sane.

----------------------------------------------------------------------

Traceback (most recent call last):

File "/macqiime/anaconda/bin/print_qiime_config.py", line 382, in test_ampliconnoise_install

"$PYRO_LOOKUP_FILE variable is not set. See %s for help." % url)

AssertionError: $PYRO_LOOKUP_FILE variable is not set. See http://qiime.org/install/install.html#ampliconnoise-install-notes for help.

----------------------------------------------------------------------

Ran 27 tests in 1.569s

FAILED (failures=1)

Jay T

unread,

Apr 28, 2017, 10:10:24 PM4/28/17

to Qiime 1 Forum

Formatdb seems to be installed

MacQIIME Todds-Mac-Pro:Bela_ITS2 $ formatdb --help

formatdb 2.2.22 arguments:

-t Title for database file [String] Optional

-i Input file(s) for formatting [File In] Optional

-l Logfile name: [File Out] Optional

default = formatdb.log

-p Type of file

T - protein

F - nucleotide [T/F] Optional

default = T

-o Parse options

T - True: Parse SeqId and create indexes.

F - False: Do not parse SeqId. Do not create indexes.

[T/F] Optional

default = F

-a Input file is database in ASN.1 format (otherwise FASTA is expected)

T - True,

F - False.

[T/F] Optional

default = F

-b ASN.1 database in binary mode

T - binary,

F - text mode.

[T/F] Optional

default = F

-e Input is a Seq-entry [T/F] Optional

default = F

-n Base name for BLAST files [String] Optional

-v Database volume size in millions of letters [Integer] Optional

default = 4000

-s Create indexes limited only to accessions - sparse [T/F] Optional

default = F

-V Verbose: check for non-unique string ids in the database [T/F] Optional

default = F

-L Create an alias file with this name

use the gifile arg (below) if set to calculate db size

use the BLAST db specified with -i (above) [File Out] Optional

-F Gifile (file containing list of gi's) [File In] Optional

-B Binary Gifile produced from the Gifile specified above [File Out] Optional

-T Taxid file to set the taxonomy ids in ASN.1 deflines [File In] Optional

MacQIIME Todds-Mac-Pro:Bela_ITS2 $

Jay T

unread,

Apr 28, 2017, 10:16:23 PM4/28/17

to Qiime 1 Forum

And finally here is where I ran the command that initially gave the error (I switched to a newer UNITE database).

MacQIIME Todds-Mac-Pro:Bela_ITS2 $ parallel_assign_taxonomy_blast.py -i Vsearch_picked_ITS_OTUs/rep_set.fna -o Vsearch_picked_ITS_OTUs/blast_assigned_taxonomy -T --jobs_to_start 8 --reference_seqs_fp refs_31.01.2016_dev.fasta --id_to_taxonomy_fp taxonomy_31.01.2016_dev.txt

Traceback (most recent call last):

File "/macqiime/anaconda/bin/parallel_assign_taxonomy_blast.py", line 117, in <module>

main()

File "/macqiime/anaconda/bin/parallel_assign_taxonomy_blast.py", line 113, in main

suppress_submit_jobs=opts.suppress_submit_jobs)

File "/macqiime/anaconda/lib/python2.7/site-packages/qiime/parallel/util.py", line 133, in __call__

self._precommand_initiation(input_fp, output_dir, working_dir, params)

File "/macqiime/anaconda/lib/python2.7/site-packages/qiime/parallel/assign_taxonomy.py", line 141, in _precommand_initiation

params['reference_seqs_fp'], output_dir=working_dir)

File "/macqiime/anaconda/lib/python2.7/site-packages/bfillings/formatdb.py", line 121, in build_blast_db_from_fasta_path

app_result = fdb(fasta_path)

File "/macqiime/anaconda/lib/python2.7/site-packages/burrito/util.py", line 285, in __call__

'StdErr:\n%s\n' % open(errfile).read())

burrito.util.ApplicationError: Unacceptable application exit status: 3

Command:

cd "/Users/twpierson/Downloads/Bela_ITS2/Vsearch_picked_ITS_OTUs/blast_assigned_taxonomy/BTA_RwGSw_/"; formatdb -l "refs_31.01.2016_dev.fasta.log" -o T -n "refs_31.01.2016_dev.fasta" -i "refs_31.01.2016_dev.fasta" -p F > "/dev/null" 2> "/dev/null"

StdOut:

StdErr:

Jay T

unread,

Apr 29, 2017, 1:19:50 AM4/29/17

to Qiime 1 Forum

Jai here is how I solved the issue and hopefully this helps other. I figured there was something wrong with the blast installation. So I attempted to run the open reference workflow without it running in parallel. I got an error stating I did not have a .ncbirc file.

I followed the instructions on the MacQiime webpage to create this profile even though it should still have been present -- we've used blast before. I suppose after upgrading our Mac Pro Desktop to MacOS Sierra the .ncbirc profile somehow got deleted. After re-creating this file...

I finally ran everything smoothly.

Thanks!! Sometimes it takes a little pain....

Jai Ram Rideout

unread,

May 1, 2017, 7:21:53 PM5/1/17

to Qiime 1 Forum

Great, thanks for posting your solution Jay!

Jai

Reply all

Reply to author

Forward

Error Problem assigning taxonomy for Fungal ITS using blast - Help

Jay T

Jai Ram Rideout

Jay T

Jay T

Jai Ram Rideout

Jay T

Jay T

Jay T

Jay T

Jai Ram Rideout