GreenGenes 12_10 - problem retraining RDP classifier

1,508 views
Skip to first unread message

Aaron

unread,
Oct 17, 2012, 12:22:59 PM10/17/12
to qiime...@googlegroups.com

Hi All,

I'm having a problem retraining the RDP classifier using assign_taxonomy.py.

I ran assign_taxonomy.py with “-t gg_12_10_otus/taxonomy/97_otu_taxonomy.txt” but got an error stating:  “Each reference assignment must contain 6 items, specifying domain, phylum, class, order, family, and genus. Detected 7 items"

Running assign_taxonomy.py with gg_12_10_otus/taxonomy/97_otu_taxonomy.txt gives the same error message as running with feb2011 greengenes_tax.txt.  Running assign_taxonomy.py with feb2011 greengenes_tax_rdp_train.txt works without error.

I'm using QIIME 1.5 (macqiime installation) with RDP Classifier version: rdp_classifier-2.2.jar

I've also attached the output from print_qiime_config.py
 
Thanks for the help!
 
-aaron
config.txt

Kyle Bittinger

unread,
Oct 17, 2012, 1:14:21 PM10/17/12
to qiime...@googlegroups.com
This issue has been fixed in newer versions of QIIME. To use with
version 1.5, the taxonomy must be trimmed to include only 6 ranks.

--Kyle
> --
>
>
>

Daniel McDonald

unread,
Oct 17, 2012, 1:17:45 PM10/17/12
to qiime...@googlegroups.com
I just spoke with Greg and this was an oversight on our part. It does
work with 1.5.0-dev. We are going to discuss this and report back
shortly.
-Daniel
> --
>
>
>

Greg Caporaso

unread,
Oct 18, 2012, 12:36:11 AM10/18/12
to qiime...@googlegroups.com
Hi all,
We have a patch to the 12_10 Greengenes OTU collection that provides files that can be used to retrain the RDP classifier version 2.2. This is necessary for QIIME 1.5.0 users, but not for users of QIIME 1.5.0-dev (or later as the relevant changes will be in the next release of QIIME). To use this, cd to the directory containing your gg_12_10_otus directory and run the following commands:

tar -xzf gg_12_10_otus.rdp22_patch.tgz

This will unpack the several files and put them in the right place in your gg_12_10_otus directory. Use the resulting taxonomy files (gg_12_10_otus/taxonomy/*rdp22_train.txt) as the id_to_taxonomy_map files for retraining the RDP classifier. 

Thanks Aaron for identifying the issue, and thanks to Daniel McDonald, Tony Walters, and Giorgio Casaburi for help with the solution!

Greg

MarineLanda

unread,
Oct 18, 2012, 11:14:35 AM10/18/12
to qiime...@googlegroups.com
Hi everyone,

Thank you Greg for posting this response. I can't make it work though, I always get a "couldn't connect to host".

Thanks for your help!

Marine

Daniel McDonald

unread,
Oct 18, 2012, 11:33:45 AM10/18/12
to qiime...@googlegroups.com
Hey Marine,

Is this from the "curl" command that Greg suggested? Can you please
open a web browser and go to ftp://thebeast.colorado.edu and report
back if it worked?

Thanks,
Daniel
> --
>
>
>

ashleyb

unread,
Oct 18, 2012, 12:13:56 PM10/18/12
to qiime...@googlegroups.com
Hi Daniel,
I got the same error "could not connect to host" using curl, but I was able to pull the file from the ftp site via web browser.

Ashley

MarineLanda

unread,
Oct 18, 2012, 12:17:54 PM10/18/12
to qiime...@googlegroups.com

Hey Daniel,

I had tried it and it wasn't working either but now it's fine, I was able to download the files and retrain the rdp classifier, many thanks!

Is there any way to get the same file going to the genus level? ... Or maybe it doesn't matter, once the rdp classifier is retrained using this one I can run the analysis again using a different "-t" file going to the genus?

I am sure my questions are very naive, I'm not so good at understanding the bases of all of this...

Thank you!

Marine

Daniel McDonald

unread,
Oct 18, 2012, 12:20:30 PM10/18/12
to qiime...@googlegroups.com
Can you try the following command please?

curl -O ftp://thebeast.colorado.edu/pub/gg_12_10_otus.rdp22_patch.tar.gz
> --
>
>
>
Message has been deleted

Daniel McDonald

unread,
Oct 18, 2012, 12:27:12 PM10/18/12
to qiime...@googlegroups.com
Great, thanks! The rest of the instructions should be fine
-Daniel

On Thu, Oct 18, 2012 at 10:23 AM, MarineLanda <land...@orange.fr> wrote:
> Yeap, this works! :-)
> --
>
>
>

Bibaswan

unread,
Oct 19, 2012, 3:42:17 PM10/19/12
to qiime...@googlegroups.com
Hi Daniel,

I downloaded the patch zip file and extracted all the contents. But when I looked into the 97_otu_taxonomy.rdp22_train.txt file, it only has taxonomic information upto the family level, where as I need the taxonomic information upto the species level. What can I do in this case?

Thanks,

Bibaswan

Luke Ursell

unread,
Oct 20, 2012, 7:42:09 PM10/20/12
to qiime...@googlegroups.com
Hi,

The latest greengenes taxonomy mapping can be found here with taxonomic information down to the species level: http://greengenes.secondgenome.com/downloads/database/12_10

Best,
Luke
--
 
 
 

Raven

unread,
Oct 21, 2012, 3:37:01 PM10/21/12
to qiime...@googlegroups.com
Hi all,

I'm having the same issue. When I open the patch file (gg_12_10_otus.rpd22_patch.tar.gz) to get the id_to_taxonomy file (97_otu_taxonomy.rdp22_train.txt), I only see taxonomic information down to family level. Do I need to run the code in note.patch to get to the genus level? Or perhaps I'm looking at the wrong file? I'm using Qiime 1.5.0, so I need to use the file in the patch instead of the original 97_otu_taxonomy.txt file that was in the gg_12_10_otus file.

Thanks!
Raven

Bibaswan

unread,
Oct 22, 2012, 2:22:31 PM10/22/12
to qiime...@googlegroups.com
Hi all,

I agree with Raven. Is that what we are supposed to do? Because if its down to the species level then the RDP classifier will fail saying that there are seven levels.

Thanks,

Bibaswan

Greg Caporaso

unread,
Oct 22, 2012, 2:44:05 PM10/22/12
to qiime...@googlegroups.com
Hi all,
This patch is designed to work at the family level, as with the
previous version of the RDP training files (in the 4feb2011 build).
This is a limitation in QIIME 1.5.0, not in Greengenes, and has been
address in QIIME 1.5.0-dev. So, you can update to work with QIIME
1.5.0-dev, or hold off until the 1.6.0 release. We don't have a date
set for that yet, but I expect that it will be sometime within the
next month or so.

Greg
> --
>
>
>

krishna Jala

unread,
Nov 6, 2012, 8:52:43 AM11/6/12
to qiime...@googlegroups.com
Hi

Does Qiime 1.5.0-dev work in virtual machines?
I am having issues to update from 1.5.0 tp 1.5.0-dev in VM.
I would like to use recent green genes library to compare and reach comparisons to the genus level.

here is the error

DEPLOYMENT SUMMARY

Packages deployed successfully:
rtax, ipython, pycogent, pynast, pprospector, qiime

Packages skipped (assumed successful):
data-lanemask, data-core, ampliconnoise, raxml, drisee, chimeraslayer, cytoscape, cdbtools, mothur, python, infernal, clearcut, blast, parsinsert, gg_otus, muscle, fasttree, cdhit, r, vienna, rdpclassifier, pplacer, uclust, numpy, tornado, SQLAlchemy, pyzmq, setuptools, mpi4py, MySQL-python, sphinx, matplotlib

Packages failed to deploy:
biom-format



Any advice would helpful.

Thanks
Jala



--
 
 
 

Jai Ram Rideout

unread,
Nov 6, 2012, 9:51:56 AM11/6/12
to qiime...@googlegroups.com
Hi Jala,

Can you please try removing the biom-format directory (probably /home/ubuntu/qiime_software/biom-format-0.9.3-release/, but may be a different version). Then, rerun the app-deploy command and it should reinstall the biom-format package.

-Jai


--
 
 
 

krishna Jala

unread,
Nov 6, 2012, 10:46:39 AM11/6/12
to qiime...@googlegroups.com
Hi Jai,
No luck. It is giving similar error

Thanks
Krishna


--
 
 
 

Jai Ram Rideout

unread,
Nov 6, 2012, 11:04:34 AM11/6/12
to qiime...@googlegroups.com
Hi Krishna,

Can you please send me the full command(s) that you are running, as well as the app-deploy config file that you are using (as an attachment)?

Thanks,
Jai


--
 
 
 

krishna Jala

unread,
Nov 6, 2012, 11:11:42 AM11/6/12
to qiime...@googlegroups.com
Hi Jai,
Here are the commands that I was using (bold).



qiime@qiime-VirtualBox:~$ wget ftp://thebeast.colorado.edu/pub/QIIME-v1.5.0-dependencies/app-deploy-qiime-1.5.0.tgz
--2012-11-06 08:43:13--  ftp://thebeast.colorado.edu/pub/QIIME-v1.5.0-dependencies/app-deploy-qiime-1.5.0.tgz
           => `app-deploy-qiime-1.5.0.tgz.1'
Resolving thebeast.colorado.edu (thebeast.colorado.edu)... 128.138.93.17
Connecting to thebeast.colorado.edu (thebeast.colorado.edu)|128.138.93.17|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD (1) /pub/QIIME-v1.5.0-dependencies ... done.
==> SIZE app-deploy-qiime-1.5.0.tgz ... 21057
==> PASV ... done.    ==> RETR app-deploy-qiime-1.5.0.tgz ... done.
Length: 21057 (21K) (unauthoritative)

100%[======================================>] 21,057       132K/s   in 0.2s   

2012-11-06 08:43:14 (132 KB/s) - `app-deploy-qiime-1.5.0.tgz.1' saved [21057]

qiime@qiime-VirtualBox:~$ tar zxvf app-deploy-qiime-1.5.0.tgz
app-deploy-qiime-1.5.0/
app-deploy-qiime-1.5.0/app-deploy.py
app-deploy-qiime-1.5.0/etc/
app-deploy-qiime-1.5.0/lib/
app-deploy-qiime-1.5.0/README
app-deploy-qiime-1.5.0/lib/__init__.py
app-deploy-qiime-1.5.0/lib/application.py
app-deploy-qiime-1.5.0/lib/config.py
app-deploy-qiime-1.5.0/lib/custom.py
app-deploy-qiime-1.5.0/lib/data.py
app-deploy-qiime-1.5.0/lib/dependencies.py
app-deploy-qiime-1.5.0/lib/deploy.py
app-deploy-qiime-1.5.0/lib/environment.py
app-deploy-qiime-1.5.0/lib/logconfig.py
app-deploy-qiime-1.5.0/lib/util.py
app-deploy-qiime-1.5.0/etc/qiime_1.5.0.conf
app-deploy-qiime-1.5.0/etc/qiime_1.5.0_repository.conf
app-deploy-qiime-1.5.0/etc/qiime_compy.conf
qiime@qiime-VirtualBox:~$ cd app-deploy-qiime-1.5.0
qiime@qiime-VirtualBox:~/app-deploy-qiime-1.5.0$ python app-deploy.py $HOME/qiime_software/ -f etc/qiime_1.5.0_repository.conf --force-remove-failed-dirs

2012-11-06 08:44:09,818 - __main__         - INFO     - Dependencies look good.
2012-11-06 08:44:09,821 - __main__         - INFO     - Starting status thread.
2012-11-06 08:44:09,823 - __main__         - INFO     - Starting 6 deploy threads.
2012-11-06 08:44:09,824 - lib.deploy       - INFO     - Still deploying applications and data...
2012-11-06 08:44:09,828 - __main__         - INFO     - Running deploy for data and apps without dependencies.
2012-11-06 08:44:09,829 - __main__         - INFO     - Searching for python...
2012-11-06 08:44:09,829 - __main__         - INFO     - python-exe is not specified.
2012-11-06 08:44:09,829 - __main__         - INFO     - Looks like python is being deployed, using that version
2012-11-06 08:44:09,830 - __main__         - INFO     - Setting python: /home/qiime/qiime_software/python-2.7.1-release/bin/python
2012-11-06 08:44:26,414 - __main__         - INFO     - Launched initial deploy, remaining apps: ['pynast', 'pprospector', 'MySQL-python', 'biom-format', 'qiime', 'pycogent', 'numpy', 'pyzmq', 'SQLAlchemy', 'tornado', 'matplotlib', 'ipython', 'sphinx', 'setuptools', 'mpi4py']
2012-11-06 08:44:26,416 - __main__         - INFO     - Deploying remaining apps.
2012-11-06 08:44:26,456 - lib.data         - INFO     - Deploying data: data-lanemask
2012-11-06 08:44:26,457 - lib.data         - INFO     - data-lanemask exists, skipping download.
2012-11-06 08:44:26,457 - lib.deploy       - INFO     - Deploy of data-lanemask skipped.
2012-11-06 08:44:26,457 - lib.data         - INFO     - Deploying data: data-core
2012-11-06 08:44:26,457 - lib.data         - INFO     - data-core exists, skipping download.
2012-11-06 08:44:26,458 - lib.deploy       - INFO     - Deploy of data-core skipped.
2012-11-06 08:44:26,458 - lib.application  - INFO     - Deploying app: clearcut
2012-11-06 08:44:26,458 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/clearcut-1.0.9-release
2012-11-06 08:44:26,458 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:26,460 - lib.application  - INFO     - Deploying app: raxml
2012-11-06 08:44:26,461 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/raxml-7.3.0-release
2012-11-06 08:44:26,461 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:26,462 - lib.deploy       - INFO     - Deploy of raxml skipped.
2012-11-06 08:44:26,461 - lib.application  - INFO     - Deploying app: cytoscape
2012-11-06 08:44:26,462 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/cytoscape-2.7.0-release
2012-11-06 08:44:26,462 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:26,463 - lib.deploy       - INFO     - Deploy of cytoscape skipped.
2012-11-06 08:44:26,463 - lib.application  - INFO     - Deploying app: chimeraslayer
2012-11-06 08:44:26,463 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/chimeraslayer-4.29.2010-release
2012-11-06 08:44:26,463 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:26,460 - lib.application  - INFO     - Deploying app: ampliconnoise
2012-11-06 08:44:26,464 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/ampliconnoise-1.25-release
2012-11-06 08:44:26,464 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:26,465 - lib.deploy       - INFO     - Deploy of ampliconnoise skipped.
2012-11-06 08:44:26,466 - lib.application  - INFO     - Deploying app: gg_otus
2012-11-06 08:44:26,460 - lib.application  - INFO     - Deploying app: cdbtools
2012-11-06 08:44:26,466 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/cdbtools-10.11.2010-release
2012-11-06 08:44:26,466 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:26,467 - lib.deploy       - INFO     - Deploy of cdbtools skipped.
2012-11-06 08:44:26,467 - lib.application  - INFO     - Deploying app: infernal
2012-11-06 08:44:26,467 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/infernal-1.0.2-release
2012-11-06 08:44:26,467 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:26,467 - lib.deploy       - INFO     - Deploy of infernal skipped.
2012-11-06 08:44:26,461 - lib.application  - INFO     - Deploying app: rtax
2012-11-06 08:44:26,468 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/rtax-0.982-release
2012-11-06 08:44:26,468 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:26,468 - lib.deploy       - INFO     - Deploy of rtax skipped.
2012-11-06 08:44:26,469 - lib.application  - INFO     - Deploying app: vienna
2012-11-06 08:44:26,469 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/vienna-1.8.4-release
2012-11-06 08:44:26,469 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:26,469 - lib.deploy       - INFO     - Deploy of vienna skipped.
2012-11-06 08:44:26,470 - lib.application  - INFO     - Deploying app: python
2012-11-06 08:44:26,470 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/python-2.7.1-release
2012-11-06 08:44:26,469 - lib.application  - INFO     - Deploying app: mothur
2012-11-06 08:44:26,466 - lib.application  - INFO     - Deploying app: drisee
2012-11-06 08:44:26,503 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/drisee-1.2-release
2012-11-06 08:44:26,468 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/gg_otus-4feb2011-release
2012-11-06 08:44:26,505 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:26,507 - lib.deploy       - INFO     - Deploy of gg_otus skipped.
2012-11-06 08:44:26,465 - lib.deploy       - INFO     - Deploy of chimeraslayer skipped.
2012-11-06 08:44:26,509 - lib.application  - INFO     - Deploying app: fasttree
2012-11-06 08:44:26,509 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/fasttree-2.1.3-release
2012-11-06 08:44:26,509 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:26,510 - lib.deploy       - INFO     - Deploy of fasttree skipped.
2012-11-06 08:44:26,510 - lib.application  - INFO     - Deploying app: pplacer
2012-11-06 08:44:26,510 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/pplacer-1.1-release
2012-11-06 08:44:26,511 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:26,460 - lib.deploy       - INFO     - Deploy of clearcut skipped.
2012-11-06 08:44:26,511 - lib.application  - INFO     - Deploying app: blast
2012-11-06 08:44:26,511 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/blast-2.2.22-release
2012-11-06 08:44:26,511 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:26,512 - lib.deploy       - INFO     - Deploy of blast skipped.
2012-11-06 08:44:26,512 - lib.application  - INFO     - Deploying app: parsinsert
2012-11-06 08:44:26,516 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/parsinsert-1.0.4-release
2012-11-06 08:44:26,516 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:26,503 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:26,516 - lib.deploy       - INFO     - Deploy of python skipped.
2012-11-06 08:44:26,517 - lib.application  - INFO     - Deploying app: r
2012-11-06 08:44:26,517 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/r-2.12.0-release
2012-11-06 08:44:26,517 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:26,517 - lib.deploy       - INFO     - Deploy of r skipped.
2012-11-06 08:44:26,517 - lib.application  - INFO     - Deploying app: muscle
2012-11-06 08:44:26,509 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:26,522 - lib.deploy       - INFO     - Deploy of drisee skipped.
2012-11-06 08:44:26,526 - lib.application  - INFO     - Deploying app: cdhit
2012-11-06 08:44:26,526 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/cdhit-3.1-release
2012-11-06 08:44:26,526 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:26,527 - lib.deploy       - INFO     - Deploy of cdhit skipped.
2012-11-06 08:44:26,512 - lib.deploy       - INFO     - Deploy of pplacer skipped.
2012-11-06 08:44:26,527 - lib.application  - INFO     - Deploying app: uclust
2012-11-06 08:44:26,527 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/uclust-1.2.22-release
2012-11-06 08:44:26,527 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:26,527 - lib.deploy       - INFO     - Deploy of uclust skipped.
2012-11-06 08:44:26,524 - lib.application  - INFO     - Deploying app: rdpclassifier
2012-11-06 08:44:26,528 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/rdpclassifier-2.2-release
2012-11-06 08:44:26,528 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:26,524 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/muscle-3.8.31-release
2012-11-06 08:44:26,533 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:26,533 - lib.deploy       - INFO     - Deploy of muscle skipped.
2012-11-06 08:44:26,524 - lib.deploy       - INFO     - Deploy of parsinsert skipped.
2012-11-06 08:44:26,500 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/mothur-1.25.0-release
2012-11-06 08:44:26,534 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:26,534 - lib.deploy       - INFO     - Deploy of mothur skipped.
2012-11-06 08:44:26,535 - lib.deploy       - INFO     - Deploy of rdpclassifier skipped.
2012-11-06 08:44:31,444 - lib.application  - INFO     - Deploying app: numpy
2012-11-06 08:44:31,445 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/numpy-1.5.1-release
2012-11-06 08:44:31,445 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:31,466 - lib.application  - INFO     - Deploying app: pyzmq
2012-11-06 08:44:31,467 - lib.application  - INFO     - Deploying app: SQLAlchemy
2012-11-06 08:44:31,475 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/SQLAlchemy-0.7.1-release
2012-11-06 08:44:31,475 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:31,468 - lib.application  - INFO     - Deploying app: setuptools
2012-11-06 08:44:31,468 - lib.application  - INFO     - Deploying app: tornado
2012-11-06 08:44:31,499 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/tornado-2.2.1-release
2012-11-06 08:44:31,499 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:31,468 - lib.application  - INFO     - Deploying app: ipython
2012-11-06 08:44:31,496 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/pyzmq-2.1.9-release
2012-11-06 08:44:31,525 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:31,518 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/setuptools-0.6c11-release
2012-11-06 08:44:31,560 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:31,555 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/ipython-0.13-release
2012-11-06 08:44:31,594 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:32,119 - lib.deploy       - INFO     - Deploy of numpy skipped.
2012-11-06 08:44:32,131 - lib.application  - INFO     - Deploying app: mpi4py
2012-11-06 08:44:32,145 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/mpi4py-1.2.2-release
2012-11-06 08:44:32,148 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:32,200 - lib.deploy       - INFO     - Deploy of SQLAlchemy skipped.
2012-11-06 08:44:32,194 - lib.deploy       - INFO     - Deploy of pyzmq skipped.
2012-11-06 08:44:32,203 - lib.deploy       - INFO     - Deploy of tornado skipped.
2012-11-06 08:44:32,233 - lib.deploy       - INFO     - Deploy of ipython skipped.
2012-11-06 08:44:32,268 - lib.deploy       - INFO     - Deploy of setuptools skipped.
2012-11-06 08:44:32,353 - lib.deploy       - INFO     - Deploy of mpi4py skipped.
2012-11-06 08:44:36,440 - lib.application  - INFO     - Deploying app: MySQL-python
2012-11-06 08:44:36,441 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/MySQL-python-1.2.3-release
2012-11-06 08:44:36,441 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:36,469 - lib.application  - INFO     - Deploying app: biom-format
2012-11-06 08:44:36,474 - lib.application  - INFO     - Deploying app: pycogent
2012-11-06 08:44:36,486 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/pycogent-1.5.1-repository-1191
2012-11-06 08:44:36,486 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:36,485 - lib.application  - INFO     - Deploying app: matplotlib
2012-11-06 08:44:36,488 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/matplotlib-1.1.0-release
2012-11-06 08:44:36,489 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:36,485 - lib.application  - INFO     - Deploying app: sphinx
2012-11-06 08:44:36,504 - lib.application  - INFO     - Making deploy directory: /home/qiime/qiime_software/biom-format-1.0.0-repository-unknown-repo-version
2012-11-06 08:44:36,550 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/sphinx-1.0.4-release
2012-11-06 08:44:36,571 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:36,572 - lib.application  - INFO     - Downloading biom-format
2012-11-06 08:44:36,650 - lib.util         - INFO     - Cloning git://github.com/biom-format/biom-format.git to /tmp/tmpeK1QaR
2012-11-06 08:44:36,760 - lib.util         - ERROR    - git clone failed: git://github.com/biom-format/biom-format.git
2012-11-06 08:44:36,762 - lib.application  - ERROR    - biom-format download failed.
2012-11-06 08:44:36,768 - lib.deploy       - ERROR    - Deploy of biom-format failed.
2012-11-06 08:44:37,110 - lib.deploy       - INFO     - Deploy of MySQL-python skipped.
2012-11-06 08:44:37,130 - lib.deploy       - INFO     - Deploy of matplotlib skipped.
2012-11-06 08:44:37,134 - lib.deploy       - INFO     - Deploy of pycogent skipped.
2012-11-06 08:44:37,148 - lib.deploy       - INFO     - Deploy of sphinx skipped.
2012-11-06 08:44:39,852 - lib.deploy       - INFO     - Still deploying applications and data...
2012-11-06 08:44:41,438 - lib.application  - INFO     - Deploying app: pynast
2012-11-06 08:44:41,439 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/pynast-1.1-repository-54
2012-11-06 08:44:41,439 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:41,438 - lib.application  - INFO     - Deploying app: pprospector
2012-11-06 08:44:41,453 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/pprospector-1.0.1-repository-99
2012-11-06 08:44:41,456 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:41,477 - lib.application  - INFO     - Deploying app: qiime
2012-11-06 08:44:41,498 - lib.application  - INFO     - Deploy directory exists: /home/qiime/qiime_software/qiime-1.5.0-repository-3313
2012-11-06 08:44:41,499 - lib.application  - INFO     - Skipping deployment, assuming successful.
2012-11-06 08:44:41,910 - lib.deploy       - INFO     - Deploy of pprospector skipped.
2012-11-06 08:44:41,921 - lib.deploy       - INFO     - Deploy of pynast skipped.
2012-11-06 08:44:41,959 - lib.deploy       - INFO     - Deploy of qiime skipped.
2012-11-06 08:45:11,964 - lib.util         - INFO     - Removing directory: /home/qiime/qiime_software/biom-format-1.0.0-repository-unknown-repo-version
2012-11-06 08:45:11,973 - __main__         - INFO     - Generating activate.sh file
2012-11-06 08:45:11,974 - lib.util         - INFO     - Backing up /home/qiime/qiime_software/activate.sh to /home/qiime/qiime_software/activate.sh.bak.1
2012-11-06 08:45:11,975 - __main__         - INFO     - Generating new ~/.qiime_config_default file
2012-11-06 08:45:11,976 - lib.util         - INFO     - Backing up /home/qiime/.qiime_config_default to /home/qiime/.qiime_config_default.bak.1
2012-11-06 08:45:11,977 - __main__         - INFO     - Generated new ~/.qiime_config_default file



DEPLOYMENT SUMMARY

Packages deployed successfully:


Packages skipped (assumed successful):
data-lanemask, data-core, cytoscape, raxml, ampliconnoise, cdbtools, rtax, infernal, vienna, chimeraslayer, fasttree, clearcut, blast, python, r, gg_otus, drisee, pplacer, uclust, muscle, parsinsert, cdhit, mothur, rdpclassifier, numpy, SQLAlchemy, pyzmq, tornado, ipython, setuptools, mpi4py, MySQL-python, matplotlib, pycogent, sphinx, pprospector, pynast, qiime


Packages failed to deploy:
biom-format


qiime@qiime-VirtualBox:~/app-deploy-qiime-1.5.0$



--
 
 
 

Jai Ram Rideout

unread,
Nov 6, 2012, 12:43:09 PM11/6/12
to qiime...@googlegroups.com
Hi Krishna,

It looks like there is a problem with git, which is used to pull the biom-format repository. Can you please send the output of the following commands:

which git

git

Thanks,
Jai


--
 
 
 

krishna Jala

unread,
Nov 6, 2012, 1:31:55 PM11/6/12
to qiime...@googlegroups.com
Hi Jai,
Here it is.
qiime@qiime-VirtualBox:~/app-deploy-qiime-1.5.0$ which git
qiime@qiime-VirtualBox:~/app-deploy-qiime-1.5.0$ git
The program 'git' is currently not installed.  You can install it by typing:
sudo apt-get install git



--
 
 
 

Jai Ram Rideout

unread,
Nov 6, 2012, 1:50:41 PM11/6/12
to qiime...@googlegroups.com
Hi Krishna,

Please run the following command, which will install git, and then try running the app-deploy script again.

sudo apt-get install git

-Jai


--
 
 
 

krishna Jala

unread,
Nov 6, 2012, 2:33:25 PM11/6/12
to qiime...@googlegroups.com
Hi Jai,

Here it is what I got.

Looks like it worked.  How do I know, it worked..


DEPLOYMENT SUMMARY

Packages deployed successfully:
biom-format

Packages skipped (assumed successful):
data-core, ampliconnoise, cdbtools, raxml, clearcut, chimeraslayer, drisee, gg_otus, rtax, mothur, infernal, cytoscape, data-lanemask, blast, parsinsert, vienna, rdpclassifier, cdhit, python, fasttree, muscle, pplacer, uclust, r, SQLAlchemy, pyzmq, tornado, numpy, setuptools, ipython, mpi4py, pycogent, sphinx, matplotlib, MySQL-python, pynast, pprospector, qiime

Packages failed to deploy:


Thanks
Jala




--
 
 
 

Jai Ram Rideout

unread,
Nov 6, 2012, 3:07:11 PM11/6/12
to qiime...@googlegroups.com
Hi Krishna,

That output looks promising, as the biom-format package was installed successfully. The next step is to source your .bashrc (if you haven't done so already):

source ~/.bashrc

Then run the following command to test your QIIME install:

print_qiime_config.py -t

-Jai


--
 
 
 

krishna Jala

unread,
Nov 6, 2012, 3:15:09 PM11/6/12
to qiime...@googlegroups.com
Hi Jai
Thanks. It came up with 2 Failures. Do I have to train the GG_12-10 to RDP and get the species level?



qiime@qiime-VirtualBox:~/app-deploy-qiime-1.5.0$ source ~/.bashrc
qiime@qiime-VirtualBox:~/app-deploy-qiime-1.5.0$ print_qiime_config.py -t

System information
==================
         Platform:    linux2
   Python version:    2.7.3 (default, Aug  1 2012, 05:25:23)  [GCC 4.6.3]
Python executable:    /home/qiime/qiime_software/python-2.7.1-release/bin/python

Dependency versions
===================
                     PyCogent version:    1.5.3-dev
                        NumPy version:    1.5.1
                   matplotlib version:    1.1.0
                  biom-format version:    1.0.0-dev
                QIIME library version:    1.5.0-dev
                 QIIME script version:    1.5.0-dev
        PyNAST version (if installed):    1.1-dev
RDP Classifier version (if installed):    rdp_classifier-2.2.jar

QIIME config values
===================
                     blastmat_dir:    /home/qiime/qiime_software/blast-2.2.22-release/data
                         sc_queue:    all.q
      topiaryexplorer_project_dir:    None
     pynast_template_alignment_fp:    /home/qiime/qiime_software/core_set_aligned.fasta.imputed
                  cluster_jobs_fp:    /home/qiime/qiime_software/qiime-1.5.0-release/bin/start_parallel_jobs.py
pynast_template_alignment_blastdb:    None
assign_taxonomy_reference_seqs_fp:    /home/qiime/qiime_software/gg_otus-4feb2011-release/rep_set/gg_97_otus_4feb2011.fasta
                     torque_queue:    friendlyq
              qiime_test_data_dir:    None
   template_alignment_lanemask_fp:    /home/qiime/qiime_software/lanemask_in_1s_and_0s
                    jobs_to_start:    1
                cloud_environment:    False
                qiime_scripts_dir:    /home/qiime/qiime_software/qiime-1.5.0-release/bin
            denoiser_min_per_core:    50
                      working_dir:    /tmp/
                    python_exe_fp:    /home/qiime/qiime_software/python-2.7.1-release/bin/python
                         temp_dir:    /tmp/
                      blastall_fp:    /home/qiime/qiime_software/blast-2.2.22-release/bin/blastall
                 seconds_to_sleep:    60
assign_taxonomy_id_to_taxonomy_fp:    /home/qiime/qiime_software/gg_otus-4feb2011-release/taxonomies/greengenes_tax_rdp_train.txt


running checks:

test_FastTree_supported_version (__main__.Qiime_config)
FastTree is in path and version is supported ... ok
test_INFERNAL_supported_version (__main__.Qiime_config)
INFERNAL is in path and version is supported ... ok
test_ParsInsert_supported_version (__main__.Qiime_config)
ParsInsert is in path and version is supported ... ok
test_R_supported_version (__main__.Qiime_config)
R is in path and version is supported ... ok
test_ampliconnoise_install (__main__.Qiime_config)
AmpliconNoise install looks sane. ... ok
test_blast_supported_version (__main__.Qiime_config)
blast is in path and version is supported ... ok
test_blastall_fp (__main__.Qiime_config)
blastall_fp is set to a valid path ... ok
test_blastmat_dir (__main__.Qiime_config)
blastmat_dir is set to a valid path. ... ok
test_cdbtools_supported_version (__main__.Qiime_config)
cdbtools is in path and version is supported ... ok
test_cdhit_supported_version (__main__.Qiime_config)
cd-hit is in path and version is supported ... ok
test_chimeraSlayer_install (__main__.Qiime_config)
no obvious problems with ChimeraSlayer install ... ok
test_clearcut_supported_version (__main__.Qiime_config)
clearcut is in path and version is supported ... ok
test_cluster_jobs_fp (__main__.Qiime_config)
cluster_jobs_fp is set to a valid path and is executable ... ok
test_denoiser_supported_version (__main__.Qiime_config)
denoiser aligner is ready to use ... ok
test_for_obsolete_values (__main__.Qiime_config)
local qiime_config has no extra params ... ok
test_matplotlib_suported_version (__main__.Qiime_config)
maptplotlib version is supported ... ok
test_mothur_supported_version (__main__.Qiime_config)
mothur is in path and version is supported ... ok
test_muscle_supported_version (__main__.Qiime_config)
muscle is in path and version is supported ... ok
test_numpy_suported_version (__main__.Qiime_config)
numpy version is supported ... ok
test_pplacer_supported_version (__main__.Qiime_config)
pplacer is in path and version is supported ... ok
test_pynast_suported_version (__main__.Qiime_config)
pynast version is supported ... ok
test_pynast_template_alignment_blastdb_fp (__main__.Qiime_config)
pynast_template_alignment_blastdb, if set, is set to a valid path ... ok
test_pynast_template_alignment_fp (__main__.Qiime_config)
pynast_template_alignment, if set, is set to a valid path ... ok
test_python_exe_fp (__main__.Qiime_config)
python_exe_fp is set to a working python env ... ok
test_python_supported_version (__main__.Qiime_config)
python is in path and version is supported ... FAIL
test_qiime_scripts_dir (__main__.Qiime_config)
qiime_scripts_dir, if set, is set to a valid path ... ok
test_qiime_test_data_dir (__main__.Qiime_config)
qiime_test_data_dir, if set, is set to a valid path ... ok
test_raxmlHPC_supported_version (__main__.Qiime_config)
raxmlHPC is in path and version is supported ... ok
test_rtax_supported_version (__main__.Qiime_config)
rtax is in path and version is supported ... ok
test_temp_dir (__main__.Qiime_config)
temp_dir, if set, is set to a valid path ... ok
test_template_alignment_lanemask_fp (__main__.Qiime_config)
template_alignment_lanemask, if set, is set to a valid path ... ok
test_uclust_supported_version (__main__.Qiime_config)
uclust is in path and version is supported ... ok
test_usearch_supported_version (__main__.Qiime_config)
usearch is in path and version is supported ... FAIL
test_working_dir (__main__.Qiime_config)
working_dir, if set, is set to a valid path ... ok

======================================================================
FAIL: test_python_supported_version (__main__.Qiime_config)
python is in path and version is supported
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/qiime/qiime_software/qiime-1.5.0-repository-3313/bin/print_qiime_config.py", line 338, in test_python_supported_version
    version_string))
AssertionError: Unsupported python version. Must be >= 2.7.1 and <= 2.7.2 , but running 2.7.3.

======================================================================
FAIL: test_usearch_supported_version (__main__.Qiime_config)
usearch is in path and version is supported
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/qiime/qiime_software/qiime-1.5.0-repository-3313/bin/print_qiime_config.py", line 676, in test_usearch_supported_version
    "which components of QIIME you plan to use.")
AssertionError: usearch not found. This may or may not be a problem depending on which components of QIIME you plan to use.

----------------------------------------------------------------------
Ran 34 tests in 2.823s

FAILED (failures=2)











=====================================================================
FAIL: test_python_supported_version (__main__.Qiime_config)
python is in path and version is supported
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/qiime/qiime_software/qiime-1.5.0-repository-3313/bin/print_qiime_config.py", line 338, in test_python_supported_version
    version_string))
AssertionError: Unsupported python version. Must be >= 2.7.1 and <= 2.7.2 , but running 2.7.3.

======================================================================
FAIL: test_usearch_supported_version (__main__.Qiime_config)
usearch is in path and version is supported
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/qiime/qiime_software/qiime-1.5.0-repository-3313/bin/print_qiime_config.py", line 676, in test_usearch_supported_version
    "which components of QIIME you plan to use.")
AssertionError: usearch not found. This may or may not be a problem depending on which components of QIIME you plan to use.

----------------------------------------------------------------------
Ran 34 tests in 2.823s

FAILED (failures=2)



--
 
 
 

Jai Ram Rideout

unread,
Nov 6, 2012, 3:30:37 PM11/6/12
to qiime...@googlegroups.com
Those failures are okay- Python 2.7.3 should work fine, and usearch can be obtained from http://www.drive5.com/usearch/.

You'll need to retrain the RDP classifier with the latest Greengenes taxonomy map and reference sequences to get species-level classifications.

-Jai


--
 
 
 

krishna Jala

unread,
Nov 6, 2012, 3:58:22 PM11/6/12
to qiime...@googlegroups.com
Hey Jai,
Can I do this by defining in parameters file with following replacements?

assign_taxonomy:id_to_taxonomy_fp    /home/qiime/qiime_software/gg_12_10_taxonomy.txt
assign_taxonomy:rdp_max_memory    4000
assign_taxonomy:reference_seqs_fp    /home/qiime/qiime_software/gg_12_10.fasta

or
Do I have to replace the "gg_12_10_taxonomy.txt" with RDP trained file...I'm just confused with it. The RDP trained file, I'm getting up to family level.

If possible, could you please explain, how do I go about it in 1.5.1-dev version to get the OTUs at genus/species level

thanks
Jala

 


--
 
 
 

Jai Ram Rideout

unread,
Nov 6, 2012, 4:26:15 PM11/6/12
to qiime...@googlegroups.com
Hi,

Please see Daniel's instructions here:


Note that you don't need to download the patch for RDP because you're using the development version of QIIME, so you can use the normal taxonomy map that comes packaged with the release (e.g. gg_12_10_otus/taxonomy/97_otu_taxonomy.txt).

Daniel's instructions will set up the new Greengenes database as the default for your QIIME analyses (because the paths are added to your .qiime_config file). You could also edit your parameters file like you suggested above with the following lines (for example, if you're running the pick_otus_through_otu_table.py workflow and want to pass a parameters file with -p). Be sure to update the paths to point to where you downloaded and extracted the Greengenes database:

assign_taxonomy:id_to_taxonomy_fp    /home/qiime/qiime_software/gg_12_10_otus/taxonomy/97_otu_taxonomy.txt
assign_taxonomy:rdp_max_memory    4000
assign_taxonomy:reference_seqs_fp    /home/qiime/qiime_software/gg_12_10_otus/rep_set/97_otus.fasta

Please be sure to use the same percent similarity threshold (e.g. 97%) for id_to_taxonomy_fp, reference_seqs_fp, and pick_otus.py's -s option. This doesn't necessarily have to be 97%, I was just using this as an example (it is the default similarity threshold used in QIIME).

-Jai


--
 
 
 

krishna Jala

unread,
Nov 6, 2012, 4:27:31 PM11/6/12
to qiime...@googlegroups.com
When I use these commands, this is log file.
Does this mean, am I comparing with old GG library?
log_20121106141908.txt
log_20121106142115.txt

krishna Jala

unread,
Nov 6, 2012, 5:49:11 PM11/6/12
to qiime...@googlegroups.com
Hey Jai
Some error is coming up

qiime@qiime-VirtualBox:~$ pick_otus_through_otu_table.py -i '/home/qiime/KU_seqs.fna' -o KU1_110612_otus_species -p '/home/qiime/qiime_parameters1.txt'Traceback (most recent call last):
  File "/home/qiime/qiime_software/qiime-1.5.0-repository-3313/bin/pick_otus_through_otu_table.py", line 132, in <module>
    main()   
  File "/home/qiime/qiime_software/qiime-1.5.0-repository-3313/bin/pick_otus_through_otu_table.py", line 129, in main
    status_update_callback=status_update_callback)
  File "/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/qiime/workflow.py", line 471, in run_pick_otus_through_otu_table
    close_logger_on_success=close_logger_on_success)
  File "/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/qiime/workflow.py", line 135, in call_commands_serially
    raise WorkflowError, msg
qiime.workflow.WorkflowError:

*** ERROR RAISED DURING STEP: Pick OTUs
Command run was:
 python /home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/scripts/pick_otus.py -i /home/qiime/KU_seqs.fna -o KU1_110612_otus_species/uclust_picked_otus --max_e_value 1e-10 --clustering_algorithm furthest --similarity 0.97 --otu_picking_method uclust --max_cdhit_memory 400
Command returned exit status: 2
Stdout:

Stderr
python: can't open file '/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/scripts/pick_otus.py': [Errno 2] No such file or directory

krishna Jala

unread,
Nov 6, 2012, 6:09:20 PM11/6/12
to qiime...@googlegroups.com
Hi Jai and Daniel,
After updating to from 1.5.0 to 1.5-dev , this was I got. Looks like my config. file does not have some thing

qiime@qiime-VirtualBox:~$ print_qiime_config.py -t


System information
==================
         Platform:    linux2
   Python version:    2.7.3 (default, Aug  1 2012, 05:25:23)  [GCC 4.6.3]
Python executable:    /home/qiime/qiime_software/python-2.7.1-release/bin/python

Dependency versions
===================
                     PyCogent version:    1.5.3-dev
                        NumPy version:    1.5.1
                   matplotlib version:    1.1.0
                  biom-format version:    1.0.0-dev
                QIIME library version:    1.5.0-dev
                 QIIME script version:    1.5.0-dev
        PyNAST version (if installed):    1.1-dev
RDP Classifier version (if installed):    rdp_classifier-2.2.jar

QIIME config values
===================
                     blastmat_dir:    None
                         sc_queue:    all.q
      topiaryexplorer_project_dir:    None
     pynast_template_alignment_fp:    None
                  cluster_jobs_fp:    None
pynast_template_alignment_blastdb:    None
assign_taxonomy_reference_seqs_fp:    None
                     torque_queue:    friendlyq
              qiime_test_data_dir:    None
   template_alignment_lanemask_fp:    None
                    jobs_to_start:    1
                cloud_environment:    False
                qiime_scripts_dir:    None
            denoiser_min_per_core:    50
                      working_dir:    None
                    python_exe_fp:    python
                         temp_dir:    /tmp/
                      blastall_fp:    blastall
                 seconds_to_sleep:    2
assign_taxonomy_id_to_taxonomy_fp:    None
cluster_jobs_fp is set to a valid path and is executable ... FAIL
FAIL: test_cluster_jobs_fp (__main__.Qiime_config)

cluster_jobs_fp is set to a valid path and is executable
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/qiime/qiime_software/qiime-1.5.0-repository-3313/bin/print_qiime_config.py", line 102, in test_cluster_jobs_fp
    self.fail("Your qiime_config file doesn't have cluster_jobs_fp\n.")
AssertionError: Your qiime_config file doesn't have cluster_jobs_fp
.


======================================================================
FAIL: test_python_supported_version (__main__.Qiime_config)
python is in path and version is supported
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/qiime/qiime_software/qiime-1.5.0-repository-3313/bin/print_qiime_config.py", line 338, in test_python_supported_version
    version_string))
AssertionError: Unsupported python version. Must be >= 2.7.1 and <= 2.7.2 , but running 2.7.3.

======================================================================
FAIL: test_usearch_supported_version (__main__.Qiime_config)
usearch is in path and version is supported
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/qiime/qiime_software/qiime-1.5.0-repository-3313/bin/print_qiime_config.py", line 676, in test_usearch_supported_version
    "which components of QIIME you plan to use.")
AssertionError: usearch not found. This may or may not be a problem depending on which components of QIIME you plan to use.

----------------------------------------------------------------------
Ran 34 tests in 2.158s

FAILED (failures=3)



--
 
 
 

krishna Jala

unread,
Nov 6, 2012, 6:47:53 PM11/6/12
to qiime...@googlegroups.com
Looks there is some issue picking up at pick_otus.py script level in my running.
Now, my commands are itself not running. Not sure what happend..

qiime@qiime-VirtualBox:~$ pick_otus_through_otu_table.py -i '/home/qiime/KU_seqs.fna' -o KU_Otus_110612 -p '/home/qiime/qiime_parameter2.txt

Will Van Treuren

unread,
Nov 6, 2012, 7:02:08 PM11/6/12
to qiime...@googlegroups.com
Hi Krishna, 

You have a '  after the -p option but you don't have another ' indicating the completion of the string. 
Try this:

pick_otus_through_otu_table.py -i /home/qiime/KU_seqs.fna -o KU_Otus_110612 -p /home/qiime/qiime_parameter2.txt

Hope this helps,

Will 


--
 
 
 

Krishna

unread,
Nov 6, 2012, 9:46:36 PM11/6/12
to qiime...@googlegroups.com
Hi
Where is this .qiime_config file?
How do we direct this to new gg database?
I tried by parameters file. No luck


Thank you
Krishna

--
 
 
 

krishna Jala

unread,
Nov 6, 2012, 10:50:33 PM11/6/12
to qiime...@googlegroups.com
This is the error I am getting with 1.5.1-dev

qiime@qiime-VirtualBox:~$ pick_otus_through_otu_table.py -i '/home/qiime/KU_seqs.fna' -o KU_Otus_110612 -f

Traceback (most recent call last):
  File "/home/qiime/qiime_software/qiime-1.5.0-repository-3313/bin/pick_otus_through_otu_table.py", line 132, in <module>
    main()   
  File "/home/qiime/qiime_software/qiime-1.5.0-repository-3313/bin/pick_otus_through_otu_table.py", line 129, in main
    status_update_callback=status_update_callback)
  File "/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/qiime/workflow.py", line 471, in run_pick_otus_through_otu_table
    close_logger_on_success=close_logger_on_success)
  File "/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/qiime/workflow.py", line 135, in call_commands_serially
    raise WorkflowError, msg
qiime.workflow.WorkflowError:

*** ERROR RAISED DURING STEP: Assign taxonomy
Command run was:
 /home/qiime/qiime_software/python-2.7.1-release/bin/python /home/qiime/qiime_software/qiime-1.5.0-release/bin/assign_taxonomy.py -o KU_Otus_110612/rdp_assigned_taxonomy -i KU_Otus_110612/rep_set//KU_seqs_rep_set.fasta
Command returned exit status: 1
Stdout:

Stderr

Traceback (most recent call last):
  File "/home/qiime/qiime_software/qiime-1.5.0-release/bin/assign_taxonomy.py", line 19, in <module>
    from qiime.assign_taxonomy import (
ImportError: cannot import name guess_rdp_version



--
 
 
 

Daniel McDonald

unread,
Nov 6, 2012, 10:55:02 PM11/6/12
to qiime...@googlegroups.com
Hey Krishna,

The output indicates you are using qiime 1.5.0. What path did you install dev under? And, can you please send the following output:

echo $PATH
echo $PYTHONPATH

Thanks
Daniel


--
 
 
 

krishna Jala

unread,
Nov 6, 2012, 11:06:39 PM11/6/12
to qiime...@googlegroups.com
Hi Daniel

qiime@qiime-VirtualBox:~$ echo $PATH
/home/qiime/picrust-dev/scripts/:/home/qiime/picrust-dev/scripts/:/home/qiime/picrust-dev/scripts/:/home/qiime/picrust-dev/scripts/:/home/qiime/picrust-dev/scripts/:/home/qiime/picrust-dev/scripts/:/home/qiime/picrust-dev/scripts/:/home/qiime/picrust-dev/scripts/:/home/qiime/picrust-dev/scripts/:/home/qiime/picrust-dev/scripts/:/home/qiime/picrust-dev/scripts/:/home/qiime/qiime_software/pprospector-1.0.1-repository-99/bin:/home/qiime/qiime_software/qiime-1.5.0-repository-3313/bin:/home/qiime/qiime_software/pynast-1.1-repository-54/bin:/home/qiime/qiime_software/biom-format-1.0.0-repository-fa1ff53e/scripts/:/home/qiime/qiime_software/tornado-2.2.1-release/bin:/home/qiime/qiime_software/pyzmq-2.1.9-release/bin:/home/qiime/qiime_software/ipython-0.13-release/bin:/home/qiime/qiime_software/cdhit-3.1-release/.:/home/qiime/qiime_software/r-2.12.0-release/bin:/home/qiime/qiime_software/uclust-1.2.22-release/.:/home/qiime/qiime_software/rdpclassifier-2.2-release/.:/home/qiime/qiime_software/muscle-3.8.31-release/.:/home/qiime/qiime_software/vienna-1.8.4-release/.:/home/qiime/qiime_software/parsinsert-1.0.4-release/.:/home/qiime/qiime_software/drisee-1.2-release/.:/home/qiime/qiime_software/blast-2.2.22-release/bin:/home/qiime/qiime_software/fasttree-2.1.3-release/.:/home/qiime/qiime_software/pplacer-1.1-release/.:/home/qiime/qiime_software/cytoscape-2.7.0-release/.:/home/qiime/qiime_software/python-2.7.1-release/bin:/home/qiime/qiime_software/mothur-1.25.0-release/.:/home/qiime/qiime_software/infernal-1.0.2-release/bin:/home/qiime/qiime_software/ampliconnoise-1.25-release/Scripts:/home/qiime/qiime_software/ampliconnoise-1.25-release/bin:/home/qiime/qiime_software/clearcut-1.0.9-release/.:/home/qiime/qiime_software/chimeraslayer-4.29.2010-release/ChimeraSlayer:/home/qiime/qiime_software/chimeraslayer-4.29.2010-release/NAST-iEr:/home/qiime/qiime_software/cdbtools-10.11.2010-release/.:/home/qiime/qiime_software/raxml-7.3.0-release/.:/home/qiime/qiime_software/rtax-0.982-release/.:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games


qiime@qiime-VirtualBox:~$ echo $PYTHONPATH
/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/qiime_software/pprospector-1.0.1-repository-99/lib/python2.7/site-packages:/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/python2.7/site-packages:/home/qiime/qiime_software/pynast-1.1-repository-54/lib/python2.7/site-packages:/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/:/home/qiime/qiime_software/sphinx-1.0.4-release/lib/python2.7/site-packages:/home/qiime/qiime_software/matplotlib-1.1.0-release/lib/python2.7/site-packages:/home/qiime/qiime_software/pycogent-1.5.1-repository-1191/lib/python2.7/site-packages:/home/qiime/qiime_software/MySQL-python-1.2.3-release/lib/python2.7/site-packages:/home/qiime/qiime_software/biom-format-1.0.0-repository-fa1ff53e/python-code/:/home/qiime/qiime_software/mpi4py-1.2.2-release/lib/python2.7/site-packages:/home/qiime/qiime_software/setuptools-0.6c11-release/lib/python2.7/site-packages:/home/qiime/qiime_software/pyzmq-2.1.9-release/lib/python2.7/site-packages:/home/qiime/qiime_software/tornado-2.2.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/SQLAlchemy-0.7.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/numpy-1.5.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/ipython-0.13-release/lib/python2.7/site-packages:
qiime@qiime-VirtualBox:~$ echo $PYTHONPATH
/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/qiime_software/pprospector-1.0.1-repository-99/lib/python2.7/site-packages:/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/python2.7/site-packages:/home/qiime/qiime_software/pynast-1.1-repository-54/lib/python2.7/site-packages:/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/:/home/qiime/qiime_software/sphinx-1.0.4-release/lib/python2.7/site-packages:/home/qiime/qiime_software/matplotlib-1.1.0-release/lib/python2.7/site-packages:/home/qiime/qiime_software/pycogent-1.5.1-repository-1191/lib/python2.7/site-packages:/home/qiime/qiime_software/MySQL-python-1.2.3-release/lib/python2.7/site-packages:/home/qiime/qiime_software/biom-format-1.0.0-repository-fa1ff53e/python-code/:/home/qiime/qiime_software/mpi4py-1.2.2-release/lib/python2.7/site-packages:/home/qiime/qiime_software/setuptools-0.6c11-release/lib/python2.7/site-packages:/home/qiime/qiime_software/pyzmq-2.1.9-release/lib/python2.7/site-packages:/home/qiime/qiime_software/tornado-2.2.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/SQLAlchemy-0.7.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/numpy-1.5.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/ipython-0.13-release/lib/python2.7/site-packages:
qiime@qiime-VirtualBox:~$ echo $PYTHONPATH
/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/qiime_software/pprospector-1.0.1-repository-99/lib/python2.7/site-packages:/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/python2.7/site-packages:/home/qiime/qiime_software/pynast-1.1-repository-54/lib/python2.7/site-packages:/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/:/home/qiime/qiime_software/sphinx-1.0.4-release/lib/python2.7/site-packages:/home/qiime/qiime_software/matplotlib-1.1.0-release/lib/python2.7/site-packages:/home/qiime/qiime_software/pycogent-1.5.1-repository-1191/lib/python2.7/site-packages:/home/qiime/qiime_software/MySQL-python-1.2.3-release/lib/python2.7/site-packages:/home/qiime/qiime_software/biom-format-1.0.0-repository-fa1ff53e/python-code/:/home/qiime/qiime_software/mpi4py-1.2.2-release/lib/python2.7/site-packages:/home/qiime/qiime_software/setuptools-0.6c11-release/lib/python2.7/site-packages:/home/qiime/qiime_software/pyzmq-2.1.9-release/lib/python2.7/site-packages:/home/qiime/qiime_software/tornado-2.2.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/SQLAlchemy-0.7.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/numpy-1.5.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/ipython-0.13-release/lib/python2.7/site-packages:
qiime@qiime-VirtualBox:~$ echo $PYTHONPATH
/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/qiime_software/pprospector-1.0.1-repository-99/lib/python2.7/site-packages:/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/python2.7/site-packages:/home/qiime/qiime_software/pynast-1.1-repository-54/lib/python2.7/site-packages:/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/:/home/qiime/qiime_software/sphinx-1.0.4-release/lib/python2.7/site-packages:/home/qiime/qiime_software/matplotlib-1.1.0-release/lib/python2.7/site-packages:/home/qiime/qiime_software/pycogent-1.5.1-repository-1191/lib/python2.7/site-packages:/home/qiime/qiime_software/MySQL-python-1.2.3-release/lib/python2.7/site-packages:/home/qiime/qiime_software/biom-format-1.0.0-repository-fa1ff53e/python-code/:/home/qiime/qiime_software/mpi4py-1.2.2-release/lib/python2.7/site-packages:/home/qiime/qiime_software/setuptools-0.6c11-release/lib/python2.7/site-packages:/home/qiime/qiime_software/pyzmq-2.1.9-release/lib/python2.7/site-packages:/home/qiime/qiime_software/tornado-2.2.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/SQLAlchemy-0.7.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/numpy-1.5.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/ipython-0.13-release/lib/python2.7/site-packages:
qiime@qiime-VirtualBox:~$ echo $PYTHONPATH
/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/qiime_software/pprospector-1.0.1-repository-99/lib/python2.7/site-packages:/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/python2.7/site-packages:/home/qiime/qiime_software/pynast-1.1-repository-54/lib/python2.7/site-packages:/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/:/home/qiime/qiime_software/sphinx-1.0.4-release/lib/python2.7/site-packages:/home/qiime/qiime_software/matplotlib-1.1.0-release/lib/python2.7/site-packages:/home/qiime/qiime_software/pycogent-1.5.1-repository-1191/lib/python2.7/site-packages:/home/qiime/qiime_software/MySQL-python-1.2.3-release/lib/python2.7/site-packages:/home/qiime/qiime_software/biom-format-1.0.0-repository-fa1ff53e/python-code/:/home/qiime/qiime_software/mpi4py-1.2.2-release/lib/python2.7/site-packages:/home/qiime/qiime_software/setuptools-0.6c11-release/lib/python2.7/site-packages:/home/qiime/qiime_software/pyzmq-2.1.9-release/lib/python2.7/site-packages:/home/qiime/qiime_software/tornado-2.2.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/SQLAlchemy-0.7.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/numpy-1.5.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/ipython-0.13-release/lib/python2.7/site-packages:
qiime@qiime-VirtualBox:~$ echo $PYTHONPATH
/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/qiime_software/pprospector-1.0.1-repository-99/lib/python2.7/site-packages:/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/python2.7/site-packages:/home/qiime/qiime_software/pynast-1.1-repository-54/lib/python2.7/site-packages:/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/:/home/qiime/qiime_software/sphinx-1.0.4-release/lib/python2.7/site-packages:/home/qiime/qiime_software/matplotlib-1.1.0-release/lib/python2.7/site-packages:/home/qiime/qiime_software/pycogent-1.5.1-repository-1191/lib/python2.7/site-packages:/home/qiime/qiime_software/MySQL-python-1.2.3-release/lib/python2.7/site-packages:/home/qiime/qiime_software/biom-format-1.0.0-repository-fa1ff53e/python-code/:/home/qiime/qiime_software/mpi4py-1.2.2-release/lib/python2.7/site-packages:/home/qiime/qiime_software/setuptools-0.6c11-release/lib/python2.7/site-packages:/home/qiime/qiime_software/pyzmq-2.1.9-release/lib/python2.7/site-packages:/home/qiime/qiime_software/tornado-2.2.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/SQLAlchemy-0.7.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/numpy-1.5.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/ipython-0.13-release/lib/python2.7/site-packages:
qiime@qiime-VirtualBox:~$ echo $PYTHONPATH
/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/qiime_software/pprospector-1.0.1-repository-99/lib/python2.7/site-packages:/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/python2.7/site-packages:/home/qiime/qiime_software/pynast-1.1-repository-54/lib/python2.7/site-packages:/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/:/home/qiime/qiime_software/sphinx-1.0.4-release/lib/python2.7/site-packages:/home/qiime/qiime_software/matplotlib-1.1.0-release/lib/python2.7/site-packages:/home/qiime/qiime_software/pycogent-1.5.1-repository-1191/lib/python2.7/site-packages:/home/qiime/qiime_software/MySQL-python-1.2.3-release/lib/python2.7/site-packages:/home/qiime/qiime_software/biom-format-1.0.0-repository-fa1ff53e/python-code/:/home/qiime/qiime_software/mpi4py-1.2.2-release/lib/python2.7/site-packages:/home/qiime/qiime_software/setuptools-0.6c11-release/lib/python2.7/site-packages:/home/qiime/qiime_software/pyzmq-2.1.9-release/lib/python2.7/site-packages:/home/qiime/qiime_software/tornado-2.2.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/SQLAlchemy-0.7.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/numpy-1.5.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/ipython-0.13-release/lib/python2.7/site-packages:
qiime@qiime-VirtualBox:~$ echo $PYTHONPATH
/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/picrust-dev/picrust/:/home/qiime/qiime_software/pprospector-1.0.1-repository-99/lib/python2.7/site-packages:/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/python2.7/site-packages:/home/qiime/qiime_software/pynast-1.1-repository-54/lib/python2.7/site-packages:/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/:/home/qiime/qiime_software/sphinx-1.0.4-release/lib/python2.7/site-packages:/home/qiime/qiime_software/matplotlib-1.1.0-release/lib/python2.7/site-packages:/home/qiime/qiime_software/pycogent-1.5.1-repository-1191/lib/python2.7/site-packages:/home/qiime/qiime_software/MySQL-python-1.2.3-release/lib/python2.7/site-packages:/home/qiime/qiime_software/biom-format-1.0.0-repository-fa1ff53e/python-code/:/home/qiime/qiime_software/mpi4py-1.2.2-release/lib/python2.7/site-packages:/home/qiime/qiime_software/setuptools-0.6c11-release/lib/python2.7/site-packages:/home/qiime/qiime_software/pyzmq-2.1.9-release/lib/python2.7/site-packages:/home/qiime/qiime_software/tornado-2.2.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/SQLAlchemy-0.7.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/numpy-1.5.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/ipython-0.13-release/lib/python2.7/site-packages:


Thanks
Jala



--
 
 
 

krishna Jala

unread,
Nov 6, 2012, 11:11:45 PM11/6/12
to qiime...@googlegroups.com
The PWD for qiime 1.5.0-dev:

qiime@qiime-VirtualBox:~/qiime_software$ cd qiime-1.5.0-repository-3313/
qiime@qiime-VirtualBox:~/qiime_software/qiime-1.5.0-repository-3313$ ls
bin    ChangeLog  examples  lib    README            scripts   tests
build  doc        INSTALL   qiime  requirements.txt  setup.py  web
qiime@qiime-VirtualBox:~/qiime_software/qiime-1.5.0-repository-3313$ pwd
/home/qiime/qiime_software/qiime-1.5.0-repository-3313
qiime@qiime-VirtualBox:~/qiime_software/qiime-1.5.0-repository-3313$


On Tue, Nov 6, 2012 at 8:55 PM, Daniel McDonald <was...@gmail.com> wrote:
--
 
 
 

krishna Jala

unread,
Nov 6, 2012, 11:12:10 PM11/6/12
to qiime...@googlegroups.com
The PWD for qiime-1.5.0-dev


qiime@qiime-VirtualBox:~/qiime_software$ cd qiime-1.5.0-repository-3313/
qiime@qiime-VirtualBox:~/qiime_software/qiime-1.5.0-repository-3313$ ls
bin    ChangeLog  examples  lib    README            scripts   tests
build  doc        INSTALL   qiime  requirements.txt  setup.py  web
qiime@qiime-VirtualBox:~/qiime_software/qiime-1.5.0-repository-3313$ pwd
/home/qiime/qiime_software/qiime-1.5.0-repository-3313
qiime@qiime-VirtualBox:~/qiime_software/qiime-1.5.0-repository-3313$

Daniel McDonald

unread,
Nov 6, 2012, 11:11:59 PM11/6/12
to qiime...@googlegroups.com
That's...overly complex, let's see if we can simplify your environment to track this issue down. Can you please send the output from the following?

cat ~/.bash_profile
cat ~/.bashrc
cat ~/qiime_software/activate.sh

Thanks
Daniel
--
 
 
 

krishna Jala

unread,
Nov 6, 2012, 11:22:45 PM11/6/12
to qiime...@googlegroups.com


it looks more complicated

qiime@qiime-VirtualBox:~$ cat ~/.bash_profile
cat: /home/qiime/.bash_profile: No such file or directory
qiime@qiime-VirtualBox:~$ cat ~/.bashrc
# ~/.bashrc: executed by bash(1) for non-login shells.
# see /usr/share/doc/bash/examples/startup-files (in the package bash-doc)
# for examples

# If not running interactively, don't do anything
[ -z "$PS1" ] && return

# don't put duplicate lines or lines starting with space in the history.
# See bash(1) for more options
HISTCONTROL=ignoreboth

# append to the history file, don't overwrite it
shopt -s histappend

# for setting history length see HISTSIZE and HISTFILESIZE in bash(1)
HISTSIZE=1000
HISTFILESIZE=2000

# check the window size after each command and, if necessary,
# update the values of LINES and COLUMNS.
shopt -s checkwinsize

# If set, the pattern "**" used in a pathname expansion context will
# match all files and zero or more directories and subdirectories.
#shopt -s globstar

# make less more friendly for non-text input files, see lesspipe(1)
[ -x /usr/bin/lesspipe ] && eval "$(SHELL=/bin/sh lesspipe)"

# set variable identifying the chroot you work in (used in the prompt below)
if [ -z "$debian_chroot" ] && [ -r /etc/debian_chroot ]; then
    debian_chroot=$(cat /etc/debian_chroot)
fi

# set a fancy prompt (non-color, unless we know we "want" color)
case "$TERM" in
    xterm-color) color_prompt=yes;;
esac

# uncomment for a colored prompt, if the terminal has the capability; turned
# off by default to not distract the user: the focus in a terminal window
# should be on the output of commands, not on the prompt
#force_color_prompt=yes

if [ -n "$force_color_prompt" ]; then
    if [ -x /usr/bin/tput ] && tput setaf 1 >&/dev/null; then
    # We have color support; assume it's compliant with Ecma-48
    # (ISO/IEC-6429). (Lack of such support is extremely rare, and such
    # a case would tend to support setf rather than setaf.)
    color_prompt=yes
    else
    color_prompt=
    fi
fi

if [ "$color_prompt" = yes ]; then
    PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '
else
    PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '
fi
unset color_prompt force_color_prompt

# If this is an xterm set the title to user@host:dir
case "$TERM" in
xterm*|rxvt*)
    PS1="\[\e]0;${debian_chroot:+($debian_chroot)}\u@\h: \w\a\]$PS1"
    ;;
*)
    ;;
esac

# enable color support of ls and also add handy aliases
if [ -x /usr/bin/dircolors ]; then
    test -r ~/.dircolors && eval "$(dircolors -b ~/.dircolors)" || eval "$(dircolors -b)"
    alias ls='ls --color=auto'
    #alias dir='dir --color=auto'
    #alias vdir='vdir --color=auto'

    alias grep='grep --color=auto'
    alias fgrep='fgrep --color=auto'
    alias egrep='egrep --color=auto'
fi

# some more ls aliases
alias ll='ls -alF'
alias la='ls -A'
alias l='ls -CF'

# Add an "alert" alias for long running commands.  Use like so:
#   sleep 10; alert
alias alert='notify-send --urgency=low -i "$([ $? = 0 ] && echo terminal || echo error)" "$(history|tail -n1|sed -e '\''s/^\s*[0-9]\+\s*//;s/[;&|]\s*alert$//'\'')"'

# Alias definitions.
# You may want to put all your additions into a separate file like
# ~/.bash_aliases, instead of adding them here directly.
# See /usr/share/doc/bash-doc/examples in the bash-doc package.

if [ -f ~/.bash_aliases ]; then
    . ~/.bash_aliases
fi

# enable programmable completion features (you don't need to enable
# this, if it's already enabled in /etc/bash.bashrc and /etc/profile
# sources /etc/bash.bashrc).
if [ -f /etc/bash_completion ] && ! shopt -oq posix; then
    . /etc/bash_completion
fi
source /home/qiime/qiime_software/activate.sh
export PYTHONPATH=~/picrust-dev/picrust/:$PYTHONPATH
export PATH=~/picrust-dev/scripts/:$PATH
export PYTHONPATH=~/picrust-dev/picrust/:$PYTHONPATH
export PATH=~/picrust-dev/scripts/:$PATH
export PYTHONPATH=~/picrust-dev/picrust/:$PYTHONPATH
export PATH=~/picrust-dev/scripts/:$PATH
export PYTHONPATH=~/picrust-dev/picrust/:$PYTHONPATH
export PATH=~/picrust-dev/scripts/:$PATH
export PYTHONPATH=~/picrust-dev/picrust/:$PYTHONPATH
export PATH=~/picrust-dev/scripts/:$PATH
export PYTHONPATH=~/picrust-dev/picrust/:$PYTHONPATH
export PATH=~/picrust-dev/scripts/:$PATH
export PYTHONPATH=~/picrust-dev/picrust/:$PYTHONPATH
export PATH=~/picrust-dev/scripts/:$PATH
export PYTHONPATH=~/picrust-dev/picrust/:$PYTHONPATH
export PATH=~/picrust-dev/scripts/:$PATH
export PYTHONPATH=~/picrust-dev/picrust/:$PYTHONPATH
export PATH=~/picrust-dev/scripts/:$PATH
export PYTHONPATH=~/picrust-dev/picrust/:$PYTHONPATH
export PYTHONPATH=~/picrust-dev/picrust/:$PYTHONPATH
export PATH=~/picrust-dev/scripts/:$PATH
export PYTHONPATH=~/picrust-dev/picrust/:$PYTHONPATH
export PATH=~/picrust-dev/scripts/:$PATH



--
 
 
 

krishna Jala

unread,
Nov 6, 2012, 11:24:14 PM11/6/12
to qiime...@googlegroups.com
qiime@qiime-VirtualBox:~$ cat ~/qiime_software/activate.sh
export QIIME=/home/qiime/qiime_software/qiime-1.5.0-repository-3313/./
export PYTHONPATH=/home/qiime/qiime_software/pprospector-1.0.1-repository-99/lib/python2.7/site-packages:/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/python2.7/site-packages:/home/qiime/qiime_software/pynast-1.1-repository-54/lib/python2.7/site-packages:/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/:/home/qiime/qiime_software/sphinx-1.0.4-release/lib/python2.7/site-packages:/home/qiime/qiime_software/matplotlib-1.1.0-release/lib/python2.7/site-packages:/home/qiime/qiime_software/pycogent-1.5.1-repository-1191/lib/python2.7/site-packages:/home/qiime/qiime_software/MySQL-python-1.2.3-release/lib/python2.7/site-packages:/home/qiime/qiime_software/biom-format-1.0.0-repository-fa1ff53e/python-code/:/home/qiime/qiime_software/mpi4py-1.2.2-release/lib/python2.7/site-packages:/home/qiime/qiime_software/setuptools-0.6c11-release/lib/python2.7/site-packages:/home/qiime/qiime_software/pyzmq-2.1.9-release/lib/python2.7/site-packages:/home/qiime/qiime_software/tornado-2.2.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/SQLAlchemy-0.7.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/numpy-1.5.1-release/lib/python2.7/site-packages:/home/qiime/qiime_software/ipython-0.13-release/lib/python2.7/site-packages:$PYTHONPATH
export TEST_DB=1
export BLASTMAT=/home/qiime/qiime_software/blast-2.2.22-release/data
export QIIME_CONFIG_FP=~/.qiime_config_default
export PYRO_LOOKUP_FILE=/home/qiime/qiime_software/ampliconnoise-1.25-release/Data/LookUp_E123.dat
export RDP_JAR_PATH=/home/qiime/qiime_software/rdpclassifier-2.2-release/rdp_classifier-2.2.jar
export PYCOGENT=/home/qiime/qiime_software/pycogent-1.5.1-repository-1191/./
export SEQ_LOOKUP_FILE=/home/qiime/qiime_software/ampliconnoise-1.25-release/Data/Tran.dat
export PATH=/home/qiime/qiime_software/pprospector-1.0.1-repository-99/bin:/home/qiime/qiime_software/qiime-1.5.0-repository-3313/bin:/home/qiime/qiime_software/pynast-1.1-repository-54/bin:/home/qiime/qiime_software/biom-format-1.0.0-repository-fa1ff53e/scripts/:/home/qiime/qiime_software/tornado-2.2.1-release/bin:/home/qiime/qiime_software/pyzmq-2.1.9-release/bin:/home/qiime/qiime_software/ipython-0.13-release/bin:/home/qiime/qiime_software/cdhit-3.1-release/.:/home/qiime/qiime_software/r-2.12.0-release/bin:/home/qiime/qiime_software/uclust-1.2.22-release/.:/home/qiime/qiime_software/rdpclassifier-2.2-release/.:/home/qiime/qiime_software/muscle-3.8.31-release/.:/home/qiime/qiime_software/vienna-1.8.4-release/.:/home/qiime/qiime_software/parsinsert-1.0.4-release/.:/home/qiime/qiime_software/drisee-1.2-release/.:/home/qiime/qiime_software/blast-2.2.22-release/bin:/home/qiime/qiime_software/fasttree-2.1.3-release/.:/home/qiime/qiime_software/pplacer-1.1-release/.:/home/qiime/qiime_software/cytoscape-2.7.0-release/.:/home/qiime/qiime_software/python-2.7.1-release/bin:/home/qiime/qiime_software/mothur-1.25.0-release/.:/home/qiime/qiime_software/infernal-1.0.2-release/bin:/home/qiime/qiime_software/ampliconnoise-1.25-release/Scripts:/home/qiime/qiime_software/ampliconnoise-1.25-release/bin:/home/qiime/qiime_software/clearcut-1.0.9-release/.:/home/qiime/qiime_software/chimeraslayer-4.29.2010-release/ChimeraSlayer:/home/qiime/qiime_software/chimeraslayer-4.29.2010-release/NAST-iEr:/home/qiime/qiime_software/cdbtools-10.11.2010-release/.:/home/qiime/qiime_software/raxml-7.3.0-release/.:/home/qiime/qiime_software/rtax-0.982-release/.:$PATH

Daniel McDonald

unread,
Nov 6, 2012, 11:37:58 PM11/6/12
to qiime...@googlegroups.com
Thanks. Can you please remove the duplicate entries at the end of .bashrc with a text editor? These are the ones with picrust in them. Then, please close and open a new terminal, and send the output for:

which assign_taxonomy.py
cat ~/.qiime_config

We're getting close...
Daniel


--
 
 
 

krishna Jala

unread,
Nov 6, 2012, 11:57:41 PM11/6/12
to qiime...@googlegroups.com
Thank you. Daniel

qiime@qiime-VirtualBox:~$ which assign_taxonomy.py
/home/qiime/qiime_software/qiime-1.5.0-repository-3313/bin/assign_taxonomy.py
qiime@qiime-VirtualBox:~$ cat ~/.qiime_config
cat: /home/qiime/.qiime_config: No such file or directory



--
 
 
 

krishna Jala

unread,
Nov 6, 2012, 11:57:52 PM11/6/12
to qiime...@googlegroups.com

Krishna

unread,
Nov 7, 2012, 12:03:36 AM11/7/12
to qiime...@googlegroups.com
I don't understand   "Can you please remove the duplicate .bashrc with a text editor? These are the ones with picrust in them. "



Thank you
Krishna


On Nov 6, 2012, at 11:37 PM, Daniel McDonald <was...@gmail.com> wrote:

Thanks. Can you please remove the duplicate .bashrc with a text editor? These are the ones with picrust in them. Then, please close and open a new terminal, and send the output for:
--
 
 
 

Daniel McDonald

unread,
Nov 7, 2012, 12:13:57 AM11/7/12
to qiime...@googlegroups.com
Please type:

kedit ~/.bashrc

And remove the duplicate picrust lines at the end of the file and then
save it? Once this is done (and following closing and reopening the
terminal), I think you should try the pick_otus_through_otu_table.py
command again.
-Daniel
> ...
>
> [Message clipped]

krishna Jala

unread,
Nov 7, 2012, 7:15:08 AM11/7/12
to qiime...@googlegroups.com
Hi Daniel,

I did it. It is the same problem. I am attaching bashrc files for your reference. Sorry for trouble. I saved them on desktop to share

qiime@qiime-VirtualBox:~$ pick_otus_through_otu_table.py -i '/home/qiime/KU_seqs.fna' -o KU_Otus_110712 -f

Traceback (most recent call last):
  File "/home/qiime/qiime_software/qiime-1.5.0-repository-3313/bin/pick_otus_through_otu_table.py", line 132, in <module>
    main()   
  File "/home/qiime/qiime_software/qiime-1.5.0-repository-3313/bin/pick_otus_through_otu_table.py", line 129, in main
    status_update_callback=status_update_callback)
  File "/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/qiime/workflow.py", line 471, in run_pick_otus_through_otu_table
    close_logger_on_success=close_logger_on_success)
  File "/home/qiime/qiime_software/qiime-1.5.0-repository-3313/lib/qiime/workflow.py", line 135, in call_commands_serially
    raise WorkflowError, msg
qiime.workflow.WorkflowError:

*** ERROR RAISED DURING STEP: Assign taxonomy
Command run was:
 /home/qiime/qiime_software/python-2.7.1-release/bin/python /home/qiime/qiime_software/qiime-1.5.0-release/bin/assign_taxonomy.py -o KU_Otus_110712/rdp_assigned_taxonomy -i KU_Otus_110712/rep_set//KU_seqs_rep_set.fasta
Command returned exit status: 1
Stdout:

Stderr
Traceback (most recent call last):
  File "/home/qiime/qiime_software/qiime-1.5.0-release/bin/assign_taxonomy.py", line 19, in <module>
    from qiime.assign_taxonomy import (
ImportError: cannot import name guess_rdp_version



--




bashrc-modified
bashrc-original

Daniel McDonald

unread,
Nov 7, 2012, 11:10:09 AM11/7/12
to qiime...@googlegroups.com
There is something latent still pointing to QIIME 1.5.0 and not dev.
Can you send me the following:

cat $QIIME_CONFIG_FP

Thanks,
Daniel
> ...
>
> [Message clipped]

krishna Jala

unread,
Nov 7, 2012, 12:19:33 PM11/7/12
to qiime...@googlegroups.com
Hi Daniel,
Here it is.

qiime@qiime-VirtualBox:~$ cat $QIIME_CONFIG_FP
# autogenerated by qiime-deploy
cluster_jobs_fp    /home/qiime/qiime_software/qiime-1.5.0-release/bin/start_parallel_jobs.py
python_exe_fp    /home/qiime/qiime_software/python-2.7.1-release/bin/python
qiime_scripts_dir    /home/qiime/qiime_software/qiime-1.5.0-release/bin
blastmat_dir    /home/qiime/qiime_software/blast-2.2.22-release/data
blastall_fp    /home/qiime/qiime_software/blast-2.2.22-release/bin/blastall
template_alignment_lanemask_fp    /home/qiime/qiime_software/lanemask_in_1s_and_0s
pynast_template_alignment_fp    /home/qiime/qiime_software/core_set_aligned.fasta.imputed
pynast_template_alignment_blastdb   
working_dir    /tmp/
jobs_to_start    1
seconds_to_sleep    60
temp_dir    /tmp/
cloud_environment    False
assign_taxonomy_reference_seqs_fp    /home/qiime/qiime_software/gg_otus-4feb2011-release/rep_set/gg_97_otus_4feb2011.fasta
assign_taxonomy_id_to_taxonomy_fp    /home/qiime/qiime_software/gg_otus-4feb2011-release/taxonomies/greengenes_tax_rdp_train.txt




--




Daniel McDonald

unread,
Nov 7, 2012, 12:22:49 PM11/7/12
to qiime...@googlegroups.com
There it is! Can you edit this file:

kedit $QIIME_CONFIG_FP

and change these lines:

cluster_jobs_fp
/home/qiime/qiime_software/qiime-1.5.0-release/bin/start_parallel_jobs.py
qiime_scripts_dir /home/qiime/qiime_software/qiime-1.5.0-release/bin

to:

cluster_jobs_fp
/home/qiime/qiime_software/qiime-1.5.0-repository-3313/bin/start_parallel_jobs.py
qiime_scripts_dir /home/qiime/qiime_software/qiime-1.5.0-repository-3313/bin

And then try the pick_otus_through_otu_table.py command again?
-Daniel
> ...
>
> [Message clipped]
Message has been deleted

krishna Jala

unread,
Nov 7, 2012, 11:43:03 PM11/7/12
to qiime...@googlegroups.com
Hi Daniel,

Here is the results. Do I have to do some thing to correct

qiime@qiime-VirtualBox:~$ pick_otus_through_otu_table.py -i '/home/qiime/KU_seqs.fna' -o KU_Otus_GG1012_110712 -p '/home/qiime/qiime_parameter2'.txt
/home/qiime/qiime_software/pycogent-1.5.1-repository-1191/lib/python2.7/site-packages/cogent/app/raxml.py:34: DeprecationWarning: use class cogent.app.raxml_v730.Raxml instead of cogent.app.raxml.Raxml, support discontinued in version 1.6
  '1.6')
qiime@qiime-VirtualBox:~$

Thanks



--




Jai Ram Rideout

unread,
Nov 8, 2012, 10:29:35 AM11/8/12
to qiime...@googlegroups.com
Hi,

That is just a deprecation warning, so you should be fine.

-Jai


...

[Message clipped]  

krishna Jala

unread,
Nov 9, 2012, 9:15:38 AM11/9/12
to qiime...@googlegroups.com
Thanks Jai and Daniel. You guys are wonderful.


...

[Message clipped]  

Wendy Demos

unread,
Nov 27, 2012, 12:35:56 PM11/27/12
to qiime...@googlegroups.com
I am using version 1.5.0 and am unable to connect to the link below. Is there an updated link? Thank you.


On Wednesday, October 17, 2012 11:36:11 PM UTC-5, Greg Caporaso wrote:
Hi all,

Daniel McDonald

unread,
Nov 27, 2012, 12:40:45 PM11/27/12
to qiime...@googlegroups.com
There was a typo, use this:

curl -O ftp://thebeast.colorado.edu/pub/gg_12_10_otus.rdp22_patch.tar.gz
> --
>
>
>

Wendy Demos

unread,
Nov 27, 2012, 1:36:18 PM11/27/12
to qiime...@googlegroups.com
Thanks I was able to get through and I used the tar command in the directory where the gg_12_10_otus directory is located. I am still returning the same error:

pick_otus_through_otu_table.py -i split_library_output/seqs.fna -o otus -p qiime_parameters.txt
Traceback (most recent call last):
  File "/usr/local/qiime/bin/pick_otus_through_otu_table.py", line 132, in <module>
    main()    
  File "/usr/local/qiime/bin/pick_otus_through_otu_table.py", line 129, in main
    status_update_callback=status_update_callback)
  File "/usr/local/qiime/lib/qiime/workflow.py", line 471, in run_pick_otus_through_otu_table
    close_logger_on_success=close_logger_on_success)
  File "/usr/local/qiime/lib/qiime/workflow.py", line 135, in call_commands_serially
    raise WorkflowError, msg
qiime.workflow.WorkflowError: 

*** ERROR RAISED DURING STEP: Assign taxonomy
Command run was:
 python /usr/local/qiime/scripts/assign_taxonomy.py -o otus/rdp_assigned_taxonomy -i otus/rep_set//seqs_rep_set.fasta --e_value 0.001 --reference_seqs_fp /data/greengenes/gg_12_10_otus/rep_set/97_otus.fasta --id_to_taxonomy_fp /data/greengenes/gg_12_10_taxonomy.txt --assignment_method rdp --confidence 0.8
Command returned exit status: 1
Stdout:

Stderr
Traceback (most recent call last):
  File "/usr/local/qiime/scripts/assign_taxonomy.py", line 226, in <module>
    main()
  File "/usr/local/qiime/scripts/assign_taxonomy.py", line 222, in main
    result_path=result_path,log_path=log_path)
  File "/usr/local/qiime/lib/qiime/assign_taxonomy.py", line 359, in __call__
    taxonomy_file, training_seqs_file = self._generate_training_files()
  File "/usr/local/qiime/lib/qiime/assign_taxonomy.py", line 395, in _generate_training_files
    training_set.add_lineage(seq_id, lineage_str)
  File "/usr/local/qiime/lib/qiime/assign_taxonomy.py", line 425, in add_lineage
    lineage = self._parse_lineage(lineage_str)
  File "/usr/local/qiime/lib/qiime/assign_taxonomy.py", line 442, in _parse_lineage
    (len(lineage), lineage_str, lineage))
ValueError: Each reference assignment must contain 6 items, specifying domain, phylum, class, order, family, and genus.  Detected 7 items in "k__Bacteria; p__Bacteroidetes; c__Flavobacteriia; o__Flavobacteriales; f__Flavobacteriaceae; g__Flavobacterium; s__": ['k__Bacteria', ' p__Bacteroidetes', ' c__Flavobacteriia', ' o__Flavobacteriales', ' f__Flavobacteriaceae', ' g__Flavobacterium', ' s__'].


attached is the custom parameters file I am using. Also 
QIIME config values
===================
                     blastmat_dir:      None
                         sc_queue:      all.q
      topiaryexplorer_project_dir:      None
     pynast_template_alignment_fp:      None
                  cluster_jobs_fp:      /usr/local/qiime/scripts/start_parallel_jobs.py
pynast_template_alignment_blastdb:      None
assign_taxonomy_reference_seqs_fp:      None
                     torque_queue:      friendlyq
              qiime_test_data_dir:      None
   template_alignment_lanemask_fp:      None
                    jobs_to_start:      1
                cloud_environment:      False
                qiime_scripts_dir:      /usr/local/qiime/scripts
            denoiser_min_per_core:      50
                      working_dir:      None
                    python_exe_fp:      python
                         temp_dir:      /tmp/
                      blastall_fp:      blastall
                 seconds_to_sleep:      60
assign_taxonomy_id_to_taxonomy_fp:      None


Do I ultimately need to install 1.5 dev?I'm hoping to avoid another install for the time being. Thanks.
qiime_parameters.txt

Daniel McDonald

unread,
Nov 27, 2012, 2:37:39 PM11/27/12
to qiime...@googlegroups.com
Hey Wendy,

It looks like you need to still update the
assign_taxonomy_id_to_taxonomy_fp field in your QIIME config to point
to the patched file you downloaded. Let me know if you need a further
breakdown of what to do.

Best,
Daniel
> --
>
>
>

Wendy Demos

unread,
Nov 27, 2012, 3:42:00 PM11/27/12
to qiime...@googlegroups.com
Thank you. yes I do need a further breakdown of what to do. Thank you.

Wendy Demos

unread,
Nov 27, 2012, 4:29:02 PM11/27/12
to qiime...@googlegroups.com
In the meantime I thought I could pass in the fp using the qiim_parameters file I created by; 
# Taxonomy assignment parameters
assign_taxonomy:id_to_taxonomy_fp /data/greengenes/gg_12_10_otus/taxonomy/97_otu_taxonomy.txt

This gave me the same error so I assume either passing in the parameter through the command is not the correct option or I am pointing to the incorrect file? Thank you in advance.

Daniel McDonald

unread,
Nov 27, 2012, 4:31:27 PM11/27/12
to qiime...@googlegroups.com
Hey Wendy,

That will work as well, but you need to specify the patched file
(97_otu_taxonomy.rdp22_train.txt)
-Daniel
> --
>
>
>

Wendy Demos

unread,
Nov 27, 2012, 4:54:43 PM11/27/12
to qiime...@googlegroups.com
I tried and it didn't work correctly, now I am throwing a different error:
bash-3.2$ cd /usr/local/Qiime-1.5.0/
bash-3.2$ pick_otus_through_otu_table.py -i /shared/users/wdemos/Qiime150_tut/qiime_tutorial/split_library_output/seqs.fna -o /shared/users/wdemos/Qiime150_tut/qiime_tutorial/otus -p /shared/users/wdemos/Qiime150_tut/qiime_tutorial/qiime_parameters.txt -f
Traceback (most recent call last):
  File "/usr/local/qiime/bin/pick_otus_through_otu_table.py", line 132, in <module>
    main()    
  File "/usr/local/qiime/bin/pick_otus_through_otu_table.py", line 129, in main
    status_update_callback=status_update_callback)
  File "/usr/local/qiime/lib/qiime/workflow.py", line 471, in run_pick_otus_through_otu_table
    close_logger_on_success=close_logger_on_success)
  File "/usr/local/qiime/lib/qiime/workflow.py", line 135, in call_commands_serially
    raise WorkflowError, msg
qiime.workflow.WorkflowError: 

*** ERROR RAISED DURING STEP: Assign taxonomy
Command run was:
 python /usr/local/qiime/scripts/assign_taxonomy.py -o /shared/users/wdemos/Qiime150_tut/qiime_tutorial/otus/rdp_assigned_taxonomy -i /shared/users/wdemos/Qiime150_tut/qiime_tutorial/otus/rep_set//seqs_rep_set.fasta --e_value 0.001 --reference_seqs_fp /data/greengenes/gg_12_10_otus/rep_set/97_otus.fasta --id_to_taxonomy_fp /data/greengenes/gg_12_10_otus/taxonomy/99_otu_taxonomy.rdp22_train.txt --assignment_method rdp --confidence 0.8
Command returned exit status: 1
Stdout:

Stderr
Traceback (most recent call last):
  File "/usr/local/qiime/scripts/assign_taxonomy.py", line 226, in <module>
    main()
  File "/usr/local/qiime/scripts/assign_taxonomy.py", line 222, in main
    result_path=result_path,log_path=log_path)
  File "/usr/local/qiime/lib/qiime/assign_taxonomy.py", line 364, in __call__
    max_memory=max_memory)
  File "/usr/local/qiime/lib/qiime/pycogent_backports/rdp_classifier.py", line 498, in train_rdp_classifier_and_assign_taxonomy
    max_memory=max_memory)
  File "/usr/local/qiime/lib/qiime/pycogent_backports/rdp_classifier.py", line 402, in assign_taxonomy
    app_result = app(data)
  File "/usr/local/python271/lib/python2.7/site-packages/cogent/app/util.py", line 250, in __call__
    % (str(exit_status),command)
cogent.app.util.ApplicationError: Unacceptable application exit status: 1, command: cd "/usr/local/Qiime-1.5.0/"; java -Xmx1000M -jar "/usr/local/bin/rdp_classifier-2.2.jar" -t "/tmp/RdpTrainer_9sgdrS/RdpClassifier.properties" -o "/tmp/RdpAssignments_Xh6s8V.txt" -f fixrank -q "/tmp/tmpp17uWSVNh7YqS7DnbVIT.txt" > "/tmp/tmpBDMRzPxEmegKSC7fmWZ2.txt" 2> "/tmp/tmp7dKL3KDIKLgcPFOMAP2s.txt"


Perhpas I need the breakdown of how to update the config file. Thank you.

Daniel McDonald

unread,
Nov 27, 2012, 6:24:31 PM11/27/12
to qiime...@googlegroups.com
Specifying it in the config file or the parameters file is the same. I
believe the issue is that you're using the 99% taxonomy file with the
97% rep_set. Can you please try using
/data/greengenes/gg_12_10_otus/taxonomy/97_otu_taxonomy.rdp22_train.txt
as assign_taxonomy:id_to_taxonomy_fp

Thanks
-Daniel
> --
>
>
>

Wendy Demos

unread,
Nov 28, 2012, 10:49:45 AM11/28/12
to qiime...@googlegroups.com
Hi Daniel,
 I did correct the typo and still am running into an error:

*** ERROR RAISED DURING STEP: Assign taxonomy
Command run was:
 python /usr/local/qiime/scripts/assign_taxonomy.py -o /shared/users/wdemos/Qiime150_tut/qiime_tutorial/otus/rdp_assigned_taxonomy -i /shared/users/wdemos/Qiime150_tut/qiime_tutorial/otus/rep_set//seqs_rep_set.fasta --e_value 0.001 --reference_seqs_fp /data/greengenes/gg_12_10_otus/rep_set/97_otus.fasta --id_to_taxonomy_fp /data/greengenes/gg_12_10_otus/taxonomy/97_otu_taxonomy.rdp22_train.txt --assignment_method rdp --confidence 0.8
Command returned exit status: 1
Stdout:

Stderr
Traceback (most recent call last):
  File "/usr/local/qiime/scripts/assign_taxonomy.py", line 226, in <module>
    main()
  File "/usr/local/qiime/scripts/assign_taxonomy.py", line 222, in main
    result_path=result_path,log_path=log_path)
  File "/usr/local/qiime/lib/qiime/assign_taxonomy.py", line 364, in __call__
    max_memory=max_memory)
  File "/usr/local/qiime/lib/qiime/pycogent_backports/rdp_classifier.py", line 498, in train_rdp_classifier_and_assign_taxonomy
    max_memory=max_memory)
  File "/usr/local/qiime/lib/qiime/pycogent_backports/rdp_classifier.py", line 402, in assign_taxonomy
    app_result = app(data)
  File "/usr/local/python271/lib/python2.7/site-packages/cogent/app/util.py", line 250, in __call__
    % (str(exit_status),command)
cogent.app.util.ApplicationError: Unacceptable application exit status: 1, command: cd "/usr/local/Qiime-1.5.0/"; java -Xmx1000M -jar "/usr/local/bin/rdp_classifier-2.2.jar" -t "/tmp/RdpTrainer_mZSbjg/RdpClassifier.properties" -o "/tmp/RdpAssignments_Orlwjd.txt" -f fixrank -q "/tmp/tmpphpdwMq009b4GTgfcGQk.txt" > "/tmp/tmpPHMTGpGwNtaPrw0yXFbD.txt" 2> "/tmp/tmplfFh5odqM0dEMr6PCnao.txt"

Thank you for your assistance in advance.

Daniel McDonald

unread,
Nov 28, 2012, 11:06:16 AM11/28/12
to qiime...@googlegroups.com
Hey Wendy,

Can you do two things? First, can you send the full output from
"print_qiime_config.py -t"? And second, can you add
"assign_taxonomy:rdp_max_memory 4000" to your parameters file and try
the command again?

Let me know how it goes,
Daniel
> --
>
>
>

Wendy Demos

unread,
Nov 28, 2012, 11:50:38 AM11/28/12
to qiime...@googlegroups.com
HI Daniel,
 here is the config output:
bash-3.2$ print_qiime_config.py -t

System information
==================
         Platform:      linux2
   Python version:      2.7.1 (r271:86832, Feb 27 2012, 12:29:47)  [GCC 4.1.2 20080704 (Red Hat 4.1.2-51)]
Python executable:      /usr/local/python271/bin/python

Dependency versions
===================
                     PyCogent version:  1.5.1
                        NumPy version:  1.5.1
                   matplotlib version:  1.1.0
                  biom-format version:  0.9.3
                QIIME library version:  1.5.0
                 QIIME script version:  1.5.0
        PyNAST version (if installed):  1.1
RDP Classifier version (if installed):  rdp_classifier-2.2.jar

QIIME config values
===================
                     blastmat_dir:      None
                         sc_queue:      all.q
      topiaryexplorer_project_dir:      None
     pynast_template_alignment_fp:      None
                  cluster_jobs_fp:      /usr/local/qiime/scripts/start_parallel_jobs.py
pynast_template_alignment_blastdb:      None
assign_taxonomy_reference_seqs_fp:      None
                     torque_queue:      friendlyq
              qiime_test_data_dir:      None
   template_alignment_lanemask_fp:      None
                    jobs_to_start:      1
                cloud_environment:      False
                qiime_scripts_dir:      /usr/local/qiime/scripts
            denoiser_min_per_core:      50
                      working_dir:      None
                    python_exe_fp:      python
                         temp_dir:      /tmp/
                      blastall_fp:      blastall
                 seconds_to_sleep:      60
assign_taxonomy_id_to_taxonomy_fp:      None


running checks:

test_FastTree_supported_version (__main__.Qiime_config)
FastTree is in path and version is supported ... ok
test_INFERNAL_supported_version (__main__.Qiime_config)
INFERNAL is in path and version is supported ... ok
test_ParsInsert_supported_version (__main__.Qiime_config)
ParsInsert is in path and version is supported ... FAIL
pplacer is in path and version is supported ... FAIL
test_pynast_suported_version (__main__.Qiime_config)
pynast version is supported ... ok
test_pynast_template_alignment_blastdb_fp (__main__.Qiime_config)
pynast_template_alignment_blastdb, if set, is set to a valid path ... ok
test_pynast_template_alignment_fp (__main__.Qiime_config)
pynast_template_alignment, if set, is set to a valid path ... ok
test_python_exe_fp (__main__.Qiime_config)
python_exe_fp is set to a working python env ... ok
test_python_supported_version (__main__.Qiime_config)
python is in path and version is supported ... ok
test_qiime_scripts_dir (__main__.Qiime_config)
qiime_scripts_dir, if set, is set to a valid path ... ok
test_qiime_test_data_dir (__main__.Qiime_config)
qiime_test_data_dir, if set, is set to a valid path ... ok
test_raxmlHPC_supported_version (__main__.Qiime_config)
raxmlHPC is in path and version is supported ... ok
test_rtax_supported_version (__main__.Qiime_config)
rtax is in path and version is supported ... FAIL
test_temp_dir (__main__.Qiime_config)
temp_dir, if set, is set to a valid path ... ok
test_template_alignment_lanemask_fp (__main__.Qiime_config)
template_alignment_lanemask, if set, is set to a valid path ... ok
test_uclust_supported_version (__main__.Qiime_config)
uclust is in path and version is supported ... ok
test_usearch_supported_version (__main__.Qiime_config)
usearch is in path and version is supported ... ok
test_working_dir (__main__.Qiime_config)
working_dir, if set, is set to a valid path ... ok

======================================================================
FAIL: test_ParsInsert_supported_version (__main__.Qiime_config)
ParsInsert is in path and version is supported
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/qiime/bin/print_qiime_config.py", line 648, in test_ParsInsert_supported_version
    "which components of QIIME you plan to use.")
AssertionError: ParsInsert not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================
FAIL: test_pplacer_supported_version (__main__.Qiime_config)
pplacer is in path and version is supported
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/qiime/bin/print_qiime_config.py", line 627, in test_pplacer_supported_version
    "which components of QIIME you plan to use.")
AssertionError: pplacer not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================
FAIL: test_rtax_supported_version (__main__.Qiime_config)
rtax is in path and version is supported
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/qiime/bin/print_qiime_config.py", line 606, in test_rtax_supported_version
    "which components of QIIME you plan to use.")
AssertionError: rtax not found. This may or may not be a problem depending on which components of QIIME you plan to use.

----------------------------------------------------------------------
Ran 34 tests in 0.119s

FAILED (failures=3)


Attached is the parameters file.

I repeated the command and it looks like the same error:

bash-3.2$ pick_otus_through_otu_table.py -i /shared/users/wdemos/Qiime150_tut/qiime_tutorial/split_library_output/seqs.fna -o /shared/users/wdemos/Qiime150_tut/qiime_tutorial/otus -p /shared/users/wdemos/Qiime150_tut/qiime_tutorial/qiime_parameters.txt -f
Traceback (most recent call last):
  File "/usr/local/qiime/bin/pick_otus_through_otu_table.py", line 132, in <module>
    main()    
  File "/usr/local/qiime/bin/pick_otus_through_otu_table.py", line 129, in main
    status_update_callback=status_update_callback)
  File "/usr/local/qiime/lib/qiime/workflow.py", line 471, in run_pick_otus_through_otu_table
    close_logger_on_success=close_logger_on_success)
  File "/usr/local/qiime/lib/qiime/workflow.py", line 135, in call_commands_serially
    raise WorkflowError, msg
qiime.workflow.WorkflowError: 

*** ERROR RAISED DURING STEP: Assign taxonomy
Command run was:
 python /usr/local/qiime/scripts/assign_taxonomy.py -o /shared/users/wdemos/Qiime150_tut/qiime_tutorial/otus/rdp_assigned_taxonomy -i /shared/users/wdemos/Qiime150_tut/qiime_tutorial/otus/rep_set//seqs_rep_set.fasta --confidence 0.8 --rdp_max_memory 4000 --assignment_method rdp --reference_seqs_fp /data/greengenes/gg_12_10_otus/rep_set/97_otus.fasta --id_to_taxonomy_fp /data/greengenes/gg_12_10_otus/taxonomy/97_otu_taxonomy.rdp22_train.txt --e_value 0.001
Command returned exit status: 1
Stdout:

Stderr
Traceback (most recent call last):
  File "/usr/local/qiime/scripts/assign_taxonomy.py", line 226, in <module>
    main()
  File "/usr/local/qiime/scripts/assign_taxonomy.py", line 222, in main
    result_path=result_path,log_path=log_path)
  File "/usr/local/qiime/lib/qiime/assign_taxonomy.py", line 364, in __call__
    max_memory=max_memory)
  File "/usr/local/qiime/lib/qiime/pycogent_backports/rdp_classifier.py", line 498, in train_rdp_classifier_and_assign_taxonomy
    max_memory=max_memory)
  File "/usr/local/qiime/lib/qiime/pycogent_backports/rdp_classifier.py", line 402, in assign_taxonomy
    app_result = app(data)
  File "/usr/local/python271/lib/python2.7/site-packages/cogent/app/util.py", line 250, in __call__
    % (str(exit_status),command)
cogent.app.util.ApplicationError: Unacceptable application exit status: 1, command: cd "/usr/local/Qiime-1.5.0/"; java -Xmx4000M -jar "/usr/local/bin/rdp_classifier-2.2.jar" -t "/tmp/RdpTrainer_dGQwJV/RdpClassifier.properties" -o "/tmp/RdpAssignments_z1hyec.txt" -f fixrank -q "/tmp/tmp0prxlHS7zwuBbsG3ThFv.txt" > "/tmp/tmpXHnF1R0BRTmiYulgieex.txt" 2> "/tmp/tmp3x4HEpw6daklErP2Qoo1.txt"

Daniel McDonald

unread,
Nov 28, 2012, 11:55:09 AM11/28/12
to qiime...@googlegroups.com
Thanks, sorry that didn't work. Can you send the version of Java used
(java -showversion)?

THanks
-Daniel
> --
>
>
>

Wendy Demos

unread,
Nov 28, 2012, 12:30:20 PM11/28/12
to qiime...@googlegroups.com
No worries Daniel, thank you for your help and patience with me through this process. Here is the output from the java command:

bash-3.2$ java -showversion
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)

Usage: java [-options] class [args...]
           (to execute a class)
   or  java [-options] -jar jarfile [args...]
           (to execute a jar file)

where options include:
    -d32          use a 32-bit data model if available

    -d64          use a 64-bit data model if available
    -server       to select the "server" VM
                  The default VM is server.
                  
    -cp <class search path of directories and zip/jar files>
    -classpath <class search path of directories and zip/jar files>
                  A : separated list of directories, JAR archives,
                  and ZIP archives to search for class files.
    -D<name>=<value>
                  set a system property
    -verbose[:class|gc|jni]
                  enable verbose output
    -version      print product version and exit
    -version:<value>
                  require the specified version to run
    -showversion  print product version and continue
    -jre-restrict-search | -jre-no-restrict-search
                  include/exclude user private JREs in the version search
    -? -help      print this help message
    -X            print help on non-standard options
    -ea[:<packagename>...|:<classname>]
    -enableassertions[:<packagename>...|:<classname>]
                  enable assertions
    -da[:<packagename>...|:<classname>]
    -disableassertions[:<packagename>...|:<classname>]
                  disable assertions
    -esa | -enablesystemassertions
                  enable system assertions
    -dsa | -disablesystemassertions
                  disable system assertions
    -agentlib:<libname>[=<options>]
                  load native agent library <libname>, e.g. -agentlib:hprof
                    see also, -agentlib:jdwp=help and -agentlib:hprof=help
    -agentpath:<pathname>[=<options>]
                  load native agent library by full pathname
    -javaagent:<jarpath>[=<options>]
                  load Java programming language agent, see java.lang.instrument
    -splash:<imagepath>
                  show splash screen with specified image

Daniel McDonald

unread,
Nov 28, 2012, 4:38:15 PM11/28/12
to qiime...@googlegroups.com
Hey Wendy,

We're discussing this internally. We'll keep you posted
-Daniel
> --
>
>
>

Wendy Demos

unread,
Nov 28, 2012, 5:05:04 PM11/28/12
to qiime...@googlegroups.com
Thank you.

Wendy Demos

unread,
Nov 30, 2012, 10:35:57 AM11/30/12
to qiime...@googlegroups.com
Hi Daniel,
 I am wondering if you would please include a second email address on the updates on this thread. I am 37 weeks pregnant and trying to get this up and running before I end up on maternity leave but just in case it doesn't happen, I need to have our intern get it going in my absence. His name is Ibrahim and email is: ivazi...@mcw.edu Thank you so much for your help with resolving this.

Wendy

>
> --
>
>
>

--






--

"Failure is not falling down but refusing to get up" - Chinese proverb


Wendy Demos

unread,
Nov 30, 2012, 10:37:04 AM11/30/12
to qiime...@googlegroups.com
Hi Daniel,
 I am wondering if you would please include a second email address on the updates on this thread. I am 37 weeks pregnant and trying to get this up and running before I end up on maternity leave but just in case it doesn't happen, I need to have our intern get it going in my absence.I have included it in a response to the last email from you in my inbox so it is not published on the forum. Thank you so much for your help with resolving this.

Wendy

Daniel McDonald

unread,
Nov 30, 2012, 10:39:38 AM11/30/12
to qiime...@googlegroups.com
I will of course! Congrats and hope everything goes well!
-Daniel
> --
>
>
>

Wendy Demos

unread,
Nov 30, 2012, 10:59:12 AM11/30/12
to qiime...@googlegroups.com
Thank you.

Daniel McDonald

unread,
Dec 3, 2012, 1:07:30 PM12/3/12
to qiime...@googlegroups.com
Hey Wendy,

Can you try running assign_taxonomy.py with the attached files?

Thanks,
Daniel
> --
>
>
>
seqs_rdp22_assigned_taxonomy.tgz
seqs.fna

Daniel McDonald

unread,
Dec 4, 2012, 3:25:25 PM12/4/12
to qiime...@googlegroups.com, ivazi...@mcw.edu
Ibrahim,

Sorry, I forgot to cc you when responding to Wendy the other day.
Below is the email,

Thanks,
Daniel
seqs_rdp22_assigned_taxonomy.tgz
seqs.fna

Ibrahim Vazirabad

unread,
Dec 5, 2012, 12:05:50 PM12/5/12
to qiime...@googlegroups.com, ivazi...@mcw.edu, was...@gmail.com, wde...@mcw.edu
Hello,

No problem. I am following this thread manually as well. 

I unpacked the .tgz file and downloaded the FASTA file, which is the one I am using.

I am running this command: 

$PWD/qiime/bin/assign_taxonomy.py -i $PWD/seqs.fna -t $PWD/rdp22_assigned_taxonomy/seqs_tax_assignments.txt -r /data/greengenes/gg_12_10_otus/taxonomy/97_otu_taxonomy.rdp22_train.txt

where $PWD is: /shared/users/ivazirabad

My output is hundreds of lines of bacterial names, which are identical to the rdp22_train.txt file. What should my parameters be? I have tried other combinations and this command is this only one that does not crash.

Respectfully,
Ibrahim Vazirabad

Ibrahim Vazirabad

unread,
Dec 5, 2012, 1:12:55 PM12/5/12
to qiime...@googlegroups.com, was...@gmail.com, wde...@mcw.edu
I have included a sample of what I see in the terminal. It appears to be a verbatim copy of the rdp22_train.txt file that I am sending the script.

-Ibrahim

tRoot; k__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Porphyromonadaceae', '538008\tRoot; k__Bacteria; p__SAR406; c__AB16; o__SSW63Au; f__SHAS460', '141806\tRoot; k__Bacteria; p__Spirochaetes; c__Spirochaetes; o__Spirochaetales; f__Spirochaetaceae', '9565\tRoot; k__Bacteria; p__Proteobacteria; c__Gammaproteobacteria; o__Pasteurellales; f__Pasteurellaceae', '232037\tRoot; k__Bacteria; p__Firmicutes; c__Clostridia; o__; f__', '105331\tRoot; k__Bacteria; p__Bacteroidetes; c__Flavobacteriia; o__Flavobacteriales; f__Flavobacteriaceae', '537662\tRoot; k__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__', '3984526\tRoot; k__Bacteria; p__Proteobacteria; c__Deltaproteobacteria; o__Desulfovibrionales; f__', '256676\tRoot; k__Bacteria; p__Proteobacteria; c__Deltaproteobacteria; o__Syntrophobacterales; f__Syntrophaceae', '189860\tRoot; k__Bacteria; p__Firmicutes; c__Clostridia; o__Clostridiales; f__Lachnospiraceae', '170504\tRoot; k__Bacteria; p__Firmicutes; c__Clostridia; o__; f__', '1110902\tRoot; k__Bacteria; p__Firmicutes; c__; o__; f__', '349786\tRoot; k__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Prevotellaceae', '746721\tRoot; k__Bacteria; p__Bacteroidetes; c__Flavobacteriia; o__Flavobacteriales; f__Flavobacteriaceae', '559451\tRoot; k__Bacteria; p__Proteobacteria; c__Deltaproteobacteria; o__Desulfovibrionales; f__Desulfovibrionaceae', '331763\tRoot; k__Bacteria; p__Firmicutes; c__Clostridia; o__Clostridiales; f__Lachnospiraceae', '639466\tRoot; k__Bacteria; p__Proteobacteria; c__Gammaproteobacteria; o__Oceanospirillales; f__Oceanospirillaceae', '1107619\tRoot; k__Bacteria; p__Acidobacteria; c__Acidobacteria; o__Acidobacteriales; f__Acidobacteriaceae', '89666\tRoot; k__Bacteria; p__Proteobacteria; c__Betaproteobacteria; o__Burkholderiales; f__Comamonadaceae', '14393\tRoot; k__Bacteria; p__Firmicutes; c__Bacilli; o__Bacillales; f__Alicyclobacillaceae', '1147375\tRoot; k__Bacteria; p__Planctomycetes; c__Planctomycetia; o__Pirellulales; f__Pirellulaceae', '140315\tRoot; k__Bacteria; p__Actinobacteria; c__Actinobacteria; o__Actinomycetales; f__Microbacteriaceae', '257082\tRoot; k__Bacteria; p__ZB3; c__; o__; f__', '343874\tRoot; k__Archaea; p__Euryarchaeota; c__[Parvarchaea]; o__YLA114; f__', '1128757\tRoot; k__Bacteria; p__Actinobacteria; c__Actinobacteria; o__Actinomycetales; f__Nocardioidaceae', '2645015\tRoot; k__Bacteria; p__Bacteroidetes; c__Flavobacteriia; o__Flavobacteriales; f__Flavobacteriaceae', '617073\tRoot; k__Archaea; p__Euryarchaeota; c__Methanomicrobia; o__Methanosarcinales; f__Methanosaetaceae', '113997\tRoot; k__Bacteria; p__Actinobacteria; c__Thermoleophilia; o__Solirubrobacterales; f__', '194304\tRoot; k__Bacteria; p__Cyanobacteria; c__Oscillatoriophycideae; o__Chroococcales; f__Xenococcaceae', '760154\tRoot; k__Bacteria; p__Firmicutes; c__Bacilli; o__Bacillales; f__', '338279\tRoot; k__Bacteria; p__Acidobacteria; c__RB25; o__; f__', '316970\tRoot; k__Bacteria; p__WS6; c__SC72; o__MAT-CR-H2-G03; f__', '1720849\tRoot; k__Bacteria; p__BHI80-139; c__; o__; f__', '2495604\tRoot; k__Bacteria; p__Chloroflexi; c__Anaerolineae; o__OPB11; f__', '572670\tRoot; k__Bacteria; p__Proteobacteria; c__Alphaproteobacteria; o__Rhodobacterales; f__Rhodobacteraceae', '246373\tRoot; k__Bacteria; p__Firmicutes; c__Clostridia; o__MBA08; f__', '736040\tRoot; k__Bacteria; p__Proteobacteria; c__Deltaproteobacteria; o__[Entotheonellales]; f__[Entotheonellaceae]', '328523\tRoot; k__Bacteria; p__Firmicutes; c__Clostridia; o__Coriobacteriales; f__', '107241\tRoot; k__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__BS11', '245548\tRoot; k__Bacteria; p__Proteobacteria; c__Deltaproteobacteria; o__Syntrophobacterales; f__Syntrophobacteraceae', '227059\tRoot; k__Bacteria; p__Cyanobacteria; c__Oscillatoriophycideae; o__Chroococcales; f__Xenococcaceae', '3348556\tRoot; k__Bacteria; p__Bacteroidetes; c__Sphingobacteriia; o__Sphingobacteriales; f__Cyclobacteriaceae', '369390\tRoot; k__Bacteria; p__Firmicutes; c__Clostridia; o__Clostridiales; f__Clostridiaceae', '1124746\tRoot; k__Bacteria; p__OD1; c__ZB2; o__; f__']

Antonio González Peña

unread,
Dec 5, 2012, 5:03:02 PM12/5/12
to Qiime Forum, Daniel McDonald, wde...@mcw.edu
Not sure if this is the case but it sounds like this is a problem with
terminology. We have used RDP for a while now; at first we used their
algorithms and taxonomy classification and now we use their algorithms
but retrain them with the GG taxonomy. This retraining is done by
_default_ in the last few releases of QIIME. Could it be the case that
your two outputs look the same because we are retraining by default
and then you are doing the same with your command? Just want to be
sure that we are not missing something. If you want to check what are
the defaults run in the terminal: assing_taxonomy.py -h and/or
print_qiime_config.py -t
> ...
>
> [Message clipped]



--
Antonio

Wendy Demos

unread,
Dec 10, 2012, 11:41:21 AM12/10/12
to qiime...@googlegroups.com, Daniel McDonald, wde...@mcw.edu
Hi Antonio,
 I'm not sure I follow. I ran the qiime_config command. This is the output:
Ran 34 tests in 1.711s

FAILED (failures=3)
 Can you tell from this info if we are running into what you are describing? Thank you.

Wendy

Wendy Demos

unread,
Dec 10, 2012, 11:43:54 AM12/10/12
to qiime...@googlegroups.com, Daniel McDonald, wde...@mcw.edu
Also could my customized qiime_paramters file be contributing to the issue? Thanks.
# qiime_parameters.txt
# WARNING: DO NOT EDIT OR DELETE Qiime/qiime_parameters.txt. Users should copy this file and edit copies of it.

# split libraries parameters
split_libraries:min-seq-length
split_libraries:max-seq-length
split_libraries:trim-seq-length
split_libraries:min-qual-score
split_libraries:keep-primer
split_libraries:keep-barcode
split_libraries:max-ambig
split_libraries:max-homopolymer
split_libraries:max-primer-mismatch
split_libraries:barcode-type
split_libraries:max-barcode-errors
split_libraries:start-numbering-at
split_libraries:remove_unassigned True
split_libraries:disable_bc_correction
split_libraries:qual_score_window
split_libraries:discard_bad_windows
split_libraries:disable_primers
split_libraries:reverse_primers
split_libraries:record_qual_scores

# OTU picker parameters
pick_otus:otu_picking_method uclust
pick_otus:clustering_algorithm furthest
pick_otus:max_cdhit_memory 400
pick_otus:refseqs_fp /data/greengenes/gg_12_10_otus/rep_set/97_otus.fasta
pick_otus:blast_db
pick_otus:similarity 0.97
pick_otus:max_e_value 1e-10
pick_otus:prefix_prefilter_length
pick_otus:trie_prefilter
pick_otus:prefix_length
pick_otus:suffix_length
pick_otus:optimal_uclust
pick_otus:exact_uclust
pick_otus:user_sort
pick_otus:suppress_presort_by_abundance_uclust
pick_otus:suppress_new_clusters
pick_otus:suppress_uclust_stable_sort
pick_otus:max_accepts
pick_otus:max_rejects
pick_otus:word_length
pick_otus:stepwords
pick_otus:suppress_uclust_prefilter_exact_match

# Parallel options
parallel:jobs_to_start 1
parallel:retain_temp_files False
parallel:seconds_to_sleep 60

# Representative set picker parameters
pick_rep_set:rep_set_picking_method first
pick_rep_set:sort_by otu

# Multiple sequence alignment parameters
align_seqs:template_fp /data/greengenes/core_set_aligned.fasta.imputed
align_seqs:alignment_method pynast
align_seqs:pairwise_alignment_method uclust
align_seqs:blast_db
align_seqs:min_length 150
align_seqs:min_percent_id 75.0

# Alignment filtering (prior to tree-building) parameters
filter_alignment:lane_mask_fp /data/greengenes/lanemask_in_1s_and_0s
filter_alignment:allowed_gap_frac 0.999999
filter_alignment:remove_outliers False
filter_alignment:threshold 3.0

# Taxonomy assignment parameters
assign_taxonomy:id_to_taxonomy_fp /data/greengenes/gg_12_10_otus/taxonomy/97_otu_taxonomy.rdp22_train.txt
assign_taxonomy:reference_seqs_fp /data/greengenes/gg_12_10_otus/rep_set/97_otus.fasta
assign_taxonomy:assignment_method rdp
assign_taxonomy:blast_db
assign_taxonomy:confidence 0.8
assign_taxonomy:e_value 0.001
assign_taxonomy:rdp_max_memory 4000

# Phylogenetic tree building parameters
make_phylogeny:tree_method fasttree
make_phylogeny:root_method tree_method_default

# Beta diversity parameters
beta_diversity:metrics weighted_unifrac,unweighted_unifrac

# Make 3D plot parameters
make_3d_plots:custom_axes
make_3d_plots:ellipsoid_smoothness   1

# Rarefaction parameters
multiple_rarefactions:num-reps 10
multiple_rarefactions:depth
multiple_rarefactions:lineages_included False

# Even-depth rarefaction parameters
multiple_rarefactions_even_depth:num-reps 100

# Alpha diversity parameters
alpha_diversity:metrics PD_whole_tree,chao1,observed_species

# Make rarefaction plots parameters
make_rarefaction_plots:imagetype png
make_rarefaction_plots:resolution 75
make_rarefaction_plots:background_color white
make_rarefaction_plots:prefs_path

# Collate alpha
collate_alpha:example_path

# OTU category significance
otu_category_significance:test
otu_category_significance:filter 3
otu_category_significance:threshold
otu_category_significance:otu_include_fp /data/greengenes/gg_12_10_otus/97_otu_map.txt

Antonio González Peña

unread,
Dec 10, 2012, 3:56:00 PM12/10/12
to Qiime Forum, Daniel McDonald, wde...@mcw.edu
From your configuration I can say that the configuration seems fine
and that you are using the new greengenes. Also that it sounds like
everything is working fine, the output of assign_taxonomy.py is a text
file with 3 columns: id of the sequence, taxonomy, and %id; which is
similar to the file that Daniel sent. With this file you should use
add_taxa.py with your OTU table (biom file) to incorporate this new
taxa to your file. In summary: I'm not sure what is wrong as
everything seems fine, maybe the error is deep in the thread and I'm
just getting lost. If this is the case, please, let me know.

Antonio González Peña

unread,
Dec 10, 2012, 5:23:26 PM12/10/12
to Qiime Forum, Daniel McDonald, Demos, Wendy
After my last email we moved the conversation out of the forum to
avoid more emails in this thread. The good news is that we figured out
the problem: the installation of RDP was incomplete. To install RDP
you need to put all the files in the package you download in the same
folder where rdp_classifier-2.2.jar is, this include the src and lib
folders.

Cheers.
--
Antonio
Reply all
Reply to author
Forward
0 new messages