Fungal analysis

549 views
Skip to first unread message

ubuntu

unread,
Nov 28, 2011, 5:33:51 PM11/28/11
to Qiime Forum
Hello Qiime users,

Does anyone use QIIME for fungal data analysis (not bacteria)? ITS
database? How to set a qiime configure file for ITS_reference
database?

Thanks,
Ng

TonyWalters

unread,
Nov 28, 2011, 5:53:26 PM11/28/11
to Qiime Forum
Hello Ng,

There is a QIIME compatible UNITE taxonomy mapping and reference
sequence file available here:
http://qiime.org/home_static/dataFiles.html

To modify your reference and id to taxonomy mapping file paths, you
can modify the qiime_config file. Instructions are found here:
http://qiime.org/install/qiime_config.html

Hope this helps,
Tony Walters

ubuntu

unread,
Nov 28, 2011, 6:31:46 PM11/28/11
to Qiime Forum
Hi Tony,

Thanks for promptly response
The ITS data set looks good, hopefully its up-to-date. I downloaded
the ITS data on my local home (~/BLAST/ITS/its_qiime.fasta) and the
qiime_config is located @ ~/.qiime_configure. I don't know how to
modify the 'qiime_config' correctly.


Here is my current environment:
[ng@csclp1 ~]$ print_qiime_config.py -t

System information
==================
Platform: linux2
Python version: 2.6.5 (r265:79063, Feb 28 2011, 21:55:45) [GCC
4.1.2 20080704 (Red Hat 4.1.2-50)]
Python executable: /usr/bin/python26

Dependency versions
===================
PyCogent version: 1.5.1
NumPy version: 1.3.0
matplotlib version: 0.98.5.3
QIIME library version: 1.3.0
QIIME script version: 1.3.0
PyNAST version (if installed): 1.1
RDP Classifier version (if installed): rdp_classifier-2.2.jar

QIIME config values
===================
blastmat_dir: None
topiaryexplorer_project_dir: None
pynast_template_alignment_fp: /stf/home/cluster/GenomicsCore/apps/
qiime/dependencies/core_set_aligned.fasta.imputed
cluster_jobs_fp: /stf/home/cluster/GenomicsCore/apps/
qiime/bin/start_parallel_jobs_sge.py
pynast_template_alignment_blastdb: None
torque_queue: all.q
template_alignment_lanemask_fp: /stf/home/cluster/GenomicsCore/apps/
qiime/dependencies/lanemask_in_1s_and_0s
jobs_to_start: 2
cloud_environment: False
qiime_scripts_dir: /stf/home/cluster/GenomicsCore/apps/
qiime/bin
denoiser_min_per_core: 50
working_dir: None
python_exe_fp: /usr/bin/python26
temp_dir: /tmp/
blastall_fp: /stf/home/cluster/GenomicsCore/apps/
qiime/dependencies/blast-2.2.22/bin/blastall
seconds_to_sleep: 60


running checks:

FastTree is in path and version is supported ... ok
INFERNAL is in path and version is supported ... ok
AmpliconNoise install looks sane. ... ok
blast is in path and version is supported ... ok
blastall_fp is set to a valid path ... ok
blastmat_dir is set to a valid path. ... ok
cdbtools is in path and version is supported ... ok
cd-hit is in path and version is supported ... ok
no obvious problems with ChimeraSlayer install ... ok
clearcut is in path and version is supported ... ok
cluster_jobs_fp is set to a valid path and is executable ... ok
denoiser aligner is ready to use ... ok
local qiime_config has no extra params ... ok
maptplotlib version is supported ... ok
mothur is in path and version is supported ... ok
muscle is in path and version is supported ... ok
numpy version is supported ... ok
pynast version is supported ... ok
pynast_template_alignment_blastdb, if set, is set to a valid path ...
ok
pynast_template_alignment, if set, is set to a valid path ... ok
python_exe_fp is set to a working python env ... ok
python is in path and version is supported ... ok
qiime_scripts_dir, if set, is set to a valid path ... ok
raxmlHPC is in path and version is supported ... ok
temp_dir, if set, is set to a valid path ... ok
template_alignment_lanemask, if set, is set to a valid path ... ok
uclust is in path and version is supported ... ok
working_dir, if set, is set to a valid path ... ok


Thanks,
Ng

Greg Caporaso

unread,
Nov 28, 2011, 7:39:26 PM11/28/11
to qiime...@googlegroups.com
Hello Ng,
In QIIME 1.3.0 you cannot store the taxonomy assignment reference
files in .qiime_config, although that feature has been added to the
development version and will be in the next release. In the meantime,
you would pass the reference sequences and taxonomy map to
assign_taxonomy.py (or one of the parallel versions) via the -r and -t
parameters. This tutorial shows how to do it (but with different
reference files):

http://qiime.org/svn_documentation/tutorials/retraining_rdp.html

Greg

ubuntu

unread,
Nov 28, 2011, 8:02:01 PM11/28/11
to Qiime Forum
Hello Greg,

on my first-step: Pick_OTUs - I try to use both BLAST (100% match) and
UCLUST

#BLAST
pick_otus.py -i split_library_output/seqs.fna -m blast -s 1.0 -r
BLAST/ITS/its_qiime.fa -o otus/blast_picked_otus

#UCLUST
pick_otus.py -i split_library_output/seqs.fna -o otus/
uclust_picked_otus

Does my 'pick_otus.py' command with BLAST/UCLUST correct?

Thanks,
Ng

On Nov 28, 4:39 pm, Greg Caporaso <gregcapor...@gmail.com> wrote:
> Hello Ng,
> In QIIME 1.3.0 you cannot store the taxonomy assignment reference
> files in .qiime_config, although that feature has been added to the
> development version and will be in the next release. In the meantime,
> you would pass the reference sequences and taxonomy map to
> assign_taxonomy.py (or one of the parallel versions) via the -r and -t
> parameters. This tutorial shows how to do it (but with different
> reference files):
>
> http://qiime.org/svn_documentation/tutorials/retraining_rdp.html
>
> Greg
>

Greg Caporaso

unread,
Nov 28, 2011, 8:04:37 PM11/28/11
to qiime...@googlegroups.com
That looks correct - this will do reference-based BLAST OTU picking
and de novo uclust OTU picking.

Greg

ubuntu

unread,
Nov 28, 2011, 8:20:44 PM11/28/11
to Qiime Forum
Hi Greg,

Would it be possible to run 'uclust' OTU picking with a reference-
base? How does a uclust work with de-noval and reference ?

Thanks,
Ng


On Nov 28, 5:04 pm, Greg Caporaso <gregcapor...@gmail.com> wrote:
> That looks correct - this will do reference-based BLAST OTU picking
> and de novo uclust OTU picking.
>
> Greg
>

Greg Caporaso

unread,
Nov 28, 2011, 8:39:12 PM11/28/11
to qiime...@googlegroups.com
Ng,
You have three options with uclust:

de novo otu picking (default, you're already doing this)
closed reference otu picking (pass -r, -C, and -m uclust_ref)
open reference otu picking (pass -r and -m uclust_ref)

The difference between open and closed reference OTU picking is in how
the sequences which don't hit the reference are handled. In closed
reference OTU picking, sequences which don't hit the reference set are
discarded (as with the BLAST OTU picker). In open reference OTU
picking, sequences which don't hit the reference set are clustered de
novo.

Greg

ubuntu

unread,
Nov 29, 2011, 2:50:11 AM11/29/11
to Qiime Forum
Hi Greg,

It works great for me so far.
How to run 'pick_otus.pl' in parallel?
#pick_otus.py -i split_library_output/seqs.fna -m blast -s 1.0 -r
BLAST/ITS/its_qiime.fa -o otus/uclust_picked_otus


Thanks,
Ng

Greg Caporaso

unread,
Nov 29, 2011, 9:08:38 AM11/29/11
to qiime...@googlegroups.com
You can use parallel_pick_otus_uclust_ref.py or parallel_pick_otus_blast.py.

See here for details on setting up parallel QIIME:

http://qiime.org/tutorials/parallel_qiime.html

ubuntu

unread,
Nov 29, 2011, 4:05:39 PM11/29/11
to Qiime Forum
Hi Greg,

The parallel works on our cluster.

>parallel_pick_otus_blast.py -i split_library_output/seqs_all.fna -r BLAST/ITS/its_qiime.fa -s 1.0 -o otus_para/blast_100_otus -T --jobs_to_start 10

However, the results are not merge. Do I have to merge all reuslt
files manually by the end?

...
rw-r--r-- 1 genomics genomics 109K Nov 29 12:49 POTU_dgD0_.0_otus.txt
-rw-r--r-- 1 genomics genomics 297K Nov 29 12:49 POTU_dgD0_.0_otus.log
-rw-r--r-- 1 genomics genomics 121K Nov 29 12:50 POTU_dgD0_.5_otus.txt
-rw-r--r-- 1 genomics genomics 297K Nov 29 12:50 POTU_dgD0_.5_otus.log
drwxr-xr-x 2 genomics genomics 106 Nov 29 12:50 POTU_dgD0_
[genomics@csclp blast_100_otus]$ ls -lrth POTU_dgD0_
total 77K
-rw-r--r-- 1 genomics genomics 939 Nov 29 09:10
expected_out_files.txt
-rw-r--r-- 1 genomics genomics 1.1K Nov 29 09:10 merge_map.txt
-rw-r--r-- 1 genomics genomics 1.8K Nov 29 09:10 deletion_list.txt


Thanks,
Ng


On Nov 29, 6:08 am, Greg Caporaso <gregcapor...@gmail.com> wrote:
> You can use parallel_pick_otus_uclust_ref.py or parallel_pick_otus_blast.py.
>
> See here for details on setting up parallel QIIME:
>
> http://qiime.org/tutorials/parallel_qiime.html
>

Jose Carlos Clemente

unread,
Nov 29, 2011, 4:40:15 PM11/29/11
to qiime...@googlegroups.com
Hi Ng,
it looks like your parallel job did not complete properly, thus the
results are not merged. Can you check whether your main job finished
correctly?
Jose

ubuntu

unread,
Nov 29, 2011, 7:07:34 PM11/29/11
to Qiime Forum
Hi,

The jobs were completed, result files are not merged. There wasn't
error files. I just 'cat' all file together.

I have another questions:

The representative sequence appear multiple times (6 times) @
otus_para/rep_set/seqs_rep_100.fasta . Should it be one represent
sequence per OTU? I may miss some thing here ?
[genomics@csclp1 fungal]$ grep 'AB019340' otus_para/rep_set/
seqs_rep_100.fasta
>AB019340 mid3_run2_10496
>AB019340 mid3_run2_30234
>AB019340 mid2_run3_17310
>AB019340 mid2_run3_47837
>AB019340 mid2_run3_101673
>AB019340 mid2_run3_135259

[genomics@csclp1 fungal]$ less otus_para/otus_table_gg_100.txt
# QIIME v1.3.0 OTU table
#OTU ID mid1_run2 mid1_run3 mid2_run2
mid2_run3 mid3_run2 mid3_run3 mid4_run2
mid4_run3 Consensus Lineage
AB019340 0 0 0 2 0 0
0 0 Root: Fungal
AB085802 0 0 0 0 0 0
0 4 Root;k__Bacteria
AB105432 0 0 0 17 0 0
0 5 Root

The 'AB019340' OTU has 2 hits on sample 'mid2_run3', which is not
match with number of sequence above.

Would you plain relationships between two files above?

Thanks,
Ng

On Nov 29, 1:40 pm, Jose Carlos Clemente <jose.cleme...@gmail.com>
wrote:


> Hi Ng,
> it looks like your parallel job did not complete properly, thus the
> results are not merged. Can you check whether your main job finished
> correctly?
> Jose
>

> ...
>
> read more »

Greg Caporaso

unread,
Nov 29, 2011, 8:57:59 PM11/29/11
to qiime...@googlegroups.com
Hello Ng,
You cannot just cat those files together at the end - the issue that
you report is exactly what I would expect. Do you have the output
generated by the initial command that you ran?

>parallel_pick_otus_blast.py -i split_library_output/seqs_all.fna -r BLAST/ITS/its_qiime.fa -s 1.0  -o otus_para/blast_100_otus -T --jobs_to_start 10

One potential issue is that you need to use full paths when calling
this script. If you don't have the output from the original (or if it
never actually completed) can you please rerun using full paths for
the -i, -r, and -o options.

Greg

ubuntu

unread,
Nov 30, 2011, 6:43:04 PM11/30/11
to Qiime Forum
Hi Greg,

Awesome! The absolute path works. Thank you for making the parallel
option, which save us lot of time.

Thanks,
Ng

ubuntu

unread,
Nov 30, 2011, 7:48:07 PM11/30/11
to Qiime Forum
Hi Greg,

I have no-match on the 'Align OTU Sequences'.

>parallel_align_seqs_pynast.py -i otus/rep_set/seqs_set.fasta -d BLAST/ITS/its_qiime.fa -o otus/pynast_aligned_seqs -T --jobs_to_star 2 &
OR
>parallel_align_seqs_pynast.py -i otus/rep_set/seqs_set.fasta -o otus/pynast_aligned_seqs -T --jobs_to_star 2 &

[genomics@csclp1 fungal]$ ls -lrth otus/pynast_aligned_seqs/
total 245K
-rw-r--r-- 1 genomics genomics 0 Nov 30 16:13 seqs_set_aligned.fasta
-rw-r--r-- 1 genomics genomics 59K Nov 30 16:13
seqs_set_failures.fasta
-rw-r--r-- 1 genomics genomics 12K Nov 30 16:13 seqs_set_log.txt

I used a reference-base (..BLAST/ITS/its_qiime.fa) picking OTUs, I
confuse the 'Align OTU sequence' steps, and why I don't have aligned
sequence?

Thanks,
Ng

Greg Caporaso

unread,
Nov 30, 2011, 7:57:58 PM11/30/11
to qiime...@googlegroups.com
Are your sequences 16S or ITS? The default reference alignment is a
16S alignment, so your ITS sequences shouldn't hit that. If you don't
have an ITS reference alignment (I don't know of one) you'll need to
use "align_seqs.py -m muscle" rather than the default of PyNAST. There
is no parallel option for muscle.

Good luck!
Greg

ubuntu

unread,
Dec 1, 2011, 1:34:06 PM12/1/11
to Qiime Forum
Hi Greg,

I've tried uclust to compare results with blast

>parallel_pick_otus_blast.py -i split_library_output/seqs_all.fna -r BLAST/ITS/its_qiime.fa -s 1.0 -o otus_para/uclust_100_otus -T --jobs_to_start 10 & <-- for 100% matching ?
OK

>parallel_pick_otus_blast.py -i split_library_output/seqs_all.fna -r BLAST/ITS/its_qiime.fa -s 1.0 -m -C -o otus_para/uclust_100_otus -T --jobs_to_start 10 &
parallel_pick_otus_blast.py: error: no such option: -m, -C

How to select option for open/close reference otu_picking on UCLUST?
Does BLAST have open/close reference?

Thanks,
Ng

On Nov 28, 5:39 pm, Greg Caporaso <gregcapor...@gmail.com> wrote:
> Ng,
> You have three options with uclust:
>
> de novo otu picking (default, you're already doing this)
> closed reference otu picking (pass -r, -C, and -m uclust_ref)
> open reference otu picking (pass -r and -m uclust_ref)
>
> The difference between open and closed reference OTU picking is in how
> the sequences which don't hit the reference are handled. In closed
> reference OTU picking, sequences which don't hit the reference set are
> discarded (as with the BLAST OTU picker). In open reference OTU
> picking, sequences which don't hit the reference set are clustered de
> novo.
>
> Greg
>

ubuntu

unread,
Dec 1, 2011, 1:38:44 PM12/1/11
to Qiime Forum
Hi Greg,

I am using ITS, so I have to use '-m muscle'
>filter_alignment.py -o otus_para2/muscle_alignment/ -i otus_para2/muscle_alignment/seqs_set_aligned.fasta (Works for me!)

What are purposes of this step? how doe sit work? brief would be ok.

Thanks,
Ng

On Nov 30, 4:57 pm, Greg Caporaso <gregcapor...@gmail.com> wrote:
> Are your sequences 16S or ITS? The default reference alignment is a
> 16S alignment, so your ITS sequences shouldn't hit that. If you don't
> have an ITS reference alignment (I don't know of one) you'll need to
> use "align_seqs.py -m muscle" rather than the default of PyNAST. There
> is no parallel option for muscle.
>
> Good luck!
> Greg
>

ubuntu

unread,
Dec 1, 2011, 1:43:58 PM12/1/11
to Qiime Forum
Hi Greg,

Pl regardless my most previous email. What I mean is:

> align_seqs.py -i otus_para2/rep_set/seqs_set.fasta -m muscle -d BLAST/ITS/its_qiime.fa -o otus_para2/muscle_alignment

What are purposes of this step? how doe sit work? brief would be ok.

Thanks,
Ng

Jose Carlos Clemente

unread,
Dec 1, 2011, 2:11:39 PM12/1/11
to qiime...@googlegroups.com
Hi NG,

BLAST only has reference OTU picking. The similarity percentage is by
default 97%, not 100%: to set it to 100%, use -s 1.

Jose

ubuntu

unread,
Dec 1, 2011, 2:25:52 PM12/1/11
to Qiime Forum
Hi Jose,

Yes, I used the '-s 1.0' option for 100% similarity. I like to test
both option blast and uclust. What are options to discard sequences
that not map the target.

>parallel_pick_otus_uclust_ref.py -i split_library_output/seqs_all.fna -r BLAST/ITS/its_qiime.fa -s 1.0 -m -C -o otus_para/uclust_100_otus -T --jobs_to_start 10 (The -m, -C option don't work)

Thanks,
Ng

On Dec 1, 11:11 am, Jose Carlos Clemente <jose.cleme...@gmail.com>
wrote:


> Hi NG,
>
> BLAST only has reference OTU picking. The similarity percentage is by
> default 97%, not 100%: to set it to 100%, use -s 1.
>
> Jose
>

Jose Carlos Clemente

unread,
Dec 1, 2011, 3:49:42 PM12/1/11
to qiime...@googlegroups.com
Hi Ng,

sorry I didn't realize you had -s already in there. Sequences that do
not match your reference set will automatically show as
seqs_all_failures.txt in your output directory.

I am not sure what are you trying to do with -m -C, those are not
valid options in either parallel_pick_otus_uclust_ref.py or
parallel_pick_otus_blast.py. If you are trying to use -C to supress
new clusters (there is such an option in pick_otus.py), you don't need
since these scripts are already doing reference based only.

Jose

ubuntu

unread,
Dec 1, 2011, 8:20:10 PM12/1/11
to Qiime Forum
Hi,

We have illumina reads (100 base long). What we like is to get only
reads that 100% match (all 100 bases matches) with target (ref
database) . Does the '-s 1.0' optin work?

>parallel_pick_otus_blast.py -i split_library_output/seqs_all.fna -r BLAST/ITS/its_qiime.fa -s 1.0 -o otus_para/uclust_100_otus -T --jobs_to_start 10 &

I've tried to verify the result (from runs above), I've blasted a
representative sequence(s) with the same ref database. I don't see
sequence aligned 100% (100% identities = 98/98 or 90/90, but not
100/100)

Do you guys have any ideas? or I miss something here?

Thanks,
Ng

On Dec 1, 12:49 pm, Jose Carlos Clemente <jose.cleme...@gmail.com>
wrote:


> Hi Ng,
>
> sorry I didn't realize you had -s already in there. Sequences that do
> not match your reference set will automatically show as
> seqs_all_failures.txt in your output directory.
>
> I am not sure what are you trying to do with -m -C, those are not
> valid options in either parallel_pick_otus_uclust_ref.py or
> parallel_pick_otus_blast.py. If you are trying to use -C to supress
> new clusters (there is such an option in pick_otus.py), you don't need
> since these scripts are already doing reference based only.
>
> Jose
>

> ...
>
> read more »

Jose Carlos Clemente

unread,
Dec 1, 2011, 8:48:33 PM12/1/11
to qiime...@googlegroups.com
Hi,

not all your sequences are going to be exactly 100 base pairs long,
you will have a distribution of lengths. 100% identity represents
perfect matches, no matter the length.

Jose

ubuntu

unread,
Dec 2, 2011, 1:33:23 PM12/2/11
to Qiime Forum
Hi,

Yes, All sequences have the same length (illumina platform). This is
sample of representative sequences (one sequence per OTU/cluster).
What I expect is each of the representative sequences has a perfect
match with the target. This is what I was setting on the
'parallel_pick_otus_blast.py' with the '-s 1.0' option.

>AF07-test
CCCATCAGGGCATGTGCACGGCCTGTCTTGACTCTTATTCATCCACCTGTGCACCTTCTGTAGTCTTCTTGGATGTGAGGAATTGCTCGTCAGGCTTTCC
>AF33-test2
AGTCGTAACAAGGTTTCCGTAGGTGAACCTGCGGAAGGATCATTACTGATTTGCTTAATTGCACCACATGTGTTTTTTATTGAACAAATTTCTTTGGTGG

Verification step: Blast these sequences
> blastall -p blastn -d BLAST/ITS/its_qiime.fa -i input -o out_blast -v 2 -b 2 &

BLAST's result (Not all)

...
Query= AF07-test
(100 letters)
Database: its_qiime.fa
30,379 sequences; 19,008,933 total letters
Searching..................................................done

Score E
Sequences producing significant alignments:
(bits) Value
AF079746
198 2e-51
EF527327
96 3e-20

>AF079746
Length = 670
Score = 198 bits (100), Expect = 2e-51
Identities = 100/100 (100%)
Strand = Plus / Plus

Query: 1
cccatcagggcatgtgcacggcctgtcttgactcttattcatccacctgtgcaccttctg 60

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Sbjct: 56
cccatcagggcatgtgcacggcctgtcttgactcttattcatccacctgtgcaccttctg 115

Query: 61 tagtcttcttggatgtgaggaattgctcgtcaggctttcc 100
||||||||||||||||||||||||||||||||||||||||
Sbjct: 116 tagtcttcttggatgtgaggaattgctcgtcaggctttcc 155

>EF527327
Length = 651
Score = 95.6 bits (48), Expect = 3e-20
Identities = 58/60 (96%), Gaps = 1/60 (1%)
Strand = Plus / Plus

Query: 8
gggcatgtgcacggcctgtcttgactcttattcatccacctgtgcaccttctgtagtctt 67
||||||||||||||||||||||||||| ||||||||||||||||||||||
|||||||||
Sbjct: 64 gggcatgtgcacggcctgtcttgactc-
tattcatccacctgtgcaccttttgtagtctt 122
....
Query= AF33-test2
(100 letters)
Database: its_qiime.fa
30,379 sequences; 19,008,933 total letters
Searching..................................................done

Score E
Sequences producing significant alignments:
(bits) Value
AF335928
168 2e-42
AJ010333
153 1e-37
>AF335928
Length = 218
Score = 168 bits (85), Expect = 2e-42
Identities = 85/85 (100%)
Strand = Plus /
Plus
Query: 16
tccgtaggtgaacctgcggaaggatcattactgatttgcttaattgcaccacatgtgttt 75

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Sbjct: 1
tccgtaggtgaacctgcggaaggatcattactgatttgcttaattgcaccacatgtgttt 60

Query: 76 tttattgaacaaatttctttggtgg 100
|||||||||||||||||||||||||
Sbjct: 61 tttattgaacaaatttctttggtgg 85

>AJ010333
Length = 464
Score = 153 bits (77), Expect = 1e-37
Identities = 77/77 (100%)
Strand = Plus / Plus

Query: 1
agtcgtaacaaggtttccgtaggtgaacctgcggaaggatcattactgatttgcttaatt 60

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Sbjct: 251
agtcgtaacaaggtttccgtaggtgaacctgcggaaggatcattactgatttgcttaatt 310


Query: 61 gcaccacatgtgttttt 77
|||||||||||||||||
Sbjct: 311 gcaccacatgtgttttt 327


1. The 'AF07-test' shows a perfect match 100/100 (Identities= 100/100
100%), which is what we like to archive.
2. The 'AF33-test2' shows a highest match only 85/100
(Identities=85/85 100%)


My question are:
1. How to set the 'parallel_pick_otus_blast.py' with options that has
'100% match' like (1)
2. What does the '-s 1.0' on 'parallel_pick_otus_blast.py' mean? (100%
similarity or identities)
3. How to get all representative sequences 100% matches to the target
like (1).

Sorry for my questions are too long. Hopefully, your answers would
help others too.

Regards,
Ng

On Dec 1, 5:48 pm, Jose Carlos Clemente <jose.cleme...@gmail.com>
wrote:


> Hi,
>
> not all your sequences are going to be exactly 100 base pairs long,
> you will have a distribution of lengths. 100% identity represents
> perfect matches, no matter the length.
>
> Jose
>

> ...
>
> read more »

Greg Caporaso

unread,
Dec 3, 2011, 9:50:01 AM12/3/11
to qiime...@googlegroups.com
Hi Ng,
The reason that you're seeing alignments to shorter fragments using
blastall (which is what QIIME is calling) is that BLAST computes a
local alignment rather than a global alignment. uclust, on the other
hand, computes global alignments. So, if the score of an alignment
drops too far (e.g., due to a stretch of mismatches) BLAST will
terminate the alignment and return the result up to the low scoring
position.

To achieve the result you're looking for you also need to pass:

--min_aligned_percent 1.0

This will discard any hits that are not to the full length read.

Greg

ubuntu

unread,
Dec 5, 2011, 5:34:44 PM12/5/11
to Qiime Forum
Hi Greg,

I may have to use uclust instead then, thank you very much for your
explanation. It is very good.
I also have another question:

If a sequence (read) has 100 match (100% similarity/identities) on
multiple places on the target (fungal ITS database). How does QIIME
select the best hit? How to get all hit with 100% matches?


Thanks,
Ng


On Dec 3, 6:50 am, Greg Caporaso <gregcapor...@gmail.com> wrote:
> Hi Ng,
> The reason that you're seeing alignments to shorter fragments using
> blastall (which is what QIIME is calling) is that BLAST computes a
> local alignment rather than a global alignment. uclust, on the other
> hand, computes global alignments. So, if the score of an alignment
> drops too far (e.g., due to a stretch of mismatches) BLAST will
> terminate the alignment and return the result up to the low scoring
> position.
>
> To achieve the result you're looking for you also need to pass:
>
>  --min_aligned_percent 1.0
>
> This will discard any hits that are not to the full length read.
>
> Greg
>

> ...
>
> read more »

Greg Caporaso

unread,
Dec 5, 2011, 8:50:33 PM12/5/11
to qiime...@googlegroups.com
Hi Ng,
pick_otus.py will choose the first that it identifies as the best hit
(so you will not get information on whether it hit more one sequence
or more than one sequence).

Greg

BB_Landa

unread,
Jan 27, 2012, 2:00:40 PM1/27/12
to Qiime Forum
Hello Greg, Hello Ng
I have followed your steps after downloading the unite_taxonomy ITS
sequences, but when performing with Blast picking method I get the
following error:

iMac-de-Blanca:~ blancalanda$ Macqiime pick_otus.py -i Piro/ITS/
split_library_output_ITS/seqs.fna -m blast -s 0.99 -r Databases/
unite_taxonomy/unite_ref_seqs_21nov2011.fasta -o Piro/ITS/OTUS/
blast_picked_otus/
Traceback (most recent call last):
File "/macqiime/QIIME/bin/pick_otus.py", line 562, in <module>
main()
File "/macqiime/QIIME/bin/pick_otus.py", line 554, in main
blast_db=opts.blast_db,refseqs_fp=opts.refseqs_fp)
File "/macqiime/lib/python2.7/site-packages/qiime/pick_otus.py",
line 188, in __call__
build_blast_db_from_fasta_path(refseqs_fp)
File "/macqiime/lib/python2.7/site-packages/cogent/app/formatdb.py",
line 116, in build_blast_db_from_fasta_path
app_result = fdb(fasta_path)
File "/macqiime/lib/python2.7/site-packages/cogent/app/util.py",
line 269, in __call__
result_paths=self._get_result_paths(data))
File "/macqiime/lib/python2.7/site-packages/cogent/app/util.py",
line 72, in __init__
raise ApplicationError, 'Could not open %s' %value.Path
cogent.app.util.ApplicationError: Could not open "Databases/
unite_taxonomy/unite_ref_seqs_21nov2011.fasta.log"
iMac-de-Blanca:~ blancalanda$

May I have some help?
Thanks
Blanca

Greg Caporaso

unread,
Jan 29, 2012, 8:50:09 PM1/29/12
to qiime...@googlegroups.com
Hi Blanca,
I'm not sure what's causing this error. Are you certain that BLAST is
installed correctly on your machine? (Can you run: blastall and
formatdb?) If everything looks OK there, can you retry your command
specifying full paths rather than relative paths (e.g.,
"/home/greg/databases" rather than "databases/")?

Greg

Blanca B. Landa

unread,
Jan 30, 2012, 12:40:37 PM1/30/12
to qiime...@googlegroups.com
Hi Greg
This is the output of blastall:

blastall 2.2.22 arguments:


-p Program Name [String]
-d Database [String]
default = nr
-i Query File [File In]
default = stdin
-e Expectation value (E) [Real]
default = 10.0
-m alignment view options:
0 = pairwise,
1 = query-anchored showing identities,
2 = query-anchored no identities,
3 = flat query-anchored, show identities,
4 = flat query-anchored, no identities,
5 = query-anchored no identities and blunt ends,
6 = flat query-anchored, no identities and blunt ends,
7 = XML Blast output,
8 = tabular,
9 tabular with comment lines
10 ASN, text
11 ASN, binary [Integer]
………..

I get an error for formatdb:
iMac-de-Blanca:~ blancalanda$ formatdb
[formatdb 2.2.22] ERROR: No database name was specified
iMac-de-Blanca:~ blancalanda$

I have tried this route:

Macqiime pick_otus.py -i Piro/ITS/
split_library_output_ITS/seqs.fna -m blast -s 0.99 -r

/home/users/blancalanda/Databases/
unite_taxonomy/unite_ref_seqs_21nov2011.fasta -o Piro/ITS/OTUS/
blast_picked_otus/
and get same error results

Blanca

-----Mensaje original-----
From: Greg Caporaso
Sent: Monday, January 30, 2012 2:50 AM
To: qiime...@googlegroups.com
Subject: Re: [qiime-forum 1.4.0] Re: Fungal analysis

Antonio González Peña

unread,
Jan 30, 2012, 1:19:02 PM1/30/12
to qiime...@googlegroups.com
Hi Blanca,

Could you run these tests and send us the output?
- Run your command:
Macqiime pick_otus.py -i Piro/ITS/split_library_output_ITS/seqs.fna


-m blast -s 0.99 -r

Databases/unite_taxonomy/unite_ref_seqs_21nov2011.fasta -o
Piro/ITS/OTUS/blast_picked_otus/
- Then run
ls -laR Piro/ITS/OTUS/blast_picked_otus/
ls -laR Databases/unite_taxonomy
df -h

Thanks

--
Antonio González Peña
Research Assistant, Knight Lab
University of Colorado at Boulder
https://chem.colorado.edu/knightgroup/

ag2l...@uco.es

unread,
Jan 30, 2012, 1:34:46 PM1/30/12
to qiime...@googlegroups.com, Antonio González Peña, blanca...@csic.es
Below the output of ecah command:

> - Run your command:
Macqiime pick_otus.py -i Piro/ITS/split_library_output_ITS/seqs.fna
-m blast -s 0.99 -r
> Databases/unite_taxonomy/unite_ref_seqs_21nov2011.fasta -o
> Piro/ITS/OTUS/blast_picked_otus/

iMac-de-Blanca:~ blancalanda$ Macqiime pick_otus.py -i

Piro/ITS/split_library_output_ITS/seqs.fna -m blast -s 0.99 -r
Databases/unite_taxonomy/unite_ref_seqs_21nov2011.fasta -o
Piro/ITS/OTUS/blast_picked_otus/

Traceback (most recent call last):
File "/macqiime/QIIME/bin/pick_otus.py", line 562, in <module>
main()
File "/macqiime/QIIME/bin/pick_otus.py", line 554, in main
blast_db=opts.blast_db,refseqs_fp=opts.refseqs_fp)
File "/macqiime/lib/python2.7/site-packages/qiime/pick_otus.py",
line 188, in __call__
build_blast_db_from_fasta_path(refseqs_fp)
File
"/macqiime/lib/python2.7/site-packages/cogent/app/formatdb.py", line
116, in build_blast_db_from_fasta_path
app_result = fdb(fasta_path)
File "/macqiime/lib/python2.7/site-packages/cogent/app/util.py",
line 269, in __call__
result_paths=self._get_result_paths(data))
File "/macqiime/lib/python2.7/site-packages/cogent/app/util.py",
line 72, in __init__
raise ApplicationError, 'Could not open %s' %value.Path
cogent.app.util.ApplicationError: Could not open
"Databases/unite_taxonomy/unite_ref_seqs_21nov2011.fasta.log"
iMac-de-Blanca:~ blancalanda$


Now


> - Then run
> ls -laR Piro/ITS/OTUS/blast_picked_otus/

total 0
drwxr-xr-x 2 blancalanda staff 68 Jan 27 19:48 .
drwxr-xr-x 6 blancalanda staff 204 Jan 27 20:23 ..
iMac-de-Blanca:~ blancalanda$


> ls -laR Databases/unite_taxonomy

iMac-de-Blanca:~ blancalanda$ ls -laR Databases/unite_taxonomy
total 41672
drwxr-xr-x@ 5 blancalanda staff 170 Jan 30 18:28 .
drwxr-xr-x 5 blancalanda staff 170 Jan 27 19:28 ..
-rw-r--r-- 1 blancalanda staff 5486 Nov 22 21:44 README.txt
-rw-r--r-- 1 blancalanda staff 1978838 Nov 21 21:48
unite_id_to_taxonomy_map_21nov2011.txt
-rw-r--r--@ 1 blancalanda staff 19342590 Nov 21 21:48
unite_ref_seqs_21nov2011.fasta
iMac-de-Blanca:~ blancalanda$

> df -h

iMac-de-Blanca:~ blancalanda$ df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/disk0s2 931Gi 63Gi 867Gi 7% /
devfs 181Ki 181Ki 0Bi 100% /dev
map -hosts 0Bi 0Bi 0Bi 100% /net
map auto_home 0Bi 0Bi 0Bi 100% /home
iMac-de-Blanca:~ blancalanda$

> Thanks
Thanks to you for the help!

Antonio González Peña

unread,
Jan 30, 2012, 2:07:16 PM1/30/12
to ag2l...@uco.es, qiime...@googlegroups.com, blanca...@csic.es
Blanca,

Everything seems fine. Lets try building the blast database by hand,
could you try doing this and report back?
cd Databases/unite_taxonomy/
formatdb -i unite_ref_seqs_21nov2011.fasta -oT -pF -l
unite_ref_seqs_21nov2011.log

Cheers

ag2l...@uco.es

unread,
Jan 30, 2012, 2:17:37 PM1/30/12
to qiime...@googlegroups.com, Antonio González Peña, qiime...@googlegroups.com, blanca...@csic.es
Some problem:
iMac-de-Blanca:~ blancalanda$ cd Databases/unite_taxonomy/
iMac-de-Blanca:unite_taxonomy blancalanda$ formatdb -i
unite_ref_seqs_21nov2011.fasta -oT -pF -l unite_ref_seqs_21nov2011.log
-bash: formatdb: command not found


Antonio González Peña <antg...@gmail.com> escribió:

Antonio González Peña

unread,
Jan 30, 2012, 2:50:52 PM1/30/12
to ag2l...@uco.es, qiime...@googlegroups.com, blanca...@csic.es
Hi Blanca,

You have reported two scenarios with formatdb:
1. it is not installed (your last reply)


iMac-de-Blanca:unite_taxonomy blancalanda$ formatdb -i
unite_ref_seqs_21nov2011.fasta -oT -pF -l unite_ref_seqs_21nov2011.log
-bash: formatdb: command not found

2. it is installed but wasn't run properly (when you replied to Greg)


iMac-de-Blanca:~ blancalanda$ formatdb
[formatdb 2.2.22] ERROR: No database name was specified

So my guess is that you are having a problem with your environment and
macqiime, can you try
macqiime formatdb -i unite_ref_seqs_21nov2011.fasta -oT -pF -l
unite_ref_seqs_21nov2011.log

Cheers

Blanca B. Landa

unread,
Jan 30, 2012, 3:12:25 PM1/30/12
to Blanca Landa, AntonioGonzález Peña, qiime...@googlegroups.com
If it helps I have been able to go through all steps with Macqiime and a 16S
dataset
Blanca

-----Mensaje original-----
From: Blanca Landa
Sent: Monday, January 30, 2012 9:02 PM
To: AntonioGonz�lez Pe�a ; ag2l...@uco.es
Cc: qiime...@googlegroups.com


Subject: Re: [qiime-forum 1.4.0] Re: Fungal analysis

I already tried before you suggested:

/usr/bin/macqiime: line 12: formatdb: command not found
iMac-de-Blanca:unite_taxonomy blancalanda$ unite_ref_seqs_21nov2011.log


I can try with the VM and see how it works?


El 1/30/12 8:50 PM, "Antonio Gonz�lez Pe�a" <antg...@gmail.com> escribi�:

Jeff Werner

unread,
Jan 31, 2012, 11:40:59 AM1/31/12
to qiime...@googlegroups.com, Blanca Landa, AntonioGonzález Peña
Hi Blanca,

It looks like the blast program formatdb isn't installed or available to you... that's odd.  How did you go about installing BLAST?  If blastall is available, I would think that formatdb should be installed right along with it... It's odd that this worked with a previous data set, and I'm not sure how blast could have gotten broken in the meantime...

What do these commands return?

macqiime which blastall

and

macqiime which formatdb


Blanca B. Landa

unread,
Jan 31, 2012, 1:06:14 PM1/31/12
to Blanca Landa, qiime...@googlegroups.com
 
 
Sent: Tuesday, January 31, 2012 7:05 PM
Subject: Re: [qiime-forum 1.4.0] Re: Fungal analysis
 
Hi Jeff
When it worked with the 16S data set i followed the instructions and I think in the tutorial Blast it is not needed also the Fungal database.
 
This is are the output of the commands:
 
iMac-de-Blanca:~ blancalanda$ macqiime which blastall
./blast-2.2.22/bin/blastall
 
iMac-de-Blanca:~ blancalanda$ macqiime which formatdb
./blast-2.2.22/bin/formatdb
 
 
Probably when I created the files you told me a few days ago I did something wrong
 
Blanca

Greg Caporaso

unread,
Jan 31, 2012, 8:40:51 PM1/31/12
to qiime...@googlegroups.com, ag2l...@uco.es, blanca...@csic.es
Hi Blanca,
I'm reviewing these emails and haven't been able to figure out what's
going on. Did you try this suggestion from Antonio, and if so what was
the result:

cd ~/Databases/unite_taxonomy/


macqiime formatdb -i unite_ref_seqs_21nov2011.fasta -oT -pF -l
unite_ref_seqs_21nov2011.log

Greg

Jeff Werner

unread,
Jan 31, 2012, 11:01:22 PM1/31/12
to qiime...@googlegroups.com
Hi Blanca,

Sorry, I'm confused by the dots before the paths output from the "which" command... has anyone else ever seen dots before a path in this context in unix? Presumably it means that the path is starting at the currrent directory?

Cheers,
Jeff

Greg Caporaso

unread,
Jan 31, 2012, 11:09:11 PM1/31/12
to qiime...@googlegroups.com
I was also confused by that - I've never seen it before. Blanca, can
you post the output of the following two commands:

echo $PATH

macqiime echo $PATH

Greg

Blanca B. Landa

unread,
Feb 1, 2012, 3:58:28 AM2/1/12
to qiime...@googlegroups.com
Hello Jeff and Greg
I think the problem is in the way I installed the RDP and something I did
wormg including the ./

First I tried to create the .ncbirc and .profile files as Jeff suggested me
in a previous email

This is my e-mail and the errors I found:

From: Blanca B. Landa
Sent: Thursday, January 26, 2012 10:17 PM
To: Jeffrey Werner
Cc: blanca...@csic.es
Subject: Re: help

Hi Jeff
Thanks for the fast answer.

I have followed your instructions and copy the blast-2.2.22 in my home
directory:
users/blancalanda

I have edited the .profile as this
export PATH=/blast-2.2.22/bin:${PATH}
but I do not receive any answer with the command
which blastall

I have also edited it like this
export PATH=/blancalanda/blast-2.2.22/bin:${PATH}

and I do not get any results with the command which blastall

I am doing something wrong?

When trying to make a directory in the terminal using mkdir it says
Permission denied,
I have been able to install all the programs that do not come with Macqiime
but I did not need any permit for installing things


Then, I read ways to create the .ncbirc files and decide to include the ./:

Hello again!
I solved it using this path:
./blast-2.2.22/

iMac-de-Blanca:~ blancalanda$ which blastall
./blast-2.2.22/bin/blastall


So, I think I should try to create new .ncbirc and .profile files ?

Sorry for this mess!
Blanca


-----Mensaje original-----
From: Jeff Werner
Sent: Wednesday, February 01, 2012 5:01 AM


To: qiime...@googlegroups.com
Subject: Re: [qiime-forum 1.4.0] Re: Fungal analysis

Hi Blanca,

Jeff Werner

unread,
Feb 1, 2012, 9:12:56 AM2/1/12
to qiime...@googlegroups.com
Hi Blanca,

Ah, yes that must be the issue.  Where exactly did you put the blast-2.2.22 directory?  If it's in your home directory, and your user name is blancalanda, the line in .profile should read this:

export PATH=/Users/blancalanda/blast-2.2.22/bin:${PATH}

Assuming this is a normal configuration on a personal mac computer, in which case your user home directory should be in the /Users/ folder  This line is telling your command line where to look for executable files (all that info is contained in the PATH variable, and you're adding the BLAST executables, in that "bin" directory, to your path).  Alternatively, it could be expressed like this:

export PATH=~/blast-2.2.22/bin:${PATH}

(with a squiggle character at the beginning of the path name, where the squiggle is a special symbol that represents your home directory) -- I think the original problem was probably that you had a dot instead of a squiggle in there.  If that were the case, the PATH to the blast binaries would only be valid when you are in your home directory (because the dot-slash ./ means "right here in my current directory"). 

Also, your .ncbirc file (just to make sure) should read as follows:

[NCBI]
Data=/Users/blancalanda/blast-2.2.22/data/

I hope that helps!

Cheers,
Jeff

Jeff Werner

unread,
Feb 1, 2012, 9:16:23 AM2/1/12
to qiime...@googlegroups.com
BTW, you can check to make sure what your user home directory is:  Right away when you open the Terminal, you start in your home.  If you then type this command:

pwd

It should print out your current location (in this case, your home directory path), which I was guessing should be

/Users/blancalanda/

but it might be something different if you're on a server or something...

Cheers,
Jeff

Blanca B. Landa

unread,
Feb 1, 2012, 4:01:06 PM2/1/12
to qiime...@googlegroups.com
Hi Jeff!
It worked great. The problem was the way I generated the .ncbirc and .profile files
I am currently running the pick_otus command and it has been working for 2 h. It is normal?
Thanks again!
Blanca
 
Sent: Wednesday, February 01, 2012 3:12 PM
Subject: Re: [qiime-forum 1.4.0] Re: Fungal analysis
 

Jose Carlos Clemente

unread,
Feb 1, 2012, 4:03:41 PM2/1/12
to qiime...@googlegroups.com
Hi Blanca,

pick_otus can take quite a while depending on what parameters you are
using. If you are doing uclust_ref, you can use
parallel_pick_otus_uclust_ref.py to speed up things by running it in
parallel.

Jose

Blanca B. Landa

unread,
Feb 1, 2012, 4:14:30 PM2/1/12
to qiime...@googlegroups.com
Ok! I got it
I think it was explained a few months ago in the Forum. I will read it
Blanca

-----Mensaje original-----
From: Jose Carlos Clemente
Sent: Wednesday, February 01, 2012 10:03 PM

BB_Landa

unread,
Feb 4, 2012, 4:46:29 AM2/4/12
to Qiime Forum
Hello again
Finally I went through the pick_otus.py script without problems using
both BLAST and UCLUST

However I am trying now to assign the OTUs to taxonomy following the
18S tutorial as well as the retraining_rdp as this:

iMac-de-Blanca:~ blancalanda$ Macqiime assign_taxonomy.py -i Piro/ITS/
OTUS/uclust_picked_otus/rep_set.fna -o Piro/ITS/OTUS/
RDP_assigned_taxonomy/ -t Databases/unite_taxonomy/
unite_id_to_taxonomy_map_21nov2011.txt -r Databases/unite_taxonomy/
unite_ref_seqs_21nov2011.fasta

An I get the following errors:

Traceback (most recent call last):
File "/macqiime/QIIME/bin/assign_taxonomy.py", line 191, in <module>
main()
File "/macqiime/QIIME/bin/assign_taxonomy.py", line 187, in main
result_path=result_path,log_path=log_path)
File "/macqiime/lib/python2.7/site-packages/qiime/
assign_taxonomy.py", line 389, in __call__
max_memory=max_memory)
File "/macqiime/lib/python2.7/site-packages/qiime/pycogent_backports/
rdp_classifier.py", line 492, in
train_rdp_classifier_and_assign_taxonomy
training_seqs_file, taxonomy_file, training_dir,
max_memory=max_memory)
File "/macqiime/lib/python2.7/site-packages/qiime/pycogent_backports/
rdp_classifier.py", line 464, in train_rdp_classifier
return app(training_seqs_file)
File "/macqiime/lib/python2.7/site-packages/qiime/pycogent_backports/
rdp_classifier.py", line 320, in __call__
result = super(RdpClassifier, self).__call__(data=data,
remove_tmp=remove_tmp)
File "/macqiime/lib/python2.7/site-packages/cogent/app/util.py",
line 250, in __call__
% (str(exit_status),command)
cogent.app.util.ApplicationError: Unacceptable application exit
status: 1, command: cd "/Users/blancalanda/"; java -Xmx1000M -cp "/
macqiime/rdp_classifier_2.2/rdp_classifier-2.2.jar"
edu.msu.cme.rdp.classifier.train.ClassifierTraineeMaker "/var/folders/
ff/r95dmshs4k3cm7yq3kgprf9r0000gn/T/RdpTaxonomy_bdQRef.txt" "/tmp/
tmps2IHM7G1sfVMmqGGHmLe.txt" 1 version1 cogent "/var/folders/ff/
r95dmshs4k3cm7yq3kgprf9r0000gn/T/RdpTrainer_flyZL1" > "/tmp/
tmpUnMYfuLR5UmLe57eo45d.txt" 2> "/tmp/tmpbRuMxiRPmYqBtGEm2ci5.txt"


Can anyone help me to fix the problem? Thanks
Blanca

Jose Carlos Clemente

unread,
Feb 4, 2012, 11:32:47 AM2/4/12
to qiime...@googlegroups.com
Hi Blanca,

can you try the following from the command line

java -Xmx1000M -cp "/macqiime/rdp_classifier_2.2/rdp_classifier-2.2.jar"

and send us the result?

Jose

ag2l...@uco.es

unread,
Feb 4, 2012, 2:22:55 PM2/4/12
to qiime...@googlegroups.com, Jose Carlos Clemente
I get no results just waiting:

iMac-de-Blanca:~ blancalanda$ java -Xmx1000M -cp
"/macqiime/rdp_classifier_2.2/rdp_classifier-2.2.jar

Jose Carlos Clemente

unread,
Feb 4, 2012, 2:56:12 PM2/4/12
to ag2l...@uco.es, qiime...@googlegroups.com
Can you please send the result of print_qiime_config.py -t?

ag2l...@uco.es

unread,
Feb 4, 2012, 3:06:18 PM2/4/12
to Jose Carlos Clemente, qiime...@googlegroups.com
Quoting Jose Carlos Clemente <jose.c...@gmail.com>:

> Can you please send the result of print_qiime_config.py -t?

iMac-de-Blanca:~ blancalanda$ macqiime print_qiime_config.py -t

System information
==================
Platform: darwin
Python version: 2.7.1 (r271:86832, Dec 15 2011, 08:41:37) [GCC
4.0.1 (Apple Inc. build 5493)]
Python executable: /macqiime/bin/python

Dependency versions
===================
PyCogent version: 1.5.1

NumPy version: 1.5.1
matplotlib version: 1.1.0
QIIME library version: 1.4.0
QIIME script version: 1.4.0


PyNAST version (if installed): 1.1
RDP Classifier version (if installed): rdp_classifier-2.2.jar

QIIME config values
===================
blastmat_dir: None
topiaryexplorer_project_dir: None

pynast_template_alignment_fp: /macqiime/greengenes/core_set_aligned.fasta.imputed
cluster_jobs_fp: /macqiime/QIIME/bin/start_parallel_jobs.py
pynast_template_alignment_blastdb: None
assign_taxonomy_reference_seqs_fp: None
torque_queue: friendlyq
template_alignment_lanemask_fp: /macqiime/greengenes/lanemask_in_1s_and_0s
jobs_to_start: 1
cloud_environment: False
qiime_scripts_dir: /macqiime/QIIME/bin/
denoiser_min_per_core: 50
working_dir: None
python_exe_fp: /macqiime/bin/python
temp_dir: /tmp/
blastall_fp: blastall
seconds_to_sleep: 60
assign_taxonomy_id_to_taxonomy_fp: None


running checks:

test_FastTree_supported_version (__main__.Qiime_config)


FastTree is in path and version is supported ... ok

test_INFERNAL_supported_version (__main__.Qiime_config)


INFERNAL is in path and version is supported ... ok

test_ampliconnoise_install (__main__.Qiime_config)
AmpliconNoise install looks sane. ... FAIL
test_blast_supported_version (__main__.Qiime_config)


blast is in path and version is supported ... ok

test_blastall_fp (__main__.Qiime_config)


blastall_fp is set to a valid path ... ok

test_blastmat_dir (__main__.Qiime_config)


blastmat_dir is set to a valid path. ... ok

test_cdbtools_supported_version (__main__.Qiime_config)


cdbtools is in path and version is supported ... ok

test_cdhit_supported_version (__main__.Qiime_config)


cd-hit is in path and version is supported ... ok

test_chimeraSlayer_install (__main__.Qiime_config)


no obvious problems with ChimeraSlayer install ... ok

test_clearcut_supported_version (__main__.Qiime_config)


clearcut is in path and version is supported ... ok

test_cluster_jobs_fp (__main__.Qiime_config)


cluster_jobs_fp is set to a valid path and is executable ... ok

test_denoiser_supported_version (__main__.Qiime_config)


denoiser aligner is ready to use ... ok

test_for_obsolete_values (__main__.Qiime_config)


local qiime_config has no extra params ... ok

test_matplotlib_suported_version (__main__.Qiime_config)


maptplotlib version is supported ... ok

test_mothur_supported_version (__main__.Qiime_config)


mothur is in path and version is supported ... ok

test_muscle_supported_version (__main__.Qiime_config)


muscle is in path and version is supported ... ok

test_numpy_suported_version (__main__.Qiime_config)


numpy version is supported ... ok

test_pynast_suported_version (__main__.Qiime_config)


pynast version is supported ... ok

test_pynast_template_alignment_blastdb_fp (__main__.Qiime_config)


pynast_template_alignment_blastdb, if set, is set to a valid path ... ok

test_pynast_template_alignment_fp (__main__.Qiime_config)


pynast_template_alignment, if set, is set to a valid path ... ok

test_python_exe_fp (__main__.Qiime_config)


python_exe_fp is set to a working python env ... ok

test_python_supported_version (__main__.Qiime_config)


python is in path and version is supported ... ok

test_qiime_scripts_dir (__main__.Qiime_config)


qiime_scripts_dir, if set, is set to a valid path ... ok

test_raxmlHPC_supported_version (__main__.Qiime_config)
raxmlHPC is in path and version is supported ... FAIL
test_temp_dir (__main__.Qiime_config)


temp_dir, if set, is set to a valid path ... ok

test_template_alignment_lanemask_fp (__main__.Qiime_config)


template_alignment_lanemask, if set, is set to a valid path ... ok

test_uclust_supported_version (__main__.Qiime_config)


uclust is in path and version is supported ... ok

test_working_dir (__main__.Qiime_config)


working_dir, if set, is set to a valid path ... ok

======================================================================
FAIL: test_ampliconnoise_install (__main__.Qiime_config)
AmpliconNoise install looks sane.
----------------------------------------------------------------------


Traceback (most recent call last):

File "/macqiime/QIIME/bin/print_qiime_config.py", line 116, in
test_ampliconnoise_install
"$PYRO_LOOKUP_FILE variable is not set. See %s for help." % url)
AssertionError: $PYRO_LOOKUP_FILE variable is not set. See
http://www.qiime.org/install/install.html#ampliconnoise-install for
help.

======================================================================
FAIL: test_raxmlHPC_supported_version (__main__.Qiime_config)


raxmlHPC is in path and version is supported

----------------------------------------------------------------------


Traceback (most recent call last):

File "/macqiime/QIIME/bin/print_qiime_config.py", line 526, in
test_raxmlHPC_supported_version
% ('.'.join(map(str,acceptable_version)), version_string))
AssertionError: Unsupported raxmlHPC version. 7.0.3 is required, but
running 7.0.4.

----------------------------------------------------------------------
Ran 28 tests in 0.590s

FAILED (failures=2)
iMac-de-Blanca:~ blancalanda$


THANKS


Here you have:

Jose Carlos Clemente

unread,
Feb 4, 2012, 4:01:16 PM2/4/12
to Qiime Forum
A few suggestions below:

- make sure you have java 1.6 or greater: from the command line, type

java -version

- increase the memory allocated using the parameter --rdp_max_memory

Just to be sure, how was the file
unite_id_to_taxonomy_map_21nov2011.txt created?

Jose

ag2l...@uco.es

unread,
Feb 4, 2012, 5:13:34 PM2/4/12
to qiime...@googlegroups.com, Jose Carlos Clemente, Qiime Forum
Hi Jose Carlos
I have that java version

iMac-de-Blanca:~ blancalanda$ java -version
java version "1.6.0_29"
Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-11M3527)
Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)
iMac-de-Blanca:~ blancalanda$


I am not sure how was the file created, I downloaded it from the files
available at qiime web page. I can see if there is some infer posted
in the forum

BB

Jose Carlos Clemente

unread,
Feb 4, 2012, 10:32:39 PM2/4/12
to Qiime Forum
Hi Blanca,

I tried those files and I'm getting the same error. We think the issue
might be this was working with RDP 2.0 not with 2.2. I will check who
produced the file and get back to you asap with a solution. Sorry for
the inconvenience!

Jose

ag2l...@uco.es

unread,
Feb 5, 2012, 3:41:38 AM2/5/12
to qiime...@googlegroups.com, Jose Carlos Clemente, Qiime Forum
Perfect!
Thanks again
Blanca

Tony Walters

unread,
Feb 5, 2012, 2:28:19 PM2/5/12
to qiime...@googlegroups.com
Hello Blanca,

I did some tests with the Unite database, and here is what I found:

As the Unite database currently is, you need to specify 12000 memory with --rdp_max_memory to run it.  It will run out of memory with 8 gigs.

I made a modified taxonomy mapping file, where the the numbers that were appended to the ends of the taxonomy strings were removed, and I was able to run this with --rdp_max_memory 4000

I've attached the filtered taxonomy mapping file.

Best regards,
Tony Walters
unite_taxonomy_mapping_filtered_numbers.txt.gz

Blanca B. Landa

unread,
Feb 5, 2012, 3:37:12 PM2/5/12
to qiime...@googlegroups.com
Thanks Tony
I have 8 GB of memory. So, I will try it and let you know the results
Blanca
 
Sent: Sunday, February 05, 2012 8:28 PM
Subject: Re: [qiime-forum 1.4.0] Re: Fungal analysis
 

Blanca B. Landa

unread,
Feb 5, 2012, 4:30:32 PM2/5/12
to qiime...@googlegroups.com
Hi Tony!
It worked great just a few seconds and I got the taxonomy solved as well as I just created the out table
THANKSSSSSSS
BLANCA
 
Sent: Sunday, February 05, 2012 8:28 PM
Subject: Re: [qiime-forum 1.4.0] Re: Fungal analysis
 
Reply all
Reply to author
Forward
0 new messages