Does anyone use QIIME for fungal data analysis (not bacteria)? ITS
database? How to set a qiime configure file for ITS_reference
database?
Thanks,
Ng
There is a QIIME compatible UNITE taxonomy mapping and reference
sequence file available here:
http://qiime.org/home_static/dataFiles.html
To modify your reference and id to taxonomy mapping file paths, you
can modify the qiime_config file. Instructions are found here:
http://qiime.org/install/qiime_config.html
Hope this helps,
Tony Walters
Thanks for promptly response
The ITS data set looks good, hopefully its up-to-date. I downloaded
the ITS data on my local home (~/BLAST/ITS/its_qiime.fasta) and the
qiime_config is located @ ~/.qiime_configure. I don't know how to
modify the 'qiime_config' correctly.
Here is my current environment:
[ng@csclp1 ~]$ print_qiime_config.py -t
System information
==================
Platform: linux2
Python version: 2.6.5 (r265:79063, Feb 28 2011, 21:55:45) [GCC
4.1.2 20080704 (Red Hat 4.1.2-50)]
Python executable: /usr/bin/python26
Dependency versions
===================
PyCogent version: 1.5.1
NumPy version: 1.3.0
matplotlib version: 0.98.5.3
QIIME library version: 1.3.0
QIIME script version: 1.3.0
PyNAST version (if installed): 1.1
RDP Classifier version (if installed): rdp_classifier-2.2.jar
QIIME config values
===================
blastmat_dir: None
topiaryexplorer_project_dir: None
pynast_template_alignment_fp: /stf/home/cluster/GenomicsCore/apps/
qiime/dependencies/core_set_aligned.fasta.imputed
cluster_jobs_fp: /stf/home/cluster/GenomicsCore/apps/
qiime/bin/start_parallel_jobs_sge.py
pynast_template_alignment_blastdb: None
torque_queue: all.q
template_alignment_lanemask_fp: /stf/home/cluster/GenomicsCore/apps/
qiime/dependencies/lanemask_in_1s_and_0s
jobs_to_start: 2
cloud_environment: False
qiime_scripts_dir: /stf/home/cluster/GenomicsCore/apps/
qiime/bin
denoiser_min_per_core: 50
working_dir: None
python_exe_fp: /usr/bin/python26
temp_dir: /tmp/
blastall_fp: /stf/home/cluster/GenomicsCore/apps/
qiime/dependencies/blast-2.2.22/bin/blastall
seconds_to_sleep: 60
running checks:
FastTree is in path and version is supported ... ok
INFERNAL is in path and version is supported ... ok
AmpliconNoise install looks sane. ... ok
blast is in path and version is supported ... ok
blastall_fp is set to a valid path ... ok
blastmat_dir is set to a valid path. ... ok
cdbtools is in path and version is supported ... ok
cd-hit is in path and version is supported ... ok
no obvious problems with ChimeraSlayer install ... ok
clearcut is in path and version is supported ... ok
cluster_jobs_fp is set to a valid path and is executable ... ok
denoiser aligner is ready to use ... ok
local qiime_config has no extra params ... ok
maptplotlib version is supported ... ok
mothur is in path and version is supported ... ok
muscle is in path and version is supported ... ok
numpy version is supported ... ok
pynast version is supported ... ok
pynast_template_alignment_blastdb, if set, is set to a valid path ...
ok
pynast_template_alignment, if set, is set to a valid path ... ok
python_exe_fp is set to a working python env ... ok
python is in path and version is supported ... ok
qiime_scripts_dir, if set, is set to a valid path ... ok
raxmlHPC is in path and version is supported ... ok
temp_dir, if set, is set to a valid path ... ok
template_alignment_lanemask, if set, is set to a valid path ... ok
uclust is in path and version is supported ... ok
working_dir, if set, is set to a valid path ... ok
Thanks,
Ng
http://qiime.org/svn_documentation/tutorials/retraining_rdp.html
Greg
on my first-step: Pick_OTUs - I try to use both BLAST (100% match) and
UCLUST
#BLAST
pick_otus.py -i split_library_output/seqs.fna -m blast -s 1.0 -r
BLAST/ITS/its_qiime.fa -o otus/blast_picked_otus
#UCLUST
pick_otus.py -i split_library_output/seqs.fna -o otus/
uclust_picked_otus
Does my 'pick_otus.py' command with BLAST/UCLUST correct?
Thanks,
Ng
On Nov 28, 4:39 pm, Greg Caporaso <gregcapor...@gmail.com> wrote:
> Hello Ng,
> In QIIME 1.3.0 you cannot store the taxonomy assignment reference
> files in .qiime_config, although that feature has been added to the
> development version and will be in the next release. In the meantime,
> you would pass the reference sequences and taxonomy map to
> assign_taxonomy.py (or one of the parallel versions) via the -r and -t
> parameters. This tutorial shows how to do it (but with different
> reference files):
>
> http://qiime.org/svn_documentation/tutorials/retraining_rdp.html
>
> Greg
>
Greg
Would it be possible to run 'uclust' OTU picking with a reference-
base? How does a uclust work with de-noval and reference ?
Thanks,
Ng
On Nov 28, 5:04 pm, Greg Caporaso <gregcapor...@gmail.com> wrote:
> That looks correct - this will do reference-based BLAST OTU picking
> and de novo uclust OTU picking.
>
> Greg
>
de novo otu picking (default, you're already doing this)
closed reference otu picking (pass -r, -C, and -m uclust_ref)
open reference otu picking (pass -r and -m uclust_ref)
The difference between open and closed reference OTU picking is in how
the sequences which don't hit the reference are handled. In closed
reference OTU picking, sequences which don't hit the reference set are
discarded (as with the BLAST OTU picker). In open reference OTU
picking, sequences which don't hit the reference set are clustered de
novo.
Greg
It works great for me so far.
How to run 'pick_otus.pl' in parallel?
#pick_otus.py -i split_library_output/seqs.fna -m blast -s 1.0 -r
BLAST/ITS/its_qiime.fa -o otus/uclust_picked_otus
Thanks,
Ng
See here for details on setting up parallel QIIME:
http://qiime.org/tutorials/parallel_qiime.html
The parallel works on our cluster.
>parallel_pick_otus_blast.py -i split_library_output/seqs_all.fna -r BLAST/ITS/its_qiime.fa -s 1.0 -o otus_para/blast_100_otus -T --jobs_to_start 10
However, the results are not merge. Do I have to merge all reuslt
files manually by the end?
...
rw-r--r-- 1 genomics genomics 109K Nov 29 12:49 POTU_dgD0_.0_otus.txt
-rw-r--r-- 1 genomics genomics 297K Nov 29 12:49 POTU_dgD0_.0_otus.log
-rw-r--r-- 1 genomics genomics 121K Nov 29 12:50 POTU_dgD0_.5_otus.txt
-rw-r--r-- 1 genomics genomics 297K Nov 29 12:50 POTU_dgD0_.5_otus.log
drwxr-xr-x 2 genomics genomics 106 Nov 29 12:50 POTU_dgD0_
[genomics@csclp blast_100_otus]$ ls -lrth POTU_dgD0_
total 77K
-rw-r--r-- 1 genomics genomics 939 Nov 29 09:10
expected_out_files.txt
-rw-r--r-- 1 genomics genomics 1.1K Nov 29 09:10 merge_map.txt
-rw-r--r-- 1 genomics genomics 1.8K Nov 29 09:10 deletion_list.txt
Thanks,
Ng
On Nov 29, 6:08 am, Greg Caporaso <gregcapor...@gmail.com> wrote:
> You can use parallel_pick_otus_uclust_ref.py or parallel_pick_otus_blast.py.
>
> See here for details on setting up parallel QIIME:
>
> http://qiime.org/tutorials/parallel_qiime.html
>
The jobs were completed, result files are not merged. There wasn't
error files. I just 'cat' all file together.
I have another questions:
The representative sequence appear multiple times (6 times) @
otus_para/rep_set/seqs_rep_100.fasta . Should it be one represent
sequence per OTU? I may miss some thing here ?
[genomics@csclp1 fungal]$ grep 'AB019340' otus_para/rep_set/
seqs_rep_100.fasta
>AB019340 mid3_run2_10496
>AB019340 mid3_run2_30234
>AB019340 mid2_run3_17310
>AB019340 mid2_run3_47837
>AB019340 mid2_run3_101673
>AB019340 mid2_run3_135259
[genomics@csclp1 fungal]$ less otus_para/otus_table_gg_100.txt
# QIIME v1.3.0 OTU table
#OTU ID mid1_run2 mid1_run3 mid2_run2
mid2_run3 mid3_run2 mid3_run3 mid4_run2
mid4_run3 Consensus Lineage
AB019340 0 0 0 2 0 0
0 0 Root: Fungal
AB085802 0 0 0 0 0 0
0 4 Root;k__Bacteria
AB105432 0 0 0 17 0 0
0 5 Root
The 'AB019340' OTU has 2 hits on sample 'mid2_run3', which is not
match with number of sequence above.
Would you plain relationships between two files above?
Thanks,
Ng
On Nov 29, 1:40 pm, Jose Carlos Clemente <jose.cleme...@gmail.com>
wrote:
> Hi Ng,
> it looks like your parallel job did not complete properly, thus the
> results are not merged. Can you check whether your main job finished
> correctly?
> Jose
>
> ...
>
> read more »
>parallel_pick_otus_blast.py -i split_library_output/seqs_all.fna -r BLAST/ITS/its_qiime.fa -s 1.0 -o otus_para/blast_100_otus -T --jobs_to_start 10
One potential issue is that you need to use full paths when calling
this script. If you don't have the output from the original (or if it
never actually completed) can you please rerun using full paths for
the -i, -r, and -o options.
Greg
Awesome! The absolute path works. Thank you for making the parallel
option, which save us lot of time.
Thanks,
Ng
I have no-match on the 'Align OTU Sequences'.
>parallel_align_seqs_pynast.py -i otus/rep_set/seqs_set.fasta -d BLAST/ITS/its_qiime.fa -o otus/pynast_aligned_seqs -T --jobs_to_star 2 &
OR
>parallel_align_seqs_pynast.py -i otus/rep_set/seqs_set.fasta -o otus/pynast_aligned_seqs -T --jobs_to_star 2 &
[genomics@csclp1 fungal]$ ls -lrth otus/pynast_aligned_seqs/
total 245K
-rw-r--r-- 1 genomics genomics 0 Nov 30 16:13 seqs_set_aligned.fasta
-rw-r--r-- 1 genomics genomics 59K Nov 30 16:13
seqs_set_failures.fasta
-rw-r--r-- 1 genomics genomics 12K Nov 30 16:13 seqs_set_log.txt
I used a reference-base (..BLAST/ITS/its_qiime.fa) picking OTUs, I
confuse the 'Align OTU sequence' steps, and why I don't have aligned
sequence?
Thanks,
Ng
Good luck!
Greg
I've tried uclust to compare results with blast
>parallel_pick_otus_blast.py -i split_library_output/seqs_all.fna -r BLAST/ITS/its_qiime.fa -s 1.0 -o otus_para/uclust_100_otus -T --jobs_to_start 10 & <-- for 100% matching ?
OK
>parallel_pick_otus_blast.py -i split_library_output/seqs_all.fna -r BLAST/ITS/its_qiime.fa -s 1.0 -m -C -o otus_para/uclust_100_otus -T --jobs_to_start 10 &
parallel_pick_otus_blast.py: error: no such option: -m, -C
How to select option for open/close reference otu_picking on UCLUST?
Does BLAST have open/close reference?
Thanks,
Ng
On Nov 28, 5:39 pm, Greg Caporaso <gregcapor...@gmail.com> wrote:
> Ng,
> You have three options with uclust:
>
> de novo otu picking (default, you're already doing this)
> closed reference otu picking (pass -r, -C, and -m uclust_ref)
> open reference otu picking (pass -r and -m uclust_ref)
>
> The difference between open and closed reference OTU picking is in how
> the sequences which don't hit the reference are handled. In closed
> reference OTU picking, sequences which don't hit the reference set are
> discarded (as with the BLAST OTU picker). In open reference OTU
> picking, sequences which don't hit the reference set are clustered de
> novo.
>
> Greg
>
I am using ITS, so I have to use '-m muscle'
>filter_alignment.py -o otus_para2/muscle_alignment/ -i otus_para2/muscle_alignment/seqs_set_aligned.fasta (Works for me!)
What are purposes of this step? how doe sit work? brief would be ok.
Thanks,
Ng
On Nov 30, 4:57 pm, Greg Caporaso <gregcapor...@gmail.com> wrote:
> Are your sequences 16S or ITS? The default reference alignment is a
> 16S alignment, so your ITS sequences shouldn't hit that. If you don't
> have an ITS reference alignment (I don't know of one) you'll need to
> use "align_seqs.py -m muscle" rather than the default of PyNAST. There
> is no parallel option for muscle.
>
> Good luck!
> Greg
>
Pl regardless my most previous email. What I mean is:
> align_seqs.py -i otus_para2/rep_set/seqs_set.fasta -m muscle -d BLAST/ITS/its_qiime.fa -o otus_para2/muscle_alignment
What are purposes of this step? how doe sit work? brief would be ok.
Thanks,
Ng
BLAST only has reference OTU picking. The similarity percentage is by
default 97%, not 100%: to set it to 100%, use -s 1.
Jose
Yes, I used the '-s 1.0' option for 100% similarity. I like to test
both option blast and uclust. What are options to discard sequences
that not map the target.
>parallel_pick_otus_uclust_ref.py -i split_library_output/seqs_all.fna -r BLAST/ITS/its_qiime.fa -s 1.0 -m -C -o otus_para/uclust_100_otus -T --jobs_to_start 10 (The -m, -C option don't work)
Thanks,
Ng
On Dec 1, 11:11 am, Jose Carlos Clemente <jose.cleme...@gmail.com>
wrote:
> Hi NG,
>
> BLAST only has reference OTU picking. The similarity percentage is by
> default 97%, not 100%: to set it to 100%, use -s 1.
>
> Jose
>
sorry I didn't realize you had -s already in there. Sequences that do
not match your reference set will automatically show as
seqs_all_failures.txt in your output directory.
I am not sure what are you trying to do with -m -C, those are not
valid options in either parallel_pick_otus_uclust_ref.py or
parallel_pick_otus_blast.py. If you are trying to use -C to supress
new clusters (there is such an option in pick_otus.py), you don't need
since these scripts are already doing reference based only.
Jose
We have illumina reads (100 base long). What we like is to get only
reads that 100% match (all 100 bases matches) with target (ref
database) . Does the '-s 1.0' optin work?
>parallel_pick_otus_blast.py -i split_library_output/seqs_all.fna -r BLAST/ITS/its_qiime.fa -s 1.0 -o otus_para/uclust_100_otus -T --jobs_to_start 10 &
I've tried to verify the result (from runs above), I've blasted a
representative sequence(s) with the same ref database. I don't see
sequence aligned 100% (100% identities = 98/98 or 90/90, but not
100/100)
Do you guys have any ideas? or I miss something here?
Thanks,
Ng
On Dec 1, 12:49 pm, Jose Carlos Clemente <jose.cleme...@gmail.com>
wrote:
> Hi Ng,
>
> sorry I didn't realize you had -s already in there. Sequences that do
> not match your reference set will automatically show as
> seqs_all_failures.txt in your output directory.
>
> I am not sure what are you trying to do with -m -C, those are not
> valid options in either parallel_pick_otus_uclust_ref.py or
> parallel_pick_otus_blast.py. If you are trying to use -C to supress
> new clusters (there is such an option in pick_otus.py), you don't need
> since these scripts are already doing reference based only.
>
> Jose
>
> ...
>
> read more »
not all your sequences are going to be exactly 100 base pairs long,
you will have a distribution of lengths. 100% identity represents
perfect matches, no matter the length.
Jose
Yes, All sequences have the same length (illumina platform). This is
sample of representative sequences (one sequence per OTU/cluster).
What I expect is each of the representative sequences has a perfect
match with the target. This is what I was setting on the
'parallel_pick_otus_blast.py' with the '-s 1.0' option.
>AF07-test
CCCATCAGGGCATGTGCACGGCCTGTCTTGACTCTTATTCATCCACCTGTGCACCTTCTGTAGTCTTCTTGGATGTGAGGAATTGCTCGTCAGGCTTTCC
>AF33-test2
AGTCGTAACAAGGTTTCCGTAGGTGAACCTGCGGAAGGATCATTACTGATTTGCTTAATTGCACCACATGTGTTTTTTATTGAACAAATTTCTTTGGTGG
Verification step: Blast these sequences
> blastall -p blastn -d BLAST/ITS/its_qiime.fa -i input -o out_blast -v 2 -b 2 &
BLAST's result (Not all)
...
Query= AF07-test
(100 letters)
Database: its_qiime.fa
30,379 sequences; 19,008,933 total letters
Searching..................................................done
Score E
Sequences producing significant alignments:
(bits) Value
AF079746
198 2e-51
EF527327
96 3e-20
>AF079746
Length = 670
Score = 198 bits (100), Expect = 2e-51
Identities = 100/100 (100%)
Strand = Plus / Plus
Query: 1
cccatcagggcatgtgcacggcctgtcttgactcttattcatccacctgtgcaccttctg 60
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Sbjct: 56
cccatcagggcatgtgcacggcctgtcttgactcttattcatccacctgtgcaccttctg 115
Query: 61 tagtcttcttggatgtgaggaattgctcgtcaggctttcc 100
||||||||||||||||||||||||||||||||||||||||
Sbjct: 116 tagtcttcttggatgtgaggaattgctcgtcaggctttcc 155
>EF527327
Length = 651
Score = 95.6 bits (48), Expect = 3e-20
Identities = 58/60 (96%), Gaps = 1/60 (1%)
Strand = Plus / Plus
Query: 8
gggcatgtgcacggcctgtcttgactcttattcatccacctgtgcaccttctgtagtctt 67
||||||||||||||||||||||||||| ||||||||||||||||||||||
|||||||||
Sbjct: 64 gggcatgtgcacggcctgtcttgactc-
tattcatccacctgtgcaccttttgtagtctt 122
....
Query= AF33-test2
(100 letters)
Database: its_qiime.fa
30,379 sequences; 19,008,933 total letters
Searching..................................................done
Score E
Sequences producing significant alignments:
(bits) Value
AF335928
168 2e-42
AJ010333
153 1e-37
>AF335928
Length = 218
Score = 168 bits (85), Expect = 2e-42
Identities = 85/85 (100%)
Strand = Plus /
Plus
Query: 16
tccgtaggtgaacctgcggaaggatcattactgatttgcttaattgcaccacatgtgttt 75
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Sbjct: 1
tccgtaggtgaacctgcggaaggatcattactgatttgcttaattgcaccacatgtgttt 60
Query: 76 tttattgaacaaatttctttggtgg 100
|||||||||||||||||||||||||
Sbjct: 61 tttattgaacaaatttctttggtgg 85
>AJ010333
Length = 464
Score = 153 bits (77), Expect = 1e-37
Identities = 77/77 (100%)
Strand = Plus / Plus
Query: 1
agtcgtaacaaggtttccgtaggtgaacctgcggaaggatcattactgatttgcttaatt 60
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Sbjct: 251
agtcgtaacaaggtttccgtaggtgaacctgcggaaggatcattactgatttgcttaatt 310
Query: 61 gcaccacatgtgttttt 77
|||||||||||||||||
Sbjct: 311 gcaccacatgtgttttt 327
1. The 'AF07-test' shows a perfect match 100/100 (Identities= 100/100
100%), which is what we like to archive.
2. The 'AF33-test2' shows a highest match only 85/100
(Identities=85/85 100%)
My question are:
1. How to set the 'parallel_pick_otus_blast.py' with options that has
'100% match' like (1)
2. What does the '-s 1.0' on 'parallel_pick_otus_blast.py' mean? (100%
similarity or identities)
3. How to get all representative sequences 100% matches to the target
like (1).
Sorry for my questions are too long. Hopefully, your answers would
help others too.
Regards,
Ng
On Dec 1, 5:48 pm, Jose Carlos Clemente <jose.cleme...@gmail.com>
wrote:
> Hi,
>
> not all your sequences are going to be exactly 100 base pairs long,
> you will have a distribution of lengths. 100% identity represents
> perfect matches, no matter the length.
>
> Jose
>
> ...
>
> read more »
To achieve the result you're looking for you also need to pass:
--min_aligned_percent 1.0
This will discard any hits that are not to the full length read.
Greg
I may have to use uclust instead then, thank you very much for your
explanation. It is very good.
I also have another question:
If a sequence (read) has 100 match (100% similarity/identities) on
multiple places on the target (fungal ITS database). How does QIIME
select the best hit? How to get all hit with 100% matches?
Thanks,
Ng
On Dec 3, 6:50 am, Greg Caporaso <gregcapor...@gmail.com> wrote:
> Hi Ng,
> The reason that you're seeing alignments to shorter fragments using
> blastall (which is what QIIME is calling) is that BLAST computes a
> local alignment rather than a global alignment. uclust, on the other
> hand, computes global alignments. So, if the score of an alignment
> drops too far (e.g., due to a stretch of mismatches) BLAST will
> terminate the alignment and return the result up to the low scoring
> position.
>
> To achieve the result you're looking for you also need to pass:
>
> --min_aligned_percent 1.0
>
> This will discard any hits that are not to the full length read.
>
> Greg
>
> ...
>
> read more »
Greg
Greg
blastall 2.2.22 arguments:
-p Program Name [String]
-d Database [String]
default = nr
-i Query File [File In]
default = stdin
-e Expectation value (E) [Real]
default = 10.0
-m alignment view options:
0 = pairwise,
1 = query-anchored showing identities,
2 = query-anchored no identities,
3 = flat query-anchored, show identities,
4 = flat query-anchored, no identities,
5 = query-anchored no identities and blunt ends,
6 = flat query-anchored, no identities and blunt ends,
7 = XML Blast output,
8 = tabular,
9 tabular with comment lines
10 ASN, text
11 ASN, binary [Integer]
………..
I get an error for formatdb:
iMac-de-Blanca:~ blancalanda$ formatdb
[formatdb 2.2.22] ERROR: No database name was specified
iMac-de-Blanca:~ blancalanda$
I have tried this route:
Macqiime pick_otus.py -i Piro/ITS/
split_library_output_ITS/seqs.fna -m blast -s 0.99 -r
/home/users/blancalanda/Databases/
unite_taxonomy/unite_ref_seqs_21nov2011.fasta -o Piro/ITS/OTUS/
blast_picked_otus/
and get same error results
Blanca
-----Mensaje original-----
From: Greg Caporaso
Sent: Monday, January 30, 2012 2:50 AM
To: qiime...@googlegroups.com
Subject: Re: [qiime-forum 1.4.0] Re: Fungal analysis
Could you run these tests and send us the output?
- Run your command:
Macqiime pick_otus.py -i Piro/ITS/split_library_output_ITS/seqs.fna
-m blast -s 0.99 -r
Databases/unite_taxonomy/unite_ref_seqs_21nov2011.fasta -o
Piro/ITS/OTUS/blast_picked_otus/
- Then run
ls -laR Piro/ITS/OTUS/blast_picked_otus/
ls -laR Databases/unite_taxonomy
df -h
Thanks
--
Antonio González Peña
Research Assistant, Knight Lab
University of Colorado at Boulder
https://chem.colorado.edu/knightgroup/
> - Run your command:
Macqiime pick_otus.py -i Piro/ITS/split_library_output_ITS/seqs.fna
-m blast -s 0.99 -r
> Databases/unite_taxonomy/unite_ref_seqs_21nov2011.fasta -o
> Piro/ITS/OTUS/blast_picked_otus/
iMac-de-Blanca:~ blancalanda$ Macqiime pick_otus.py -i
Piro/ITS/split_library_output_ITS/seqs.fna -m blast -s 0.99 -r
Databases/unite_taxonomy/unite_ref_seqs_21nov2011.fasta -o
Piro/ITS/OTUS/blast_picked_otus/
Traceback (most recent call last):
File "/macqiime/QIIME/bin/pick_otus.py", line 562, in <module>
main()
File "/macqiime/QIIME/bin/pick_otus.py", line 554, in main
blast_db=opts.blast_db,refseqs_fp=opts.refseqs_fp)
File "/macqiime/lib/python2.7/site-packages/qiime/pick_otus.py",
line 188, in __call__
build_blast_db_from_fasta_path(refseqs_fp)
File
"/macqiime/lib/python2.7/site-packages/cogent/app/formatdb.py", line
116, in build_blast_db_from_fasta_path
app_result = fdb(fasta_path)
File "/macqiime/lib/python2.7/site-packages/cogent/app/util.py",
line 269, in __call__
result_paths=self._get_result_paths(data))
File "/macqiime/lib/python2.7/site-packages/cogent/app/util.py",
line 72, in __init__
raise ApplicationError, 'Could not open %s' %value.Path
cogent.app.util.ApplicationError: Could not open
"Databases/unite_taxonomy/unite_ref_seqs_21nov2011.fasta.log"
iMac-de-Blanca:~ blancalanda$
Now
> - Then run
> ls -laR Piro/ITS/OTUS/blast_picked_otus/
total 0
drwxr-xr-x 2 blancalanda staff 68 Jan 27 19:48 .
drwxr-xr-x 6 blancalanda staff 204 Jan 27 20:23 ..
iMac-de-Blanca:~ blancalanda$
> ls -laR Databases/unite_taxonomy
iMac-de-Blanca:~ blancalanda$ ls -laR Databases/unite_taxonomy
total 41672
drwxr-xr-x@ 5 blancalanda staff 170 Jan 30 18:28 .
drwxr-xr-x 5 blancalanda staff 170 Jan 27 19:28 ..
-rw-r--r-- 1 blancalanda staff 5486 Nov 22 21:44 README.txt
-rw-r--r-- 1 blancalanda staff 1978838 Nov 21 21:48
unite_id_to_taxonomy_map_21nov2011.txt
-rw-r--r--@ 1 blancalanda staff 19342590 Nov 21 21:48
unite_ref_seqs_21nov2011.fasta
iMac-de-Blanca:~ blancalanda$
> df -h
iMac-de-Blanca:~ blancalanda$ df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/disk0s2 931Gi 63Gi 867Gi 7% /
devfs 181Ki 181Ki 0Bi 100% /dev
map -hosts 0Bi 0Bi 0Bi 100% /net
map auto_home 0Bi 0Bi 0Bi 100% /home
iMac-de-Blanca:~ blancalanda$
> Thanks
Thanks to you for the help!
Everything seems fine. Lets try building the blast database by hand,
could you try doing this and report back?
cd Databases/unite_taxonomy/
formatdb -i unite_ref_seqs_21nov2011.fasta -oT -pF -l
unite_ref_seqs_21nov2011.log
Cheers
Antonio González Peña <antg...@gmail.com> escribió:
You have reported two scenarios with formatdb:
1. it is not installed (your last reply)
iMac-de-Blanca:unite_taxonomy blancalanda$ formatdb -i
unite_ref_seqs_21nov2011.fasta -oT -pF -l unite_ref_seqs_21nov2011.log
-bash: formatdb: command not found
2. it is installed but wasn't run properly (when you replied to Greg)
iMac-de-Blanca:~ blancalanda$ formatdb
[formatdb 2.2.22] ERROR: No database name was specified
So my guess is that you are having a problem with your environment and
macqiime, can you try
macqiime formatdb -i unite_ref_seqs_21nov2011.fasta -oT -pF -l
unite_ref_seqs_21nov2011.log
Cheers
-----Mensaje original-----
From: Blanca Landa
Sent: Monday, January 30, 2012 9:02 PM
To: AntonioGonz�lez Pe�a ; ag2l...@uco.es
Cc: qiime...@googlegroups.com
Subject: Re: [qiime-forum 1.4.0] Re: Fungal analysis
I already tried before you suggested:
/usr/bin/macqiime: line 12: formatdb: command not found
iMac-de-Blanca:unite_taxonomy blancalanda$ unite_ref_seqs_21nov2011.log
I can try with the VM and see how it works?
El 1/30/12 8:50 PM, "Antonio Gonz�lez Pe�a" <antg...@gmail.com> escribi�:
cd ~/Databases/unite_taxonomy/
macqiime formatdb -i unite_ref_seqs_21nov2011.fasta -oT -pF -l
unite_ref_seqs_21nov2011.log
Greg
Sorry, I'm confused by the dots before the paths output from the "which" command... has anyone else ever seen dots before a path in this context in unix? Presumably it means that the path is starting at the currrent directory?
Cheers,
Jeff
echo $PATH
macqiime echo $PATH
Greg
First I tried to create the .ncbirc and .profile files as Jeff suggested me
in a previous email
This is my e-mail and the errors I found:
From: Blanca B. Landa
Sent: Thursday, January 26, 2012 10:17 PM
To: Jeffrey Werner
Cc: blanca...@csic.es
Subject: Re: help
Hi Jeff
Thanks for the fast answer.
I have followed your instructions and copy the blast-2.2.22 in my home
directory:
users/blancalanda
I have edited the .profile as this
export PATH=/blast-2.2.22/bin:${PATH}
but I do not receive any answer with the command
which blastall
I have also edited it like this
export PATH=/blancalanda/blast-2.2.22/bin:${PATH}
and I do not get any results with the command which blastall
I am doing something wrong?
When trying to make a directory in the terminal using mkdir it says
Permission denied,
I have been able to install all the programs that do not come with Macqiime
but I did not need any permit for installing things
Then, I read ways to create the .ncbirc files and decide to include the ./:
Hello again!
I solved it using this path:
./blast-2.2.22/
iMac-de-Blanca:~ blancalanda$ which blastall
./blast-2.2.22/bin/blastall
So, I think I should try to create new .ncbirc and .profile files ?
Sorry for this mess!
Blanca
-----Mensaje original-----
From: Jeff Werner
Sent: Wednesday, February 01, 2012 5:01 AM
To: qiime...@googlegroups.com
Subject: Re: [qiime-forum 1.4.0] Re: Fungal analysis
Hi Blanca,
pick_otus can take quite a while depending on what parameters you are
using. If you are doing uclust_ref, you can use
parallel_pick_otus_uclust_ref.py to speed up things by running it in
parallel.
Jose
-----Mensaje original-----
From: Jose Carlos Clemente
Sent: Wednesday, February 01, 2012 10:03 PM
can you try the following from the command line
java -Xmx1000M -cp "/macqiime/rdp_classifier_2.2/rdp_classifier-2.2.jar"
and send us the result?
Jose
iMac-de-Blanca:~ blancalanda$ java -Xmx1000M -cp
"/macqiime/rdp_classifier_2.2/rdp_classifier-2.2.jar
> Can you please send the result of print_qiime_config.py -t?
iMac-de-Blanca:~ blancalanda$ macqiime print_qiime_config.py -t
System information
==================
Platform: darwin
Python version: 2.7.1 (r271:86832, Dec 15 2011, 08:41:37) [GCC
4.0.1 (Apple Inc. build 5493)]
Python executable: /macqiime/bin/python
Dependency versions
===================
PyCogent version: 1.5.1
NumPy version: 1.5.1
matplotlib version: 1.1.0
QIIME library version: 1.4.0
QIIME script version: 1.4.0
PyNAST version (if installed): 1.1
RDP Classifier version (if installed): rdp_classifier-2.2.jar
QIIME config values
===================
blastmat_dir: None
topiaryexplorer_project_dir: None
pynast_template_alignment_fp: /macqiime/greengenes/core_set_aligned.fasta.imputed
cluster_jobs_fp: /macqiime/QIIME/bin/start_parallel_jobs.py
pynast_template_alignment_blastdb: None
assign_taxonomy_reference_seqs_fp: None
torque_queue: friendlyq
template_alignment_lanemask_fp: /macqiime/greengenes/lanemask_in_1s_and_0s
jobs_to_start: 1
cloud_environment: False
qiime_scripts_dir: /macqiime/QIIME/bin/
denoiser_min_per_core: 50
working_dir: None
python_exe_fp: /macqiime/bin/python
temp_dir: /tmp/
blastall_fp: blastall
seconds_to_sleep: 60
assign_taxonomy_id_to_taxonomy_fp: None
running checks:
test_FastTree_supported_version (__main__.Qiime_config)
FastTree is in path and version is supported ... ok
test_INFERNAL_supported_version (__main__.Qiime_config)
INFERNAL is in path and version is supported ... ok
test_ampliconnoise_install (__main__.Qiime_config)
AmpliconNoise install looks sane. ... FAIL
test_blast_supported_version (__main__.Qiime_config)
blast is in path and version is supported ... ok
test_blastall_fp (__main__.Qiime_config)
blastall_fp is set to a valid path ... ok
test_blastmat_dir (__main__.Qiime_config)
blastmat_dir is set to a valid path. ... ok
test_cdbtools_supported_version (__main__.Qiime_config)
cdbtools is in path and version is supported ... ok
test_cdhit_supported_version (__main__.Qiime_config)
cd-hit is in path and version is supported ... ok
test_chimeraSlayer_install (__main__.Qiime_config)
no obvious problems with ChimeraSlayer install ... ok
test_clearcut_supported_version (__main__.Qiime_config)
clearcut is in path and version is supported ... ok
test_cluster_jobs_fp (__main__.Qiime_config)
cluster_jobs_fp is set to a valid path and is executable ... ok
test_denoiser_supported_version (__main__.Qiime_config)
denoiser aligner is ready to use ... ok
test_for_obsolete_values (__main__.Qiime_config)
local qiime_config has no extra params ... ok
test_matplotlib_suported_version (__main__.Qiime_config)
maptplotlib version is supported ... ok
test_mothur_supported_version (__main__.Qiime_config)
mothur is in path and version is supported ... ok
test_muscle_supported_version (__main__.Qiime_config)
muscle is in path and version is supported ... ok
test_numpy_suported_version (__main__.Qiime_config)
numpy version is supported ... ok
test_pynast_suported_version (__main__.Qiime_config)
pynast version is supported ... ok
test_pynast_template_alignment_blastdb_fp (__main__.Qiime_config)
pynast_template_alignment_blastdb, if set, is set to a valid path ... ok
test_pynast_template_alignment_fp (__main__.Qiime_config)
pynast_template_alignment, if set, is set to a valid path ... ok
test_python_exe_fp (__main__.Qiime_config)
python_exe_fp is set to a working python env ... ok
test_python_supported_version (__main__.Qiime_config)
python is in path and version is supported ... ok
test_qiime_scripts_dir (__main__.Qiime_config)
qiime_scripts_dir, if set, is set to a valid path ... ok
test_raxmlHPC_supported_version (__main__.Qiime_config)
raxmlHPC is in path and version is supported ... FAIL
test_temp_dir (__main__.Qiime_config)
temp_dir, if set, is set to a valid path ... ok
test_template_alignment_lanemask_fp (__main__.Qiime_config)
template_alignment_lanemask, if set, is set to a valid path ... ok
test_uclust_supported_version (__main__.Qiime_config)
uclust is in path and version is supported ... ok
test_working_dir (__main__.Qiime_config)
working_dir, if set, is set to a valid path ... ok
======================================================================
FAIL: test_ampliconnoise_install (__main__.Qiime_config)
AmpliconNoise install looks sane.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/macqiime/QIIME/bin/print_qiime_config.py", line 116, in
test_ampliconnoise_install
"$PYRO_LOOKUP_FILE variable is not set. See %s for help." % url)
AssertionError: $PYRO_LOOKUP_FILE variable is not set. See
http://www.qiime.org/install/install.html#ampliconnoise-install for
help.
======================================================================
FAIL: test_raxmlHPC_supported_version (__main__.Qiime_config)
raxmlHPC is in path and version is supported
----------------------------------------------------------------------
Traceback (most recent call last):
File "/macqiime/QIIME/bin/print_qiime_config.py", line 526, in
test_raxmlHPC_supported_version
% ('.'.join(map(str,acceptable_version)), version_string))
AssertionError: Unsupported raxmlHPC version. 7.0.3 is required, but
running 7.0.4.
----------------------------------------------------------------------
Ran 28 tests in 0.590s
FAILED (failures=2)
iMac-de-Blanca:~ blancalanda$
THANKS
Here you have:
- make sure you have java 1.6 or greater: from the command line, type
java -version
- increase the memory allocated using the parameter --rdp_max_memory
Just to be sure, how was the file
unite_id_to_taxonomy_map_21nov2011.txt created?
Jose
iMac-de-Blanca:~ blancalanda$ java -version
java version "1.6.0_29"
Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-11M3527)
Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)
iMac-de-Blanca:~ blancalanda$
I am not sure how was the file created, I downloaded it from the files
available at qiime web page. I can see if there is some infer posted
in the forum
BB
I tried those files and I'm getting the same error. We think the issue
might be this was working with RDP 2.0 not with 2.2. I will check who
produced the file and get back to you asap with a solution. Sorry for
the inconvenience!
Jose