filter_samples_from_otu_table.py: command not found

[lucid]

Hey all,

I have recently started using QIIME for my research involving
pyrosequencing results (130k sequences). Just as I thought I was
getting the hang of it, I hit a hiccup in my workflow..

Just to give a little background, this morning I figured out how to
find out which sequences are most significant for my hypothesis
(otu_category_significance.py) and even brushed up on statistics in
the meantime to avoid being completely lost! The positive results led
me back to the original otu table containing all the samples. What
truly interests me now is isolating the significant OTUs in the otu
table with the function filter_samples_from_otu_table.py in order to
use the OTUs that are randomly throughout the otu table to construct
figures and use for other downstream analyses.

For some reason whenever I attempt this script
(filter_samples_from_otu_table.py), I get a "command not found"
message. I know that I typed it correctly, as all the other commands
I've used thus far with QIIME have all worked. Perhaps this is a
simple path issue? Is there something particularly different with this
script in relation to the others? I am a mac user and somewhat savvy
regarding the terminal, but am still learning about particularities of
path, etc. Any type of suggestion along those lines would be greatly
appreciated so I can keep on rolling with these results! Thanks so
much!

Josh (aka lucid)

[Tony Walters]

Hello Josh,

It's possible that this is a version issue (the names of the scripts for filtering OTU tables changed from 1.4.0 to 1.5.0).

Could you run print_qiime_config.py and print the results?  This will tell us what version of the software you have.

-Tony

[lucid]

Hey Tony,

Thanks for the heads up. Looks like I'm running the 1.4.0 version!

System information
==================
Platform: darwin
Python version: 2.7.1 (r271:86832, Dec 15 2011, 08:41:37) [GCC
4.0.1 (Apple Inc. build 5493)]
Python executable: /macqiime/bin/python

Dependency versions
===================
PyCogent version: 1.5.1
NumPy version: 1.5.1
matplotlib version: 1.1.0
QIIME library version: 1.4.0
QIIME script version: 1.4.0
PyNAST version (if installed): 1.1
RDP Classifier version (if installed): rdp_classifier-2.2.jar

QIIME config values
===================
blastmat_dir: None
topiaryexplorer_project_dir: None
pynast_template_alignment_fp: /macqiime/greengenes/
core_set_aligned.fasta.imputed
cluster_jobs_fp: /macqiime/QIIME/bin/
start_parallel_jobs.py
pynast_template_alignment_blastdb: None
assign_taxonomy_reference_seqs_fp: None
torque_queue: friendlyq
template_alignment_lanemask_fp: /macqiime/greengenes/
lanemask_in_1s_and_0s
jobs_to_start: 1
cloud_environment: False
qiime_scripts_dir: /macqiime/QIIME/bin/
denoiser_min_per_core: 50
working_dir: None
python_exe_fp: /macqiime/bin/python
temp_dir: /tmp/
blastall_fp: blastall
seconds_to_sleep: 60
assign_taxonomy_id_to_taxonomy_fp: None

[Tony Walters]

Hello again Josh,

I think (other QIIMEers can correct me if I'm wrong) that there isn't the functionality in place for filtering specifically by OTU IDs in QIIME 1.4.0.  Unfortunately, you may need to manually filter the OTUs from the OTU table.  If you open the OTU table in Excel, copy the top row into another spreadsheet, and find each row that starts with the OTU ids in question to copy to the new OTU table.

You can filter by taxonomy if you happened to look at category significance after summarizing taxonomic categories together (it sounds as though you ran it for the individual OTUs though rather than by taxonomy, so this might not work for you).

Sorry I don't have a more elegant answer for this.

-Tony

[lucid]

Hey Tony,

Thanks again for your help - I wouldn't have known that there was a
more up-to-date version out there. I just downloaded 1.5.0 (http://
www.wernerlab.org/software/macqiime) for mac and it appears to be
working fine.. just need to get used to some of the new commands -
haven't tested the original script yet though. Thinks are looking up,
thanks again!

cheers

Josh

[Tony Walters]

Hello Josh,

You probably are already getting this software, but you'll need to update to the new .biom format for OTU tables with 1.5.0 (http://biom-format.org/).  The scripts will balk at the tab-separated OTU table.

-Tony

[lucid]

Thanks for software info - I got the biom-format up and running. I did
run into another hitch though after converting the original OTU table
from .txt format to biom - when I attempt to filter using the
filter_samples_from_otu_table.py command
[filter_samples_from_otu_table.py -i otus/otu_table.biom -o
otu_table_paired_T_05.biom --sample_id_fp otus/Map_keep.txt].. I get
this response:

Traceback (most recent call last):
File "/macqiime/QIIME/bin/filter_samples_from_otu_table.py", line
127, in <module>
main()
File "/macqiime/QIIME/bin/filter_samples_from_otu_table.py", line
113, in main
max_count)
File "/macqiime/lib/python2.7/site-packages/qiime/filter.py", line
220, in filter_samples_from_otu_table
return otu_table.filterSamples(filter_f)
File "/macqiime/lib/python2.7/site-packages/biom/table.py", line
755, in filterSamples
raise TableException, "All samples filtered out!"
biom.exception.TableException: All samples filtered out!

Which generates a biom file that contains nothing! Not sure if its the
way I formatted my "to keep" txt file, where I just listed the OTUs
under a column "OTU ID"...

On Jun 5, 7:13 pm, Tony Walters <william.a.walt...@gmail.com> wrote:
> Hello Josh,
>

[Greg Caporaso]

Hi Josh,
You mention that you're listing OTU ids in your file, but the script
that you're using is filtering samples, so you should have sample ids
in your file. Was that a typo in your post? If not, you want to be
using the filter_otus_from_otu_table.py script, not
filter_samples_from_otu_table.py. You'll also need to pass
--negate_ids_to_exclude if you want to keep the OTUs in your
Map_keep.txt file (rather than exclude them).

Greg

[lucid]

Thanks Greg! that finally did the trick - I was trying to work with
OTUs and somehow got them mixed up with the samples in code! Now I'm
booking for sure!!

[Greg Caporaso]

Great, glad it's working for you!

Greg