No taxa in otu_table from closed ref OTU picking

[Paul]

I am having problems getting taxonomy assignments for my .biom tables.  The command I used was as follows:

pick_closed_reference_otus.py --parallel -O 10 -i seqs.fna -o 16SDDucrCJune_19_2013taxa/ -r gg_13_5.fasta

-t gg_13_5_taxonomy.txt

where the reference is the latest greengenes file and the -t points to the associated greengenes taxonomy.txt file.  When I run per_library_stats.py this is the outcome

per_library_stats.py -i OTU_full_taxa.biom -o statsNum samples: 48
Num otus: 19604
Num observations (sequences): 9722937.0
Table density (fraction of non-zero values): 0.0995

Seqs/sample summary:
 Min: 38878.0
 Max: 345039.0
 Median: 195587.0
 Mean: 202561.1875
 Std. dev.: 71749.5468144
 Median Absolute Deviation: 47055.5
 Default even sampling depth in
  core_qiime_analyses.py (just a suggestion): 107967.0
 Sample Metadata Categories: None provided
 Observation Metadata Categories: None provided

Seqs/sample detail:
 3: 38878.0
 44: 73125.0
 36: 76103.0
 17: 107967.0
 26: 111650.0
 2: 112164.0
 45: 116854.0
 9: 120217.0
 19: 134811.0
 12: 147021.0
 8: 152398.0
 40: 158319.0
 10: 160512.0
 32: 162254.0
 21: 163782.0
 25: 164074.0
 5: 167746.0
 43: 172781.0
 30: 177045.0
 18: 179168.0
 47: 184489.0
 15: 186813.0
 27: 188067.0
 33: 192795.0
 7: 198379.0
 22: 205504.0
 14: 205661.0
 16: 208731.0
 39: 213181.0
 28: 216367.0
 4: 222058.0
 41: 240064.0
 24: 240187.0
 13: 241132.0
 23: 260324.0
 37: 262419.0
 11: 264712.0
 48: 265596.0
 38: 267349.0
 6: 277315.0
 1: 291987.0
 29: 292299.0
 46: 301143.0
 35: 301402.0
 42: 308366.0
 31: 319616.0
 34: 325073.0
 20: 345039.0

Obviously I have plenty of depth, but the fact that it still says Observation metadata: None provided concerns me.  It is my understanding that if the taxonomy data is included that should list 'Taxonomy'.  When I try to use this file downstream in summarize_taxa_through_plots.py I get an error

Traceback (most recent call last):
  File "/macqiime/QIIME/bin/summarize_taxa_through_plots.py", line 129, in <module>
    main()
  File "/macqiime/QIIME/bin/summarize_taxa_through_plots.py", line 126, in main
    status_update_callback=status_update_callback)
  File "/macqiime/lib/python2.7/site-packages/qiime/workflow.py", line 1780, in run_summarize_taxa_through_plots
    close_logger_on_success=close_logger_on_success)
  File "/macqiime/lib/python2.7/site-packages/qiime/workflow.py", line 135, in call_commands_serially
    raise WorkflowError, msg
qiime.workflow.WorkflowError:

*** ERROR RAISED DURING STEP: Summarize Taxonomy
Command run was:
 /macqiime/bin/python /macqiime/QIIME/bin//summarize_taxa.py -i /Users/pplummer/16S_DD_Qiime/OTU_full_taxa.biom -o /Users/pplummer/16S_DD_Qiime/taxa_summary_by_stage
Command returned exit status: 1
Stdout:

Stderr
Traceback (most recent call last):
  File "/macqiime/QIIME/bin//summarize_taxa.py", line 179, in <module>
    main()
  File "/macqiime/QIIME/bin//summarize_taxa.py", line 172, in main
    md_identifier)
  File "/macqiime/lib/python2.7/site-packages/qiime/summarize_taxa.py", line 42, in make_summary
    md_identifier)
  File "/macqiime/lib/python2.7/site-packages/qiime/summarize_taxa.py", line 85, in sum_counts_by_consensus
    if md_identifier not in otu_metadata:
TypeError: argument of type 'NoneType' is not iterable


I assume this error is due to the taxonomy missing.  Any thoughts on what I am doing incorrectly? Is the -t taxa.txt supposed to be a different file than one from greengenes? 

On a related note I would really like to take these to the genus level.  I realize this is risky but would like to see what we get.  Will the greengenes reference above let me get to genus or do I need a different file? Any input greatly appreciated.

Thanks, Paul

[Tony Walters]

Hello Paul,

I think you might be pointing to the wrong reference/taxonomy file. Can you go to http://greengenes.secondgenome.com/downloads/database/13_5 and download the gg_13_5_otus.tar.gz file? From this one, can you point to the /rep_set/97_otus.fasta for the -r parameter and -t /taxonomy/97_otu_taxonomy.txt for the pick_closed_reference_otus.py

Note that the taxa strings go down to species, but, many of these aren't named species (e.g. "s__").

The resulting OTU table should have the taxonomy strings in it-if you use convert_biom.py (http://biom-format.org/documentation/biom_conversion.html), use the -b --header_key taxonomy conversion option as shown under the "Convert biom format to classic format" example.

-Tony

--
 
---
You received this message because you are subscribed to the Google Groups "Qiime Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qiime-forum...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Paul]

Thanks so much Tony.  I will give it a try and let you know.  I appreciate your help.  Paul

[Paul]

Tony,
 Still seem to be failing.  I downloaded the tarball you suggested and opened it in the working directory

Ran the following script on a subset (2,000 reads) of the dataset to test

pick_closed_reference_otus.py --parallel -O 10 -i seqs2k.fna -o 16SDDucrCJune_19_2013taxanewgg/ -r gg_13_5_otus/rep_set/97_otus.fasta
-t gg_13_5_otus/taxonomy/97_otu_taxonomy.txt

And then recheck the per_library _stats.py

still no evidence of taxonomy

per_library_stats.py -i OTU_2k_taxanewgg.biom -o stats
Num samples: 48
Num otus: 140
Num observations (sequences): 616.0
Table density (fraction of non-zero values): 0.0501

Seqs/sample summary:
 Min: 1.0
 Max: 30.0
 Median: 11.0
 Mean: 12.8333333333
 Std. dev.: 6.62486897145
 Median Absolute Deviation: 4.0

 Default even sampling depth in

  core_qiime_analyses.py (just a suggestion): 3.0

 Sample Metadata Categories: None provided
 Observation Metadata Categories: None provided

Seqs/sample detail:

 3: 1.0
 36: 3.0
 47: 3.0
 10: 4.0
 17: 5.0
 19: 6.0
 9: 6.0
 2: 7.0
 21: 7.0
 44: 7.0
 25: 8.0
 43: 8.0
 45: 8.0
 12: 9.0
 27: 9.0
 40: 9.0
 13: 10.0
 15: 10.0
 16: 10.0
 32: 10.0
 41: 10.0
 22: 11.0
 23: 11.0
 24: 11.0
 26: 11.0
 18: 12.0
 8: 12.0
 30: 13.0
 48: 13.0
 28: 14.0
 7: 14.0
 33: 15.0
 38: 15.0
 4: 15.0
 42: 15.0
 5: 17.0
 29: 18.0
 37: 18.0
 39: 18.0
 11: 20.0
 14: 21.0
 31: 21.0
 34: 21.0
 46: 22.0
 1: 26.0
 20: 26.0
 6: 26.0
 35: 30.0

Any additional thoughts or ideas?

Thanks so much, again.

[Tony Walters]

Paul, did you convert to the tab separated text format? It's easier to observe the taxonomies with this (under the 'taxonomy' column).

[Paul]

Yes, I ran this command

convert_biom.py -i OTU_2k_taxanewgg2.biom -o table.from_biom_w_taxonomy.txt -b --header_key taxonomy

When I look at the output in excel there is no taxonomy file

On Wednesday, June 19, 2013 2:36:13 PM UTC-5, Paul wrote:

[Tony Walters]

Paul, can you type:

head table.from_biom_w_taxonomy.txt > sample_otu_table.txt

and post the sample_otu_table.txt file?

-Tony

--

[Paul]

You bet.  I sure appreciate the help.

It should be attached.

[Tony Walters]

Let's take a step back, can you look in the output folder of the workflow you called earlier (16SDDucrCJune_19_2013taxanewgg/) and send the OTU table that's there to me (william(dot)a(dot)walters(at)gmail(dot)com)

On Wed, Jun 19, 2013 at 6:17 PM, Paul <pplu...@iastate.edu> wrote:

You bet.  I sure appreciate the help.

It should be attached.

[Tony Walters]

There should be a .log file in that output folder as well-can you send that file?