Errors running diversity analysis after summarize_otu_by_cat.py

[Dylan Bodington]

Hi,

I'm using summarize_otu_by_cat.py to create separate otu tables based on different categories in the mapping file, but my analysis of the sumarized otu tables fails at the beta diversity step when using the original mapping file:

summarize_otu_by_cat.py -i map.txt -c 99/otu/otu_table.biom -o 99/otu/otu_table_Set.biom -m Set

beta_diversity_through_plots.py -i 99/otu/otu_table_Set.biom -o 99/beta/Set -m map.txt -p parameters.txt -t 99/otu/rep_set.tre -a

Traceback (most recent call last):
  File "/opt/qiime/qiime-1.5.0-release/bin/beta_diversity_through_plots.py", line 166, in <module>
    main()
  File "/opt/qiime/qiime-1.5.0-release/bin/beta_diversity_through_plots.py", line 163, in main
    status_update_callback=status_update_callback)
  File "/opt/qiime/qiime-1.5.0-release/lib/qiime/workflow.py", line 867, in run_beta_diversity_through_plots
    close_logger_on_success=close_logger_on_success)
  File "/opt/qiime/qiime-1.5.0-release/lib/qiime/workflow.py", line 135, in call_commands_serially
    raise WorkflowError, msg
qiime.workflow.WorkflowError:

*** ERROR RAISED DURING STEP: Make 3D plots (continuous coloring, bray_curtis)
Command run was:
 /opt/qiime/python-2.7.1-release/bin/python /opt/qiime/qiime/bin/make_3d_plots.py -p 97/beta/Set/prefs.txt -i 97/beta/Set/bray_curtis_pc.txt -o 97/beta/Set/bray_curtis_3d_continuous/ -m map.csv --n_taxa_keep 5 --ellipsoid_smoothness 1
Command returned exit status: 1
Stdout:

Error: OTU table and mapping file had no samples in common


Stderr


Everything runs fine when using the original otu table and mapping file. I thought to use pool_by_metadata.py to create the individual mapping files for beta diversity and jackknife analysis of the summarized otu tables, but I can't seem to find pool_by_metadata.py in my new 1.5.0 installation, though it is still listed in the documentation.

Dylan Bodington
Tokyo Institute of Technology

[Jesse Stombaugh]

Hello Dylan,

The reason is that you summarized the OTU table by a category. The scripts assume those ids are SampleIDs which is where the confusion occurs. What you need to do is to generate a new mapping file, where the SampleID column only contains the Category values you have in the summarized OTU table.  Once you have that updated mapping file, you can pass it to the workflow scripts you are working with.

-Jesse

--
Jesse Stombaugh, Ph.D.
Research Associate
University of Colorado, Boulder

[Dylan Bodington]

Hi Jesse,

Yes, I realized that was the issue, but I was looking for way to automate the generation of the new mapping files, as the old pool_by_metadata.py did, so I can repeat the process on a number of categories without having to manually generate mapping files each time.

I have now found the feature request on this very problem http://sourceforge.net/tracker/?func=detail&aid=3510927&group_id=272178&atid=1157167 but I'm not sure if this request will cover the generation of new mapping files.

Dylan

[Antonio González Peña]

Hi Dylan,

We know that the feature you pointed out could be useful but we
haven't being able to address it yet, our current solutions can be
found in here:
https://groups.google.com/group/qiime-forum/browse_thread/thread/2d3a2476e36db39e

In my opinion this is not a trivial problem because the script will
need to create a new mapping file, a new collapsed tree, and the biom
format file.

By the way, we always welcome new code (after revision and that most
comply with the QIIME standards, more info
http://qiime.org/developer/index.html), if you are intereted.

Cheers

--
Antonio González Peña
Research Assistant, Knight Lab
University of Colorado at Boulder
https://chem.colorado.edu/knightgroup/

[Dylan Bodington]

Hi Antonio,

This is as good an incentive as any to level up my python. After being inspired to code in Fort Collins last year, I was halted by endless diapers and lack of sleep.

I'll give it a go, but I can't promise it'll be clean.

Dylan

[Antonio González Peña]

Good to hear that Dylan, if you need pointers or code examples send me
an email to antgonza @ gmail. com

[Hagen]

Hi guys,

Sorry for intruding this thread, but I ran into a similar issue: I want to generate beta div plots based on UniFrac distance from a collapsed otu table. I have changed the IDs in the mapping file so they are compatible with the otu table, but is there any way to generate a new, collapsed tree for diversity analysis? Or should I just settle with a BrayCurtis distance instead?

Br
Live Hagen

[Greg Caporaso]

Hello,

You'll need to use a non-phylogenetic diversity metric for this. Since the taxonomy (unfortunately) does not map directly on to phylogeny, you can't generate a collapsed tree to reflect the taxonomy. I would recommend Bray-Curtis for this.

Greg

[Hagen]

Ok, thanks!

Br,
LH