Beta_significance.py using OTU table summarized by category Options

[JQL]

This is an old thread, but very informative (see link below). But it
has been deactivated so I am unable to reply to the thread itself. But
I have a questions.

I am trying to use summarize_taxa_through_plots.py instead of
summarize_OTU_by_cat.py. I believe both creat the same
cat_otu_table.txt when supplied with category information. But the
former script can do much more, for example, categoried OTU table by
taxa levels (e.g. genus) and plots.

If I want to do beta_significance tests on cat_otu_table_L6 (genus?),
how do I suppress the floating points?

thanks
John


http://groups.google.com/group/qiime-forum/browse_thread/thread/f80a0ded886009bf/5b1ae1efcc15a5a6?lnk=gst&q=beta_significance+justink#5b1ae1efcc15a5a6

[Greg Caporaso]

Hi John,

summarize_taxa.py -a

will output actual abundances rather than relative abundances. That
should get you want you want. I recommend using summarize_taxa.py here
instead of summarize_taxa_through_plots.py as you need relative
abundances for the plots.

Greg

[JQL]

Thank you Greg!

I ran the summarize_taxa.py with -a using the cat_otu_table.txt
(output from either the summarize_otu_by_category or
summarize_taxa_through_plots.py)
it produces categorized otu summary by various levels, say
cat_otu_table_L4.txt

then I trid to run beta_significance.py, it gaves errors (see below).
The same script works with cat_otu_table.txt.

I notice there are slight difference between the two input files.
The original one:
#Category OTU Counts-Grp
#OTU ID T1 T2 T3 Consensus Lineage
1994 0 0 1 k__Bacteria; p__Bacteroidetes;
c__Bacteroidia; o__Bacteroidales;
f__Bacteroidaceae; g__Bacteroides; s__

The L4:
Taxon T1 T2 T3
k__Bacteria;p__Actinobacteria;c__Actinobacteria;o__Actinomycetales
13 10 20
k__Bacteria;p__Actinobacteria;c__Actinobacteria;o__Bifidobacteriales
23 45 10

I wonder if that causes errors? Do I need to reformat the L4 file so
that it can be used for beta_signif.py?
thanks,
John





Traceback (most recent call last):
File "/usr/bin/beta_significance.py", line 149, in <module>
main()
File "/usr/bin/beta_significance.py", line 133, in main
weighted=True, num_iters=opts.num_iters, verbose=opts.verbose)
File "/usr/lib64/python2.6/site-packages/cogent/maths/unifrac/
fast_unifrac.py", line 99, in fast_unifrac_permutations_file
is_symmetric=True, modes=[UNIFRAC_DIST_MATRIX])
File "/usr/lib64/python2.6/site-packages/cogent/maths/unifrac/
fast_unifrac.py", line 466, in fast_unifrac
envs, count_array, unique_envs, env_to_index, node_to_index,
env_names, branch_lengths, nodes, t = _fast_unifrac_setup(t, envs,
make_subtree)
File "/usr/lib64/python2.6/site-packages/cogent/maths/unifrac/
fast_unifrac.py", line 194, in _fast_unifrac_setup
raise ValueError, "No valid samples/environments found. Check
whether tree tips match otus/taxa present in samples/environments"
ValueError: No valid samples/environments found. Check whether tree
tips match otus/taxa present in samples/environments
Traceback (most recent call last):
File "/usr/bin/beta_significance.py", line 149, in <module>
main()
File "/usr/bin/beta_significance.py", line 133, in main
b weighted=True, num_iters=opts.num_iters, verbose=opts.verbose)
File "/usr/lib64/python2.6/site-packages/cogent/maths/unifrac/
fast_unifrac.py", line 99, in fast_unifrac_permutations_file
e is_symmetric=True, modes=[UNIFRAC_DIST_MATRIX])
File "/usr/lib64/python2.6/site-packages/cogent/maths/unifrac/
fast_unifrac.py", line 466, in fast_unifrac
t envs, count_array, unique_envs, env_to_index, node_to_index,
env_names, branch_lengths, nodes, t = _fast_unifrac_setup(t, envs,
make_subtree)
a File "/usr/lib64/python2.6/site-packages/cogent/maths/unifrac/
fast_unifrac.py", line 194, in _fast_unifrac_setup
raise ValueError, "No valid samples/environments found. Check
whether tree tips match otus/taxa present in samples/environmentsjobs

[Antonio González Peña]

Hi John,

In the development version of QIIME the convert_biom.py script will
change your summary table to a valid OTU table in biom format using
this command:
convert_biom.py -i cat_otu_table_L4.txt -o cat_otu_table_L4.biom
--biom_table_type "taxon table"

Now beware that you will not be able to do phylogenetic alpha or beta
analysis on this table because the current methods rely on the names
on the tip of the tree and not in clades (basically what you are doing
when you summarize or split the otu table in taxonomic levels) so the
suggestion is to use only non phylogenetic metrics for this kind of
analyses. By the way, I have added a feature request in QIIME:
https://sourceforge.net/tracker/?func=detail&aid=3510927&group_id=272178&atid=1157167

Cheers

--
Antonio González Peña
Research Assistant, Knight Lab
University of Colorado at Boulder
https://chem.colorado.edu/knightgroup/

[JQL]

Thank you Antonio.

It tried p-test, which gave the same error. I guess P test is also
phylogenetic metrics based test.

It seems that there are only three tests availalbe (two unifrac, and p
test), all phylogenetic tree based. When you said non-phylogenetic
metrics, which else did you mean?
-s, --significance_test
Significance test to use, options are ‘unweighted_unifrac’,
‘weighted_unifrac’, or ‘p-test’

When I was reading uniFrac tutorial, it mentioned "lineage specific
analysis", which allows us to ask which lineages contribute to
differences between those samples. I thought the beta-significance
test can just do that. We first ask if the treatments are different by
using cat_otu_table.txt, then we ask which lineage contributes to the
difference by using cat_otu_table_L4.txt. Now that it doesn't work at
lineage level, what does lineage specific analysis exactly mean? Or it
means something totally different?

thanks
John

On Mar 25, 12:38 am, Antonio González Peña <antgo...@gmail.com> wrote:
> Hi John,
>
> In the development version of QIIME the convert_biom.py script will
> change your summary table to a valid OTU table in biom format using
> this command:
> convert_biom.py -i cat_otu_table_L4.txt -o cat_otu_table_L4.biom
> --biom_table_type "taxon table"
>
> Now beware that you will not be able to do phylogenetic alpha or beta
> analysis on this table because the current methods rely on the names
> on the tip of the tree and not in clades (basically what you are doing
> when you summarize or split the otu table in taxonomic levels) so the
> suggestion is to use only non phylogenetic metrics for this kind of

> analyses. By the way, I have added a feature request in QIIME:https://sourceforge.net/tracker/?func=detail&aid=3510927&group_id=272...

> University of Colorado at Boulderhttps://chem.colorado.edu/knightgroup/- Hide quoted text -
>
> - Show quoted text -

[Antonio González Peña]

Hi John,

I was thinking in non phylogenetic methods in alpha, for example Chao1
or observed species vs. PD, or beta, in example Bray Curtis vs.
UniFrac, analyses.

Now when you use unifrac in beta_diversity.py you are assessing the
community differences between samples using the phylogenetic tree
(which OTUs/lineages contribute to the difference based on the tree),
step 6 of the tutorial http://bmf2.colorado.edu/unifrac/tutorial.psp ;
when you use beta_significance.py you are testing whether the pattern
you are seeing is significant or not using unifrac, step 7. In the current
implementations you need to pass the full OTU table and the full tree,
not summarized.

Hope this helps.

[JQL]

Hi Antonio,

In the tutorial, step 10, lineage specific analysis, is it implemented
in Qiime? or one has to use the unifrac web application to do that?

thanks
John

On Mar 27, 12:49 am, Antonio González Peña <antgo...@gmail.com> wrote:
> Hi John,
>

> I was thinking in non phylogenetic methods in alpha, for example Chao1
> or observed species vs. PD, or beta, in example Bray Curtis vs.
> UniFrac, analyses.
>
> Now when you use unifrac in beta_diversity.py you are assessing the
> community differences between samples using the phylogenetic tree
> (which OTUs/lineages contribute to the difference based on the tree),

> step 6 of the tutorialhttp://bmf2.colorado.edu/unifrac/tutorial.psp;

> >> University of Colorado at Boulderhttps://chem.colorado.edu/knightgroup/-Hide quoted text -

[Antonio González Peña]

Hi John,

That one is otu_category_significance.py

Cheers.

[JQL]

that makes sense! thank you again!
John

On Mar 30, 7:16 pm, Antonio González Peña <antgo...@gmail.com> wrote:
> Hi John,
>

> >> >> University of Colorado at Boulderhttps://chem.colorado.edu/knightgroup/-Hidequoted text -