Alpha diversity : evenness

[MarineLanda]

Hi everyone,

My question is not really a technical one.

With the alpha_diversity.py script, a lot of different metrics are
available. I understand why the whereabouts of each of them isn't the
point of the tut, but I could use some help here.

I'm not a pure ecologist and I'm wondering, among those metrics, which
one(s) would be the most appropriate to measure evenness.

If someone here has a clue about this, thanks a lot!!

Marine

[Jeff Werner]

Hi Marine,

The "equitability" measure is a metric of evenness on a scale of 0 to 1, where 1 is perfectly even.  I'm not sure what the formula is for this -- QIIME folks?  However, if you're doing fairly deep sequencing and using large rarefaction levels, the precision of the equitability metric can be problematic, because most of your numbers end up being something like 0.998, 0.986, etc.  Another useful metric that we've been using a lot is the Gini Coefficient (e.g., see Wittebolle et al, 2009 :  http://www.nature.com/nature/journal/v458/n7238/full/nature07840.html ). However, this metric is not part of QIIME, and you may have to make some very large spreadsheets and do a little math to calculate it yourself, based on sorted OTU tables.

And, is "simpson_e" also a measure of evenness, QIIME folks?  I haven't used that one...

Cheers,
Jeff

[justink]

For evenness, some options are equitability (in qiime, it's defined as
(the shannon entropy) / log_2(the number of observed species), simpson
(defined in qiime as 1-dominance), or the gini index, which sadly
isn't in qiime yet, but is somewhere on our to do list.


some useful resources are:

http://folk.uio.no/ohammer/past/diversity.html
and
http://www.pisces-conservation.com/sdrhelp/


and of course books like magurran 2004, or Pielou 1975.


And simpson_e as used in QIIME is the reciprocal simpson index defined
in http://www.pisces-conservation.com/sdrhelp/.

If you're comfortable looking at python code, most of the formulas for
alpha diversity (evenness and otherwise) are in pycogent: cogent/maths/
stats/alpha_diversity.py. Usually that code is called by QIIME as-is.

[MarineLanda]

Thank you so much for your help, this is very useful for me.

Just out of curiosity, someone told me that there were ways to
calculate the error on those alpha diversity indices (Chao1, Shannon,
etc). Is that something you are going to add to Qiime someday? Do you
think the values cannot be used if you don't have the error that goes
with them?

Thanks a lot

Marine

On 8 sep, 19:51, justink <justi...@gmail.com> wrote:
> For evenness, some options are equitability (in qiime, it's defined as
> (the shannon entropy) / log_2(the number of observed species), simpson
> (defined in qiime as 1-dominance), or the gini index, which sadly
> isn't in qiime yet, but is somewhere on our to do list.
>
> some useful resources are:
>
> http://folk.uio.no/ohammer/past/diversity.html

> andhttp://www.pisces-conservation.com/sdrhelp/

>
> and of course books like magurran 2004, or Pielou 1975.
>
> And simpson_e as used in QIIME is the reciprocal simpson index defined

> inhttp://www.pisces-conservation.com/sdrhelp/.
>
> If you're comfortable looking at python code, most of the formulas foralphadiversity(evenness and otherwise) are in pycogent: cogent/maths/

[justink]

The chao1 has confidence intervals in qiime, check out
alpha_diversity.py -m chao1_confidence ...

I'm not sure for shannon, but I think bootstrap/jackknife tests are
appropriate, see the qiime overview tutorial for more details on how
to do rarefactions / jackknifing.

-Justin

[mike]

I am running alpha diversity metrics in QIIME, most notably shannon's
and equitability (evenness). I am also interested in calculating the
upper and lower 95% confidense intervals. Justin implied it was
possible in QIIME and that there was documentation in the overview
tutorial. Could someone please clarify whether how this is possible
in QIIME? Thanks for your help.

Mike

[Michael Matiasek]

I am looking through my output files to the alpha_rarefaction.py
workflow script and in the alpha_rarefaction_plots/average_tables
folder there are a bunch of text files which seem to have what I am
looking for. In this folder, I in the "shannon treatment" file I can
see the shannon indicies that are calculated and their error at each
iteration. Is the error 0.05%? and is the shannon value in the last
interation my reported value? thanks for clarifying this!

Mike

[justink]

Just a quick clarification: the 'error' listed in
alpha_rarefaction_plots/average_tables refers to the variation across
communities. Not the variation [within a community and across
rarefactions/jackknife replicates].

If you're looking for confidence intervals on a single sample/
community and across rarefactions/jackknife replicates, see a file
such as alpha_div_collated/observed_species.txt:

sequences per sample iteration PC.354
alpha_rarefaction_10_0.txt 10 0 8
alpha_rarefaction_10_1.txt 10 1 8
alpha_rarefaction_10_2.txt 10 2 9
...

In this example, repeated sampling yielded 8,8, and 9 species For 95%
confidence intervals I might run 100 iterations, exclude the highest 3
values and lowest 3 values, and quote the max and min of the remaining
94 iterations.

It'd be nice if this were more straightforward for sure - sorry about
that.

[Dylan Bodington]

Hi,

I'm a little confused by the calculation of the reciprocal Simpson's index. From my qiime output, it seems simpsons is calculated as 1-dominance and reciprocal_simpsons is calculated as 1/simpsons ie 1/(1-dominance). As far as I know, reciprocal Simpson's should be calculated as 1/dominance as described by http://www.countrysideinfo.co.uk/simpsons.htm

Dylan Bodington
Tokyo Institute of Technology

[justink]

I agree, that's misleading. Thanks for bringing that to our attention. Currently in qiime, reciprocal_simpson is in fact calculated as 1/(simpson's index), or 1/(1-D). The svn version of qiime has, and future releases of qiime will also have a 'simpson_reciprocal' measure, which is 1/D, following the definition at http://www.countrysideinfo.co.uk/simpsons.htm among other places.

Sorry about that.