dereplication step and abundance data via OTUpipe

[Rachel S]

Hello,
I have a question about how OTUpipe works. I have a 16S analysis (one
SFF file, about 200,000 sequences, eighteen samples) which I would
like to denoise, chimera filter, and otu-pick using Otupipe. I am
interested in the diversity and also the structure of the communities,
so abundance data in the form of percent of total (good) reads
represented by a taxon per sample is something I want to be sure I am
getting reliable data on. As I understand it, OTUpipe has four steps
where the abundance of sequences is reduced: initial dereplication,
denoising, merging of nonchimeras, and sorting by abundance (removing
clusters with less than minsize=x). Obviously, I dont mind losing
noisy, chimeric, or low-abundance sequences, but what about the
dereplication step? If there were really fifty identical sequences
that were dereplicated into one, and that one makes it through all the
denoising, chimera filtering, and low-abundance-of-similar-sequences
(sorry to say that so clumsily!) steps, is it finally "reinflated" to
fifty sequences at some point? Is that information appended somehow to
the OTU map file?

thank you very much for your time and advice,

Rachel Strickman

[Tony Walters]

Hello Rachel,

You don't lose those sequences that are dereplicated as exact matches, unless they happen to be flagged as chimeric.  What will happen is that in one of the final steps, where "good" sequences remain (i.e., non-low abundance, non-chimeric), the original sequence file passed through, in which each sequence globally aligned against the "good" sequences to connect the original sequence ID to the "good" sequences (corresponds to step 11 of the process listed here: http://www.qiime.org/tutorials/usearch_quality_filter.html#step-11-classify-reads), so any sequences that were dereplicated initially should not be lost, unless they happen to be chimeric or low abundance.

Hope this helps,

Tony Walters

[Rachel S]

Hello,
Thank you very much for clearing this up. If you have time, what is
your opinion on the great 454 abundance data debate? Amend et al 2010
(and many others) rightly point out its limitations... but do you
think comparing abundance data at the order rather than species level,
along with less-permissive denoising, chimera filtering, and OTU
picking steps, would generate reliable enough data? I have been
playing around with the MINSIZE setting in OTUpipe and find it has a
BIG effect on the final number of OTUs... I am interested in the rare
biosphere, but not in imaginary diversity! What is your perspective?

thank you for any advice!

Rachel

On May 29, 5:14 pm, Tony Walters <william.a.walt...@gmail.com> wrote:
> Hello Rachel,
>
> You don't lose those sequences that are dereplicated as exact matches,
> unless they happen to be flagged as chimeric.  What will happen is that in
> one of the final steps, where "good" sequences remain (i.e., non-low
> abundance, non-chimeric), the original sequence file passed through, in
> which each sequence globally aligned against the "good" sequences to
> connect the original sequence ID to the "good" sequences (corresponds to

> step 11 of the process listed here:http://www.qiime.org/tutorials/usearch_quality_filter.html#step-11-cl...),

> so any sequences that were dereplicated initially should not be lost,
> unless they happen to be chimeric or low abundance.
>
> Hope this helps,
> Tony Walters
>
> On Tue, May 29, 2012 at 3:00 PM, Rachel S

> <rachel.strick...@googlemail.com>wrote:

[Tony Walters]

Hello Rachel,

I would say this depends upon the environment (a highly diverse soil environment probably has more important rare OTUs than say a skin community) and the sequencing abundance itself (you may want filter out singletons for samples with about 1000 seqs, maybe double or tripletons for some with 5000 sequences, and maybe disable size filtering altogether if you only had a few hundred sequences per sample).

The safest bet would be to do formal denoising, rather than OTUPipe, if you want to feel more confidant about rare OTUs, despite it taking more computational power.  As far as clustering a lower percent identity (e.g. 94% rather than 97%) I'm not really sure what effect that would have on chimera detection, although it would notionally limit some of the rare OTU filtering.

Hope this helps,

Tony Walters