Combining Illumina Miseq and Hiseq preprocessed sequences

[Alain I.]

Hi developers and Qiimers!

I am studying bacterial communities from environmental samples. I have received raw and quality controlled sequence (preprocessed) data from the sequencing company in two batches: 40 and 60 samples from MiSeq and HiSeq platforms, respectively. I want to obtain a combined OTU table and run downstream analyses in QIIME. I have merged "quality controlled" sequences of both MiSeq and HiSeq outside QIIME and jumped to the pick_open_reference.py script. 

The first lines of the merged file (HX16S.fna) looks like:

>Y12D01_0

TGGGGATATTGGACAATGG....

>Y12D01_1

TGGGGAATAATTGGACAA...

>Y12D01_2

TGGGGAATTTTGGA...

(12D01 is the name of the first of 100 samples)

On page 390 of Navas-Molina et al. "Advancing our understanding of the human microbiome using QIIME." Methods in enzymology 531 (2013): 371, it says:

"QIIME can perform all the steps for generating the OTU table and the phylogenetic tree from the preprocessed data in a single command"....

and 

"For open-reference (run time 27 h on 20 processors):
pick_open_reference_otus.py -o $PWD/open_ref_otus -i $PWD/slout/seqs.fna -r $PWD/gg_12_10_otus/rep_set/97_otus.fasta -a -O 20"

I run a similar script (assuming QIIME defaults: uclust; and gg_13_8) on the combined file (HX16S.fna):

pick_open_reference_otus.py -i $PWD/02_Raw/HX16S.fna -o $PWD/03_Open_ref_picked_otus

It resulted into 6 folders (pynast_aligned_seqs; step1_otus; step2_otus; step3_otus; step4_otus and uclust_assigned_taxonomy) and 10 files (final_otu_map.txt; final_otu_map_mc2.txt, index.html; log file; new_refseqs.fna; otu_table_mc2.biom; otu_table_mc2_w_tax.biom; otu_table_mc2_w_tax_no_pynast_failures.biom; rep_set.fna; rep_set.tree).

1. Does this workflow make sense?

2. In the above Navas-Molina et al.2013 single script after preprocessing, where are chimeras removed?

I thank you in advance.

Alain

[Colin Brislawn]

Good morning Alain,

Yes, that workflow makes sense. This should produce the desired result. 

I believe that chimera checking must take place in a separate step, directly before OTU picking. I've cc'ed one of the qiime devs who can clarify.

Colin

[Alain I.]

Good morning dear Colin,

Thank you very much for your clarifications. 

I understand that in my current situation chimeras were not removed.

I think I have to re-run the script and use usearch61 algorithm method instead of the default uclust in order to detect chimeras.

Are the detected chimera automatically removed by usearch or I have to exclude them via make_otu_table.py -e chimeras?

Thank you very much!

[Colin Brislawn]

Hello Alain,

Correct; chimeras were not removed by this workflow.

Passing -m usearch61 would use usearch v6.1 for clustering, but I'm not sure if it would also do the chimera filtering step. I'm not super familiar with chimera filtering using qiime 1.9.1, which is why I've forwarded this to one of the qiime devs who can tell you more.

I'm glad you are looking into this and inspecting the details of this script. Chimera detection is super important, especially for accurate alpha diversity measurements.

Keep in touch,

Colin