Found double ID 3 (Extending OTU Seeds step)

[qinglong wu]

Dear Falk,

I got a error during the lotus run:

Command:

perl /export/apps/lotus_pipeline/lotus.pl -i ./Stitched_reads_header_renamed/ -map ./CDUO1_LotuS_filtering_map.txt -s ./sdm_miSeq_stitched_reads.txt -saveDemultiplex 2 -o ./LotuS_filtering_v2/ -c /export/apps/lotus_pipeline/lOTUs.cfg

My raw reads format (example in one of sample seq reads):

@1

ATACGTAGGTGGCAAGCGTTATCCGGATTTATTGGGCGTAAAGCGAGCGCAGGCGGTTTTTTAAGTCTGATGTGAAAGCCCTCGGCTTAACCGAGGAAGTGCATCGGAAACTGGGAAACTTGAGTGCAGAAGAGGACAGTGGAACTCCATGTGTAGCGGTGAAATGCGTAGATATATGGAAGAACACCAGTGGCGAAGGCGGCTGTCTGGTCTGTAACTGACGCTGAGGCTCGAAAGCATGGGTAGCGAACAGGA

+1

GHHHHGHHGAFFHFHGHGGGGHHHGGGGGHHHHHHHFFGIIIHIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIFIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII@IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIGGHHHHHGGHGGHHHHHHHHHHGGGGGHHHHHHHH

Error message:

=========================================================================

          Extending OTU Seeds

          elapsed time: 347 s

=========================================================================

This is sdm (simple demultiplexer) 1.35 beta.

Checking for reverse primers on 1st read.

sdm run in No Map Mode.

Reading Fasta.

   [#############################################################] 100.00%

Found double ID 3

Aborting..

Failed /export/apps/lotus_pipeline//sdm -i_fastq ./LotuS_filtering_v2//tmpFiles//derep.hq.fq -o_fna ./LotuS_filtering_v2//tmpFiles//otu_seeds.fna -optimalRead2Cluster ./LotuS_filtering_v2//tmpFiles//finalOTU.uc -paired 1 -sample_sep ___ -derep_map ./LotuS_filtering_v2//tmpFiles//derep.map -options sdm_miSeq_stitched_reads.txt -o_qual_offset 33 -log ./LotuS_filtering_v2//LotuSLogS//SeedExtensionStats.txt -mergedPairs 0 -OTU_fallback ./LotuS_filtering_v2//tmpFiles//tmp_otu.fa -ucAdditionalCounts ./LotuS_filtering_v2//tmpFiles//finalOTU.uc.ADD -ucAdditionalCounts1 ./LotuS_filtering_v2//tmpFiles//finalOTU.uc.REST -otu_matrix ./LotuS_filtering_v2//OTU.txt -count_chimeras F

Fallback to OTU median sequences.

usearch v9.2.64_i86linux64, 65.7Gb RAM, 32 cores

No idea about this error, can you advise me about this error?

Thanks!

Qinglong

[Falk Hildebrand]

Hey Qinglong,

Normally the pipeline would expect some sort of tag on reads, that indicate whether it's forward or reverse (e.g. /1 or /2 or the illumina format which is "XXX 1:N:0:", but I think even like this it should work. Can you ensure that your renamed fasta headers are unique? This woud be the first thing that comes to my mind, despite just "complicating" the header of the fastqs.

best,
Falk

[qinglong wu]

This is special case study, I got stitched reads from others.

I used fastx_toolkit_0.0.13_binaries_Linux_2.6_amd64/fastx_renamer to rename the header for each sequence read in each sample; But I am sure it give me the unique header for each read before I go to LotuS step.

Sample ID is unique.

Thanks.

Qinglong

[Falk Hildebrand]

Hey Qinglong,

sorry for the late answer. Can you check in this file:

./LotuS_filtering_v2//tmpFiles//derep.hq.fq

if there are double IDs inf the fastq header? E.g. "grep '^@3'  | head" might be a good start on this file. To be honest, LotuS was never designed for these cases where headers were tampered extremely with, but maybe we can figure at least out, what went wrong.

best,

Falk

[qinglong wu]

Thanks, Falk!

Yes, as you expected, I found the repeated header name in derep.hq.fq file. That means LotuS will merge the filtered the high-quality reads during dereplication process by UPARSE; if the sequence header is not unique among all the sequence reads of all samples, then I will get this error.

Thanks!

Qinglong