Failed sdm demultiplexing step: What is the problem?

[amin...@hotmail.com]

I am running Lotus on a single fastq file that is multiplexed.  That is, the reads are structured like this:

barcodelinkerprimersequenceread    (barcode_linkerprimersequence_read).

I created the above fastq file from two separate fastq files; one with just the barcodes and one with just the reads-no linker primer sequence was in either file.  I have the linker primer sequence from the mapping file that was provided with the two fastq files.

Has my joining the two fastq files into a single multiplexed file created an issue with Lotus?  Can I use lotus with the two separate barcode fastq and reads fast that I received?

Here is the log file from running Lotus:

=========================================================================

LotuS 1.562

=========================================================================

COMMAND

/usr/bin/perl /home/smitha/tools/lotus_pipeline/lotus.pl -i /home/smitha/NE -o /home/smitha/NE/lOTUS-test -m /home/smitha/NE/Final_NE_16S_MappingFile.txt -s /home/smitha/tools/lotus_pipeline/sdm_miSeq.txt -c /home/smitha/tools/lotus_pipeline/lOTUs.cfg -threads 8

　

=========================================================================

Reading mapping file

=========================================================================

Running UPARSE de novo sequence clustering..

Running fast LotuS mode..

------------ I/O configuration --------------

Input= /home/smitha/NE

Output= /home/smitha/NE/lOTUS-test

TempDir= /home/smitha/NE/lOTUS-test/tmpFiles/

------------ Configuration LotuS --------------

Sequencing platform=454

AmpliconType=SSU

OTU id=0.97

min unique read abundance=2

UCHIME_REFDB, ABSKEW=/home/smitha/tools/lotus_pipeline/DB/rdp_gold.fa, 2

OTU, Chimera prefix=OTU_, CHIMERA_

TaxonomicGroup=bacteria

--------------------------------------------

=========================================================================

Demultiplexing input files

elapsed time: 0 s

=========================================================================

FAILED sdm demultiplexing step: /home/smitha/tools/lotus_pipeline/sdm -i_path /home/smitha/NE  -o_fna /home/smitha/NE/lOTUS-test/tmpFiles//demulti.fna -sample_sep ___ -log /home/smitha/NE/lOTUS-test/LotuSLogS//demulti.log -map /home/smitha/NE/Final_NE_16S_MappingFile.txt -options /home/smitha/tools/lotus_pipeline/sdm_miSeq.txt -o_dereplicate /home/smitha/NE/lOTUS-test/tmpFiles//derep.fas -dere_size_fmt 0 -min_derep_copies 2 -suppressOutput 1 -o_qual_offset 33 -paired 1 -maxReadsPerOutput 4000000 -oneLineFastaFormat 1

Here are a few lines from my mapping file:

#SampleID BarcodeSequence LinkerPrimerSequence sampleType nucleicAcidProtocolID kitLotID extractionDate concentrationNucleicAcid Description

STER_033_M_23_30_20140723 AGCCTTCGTCGC GTGTGYCAGCMGCCGCGGTAA soil PowerSoil96WellExtractionv1.pdf PS14K4 20150327 7.7 STER.033.M.23.30.20140723

STER_027_M_33_0_20140723 TCCATACCGGAA GTGTGYCAGCMGCCGCGGTAA soil PowerSoil96WellExtractionv1.pdf PS14K4 20150327 6.67 STER.027.M.33.0.20140723

STER_033_M_33_4_20140723 AGCCCTGCTACA GTGTGYCAGCMGCCGCGGTAA soil PowerSoil96WellExtractionv1.pdf PS14K4 20150327 4.77 STER.033.M.33.4.20140723

Here are a couple of fastq reads from the original fastq file I received:

@M02149:250:000000000-AHCGJ:1:1101:18130:1801 1:N:0:0

TACGTAGGCAGCGAGCGTTGTTCGGAGTTACTGGGCGTAAAGGGTGCGTAGGCGGTTTCCTAAGTTTGGTGTGAAATCTCCCGGCTCAACTGGGAGGGTGCGGCGAAAACTGGGAGGCTGGAGTACCGGAGAGGTAAGCGGAATTCCTGGTGTAGCGGTGAAATGCGTAGAGATCAGGAGGAACACCTGTGGTGAAGACAGCTTACTGGACGGTCACTGACGCTGAGGAACGAAAGCGAGGGTAGCAAACAGG

+

A>A1>11A11>A11EEAEG?EFHEEA0EFD2FDBFAEE/ECG1/A/AAFAAEEEEEEE@2B2@BF@B1/??GGEBFBFED1>/EEG/<BF>00<0<//?CC///?1@/??G?<<0?<@G1GF?GB10CFHHFGC:0E<0FFBB;C;BB0B;E/EE/FG2FHEB/EFF2HGFGB11FEEG2HEB1B/B/A1FB2D2DA1B1A121BB/AEB/1112B0AEAA1B1FGBA0AEGFECE1A1GGFFFDB@FA>1AA

@M02149:250:000000000-AHCGJ:1:1101:16127:1806 1:N:0:0

TACAGAGGTCTCAAGCGTTGTTCGGATTCATTGGGCGTAAAGGGTGCGTAGGCGGCGCGGTAAGTCGGATGTGAAATCTCGGAGCTTAACTCCGAAACTGCATTCGATACTGCCATGCTTGAGGACTGGAGAGGAGACTGGAATTTACGGTGTAGCGGTGAAATGCGTAGATATCGTAAGGAAGACCAGTGGCGAAGGCGGGTCTCTGGACAGTTCCTGACGCTGAGGCACGAAGGTCAGGGGAGCAAACGGG

+

AAA111FAF1DF3BFGGGG1FEHEGEAE1D2FFGHCEGEEAG1E0/AAFEEGFGG/////???GHD///?EGHFHFBFDE0//>BGCBBG>1<?///<FGG1??>?@?FCGB1<1?<GHDGG<B>1<>FHGFGG>1B<<GHGF<C0B0:B;E/GEE@F2FHEE>GFGBGG>>B//12@B1HGGBF0B@>/GEAHEB/EA/AA0/1F2ABD1B0010BAFEB1GFCGB00AGGGB1B111GGFFFD>CF>>1AA

@M02149:250:000000000-AHCGJ:1:1101:14898:1808 1:N:0:0

TACAGAGGGTGCAAGCGTTGTTCGGAATTATTGGGCGTAAAGGGCGTGTAGGCGGTGCGGTAAGTTTGTGGTGAAATCTCCGGGCTCAACTCGGAAATTGCCGCGAAGACTGCCGTGCTGGAGTATGGGAGAGGTGAGTGGAATTCCCGGTGTAGCGGTGAAATGCGTAGATATCGGGAGGAACACCTGCGGCGAAAGCGGCTTACTGGACCATAACTGACGCTGAGACGCGAAAGCTAGGGGAGCAAACAGG

+

AAA111BAAA?A1GGGGGGAEEHFE00BBDAFBBGCEGEE/G1A//AAAGGHFEGE////?//BF2B1B0?FFHHFBFEE1//>EE/B1B>1/?///<BFG1//D0D/CGGF0<<>CFG1HFFF?=F<GHHGGF<<F@;GEGC;C:C0<CAEEGGEEGBFHGEEGGG>GF>>>/>CFFGBHFGBG>>///GGGHGEAEA1A121BB1BFD1122BGFCGEB1HGFFEA00EGGFAGF1FGGFFFFBDFA>1AA

@M02149:250:000000000-AHCGJ:1:1101:13858:1817 1:N:0:0

TACGGAGGGTGCAAGCGTTATCCGGATTCACTGGGTTTAAAGGGTGCGTAGGCGGGCAGGTAAGTCCGTGGTGAAATCTCCCAGCTTAACTTGGAAACTGCCGTGGATACTATTTGTCTTGAATATAGTAGAGGTGAGCGGAATATGTCATGTAGCGGTGAAATGCTTAGATATGACATAGAACACCCATTGCGAAGGCAGCTCGCTACACTATTATTGACGCTGAGGCACGAAAGCGAGGGGATCAAACAGG

+

AAAAAAADD>AA1BGGGFGEC3FEEGAE1D1FGHH0FEGGHHDEB/AEFEEGFEG///>/BBGHHE1/>/EGGGHHFGFE<00BGHDBFGFF1B111<FGH01?@@C?FGG2><?<@FGGHG2FB?BFD1GFGFC/<?1FFFCC2=1==1=EEGGEDGGFHGFBFB@BGFBBFB2B@B2EHEE/GFB1>/GGHHGB0BB/////2AD22D22A12E/GGEB1HHGGF00AGGFFAEAA1GGFFF1FFFAA>A>

@M02149:250:000000000-AHCGJ:1:1101:15859:1823 1:N:0:0

TACGAGGGGGGCAAGCGTTGTTCGGAATTATTGGGCGTAAAGGGCGCGTAGGCGGTGCGGTAAGTCACCTGTGAAATCTCTGGGCTCAACTCAGAGCGTGCAGGCGAAACTGCCGTGCTCGAGGGTGGGAGAGGTGCGTGGAATTCCCGGTGTAGCGGTGAAATGCGTAGATATCGGGAGGAACACCTGTGGCGAAAGCGGCGCACTGGACCACATCTGACGCTGAGGCGCGAAAGCTAGGGGAGCAAACAGG

+

AAA?11>@10/A/FFGGGGEEEHGEE/?B1>FBBGECC<CCH1<<//A@CCCDCG@----<<CFHC0:::CGHFGG<CCC;00CGH.9;F;90;009-CFF.::.CC-G@FB/9.;9BF/HGA>0<?9HHHCCBB//@9FFFF;FBB90C@EEEE<<GDFHGEEEFB2GGEEE?/>B@GBHEG1FB>0>/GGCHFEAEAA//A1AB1ABB11A1D1EEEEB1GGEFA000AGGGDGB11GGFFFF>DFAA1AA

@M02149:250:000000000-AHCGJ:1:1101:15659:1825 1:N:0:0

GACGTAGGGGGCAAGCGTTGTTCGGAATTATTGGGCGTAAAGGGCGCGTAGGCGGCGCAGCAAGTCACCTGTGAAATCTCTGGGCTTAACTCAGAGCGTGCAGGCGAAACTGCTGTGCTGGAGTATGGGAGAGGTGCGTGGAATTCCCGGTGTAGCGGTGAAATGCGTAGATATCGGGAGGAACACCTGTGGCGAAAGCGGCGCACTGGACCATAACTGACGCTGAGGCGCGAAAGCTAGGGGAGCAAACAGG

+

AAB3>AFB@2DD2FFGGGGEFEHGGE2FFEAFFFGEFGEGHH3EA0AEEEGGFEG?//<//BFGHD4??FGHHHHHFGDFD1BGHHFGFHFD1?1<?/CDF/<0/FD0GCHF0=<=<GH1HGFGB=FCFHHGDGCCB<>GGGF<FBF9CF@E<EEE<GDFHFEEGFF@HF>>E>/?F@G2HEF1GEB0/?GGCHFEAEA///B1ABABFD1122DDAEEBA1EGEEE000GGGFDGF1AGGFFFFCDFA>1AA

@M02149:250:000000000-AHCGJ:1:1101:13907:1825 1:N:0:0

TACAGAGGTCCCAAGCGTTGTTCGGATTCACTGGGCGTAAAGGGTGCGTAGGCGGTTGGGTAAGTCTGACGTGAAATCTCCGGGCTTAACCCGGAAACTGCGTTGGATACTATCCAGCTAGAGGAATGGAGGGGAGACTGGAATACTTGGTGTAGCAGTGAAATGCGTAGATATCAAGTGGAACACCAGTGGCGAAGGCGAGTCTCTGGACAATTCCTGACGCTGAGGCACGAAAGCCAGGGGAGCAAACGGG

+

A1A>11FAF>1C1BFFAEG0ACGGGCAE1FAFFAGCEEEECH1EA/AEFEEGDEGE>///?@0GG@@111?FEEFFBF@@1//>E/>BBG1</<///<FDG/??>D@?FGG>>><>>GHDHGFFB1<?EGF<GF>1>1<FGFH</<>=0E;G@HGF2F2FHFE/EFFBGFB@B11@@@22HEG0F0BAA/GFCHEEAABA1B0/1B2EFD1A101000EEB1FCFGF00AGGGGAGA1BGGFFFF>CFA>1AA

@M02149:250:000000000-AHCGJ:1:1101:16069:1832 1:N:0:0

TACAGAGGTCCCAAGCGTTGTTCGGATTCACTGGGCGTAAAGGGTGCGTAGGTGGCCAGGTAAGTTTGATGTGAAATCCCGCAGCTCAACTGCGGAACTGCATTGAATACTATTTGGCTCGAGGAATGGAGGGGAGACTGGAATTCTCGGTGTAGCAGTGAAATGCGTAGATATCAGGAGGAACACCCGTGGTGAAGGCGGCTTTCTGGACCATAGCTGACGCTGAGACACGAAAGCGTGGGTAGCAAACAGG

+

AAAA11FAF>1C1FCFGGGEFGHGGGEF1FAFGFHEEGGGHH1EA/AEFGGG1BF001//B@FGF@D11DFGHFHHFF1B/>/EEH@FGHF0@E@//>GHH1BB?1BBGGHB>@><1FHCGGFCB1@FCGF<GC><B<<GHGGFGD<=>FFF1HGF2G2GHFE>GGF@GGFFFBB0FDGBHEE/E/>/A1FB1A/EA/E001101GFAFDB1A1B1EEGEEBHGFGFA0AGGGFAGEE1GGFFFFBFFAA1AA

@M02149:250:000000000-AHCGJ:1:1101:13778:1833 1:N:0:0

TACGGGGGGTGCGAGCGTTGTCCGGAATCACTGGGCGTAAAGGGCGCGTAGGCGGTCTGTTAAGGGTGTGGTGAAATGCCGGGGCTCAACTCCGTGCCTGCCGCGGATACTGGGAAGCTGGAGGCGAGTAGAGGCAAGAGGAATTCCGGGTGTAGCGGTGAAATGCGTAGAGATCCGGAGGAAGACCGGTGGCGAAGGCGGACTGAGGGAAAGAAAATGAAAATGAGGAGAGAAAGAGAGGGGAGCAAACAGG

+

AAAAA1>DDADA//EEEGG/?1FBCG/?1B1F?F/<BGCFHD1C//->@@GG0<C@::/0;:<CB.:.;.;FEEFF;009---;A@-;BF/99999999;A-//D?@9FBF<<<<1<@F??BF//1>;<BFBFF11B>2GFGF>B9B?9>;E>EE/EGFE@FEEEFF2GGGEB?//>BB1FDFAE/B0A/GGGHGEAABA1110011AD11D222233DA3311AFBA333FA3AB1A1GGFDBFBDFA>>AA

@M02149:250:000000000-AHCGJ:1:1101:14018:1834 1:N:0:0

TACAGAGGGTGCGAGCGTTGTTCGGATTTATTGGGCGTAAAGGGCGCGTAGGCGGTTCCTCAAGTCACCTGTGAAATCTCCGGGCTTAACTCCGAGCCTGCTGGCGAAACTGTCGTGCTGGAGTATGGGAGAGGTGCGTGGAATTCCCGGTGTAGCGGTGAAATGCGTAGATATCGGGAGGAACACCAGTGGCGAAAGCGGCGCACTGGACCATAACTGACGCTGAGGCGCGAAAGCTAGGGGAGCAAACAGG

+

AAA111CA1>AA1EEEEFE0EEH0AA0BBD2FDB/AEEAEFH1A///A>EEEDEE>//1211BBF@2>1B>FCGFGBE>>1//<>E<BBF>11////<FFG1<11DD/?CG<<10?0>F1GF>G?=?<BHGCC></0@<GGFC:C:C00EDE<GE?>GBFHEE/EFGBGF>>>?>>B1G2HEF0B0B1>/GGEHFE/EA///A1BAAADD1122BFEAEEA1GGEEA000FGGFAGB11GGFFFDBDFA>1AA

　

Here are the corresponding fastq barcode reads from the original fastq I received:

@M02149:250:000000000-AHCGJ:1:1101:18130:1801 1:N:0:0

CTACACGGTGAA

+

111>1>111111

@M02149:250:000000000-AHCGJ:1:1101:16127:1806 1:N:0:0

TCGTTGGGTCTA

+

1>1>111111>1

@M02149:250:000000000-AHCGJ:1:1101:14898:1808 1:N:0:0

CGAAGCTTAACT

+

11111>BB11B@

@M02149:250:000000000-AHCGJ:1:1101:13858:1817 1:N:0:0

CCTTCCCCTTCC

+

111111111111

@M02149:250:000000000-AHCGJ:1:1101:15859:1823 1:N:0:0

CGTGCAGACAGA

+

11>1>111>111

@M02149:250:000000000-AHCGJ:1:1101:15659:1825 1:N:0:0

CAGACCGCACGT

+

1111>1111110

@M02149:250:000000000-AHCGJ:1:1101:13907:1825 1:N:0:0

TTAGGTATAACC

+

1>11113@331@

@M02149:250:000000000-AHCGJ:1:1101:16069:1832 1:N:0:0

TCGAGCCGATCT

+

1>1111111>>>

@M02149:250:000000000-AHCGJ:1:1101:13778:1833 1:N:0:0

TCTTGGTGTAAG

+

1>>>11B1>133

@M02149:250:000000000-AHCGJ:1:1101:14018:1834 1:N:0:0

TCTCTTTCTTTT

+

1111133@B33@

　

Here are a few reads from the fastq file I created to join the barcode, linkerprimer sequence, and reads:

@M02149:250:000000000-AHCGJ:1:1101:18130:1801 1:N:0:0

CTACACGGTGAAGTGTGYCAGCMGCCGCGGTAATACGTAGGCAGCGAGCGTTGTTCGGAGTTACTGGGCGTAAAGGGTGCGTAGGCGGTTTCCTAAGTTTGGTGTGAAATCTCCCGGCTCAACTGGGAGGGTGCGGCGAAAACTGGGAGGCTGGAGTACCGGAGAGGTAAGCGGAATTCCTGGTGTAGCGGTGAAATGCGTAGAGATCAGGAGGAACACCTGTGGTGAAGACAGCTTACTGGACGGTCACTGACGCTGAGGAACGAAAGCGAGGGTAGCAAACAGG

+

111>1>111111?????????????????????A>A1>11A11>A11EEAEG?EFHEEA0EFD2FDBFAEE/ECG1/A/AAFAAEEEEEEE@2B2@BF@B1/??GGEBFBFED1>/EEG/<BF>00<0<//?CC///?1@/??G?<<0?<@G1GF?GB10CFHHFGC:0E<0FFBB;C;BB0B;E/EE/FG2FHEB/EFF2HGFGB11FEEG2HEB1B/B/A1FB2D2DA1B1A121BB/AEB/1112B0AEAA1B1FGBA0AEGFECE1A1GGFFFDB@FA>1AA

@M02149:250:000000000-AHCGJ:1:1101:16127:1806 1:N:0:0

TCGTTGGGTCTAGTGTGYCAGCMGCCGCGGTAATACAGAGGTCTCAAGCGTTGTTCGGATTCATTGGGCGTAAAGGGTGCGTAGGCGGCGCGGTAAGTCGGATGTGAAATCTCGGAGCTTAACTCCGAAACTGCATTCGATACTGCCATGCTTGAGGACTGGAGAGGAGACTGGAATTTACGGTGTAGCGGTGAAATGCGTAGATATCGTAAGGAAGACCAGTGGCGAAGGCGGGTCTCTGGACAGTTCCTGACGCTGAGGCACGAAGGTCAGGGGAGCAAACGGG

+

1>1>111111>1?????????????????????AAA111FAF1DF3BFGGGG1FEHEGEAE1D2FFGHCEGEEAG1E0/AAFEEGFGG/////???GHD///?EGHFHFBFDE0//>BGCBBG>1<?///<FGG1??>?@?FCGB1<1?<GHDGG<B>1<>FHGFGG>1B<<GHGF<C0B0:B;E/GEE@F2FHEE>GFGBGG>>B//12@B1HGGBF0B@>/GEAHEB/EA/AA0/1F2ABD1B0010BAFEB1GFCGB00AGGGB1B111GGFFFD>CF>>1AA

@M02149:250:000000000-AHCGJ:1:1101:14898:1808 1:N:0:0

CGAAGCTTAACTGTGTGYCAGCMGCCGCGGTAATACAGAGGGTGCAAGCGTTGTTCGGAATTATTGGGCGTAAAGGGCGTGTAGGCGGTGCGGTAAGTTTGTGGTGAAATCTCCGGGCTCAACTCGGAAATTGCCGCGAAGACTGCCGTGCTGGAGTATGGGAGAGGTGAGTGGAATTCCCGGTGTAGCGGTGAAATGCGTAGATATCGGGAGGAACACCTGCGGCGAAAGCGGCTTACTGGACCATAACTGACGCTGAGACGCGAAAGCTAGGGGAGCAAACAGG

+

11111>BB11B@?????????????????????AAA111BAAA?A1GGGGGGAEEHFE00BBDAFBBGCEGEE/G1A//AAAGGHFEGE////?//BF2B1B0?FFHHFBFEE1//>EE/B1B>1/?///<BFG1//D0D/CGGF0<<>CFG1HFFF?=F<GHHGGF<<F@;GEGC;C:C0<CAEEGGEEGBFHGEEGGG>GF>>>/>CFFGBHFGBG>>///GGGHGEAEA1A121BB1BFD1122BGFCGEB1HGFFEA00EGGFAGF1FGGFFFFBDFA>1AA

@M02149:250:000000000-AHCGJ:1:1101:13858:1817 1:N:0:0

CCTTCCCCTTCCGTGTGYCAGCMGCCGCGGTAATACGGAGGGTGCAAGCGTTATCCGGATTCACTGGGTTTAAAGGGTGCGTAGGCGGGCAGGTAAGTCCGTGGTGAAATCTCCCAGCTTAACTTGGAAACTGCCGTGGATACTATTTGTCTTGAATATAGTAGAGGTGAGCGGAATATGTCATGTAGCGGTGAAATGCTTAGATATGACATAGAACACCCATTGCGAAGGCAGCTCGCTACACTATTATTGACGCTGAGGCACGAAAGCGAGGGGATCAAACAGG

+

111111111111?????????????????????AAAAAAADD>AA1BGGGFGEC3FEEGAE1D1FGHH0FEGGHHDEB/AEFEEGFEG///>/BBGHHE1/>/EGGGHHFGFE<00BGHDBFGFF1B111<FGH01?@@C?FGG2><?<@FGGHG2FB?BFD1GFGFC/<?1FFFCC2=1==1=EEGGEDGGFHGFBFB@BGFBBFB2B@B2EHEE/GFB1>/GGHHGB0BB/////2AD22D22A12E/GGEB1HHGGF00AGGFFAEAA1GGFFF1FFFAA>A>

@M02149:250:000000000-AHCGJ:1:1101:15859:1823 1:N:0:0

CGTGCAGACAGAGTGTGYCAGCMGCCGCGGTAATACGAGGGGGGCAAGCGTTGTTCGGAATTATTGGGCGTAAAGGGCGCGTAGGCGGTGCGGTAAGTCACCTGTGAAATCTCTGGGCTCAACTCAGAGCGTGCAGGCGAAACTGCCGTGCTCGAGGGTGGGAGAGGTGCGTGGAATTCCCGGTGTAGCGGTGAAATGCGTAGATATCGGGAGGAACACCTGTGGCGAAAGCGGCGCACTGGACCACATCTGACGCTGAGGCGCGAAAGCTAGGGGAGCAAACAGG

+

11>1>111>111?????????????????????AAA?11>@10/A/FFGGGGEEEHGEE/?B1>FBBGECC<CCH1<<//A@CCCDCG@----<<CFHC0:::CGHFGG<CCC;00CGH.9;F;90;009-CFF.::.CC-G@FB/9.;9BF/HGA>0<?9HHHCCBB//@9FFFF;FBB90C@EEEE<<GDFHGEEEFB2GGEEE?/>B@GBHEG1FB>0>/GGCHFEAEAA//A1AB1ABB11A1D1EEEEB1GGEFA000AGGGDGB11GGFFFF>DFAA1AA

@M02149:250:000000000-AHCGJ:1:1101:15659:1825 1:N:0:0

CAGACCGCACGTGTGTGYCAGCMGCCGCGGTAAGACGTAGGGGGCAAGCGTTGTTCGGAATTATTGGGCGTAAAGGGCGCGTAGGCGGCGCAGCAAGTCACCTGTGAAATCTCTGGGCTTAACTCAGAGCGTGCAGGCGAAACTGCTGTGCTGGAGTATGGGAGAGGTGCGTGGAATTCCCGGTGTAGCGGTGAAATGCGTAGATATCGGGAGGAACACCTGTGGCGAAAGCGGCGCACTGGACCATAACTGACGCTGAGGCGCGAAAGCTAGGGGAGCAAACAGG

+

1111>1111110?????????????????????AAB3>AFB@2DD2FFGGGGEFEHGGE2FFEAFFFGEFGEGHH3EA0AEEEGGFEG?//<//BFGHD4??FGHHHHHFGDFD1BGHHFGFHFD1?1<?/CDF/<0/FD0GCHF0=<=<GH1HGFGB=FCFHHGDGCCB<>GGGF<FBF9CF@E<EEE<GDFHFEEGFF@HF>>E>/?F@G2HEF1GEB0/?GGCHFEAEA///B1ABABFD1122DDAEEBA1EGEEE000GGGFDGF1AGGFFFFCDFA>1AA

@M02149:250:000000000-AHCGJ:1:1101:13907:1825 1:N:0:0

TTAGGTATAACCGTGTGYCAGCMGCCGCGGTAATACAGAGGTCCCAAGCGTTGTTCGGATTCACTGGGCGTAAAGGGTGCGTAGGCGGTTGGGTAAGTCTGACGTGAAATCTCCGGGCTTAACCCGGAAACTGCGTTGGATACTATCCAGCTAGAGGAATGGAGGGGAGACTGGAATACTTGGTGTAGCAGTGAAATGCGTAGATATCAAGTGGAACACCAGTGGCGAAGGCGAGTCTCTGGACAATTCCTGACGCTGAGGCACGAAAGCCAGGGGAGCAAACGGG

+

1>11113@331@?????????????????????A1A>11FAF>1C1BFFAEG0ACGGGCAE1FAFFAGCEEEECH1EA/AEFEEGDEGE>///?@0GG@@111?FEEFFBF@@1//>E/>BBG1</<///<FDG/??>D@?FGG>>><>>GHDHGFFB1<?EGF<GF>1>1<FGFH</<>=0E;G@HGF2F2FHFE/EFFBGFB@B11@@@22HEG0F0BAA/GFCHEEAABA1B0/1B2EFD1A101000EEB1FCFGF00AGGGGAGA1BGGFFFF>CFA>1AA

@M02149:250:000000000-AHCGJ:1:1101:16069:1832 1:N:0:0

TCGAGCCGATCTGTGTGYCAGCMGCCGCGGTAATACAGAGGTCCCAAGCGTTGTTCGGATTCACTGGGCGTAAAGGGTGCGTAGGTGGCCAGGTAAGTTTGATGTGAAATCCCGCAGCTCAACTGCGGAACTGCATTGAATACTATTTGGCTCGAGGAATGGAGGGGAGACTGGAATTCTCGGTGTAGCAGTGAAATGCGTAGATATCAGGAGGAACACCCGTGGTGAAGGCGGCTTTCTGGACCATAGCTGACGCTGAGACACGAAAGCGTGGGTAGCAAACAGG

+

1>1111111>>>?????????????????????AAAA11FAF>1C1FCFGGGEFGHGGGEF1FAFGFHEEGGGHH1EA/AEFGGG1BF001//B@FGF@D11DFGHFHHFF1B/>/EEH@FGHF0@E@//>GHH1BB?1BBGGHB>@><1FHCGGFCB1@FCGF<GC><B<<GHGGFGD<=>FFF1HGF2G2GHFE>GGF@GGFFFBB0FDGBHEE/E/>/A1FB1A/EA/E001101GFAFDB1A1B1EEGEEBHGFGFA0AGGGFAGEE1GGFFFFBFFAA1AA

@M02149:250:000000000-AHCGJ:1:1101:13778:1833 1:N:0:0

TCTTGGTGTAAGGTGTGYCAGCMGCCGCGGTAATACGGGGGGTGCGAGCGTTGTCCGGAATCACTGGGCGTAAAGGGCGCGTAGGCGGTCTGTTAAGGGTGTGGTGAAATGCCGGGGCTCAACTCCGTGCCTGCCGCGGATACTGGGAAGCTGGAGGCGAGTAGAGGCAAGAGGAATTCCGGGTGTAGCGGTGAAATGCGTAGAGATCCGGAGGAAGACCGGTGGCGAAGGCGGACTGAGGGAAAGAAAATGAAAATGAGGAGAGAAAGAGAGGGGAGCAAACAGG

+

1>>>11B1>133?????????????????????AAAAA1>DDADA//EEEGG/?1FBCG/?1B1F?F/<BGCFHD1C//->@@GG0<C@::/0;:<CB.:.;.;FEEFF;009---;A@-;BF/99999999;A-//D?@9FBF<<<<1<@F??BF//1>;<BFBFF11B>2GFGF>B9B?9>;E>EE/EGFE@FEEEFF2GGGEB?//>BB1FDFAE/B0A/GGGHGEAABA1110011AD11D222233DA3311AFBA333FA3AB1A1GGFDBFBDFA>>AA

@M02149:250:000000000-AHCGJ:1:1101:14018:1834 1:N:0:0

TCTCTTTCTTTTGTGTGYCAGCMGCCGCGGTAATACAGAGGGTGCGAGCGTTGTTCGGATTTATTGGGCGTAAAGGGCGCGTAGGCGGTTCCTCAAGTCACCTGTGAAATCTCCGGGCTTAACTCCGAGCCTGCTGGCGAAACTGTCGTGCTGGAGTATGGGAGAGGTGCGTGGAATTCCCGGTGTAGCGGTGAAATGCGTAGATATCGGGAGGAACACCAGTGGCGAAAGCGGCGCACTGGACCATAACTGACGCTGAGGCGCGAAAGCTAGGGGAGCAAACAGG

+

1111133@B33@?????????????????????AAA111CA1>AA1EEEEFE0EEH0AA0BBD2FDB/AEEAEFH1A///A>EEEDEE>//1211BBF@2>1B>FCGFGBE>>1//<>E<BBF>11////<FFG1<11DD/?CG<<10?0>F1GF>G?=?<BHGCC></0@<GGFC:C:C00EDE<GE?>GBFHEE/EFGBGF>>>?>>B1G2HEF0B0B1>/GGEHFE/EA///A1BAAADD1122BFEAEEA1GGEEA000FGGFAGB11GGFFFDBDFA>1AA

　

　

[Falk Hildebrand]

Hey,

I am currently investigating a bug that was introduced to sdm in 1.562 that causes sdm in some cases (that were not in my default test sets) to crash, if you have on your console some error about sdm crashing, then that's probably the same error. I'll post again here, as soon as  I had time to investigate the bug.

sorry for the delay,

Falk

[amin...@hotmail.com]

Thank you for continuing to work on this.  I have no error messages from SDM.  I did run the example that is included with LOTUs and it ran with no errors.  This makes me think there is something wrong with how I have set up my data.

[amin...@hotmail.com]

I switched out the SDM with a previous version that was used in an older version of LOTUs I had and was able to successfully run LOTUs on my data.

[Falk Hildebrand]

Hey Aminards,

well I'm glad to hear that was just a version issue. Just so you know, I'll upload a new sdm version hopefully soonish, that has a more stable memory handling model (and possibly uses a lot less mem as well, I'm still testing this).

best, Falk

[amin...@hotmail.com]

Coming back to this issue.   Has the version of sdm been fixed relating to the error I have been getting?

[Falk Hildebrand]

Hey Aminards,

I'm still not sure what the problem exactly is with your run, but given the convoluted nature of how the files were created, there might be some problems. I would just give it a try and see if the numbers your get out make sense. btw, if you save your barcodes to an extra file, you could insert these in the mapping file ("MIDfqFile"), actually now that I read through the email again, I wonder if there was a specifc reason to integrate the barcodes onto the sequence, when you can just have a MID fastq file?

hth,

Falk

[amin...@hotmail.com]

I should have noted that I am working with a new dataset.  These are simple 300bp paired fastq files from a miSeq.  I have done nothing to the fastq files.  I created the mapping file like this:

#SampleID    BarcodeSequence    LinkerPrimerSequence    fastqFile    Description

11-ITS-A1 11-ITS-A1_S28_L001_R1_001.fastq,11-ITS-A1_S28_L001_R2_001.fastq ITS

11-ITS-A10 11-ITS-A10_S29_L001_R1_001.fastq,11-ITS-A10_S29_L001_R2_001.fastq ITS

Now, no matter which version of SDM I use I am getting the same error as above:

FAILED sdm demultiplexing step: /hpc/home/smitha/tools/lotus_pipeline/sdm -i_path /hpc/home/smitha/NE/Sept_1_Run -o_fna /hpc/home/smitha/NE/Sept_1_Run/lOTUs_ITS/tmpFiles//demulti.1.fna,/hpc/home/smitha/NE/Sept_1_Run/lOTUs_ITS/tmpFiles//demulti.2.fna -sample_sep ___ -log /hpc/home/smitha/NE/Sept_1_Run/lOTUs_ITS/LotuSLogS//demulti.log -map /hpc/home/smitha/NE/Sept_1_Run/Sep_1_Run_ITS_mapping.txt -options /hpc/home/smitha/tools/lotus_pipeline/sdm_miSeq.txt -o_dereplicate /hpc/home/smitha/NE/Sept_1_Run/lOTUs_ITS/tmpFiles//derep.fas -dere_size_fmt 0 -min_derep_copies 2 -suppressOutput 1 -o_qual_offset 33 -paired 2 -maxReadsPerOutput 4000000 -oneLineFastaFormat 1

　

The "-sample_sep __"  in the error message doesn't make sense to me since my sample separator is the ",".   I tried replacing the "," in my mapping file with "__" and still got the exact same error. 

What might be helpful is if you can point me to an older version of SDM so I can re-download it.

Thank you.

[Falk Hildebrand]

Hey,

-sample_sep is an internal command, please don't modify it. 

Please send your mapping file to my email (falk.hildebrand at gmail), I'll have a look at it. I really don't recommend to use old sdm versions, but if the error persists, I'll make it available to you.

best,

Falk