Automating Qiime Split Libraries

[Sofia P]

I have 100s of joined.fastq.extendedFrags.fastqfiles that I want to run split libraries on. It works when I do this for each file:

split_libraries_fastq.py -i 03142103_joined.fastq.extendedFrags.fastq --sample_id 03142103 -o split_lib_03142103/ -m solo\ mapping\ files//03142103_mapping.txt --barcode_type 'not-barcoded'

However, I would like to automate it in the terminal so that I can simply apply this to all the files. Any suggestions on what I can add to my script to do this? All files are numbered like the example above.

I have tried this, but it comes up with an error saying it cant find the solo mapping files folder or the mapping files. Not sure how to identify these files in the syntax..

for i in $(ls *_joined.fastq.extendedFrags.fastq); do split_libraries_fastq.py ${i}_joined.fastq.extendedFrags.fastq —sample_id ${i} -o $split_lib_{i} -m solo\mapping\files//{i}_mapping.txt —barcode_type ’not-barcoded’ ; done

Thank you!

[TonyWalters]

Sofia, you might try this script (if you have QIIME 1.9.0 or later): http://qiime.org/scripts/multiple_split_libraries_fastq.html

If you have an older version of QIIME, I think you might want to create a faux mapping file with a single SampleID in it and have it pointed to as a constant, rather than using the i variable (it's actually not used directly in the command, hence putting a single sample in it and nothing else, and was removed as a required parameter in 1.9.x).

[Kyle Bittinger]

Sofia,

You might not need to do BASH scripting for this, but just a few tips in case you do:

1. You almost never need to use $( ... ) with the ls command.  This would probably work better:

    for i in *_joined.fastq.extendedFrags.fastq

instead of

    for i in $(ls *_joined.fastq.extendedFrags.fastq)

2. I see that you have some $'s and {}'s in odd places.  The ultimate best safe practice is to put each filepath in double-quotes, then use the full ${ ... } construct to surround each variable.  The first filepath would then be: "${i}_joined.fastq.extendedFrags.fastq" -- not sure what the other filepaths are in your command.  Whatever they are, you want to make sure the variable, i, always appears as ${i} when you include it in a file path, and that the file path is always surrounded by double quotes.

[Sofia P]

Thank you Kyle. I was able to make it work!