Looping split_libraries with bash

[André Soares]

Hello all,

I need to loop split_libraries across a list of .fna and mapping files in two different folders. Each two corresponding mapping and .fna files' names will correspond.

For each run, I need the output folder to be incremented in its name (e.g. f1, f2,...) and -n to be incremented from 10000000 by 10000000 each time.

split_libraries.py -m $i.map.txt \
    -f $i.fna \
    -l 50 \
    -b 0 \
    -p \
    -o ./f_x \
    -n x*10000000

I'm trying to get a bash script going but I'm new at this stuff and not sure of how to create the right variables and get files from different folders...
Can anyone help?

Thanks in advance!

[Jamie Morton]

Hi Andre,

bash is a little different.

If you want to reference the variable x, you need to use $x.

So the code you want should be something as follows

#!/bin/bash

x=0;

for i in `seq 1 10`;

do

    split_libraries.py -m ($i).map.txt \
        -f ($i).fna \

        -l 50 \
        -b 0 \
        -p \

        -o ./f_($x) \
        -n $x

    ((x=x+100000)

    echo $i   # print out value of i

    echo $x  # print out value of x

done

You check out the following sources if you want more details on how bash works.

http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO-7.html

http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO-10.html

Let me know how it works out!

Cheers,

Jamie

[André Soares]

Hey Jamie,

Thanks for the help.

In this case, is 'seq 1 10' a kind of range(1,10) function?

To call the mappings and .fna from different folders, should I do something like:

    split_libraries.py -m (~/Desktop/PRJEB0000/maps/$i).map.txt \
        -f (~/Desktop/PRJEB0000/fnas/$i).fna \

?

Thanks again!

[André Soares]

Sorry for the spam, but with regards to
        -n $x

    ((x=x+100000)

Will this increment only the $x associated to -n each time?

Thanks again!

[Jamie Morton]

Hi Andres,

If you try running the following on the command line

i=0
echo "(~/Desktop/PRJEB0000/maps/$i)"

Then you'll get

(~/Desktop/PRJEB0000/maps/0)

Now, if you run

i=0
echo "~/Desktop/PRJEB0000/maps/$i"

Then you'll get

~/Desktop/PRJEB0000/maps/0

Rather, you probably want

split_libraries.py -m ~/Desktop/PRJEB0000/maps/$i.map.txt \
        -f ~/Desktop/PRJEB0000/fnas/$i.fna \

...

This will require a bit of tinkering - but fortunately, these can all be tested directly on the command line.

And yes, it'll only increment $x.  But to double check this, try running

echo $x

Cheers,

Jamie

[André Soares]

Hey again,

Thanks again for this, but can you just clarify the 'seq 1 10' part?
Doesn't that include only the first 10 files in the folder?
How can I include all files in the folders?

Thanks again!

[Jamie Morton]

Hi Andre,

Replace 'seq 1 10' with 'ls'.

Recommend reading the linked bash tutorial

Cheers,

Jamie

[André Soares]

Hello again,

I'm trying to loop another command - merge_otu_maps.py - through two folders of OTU picking outputs.

How can I append each run of the command within the for loop to a single 'otus.txt'?

Thanks!
André

[Colin Brislawn]

Hello Andre,

I should warn you that most OTU picking methods produce outputs which cannot be merged afterwards. The exception to this is closed-ref OTU picking. What kind of OTU picking are you performing? (I want to make sure I can recommend a method that works for you).

Colin Brislawn

[André Soares]

Hey there,

This comes from closed-ref OTU picking :)

[Colin Brislawn]

Glad you are on the right track! :-)

[André Soares]

Thanks, but can you help with the looping step?

I basically have this;

pick_otus_a
   1
      seqs_otus.txt
   2
      seqs_otus.txt
   3
      seqs_otus.txt

pick_otus_b
   1
      seqs_otus.txt
   2
      seqs_otus.txt
   3
      seqs_otus.txt

And have to merge all of them together!
Can you help?

[Colin Brislawn]

Can you post your current bash loop? I can help get it up and running. 

For reference, here is a loop of mine:

ls > folder_list.txt
head folder_list.txt
while read -r line
do
  echo "Folder name: $line"
  mkdir $line
done < folder_list.txt

This first makes a list of folder names, then loops through those names, making a folder for each one. 

Colin