meme -p incompatible with (my) bash while loops?

[Skyler]

Hello All,

I can loop a directory of fasta files through meme only in single processor mode. If I use the -p option for parallel meme, the loop breaks after successfully completing the first file. 

Below are abbreviated commands showing what I've done to try to identify the problem. I've tried reinstalling meme to no avail. I'm running on Ubuntu in windows subsystem for Linux. I am aware of a warning that can be silenced by temporarily setting ptrace_scope to zero, but this has not changed my results below.

while read i; do meme -oc ${i} $i; done < <(ls | grep .fasta)                                                            #this loops through all the files successfully

while read I; do meme -p 2 -oc ${i} $i; done < <(ls | grep .fasta)                                                     #this stops after successfully finishing first file

meme -p 2 single_file.fasta                                                                                                             #this works just fine, I see N processes in the task manager

while read I; do meme -p 2 -oc ${i} $i; done < <(ls | grep single_file.fasta)                                     #feeding only a single file into the loop works the same as no loop

meme -p 7 -oc first first_file.fasta && meme -p 7 -oc second second_file.fasta                             #both files are successfully completed with this approach

It's beyond me whether its a meme or openmpi or bash problem. Does meme -p throw some extra 'exit' signals that terminate the loop?

I appreciate any thoughts as to how this can be addressed.

Thanks

Skyler

[cegrant]

while read i; do meme -oc ${i} $i; done < <(ls | grep .fasta)

I wouldn't expect that command line to work, and in fact it didn't work for me.

This command line tells meme to create an output directory with the same name as the input fasta file. You should be getting an error message like:

A non-directory file named 'crp0.fasta' already exists,
so that name can't be used for an output directory.

Are you sure that is the exact command line you used?

[Skyler]

 EDIT: In the original post, I misrepresented the code I was actually using. The ${i} should be replaced with ${i}_out to give this:

while read i; do meme -p 6 -oc ${i}_out $i; done < <(ls | grep .fasta)

[Joel Wallenius]

I'm no expert on meme, but the better bash syntax for that is

for i in *.fasta ; do ... ; done

No arrows needed.

[Skyler]

You are right cegrant, I mistyped the problem in the original post. Thank you for your support.

Below are exact copy-paste of commands I repeated today (in a directory containing file1.fasta and file2.fasta) to replicate the problem.

while read i; do meme -p 6 -oc ${i}_out $i; done < <(ls | grep .fasta)

while read i; do meme -oc ${i}_out $i; done < <(ls | grep .fasta)

A transcript of the commands and standard out is below. Sorry it's ugly. I can't seem to get the code syntax highlighter to work.

:$ ls

file1.fasta  file2.fasta

:$ while read i; do meme -oc ${i}_out $i; done < <(ls | grep .fasta)

Writing results to output directory 'file1.fasta_out'.
BACKGROUND: using background model of order 0
PRIMARY (classic): n 39 p0 39 p1 0 p2 0
SEQUENCE GROUP USAGE-- Starts/EM: p0; Trim: p0; pvalue: p0; nsites: p0,p1,p2
SEEDS: maxwords 15600 highwater mark: seq 39 pos 392
BALANCE: samples 39 chars 15600 nodes 1 chars/node 15600
Initializing the motif probability tables for 2 to 39 sites...
nsites = 39
Done initializing.

seqs=    39, min_w= 400, max_w=  400, total_size=    15600

motif=1
SEED DEPTHS: 2 4 8 16 32 39
SEED WIDTHS: 8 11 15 21 29 41 50
em: w=  50, psites=  39, iter=   0
Writing results to output directory 'file2.fasta_out'.
BACKGROUND: using background model of order 0
PRIMARY (classic): n 39 p0 39 p1 0 p2 0
SEQUENCE GROUP USAGE-- Starts/EM: p0; Trim: p0; pvalue: p0; nsites: p0,p1,p2
SEEDS: maxwords 15600 highwater mark: seq 39 pos 392
BALANCE: samples 39 chars 15600 nodes 1 chars/node 15600
Initializing the motif probability tables for 2 to 39 sites...
nsites = 39
Done initializing.

seqs=    39, min_w= 400, max_w=  400, total_size=    15600

motif=1
SEED DEPTHS: 2 4 8 16 32 39
SEED WIDTHS: 8 11 15 21 29 41 50
em: w=  50, psites=  39, iter=   0

:$ while read i; do meme -p 6 -oc ${i}_out $i; done < <(ls | grep .fasta)

The output directory 'file1.fasta_out' already exists.
Its contents will be overwritten.
BACKGROUND: using background model of order 0
PRIMARY (classic): n 39 p0 39 p1 0 p2 0
SEQUENCE GROUP USAGE-- Starts/EM: p0; Trim: p0; pvalue: p0; nsites: p0,p1,p2
SEEDS: maxwords 15600 highwater mark: seq 39 pos 392
BALANCE: samples 39 chars 15600 nodes 6 chars/node 2600
Initializing the motif probability tables for 2 to 39 sites...
nsites = 36
Done initializing.

seqs=    39, min_w= 400, max_w=  400, total_size=    15600

motif=1
SEED DEPTHS: 2 4 8 16 32 39
SEED WIDTHS: 8 11 15 21 29 41 50
em: w=  50, psites=   2, iter=   0

:$ ls

file1.fasta  file1.fasta_out  file2.fasta  file2.fasta_out

[Skyler]

Thank you Joel. That is more precise/concise and it solves the problem.

Expert or no, I appreciate your support.