How to capture command line output besides echo?

[Julian Neuhaus]

Hi.

I want to measure some execution times of a bwa mapping process in order to decide which is the best chunk size. Therefore i want to capture the command line output produced by the bwa mem command which shoud look like this, when i execute the command on my local shell:

[M::main_mem] read 48000 sequences (4800000 bp)...

[main] Version: 0.7.5a-r405

[main] CMD: bwa mem hg18_chr21_micro.fa reads_pe_N48000.L.fastq

[main] Real time: 10.348 sec; CPU: 9.214 sec

I already tried to use the stdout channel (described here). My process looks like this:

process map {

input:

set left_fastq, right_fastq from jobs

output:

set left_fastq, file('out_bam') into records mode flatten

stdout out_channel

"""

bwa mem '${params.ref}' $left_fastq $right_fastq > out.sam

samtools view -bS out.sam > out_bam

"""

}

out_channel.println()

But no command line output is produced. Is there a way to gather it, or am i using it wrong? It would also be ok, if there is another possibility to measure the processing time of a nextflow process itself (but did not found something like this).

Regards,

Julian

[Marc Logghe]

Hi Julian

I think it should work if you don't send the samtools output to a file but to stdout instead:

process map {


 input:
 set left_fastq, right_fastq from jobs


 output:

 set left_fastq, file('out.sam') into records mode flatten
 stdout out_channel


 """

 bwa mem '${params.ref}' $left_fastq $right_fastq > out.sam
 samtools view -bS out.sam
 """

}

In that way, the bwa output file 'out.sam' is put into the records channel, the stdout of samtools is put into the out_channel.

HTH,

M

Op vrijdag 20 november 2015 10:13:10 UTC+1 schreef Julian Neuhaus:

[Marc Logghe]

Correction, you wanted it the way around ...

process map {


 input:
 set left_fastq, right_fastq from jobs


 output:
 set left_fastq, file('out_bam') into records mode flatten
 stdout out_channel


 """
 bwa mem '${params.ref}' $left_fastq $right_fastq

[Paolo Di Tommaso]

I guess the problem here is that the bwa tool writes that information on the stderr pseudofile. 

So you won't be able to capture it in using an "stdout" output channel. 

You have two options: 

1) Save the bwa stderr to a file, as shown below, and declare that file as an output 

bwa mem '${params.ref}' $left_fastq $right_fastq > out.sam 2> info

2) Nextflow is able to profile process runtime information adding the `-with-trace` command line option. 

You can find the documentation at this link.

Cheers,

Paolo

--
You received this message because you are subscribed to the Google Groups "Nextflow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+u...@googlegroups.com.
Visit this group at http://groups.google.com/group/nextflow.
For more options, visit https://groups.google.com/d/optout.

[Julian Neuhaus]

Thanks Paolo,

that is what i was locking for.

Regards,

Julian