How to pass all files in a Channel to a process at once?

[Matteo Schiavinato]

Hi,

I have what seems to be a basic question, but I can't find an answer. Chances are I misunderstood how collectFile() works, so I would appreciate anyone pointing me in the right direction.

I have a channel which is the output of a process. This channel contains multiple files of the same format.

In the process that follows, I would like to process all the files in this channel at the same time, that is, as if you were doing:

cat *.bam | <do something>

Where the *.bam represents all files ending in "bam" (which are all in the channel I pass to this process). What I want to do is to write something that does like:

cat <all_channel_files> | <do something>

Is there an elegant and concise way of doing it?

[Matteo Schiavinato]

For future readers, here's how I solved it (I made a general example from my code):

ch1 = Channel.from("<path_to_files>/*<extension_of_files>)
ch1
  .concat(ch2, ch3)
  .set { Concat_channel }

process process_all_files {

  input:
    file all_the_files from Concat_channel.collect()

  output:
    file "merged_file.txt" into merged_file

  script:
    """

    cat ${all_the_files} | \

    <some commands> \
    > merged_file.txt

[Li Wang]

Does it work in DSL2?

[Alan Hoyle]

As an aside, if you need to prepend those files with the same repeated command line option you can do it using Groovy's .join() method:

script:

all_the_files_with_option_inserted = all_the_files.join(" --option ")

# Note:  the spaces in " --option " above are needed for this to work in most cases

"""

command_that_needs_option --option ${all_the_files_with_option_inserted}

"""

--
  -  Alan Hoyle  -  al...@alanhoyle.com  -  http://www.alanhoyle.com/  -

--
You received this message because you are subscribed to the Google Groups "Nextflow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nextflow/1504554b-9c30-4bf6-ba6a-8d6e7e325a5fn%40googlegroups.com.