Use value inside file as input to process

[Chris D]

Hey,

Suppose I have a process that writes a random integer to N files. In another process I would like to use each of those integers (from each file) as input into some program. How can I do that?

For example,

process X {

    // produces N files each with a random integer

}

process Y {

    input:

    val x from input_files

    """

    executable -k x (where x is the integer in the file and -k is a command line option)

    """

}

Best,

Chris D

[Paolo Di Tommaso]

If each of that file contains just a value you can simply read the file content and return that value with a `map` operator eg. 

input_files.map{ file -> file.text.trim() } .set { input_values } 

then use `input_values` in the Y process. 

In the case the file content is more complicated you may need of a parsing function to find out the value you are looking for. 

Another approach, is to input that files in the Y process and use grep or sed to get the value and assign it to a BASH variable. 

Hope it helps.

Cheers,

Paolo

 

--
You received this message because you are subscribed to the Google Groups "Nextflow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/nextflow.
For more options, visit https://groups.google.com/d/optout.

[Chris D]

Hey Paolo,

Thanks for the reply.

This is the error message I get when I attempt your first suggestion.

ERROR ~ No such variable: Exception evaluating property 'text' for java.util.ArrayList, Reason: groovy.lang.MissingPropertyException: No such property: text for class: java.lang.String

I also tried your third suggestion, but was unsuccessful.

Any advice?

Best,
Chris D

[Paolo Di Tommaso]

Could u post your pipeline code?

p

--

[Chris D]

Hey Paolo,

process run_kmer_genie {
        publishDir "$results_path/kmer_genie"

        input:
        set dataset_id, file(forward), file(reverse) from kmer_genie_read_pairs

        output:
        set dataset_id, file("${dataset_id}_best-k.txt") into best_kmer_results

        """
        echo $forward > ${dataset_id}_read_pair_list.txt
        echo $reverse >> ${dataset_id}_read_pair_list.txt
        kmergenie "${dataset_id}_read_pair_list.txt" -t 10 | tail -n 1 | awk '{print \$3}' > ${dataset_id}_best-k.txt
        """
}

best_kmer_results.map{ file -> file.text.trim() } .set { input_values }

To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+u...@googlegroups.com.

[Paolo Di Tommaso]

Since the output is declared as 

     set dataset_id, file("${dataset_id}_best-k.txt") into best_kmer_results

it contains pairs not just files, thus you will need to map it to a new pairs as shown below 

best_kmer_results

       .map{ id, file -> [id, file.text.trim()] } 

       .set { input_values } 

Hope it helps. 

p

To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+unsubscribe@googlegroups.com.

[Chris D]

Perfect, thanks!