how to specify publishDir directory name as input for downstream process

[Anand Venkatraman]

Hi all

How do I specify a directory coming from the publishDir directive in a given process to be one of the input parameters for a downstream process?

To elaborate:

process quant {

    tag "$pair_id"

    publishDir "${params.outdir}/quant", mode: 'copy'

 

   Other steps etc

}

process summarize{

  need to specify the above dir "${params.outdir}/quant" as my input

               

               also need to ensure all relevant jobs for the quant process is finished before this process should start 

}

Thanks in advance.

[Paolo Di Tommaso]

You should not do that. PublishDir is meant to be used to copy final result files, not to make output files available to a downstream task. 

Process input/output files are automatically staged link the process with corresponding upstream output(s). 

Cheers, Paolo

--
You received this message because you are subscribed to the Google Groups "Nextflow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nextflow/7d3d7a65-9bf5-4ca7-b755-4c513ed7f427%40googlegroups.com.

[Anand Venkatraman]

Thanks for your response Paolo. So, how should I go about in this case 

• I have the salmon quant process that outputs quant.sf files to different folders under params.outdir/quant.
• I have a custom R script that takes a directory as command line argument, then looks up all the folders under there to find all quant.sf files and generate a summary.

[Paolo Di Tommaso]

Thanks for your response Paolo. So, how should I go about in this case 

• I have the salmon quant process that outputs quant.sf files to different folders under params.outdir/quant.

Umm, any NF task writes (or should write) in the task work dir, then the `publishDir` allows you to copy somewhere else. 

Therefore there's nothing special to do other the declare those files as output the process and as input in the downstream task. 

[Anand Venkatraman]

Thanks Paolo. I was able to use your suggestions and incorporate my Rscript within the NF framework and summarize data from the quant.sf files from the different folders.

I can share that code - it would be beneficial to people on this forum; the Rscript code can be useful extensions to these nf-core workflows as well - https://github.com/nextflow-io/rnaseq-encode-nf and https://github.com/nextflow-io/rnaseq-nf

[Paolo Di Tommaso]

Nice, thanks.

p