Import parameters from another file (or other variables)

[Z]

Hello Paolo,

I hope you're doing good !

I tried to search if someone asked this question previously but I couldn't find anything. Let's say I have a main.nf that contains the parameters defined at the top along other variables and a nextflow.config. I tried to create a params.config and put them inside the file. Then I edited nextflow.config to includeConfig "params.config".  Is there a way to tell Nextflow that the parameters given at the terminal are for params.config and not for main.nf ?

Thanks !

[Z]

Oh and I have another question please. As you know, I am using SLURM to run my jobs. When I use publishDir, the outputs are copied/moved to the local path given. How can I tell Nextflow to pass the path of publishDir to SLURM and outputs the results in the directory of the compute node instead of a local directory. if it is not possible, then I guess it must be a shared directory too ?

[Paolo Di Tommaso]

> Is there a way to tell Nextflow that the parameters given at the terminal are for params.config and not for main.nf ?

Quick answer: no. 

p

 

--
You received this message because you are subscribed to the Google Groups "Nextflow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+u...@googlegroups.com.
Visit this group at https://groups.google.com/group/nextflow.
For more options, visit https://groups.google.com/d/optout.

[Paolo Di Tommaso]

> How can I tell Nextflow to pass the path of publishDir to SLURM and outputs the results in the directory of the compute node instead of a local directory

This does not make much sense for what publishDir is designed for. 

p

--

[Z]

Hey Paolo,

Sorry could you please explain me what do you mean by that ? Maybe I didn't explain myself better. I am using publishDir to output my results in a particular directory. When using SLURM, he publishes the results in a path on the controller node (local) instead of the computing nodes directory (remote).

Thanks !

[Paolo Di Tommaso]

The publishDir is not aware if the path is local or remote. Therefore if you want that files in a remote directory, you need to provide a shared file system path (accessible from the computing node) that allows you to write that files remotely. 

p

[Steve]

In regards to "importing parameters from another file", I have found it useful to import from a JSON file instead. Example usage here:

https://github.com/NYU-Molecular-Pathology/demux-nf/blob/a86b6aeeb73933a5e22574aa22772ecf016bb450/main.nf#L9

The basic code would look something like this:

import groovy.json.JsonSlurper
def jsonSlurper = new JsonSlurper()


// new File object from your JSON file

def ConfigFile = new File("config.json")


// load the text from the JSON

String ConfigJSON = ConfigFile.text


// create a dictionary object from the JSON text

def myConfig = jsonSlurper.parseText(ConfigJSON)


// access values in the dict
log.info "Some val: ${myConfig.some_val}"

[Paolo Di Tommaso]

This is a good candidate for the patterns page 

https://github.com/nextflow-io/patterns

p

--