Can we run multiple instances of the same nextflow pipeline on a single server?

[EDoster]

Hello everyone, 

Our group is trying to run two instances of the same nextflow pipeline on a large cluster computer and keep getting a generic error that I can't decipher. I was wondering if there could be an issue with trying to run the same pipeline, using two accounts, on a server that uses the slurm scheduler? 

We know that the pipeline will run successfully on a single account, but when we tried starting a second one we get this error:

[Task monitor] DEBUG nextflow.Session - Session aborted -- Cause: Process `RunQC (IE20_concatenated)` terminated for an unknown reason -- Likely it has been terminated by the external system

However, as I try to figure out what caused the error, I noticed that the process was actually successfully completed:

 Task completed > TaskHandler[jobId: 1870929; id: 11; name: RunQC (IE20_concatenated); status: COMPLETED

And all the output files were in the work directory as expected. I'm wondering if I'm missing something obvious or recommendations for how else I should go about troubleshooting this error.  

Thank you for your time!

-Enrique

[Paolo Di Tommaso]

Even though is suggested to launch different runs in separate directories, it's generally safe. 

If the problem persists open an issue on GitHub including the .nextflow.log file generated using the following command line: 

nextflow -trace nextflow.executor run ... [you command line]

Hope it helps

p

--
You received this message because you are subscribed to the Google Groups "Nextflow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+u...@googlegroups.com.
Visit this group at https://groups.google.com/group/nextflow.
For more options, visit https://groups.google.com/d/optout.

[Steve]

I have been getting these exact errors a lot and I am 99% certain it is due to the storage device stalling out and being inaccessible when Nextflow tries to read the task's work dir contents to determine its status.

[Xin Gong]

I was wondering if there is a solution to this issue.

Best,

Xin

[Xin Gong]

I think this would a solution:

1. edit a new nextflow.config file

$ cat run1.nextflow.config

workDir="${params.wd}"

2. every run uses it's own config file

$nextflow run pipeline.nf --wd /<path>/work_run1 -c run1.nextflow.config

Xin