Wait for nextflow -bg process in wrapper script

[Susanne Motameny]

Hi,

I have a bash wrapper script that runs a nextflow pipeline with the -bg option. The wrapper script should wait for the nextflow process to finish and then execute some commands. However, I do not manage to make the wrapper script wait for the nextflow process properly and then continue. I tried it like this:

nextflow -bg run main.nf

wait

echo "nextflow finished"

But the wrapper script exits right after starting the nextflow process (which keeps running and executes normally). I start the wrapper script with the nohup command and use nextflow version 20.10.0. Can anybody suggest a working solution for this?

[Paolo Di Tommaso]

Why using `-bg` to have a `wait` immediately after? 

--
You received this message because you are subscribed to the Google Groups "Nextflow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nextflow/0d1d2622-3a25-472e-a03b-cce7db5071cdn%40googlegroups.com.

[Susanne Motameny]

Because if I don't use `-bg`, the wrapper process is stopped after the nextflow call and the nextflow process does not run.

[Susanne Motameny]

I digged a little deeper and ran some more tests. The following solution works as desired:

wrapper.sh:

#!/bin/bash

nextflow run main.nf

echo "nextflow finished"

Command line to run the wrapper script:

nohup ./wrapper.sh

So the `-bg`option is actually not necessary to run the wrapper script in the desired way.

The problem with the stopped processes appears when starting the wrapper script as a background process like this:

nohup ./wrapper.sh &

In this case, both the wrapper script and the nextflow process are stopped and do not run, while being listed as stopped processes in the process table.

The only inconvenience with this solution is that when I launch without the ampersand, I cannot continue working on the command line but have to open a different terminal.

[Felix Krueger]

I am not sure if it is helpful in this context, but we had a similar issue with running Nextflow from within automated analysis scripts.

The main issue was that the `nextflow` command needed to be started in either "-ansi-log false" or "-quiet" mode, as it would otherwise ask for input from the Terminal - which meant it would go into an indefinite sleep state if wrapped within a nohup & command.