how to print out all sbatch commends?

[dk lcdbi]

Hello, I'm new with nextflow, would appreciate any help.

One process was not submitted, no error message, and the pipeline finished "successfully". On the monitoring screen, you can see the process name, but no executor, no updates (see the screenshot). There is no error from the execution of the whole pipeline and no error in  .nextflow.log. There are logs for the finished processes, but no log for this process.  No entry for this process when run "nextflow log <run name>". 

Seems this process was not submitted at all. Why not submitted? Failed after submission? How can I check if it is submitted or not? Is there a way from nexflow to print out a sbatch commend for a process, no matter it's submitted or not? I'd like manually test if the commend works or not. I use SLURM. 

Thanks a lot!

Dave

[Alan Hoyle]

I've learned two things about this:

1.  if you  turn on nextflow run -with-trace, you can see more details about each process including the JOB_ID, which you can dump into sacct or squeue to get information about past and currently running jobs.  

2.  in the screen output where you see the [12/34abcd] that tells you the path to the working directory for that processes, so you can go there and look at the scripts and the temporary outputs.

For example there's a [44/cf9ee6] for one of your processes.  If you cd $NXF_WORK/44, you'll see a subdirectory there that begins with cf9ee6, and has a bunch more characters.

like 44/cf9ee637e38f15291c04b7e99544ad or something (those seem to be an md5sum, so this obviously isn't correct for your context.  

go there and your temporary files are saved, and the actual process that gets run is in the file .command.sh  

If you run in a container environment, you might need to look at the .command.run to extract that context information.  I don't know how it would work for Conda or Docker, but for singularity, you can figure it out by running "grep singularity .command.run" in that directory.  

--
  -  Alan Hoyle  -  al...@alanhoyle.com  -  http://www.alanhoyle.com/  -

--
You received this message because you are subscribed to the Google Groups "Nextflow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nextflow/2e09ce3d-a237-4c6f-a5ba-75bb0eec0a8cn%40googlegroups.com.

[dk lcdbi]

Thank you very much, Alan. 

All your tips are helpful for a newbie like me. I found out that the process was certainly not submitted. It's odd because the pipeline script actually ran through that part and didn't complain about anything. The trace and all the other tips you mentioned take care of the submitted processes only. How should I troubleshoot a process not submitted yet no error message? I put some println lines in the process block of the Groovy script, and I can see it went through. What happened to the process? Why not submitted? If failed, why no error? 

I suspect that the excurator took that process code and didn't do anything for some reason. Any suggestions for troubleshooting? 

I was using Conda and some module environments. 

thanks a lot again,

Dave

To view this discussion on the web visit https://groups.google.com/d/msgid/nextflow/CAKxVof1SqHe2z-i96mJD0mOVWqF%3D8XGbGDU41%3DHyK6cz46S%2BpA%40mail.gmail.com.