SLURM juicer sending jobs to the incorrect nodes

[Gabrielle Coffing]

Hi! 

I hope this message finds you well. I've made several attempts at getting juicer to run over the past few months and continue to run into problems. I hope to get some help with getting it running!

When I follow the instructions to make the appropriate edits in the juicer.sh script, I get a set of errors (pasted at the bottom of this message) indicating that batch job submission has failed due to invalid account information and job dependency problems.

I was able to remove these errors by editing the batch submissions within the juicer.sh script to fit our cluster. Juicer successfully runs on a small subset of my data. However, most of the jobs are still being launched onto the short nodes rather than our lab-specific nodes (kern) that I specify in the script. Our short nodes are default, so I think for some reason many of the jobs get sent to the default partitions. This is a problem because the maximum about of time for short nodes is 24 hours. Juicer ultimately fails on our full dataset at the fragmerge step because 24 hours is not enough time. 

I have attached a link to my edited juicer.sh script. I would really appreciate if I could get some insight to what might be causing the problem. 

Thank you!!

Gabby

https://drive.google.com/file/d/1Kb8pKI8WLuybQcqOkCsPcDMfaLq43quA/view?usp=sharing 

sbatch: error: Batch job submission failed: Invalid account or account/partition combination specified
sbatch: error: Batch job submission failed: Invalid account or account/partition combination specified
sbatch: error: Batch job submission failed: Invalid account or account/partition combination specified
sbatch: error: Batch job submission failed: Invalid account or account/partition combination specified
sbatch: error: Batch job submission failed: Invalid account or account/partition combination specified
sbatch: error: Batch job submission failed: Invalid account or account/partition combination specified
sbatch: error: Batch job submission failed: Invalid account or account/partition combination specified
sbatch: error: Batch job submission failed: Job dependency problem
sbatch: error: Batch job submission failed: Invalid account or account/partition combination specified
sbatch: error: Batch job submission failed: Invalid account or account/partition combination specified
sbatch: error: Batch job submission failed: Invalid account or account/partition combination specified
sbatch: error: Batch job submission failed: Invalid account or account/partition combination specified
sbatch: error: Batch job submission failed: Invalid account or account/partition combination specified
sbatch: error: Batch job submission failed: Invalid account or account/partition combination specified
sbatch: error: Batch job submission failed: Job dependency problem
sbatch: error: Batch job submission failed: Job dependency problem
sbatch: error: Batch job submission failed: Job dependency problem
sbatch: error: Batch job submission failed: Job dependency problem
sbatch: error: Batch job submission failed: Invalid account or account/partition combination specified
sbatch: error: Batch job submission failed: Job dependency problem
sbatch: error: Batch job submission failed: Invalid account or account/partition combination specified

[Moshe Olshansky]

Hi Gabby,

Have you tried using -L and -L flags (for queue time limit and long queue time limits)?

Regards,

Moshe.

[Gabrielle Coffing]

Yes, I used those flags before making edits to the script and after and jobs were still sent to the incorrect nodes. 

[Gabrielle Coffing]

Hello! 

I have tried using Juicer 2 instead of 1.6 hoping that would fix my issue. However, jobs are still being sent to the short nodes rather than the nodes that I specify. Any ideas on what I could try to fix this issue? I have pasted my launch command I am using with my test dataset below. 

bash juicer.sh -t 10 -g obimac -D /home/gcoffing/kernlab/genomeAssembly/outputs/juicer_github -p contigsMiddlePurged.purged.MITO.contig.sizes -y /home/gcoffing/kernlab/genomeAssembly/outputs/juicer_github/restriction_sites/contigsMiddlePurged.MITO_MboI.txt -z /home/gcoffing/kernlab/genomeAssembly/outputs/juicer_github/references/contigsMiddlePurged.purged.MITO.fa -A kernlab -q kern -l kern -C 200000 -Q 12:00:00 -L 72:00:00 -e

Thank you!

Gabby