HiC-Pro error / make warning when using Singularity (make[1]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule.)

[Anke Busch]

Hello,

I'm trying to use HiC-Pro through Singularity. I created my own Singularity image using the 'Singularity' recipe file provided with HiC-Pro v3.1.0. I ran the following command to create the image:

SINGULARITY_TMPDIR=/mnt/storage/scratch singularity build --fakeroot hicpro.simg Singularity

This worked nicely (incl. showing HiC-Pro's help) apart from the following warnings:

INFO:    Running testscript
source: /usr/local/conda/envs/hicpro/etc/conda/activate.d/activate-binutils_linux-64.sh:10:40: parameter expansion requires a literal
source: /usr/local/conda/envs/hicpro/etc/conda/activate.d/activate-gcc_linux-64.sh:10:40: parameter expansion requires a literal
source: /usr/local/conda/envs/hicpro/etc/conda/activate.d/activate-gfortran_linux-64.sh:10:40: parameter expansion requires a literal
source: /usr/local/conda/envs/hicpro/etc/conda/activate.d/activate-gxx_linux-64.sh:10:40: parameter expansion requires a literal

Then I tried running the Singularity image on the provided HiC-Pro test dataset with the following command:

singularity exec hicpro.simg HiC-Pro -c config-hicpro.txt -i testdata_HiCPro/test_data/ -o hicpro_latest_test

This throws the following warnings and error messages:

source: /usr/local/conda/envs/hicpro/etc/conda/activate.d/activate-binutils_linux-64.sh:10:40: parameter expansion requires a literal
source: /usr/local/conda/envs/hicpro/etc/conda/activate.d/activate-gcc_linux-64.sh:10:40: parameter expansion requires a literal
source: /usr/local/conda/envs/hicpro/etc/conda/activate.d/activate-gfortran_linux-64.sh:10:40: parameter expansion requires a literal
source: /usr/local/conda/envs/hicpro/etc/conda/activate.d/activate-gxx_linux-64.sh:10:40: parameter expansion requires a literal

Run HiC-Pro 3.1.0
--------------------------------------------
--------------------------------------------
Mon Jul  4 10:44:48 CEST 2022
Mon Jul  4 10:44:48 CEST 2022
Bowtie2 alignment step1 ...
--------------------------------------------
Bowtie2 alignment step2 ...
--------------------------------------------
Mon Jul  4 10:44:48 CEST 2022
Mon Jul  4 10:44:48 CEST 2022
Combine R1/R2 alignment files ...
--------------------------------------------
Mapping statistics for R1 and R2 tags ...
--------------------------------------------
Mon Jul  4 10:44:48 CEST 2022
make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
Mon Jul  4 10:44:48 CEST 2022
make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
Pairing of R1 and R2 tags ...
--------------------------------------------
Assign alignments to restriction fragments ...
--------------------------------------------
Mon Jul  4 10:44:48 CEST 2022
Mon Jul  4 10:44:48 CEST 2022
make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
Merge chunks from the same sample ...
Merge stat files per sample ...
make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
.
. [MY COMMENT: multiple times the same warning message]
.
make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
find: 'bowtie_results/bwt2_global': No such file or directory
Nothing to align ! Please check input files and R1/R2 extension.
make: *** [/usr/local/bin/HiC-Pro_3.1.0/bin/../scripts//Makefile:101: bowtie_local] Error 1
make: *** Waiting for unfinished jobs....
find: 'bowtie_results/bwt2_global': No such file or directory
find: 'bowtie_results/bwt2_global': No such file or directory
Logs: logs/dixon_2M/merge_stats.log
find: 'bowtie_results/bwt2': No such file or directory

Logs: logs/dixon_2M_2/merge_stats.log

find: 'hic_results/data/': No such file or directory
make: *** [/usr/local/bin/HiC-Pro_3.1.0/bin/../scripts//Makefile:160: merge_valid_interactions] Error 1
find: 'bowtie_results/bwt2': No such file or directory

Logs: logs/dixon_2M/mapping_step1.log
Logs: logs/dixon_2M_2/mapping_step1.log

To me it looks like the first step of mapping is successfully started and run (I can see the logs and result files later on). However, the main (parent) process doesn't seem to communicate well with the child process and doesn't recognize it's a make file. Thus, the main process does not wait for the child process to finish before continuing to the next task. Since the first child (step1) is not finished when the second step is started, the latter is missing its input files and, thus, crashes saying "find: 'bowtie_results/bwt2_global': No such file or directory." and "Nothing to align ! Please check input files and R1/R2 extension."

Do you have any idea what I am doing wrong and how this could be fixed?

Thanks a lot in advance.
Best,
Anke.