build exceeded max build time

[Dianne Patterson]

Dear experts,

From our HPC, I tried this: 

singularity build --remote fmriprep.sif docker://poldracklab/fmriprep

However, it eventually timed out and thus failed: 

5cf17cfd46f1aa23bc9fcd5b build exceeded max build time FATAL:   While performing build: build has not completed 

Can you tell me whether the problem is on the HPC side or the remote builder at Sylabs side?

Thanks so much,

Dianne

[v]

Why do you need the --remote if you just want to build from a docker container? If you did build with the remote, you'd have to download the result anyway. Why not try:

$ singularity build fmriprep.sif docker://poldracklab/fmriprep

In the time it took me to read your email, and write up this response, the image has built and is on my computer. The creation of the sif binary is what takes the longest, not downloading the layers.

If you run into issues with that command, you likely need to export the singularity cache directory to be somewhere with more space (e.g., not home on a cluster) or you ran out of memory. This would have been killed on my cluster login node due to that. If the remote builder did bug out (and the command for a docker uri is valid) I'd guess he ran out of memory too :)

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.
To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/singularity/c77fd727-4d70-4554-8eb2-ed2bf902ca6a%40lbl.gov.

--

Vanessa Villamia Sochat
Stanford University '16

(603) 321-0676

[Dianne Patterson]

Thanks, before trying the remote option, I tried building directly on the HPC, with exactly the command you suggested:

singularity build fmriprep.sif docker://poldracklab/fmriprep

but got this message every time:

FATAL:   While performing build: While running mksquashfs: signal: killed: 

I could build lolcow directly on the HPC, but neither fmriprep not bids/mrtrix3_connectome (both got almost all the way through and then failed with the highlighted message above). 

I have built successfully on our linux box at home and transferred the resulting sif files...but I'd LOVE to be able to build directly on the HPC.

Thanks for your quick reply. I'll try to move the cache and see if it'll run.

-Dianne

[v]

Yep, looks like it was killed because it took too much memory to squash. If you are on HPC, make sure you are on an interactive node (and not the login node). If you need to get an interactive node with more memory, you can usually ask for one with srun:

srun --mem 32000 --time 8:00:00 --pty bash

You did everything right, you just need more memory :) 

--

You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/singularity/0ef0f32e-0db6-44e2-a98f-cea211da06ba%40lbl.gov.

[Dianne Patterson]

Thanks!  We don't have slurm, but I got it working with PBS and qsub.  This is very cool.

-Dianne

On Friday, May 31, 2019 at 4:04:15 PM UTC-7, vanessa wrote:

Yep, looks like it was killed because it took too much memory to squash. If you are on HPC, make sure you are on an interactive node (and not the login node). If you need to get an interactive node with more memory, you can usually ask for one with srun:

srun --mem 32000 --time 8:00:00 --pty bash

You did everything right, you just need more memory :) 

On Fri, May 31, 2019 at 6:50 PM Dianne Patterson <dian...@gmail.com> wrote:

Thanks, before trying the remote option, I tried building directly on the HPC, with exactly the command you suggested:

singularity build fmriprep.sif docker://poldracklab/fmriprep

but got this message every time:

FATAL:   While performing build: While running mksquashfs: signal: killed: 

I could build lolcow directly on the HPC, but neither fmriprep not bids/mrtrix3_connectome (both got almost all the way through and then failed with the highlighted message above). 

I have built successfully on our linux box at home and transferred the resulting sif files...but I'd LOVE to be able to build directly on the HPC.

Thanks for your quick reply. I'll try to move the cache and see if it'll run.

-Dianne

On Friday, May 31, 2019 at 2:08:53 PM UTC-7, Dianne Patterson wrote:

Dear experts,

From our HPC, I tried this: 

singularity build --remote fmriprep.sif docker://poldracklab/fmriprep

However, it eventually timed out and thus failed: 

5cf17cfd46f1aa23bc9fcd5b build exceeded max build time FATAL:   While performing build: build has not completed 

Can you tell me whether the problem is on the HPC side or the remote builder at Sylabs side?

Thanks so much,

Dianne

--
You received this message because you are subscribed to the Google Groups "singularity" group.

To unsubscribe from this group and stop receiving emails from it, send an email to singu...@lbl.gov.

To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/singularity/0ef0f32e-0db6-44e2-a98f-cea211da06ba%40lbl.gov.

[v]

Awesome!!

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.
To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/singularity/a269a864-9088-4c67-b844-447d2941dbaa%40lbl.gov.