Running Singularity images in Docker

[Sundar Vijayakumar]

Hi Guys,

Apologies if this topic has been covered elsewhere.  Has anyone successfully run a singularity image within Docker?  If so, what was the Docker platform?  Also, wondering if there are any tools for converting a Singularity image file into Docker image.  

Thanks,

-Vijay

[v]

Yep, you just need to add the --privileged tag, here are some examples:

 - docker2singularity converts a Docker image to Singularity, inside a Docker image.

- tunel is a Dockerized interface that runs singularity commands internally, it's under development

The easiest thing to do to convert singularity back to Docker is just to "undo" the Singularity-ization of the recipe. Tunel (above) has an interface that does this:

but I haven't updated the Docker image with recent changes, so I'd advise to use this tool on the command line, with the container tool Global Clients, instructions and examples are here -->

https://singularityhub.github.io/singularity-cli/recipes

Please file an issue if you have questions, and I'd be glad to help!

https://github.com/singularityhub/singularity-cli/issues

Best,

Vanessa

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

--

Vanessa Villamia Sochat
Stanford University '16

(603) 321-0676

[Sundar Vijayakumar]

Hi Vanessa,

Thanks for the prompt response.  We are NOT looking to rebuild a docker image from a recipe file since we have an image with many custom application installs and configuration that has been made to the Singularity image.  We are looking for a way to install singularity within Docker and execute a ready made Singularity container in .simg or *.img formats.   Does it make sense?  We would like to execute containers developed on HPC (that have been qualified) on the user's desktop for running quick  analysis for model development and/or reporting.

-Vijay

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

--

Vanessa Villamia Sochat
Stanford University '16

(603) 321-0676

[v]

Why not start with a scratch /empty / equivalent OS base and just export the entire contents? Metadata you would just need to map from one image to the other. I can throw something together a bit later today if you are interested!

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

[Sundar Vijayakumar]

Hi Vanessa,

Not sure I understand your proposal.  Are you suggesting exporting the *.simg to a sandbox and then adjusting the metadata so that Docker can recognize it?  We did try to install Singularity inside docker but couldn't mount the image.  It complained about support for Squash FS following which we exported the contents out and tried exec'ing from a sandbox.  Cant recall offhand what error it threw but it did not work.

Anyway, would appreciate if you can come up with a script/recipe to execute this without requiring a container rebuild.

Thanks,

-Vijay

To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

--

Vanessa Villamia Sochat
Stanford University '16

(603) 321-0676

--
You received this message because you are subscribed to the Google Groups "singularity" group.

To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

--
You received this message because you are subscribed to the Google Groups "singularity" group.

To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

--
You received this message because you are subscribed to the Google Groups "singularity" group.

To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

[v]

Definitely! Let me poke around and try some things out :)

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

--

Vanessa Villamia Sochat
Stanford University '16

(603) 321-0676

--
You received this message because you are subscribed to the Google Groups "singularity" group.

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

--
You received this message because you are subscribed to the Google Groups "singularity" group.

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

--
You received this message because you are subscribed to the Google Groups "singularity" group.

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

[v]

okay, poking is complete!

https://github.com/vsoch/singularity2docker

There are definitely tweaks that can be done to improve this, but this is the simplest way to convert singularity2docker.sh. If you have suggestions please open PR! Thanks for the fun idea, dinosaur off to bed! :)

[Sundar Vijayakumar]

Hi Vanessa,

Thank you so much for pulling this script together.  I really appreciate your prompt efforts! I ran into a few issues with space in /tmp, not having jq installed, etc. but moved past them.  It finally started building Docker image around 4 AM this morning and hasn't finished yet.  I reviewed the /var/lib/Docker folder and its size in not changing anymore, so it looks like the Docker build seems to have stalled.  The squash FS image size was 970M and uncompressed sandbox was 5.5 GB.  Wondering if size could be an issue or something else is causing this?  I just used one of our existing containers but will try with a smaller container next.  Stay tuned...

-Vijay

To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

[v]

I think that is likely (5.5GB might be big!) so the first sanity check is to check it running with a smaller image. For the jq issue, @tru pointed out that I didn't check that it was isntalled and tere

is a PR in to do that --> https://github.com/vsoch/singularity2docker/pull/2 If you would like to use this branch and check that it is A-OK I can merge into master so you have this check by default.

We will take it from there! As a second sanity check to this, we would want to try stepping through the various commands in the script and seeing where it gets tied up. It's likely the build step but who knows without seeing what is going on. Let me know how it goes and we will figure out what to test next.

Best,

- another V :)

To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.