Build singularity image from local docker image

[Merket, Noel]

I'm trying to convert a docker image to a singularity image, but I'd prefer to do it all locally, rather than uploading it all to docker hub or another repository. I've had limited success with running the image and using the docker export command on the container and then building a singularity image from that, but it seems that environment variables set it the Dockerfile get lost in that kind of conversion. I think it's because docker export only exports the container's file system.

 

docker build -t my-image .
docker run -it --name for_export my-image /bin/true
docker export for_export > my-image.tar
singularity build my-image.simg my-image.tar

singularity shell my-image.simg
echo $ENVVAR_SET_IN_DOCKERFILE
(nothing returned)

 

 

When I push the docker image to docker hub and build from there, it works as expected:

 

singularity build my-image.simg docker://nmerket/my-image

singularity shell my-image.simg
echo $ENVVAR_SET_IN_DOCKERFILE
(expected value returned)

 

I'd like to not have to upload and then download my image to Docker Hub to get this to work. Is there any way I can take a local docker image and convert it into a singularity image using the same mechanisms it uses to build a singularity image from layers on Docker Hub? I tried using the docker image save command and using the resulting tar file to build a singularity image, but that didn't work at all.

 

Thanks for your help,

Noel

 

[v]

The quickest thing to try (without needing to set up a local registry) is docker2singularity --> https://github.com/singularityware/docker2singularity/tree/v2.5

I'm linking the branch with version 2.5 of Singularity.

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

--

Vanessa Villamia Sochat
Stanford University '16

(603) 321-0676

[David Trudgian]

Hi Noel,

Here's a general pattern for doing this... pushing the image into a local docker registry, and then pulling out to singularity:

# Start a docker registry

$ docker run -d -p 5000:5000 --restart=always --name registry registry:2

# Push local docker container to it

$ docker tag alpine localhost:5000/alpine

$ docker push localhost:5000/alpine

# Create def file for singularity like this..

# (add your modifications)

Bootstrap: docker

Registry: http://localhost:5000

Namespace:

From: alpine:latest

# Build singularity container

$ sudo SINGULARITY_NOHTTPS=1 singularity build alpine.simg def

You'll be glad to know that this will improve a lot with Singularity 3.0, our rewrite in Golang. We've just merged functionality into the development branch for 3.0 that allows direct import of docker save archives, oci archives and layouts, and even direct imports pulling images from the docker daemon itself with no intermediary. Leveraging the excellent container libraries available for Go is letting us streamline a lot of things.

Cheers,

Dave T

[haris zaf]

Hi Dave, 

I tried to do as you said but I got this error:

FATAL:   While performing build: conveyor failed to get: Error reading manifest v1 in localhost:5000/hariszaf/pema: manifest unknown: manifest unknown

Do you have any idea what this is about?

[Randy Herban]

Haris,

What version of Singularity are you running and have you followed the directions in this page:

https://sylabs.io/guides/3.2/user-guide/singularity_and_docker.html#locally-available-images-stored-archives

--

You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/singularity/1255d8ce-b7a9-4cf6-ba05-11bcf2cb8a4f%40lbl.gov.

[haris zaf]

Randy thanks for the reply. 

I made it using your link!

Thanks again,

Haris

To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/singularity/CAPw58N6AW9KS6LgyT7NOtp-Ay9p5W2gyQ7%2BKVQ1BG1BP%3D7zgwQ%40mail.gmail.com.

--

-- 

Haris Zafeiropoulos, PhD student
Biodiversity Informatics and Ecological Network Analysis
Institute of Marine Biology Biotechnology and Aquaculture (IMBBC)
Hellenic Centre for Marine Research (HCMR)
Departmnet of Biology, University of Crete

Crete, Greece