Getting started in Singularity

[Koushik Raghavan]

Hey guys,

I'm currently working on a project with Singularity containers and I have a few queries which I was hoping someone could help me out with. I'm just getting started with Singularity so please excuse if my questions are a bit trivial. 

1. From a storage perspective, I want to understand how Singularity handles I/O differently than Docker.

2. If I'm using a driver from the host machine on a Singularity container and the host driver is updated, does it affect the running container?

3. I noticed that Singularity has been renamed to Apptainer according to this link :- https://apptainer.org/news/community-announcement-20211130/ . So are there any major differences b/w singularity and apptainer currently?

Regards,

Koushik Raghavan

[David Trudgian]

Hi Koushik,

 

1. From a storage perspective, I want to understand how Singularity handles I/O differently than Docker.

With Docker, container images are generally built up from multiple overlayed layers, where each layer is stored on disk somewhere. The layers are managed by some storage driver - there are many and different ones are default for different deployments. If you want to run the same container on different machines, then the docker daemon on each machine typically has to keep all of the layers locally on each machine.

Singularity generally uses SIF images. These are a single file. The container is inside the SIF as a SquashFS filesystem. This means that you put a SIF on a shared filesystem, and run it from any machines that can access that filesystem. This is one of the reasons Singularity has been favored in HPC environments. Also, it 'flattens' the I/O traffic to being against a single file, rather than lots of files in an extracted set of layers.

 

2. If I'm using a driver from the host machine on a Singularity container and the host driver is updated, does it affect the running container?

Drivers on Linux are built in to, or are modules that load in to, the host system's kernel. Containers (Singularity or Docker) use the host Kernel. Generally you cannot change a driver while a container is running ... you can only unload and reload kernel modules that are not in use. Behavior of containers that need a kernel driver for certain functionality (such as GPU compute) can depend on the version of the driver on the host. Forward / backward compatibility between drivers and libraries/applications has been improving in things like CUDA, so this is less of an issue than it used to be.

 

3. I noticed that Singularity has been renamed to Apptainer according to this link :- https://apptainer.org/news/community-announcement-20211130/ . So are there any major differences b/w singularity and apptainer currently?

The prior lineage of Singularity was forked twice. From our (SingularityCE) perspective there are now two forks (SingularityCE and Apptainer). Singularity has not been renamed.

• SingularityCE is primarily maintained by Sylabs, who have been maintaining Singularity and now SingularityCE since 2017 and this google group is related to SingularityCE.
• Apptainer is a separate project, though it imports a lot of code from SingularityCE, and SingularityCE imports some code from Apptainer.

Apptainer 1.0 (released Mar 2022) is roughly feature equivalent to SingularityCE 3.9 (released Nov 2021), except that it removes support for the Sylabs Cloud remote builds, and adds some checkpointing support for instances. At this time SingularityCE and Apptainer are almost entirely compatible. See the release notes of each project for some more detail / minor differences.

SingularityCE 3.11 was released in May 2022, adding a range of new features including resource limit support, improved OCI compatibility flags etc. Apptainer has pulled in some of these features for their next release, but that release has not yet been made.

The projects have distinct roadmaps, which we'd recommend reviewing to see the future directions:

• SingularityCE - https://github.com/sylabs/singularityce-community/blob/main/roadmap.md
• Apptainer - https://drive.google.com/drive/u/0/folders/1npfBhIDxqeJIUHZ0tMeuHPvc_iB4T2B6

Hope this is useful.

Cheers,

DT

[Koushik Raghavan]

Hi David,

Thank you for your prompt response, I got what I'm looking for from the 2nd and 3rd query answers. WRT my 1st question, let me try and rephrase it better. So I'm trying to run a container which reads and writes data into an NVME SSD drive in the host machine and I would like to know as to how docker and singularity containers would deal with this. Will they both have the same approach and performance from the storage perspective of the drive solely or will one be better than the other?

Regards,

Koushik Raghavan

[David Trudgian]

Hi Koushik,

If you are working with data on a drive on the host machine, i.e. you have an existing directory holding data, and you are binding that directory into the container with Docker, or Singularity, there will be no difference. Both just use a bind mount, which will pass through I/O to the normal kernel filesystem code without impact.

If that's not the case, let us know a bit more detail about where the data is, how you get it into the docker container etc.

Cheers,

DT

[Koushik Raghavan]

Hi David,

Thank you for the clarification, and yes I was doing some tests wrt the storage IO performance and it seems to concur with what you stated. One last thing I'd like your opinion on, I found this short paper online (https://iopscience.iop.org/article/10.1088/1742-6596/1948/1/012005/pdf) and it compares the performance of AI applications on docker and singularity. They come to the conclusion that singularity gives a better performance wrt CPU and GPU utilization. I was hoping you could shed some light on this if you could, thank you.

Regards,

Koushik Raghavan

[David Trudgian]

The results in the paper are somewhat surprising w.r.t. the magnitude of the performance difference.

I would guess that the Docker GPU results may be including some overhead in container startup, that would be less noticeable with a longer test. Here's an example comparing Docker vs bare metal for tensorflow, with a much smaller (if any) performance difference observed vs bare metal.

https://www.exxactcorp.com/blog/Deep-Learning/is-docker-ideal-for-running-tensorflow-gpu-let-s-measure-using-the-rtx-2080-ti

I would advise that you test for your workload, rather than rely on a single benchmark paper - and would expect only small differences.

--
You received this message because you are subscribed to the Google Groups "Singularity Community Edition" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity-c...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/singularity-ce/85238dde-ec1f-4516-b775-b086d9fc9077n%40googlegroups.com.

--

David Trudgian

Sylabs Inc.