Singularity and runC

[Taras Shapovalov]

Hi guys,

Could you help me to understand the difference (from user prospective) between Singularity and runC.

In case of runC you need to have some unpacked bundle in a filesystem directory, then you go inside it and run runc command (you need to have some spec file inside the directory that describes the container). And runC starts a new container for you. It seems that Singularity does almost the same. The only difference I see is how Singularity helps to package the bundle (resolves dependencies) + how it is integrated with OpenMPI + in v2.0 the image is not unpackaged into a cache. Is my understanding correct?

Have you ever considered using runC as a backend of Singularity?

Best regards,

Taras

[Gregory M. Kurtzer]

Hi Taras,

There are a number of reasons why runC will not work on my shared multi-tenant environments:

- requires root to run (there is however a submitted patch to allow non-root, but it has not been accepted at this point)

- even with the proposed patch, no mitigation of user escalation within the container

- the container files themselves are owned by root, thus a user can not "bring their own environment"

- no facility or optimizations for MPI or parallel job launch

- requires a very recent host operating system (RHEL7 and compats, and similar vintage Debian derivatives)

- no automatic control of which namespaces to use (e.g. automatic disable PID namespace separation for OMPI shared memory optimizations)

- it is not a "mobility of compute" solution (it is an example implementation of the OCI)

Singularity on the other hand addresses these issues and more:

- Because Singularity uses a single file for the container, that single file can be owned by a user but contain root owned files inside (thus a user can copy from another system)

- Single file also optimizes parallel runs with lots of open()s (large python runs can take 10-30 minutes to start on a big system, but not in a container image)

- Designed for: mobility/portability, speed, HPC, application virtualization (running apps within the container as if they are running on the host)

- Works on all currently maintained vintages of Linux (e.g. RHEL 5 and compats)

- No limitations on vintage of Container OS (e.g. I have a 17 year old install (RHL8) running in a Singularity image)

I think there are a number more, but this should frame some of the primary bits. Anybody else have any differentiators of runC and Singularity?

This might make a great FAQ entry! I will work on that...

Greg

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

--

Gregory M. Kurtzer
High Performance Computing Services (HPCS)
University of California
Lawrence Berkeley National Laboratory
One Cyclotron Road, Berkeley, CA 94720

[Tru Huynh]

Hi,

On Fri, May 27, 2016 at 04:45:10PM +0300, Taras Shapovalov wrote:
> Hi guys,
>
> Could you help me to understand the difference (from user prospective)

> between Singularity and runC <https://github.com/opencontainers/runc>.

I haven't played with runC, but your mail has stirred my curiosity.

>
> In case of runC you need to have some unpacked bundle in a filesystem
> directory, then you go inside it and run runc command (you need to have
> some spec file inside the directory that describes the container). And runC
> starts a new container for you. It seems that Singularity does almost the
> same. The only difference I see is how Singularity helps to package the
> bundle (resolves dependencies) + how it is integrated with OpenMPI + in
> v2.0 the image is not unpackaged into a cache. Is my understanding correct?
>
> Have you ever considered using runC as a backend of Singularity?

- I am still running CentOS-5, and runC needs golang which does not
support my current soon EOL distribution of choice.
I failed to compile the latest go 1.6.2 from go 1.4.2/1.4.3:
<snip>
[tru@sillage src]$ GOOS=linux GOARCH=amd64 ./bootstrap.bash
#### Copying to ../../go-linux-amd64-bootstrap

#### Cleaning ../../go-linux-amd64-bootstrap

#### Building ../../go-linux-amd64-bootstrap

##### Building Go bootstrap tool.
cmd/dist
FATAL: kernel too old
go build _/home/tru/build/go-linux-amd64-bootstrap/src/cmd/dist: /c5/shared/go/amd64/1.4.2/pkg/tool/linux_amd64/6g: signal: segmentation fault
</snip>

That pretty much a road block for me: I can't even compile the compiler to build the tool!
Of course that not an issue if your are running a recent linux distribution, but what
will happen in a few years when golang and runC move forward?

OTOH, I just need a basic environment and I can build singularity on all my
platforms, (CentOS-5 CentOS-6 and now CentOS-7) the idea is being able to use
singularity to allow the newer distribution to run the older binaries without
too much effort.
RHEL is providing N->N+1 runtime "assurance" between major releases,
singularity might give me N->N+2 or more without too much effort (ie by
packaging and not recompiling).


With singularity I gain a backward compatibility for free: a CentOS-6 (glibc
2.12) getconf running on CentOS-5 is neat:
[tru@sillage singularity]$ rpm -q glibc
glibc-2.5-123.el5_11.3.i686
glibc-2.5-123.el5_11.3.x86_64
[tru@sillage singularity]$ singularity exec Centos-6.img getconf GNU_LIBC_VERSION
glibc 2.12

Of course there are limitation, a CentOS-7 container does not run on my CentOS-5:
[tru@sillage singularity]$ singularity exec Centos-7.img df
FATAL: kernel too old

Fair enough, retro compatibility has limits!

Likewise, I hope to be able to package executable accross linux distributions:
debian/ubuntu/alpine. Portability and simplicity are strong selling points.

I still build from sources, rpm build for a specific distribution, using
modules (or lmod) to isolate the dependencies and software versions (or use
easybuild/spark/SCL/... your choice of framework). Singularity would allow
me to give these prebuilt tools to others distributions of the same family
or not.

Bottom line, I believe that singularity is adressing my needs, ymmv :D

Best regards,

Tru
--
Dr Tru Huynh | http://www.pasteur.fr/research/bis
mailto:t...@pasteur.fr | tel/fax +33 1 45 68 87 37/19
Institut Pasteur, 25-28 rue du Docteur Roux, 75724 Paris CEDEX 15 France

[Taras Shapovalov]

Hi Gregory and Tru,

Thanks for the answers. Indeed, you convinced me that runC does not substitute Singularity now.

But I still have some doubts about the future of Singularity ... when runC becomes a part of each linux distribution. Docker since 1.11 uses runC as a backend. And runC is a standard since last Summer. Although for now runc is not a part of each linux distro, I have feeling it will be. So I wonder if you, Gregory, is thinking about using runC as a backend keeping all the current Singularity features.

Best regards,

Taras

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

--

/T

[Gregory M. Kurtzer]

I am not considering using runC as a backend for Singularity because it would be impossible to keep feature parity. The solutions from my perspective are like comparing apples and oranges. While on the surface distinction maybe difficult to make, runC can not be twisted to do what we need (nor should it be). 

HTH, Greg. 

Sent from my iPhone