Anyone gotten snap working in a container?

[Jonathan Greenberg]

I'm trying to install a piece of software called "cloudcompare" that has a "snap" installer.  When I add this into a bootstrapped xenial container:

apt-get install -y snapd

snap install cloudcompare

I get a:

error: cannot communicate with server: Post http://localhost/v2/snaps/cloudcompare: dial unix /run/snapd.socket: connect: no such file or directory

There were some suggestions to:

systemctl restart snapd.service

But I get:

Running in chroot, ignoring request.

Any thoughts?  

Here's the GIT path to the full definition that keeps failing:

https://github.com/gearslaboratory/gears-singularity/blob/master/Singularity/specialized_use/gears-singularity.cloudcompare

--j

[David Godlove]

Did you try as root? I'm guessing that might require root privs. 

Snaps are basically containerized apps afaik. This could lead to some weirdness.

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

[Jonathan Greenberg]

I'm trying to build on singularity hub -- are you suggesting:

sudo apt-get snapd 

in the singularity def (I thought normally we avoid sudos)?

--j

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

[v]

You would want to find a solution that works for you locally too - if this doesn't work on your local machine, it's not going to work on Singularity Hub.

To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

--

Vanessa Villamia Sochat
Stanford University '16

(603) 321-0676

[David Godlove]

Oh sorry.  I had a brain fart and thought you were talking about runtime.  

Based on what you are describing, it sounds like snaps are orchestrated by a daemon process.  Right now, Singularity doesn't support running system services (as you discovered when your systemctl command failed I think).  I believe that Cedric Clerget has been working on a way to "boot" a Singularity container so that the init system will work as expected.  This might solve your problem?  There may be some code to do that in the development branch right now.

Dave

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

--

Vanessa Villamia Sochat
Stanford University '16

(603) 321-0676

[Jonathan Greenberg]

Yep, it didn't work locally.  It would be great to have a solution for this, as some folks distribute software using snap.  

--j

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

[v]

This is interesting, snap is another kind of container in itself, one that seems like it was developed for ubuntu phone's OS (not sure that turned out as expected) and thus is now being used more broadly. You really could sub out "snap" with another container technology:

The snap format is a single compressed filesystem that is mounted dynamically by the host operating system, together with declarative metadata that is interpreted by the snap system to set up an appropriately shaped secure sandbox or container for that application. The file format extension is .snap.

https://en.wikipedia.org/wiki/Snappy_(package_manager)

So in the same way you would have trouble running Docker or another Singularity while building without some kind of hack, this is likely to be the case (and I tested out your recipe and came to the same issues with not having control of those services). But it also means that there is likely just a dump of files and metadata that needs to happen, and if you can find another way to do that and get it into the container, this would work! Docker Hub has an API for the metadata and layers, and snap doesn't, but it can be used via your host.

So the direction I would go in, and have caution in doing this because it relies on your host, is to use snap as it was intended - as a package manager. I would try installing the snap in the %setup section (still working on your host) and then basically move it from host to container. Here is a little test I just did - here is what I can figure out.

snap is already installed on my host, install hello-world

snap install hello-world
snap list
Name         Version    Rev   Developer  Notes
core         16-2.31.1  4110  canonical  core
hello-world  6.3        27    canonical  -

Note if you do the above before install, it tells you that you don't have any, and you should install hello-world :) Next, get information about the snap

snap info hello-world

name:      hello-world

summary:   The 'hello-world' of snaps

publisher: canonical

contact:   snappy...@lists.ubuntu.com

license:   unknown

description: |

  This is a simple hello world example.

snap-id: buPKUD3TKqCOgLEjjHx5kSiCpIs5cMuQ

commands:

  - hello-world.env

  - hello-world.evil

  - hello-world

  - hello-world.sh

tracking:    stable

installed:   6.3 (27) 20kB -

refreshed:   2016-07-11 17:20:44 -0400 EDT

channels:             

  stable:    6.3 (27) 20kB -

  candidate: 6.3 (27) 20kB -

  beta:      6.3 (27) 20kB -

  edge:      6.3 (28) 20kB -

Ah I found a download command, let's try that. It looks like it will download to the directory I'm in, and I can't control output name.

$ mkdir dump && cd dump

$ snap download hello-world

Fetching snap "hello-world"
Fetching assertions for "hello-world"
Install the snap with:
   snap ack hello-world_27.assert
   snap install hello-world_27.snap

That could be informative if there is more than just the file structure. But actually if you want the snap contents, just look in the /snap folder:

 tree /snap/hello-world/current

/snap/hello-world/current

├── bin

│   ├── echo

│   ├── env

│   ├── evil

│   └── sh

└── meta

    ├── gui

    │   └── icon.png

    └── snap.yaml

and the metadata is in the yaml file, as you would guess:

$ cat /snap/hello-world/current/meta/snap.yaml 
name: hello-world
version: 6.3
architectures: [ all ]
summary: The 'hello-world' of snaps
description: |
    This is a simple snap example that includes a few interesting binaries
    to demonstrate snaps and their confinement.
    * hello-world.env  - dump the env of commands run inside app sandbox
    * hello-world.evil - show how snappy sandboxes binaries
    * hello-world.sh   - enter interactive shell that runs in app sandbox
    * hello-world      - simply output text
apps:
 env:
   command: bin/env
 evil:
   command: bin/evil
 sh:
   command: bin/sh
 hello-world:
   command: bin/echo

So! This is good news I think, because you would basically install these snaps to your container by copying the contents from your host. You won't be able to interact with the service / daemon and use them as containers, but you should be able to interact with the extracted contents. And given a consistent structure, this very likely could be added as a base to bootstrap from (or a particular kind of other container to input). I would open an issue and ask for this feature if you think it is important, and others can discuss. In the meantime so you don't have to wait, you can add some basic logic/commands in your setup (on the host) to minimally "get the snap guts" into your container!

%setup

    # snap is already installed on my host

    snap install hello-world

    # copy a snap directory to your container somewhere

    SNAP_BASE=/snap/current

    mkdir -p ${SINGULARITY_ROOTFS}$SNAP_BASE

    # Copy hello-world

    cp -R $SNAP_BASE/hello-world ${SINGULARITY_ROOTFS}$SNAP_BASE

or something like that :) I think it would be up to you to determine how to run the contents, since snap is likely not to work (unless the running bit doesn't require the package manager / service thing!) But since they give you an obvious "bin" I think it would be pretty easy to add that to a path. If you make it a SCIF app and install the snap to /scif/apps/hello-world for example, then the contents of the "bin" would automatically be on the PATH when you do a run with --app hello-world. That's pretty neat!

Good luck!

Best,

V

To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

[v]

Speaking of snap... this is pretty awesome that Ubuntu is taking the whole package manager + container thing and integrating it directly into their OS:

https://www.omgubuntu.co.uk/2018/03/ubuntu-budgie-18-04

This might be an interesting general strategy for other container software to consider *wink*  

[John Hearns]

Vanessa, in the immortal words of Scooby Doo...   could be

There is CoreOS of course https://coreos.com/

But lets think of the HPC environment.

We SHOULD have applications which use all cores on a server at 100% utilisation.

The HPC processes (or containers!) should be pinned to particular cores, and have their resources restricted.

We do not want OS processes causing context switches and stopping HPC containers running at full tilt.

My vision of an HPC server building block would be

A lightweight OS layer which exists to start up the containers.

Lightweight OS layer support the high bandwidth network and filesystems.

As little as possible else is made available. Everything else needed should be in a container.

A 'donkey engine'  CPU core.

A donkey engine is a small engine used to start up a big huge engine.

With cpusets you can reserve a 'boot cpuset'with a couple of cores and some memory.

Your OS is then constrained within those cores.

But how about a dedicated low-power CPU core in the package which does the OS tasks?

To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.