Using bind with overlay to gain access to large directory tree

[David Fletcher]

Hello all,

I'm new to singularity and to this group.  First, thanks for making this tool available.

I've got a container image that has an OS (CentOS 6 for now) and a collection of C/C++ compilers and so on.  This is working fine.

What I'd like to do is get access to a large directory tree inside the container.  For many reasons, I don't want to copy the (large)

tree into the container image.  What I'd like to do is form an overlay file system inside the container where the lower, read-only

layer is the large directory tree on the host's disk and the upper, read-write level of the overlay file system would point to another

directory tree that is located elsewhere.

I've tried this:

        $ singularity image.create overlay-experiment.img

        $ singularity shell --bind /path/to/dev/tree:/work:ro --overlay overlay-experiment.img Centos_6_7.img

This didn't work.  Inside the container, if I try to touch/modify a file, Singularity complains that the file system is read only.  If I

remove the ':ro', I can touch/edit files but the /path/to/dev/tree file system is modified.  ie, it's not functioning as an overlay.

Maybe my understanding of overlays is weak.  Is there a combination of arguments that will give me the arrangement I'm after?

Thanks, in advance, for any help you can provide,

David

[Jason Stover]

Hi David,

I don't believe that will work... IIRC the overlyfs image is placed
on top of /. Then those other mounts are placed. So their permissions
(i.e. read-only) win since the overlay is already in place.

Generally, we take file system permissions to restrict writing to
bound directories. Otherwise, in most use cases, they are actually
used for program output.

If you look at "src/start.c" (I'm pretty sure this is the right
place, but others know the C code better than I do) you can see the
order that the mounts happen... So by default it goes:

singularity_runtime_overlayfs();
singularity_runtime_mounts();

You could _try_ swapping singularity_runtime_overlayfs() and
singularity_runtime_mounts()... and see if that gets the expected
behavior you are after. But test on a junk directory...

I was looking at 2.4.4 tag when going through this, and the above is
a guess on my part from scanning over the code here.

-J

> --
> You received this message because you are subscribed to the Google Groups
> "singularity" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to singularity...@lbl.gov.

[David Fletcher]

Hi Jason,

Thanks for the response.  I was running an older version of singularity installed by our IT

dept so I downloaded/installed the 2.4.5 release and made the change as you suggested.

The change in ordering didn't affect the behaviour.  ie, with the :ro qualifier on the --bind

argument, the mounted directory cannot be modified; without the :ro qualifier, data is "written

through" to the host's directory tree.  (I removed the --overlay argument just to see if this had

any effect.  It didn't.)

If I run the mount command inside the container, I'm seeing just the / partition.  There's no

indication of an overlay mount.  Is this expected?

I ran singularity with the --debug option and saw lots of output but nothing that told my

inexperienced eyes anything useful.

Any other suggestions?  Thanks for the help, by the way.  Much appreciated,

David