[lxc-users] Using shiftfs with NFS mounted filesystem

[Michael Eager]

I'm running LXD on a Centos 8 host, kernel-4.18.0. I built and
installed the shiftfs module.

I added a device to a container which mounts a local file system in the
container:
$ lxc config device add devel devel disk path=/<container-path>
source=/<host-path> shift=true
(Both container and mounted fs are named "devel".)

This appears to work with a local FS. Files which were owned by "root"
on the host system show up as owned by "root" in the container, not by
"nobody". I can create and delete files normally.

When the host filesystem is NFS mounted, it initially appears correct in
the container, but there are several odd behaviors:

Creating a file:
[eager@devel devel]$ cd tmp
[eager@devel tmp]$ ls
[eager@devel tmp]$ touch x
touch: cannot touch 'x': Permission denied
[eager@devel tmp]$ ls -l
total 512
-rw-rw-r-- 1 eager eager 0 Jul 18 17:56 x

Deleting a file:
[eager@devel tmp]$ rm x
rm: remove write-protected regular empty file 'x'? y
rm: cannot remove 'x': Permission denied
[eager@devel tmp]$ ls -l
total 512
-rw-rw-r-- 1 eager eager 0 Jul 18 17:56 x

When I remove the file 'x' on the host, it's gone in the container, but
trying to create 'x' again fails with a different error:
[eager@devel tmp]$ ls -l
total 0
[eager@devel tmp]$ touch x
touch: cannot touch 'x': Stale file handle
[eager@devel tmp]$ ls -l
total 0

But I can create a different file 'y', albeit with the permission error:
[eager@devel tmp]$ touch y
touch: cannot touch 'y': Permission denied
[eager@devel tmp]$ ls -l
total 1.0K
-rw-rw-r-- 1 eager eager 0 Jul 18 18:07 y

Something seems to be broken. Is there a way to fix this?

--
Michael Eager

[Narcis Garcia]

This issue seems more apropriate here:
https://github.com/containers/fuse-overlayfs

Narcis Garcia

__________
I'm using this dedicated address because personal addresses aren't
masked enough at this mail public archive. Public archive administrator
should fix this against automated addresses collectors.
El 18/7/20 a les 20:16, Michael Eager ha escrit:

[Michael Eager]

On 7/18/20 12:14 PM, Narcis Garcia wrote:
> This issue seems more apropriate here:
> https://github.com/containers/fuse-overlayfs

Why?

The shiftfs module is a kernel module, not a userspace FUSE implementation.

> _______________________________________________
> lxc-users mailing list
> lxc-...@lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users
>

[Serge E. Hallyn]

On Sat, Jul 18, 2020 at 11:16:16AM -0700, Michael Eager wrote:
> I'm running LXD on a Centos 8 host, kernel-4.18.0. I built and installed
> the shiftfs module.

>From what source?

[Narcis Garcia]

https://bugzilla.kernel.org/

Narcis Garcia

__________
I'm using this dedicated address because personal addresses aren't
masked enough at this mail public archive. Public archive administrator
should fix this against automated addresses collectors.

El 19/7/20 a les 1:35, Michael Eager ha escrit:

[Michael Eager]

https://github.com/toby63/shiftfs-dkms

[Serge E. Hallyn]

Interesting. Yeah the shiftfs.c looks identical to what's in
ubuntu's, but when I backported that to 4.14 I had to also take
a bunch of other patches (https://git.kernel.org/pub/scm/linux/kernel/git/sergeh/linux.git/log/?h=v4.14.180-shiftfs)
which may or may not be in 4.18. Still, I don't actually think
that's the problem. NFS has always been difficult to use
with layered filesystems.

Amir, as far as you know, does overlay on top of NFS work
these days?

-serge

[Amir Goldstein]

I know people are using it, but only as lower fs, which are files that
are not expected to be changed.
NFS is not supported as upper fs.

Thanks,
Amir.

[Serge E. Hallyn]

Thanks Amir. I over-snipped, but Michael is having problems
when NFS is the underlay for shiftfs. If overlay works fine,
then that's an interesting complication :)

I'm hoping to have some time to try it out soon.

thanks,
-serge

[Michael Eager]

If you would like me to submit a bug report, just tell me where.

[Serge E. Hallyn]

Since it's shipped with the Ubuntu kernel, you could test it in an
Ubuntu VM and then file it at
https://bugs.launchpad.net/ubuntu/+source/linux/+filebug I don't
have any better place to suggest tracking it.

[Michael Eager]

I'm running LXD on CentOS 8, kernel-4.18.0. I built the shiftfs module
from https://github.com/toby63/shiftfs-dkms.

I can submit an issue on GitHub or bugzilla.kernel.org. Setting up a
Ubunut VM with LXD seems like a bit of work.

[Serge E. Hallyn]

So just to make sure, you've not reproduced this by hand doing a shiftfs
mount on top of NFS?

[Michael Eager]

No, I didn't try that. I only used the option with LXD.