issues with --fakeroot

272 views
Skip to first unread message

Haopeng Han

unread,
Dec 6, 2021, 12:20:45 AM12/6/21
to singularity
Hi,
I am using singularity on my company's server. So far I am pretty happy about its capabilities and performance.
I got two issues with the --fakeroot option:

1:couldn't delete all the sandbox files:
singularity build --fakeroot --sandbox demo docker://ubuntu:20.04
singularity shell --fakeroot --writable demo
        singularity->./to_install.sh
        singularity->./startxfce.sh
        singularity-> ctrl+c, ctrl+d
rm -rf demo/ > rmdemo.log
ll demo/var/cache
drwxr-xr-x. 25 100005 100011 4096 Dec  6 12:33 man

2:--fakeroot doesn't work with Nviida docker images
singularity build --fakeroot --sandbox nvdemo docker://nvcr.io/nvidia/cuda:11.3.1-devel-ubuntu16.04
singularity shell --fakeroot --writable nvdemo 
        singularity-> apt update
Ign:1 http://security.ubuntu.com/ubuntu xenial-security InRelease              
Ign:2 http://archive.ubuntu.com/ubuntu xenial InRelease                        
Ign:3 http://archive.ubuntu.com/ubuntu xenial-updates InRelease                
  Could not open file /var/lib/apt/lists/partial/developer.download.nvidia.com_compute_cuda_repos_ubuntu1604_x86%5f64_InRelease - open (13: Permission denied)
Ign:5 http://archive.ubuntu.com/ubuntu xenial-backports InRelease              
Err:6 http://archive.ubuntu.com/ubuntu xenial Release                       
  Could not open file /var/lib/apt/lists/partial/archive.ubuntu.com_ubuntu_dists_xenial_Release - open (13: Permission denied) [IP: 91.189.88.142 80]
Err:7 http://security.ubuntu.com/ubuntu xenial-security Release             
  Could not open file /var/lib/apt/lists/partial/security.ubuntu.com_ubuntu_dists_xenial-security_Release - open (13: Permission denied) [IP: 91.189.88.142 80]
..................
..................
however it can be done using:
sudo singularity shell --writable nvdemo 

The server is a Dell 4U PowerEdge R940XA with 4xIntel 6230, running CentOS Stream 8.
Attached the files. 
Any help would be appreciated!
Best regards,
Haopeng

to_install.sh
startxfce.sh
rmdemo.log

Haopeng Han

unread,
Dec 6, 2021, 12:34:52 AM12/6/21
to singularity, Haopeng Han
Forgot to mention: 
singularity version 3.8.3


Dave Dykstra

unread,
Dec 6, 2021, 4:51:42 PM12/6/21
to singu...@lbl.gov
On Sun, Dec 05, 2021 at 09:20:45PM -0800, Haopeng Han wrote:
> Hi,
> I am using singularity on my company's server. So far I am pretty happy
> about its capabilities and performance.
> I got two issues with the --fakeroot option:
>
> 1:couldn't delete all the sandbox files:
> singularity build --fakeroot --sandbox demo docker://ubuntu:20.04
> singularity shell --fakeroot --writable demo
> singularity->./to_install.sh
> singularity->./startxfce.sh
> singularity-> ctrl+c, ctrl+d
> rm -rf demo/ > rmdemo.log
> ll demo/var/cache
> drwxr-xr-x. 25 100005 100011 4096 Dec 6 12:33 man

I think that's just the way it has to be, a consequence of the way
"rootless containers" deal with multiple user ids by enabling each user
to have access to multiple other user ids while inside the container.
Those files will just have to be removed from inside a container.

> 2:--fakeroot doesn't work with Nvidia docker images
> singularity build --fakeroot --sandbox nvdemo
> docker://nvcr.io/nvidia/cuda:11.3.1-devel-ubuntu16.04
> singularity shell --fakeroot --writable nvdemo
> singularity-> apt update
> Ign:1 http://security.ubuntu.com/ubuntu xenial-security InRelease
> Ign:2 http://archive.ubuntu.com/ubuntu xenial InRelease
> Ign:3 http://archive.ubuntu.com/ubuntu xenial-updates InRelease
> Err:4 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64
> InRelease
> Could not open file
> /var/lib/apt/lists/partial/developer.download.nvidia.com_compute_cuda_repos_ubuntu1604_x86%5f64_InRelease
> - open (13: Permission denied)
> Ign:5 http://archive.ubuntu.com/ubuntu xenial-backports InRelease
> Err:6 http://archive.ubuntu.com/ubuntu xenial Release
> Could not open file
> /var/lib/apt/lists/partial/archive.ubuntu.com_ubuntu_dists_xenial_Release -
> open (13: Permission denied) [IP: 91.189.88.142 80]
> Err:7 http://security.ubuntu.com/ubuntu xenial-security Release
> Could not open file
> /var/lib/apt/lists/partial/security.ubuntu.com_ubuntu_dists_xenial-security_Release
> - open (13: Permission denied) [IP: 91.189.88.142 80]
> ..................
> ..................

I cannot reproduce your symptoms, those commands worked for me.
Maybe you have to say more about your environment.

Dave

> however it can be done using:
> sudo singularity shell --writable nvdemo
>
> The server is a Dell 4U PowerEdge R940XA with 4xIntel 6230, running CentOS
> Stream 8.
> Attached the files.
> Any help would be appreciated!
> Best regards,
> Haopeng
>
> --
> You received this message because you are subscribed to the Google Groups "singularity" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.
> To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/singularity/9e188b67-3e02-4bf2-bea3-25cc359df8e6n%40lbl.gov .



> rm: cannot remove 'demo/var/cache/man/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/ru/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/ru/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/ru/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/ru/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/ru/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/pt/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/pt/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/pt/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/pt/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/pt/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/es/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/es/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/es/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/es/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/es/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/hu/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/hu/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/hu/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/ja/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/ja/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/ja/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/ja/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/ja/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/tr/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/tr/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/tr/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/cs/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/cs/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/cs/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/cs/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/cs/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/nl/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/nl/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/nl/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/nl/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/nl/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/ko/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/ko/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/ko/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/de/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/de/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/de/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/de/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/de/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/sv/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/sv/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/sv/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/sv/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/sv/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/da/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/da/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/da/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/da/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/da/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/zh_TW/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/zh_TW/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/zh_TW/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/zh_CN/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/zh_CN/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/zh_CN/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/zh_CN/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/zh_CN/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/pl/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/pl/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/pl/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/pl/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/pl/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/id/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/id/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/id/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/fr/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/fr/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/fr/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/fr/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/fr/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/cat3': Permission denied
> rm: cannot remove 'demo/var/cache/man/cat7': Permission denied
> rm: cannot remove 'demo/var/cache/man/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/it/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/it/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/it/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/it/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/it/cat5': Permission denied
> rm: cannot remove 'demo/var/lib/gdm3/.local/share/applications': Permission denied
> rm: cannot remove 'demo/var/lib/gdm3/.config/pulse/default.pa': Permission denied
> rm: cannot remove 'demo/var/lib/gdm3/.config/dconf': Permission denied
> rm: cannot remove 'demo/var/lib/colord/icc': Permission denied
> rm: cannot remove 'demo/run/systemd/netif/lldp': Permission denied
> rm: cannot remove 'demo/run/systemd/netif/leases': Permission denied
> rm: cannot remove 'demo/run/systemd/netif/links': Permission denied
> [hhan@dv-srv01 dv-app-11]$ rm -rf demo/ > rmdemo.log
> rm: cannot remove 'demo/var/cache/man/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/ru/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/ru/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/ru/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/ru/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/ru/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/pt/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/pt/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/pt/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/pt/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/pt/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/es/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/es/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/es/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/es/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/es/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/hu/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/hu/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/hu/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/ja/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/ja/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/ja/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/ja/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/ja/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/tr/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/tr/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/tr/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/cs/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/cs/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/cs/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/cs/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/cs/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/nl/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/nl/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/nl/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/nl/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/nl/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/ko/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/ko/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/ko/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/de/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/de/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/de/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/de/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/de/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/sv/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/sv/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/sv/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/sv/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/sv/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/da/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/da/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/da/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/da/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/da/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/zh_TW/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/zh_TW/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/zh_TW/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/zh_CN/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/zh_CN/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/zh_CN/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/zh_CN/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/zh_CN/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/pl/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/pl/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/pl/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/pl/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/pl/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/id/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/id/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/id/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/fr/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/fr/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/fr/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/fr/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/fr/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/cat3': Permission denied
> rm: cannot remove 'demo/var/cache/man/cat7': Permission denied
> rm: cannot remove 'demo/var/cache/man/cat5': Permission denied
> rm: cannot remove 'demo/var/cache/man/it/index.db': Permission denied
> rm: cannot remove 'demo/var/cache/man/it/cat1': Permission denied
> rm: cannot remove 'demo/var/cache/man/it/CACHEDIR.TAG': Permission denied
> rm: cannot remove 'demo/var/cache/man/it/cat8': Permission denied
> rm: cannot remove 'demo/var/cache/man/it/cat5': Permission denied
> rm: cannot remove 'demo/var/lib/gdm3/.local/share/applications': Permission denied
> rm: cannot remove 'demo/var/lib/gdm3/.config/pulse/default.pa': Permission denied
> rm: cannot remove 'demo/var/lib/gdm3/.config/dconf': Permission denied
> rm: cannot remove 'demo/var/lib/colord/icc': Permission denied
> rm: cannot remove 'demo/run/systemd/netif/lldp': Permission denied
> rm: cannot remove 'demo/run/systemd/netif/leases': Permission denied
> rm: cannot remove 'demo/run/systemd/netif/links': Permission denied
>

Haopeng Han

unread,
Dec 7, 2021, 11:36:02 PM12/7/21
to singularity, Dave Dykstra
Thanks Dave for your reply!
For the first one, my concern is: if a user (without sudo privilege on the host, with --fakeroot on) issue "rm -rf /*" inside the container, will all the mounted/bound directories on the host that the user has rw access (eg home) get deleted?
For the second one, I guess it was the server's problem. I tried the same command on another server, it was ok.

Thanks & Best,
Haopeng

Thomas Hartmann

unread,
Dec 9, 2021, 4:05:40 AM12/9/21
to singu...@lbl.gov
Hi Haopeng,

iirc - bin mounts into a container's context behave just like the same
directories on other paths.

e.g., if you bind mount a directory onto another path

> mount --bind /original/path /new/path

and your user has r/w on `/original/path`, the same capabilities apply
to your user on the new view `/new/path` as well

Cheers,
Thomas
> <http://nvcr.io/nvidia/cuda:11.3.1-devel-ubuntu16.04>
> <http://default.pa>': Permission denied
> <http://default.pa>': Permission denied
> > rm: cannot remove 'demo/var/lib/gdm3/.config/dconf': Permission
> denied
> > rm: cannot remove 'demo/var/lib/colord/icc': Permission denied
> > rm: cannot remove 'demo/run/systemd/netif/lldp': Permission denied
> > rm: cannot remove 'demo/run/systemd/netif/leases': Permission denied
> > rm: cannot remove 'demo/run/systemd/netif/links': Permission denied
> >
>
> --
> You received this message because you are subscribed to the Google
> Groups "singularity" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to singularity...@lbl.gov
> <mailto:singularity...@lbl.gov>.
> To view this discussion on the web visit
> https://groups.google.com/a/lbl.gov/d/msgid/singularity/df3e1fee-4cff-4536-910d-762ce9470bc4n%40lbl.gov
> <https://groups.google.com/a/lbl.gov/d/msgid/singularity/df3e1fee-4cff-4536-910d-762ce9470bc4n%40lbl.gov?utm_medium=email&utm_source=footer>.

Haopeng Han

unread,
Dec 9, 2021, 6:18:12 AM12/9/21
to singularity, Thomas Hartmann
Hi Thomas,
I just did an experiment on running "rm -rf /*" insider container with --fakeroot on a virtual machine. Yes, it wiped out everything under the home directory. And I think it will be the same result without --fakeroot.
Thanks & Best,
Haopeng

Reply all
Reply to author
Forward
0 new messages