with singularity version 3.0.0 : Cant' open /proc/self/auxv: Permission denied

[toufique]

Hi, 

Trying to run an .img file created with version 3.0.0, getting the following: 

016 $ singularity -d run /dfs1/som/tvanerp_col/bin/singularity_containers/mriqc/mriqc-hpc.img

VERBOSE [U=2906,P=5180]    singularity_message_level()               Set messagelevel to: 5

DEBUG   [U=2906,P=5180]    init()                                    PIPE_EXEC_FD value: 7

VERBOSE [U=2906,P=5180]    init()                                    Container runtime

VERBOSE [U=2906,P=5180]    is_suid()                                 Check if we are running as setuid

ERROR   [U=2906,P=5180]    is_suid()                                 Cant' open /proc/self/auxv: Permission denied

I don't recall this from version 2.6.0. Anything I need to do get this run with version 3.0.0? This is a user running it, just with his user account, not as root

thanks.

[Justin Cook]

What happens when you try to run it yourself? Do you get the same error?

What are the permissions of /proc/self/auxv?

Is the filesytem that holds that file mounted with the no setuid option set?

Thanks,

Justin

[Jason Stover]

Can we also see (assuming /usr/local prefix):
ls -l /usr/local/libexecdir/singularity/bin/
ls -l /proc/self/auxv

Something doesn't look quite right in the output ... It's running
`is_suid` .. but it _should_ be running as root (U=0) if starter-suid
is being called, and the permissions on it are correct. But again,
that doesn't explain why you're getting a permission denied error on
opening /proc/self/auxv ... the user should have access to their
own...

Do you get output with:
LD_SHOW_AUX=1 /bin/dd
(press ^C)

-J

> --
> You received this message because you are subscribed to the Google Groups "singularity" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

[Jason Stover]

*Edit ... That would be:

ls -l /usr/local/libexec/singularity/bin

... yeah ...

-J

[toufique]

We have singularity installed in a file server, so everyone can use it.

1011 $ ls -l /dfs3/apps/singularity/3.0.0/libexec/singularity/bin/

total 9130

-rwxr-xr-x 1 sw staff 21195848 Oct 29 15:41 starter

-rwsr-xr-x 1 sw staff 21195848 Oct 29 15:41 starter-suid

1017 $ ls -l /proc/self/auxv

-r-------- 1 itoufiqu staff 0 Nov 14 13:46 /proc/self/auxv

1018 $ LD_SHOW_AUX=1 /bin/dd 

^C0+0 records in

0+0 records out

0 bytes (0 B) copied, 2.208 s, 0.0 kB/s

[toufique]

Hi Justin, 

If I run it as myself, same error. 

example , when the container is located in parallel file system : 

1019 $ singularity -d run /dfs1/som/tvanerp_col/bin/singularity_containers/mriqc/mriqc-hpc.img xterm

VERBOSE [U=2906,P=9650]    singularity_message_level()               Set messagelevel to: 5

DEBUG   [U=2906,P=9650]    init()                                    PIPE_EXEC_FD value: 7

VERBOSE [U=2906,P=9650]    init()                                    Container runtime

VERBOSE [U=2906,P=9650]    is_suid()                                 Check if we are running as setuid

ERROR   [U=2906,P=9650]    is_suid()                                 Cant' open /proc/self/auxv: Permission denied

example, then I copy the container  in /tmp:

1020 $ singularity -d exec /tmp/mriqc-hpc.img xterm

VERBOSE [U=2906,P=29072]   singularity_message_level()               Set messagelevel to: 5

DEBUG   [U=2906,P=29072]   init()                                    PIPE_EXEC_FD value: 7

VERBOSE [U=2906,P=29072]   init()                                    Container runtime

VERBOSE [U=2906,P=29072]   is_suid()                                 Check if we are running as setuid

ERROR   [U=2906,P=29072]   is_suid()                                 Cant' open /proc/self/auxv: Permission denied

about the mount point, in the disk where the container is located and where singularity is installed .   

beegfs_dfs1 on /dfs1 type beegfs (rw,_netdev,cfgFile=/etc/beegfs/dfs1.d/beegfs-client.conf,)

beegfs_dfs2 on /dfs2 type beegfs (rw,_netdev,cfgFile=/etc/beegfs/dfs2.d/beegfs-client.conf,)

beegfs_dfs3 on /dfs3 type beegfs (rw,_netdev,cfgFile=/etc/beegfs/dfs3.d/beegfs-client.conf,)

From cat /proc/mounts : 

1023 $ cat /proc/mounts | grep dfs3

beegfs_dfs3 /dfs3 beegfs rw,relatime,cfgFile=/etc/beegfs/dfs3.d/beegfs-client.conf 0 0

ss

As you can see , there is no setuid option set in the file system mount  - either from the mount command or from /proc/mounts command. 

Thanks.

[Jason Stover]

Hi,

-rwsr-xr-x 1 sw staff 21195848 Oct 29 15:41 starter-suid

This is going to run as the `sw` user ... This needs to be owned by
root. Can you do:

chown root:staff
/dfs3/apps/singularity/3.0.0/libexec/singularity/bin/starter-suid
chmod 4755
/dfs3/apps/singularity/3.0.0/libexec/singularity/bin/starter-suid

> $ LD_SHOW_AUX=1 /bin/dd
Doh! I messed up on the var there...

LD_SHOW_AUXV=1 /bin/dd

-J

[toufique]

Thanks Jason, That seem to have fix it.  I can run the exec command and list stuff in the container. 

here is LD_SHOW_AUXV output: 

1012 $ LD_SHOW_AUXV=1  /bin/dd 

AT_SYSINFO_EHDR: 0x7ffd758e1000

AT_HWCAP:        178bfbff

AT_PAGESZ:       4096

AT_CLKTCK:       100

AT_PHDR:         0x400040

AT_PHENT:        56

AT_PHNUM:        8

AT_BASE:         0x7f6ae5386000

AT_FLAGS:        0x0

AT_ENTRY:        0x401ae0

AT_UID:          2527

AT_EUID:         2527

AT_GID:          200

AT_EGID:         200

AT_SECURE:       0

AT_RANDOM:       0x7ffd758aac49

AT_EXECFN:       /bin/dd

AT_PLATFORM:     x86_64

^C0+0 records in

0+0 records out

0 bytes (0 B) copied, 1.07277 s, 0.0 kB/s

thanks again!

[Josep Manel Andrés i Moscardó]

However they do not recommend this way of deployment but installing it through rpm/deb packages and provide them as a OS package

[Jason Stover]

Hi,

The installation is incorrect though. The issue here boils down to
the starter-suid process. It was setuid to the sw user. So, when it
was ran, the EUID was that of the sw user. The `sw` user doesn't have
access to the /proc/self/auxv file for a process that your user
started.

starter-suid needs to be owned by root and setuid. When you install,
you should running: sudo make install (to install as root)

See: https://www.sylabs.io/guides/3.0/admin-guide/admin_quickstart.html#source-installation

You can get away with everything else being owned by the sw user,
but the starter-suid binary must be owned by root, otherwise you're
going to get the error you did. Or other errors where you need root
privilege to do something.

-J

[Jason Stover]

Note: I just noticed that the build instructions are completely wrong
in the admin-guide... :/ The *Note* there is relevant though.

So, we'll go with the instructions here until we get that fixed...

https://www.sylabs.io/guides/3.0/user-guide/quick_start.html#compile-the-singularity-binary

Seems we completely overlook the source build in the admin-guide ... ...

-J