run command as USER inside %post

1,152 views
Skip to first unread message

Till

unread,
Jun 26, 2017, 4:10:45 PM6/26/17
to singularity
Hi,

one basic question:
How do I run an USER command in the image definition?
Do I have to add an user and change to that user in oder to do that or
is there a dedicated statement  that should be used (like USER in dockerfiles) ?

Thanks,

Till

David Godlove

unread,
Jun 26, 2017, 4:16:13 PM6/26/17
to singu...@lbl.gov
I don’t think there is way to do this built into Singularity.  You would have to add the user, change to that user and then execute the command.  I think you would also want to remove the user when you were finished unless you wanted a user account to persist in your /etc/passwd file within your singularity container.  

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

vanessa s

unread,
Jun 26, 2017, 4:23:19 PM6/26/17
to singu...@lbl.gov
Could you instead run as root, and then change permissions as needed?

What I think you (might) be after is more along the lines of having software that needs to be associated with a particular user. Here is a snippet (I think would do the trick) that is from this container:

# Install homebrew science, can't use root
useradd -m singularity
cd /Software
su -c 'git clone https://github.com/Linuxbrew/brew.git' singularity
su -c '/Software/brew/bin/brew install bsdmainutils parallel util-linux' singularity
su -c '/Software/brew/bin/brew tap homebrew/science' singularity
su -c '/Software/brew/bin/brew install art bwa samtools' singularity
su -c 'rm -r $(/Software/brew/bin/brew --cache)' singularity
su -c 'wget http://repo.continuum.io/archive/Anaconda3-4.1.1-Linux-x86_64.sh' singularity

If it's just for a general user (eg, not "singularity") then you might want to do this by way of changing permissions. For the above, the basic steps were to create the user (as root) and then run commands on its behalf. 

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.



--
Vanessa Villamia Sochat
Stanford University '16

vanessa s

unread,
Jun 26, 2017, 4:23:57 PM6/26/17
to singu...@lbl.gov
+1! I think there is a lot of "it depends on what you are trying to do" for this answer.

To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

Till

unread,
Jun 26, 2017, 4:54:23 PM6/26/17
to singularity
I'd like to install a javascript app thats build with the Meteor framework.
When the app starts it updates all components in user-space and aborts 
if you're root. Since we don't have internet on the cluster I have to do this 
update thing in the bootstrap process (%post), which is run as root. That means I
have to switch to a user somehow. 
That we're offline just complicates everything for me. Hope this was understandable.
/Till 

On Monday, June 26, 2017 at 1:23:57 PM UTC-7, vanessa wrote:
+1! I think there is a lot of "it depends on what you are trying to do" for this answer.
On Mon, Jun 26, 2017 at 4:16 PM, David Godlove <davidg...@gmail.com> wrote:
I don’t think there is way to do this built into Singularity.  You would have to add the user, change to that user and then execute the command.  I think you would also want to remove the user when you were finished unless you wanted a user account to persist in your /etc/passwd file within your singularity container.  
On Jun 26, 2017, at 4:10 PM, Till <tsche...@gmail.com> wrote:

Hi,

one basic question:
How do I run an USER command in the image definition?
Do I have to add an user and change to that user in oder to do that or
is there a dedicated statement  that should be used (like USER in dockerfiles) ?

Thanks,

Till

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

Martin Cuma

unread,
Jun 27, 2017, 6:21:33 PM6/27/17
to singularity
Till,

you can try to do the userspace installation in the %runscript section on the container's first invocation, as done here:

https://github.com/CHPC-UofU/Singularity-ubuntu-wine-peakselector

Though in this case the user stack ($APPDIR/wineprefix.tgz) is stored in the user space (or here, in "some" fixed space), not the container itself.

MC

Till

unread,
Jun 28, 2017, 2:05:34 PM6/28/17
to singularity
Thanks for all your suggestions!

It seems to work at least on my build system.
I've set up a definition like this. But on the cluster I get an error ... 
Do you know what might be wrong??

Thanks,

Till



Bootstrap: docker
From: ubuntu

%post
        # Creating bind point for home directory!

        mkdir /tsd
       mkdir -p /tsd/p158/home/p158-uxscti



        # Install software needed during bootstrap!

        apt-get update && apt-get install -y git \
                                       curl \
                                       sudo \
                                       locales \
                                       python \
                                       build-essential

 # Adding meteor user and install meteor

        adduser meteor-user
       curl https://install.meteor.com/ | sh
        cd /usr/local
       git clone https://github.com/akeshavan/mindcontrol
        cd /usr/local/mindcontrol      



        # Adding bind point  for writable meteor directory!

        mkdir -p /usr/local/mindcontrol/.meteor/local
       chown -Rh meteor-user /usr/local/mindcontrol



        # Meteor needs locale en_US.UTF-8

        locale-gen en_US.UTF-8
       export LC_ALL=en_US.UTF-8
   # Run meteor as user to install updated packages

        su -c "meteor update" meteor-user
       su -c "meteor npm install --save babel-runtime" meteor-user
       su -c "meteor npm install --save bcrypt" meteor-user


%runscript
       
       echo "Starting mindcontrol!"
       cd /usr/local/mindcontrol
       METEOR_OFFLINE_CATALOG=1 meteor


When I run the container on my build linux instance it starts without any problems!

singularity run -B .meteor/local:/usr/local/mindcontrol/.meteor/local mindcontrol.img
Starting mindcontrol!
[[[[[ /usr/local/mindcontrol ]]]]]            

=> Started proxy.                            
=> Started MongoDB.                          
=> Started your app.                          

=> App running at: http://localhost:3000/


On the cluster I'm getting a permission denied error:


Increasing verbosity level (6)
Ending argument loop
Singularity version: 2.3.1-dist
Exec'ing: /usr/local/libexec/singularity/cli/run.exec
Evaluating args: '-B .meteor/local:/usr/local/mindcontrol/.meteor/local mindcontrol.img'
VERBOSE [U=0,P=60628]      message_init()                            Set messagelevel to: 6
VERBOSE [U=0,P=60628]      singularity_config_parse()                Initialize configuration file: /usr/local/etc/singularity/singularity.conf
DEBUG   [U=0,P=60628]      singularity_config_parse()                Starting parse of configuration file /usr/local/etc/singularity/singularity.conf
VERBOSE [U=0,P=60628]      singularity_config_parse()                Got config key allow setuid = 'yes'
VERBOSE [U=0,P=60628]      singularity_config_parse()                Got config key max loop devices = '256'
VERBOSE [U=0,P=60628]      singularity_config_parse()                Got config key allow pid ns = 'yes'
VERBOSE [U=0,P=60628]      singularity_config_parse()                Got config key config passwd = 'yes'
VERBOSE [U=0,P=60628]      singularity_config_parse()                Got config key config group = 'yes'
VERBOSE [U=0,P=60628]      singularity_config_parse()                Got config key config resolv_conf = 'yes'
VERBOSE [U=0,P=60628]      singularity_config_parse()                Got config key mount proc = 'yes'
VERBOSE [U=0,P=60628]      singularity_config_parse()                Got config key mount sys = 'yes'
VERBOSE [U=0,P=60628]      singularity_config_parse()                Got config key mount dev = 'yes'
VERBOSE [U=0,P=60628]      singularity_config_parse()                Got config key mount home = 'yes'
VERBOSE [U=0,P=60628]      singularity_config_parse()                Got config key mount tmp = 'yes'
VERBOSE [U=0,P=60628]      singularity_config_parse()                Got config key mount hostfs = 'no'
VERBOSE [U=0,P=60628]      singularity_config_parse()                Got config key bind path = '/etc/localtime'
VERBOSE [U=0,P=60628]      singularity_config_parse()                Got config key bind path = '/etc/hosts'
VERBOSE [U=0,P=60628]      singularity_config_parse()                Got config key user bind control = 'yes'
VERBOSE [U=0,P=60628]      singularity_config_parse()                Got config key enable overlay = 'yes'
VERBOSE [U=0,P=60628]      singularity_config_parse()                Got config key mount slave = 'yes'
VERBOSE [U=0,P=60628]      singularity_config_parse()                Got config key sessiondir max size = '16'
DEBUG   [U=0,P=60628]      singularity_config_parse()                Finished parsing configuration file '/usr/local/etc/singularity/singularity.conf'
VERBOSE [U=0,P=60628]      singularity_registry_init()               Initializing Singularity Registry
VERBOSE [U=0,P=60628]      singularity_registry_set()                Adding value to registry: 'LIBEXECDIR' = '/usr/local/libexec'
DEBUG   [U=0,P=60628]      singularity_registry_set()                Returning singularity_registry_set(libexecdir, /usr/local/libexec) = 0
VERBOSE [U=0,P=60628]      singularity_registry_set()                Adding value to registry: 'COMMAND' = 'run'
DEBUG   [U=0,P=60628]      singularity_registry_set()                Returning singularity_registry_set(COMMAND, run) = 0
VERBOSE [U=0,P=60628]      singularity_registry_set()                Adding value to registry: 'MESSAGELEVEL' = '6'
DEBUG   [U=0,P=60628]      singularity_registry_set()                Returning singularity_registry_set(MESSAGELEVEL, 6) = 0
VERBOSE [U=0,P=60628]      singularity_registry_set()                Adding value to registry: 'VERSION' = '2.3.1-dist'
DEBUG   [U=0,P=60628]      singularity_registry_set()                Returning singularity_registry_set(version, 2.3.1-dist) = 0
VERBOSE [U=0,P=60628]      singularity_registry_set()                Adding value to registry: 'LOCALSTATEDIR' = '/usr/local/var'
DEBUG   [U=0,P=60628]      singularity_registry_set()                Returning singularity_registry_set(localstatedir, /usr/local/var) = 0
VERBOSE [U=0,P=60628]      singularity_registry_set()                Adding value to registry: 'SYSCONFDIR' = '/usr/local/etc'
DEBUG   [U=0,P=60628]      singularity_registry_set()                Returning singularity_registry_set(sysconfdir, /usr/local/etc) = 0
VERBOSE [U=0,P=60628]      singularity_registry_set()                Adding value to registry: 'BINDPATH' = ',.meteor/local:/usr/local/mindcontrol/.meteor/local'
DEBUG   [U=0,P=60628]      singularity_registry_set()                Returning singularity_registry_set(BINDPATH, ,.meteor/local:/usr/local/mindcontrol/.meteor/local) = 0
VERBOSE [U=0,P=60628]      singularity_registry_set()                Adding value to registry: 'BINDIR' = '/usr/local/bin'
DEBUG   [U=0,P=60628]      singularity_registry_set()                Returning singularity_registry_set(bindir, /usr/local/bin) = 0
VERBOSE [U=0,P=60628]      singularity_registry_set()                Adding value to registry: 'IMAGE' = 'mindcontrol.img'
DEBUG   [U=0,P=60628]      singularity_registry_set()                Returning singularity_registry_set(IMAGE, mindcontrol.img) = 0
DEBUG   [U=0,P=60628]      singularity_registry_get()                Returning NULL on 'HOME'
DEBUG   [U=0,P=60628]      singularity_registry_get()                Returning NULL on 'TARGET_UID'
DEBUG   [U=0,P=60628]      singularity_registry_get()                Returning NULL on 'TARGET_GID'
DEBUG   [U=0,P=60628]      singularity_priv_init()                   Initializing user info
DEBUG   [U=0,P=60628]      singularity_priv_init()                   Set the calling user's username to: p158-uxscti
DEBUG   [U=0,P=60628]      singularity_priv_init()                   Marking uinfo structure as ready
DEBUG   [U=0,P=60628]      singularity_priv_init()                   Obtaining home directory
VERBOSE [U=0,P=60628]      singularity_priv_init()                   Set home (via getpwuid()) to: /tsd/p158/home/p158-uxscti
VERBOSE [U=0,P=60628]      singularity_suid_init()                   Running SUID program workflow
VERBOSE [U=0,P=60628]      singularity_suid_init()                   Checking program has appropriate permissions
VERBOSE [U=0,P=60628]      singularity_suid_init()                   Checking configuration file is properly owned by root
VERBOSE [U=0,P=60628]      singularity_suid_init()                   Checking if singularity.conf allows us to run as suid
DEBUG   [U=0,P=60628]      singularity_config_get_bool_char_impl()   Called singularity_config_get_bool(allow setuid, yes)
DEBUG   [U=0,P=60628]      singularity_config_get_value_impl()       Returning configuration value allow setuid='yes'
DEBUG   [U=0,P=60628]      singularity_config_get_bool_char_impl()   Return singularity_config_get_bool(allow setuid, yes) = 1
DEBUG   [U=0,P=60628]      singularity_registry_get()                Returning NULL on 'NOSUID'
VERBOSE [U=0,P=60628]      singularity_priv_userns()                 Invoking the user namespace
DEBUG   [U=0,P=60628]      singularity_config_get_bool_char_impl()   Called singularity_config_get_bool(allow user ns, yes)
DEBUG   [U=0,P=60628]      singularity_config_get_value_impl()       No configuration entry found for 'allow user ns'; returning default value 'yes'
DEBUG   [U=0,P=60628]      singularity_config_get_bool_char_impl()   Return singularity_config_get_bool(allow user ns, yes) = 1
VERBOSE [U=0,P=60628]      singularity_priv_userns()                 Not virtualizing USER namespace: running as SUID
DEBUG   [U=0,P=60628]      singularity_priv_userns()                 Returning singularity_priv_init(void)
DEBUG   [U=0,P=60628]      singularity_priv_drop()                   Dropping privileges to UID=1947, GID=2712 (5 supplementary GIDs)
DEBUG   [U=0,P=60628]      singularity_priv_drop()                   Restoring supplementary groups
DEBUG   [U=1947,P=60628]   singularity_priv_drop()                   Confirming we have correct UID/GID
DEBUG   [U=1947,P=60628]   singularity_registry_get()                Returning NULL on 'CLEANUPDIR'
DEBUG   [U=1947,P=60628]   singularity_registry_get()                Returning NULL on 'NOSESSIONCLEANUP'
DEBUG   [U=1947,P=60628]   singularity_registry_get()                Returning NULL on 'NOCLEANUP'
DEBUG   [U=1947,P=60628]   singularity_cleanupd()                    Not running a cleanup thread, no 'SINGULARITY_CLEANUPDIR' defined
DEBUG   [U=1947,P=60628]   singularity_runtime_ns()                  Calling: _singularity_runtime_ns_pid()
DEBUG   [U=1947,P=60628]   singularity_config_get_bool_char_impl()   Called singularity_config_get_bool(allow pid ns, yes)
DEBUG   [U=1947,P=60628]   singularity_config_get_value_impl()       Returning configuration value allow pid ns='yes'
DEBUG   [U=1947,P=60628]   singularity_config_get_bool_char_impl()   Return singularity_config_get_bool(allow pid ns, yes) = 1
DEBUG   [U=1947,P=60628]   singularity_registry_get()                Returning NULL on 'UNSHARE_PID'
VERBOSE [U=1947,P=60628]   singularity_runtime_ns_pid()              Not virtualizing PID namespace on user request
DEBUG   [U=1947,P=60628]   singularity_runtime_ns()                  Calling: _singularity_runtime_ns_ipc()
DEBUG   [U=1947,P=60628]   singularity_config_get_bool_char_impl()   Called singularity_config_get_bool(allow ipc ns, yes)
DEBUG   [U=1947,P=60628]   singularity_config_get_value_impl()       No configuration entry found for 'allow ipc ns'; returning default value 'yes'
DEBUG   [U=1947,P=60628]   singularity_config_get_bool_char_impl()   Return singularity_config_get_bool(allow ipc ns, yes) = 1
DEBUG   [U=1947,P=60628]   singularity_registry_get()                Returning NULL on 'UNSHARE_IPC'
VERBOSE [U=1947,P=60628]   singularity_runtime_ns_ipc()              Not virtualizing IPC namespace on user request
DEBUG   [U=1947,P=60628]   singularity_runtime_ns()                  Calling: _singularity_runtime_ns_mnt()
DEBUG   [U=1947,P=60628]   singularity_config_get_bool_char_impl()   Called singularity_config_get_bool(mount slave, yes)
DEBUG   [U=1947,P=60628]   singularity_config_get_value_impl()       Returning configuration value mount slave='yes'
DEBUG   [U=1947,P=60628]   singularity_config_get_bool_char_impl()   Return singularity_config_get_bool(mount slave, yes) = 1
DEBUG   [U=1947,P=60628]   singularity_priv_escalate()               Temporarily escalating privileges (U=1947)
DEBUG   [U=0,P=60628]      singularity_priv_escalate()               Clearing supplementary GIDs.
DEBUG   [U=0,P=60628]      singularity_runtime_ns_mnt()              Virtualizing FS namespace
DEBUG   [U=0,P=60628]      singularity_runtime_ns_mnt()              Virtualizing mount namespace
DEBUG   [U=0,P=60628]      singularity_runtime_ns_mnt()              Making mounts slave
DEBUG   [U=0,P=60628]      singularity_priv_drop()                   Dropping privileges to UID=1947, GID=2712 (5 supplementary GIDs)
DEBUG   [U=0,P=60628]      singularity_priv_drop()                   Restoring supplementary groups
DEBUG   [U=1947,P=60628]   singularity_priv_drop()                   Confirming we have correct UID/GID
DEBUG   [U=1947,P=60628]   singularity_sessiondir()                  Setting sessiondir
VERBOSE [U=1947,P=60628]   singularity_sessiondir()                  Using session directory: /usr/local/var/singularity/mnt/session
DEBUG   [U=1947,P=60628]   singularity_sessiondir()                  Checking for session directory: /usr/local/var/singularity/mnt/session
DEBUG   [U=1947,P=60628]   singularity_sessiondir()                  Obtaining the default sessiondir size
DEBUG   [U=1947,P=60628]   singularity_config_get_value_impl()       Returning configuration value sessiondir max size='16'
DEBUG   [U=1947,P=60628]   singularity_sessiondir()                  Converted sessiondir size to: 16
DEBUG   [U=1947,P=60628]   singularity_sessiondir()                  Creating the sessiondir size mount option length
DEBUG   [U=1947,P=60628]   singularity_sessiondir()                  Got size length of: 9
DEBUG   [U=1947,P=60628]   singularity_sessiondir()                  Creating the sessiondir size mount option string
DEBUG   [U=1947,P=60628]   singularity_sessiondir()                  Checking to make sure the string was allocated correctly
DEBUG   [U=1947,P=60628]   singularity_priv_escalate()               Temporarily escalating privileges (U=1947)
DEBUG   [U=0,P=60628]      singularity_priv_escalate()               Clearing supplementary GIDs.
DEBUG   [U=0,P=60628]      singularity_sessiondir()                  Mounting sessiondir tmpfs: /usr/local/var/singularity/mnt/session
DEBUG   [U=0,P=60628]      singularity_priv_drop()                   Dropping privileges to UID=1947, GID=2712 (5 supplementary GIDs)
DEBUG   [U=0,P=60628]      singularity_priv_drop()                   Restoring supplementary groups
DEBUG   [U=1947,P=60628]   singularity_priv_drop()                   Confirming we have correct UID/GID
VERBOSE [U=1947,P=60628]   singularity_registry_set()                Adding value to registry: 'SESSIONDIR' = '/usr/local/var/singularity/mnt/session'
DEBUG   [U=1947,P=60628]   singularity_registry_set()                Returning singularity_registry_set(SESSIONDIR, /usr/local/var/singularity/mnt/session) = 0
DEBUG   [U=1947,P=60628]   singularity_registry_get()                Returning value from registry: 'IMAGE' = 'mindcontrol.img'
DEBUG   [U=1947,P=60628]   singularity_registry_get()                Returning NULL on 'WRITABLE'
DEBUG   [U=1947,P=60628]   singularity_config_get_value_impl()       No configuration entry found for 'limit container owners'; returning default value 'NULL'
DEBUG   [U=1947,P=60628]   singularity_config_get_value_impl()       No configuration entry found for 'limit container paths'; returning default value 'NULL'
DEBUG   [U=1947,P=60628]   singularity_image_open()                  Opening file descriptor to image: mindcontrol.img
VERBOSE [U=1947,P=60628]   singularity_image_check()                 Checking what kind of image we are mounting
VERBOSE [U=1947,P=60628]   singularity_image_check_image()           Checking that file pointer is a Singularity image
DEBUG   [U=1947,P=60628]   singularity_image_check_image()           First line of image(fd=4): #!/usr/bin/env run-singularity

DEBUG   [U=1947,P=60628]   singularity_image_check_image()           Checking if first line matches key
VERBOSE [U=1947,P=60628]   singularity_image_check_image()           File is a valid Singularity image
DEBUG   [U=1947,P=60628]   singularity_config_get_value_impl()       Returning configuration value max loop devices='256'
DEBUG   [U=1947,P=60628]   singularity_image_bind()                  Entered singularity_image_bind()
DEBUG   [U=1947,P=60628]   singularity_image_bind()                  Converting max_loop_devs_string to int: '256'
DEBUG   [U=1947,P=60628]   singularity_image_bind()                  Converted max_loop_devs_string to int: '256' -> 256
DEBUG   [U=1947,P=60628]   singularity_image_bind()                  Checking if this image has been properly opened
DEBUG   [U=1947,P=60628]   singularity_image_bind()                  Checking if image is valid file
DEBUG   [U=1947,P=60628]   singularity_image_bind()                  Checking if image is already bound to a loop device
DEBUG   [U=1947,P=60628]   singularity_registry_get()                Returning NULL on 'WRITABLE'
DEBUG   [U=1947,P=60628]   singularity_image_bind()                  Setting loopdev open to: O_RDONLY
DEBUG   [U=1947,P=60628]   singularity_image_bind()                  Setting LO_FLAGS_AUTOCLEAR
DEBUG   [U=1947,P=60628]   singularity_image_bind()                  Calculating image offset
VERBOSE [U=1947,P=60628]   singularity_image_check()                 Checking what kind of image we are mounting
VERBOSE [U=1947,P=60628]   singularity_image_check_image()           Checking that file pointer is a Singularity image
DEBUG   [U=1947,P=60628]   singularity_image_check_image()           First line of image(fd=4): #!/usr/bin/env run-singularity

DEBUG   [U=1947,P=60628]   singularity_image_check_image()           Checking if first line matches key
VERBOSE [U=1947,P=60628]   singularity_image_check_image()           File is a valid Singularity image
VERBOSE [U=1947,P=60628]   singularity_image_offset()                Calculating image offset
VERBOSE [U=1947,P=60628]   singularity_image_offset()                Found image at an offset of 31 bytes
DEBUG   [U=1947,P=60628]   singularity_image_offset()                Returning image_offset(image_fp) = 31
DEBUG   [U=1947,P=60628]   singularity_priv_escalate()               Temporarily escalating privileges (U=1947)
DEBUG   [U=0,P=60628]      singularity_priv_escalate()               Clearing supplementary GIDs.
DEBUG   [U=0,P=60628]      singularity_image_bind()                  Finding next available loop device...
VERBOSE [U=0,P=60628]      singularity_image_bind()                  Found available loop device: /dev/loop0
DEBUG   [U=0,P=60628]      singularity_image_bind()                  Setting loop device flags
DEBUG   [U=0,P=60628]      singularity_priv_drop()                   Dropping privileges to UID=1947, GID=2712 (5 supplementary GIDs)
DEBUG   [U=0,P=60628]      singularity_priv_drop()                   Restoring supplementary groups
DEBUG   [U=1947,P=60628]   singularity_priv_drop()                   Confirming we have correct UID/GID
VERBOSE [U=1947,P=60628]   singularity_image_bind()                  Using loop device: /dev/loop0
VERBOSE [U=1947,P=60628]   singularity_runtime_rootfs()              Set container directory to: /usr/local/var/singularity/mnt/container
DEBUG   [U=1947,P=60628]   singularity_runtime_rootfs()              Checking for container directory
DEBUG   [U=1947,P=60628]   singularity_runtime_rootfs()              Returning container_directory: /usr/local/var/singularity/mnt/container
DEBUG   [U=1947,P=60628]   chk_mode()                                Checking exact mode (40755) on: /usr/local/var/singularity/mnt/container
DEBUG   [U=1947,P=60628]   chk_mode()                                Found appropriate mode on file: /usr/local/var/singularity/mnt/container
VERBOSE [U=1947,P=60628]   singularity_image_mount()                 Checking what kind of image we are mounting
VERBOSE [U=1947,P=60628]   singularity_image_mount()                 Attempting to mount as singularity image
DEBUG   [U=1947,P=60628]   singularity_registry_get()                Returning NULL on 'WRITABLE'
DEBUG   [U=1947,P=60628]   singularity_priv_escalate()               Temporarily escalating privileges (U=1947)
DEBUG   [U=0,P=60628]      singularity_priv_escalate()               Clearing supplementary GIDs.
VERBOSE [U=0,P=60628]      singularity_image_mount_image_mount()     Mounting /dev/loop0 in read/only to: /usr/local/var/singularity/mnt/container
DEBUG   [U=0,P=60628]      singularity_priv_drop()                   Dropping privileges to UID=1947, GID=2712 (5 supplementary GIDs)
DEBUG   [U=0,P=60628]      singularity_priv_drop()                   Restoring supplementary groups
DEBUG   [U=1947,P=60628]   singularity_priv_drop()                   Confirming we have correct UID/GID
DEBUG   [U=1947,P=60628]   singularity_runtime_rootfs()              Returning container_directory: /usr/local/var/singularity/mnt/container
DEBUG   [U=1947,P=60628]   action_ready()                            Checking if container is valid at: /usr/local/var/singularity/mnt/container
DEBUG   [U=1947,P=60628]   singularity_runtime_rootfs()              Returning container_directory: /usr/local/var/singularity/mnt/container
DEBUG   [U=1947,P=60628]   singularity_runtime_overlayfs()           Checking if overlayfs should be used
DEBUG   [U=1947,P=60628]   singularity_config_get_bool_char_impl()   Called singularity_config_get_bool(enable overlay, yes)
DEBUG   [U=1947,P=60628]   singularity_config_get_value_impl()       Returning configuration value enable overlay='yes'
DEBUG   [U=1947,P=60628]   singularity_config_get_bool_char_impl()   Return singularity_config_get_bool(enable overlay, yes) = 1
DEBUG   [U=1947,P=60628]   singularity_registry_get()                Returning NULL on 'DISABLE_OVERLAYFS'
DEBUG   [U=1947,P=60628]   singularity_registry_get()                Returning NULL on 'WRITABLE'
VERBOSE [U=1947,P=60628]   singularity_runtime_overlayfs()           OverlayFS not supported by host build
DEBUG   [U=1947,P=60628]   singularity_runtime_rootfs()              Returning container_directory: /usr/local/var/singularity/mnt/container
VERBOSE [U=1947,P=60628]   singularity_runtime_mounts()              Running all mount components
DEBUG   [U=1947,P=60628]   singularity_runtime_rootfs()              Returning container_directory: /usr/local/var/singularity/mnt/container
DEBUG   [U=1947,P=60628]   singularity_config_get_bool_char_impl()   Called singularity_config_get_bool(mount hostfs, no)
DEBUG   [U=1947,P=60628]   singularity_config_get_value_impl()       Returning configuration value mount hostfs='no'
DEBUG   [U=1947,P=60628]   singularity_config_get_bool_char_impl()   Return singularity_config_get_bool(mount hostfs, no) = 0
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_hostfs()        Not mounting host file systems per configuration
DEBUG   [U=1947,P=60628]   singularity_runtime_rootfs()              Returning container_directory: /usr/local/var/singularity/mnt/container
DEBUG   [U=1947,P=60628]   singularity_registry_get()                Returning NULL on 'CONTAIN'
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_binds()         Checking configuration file for 'bind path'
VERBOSE [U=1947,P=60628]   singularity_runtime_mount_binds()         Found 'bind path' = /etc/localtime, /etc/localtime
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_binds()         Checking if bind point is already mounted: /etc/localtime
DEBUG   [U=1947,P=60628]   singularity_runtime_rootfs()              Returning container_directory: /usr/local/var/singularity/mnt/container
DEBUG   [U=1947,P=60628]   check_mounted()                           Opening /proc/mounts
DEBUG   [U=1947,P=60628]   check_mounted()                           Iterating through /proc/mounts
DEBUG   [U=1947,P=60628]   singularity_priv_escalate()               Temporarily escalating privileges (U=1947)
DEBUG   [U=0,P=60628]      singularity_priv_escalate()               Clearing supplementary GIDs.
VERBOSE [U=0,P=60628]      singularity_runtime_mount_binds()         Binding '/etc/localtime' to '/usr/local/var/singularity/mnt/container//etc/localtime'
DEBUG   [U=0,P=60628]      singularity_priv_drop()                   Dropping privileges to UID=1947, GID=2712 (5 supplementary GIDs)
DEBUG   [U=0,P=60628]      singularity_priv_drop()                   Restoring supplementary groups
DEBUG   [U=1947,P=60628]   singularity_priv_drop()                   Confirming we have correct UID/GID
VERBOSE [U=1947,P=60628]   singularity_runtime_mount_binds()         Found 'bind path' = /etc/hosts, /etc/hosts
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_binds()         Checking if bind point is already mounted: /etc/hosts
DEBUG   [U=1947,P=60628]   singularity_runtime_rootfs()              Returning container_directory: /usr/local/var/singularity/mnt/container
DEBUG   [U=1947,P=60628]   check_mounted()                           Opening /proc/mounts
DEBUG   [U=1947,P=60628]   check_mounted()                           Iterating through /proc/mounts
DEBUG   [U=1947,P=60628]   singularity_priv_escalate()               Temporarily escalating privileges (U=1947)
DEBUG   [U=0,P=60628]      singularity_priv_escalate()               Clearing supplementary GIDs.
VERBOSE [U=0,P=60628]      singularity_runtime_mount_binds()         Binding '/etc/hosts' to '/usr/local/var/singularity/mnt/container//etc/hosts'
DEBUG   [U=0,P=60628]      singularity_priv_drop()                   Dropping privileges to UID=1947, GID=2712 (5 supplementary GIDs)
DEBUG   [U=0,P=60628]      singularity_priv_drop()                   Restoring supplementary groups
DEBUG   [U=1947,P=60628]   singularity_priv_drop()                   Confirming we have correct UID/GID
DEBUG   [U=1947,P=60628]   singularity_runtime_rootfs()              Returning container_directory: /usr/local/var/singularity/mnt/container
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_kernelfs()      Checking configuration file for 'mount proc'
DEBUG   [U=1947,P=60628]   singularity_config_get_bool_char_impl()   Called singularity_config_get_bool(mount proc, yes)
DEBUG   [U=1947,P=60628]   singularity_config_get_value_impl()       Returning configuration value mount proc='yes'
DEBUG   [U=1947,P=60628]   singularity_config_get_bool_char_impl()   Return singularity_config_get_bool(mount proc, yes) = 1
DEBUG   [U=1947,P=60628]   singularity_registry_get()                Returning NULL on 'PIDNS_ENABLED'
DEBUG   [U=1947,P=60628]   singularity_priv_escalate()               Temporarily escalating privileges (U=1947)
DEBUG   [U=0,P=60628]      singularity_priv_escalate()               Clearing supplementary GIDs.
VERBOSE [U=0,P=60628]      singularity_runtime_mount_kernelfs()      Bind-mounting host /proc
DEBUG   [U=0,P=60628]      singularity_priv_drop()                   Dropping privileges to UID=1947, GID=2712 (5 supplementary GIDs)
DEBUG   [U=0,P=60628]      singularity_priv_drop()                   Restoring supplementary groups
DEBUG   [U=1947,P=60628]   singularity_priv_drop()                   Confirming we have correct UID/GID
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_kernelfs()      Checking configuration file for 'mount sys'
DEBUG   [U=1947,P=60628]   singularity_config_get_bool_char_impl()   Called singularity_config_get_bool(mount sys, yes)
DEBUG   [U=1947,P=60628]   singularity_config_get_value_impl()       Returning configuration value mount sys='yes'
DEBUG   [U=1947,P=60628]   singularity_config_get_bool_char_impl()   Return singularity_config_get_bool(mount sys, yes) = 1
DEBUG   [U=1947,P=60628]   singularity_priv_escalate()               Temporarily escalating privileges (U=1947)
DEBUG   [U=0,P=60628]      singularity_priv_escalate()               Clearing supplementary GIDs.
VERBOSE [U=0,P=60628]      singularity_runtime_mount_kernelfs()      Mounting /sys
DEBUG   [U=0,P=60628]      singularity_priv_drop()                   Dropping privileges to UID=1947, GID=2712 (5 supplementary GIDs)
DEBUG   [U=0,P=60628]      singularity_priv_drop()                   Restoring supplementary groups
DEBUG   [U=1947,P=60628]   singularity_priv_drop()                   Confirming we have correct UID/GID
DEBUG   [U=1947,P=60628]   singularity_runtime_rootfs()              Returning container_directory: /usr/local/var/singularity/mnt/container
DEBUG   [U=1947,P=60628]   singularity_registry_get()                Returning NULL on 'CONTAIN'
DEBUG   [U=1947,P=60628]   singularity_config_get_value_impl()       Returning configuration value mount dev='yes'
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_dev()           Checking configuration file for 'mount dev'
DEBUG   [U=1947,P=60628]   singularity_config_get_bool_char_impl()   Called singularity_config_get_bool(mount dev, yes)
DEBUG   [U=1947,P=60628]   singularity_config_get_value_impl()       Returning configuration value mount dev='yes'
DEBUG   [U=1947,P=60628]   singularity_config_get_bool_char_impl()   Return singularity_config_get_bool(mount dev, yes) = 1
DEBUG   [U=1947,P=60628]   singularity_priv_escalate()               Temporarily escalating privileges (U=1947)
DEBUG   [U=0,P=60628]      singularity_priv_escalate()               Clearing supplementary GIDs.
VERBOSE [U=0,P=60628]      singularity_runtime_mount_dev()           Bind mounting /dev
DEBUG   [U=0,P=60628]      singularity_priv_drop()                   Dropping privileges to UID=1947, GID=2712 (5 supplementary GIDs)
DEBUG   [U=0,P=60628]      singularity_priv_drop()                   Restoring supplementary groups
DEBUG   [U=1947,P=60628]   singularity_priv_drop()                   Confirming we have correct UID/GID
DEBUG   [U=1947,P=60628]   singularity_runtime_rootfs()              Returning container_directory: /usr/local/var/singularity/mnt/container
DEBUG   [U=1947,P=60628]   singularity_config_get_bool_char_impl()   Called singularity_config_get_bool(mount home, yes)
DEBUG   [U=1947,P=60628]   singularity_config_get_value_impl()       Returning configuration value mount home='yes'
DEBUG   [U=1947,P=60628]   singularity_config_get_bool_char_impl()   Return singularity_config_get_bool(mount home, yes) = 1
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_home()          Checking that home directry is configured: /tsd/p158/home/p158-uxscti
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_home()          Checking if SINGULARITY_CONTAIN is set
DEBUG   [U=1947,P=60628]   singularity_registry_get()                Returning NULL on 'CONTAIN'
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_home()          Checking if home directories are being influenced by user
DEBUG   [U=1947,P=60628]   singularity_registry_get()                Returning NULL on 'HOME'
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_home()          Checking ownership of home directory source: /tsd/p158/home/p158-uxscti
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_home()          Checking to make sure home directory destination is a full path: /tsd/p158/home/p158-uxscti
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_home()          Checking if home directory is already mounted: /tsd/p158/home/p158-uxscti
DEBUG   [U=1947,P=60628]   singularity_runtime_rootfs()              Returning container_directory: /usr/local/var/singularity/mnt/container
DEBUG   [U=1947,P=60628]   check_mounted()                           Opening /proc/mounts
DEBUG   [U=1947,P=60628]   check_mounted()                           Iterating through /proc/mounts
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_home()          Checking if overlay is enabled
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_home()          Staging home directory
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_home()          Checking if sessiondir/tmpdir is set
DEBUG   [U=1947,P=60628]   singularity_registry_get()                Returning value from registry: 'SESSIONDIR' = '/usr/local/var/singularity/mnt/session'
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_home()          Identifying the base home directory: /tsd/p158/home/p158-uxscti
DEBUG   [U=1947,P=60628]   basedir()                                 Obtaining basedir for: /tsd/p158/home/p158-uxscti
DEBUG   [U=1947,P=60628]   basedir()                                 Iterating basedir: /tsd/p158/home/p158-uxscti
DEBUG   [U=1947,P=60628]   basedir()                                 Iterating basedir: /tsd/p158/home
DEBUG   [U=1947,P=60628]   basedir()                                 Iterating basedir: /tsd/p158
DEBUG   [U=1947,P=60628]   basedir()                                 Iterating basedir: /tsd
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_home()          Checking home directory base exists in container: /tsd
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_home()          Creating temporary directory to stage home: /usr/local/var/singularity/mnt/session/tsd/p158/home/p158-uxscti
DEBUG   [U=1947,P=60628]   s_mkpath()                                Creating directory: /usr/local/var/singularity/mnt/session/tsd
DEBUG   [U=1947,P=60628]   s_mkpath()                                Creating directory: /usr/local/var/singularity/mnt/session/tsd/p158
DEBUG   [U=1947,P=60628]   s_mkpath()                                Creating directory: /usr/local/var/singularity/mnt/session/tsd/p158/home
DEBUG   [U=1947,P=60628]   s_mkpath()                                Creating directory: /usr/local/var/singularity/mnt/session/tsd/p158/home/p158-uxscti
DEBUG   [U=1947,P=60628]   singularity_priv_escalate()               Temporarily escalating privileges (U=1947)
DEBUG   [U=0,P=60628]      singularity_priv_escalate()               Clearing supplementary GIDs.
VERBOSE [U=0,P=60628]      singularity_runtime_mount_home()          Mounting home directory source to stage: /tsd/p158/home/p158-uxscti -> /usr/local/var/singularity/mnt/session/tsd/p158/home/p158-uxscti
ERROR   [U=0,P=60628]      singularity_runtime_mount_home()          Failed to mount home directory /tsd/p158/home/p158-uxscti -> /usr/local/var/singularity/mnt/session/tsd/p158/home/p158-uxscti: Permission denied
ABORT   [U=0,P=60628]      singularity_runtime_mount_home()          Retval = 255

Is there any obvious reason for this?
Thanks,
Till

Till

unread,
Jun 28, 2017, 2:10:27 PM6/28/17
to singularity
Here comes the rest of the debug output:

DEBUG   [U=1947,P=60628]   singularity_registry_get()                Returning NULL on 'OVERLAYFS_ENABLED'
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_home()          Staging home directory
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_home()          Checking if sessiondir/tmpdir is set
DEBUG   [U=1947,P=60628]   singularity_registry_get()                Returning value from registry: 'SESSIONDIR' = '/usr/local/var/singularity/mnt/session'
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_home()          Identifying the base home directory: /tsd/p158/home/p158-uxscti
DEBUG   [U=1947,P=60628]   basedir()                                 Obtaining basedir for: /tsd/p158/home/p158-uxscti
DEBUG   [U=1947,P=60628]   basedir()                                 Iterating basedir: /tsd/p158/home/p158-uxscti
DEBUG   [U=1947,P=60628]   basedir()                                 Iterating basedir: /tsd/p158/home
DEBUG   [U=1947,P=60628]   basedir()                                 Iterating basedir: /tsd/p158
DEBUG   [U=1947,P=60628]   basedir()                                 Iterating basedir: /tsd
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_home()          Checking home directory base exists in container: /tsd
DEBUG   [U=1947,P=60628]   singularity_runtime_mount_home()          Creating temporary directory to stage home: /usr/local/var/singularity/mnt/session/tsd/p158/home/p158-uxscti
DEBUG   [U=1947,P=60628]   s_mkpath()                                Creating directory: /usr/local/var/singularity/mnt/session/tsd
DEBUG   [U=1947,P=60628]   s_mkpath()                                Creating directory: /usr/local/var/singularity/mnt/session/tsd/p158
DEBUG   [U=1947,P=60628]   s_mkpath()                                Creating directory: /usr/local/var/singularity/mnt/session/tsd/p158/home
DEBUG   [U=1947,P=60628]   s_mkpath()                                Creating directory: /usr/local/var/singularity/mnt/session/tsd/p158/home/p158-uxscti
DEBUG   [U=1947,P=60628]   singularity_priv_escalate()               Temporarily escalating privileges (U=1947)
DEBUG   [U=0,P=60628]      singularity_priv_escalate()               Clearing supplementary GIDs.
VERBOSE [U=0,P=60628]      singularity_runtime_mount_home()          Mounting home directory source to stage: /tsd/p158/home/p158-uxscti -> /usr/local/var/singularity/mnt/session/tsd/p158/home/p158-uxscti
ERROR   [U=0,P=60628]      singularity_runtime_mount_home()          Failed to mount home directory /tsd/p158/home/p158-uxscti -> /usr/local/var/singularity/mnt/session/tsd/p158/home/p158-uxscti: Permission denied
ABORT   [U=0,P=60628]      singularity_runtime_mount_home()          Retval = 255

Martin Cuma

unread,
Jun 28, 2017, 2:16:41 PM6/28/17
to singularity
Any chance you happen to have /tsd/p158/home/p158-uxscti as your home
directory on the cluster?
 
Based on the fact that it's failing to mount /tsd/p158/home/p158-uxscti as   
home, I suspect Singularity is assuming this to be your home on the
cluster, but, you mkdir -p it in the container.

So, remove that line, but keep the "mkdir /tsd" since that creates the   
mount point for your home on the cluster.
 
HTH,
MC

Till

unread,
Jun 28, 2017, 2:55:28 PM6/28/17
to singularity
Thanks,
apparently I get the exact same error msg. 
Might there be a problem with the permissions?

/till

Till

unread,
Jun 28, 2017, 3:54:07 PM6/28/17
to singularity
I tried different images and get always the same error!
Singularity has just been upgraded to 2.3.1 on the cluster, maybe 
something went wrong during the process......
/till

Till

unread,
Jul 6, 2017, 1:32:11 PM7/6/17
to singularity
Hi,

just wanted to give a quick feedback on this issue.

The javascript meteor-app that I wanted to include in the singularity container was on GitHub in development mode. 
And, meteor development mode is not suited to be deployed in a container. What I needed to do was to package the
app and run it as a node.js app which made it necessary to install a mongo-db server. Both the mongo-db server and 
the node.js app can easily be started from userspace. 

The other problem seems to be related to the fact that our home directories on the cluster are placed on a mounted filesystem.
And apparently, singularity quits when it can't mount the home directory before it binds the directories specified in the -B flag. I circumvented
this by specifying a home directory on the local filesystem with the -H flag. I think the admin could also just have disabled the automounting of
the home directory in the singularity configuration file. 

So, all after all the app is running now in the singularity container on the cluster. Great. Thank you for singularity.

/till 
Reply all
Reply to author
Forward
0 new messages