TMPDIR environment variable missing from container environment

[Trey Dockendorf]

I have noticed that the environment variable TMPDIR is not present when using shell or exec subcommands.  This is running Singularity 2.3.1 on RHEL 7.3.  Is there some special treatment in Singularity with regard to the TMPDIR environment variable?  The script we use to set TMPDIR also sets PFSDIR and that environment variable is present.

$ echo $TMPDIR

/tmp/pbstmp.1586923

$ singularity shell centos7.img 

Singularity: Invoking an interactive shell within container...

Singularity centos7.img:~> echo $TMPDIR

Singularity centos7.img:~>

$ echo $PFSDIR

/fs/scratch/tdockendorf/1586923

$ singularity shell centos7.img 

Singularity: Invoking an interactive shell within container...

Singularity centos7.img:~> echo $PFSDIR

/fs/scratch/tdockendorf/1586923

$ singularity exec  centos7.img env | grep TMPDIR

$

I did a diff between my host environment and the singularity container environment and that is below.

--- /tmp/host.env       2017-08-18 08:26:43.036541513 -0400

+++ /tmp/container.env  2017-08-18 08:26:35.040501156 -0400

@@ -1,10 +1,3 @@

-}

-}

- [ $? = 0 ] && eval $(${LMOD_SETTARG_CMD:-:} -s sh)

-BASH_ENV=/usr/share/lmod/lmod/init/bash

-BASH_FUNC_ml()=() {  eval $($LMOD_DIR/ml_cmd "$@")

-BASH_FUNC_module()=() {  eval $($LMOD_CMD bash "$@");

-_=/bin/env

 CPATH=/opt/intel/compilers_and_libraries_2016.3.210/linux/mkl/include:/opt/intel/compilers_and_libraries_2016.3.210/linux/tbb/include

 CVS_RSH=ssh

 DAALROOT=/opt/intel/compilers_and_libraries_2016.3.210/linux/daal

@@ -21,7 +14,7 @@

 IPPROOT=/opt/intel/compilers_and_libraries_2016.3.210/linux/ipp

 KDEDIRS=/usr

 LANG=en_US.UTF-8

-LD_LIBRARY_PATH=/opt/mvapich2/intel/16.0/2.2/lib:/opt/intel/debugger_2016/libipt/intel64/lib:/opt/intel/compilers_and_libraries_2016.3.210/linux/compiler/lib/intel64_lin:/opt/intel/compilers_and_libraries_2016.3.210/linux/daal/lib/intel64_lin:/opt/intel/compilers_and_libraries_2016.3.210/linux/ipp/lib/intel64_lin:/opt/intel/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64_lin:/opt/intel/compilers_and_libraries_2016.3.210/linux/tbb/lib/intel64_lin/gcc4.4:/opt/torque/lib64:/opt/torque/lib::

+LD_LIBRARY_PATH=/.singularity.d/libs

 LESSOPEN=||/usr/bin/lesspipe.sh %s

 LIBRARY_PATH=/opt/intel/compilers_and_libraries_2016.3.210/linux/compiler/lib/intel64_lin:/opt/intel/compilers_and_libraries_2016.3.210/linux/daal/lib/intel64_lin:/opt/intel/compilers_and_libraries_2016.3.210/linux/ipp/lib/intel64_lin:/opt/intel/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64_lin:/opt/intel/compilers_and_libraries_2016.3.210/linux/tbb/lib/intel64_lin/gcc4.4

 _LMFILES_=/usr/local/share/lmodfiles/Core/intel/16.0.3.lua:/usr/local/share/lmodfiles/Compiler/intel/16.0/mvapich2/2.2.lua:/usr/local/share/lmodfiles/Core/modules/au2016.lua:/users/sysp/tdockendorf/modulefiles/singularity/2.3.1.lua

@@ -67,12 +60,11 @@

 MPI_LIBS=-L/opt/mvapich2/intel/16.0/2.2/lib -lmpich -libverbs -lpthread

 MV2_ENABLE_AFFINITY=0

 MV2_IBA_HCA=mlx5_0

-NLSPATH=/opt/intel/debugger_2016/gdb/intel64/share/locale/%l_%t/%N:/opt/intel/compilers_and_libraries_2016.3.210/linux/compiler/lib/intel64_lin/locale/%l_%t/%N:/opt/intel/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64_lin/locale/%l_%t/%N

 OSC_COMPILER=intel

 OSC_COMPILER_MAJOR=16

 OSC_COMPILER_MINOR=0

 OSC_DEFAULT_ACCOUNT=PZS0090

-PATH=/opt/singularity/bin:/opt/mvapich2/intel/16.0/2.2/bin:/opt/intel/itac_9.1/bin:/opt/intel/advisor_xe_2016/bin64:/opt/intel/vtune_amplifier_xe_2016/bin64:/opt/intel/inspector_xe_2016/bin64:/opt/intel/compilers_and_libraries_2016.3.210/linux/bin/intel64:/opt/torque/bin:/usr/lib64/qt-3.3/bin:/opt/osc/bin:/opt/moab/bin:/bin:/usr/bin:/usr/local/bin:/usr/local/sbin:/usr/sbin:/opt/ibutils/bin:/opt/ddn/ime/bin:/users/sysp/tdockendorf/.rvm/bin

+PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

 PBSCOREDUMP=""

 PBS_DEFAULT=owens-batch.ten.osc.edu

 PBS_ENVIRONMENT=PBS_INTERACTIVE

@@ -107,6 +99,7 @@

 PDSH_RCMD_TYPE=ssh

 PDSH_SSH_ARGS_APPEND=-oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oLogLevel=ERROR

 PFSDIR=/fs/scratch/tdockendorf/1586923.owens-batch.ten.osc.edu

+PS1=Singularity> 

 PWD=/users/sysp/tdockendorf

 QTDIR=/usr/lib64/qt-3.3

 QT_GRAPHICSSYSTEM_CHECKED=1

@@ -119,6 +112,8 @@

 rvm_version=1.27.0 (latest)

 SHELL=/bin/bash

 SHLVL=1

+SINGULARITY_CONTAINER=centos7.img

+SINGULARITY_NAME=centos7.img

 SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass

 _system_arch=x86_64

 _system_name=RedHat

@@ -128,5 +123,4 @@

 TACC_FAMILY_MPI=mvapich2

 TBBROOT=/opt/intel/compilers_and_libraries_2016.3.210/linux/tbb

 TERM=screen

-TMPDIR=/tmp/pbstmp.1586923

 USER=tdockendorf

[Kenneth Hoste]

Hi Trey,

This happens because of the setuid permissions on the Singularity, afaik...

If you want to preserve certain environment variables, you should 'copy' them first into an $SINGULARITYENV_* environment variables, see also http://singularity.lbl.gov/docs-environment-metadata .


regards,

Kenneth

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

[Dockendorf, Trey]

Kenneth,

 

Thanks, that was the issue.  Setting SINGULARITYENV_TMPDIR=$TMPDIR worked.

 

Thanks,

- Trey

 

-- 

Trey Dockendorf

HPC Systems Engineer

Ohio Supercomputer Center