Keeep working directory with different home mount point

[mle...@gmail.com]

Hi,

I am using Singularity to have reproducible paths for our Analyses. I also provide shell scripts to my colleagues so that they have better reproducibility without caring too much about the technical details of Singularity (they have no CS background). I am working on a script that can execute a command in the container, like "./container exec some_command". This will execute the command in a pre-defined Singularity image.

To improve reproducibility, we are mapping the root directory of the Analysis to /proj, so that the paths are the same on each machine. (Not all users can be trusted to not use absolute paths in their analyses). It is done by providing an argument to Singularity like "-H $PROJECT_DIR:/proj".

But this causes a problem: The working directory is not preserved. If my project and the container starting script is on /home/user/project, and my working directory is /home/user/project/subdir, I want the working directory in singularity to be set to /proj/subdir. Currently I do this by a lengthy bash script which has some downsides like (1) lengthy bash scripts often have subtle bugs and portability issues (2) it does not work with other bind-mounts outside of the project directory

Is there a function in Singularity to map a host location to a within-container path?

Cheers,

Moritz

[Thomas Hartmann]

Hi Moritz,

you can bind a path from outside the container's context to a different
path in the container
--bind /hst/src/pth:/cnt/dst/pth {:option like rw or ro)

(the `--home` flag works similarly for setting the HOME dir)

Cheers,
Thomas

> --
> You received this message because you are subscribed to the Google
> Groups "singularity" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to singularity...@lbl.gov
> <mailto:singularity...@lbl.gov>.
> To view this discussion on the web visit
> https://groups.google.com/a/lbl.gov/d/msgid/singularity/7bd473a2-4e4a-47c5-9d05-c7f4c7f2daadn%40lbl.gov
> <https://groups.google.com/a/lbl.gov/d/msgid/singularity/7bd473a2-4e4a-47c5-9d05-c7f4c7f2daadn%40lbl.gov?utm_medium=email&utm_source=footer>.

[mle...@gmail.com]

Thomas Hartmann schrieb am Mittwoch, 28. Juli 2021 um 13:34:15 UTC+2:

you can bind a path from outside the container's context to a different
path in the container
--bind /hst/src/pth:/cnt/dst/pth {:option like rw or ro)

(the `--home` flag works similarly for setting the HOME dir)

Hi, thanks for the answer. I tried what you suggested, but it turns out that the working dir is not preserved. I'll try to make it clearer with a reproducible example:

singularity pull library://default/alpine

mkdir test

cd test

singularity exec -H ..:/proj ../alpine_latest.sif pwd

#> /proj

expected:

#> /proj/test

So the problem I have is that if I set the home directory, the current working directory is not preserved but set to the home directory.

There is the option of using --pwd:

singularity exec -H ..:/proj --pwd /proj/test ../alpine_latest.sif pwd

#> /proj/test

But the problem here is that I need to figure out the inside-container path /proj/test by myself with the bash script I provide to my colleagues. I have to consider not only the mount to /home, but also to other directories that are mounted with --bind (read-only analysis input data). An example is below. But that is essentially rebuilding the singularity file namespacing logic in bash; I want to avoid that as it will be fragile and bug-ridden. Can singularity start with the current WD translated to the in-container path, taking into account also all other bind mounts of the container, not only /home?

cd /home/user/test

ls

#>  alpine_latest.sif

mkdir home home/dir1 external_files

sing_pwd(){

  # mount project folder and external files to /proj and /ext

  # and run "pwd"

  singularity exec \

    -H /home/user/test:/proj \

    -B /home/user/external_files:/ext \

    /home/user/test/alpine_latest.sif \

    pwd

}

cd /home/user/test/home/dir1 

sing_pwd

#> /proj/dir1   # (expected; actual output is: /proj)

cd /home/user/test/external_files

sing_pwd

#> /ext         # (expected; actual output is: /proj)

Cheers,

Moritz

[Thomas Hartmann]

Hi Moritz,

I am not sure, if I follow you

Would it not work to define a input path in the container context and
bind whatever external path to it?

I.e.
singularity cmd --bind /foo:/data
and point the application to `/data/...`

Cheers,
Thomas

> --
> You received this message because you are subscribed to the Google
> Groups "singularity" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to singularity...@lbl.gov
> <mailto:singularity...@lbl.gov>.
> To view this discussion on the web visit

> https://groups.google.com/a/lbl.gov/d/msgid/singularity/ad6fbf3c-0be0-436f-8968-a98e28061d17n%40lbl.gov
> <https://groups.google.com/a/lbl.gov/d/msgid/singularity/ad6fbf3c-0be0-436f-8968-a98e28061d17n%40lbl.gov?utm_medium=email&utm_source=footer>.