Singularity running software that is not installed in the container

[Oriol Guitart Pla]

Hello,


I have been testing singularity to use it in our cluster. I was
surprised to see that the software that is installed in the computer
where singularity runs, it is also available from inside the container.
For instance, I have an R script that calls a java jar and I'm calling
it from singularity. I do not have java installed in my container but it
is still working because it uses the java that is installed in my computer.

Is this behavior normal in containers? How to make sure that is using,
for instance, the java that is installed in the container and not the
one in my computer? Is it just a matter of playing with the PATH? By the
way, the java issue is just an example.

Thanks in advance for your comments,


oriol

[vanessa s]

Hey Oriol!

In a nutshell, yes - one of the features of Singularity is seamlessness between the container and environment of the host, and this is very different from other container solutions (e.g. Docker) and one of the reasons it works really great for scientific computing. You can fine turn which software gets used by way of PATH or by calling the executable directly (the latter is usually my preference). If you have specific use cases or example workflows that you want help with, feel free to post on here and we can walk through different things to try.

Best,

Vanessa

> --
> You received this message because you are subscribed to the Google Groups "singularity" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

[Paolo Di Tommaso]

IMO it would have useful to have command line option to *not* import the hosting environment in order to isolate the contained application(s) from the external one. 

Cheers,
Paolo

On Thu, Mar 30, 2017 at 5:42 PM, vanessa s <vso...@gmail.com> wrote:

Hey Oriol!

In a nutshell, yes - one of the features of Singularity is seamlessness between the container and environment of the host, and this is very different from other container solutions (e.g. Docker) and one of the reasons it works really great for scientific computing. You can fine turn which software gets used by way of PATH or by calling the executable directly (the latter is usually my preference). If you have specific use cases or example workflows that you want help with, feel free to post on here and we can walk through different things to try.

Best,

Vanessa

> On Mar 30, 2017, at 11:14 AM, Oriol Guitart Pla <oriol.guitart@irbbarcelona.org> wrote:
>
> Hello,
>
>
> I have been testing singularity to use it in our cluster. I was surprised to see that the software that is installed in the computer where singularity runs, it is also available from inside the container. For instance, I have an R script that calls a java jar and I'm calling it from singularity. I do not have java installed in my container but it is still working because it uses the java that is installed in my computer.
>
> Is this behavior normal in containers? How to make sure that is using, for instance, the java that is installed in the container and not the one in my computer? Is it just a matter of playing with the PATH? By the way, the java issue is just an example.
>
> Thanks in advance for your comments,
>
>
> oriol
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "singularity" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

--
You received this message because you are subscribed to the Google Groups "singularity" group.

To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

[Amir Shams]

Is there any way that we can grab the singularity exec command line??

I want the assigned path and direction for my Logging

Amir

On Thu, Mar 30, 2017 at 1:27 PM, Paolo Di Tommaso <paolo.d...@gmail.com> wrote:

IMO it would have useful to have command line option to *not* import the hosting environment in order to isolate the contained application(s) from the external one. 

Cheers,
Paolo

On Thu, Mar 30, 2017 at 5:42 PM, vanessa s <vso...@gmail.com> wrote:

Hey Oriol!

In a nutshell, yes - one of the features of Singularity is seamlessness between the container and environment of the host, and this is very different from other container solutions (e.g. Docker) and one of the reasons it works really great for scientific computing. You can fine turn which software gets used by way of PATH or by calling the executable directly (the latter is usually my preference). If you have specific use cases or example workflows that you want help with, feel free to post on here and we can walk through different things to try.

Best,

Vanessa

> On Mar 30, 2017, at 11:14 AM, Oriol Guitart Pla <oriol....@irbbarcelona.org> wrote:
>
> Hello,
>
>
> I have been testing singularity to use it in our cluster. I was surprised to see that the software that is installed in the computer where singularity runs, it is also available from inside the container. For instance, I have an R script that calls a java jar and I'm calling it from singularity. I do not have java installed in my container but it is still working because it uses the java that is installed in my computer.
>
> Is this behavior normal in containers? How to make sure that is using, for instance, the java that is installed in the container and not the one in my computer? Is it just a matter of playing with the PATH? By the way, the java issue is just an example.
>
> Thanks in advance for your comments,
>
>
> oriol
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "singularity" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

--
You received this message because you are subscribed to the Google Groups "singularity" group.

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

[Gregory M. Kurtzer]

Hi Oriol,

Can you confirm (e.g. with an strace from within the container) and follow the execution of your R script where calls the java jar file (e.g. the exact path of the binary calling your jar file)?

Do you have a java installation in a shared location (e.g. `$HOME/bin`) or another location that has been bound into the container?

Greg

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

--

Gregory M. Kurtzer

HPC Systems Architect and Technology Developer

Lawrence Berkeley National Laboratory HPCS
University of California Berkeley Research IT
Singularity Linux Containers (http://singularity.lbl.gov/)

Warewulf Cluster Management (http://warewulf.lbl.gov/)

GitHub: https://github.com/gmkurtzer, Twitter: https://twitter.com/gmkurtzer

[David Godlove]

Hey Paolo,

Check out the -c option.  (Can be used with things like shell and exec)

    -c/--contain        This option disables the automatic sharing of writable

                        filesystems on your host (e.g. $HOME and /tmp).

On Thu, Mar 30, 2017 at 1:53 PM, Gregory M. Kurtzer <gmku...@lbl.gov> wrote:

Hi Oriol,

Can you confirm (e.g. with an strace from within the container) and follow the execution of your R script where calls the java jar file (e.g. the exact path of the binary calling your jar file)?

Do you have a java installation in a shared location (e.g. `$HOME/bin`) or another location that has been bound into the container?

Greg

On Thu, Mar 30, 2017 at 8:14 AM, Oriol Guitart Pla <oriol.guitart@irbbarcelona.org> wrote:

Hello,


I have been testing singularity to use it in our cluster. I was surprised to see that the software that is installed in the computer where singularity runs, it is also available from inside the container. For instance, I have an R script that calls a java jar and I'm calling it from singularity. I do not have java installed in my container but it is still working because it uses the java that is installed in my computer.

Is this behavior normal in containers? How to make sure that is using, for instance, the java that is installed in the container and not the one in my computer? Is it just a matter of playing with the PATH? By the way, the java issue is just an example.

Thanks in advance for your comments,


oriol



--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

--

Gregory M. Kurtzer

HPC Systems Architect and Technology Developer

Lawrence Berkeley National Laboratory HPCS
University of California Berkeley Research IT
Singularity Linux Containers (http://singularity.lbl.gov/)

Warewulf Cluster Management (http://warewulf.lbl.gov/)

GitHub: https://github.com/gmkurtzer, Twitter: https://twitter.com/gmkurtzer

--
You received this message because you are subscribed to the Google Groups "singularity" group.

To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

[Paolo Di Tommaso]

Hi David, 

I'm not referring the automatic mount of host file paths, but to the automatically import of the host environment variables in the container.  

However checking the latest version in the development branch it turns out that it include a new `--cleanenv ` option that's exactly what I was mentioning. 

Great work Singularity team 👏👏 

Cheers,

Paolo

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

[Thomas Hartmann]

Hi Amir,

singularity child execs PID should be available in the host's
proc/namespace. You can check the command line with something like
cat /proc/${SINGEXEC}/cmdline | tr '\0' '\n'
/proc/${SINGEXEC}/fd{info}/ might also a place to check for open
files/sockets.
I haven't checked but I guess the container namespace's bind mounts etc
might show up somewhere in /proc/${SINGEXEC}/mounts{tats,info}

Cheers,
Thomas

> <mailto:oriol....@irbbarcelona.org>> wrote:
> >
> > Hello,
> >
> >
> > I have been testing singularity to use it in our cluster. I
> was surprised to see that the software that is installed in the
> computer where singularity runs, it is also available from
> inside the container. For instance, I have an R script that
> calls a java jar and I'm calling it from singularity. I do not
> have java installed in my container but it is still working
> because it uses the java that is installed in my computer.
> >
> > Is this behavior normal in containers? How to make sure that
> is using, for instance, the java that is installed in the
> container and not the one in my computer? Is it just a matter of
> playing with the PATH? By the way, the java issue is just an
> example.
> >
> > Thanks in advance for your comments,
> >
> >
> > oriol
> >
> >
> >
> > --
> > You received this message because you are subscribed to the
> Google Groups "singularity" group.
> > To unsubscribe from this group and stop receiving emails from
> it, send an email to singularity...@lbl.gov

> <mailto:singularity%2Bunsu...@lbl.gov>.

>
> --
> You received this message because you are subscribed to the
> Google Groups "singularity" group.
> To unsubscribe from this group and stop receiving emails from
> it, send an email to singularity...@lbl.gov

> <mailto:singularity%2Bunsu...@lbl.gov>.

>
>
> --
> You received this message because you are subscribed to the Google
> Groups "singularity" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to singularity...@lbl.gov

> <mailto:singularity...@lbl.gov>.

>
>
> --
> You received this message because you are subscribed to the Google
> Groups "singularity" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to singularity...@lbl.gov

> <mailto:singularity...@lbl.gov>.

[Oriol Guitart]

Hi Greg,

Yes, my java installation is on a shared location that is bounded to the container. As Vanessa said, I see this behavior is done on purpose and the way to deal with it would be to make sure the PATH only have what I need to run in the container or call the executable directly.
Thanks for your help, I need to think more about it to find the best way to manage that.
Regards,

oriol