Proprietary SW: howto restrict container access to local computing site?

[Stefan Kombrink]

Hi,

 I wonder what are the possibilities to offer licensed SW containerized by singularity while still preventing users to copy the container.
Right now we achieve this by allowing certain binaries to be executable but not readable.
If I get it right there is no way to let users run containers they can't read but execute.
Anyways this approach using access rights seems very limited.

I had that idea that containers might be encrypted using a key which can be installed in the system alongside singularity and allow to decrypt the container during loading.
The appropriate key could be loaded prior during the suid_exec part.
Am I overlooking something?
Is there an easier way to achieve what I want?

I'd really love to come up with a viable solution for that problem as it is an obstacle for containerization/virtualization which affects many SW suites we are currently offering...

greets and thanks
Stefan

[Gregory M. Kurtzer]

Hi Stefan,

This is an interesting idea! There are a couple of ways to handle it considering that Singularity has a privileged mode of operation... We can look into encrypting the file system contained within the image and locate the key within a file that only has root access, then count on our privileged code to read that in.

Technically we can also do this by circumventing POSIX permissions on the image files, and allow containers that can only be read by root to be run by normal users, but then how do we implement this securely and reliably? ... I am open to suggestions. :)

Greg

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

--

Gregory M. Kurtzer

HPC Systems Architect and Technology Developer

Lawrence Berkeley National Laboratory HPCS
University of California Berkeley Research IT
Singularity Linux Containers (http://singularity.lbl.gov/)

Warewulf Cluster Management (http://warewulf.lbl.gov/)

GitHub: https://github.com/gmkurtzer, Twitter: https://twitter.com/gmkurtzer

[r...@open-mpi.org]

I’ve been pondering a related issue lately. Suppose I create a container that includes a licensed library, and then I migrate that container to another cluster. Depending on how the license is enforced (e.g., a license manager daemon vs a local “token”), I may now be operating the container in violation of the license.

Now suppose I distribute the container - have I now enabled everyone to use that licensed software without paying for it? You have a similar problem with your proposal - what if I distribute the encryption key along with the container?

Related question: suppose the library is actually GPL. Now GPL doesn’t hold sway unless you distribute your software. If you statically link your code against a GPL library and then distribute that binary, we know that makes your code subject to the GPL.

However, if I put a GPL library in a container, and then distribute the container - have I now caused my code that links against that library at runtime to become subject to GPL? I am distributing the combination, albeit they are not actually combined until you run the container.

Makes my head hurt...

Ralph

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

[Stefan Kombrink]

Yeah, all these licensing issues are really complex .. :D

Am Mittwoch, 26. Oktober 2016 21:00:58 UTC+2 schrieb r...@open-mpi.org:

I’ve been pondering a related issue lately. Suppose I create a container that includes a licensed library, and then I migrate that container to another cluster. Depending on how the license is enforced (e.g., a license manager daemon vs a local “token”), I may now be operating the container in violation of the license.

Now suppose I distribute the container - have I now enabled everyone to use that licensed software without paying for it? You have a similar problem with your proposal - what if I distribute the encryption key along with the container?

Sure, if you distributed the private key that allows the access to the encrypted container. In my use case, however, I'd prevent this from happening  by having it installed on our cluster with root-only access exclusively.
 

Related question: suppose the library is actually GPL. Now GPL doesn’t hold sway unless you distribute your software. If you statically link your code against a GPL library and then distribute that binary, we know that makes your code subject to the GPL.

However, if I put a GPL library in a container, and then distribute the container - have I now caused my code that links against that library at runtime to become subject to GPL? I am distributing the combination, albeit they are not actually combined until you run the container.

Aaah took me long to understand what you actually meant: Technically no. You're just not allowed to execute the container ... hehe
Here's another one: If I containerize licensed software into encrypted containers, can I be sued for distributing it? Or just for distributing the private key?
 

Makes my head hurt...

Ralph

On Oct 26, 2016, at 11:21 AM, Gregory M. Kurtzer <gmku...@lbl.gov> wrote:

Hi Stefan,

This is an interesting idea! There are a couple of ways to handle it considering that Singularity has a privileged mode of operation... We can look into encrypting the file system contained within the image and locate the key within a file that only has root access, then count on our privileged code to read that in.

Technically we can also do this by circumventing POSIX permissions on the image files, and allow containers that can only be read by root to be run by normal users, but then how do we implement this securely and reliably? ... I am open to suggestions. :)

Greg

On Mon, Oct 24, 2016 at 2:40 AM, 'Stefan Kombrink' via singularity <singu...@lbl.gov> wrote:

Hi,

 I wonder what are the possibilities to offer licensed SW containerized by singularity while still preventing users to copy the container.
Right now we achieve this by allowing certain binaries to be executable but not readable.
If I get it right there is no way to let users run containers they can't read but execute.
Anyways this approach using access rights seems very limited.

I had that idea that containers might be encrypted using a key which can be installed in the system alongside singularity and allow to decrypt the container during loading.
The appropriate key could be loaded prior during the suid_exec part.
Am I overlooking something?
Is there an easier way to achieve what I want?

I'd really love to come up with a viable solution for that problem as it is an obstacle for containerization/virtualization which affects many SW suites we are currently offering...

greets and thanks
Stefan

--
You received this message because you are subscribed to the Google Groups "singularity" group.

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

--

Gregory M. Kurtzer

HPC Systems Architect and Technology Developer

Lawrence Berkeley National Laboratory HPCS
University of California Berkeley Research IT
Singularity Linux Containers (http://singularity.lbl.gov/)

Warewulf Cluster Management (http://warewulf.lbl.gov/)

GitHub: https://github.com/gmkurtzer, Twitter: https://twitter.com/gmkurtzer

[r...@open-mpi.org]

On Oct 26, 2016, at 1:10 PM, Stefan Kombrink <stefan....@googlemail.com> wrote:

Yeah, all these licensing issues are really complex .. :D

Am Mittwoch, 26. Oktober 2016 21:00:58 UTC+2 schrieb r...@open-mpi.org:

I’ve been pondering a related issue lately. Suppose I create a container that includes a licensed library, and then I migrate that container to another cluster. Depending on how the license is enforced (e.g., a license manager daemon vs a local “token”), I may now be operating the container in violation of the license.

Now suppose I distribute the container - have I now enabled everyone to use that licensed software without paying for it? You have a similar problem with your proposal - what if I distribute the encryption key along with the container?

Sure, if you distributed the private key that allows the access to the encrypted container. In my use case, however, I'd prevent this from happening  by having it installed on our cluster with root-only access exclusively.
 

Related question: suppose the library is actually GPL. Now GPL doesn’t hold sway unless you distribute your software. If you statically link your code against a GPL library and then distribute that binary, we know that makes your code subject to the GPL.

However, if I put a GPL library in a container, and then distribute the container - have I now caused my code that links against that library at runtime to become subject to GPL? I am distributing the combination, albeit they are not actually combined until you run the container.

Aaah took me long to understand what you actually meant: Technically no. You're just not allowed to execute the container ... hehe
Here's another one: If I containerize licensed software into encrypted containers, can I be sued for distributing it? Or just for distributing the private key?

Oooohh...now my head really hurts! I honestly don’t know. You distributed the software, but it isn’t usable without the key...but you _did_ distribute it.

Sounds like something to ask a lawyer.

[Gregory M. Kurtzer]

Even worse when you are dealing with users who have controlled software. Most container platforms don't offer the granularity of ACLs necessary to limit container access to a subset of users on a particular host.

On second thought,... I don't even want to think about this... Purging from memory so I can plead ignorance (which isn't hard for me). haha

On Wed, Oct 26, 2016 at 1:24 PM, r...@open-mpi.org <r...@open-mpi.org> wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

[Stefan Kombrink]

Okay, after giving it more thought I found another solution for our needs:

Have a look at this little patch:

--- src/lib/rootfs/image/image.c    2016-10-10 19:25:59.000000000 +0200
+++ /home/ul/ul_kiz/ul_l_nsn25/image.c    2016-10-28 10:04:26.974707586 +0200
@@ -81,7 +81,16 @@
     } else {
         if ( ( image_fp = fopen(source, "r") ) == NULL ) { // Flawfinder: ignore
             singularity_message(ERROR, "Could not open image (read only) %s: %s\n", source, strerror(errno));
-            ABORT(255);
+            singularity_message(ERROR, "Trying with escalated privileges...\n");
+            singularity_priv_escalate();
+            if ( ( image_fp = fopen(source, "r") ) != NULL ) {
+                singularity_message(ERROR, "Now we succeeded :)\n");
+            } else {
+                singularity_message(ERROR, "No, we again failed :('\n");
+                singularity_priv_drop();
+                ABORT(255);
+            }
+            singularity_priv_drop();
         }
     }

----

Here's the output when I run the patched exec:

[ul_l_nsn25@adm02 ~]$ ls -lah vre2 && singularity exec -p vre2 ps aux  
-rwx------ 1 root root 1.1G Oct 27 14:32 vre2
ERROR  : Could not open image (read only) vre2: Permission denied
ERROR  : Trying with escalated privileges...
ERROR  : Now we succeeded :)
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
ul_l_ns+     1  0.0  0.0   7132   636 pts/0    R+   08:12   0:00 ps aux

If I apply this exec.exec falls back to escalated privileges to open the singularity container. That way root can be the owner and have exclusive read/write access to it.
As I understand any other command shouldn't have access to that container then. Is there anything to be considered security-wise?
Would be lovely to have such functionality merged into master.

thanks and greetings Stefan

[Gregory M. Kurtzer]

Hi Stefan,

This will end up circumventing standard POSIX permissions ACLs on containers, so I don't like the implementation. But, I do like the feature idea!

So the question in my mind though is how to we do this in a configurable manner? Perhaps we do it by group ownership? Or perhaps we have a different configuration file for special containers and in each configuration file, the system admin can define things like the path of the file, and if they match, then allow Singularity to open it as you defined...? <just brainstorming outloud here>...

Thoughts?

To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

[Stefan Kombrink]

This was just a hack to check if the idea works in principle, and I understand that such a function needs to be configurable in the cfg file.
I'd prefer a global option, where I don't touch /etc/singularity.conf whenever I add a container to the system.

I'll be offline for the next week.
After I return we can discuss and agree on the best way to implement this.
I can do a pull request and try on my own :)

[Brian Bockelman]

A few thoughts:

- it'd be straightforward to setup a group at install time and allow the runtime to execute containers owned by that GID.  Then, the sysadmin could allow any logged in user to execute - but not read - the container.

- things get significantly trickier if you want to allow only a subset of users in a certain container. I suppose the image itself could declare the allowed users/groups - but then you have a likely layering problem (image shouldn't need knowledge about the host system) and might not be an effective protection inside a user namespace.

- we likely need to have some level of access control for users in general.  I forget - do we already limit to users with an enabled login?  I'd hate to see user nobody and httpd be able to start containers by default.

  - most platforms have an idea of a system daemon UID range (historically 1-100, but I think 1-500 and 1-1000 are more modern values for this).  It might be reasonable and simple to have a minimum UID configuration value for starting containers.

- handling key matter and doing any sort of encryption gets nasty and complicated awfully quickly. I'd rather rely on POSIX mechanisms.

  - best save our energy in cryptography to the simple stuff, like signing releases!

HTH,

Brian

Sent from my iPhone

[Gregory M. Kurtzer]

How about.... If the user does not have read access to the container, but the container is owned by root and in a group which is not root and that user is also in that group, and the singularity.conf has enabled this feature, then allow the user to open the container.

I am honestly concerned about this feature (overriding standard POSIX file system ACLs), but as long as we default this feature to off in the configuration file and document it appropriately... ok.

[Stefan Kombrink]

Hi Greg & Brian,

 I am back :) The way proposed by Greg sounds reasonable to me.
I will explain the intended behavior to my colleagues and try to implement it.

Greets Stefan