Salt multi-tenancy

[Xin Liu]

Hi guys, 

I've been looking at salt deployment for multi-tenant scenario. My understanding is the same AES key used for all minions. Base on this one security concern comes up:

Suppose there are two users in the system, each having some minions. There is one master managing all minions. User A sends a command to his minions. User B is a malicious user and he gets hold of User A's packet. Given they have the same key, user B can decode the messages sent by user A. 

My question is is the above true in salt? Or I might have missed something. 

Regards,

Xin

[Mike Chesnut]

You can have different keys.  The way I do this is to use different config files for the various minions, with the various directory configs (starting with root_dir) set to different values for each.

You then add an option (-c) to the minion when starting it to point it to the directory where its particular config lives.

[Thomas S Hatch]

This is partially true, the publications are sent out with a single AES key because all minions need to decrypt them to determine if they need to execute the command. Single minion operations do require independent authentication and session management though, which is why the minion re-auth for file server and pillar operations.

The question I think should be about your multi tenant setup, are your tenants accessing the same systems with different right? Are you using client_acl or eauth? If you are in an environment like a hosting service with separate customers owning separate vms then each customer would need a single master, and then bind them together with a syndic if you want to be able to override the customer systems.

[Xin Liu]

Thanks Thomas. 

I am setting up a salt service for multiple tenants. Each tenant is able to add his own minions in the service. Each tenant should have complete rights over his own minions, but should have no rights over others' minions. I'm not using client_acl or eauth. I'm building my own authentication/authorization system, and then use salt client API to talk to minions. In summary, the service requires:

1. Multiple tenants coexist in the same system

2. Each tenant having different set of minions

3. Tenant A cannot access Tenant B's minions

4. Tenant A and Tenant B use different AES keys

If I understand correctly, you are suggesting that each tenant should have a different master. Could you explain more on binding them together with a syndic? I thought only syndics can be binded with a master, not the other way around. If I need to start a new master for each different tenant, then it doesn't seem to be scalable, considering we could have tens of thousands of tenants. 

Could you please provide some suggestions on this problem, the correct way to resolve it and where I should hack into salt to make it work? I am thinking a two-layer AES encoding approach. Each tenant should have another AES key, encoding the entire command except the minion id. Outside the message is encoded using the original AES key. Upon receiving the message, minion should decode the message with the original AES key to reveal the minion id. Once the minion id is known and matches the minion's own id, it will go ahead to decode the message with the second AES key. 

Thanks,

Xin

[Xin Liu]

Thanks Mike. 

I guess I don't fully understand. Are you suggesting one master could have multiple config files, each config file for different tenant? If this is the case, how do I start the master process? Could you please elaborate a bit more?

Thanks,

Xin

[Ryan Schneider]

Multi-tenacy sounds very dangerous.  Even if you get the AES encryption right. don't forget Salt is built on zeromq, and all those tenants would be sharing a single master socket.  Zeromq hasn't been around long enough to get a lot of security exploits shaken out of it, if the PUB/SUB sockets end up having an exploit in them you've potentially allowed one tenant to compromise all the others.

I'd say investing your time in automating the infrastructure for spinning up new per-tenant masters is a better long-term use of your time.  Perhaps even use salt to spin up new masters in a chroot (a chroot module would be awesome!), or use salty-vagrant or salt-cloud once vagrant or one of the other frameworks has LXC support.

Ryan

[Xin Liu]

Thanks Ryan. 

Yes ideally it should be one master per tenant setup for multi-tenant scenario. We are exploring sensible ways to solve the issue. Will share when we finalize :--)

Regards,

Xin

[Thomas S Hatch]

Thanks for your help here Ryan

Yes, we are looking at a multi tennant system right now and leaning towards using chroots for the masters, I think that total isolation is the best way here. As for tying them together, take a look at the syndic docs:

http://docs.saltstack.org/en/latest/ref/syndic.html

[Ryan Schneider]

I could see two very useful general purpose modules/states come out of this:

- A chroot state for creating and managing new chroots (much like virtualenv.managed)

- A module for creating new masters as syndic minions and accepting their keys (generalized to a module for creating new minions and managing their keys)

Thanks,

Ryan

[Jeff Schroeder]

On Monday, December 3, 2012 3:46:12 PM UTC-6, Ryan Schneider wrote:

I could see two very useful general purpose modules/states come out of this:

- A chroot state for creating and managing new chroots (much like virtualenv.managed)

- A module for creating new masters as syndic minions and accepting their keys (generalized to a module for creating new minions and managing their keys)

Thanks,

Ryan

Or perhaps wrap this with lxc (which is easily managed via libvirt) and call it a day :)

much better than chroots.
 

[Benjamin Coetzer]

Sorry to bump up an old thread

Did you find a way to accomplish this multi-tenancy? Using syndic? I would love to hear how you accomplished it.

[Pedro Paulo Vezzá Campos]

Hello everyone,

I, too, am interested in finding how can we achieve multi-tenancy using saltstack. Elaborating a bit on Jeff Schroeder's idea, spinning one salt-master for each tenant in a Docker container would solve this problem except for the excessive memory consumption, right?

Best,

Pedro Paulo

[Alex Stewart]

Hello all,

We're investigating salt for use in a multi-tenant environment. While we agree usage of an LXC container or docker could be a viable solution, we're investigating the possibility of a per minion AES key. I wanted to reach out and see what the general thought/feeling was around this idea? To be more specific are proposed solution is:

1) Enable zmq_filtering:

- This will allow commands to be sent only to minions that are the intended recipients  (viable for only things that can be matched master side: i.e. minion-id, GLOB, PCRE only)

2) A Per Minion AES key. 

- Even with zmq_filtering, it is possible to fake out the routing minion side. So a compromised box could still gather data from other targets (based off minion-id). Idea here is to minimize that route by encrypting per minion.

Thank you in advance,

-Alex 

[V42]

Might not be relevant, but since you mentioned Docker for multi-tenancy, I'm gonna share this anyway. Don't use Docker for multi-tenancy, especially if there private data/keys involved. Root access in a Docker container === root access on the host and thus possible root access to all other tenant containers. While Docker is great for many things, multi-tenancy is NOT one of them. Hopefully, this Feb update should fix it, as it'll bring user namespaces.

--
You received this message because you are subscribed to the Google Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alex Stewart]

Good point. Thanks for the note. We hadn't looked to close at docker, for our uses a fleet of lxc containers was what things were going to look like. Problem is the over head. I personally think in the case someone enables zmq_filtering, I could add a sub option --> if you have zmq_filtering on you essentially agree to a contract that you can only match minion targets master side. This means you can enable a sub option for per minion AES keys since master is/should be sending a payload to each minion one by one.  

At least that's the idea. 

Cheers and best regards,

-Alex

You received this message because you are subscribed to a topic in the Google Groups "Salt-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/salt-users/hu9EugfpM-E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to salt-users+...@googlegroups.com.