Best practices for cluster permissions defined in a CSV

[Bob Tinsman]

I have been working as a senior OpenShift consultant for several months at a large customer where security is a high priority. I've installed a third-party operator-based stack onto various clusters for testing. It's a pretty complex product which includes a couple dozen operators.

The following may be review for most.

When an operator gets installed, the OLM will create service accounts, roles, and role bindings, which determine the cluster privileges that the operator pods will have, including:

• What type of resources the pod can work with
• What operations it can perform with each type of resource (list, get, create, update, delete, etc.)
  
• Where it can do these operations (within a single namespace, or cluster-wide)

This is all specified in the CSV (cluster service version).

To get to my point--I observed that many of the CSV's for this product created cluster roles which essentially allow the operators to do anything, because they include the ability to create cluster role bindings and update cluster roles, among other things.

If you can create a cluster role binding, then you can make anybody cluster admin.

It wasn't just a single service account that had this capability--I counted 10 or 15.

Now I understand that there are legitimate uses for this capability, but a developer of operators should be careful about handing it out; otherwise this is a violation of the principle of least privilege--in other words, code should be running only with the access that it requires, and no more.

There are a couple security holes this could open up; maybe you can think of more:

• A bad actor who is aware that pods are running with these privileges can look for security holes that inject code into that pod
• A careless programmer writing code to update cluster roles (say, to add access to new CR types for existing roles) could introduce a bug that removes access necessary for code to run, or adds excessive access

Questions:

• Am I off-base in my concerns?
• What are some best practices for developers of operators?
• What are some best practices for cluster admins installing operator-based software?

I have been googling a lot without finding very much; thanks for any help!

Bob Tinsman

Senior OpenShift Consultant