CNI's running as root

[Jason Culligan]

Must all Container Networking Interface (CNI) deployments run as root or with elevated privilege?

[Casey Callendrello]

Hi there, Jason,

There are a few different privileges we're talking about here, so let's list a few of them and see what can be done about them.

CNI plugins generally need to be able to

- create and delete network interfaces, ip addresses, and routes

- tweak sysctls

- read files "pointing" to network namespaces

- enter those namespaces

- manage iptables rules.

In general, CNI plugins are executed as root, in the root mount namespace, with full capabilities. This is, as you observe, quite a lot of privilege. What might we be able to drop?

First of all, it might be pretty easy to run in a chroot / different mount namespace. As long as we have read access to "/run/netns (or /proc), write access to "/run/cni" and "/proc/sys/net", and the ability to find any chained plugins, then that would work fine. The container runtime would be responsible for setting this up.

Secondly, can we drop capabilities to something more restrictive? Currently, the answer is no. This is for two reasons:

1. iptables requires CAP_SYS_ADMIN

2. the setns() system call requires CAP_SYS_ADMIN.

Everything else mentioned just needs CAP_NET_ADMIN.

Some plugins don't need to execute iptables, but essentially every plugin needs the ability to switch between namespaces. So that's an unfortunate constraint. It seems like we won't be able to get away with dropping capabilities.

Lastly, can we run as non-root? Well, theoretically yes: we could run as a non-root user, but with CAP_SYS_ADMIN anyways. Now, CAP_SYS_ADMIN is effectively running as root, but it does require jumping through some hoops to gain full control of the system. So, doing this would protect against accidental damage (e.g. rm -rf /), but wouldn't stop a specific attack.

What about dropping unnecessary privileges in the binary itself? Well, that could definitely work! We could fork off a process in the child namespace *then* drop ourselves to CAP_NET_ADMIN. We might even want to switch to a non-root user. All of these would add a bit of complexity but add security. Unfortunately, golang doesn't make doing these sorts of things easy (or even possible, really).

I would love to see some work done on the runtime side to execute CNI plugins in a separate mount namespace. Care would have to be taken to prevent namespace escape (trivial if you have access root /proc), but it would certainly help prevent a rogue cni plugin from doing too much damage.

Cheers,

-- Casey Callendrello

On Fri, Sep 3, 2021 at 12:05 PM Jason Culligan <jasonc...@gmail.com> wrote:

Must all Container Networking Interface (CNI) deployments run as root or with elevated privilege?

--
You received this message because you are subscribed to the Google Groups "cni-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cni-dev+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cni-dev/5a96cb80-aafe-42dc-8c1a-e4b34d0f3099n%40googlegroups.com.