KEP Proposal: Hermetic Pods

[Antonio Ojea]

Hi everyone,

We all know the pain of trying to fit multi-network setups and complex service meshes into the standard Kubernetes network model. With the recent rise of agentic workloads and custom schedulers, this is getting even more complicated. These new workloads demand absolute network isolation, often building their own proxies to guarantee zero traffic leaks into the cluster network.

To help unblock this, I've drafted a KEP proposing "Hermetic Pods" (PodSpec.Hermetic = true). 

The idea is simple: Kubelet tells the container runtime to create the pod's network namespace, but completely skips the CNI plumbing step. The pod just gets a loopback interface and nothing else.

This way agentic workloads and Service Meshes get a completely empty, air gapped environment to build their own "network" without fighting or leaking into the default network.

We solve the multi-network/tenant problem cleanly without adding complex APIs to K8s core. Using the new DRA Resource Claim Status (KEP-4817), out-of-tree controllers can inject custom interfaces right into the sandbox and report the IPs back.

I've chatted with a few of you already and this feels like a very pragmatic path forward, the prototype looks relatively simple https://github.com/kubernetes/kubernetes/pull/141604. I'd love to get your option on it to see if anyone sees any red flags.

Please comment on the doc attached and if there are no strong objections, I'd really like to target this for the upcoming 1.38 cycle.

 Kubernetes Hermetic Pods

Thanks,
Antonio

[Benjamin Elder]

I like it, I think we should also include the use case: "I have a daemonset that does not need to use the network because its job is to configure the host" (e.g. imagine a GPU driver installer with driver contents self-contained to the container image).

Today those pods must either use hostNetwork or consume pod IPs.

While I don't want to increase the scope of this proposal, it feels like another reminder that the pods-per-node knob is rather blunt (Perhaps another thread for that topic ...)

- Ben

--
You received this message because you are subscribed to the Google Groups "sig-node" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sig-node+u...@kubernetes.io.
To view this discussion visit https://groups.google.com/a/kubernetes.io/d/msgid/sig-node/CABhP%3DtaFQEtdFXR_017qPy3GQ-1vMQfwQJpOCLcFDPLAG%2B8MfA%40mail.gmail.com.

[Michael Brown]

Nod we've talked about this for sometime now.  SGTM.. A new netns mode set lo up only.. which may or may not be via a cni plugin/container runtime built in func. 

[Michael Zappa]

@mikebrown we talked about this years and years ago! It's about time a formal proposal has been made. 

--
You received this message because you are subscribed to the Google Groups "sig-network" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sig-network...@kubernetes.io.
To view this discussion visit https://groups.google.com/a/kubernetes.io/d/msgid/sig-network/e72b24b2-b91c-4ea1-892f-4b9cb7534ca4n%40kubernetes.io.

[Whoro Ochuko]

Hi all, I read through this and I think it's a nice proposal.

I have a question 

When `Hermetic=true` causes the normal CNI setup to be skipped, what is the expected lifecycle of the resulting network namespace? Does it simply remain with only a `loopback interface` until an external networking provider attaches additional interfaces, or is there an intended mechanism in the PodSpec (e.g. a resource claim/provider reference) for specifying which component should configure the network if needed?

[Michael Zappa]

@Whoro that is in the POC at the bottom of the document in the
containerd code base.

On Wed, Aug 26, 2026 at 3:26 PM Whoro Ochuko <whoro...@gmail.com> wrote:
>
> Hi all, I read through this and I think it's a nice proposal.
>
> I have a question
>
> When `Hermetic=true` causes the normal CNI setup to be skipped, what is the expected lifecycle of the resulting network namespace? Does it simply remain with only a `loopback interface` until an external networking provider attaches additional interfaces, or is there an intended mechanism in the PodSpec (e.g. a resource claim/provider reference) for specifying which component should configure the network if needed?
>
>
> On Wednesday, August 26, 2026 at 9:33:58 PM UTC+1 Michael Zappa wrote:
>>
>> @mikebrown we talked about this years and years ago! It's about time a formal proposal has been made.
>>
>> On Wed, Aug 26, 2026 at 2:21 PM Michael Brown <mike...@gmail.com> wrote:
>>>
>>> Nod we've talked about this for sometime now. SGTM.. A new netns mode set lo up only.. which may or may not be via a cni plugin/container runtime built in func.
>>>
>>> On Wednesday, August 26, 2026 at 2:18:34 PM UTC-5 Antonio Ojea wrote:
>>>>
>>>> Hi everyone,
>>>>
>>>> We all know the pain of trying to fit multi-network setups and complex service meshes into the standard Kubernetes network model. With the recent rise of agentic workloads and custom schedulers, this is getting even more complicated. These new workloads demand absolute network isolation, often building their own proxies to guarantee zero traffic leaks into the cluster network.
>>>>
>>>> To help unblock this, I've drafted a KEP proposing "Hermetic Pods" (PodSpec.Hermetic = true).
>>>>
>>>> The idea is simple: Kubelet tells the container runtime to create the pod's network namespace, but completely skips the CNI plumbing step. The pod just gets a loopback interface and nothing else.
>>>>
>>>> This way agentic workloads and Service Meshes get a completely empty, air gapped environment to build their own "network" without fighting or leaking into the default network.
>>>>
>>>> We solve the multi-network/tenant problem cleanly without adding complex APIs to K8s core. Using the new DRA Resource Claim Status (KEP-4817), out-of-tree controllers can inject custom interfaces right into the sandbox and report the IPs back.
>>>>
>>>> I've chatted with a few of you already and this feels like a very pragmatic path forward, the prototype looks relatively simple https://github.com/kubernetes/kubernetes/pull/141604. I'd love to get your option on it to see if anyone sees any red flags.
>>>>
>>>> Please comment on the doc attached and if there are no strong objections, I'd really like to target this for the upcoming 1.38 cycle.
>>>>

[Antonio Ojea]

The  proposal is that the Pod has only an isolated network namespace , period, that is the Kubernetes guarantee .... any possible extension on that model is outside of kubernetes. Commonly (most of the time) people want to add some connectivity, but there are examples where you may want to run something that does some processing and outputs something to a file, when you do not require connectivity (although is very uncommon), docker --network none has been a thing for a long time

[Antonio Ojea]

Hi,

after presenting the idea on SIG Network and Node meetings and getting positive feedback from the community I filed the KEP incorporating some of the early feedback https://github.com/kubernetes/enhancements/pull/6317

Thanks all