Idea of NetworkClass

[Sascha Spreitzer]

Hi there

I just wanted ask in this SIG whether an addition or replacement of CNI was ever considered that goes into the same flexibility/plugability as StorageClass?

For example:

* NetworkClass
* NetworkInterface

Pods referencing one or more NetworkClass will cause network-controllers to produce and bind NetworkInterface's to a Pod?

After a CNI implementation replacement (calico to cilium) this question came to my mind.

Kind regards
Sascha

[Antonio Ojea]

We are going to do that with DRA, see this presentation that provides
a bit of historical context
https://docs.google.com/presentation/d/1Vdr7BhbYXeWjwmLjGmqnUkvJr_eOUdU0x-JxfXWxUT8/edit#slide=id.g2f0d619180b_0_17

> After a CNI implementation replacement (calico to cilium) this question came to my mind.

We should stop building network monoliths to control "all network"
related things in kubernetes
That is the idea behind Kubernetes Network Drivers and projects like
kube-proxy and https://github.com/kubernetes-sigs/kube-network-policies,
make the network composable

> --
> You received this message because you are subscribed to the Google Groups "kubernetes-sig-network" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-ne...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-network/bfdbb48b-5720-41e5-a304-71d101711643n%40googlegroups.com.

[Sascha Spreitzer]

Thank you very much. DRA looks very promising. Looking forward to use it.

[Antonio Ojea]

For network interfaces there is already a working implementation in https://github.com/google/dranet

On Wed, Mar 5, 2025 at 12:50 PM Sascha Spreitzer <sas...@spreitzer.ch> wrote:

Thank you very much. DRA looks very promising. Looking forward to use it.

--

You received this message because you are subscribed to the Google Groups "kubernetes-sig-network" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-ne...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-network/ab1f0c77-1897-4fa6-9612-d9e5852f7f4bn%40googlegroups.com.

[Lionel Jouin]

We are also currently trying to explore this area keeping the CNI API in the picture: https://github.com/kubernetes-sigs/cni-dra-driver

Some PoCs have been achieved previously (Mentioned in the slides Antonio sent):

https://github.com/LionelJouin/network-dra

https://github.com/aojea/kubernetes-network-driver

---

From: 'Antonio Ojea' via kubernetes-sig-network <kubernetes-...@googlegroups.com>
Sent: Wednesday, March 5, 2025 1:20 PM
To: Sascha Spreitzer <sas...@spreitzer.ch>
Cc: kubernetes-sig-network <kubernetes-...@googlegroups.com>
Subject: Re: [k8s-sig-net] Idea of NetworkClass

 

To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-network/CAAdXToRURDsZ2Q-EqmatnboYG2_h26-U2kKJTYEVQXKgwcXVSw%40mail.gmail.com.

[Shane Utt]

This idea has been discussed before. We considered NetworkClass (with that same name) as part of the “Kubernetes Networking reImagined (KNI)” effort that started in late 2023:

https://github.com/kubernetes/enhancements/issues/4410

Note: KNI was paused during its draft phase when DRA for networking became a higher priority for some SIG Network members. If revisited, it would need major updates.

Personally, I have wanted networking to have discrete, composable APIs since the earliest days of Kubernetes. This has proven very difficult to prioritize and build consensus for over the last decade.

If you're very eager to push this forward, I would highly recommend joining and participating in the CNI DRA Driver experiment.

To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-network/AM7P189MB09320A481BDD0D2E3EF68B88F8CB2%40AM7P189MB0932.EURP189.PROD.OUTLOOK.COM.

[Bowei Du]

Hi Sascha -- 

I am curious on more details of your use case if you don't mind sharing.

Are you looking for a way to migrate between CNI providers? Trying to access more than one interface/network etc?

Thanks,

Bowei

To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-network/CAHEVDqPe_mNyLjZNFKOLv9djVrkEEXHj6zYVegQ-OgaEvLBKbQ%40mail.gmail.com.

[Sascha Spreitzer]

Hi Bowei

In two different companies I made the following experiences:

1. company: 

Choosing a CNI to cover a flexible multi-cloud scenario. VMware vShpere & OpenShift on-premises, Azure Kubernetes Service in the public cloud.

The discussions around architecture where quite cumbersome, as the CNI implementations are not composable.

And the CNIs in question are often incompatible to each other: Antrea, Multus, Cilium.

A flexible composable networking stack would have been very helpful. In comparison to the flexibility that is offered by the StorageClass concept.

The hypothesis: Using one control-plane to control nodes which reside on-premises next to nodes that reside in public clouds, using the region and zone node labels to control workload distribution and probably additionally storage and networking.

 

2. company:

Forceful change of CNI technology by the enterprise Kubernetes vendor. Resulting in downtime of mission critical applications for that company. This was given because of the use of specifics of the CNI (EgressIP source identification on hardware firewalls).

Very unfortunate the solution is marketed as self-healing and highly available when it has to undergo a service downtime to replace the CNI.

The hypothesis: A rolling upgrade of the network stack without service downtime.

I hope in future Kubernetes networking will be as composable as storage.

Thank you and kind regards

Sascha

[Antonio Ojea]

Hey Sascha,

Your networking experiences really confirm our observations and
highlight the bigger pain point for all users in this space.
Let me send you a more recent slide deck that I think explains this
better, is based on the previous one so some slides may be similar:

https://docs.google.com/presentation/d/16eT_EYVbm75UvqKVg8L55VtRuGJ1_dr463OpbrRN2gg/edit#slide=id.p

Basically, we're working on fixing two key things to make Kubernetes
networking more flexible: the high-level and low-level APIs.

For the high-level, we're using DRA to make it easy to say "Pod,
connect to this network," treating the connection like a device.

The low-level is trickier, and is practically a consequence of CNI's
chaining architecture causing a bottleneck.

I was talking with the Node folks about a different problem, when I
found NRI, which started like CNI , execing json
https://github.com/containerd/nri?tab=readme-ov-file#background, but
moved to a plugin model because of the same problems we want to solve.

NRI maintainers were happy to add networking stuff to it, so we can
read now PodIPs and attach interfaces
(https://github.com/containerd/nri/pulls?q=is%3Apr+author%3Aaojea+is%3Aclosed)
with NRI,

This new interface with the runtime lets us break down networking into
smaller parts, so we don't have to do everything in one long chain. I
expand a bit more in this PR:
https://github.com/containerd/nri/pull/119 on the immediate benefits.

These lines of work fixes the composability and upgrade issues you
mentioned, and should make networking way more flexible

> --
> You received this message because you are subscribed to the Google Groups "kubernetes-sig-network" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-ne...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-network/5D4E1C64-E461-4E59-BD7C-9DE3A87B09F6%40spreitzer.ch.