[idea] crazytalk to move windows proxy out-of-tree?

[jay vyas]

Hi folks.   

Overall the bifurcation of the windows and linux intree proxys seems unnatural ... I think it served an initial purpose to get core windows support into kubernetes, but, with networking becoming increasingly sophisticated, maybe its time to build a faster moving windows proxy that can be 100% maintained by the sig-windows community (avoiding the deprecation policy and other well known guardrails debates here... just want to ask - is this something people want )? 

If so , here are a few tidbits that will move us in that direction , that we can start on.

 So, thus far it hasnt been obvious about how windows specific kube proxy features and knobs can be added.  Examples:

- enable-dsr command line option https://github.com/kubernetes-sigs/windows-testing/issues/206 

- prioritization of kube proxy niceness in windows processes https://github.com/kubernetes/kubernetes/issues/96935 

- where would we put an ebpf implementation ? 

- how could we test the kube proxy on windows in isolation, without requiring a full apiserver (theres tons of MSFT networking expertise we could pull on, if we had a simpler developer experience for the kube proxy).

- Alt windows proxies (like antrea OVS proxy) dont have an obvious extensibility model in the community, either. https://github.com/antrea-io/antrea/blob/main/docs/design/windows-design.md#service-traffic 

As folks, we have a generic solution to these problems,  the KPNG project,  and several prototypes that hinting at a much more modular future.  Maybe as a sig we could spearhead this with a kube-proxy-windows out of tree implementation, that plugs into  KPNG as a plugin ?  

Some examples of the types of things that are emerging in the community: 

https://github.com/jsturtevant/win-ebpf-demo-go 

https://github.com/kubernetes-sigs/kpng 

https://github.com/kubernetes-sigs/kpng/tree/master/backends/ebpf 

https://jayunit100.blogspot.com/2022/03/my-dev-workflow-for-kpng-windows.html 

 

So , now whats next ?  One possibility, is end to end ownership of the windows proxy impl, ... 

- A windows specific development experience 

- Windows specific command line arguments where applicable

- Windows owners for the end to end experience 

This would 

- reduce the kube proxy tech debt in tree.

- make it easier to rapidly debug and triage incoming kube proxy windows bugs

- avoid us having to resolve command line option bifurcation (i.e. like what we have to do now ...

type serviceInfo struct {
    *proxy.BaseServiceInfo
    targetPort             int
    externalIPs            []*externalIPInfo
    loadBalancerIngressIPs []*loadBalancerIngressInfo
    hnsID                  string
    nodePorthnsID          string
    policyApplied          bool
    remoteEndpoint         *endpointsInfo
    hns                    HostNetworkService
    preserveDIP            bool
    localTrafficDSR        bool
}

So that we could have windows specific data model end to end for K8s service proxying.   This also could liberate us to build an ebpf proxy that is maintained. 

If folks at MSFT would be interested in investing in this, we could propose a kubernetes-sigs/kpng-windows repository today and I could help get the initial vendoring and librarying etc in place.  

[jay vyas]

(alternative, which might evolve here, is kubernetes-sigs/kube-proxy-lib, where we'd be first adopters, that conversation is evolving here : https://groups.google.com/g/kubernetes-sig-network/c/IxPZlOPj3EA/m/rO-FaWFRBQAJ ) 

[jay vyas]

FWIW, we concluded we're going to give it a shot ! https://github.com/kubernetes-sigs/windows-service-proxy