Gateway API Conformance Profiles

[Shane Utt]

Hi all,

In the SIG Network Gateway API sub-project we have a feature called Conformance Profiles we're actively developing. Conformance Profiles add high-level configuration options for running our conformance test suite on downstream implementations and producing reports which can be submitted back to upstream as part of a "certification process".

For more details we have a Gateway Enhancement Proposal (GEP) for this:

https://gateway-api.sigs.k8s.io/geps/gep-1709/

(Note that this is currently in a provisional state as we continue to iterate on it and its final form could end up being significantly different as we collect more feedback)

Riaan Kleinhans suggested sharing with this mailing list to get feedback and put it on people's radar. We're very interested to hear thoughts and ideas about this. We would love feedback in the form of PRs to update the GEP with the tweaks, changes and additions that you would like to propose. We also will be bringing this to the agenda for one of the upcoming SIG Arch meetings to discuss it.

Thank you,

Shane

[Antonio Ojea]

Not directly related but I want to use this opportunity to raise a problem we are having in Sig-network due to the lack of consistency on implementations, but we can't promote the test to Conformance since the features are not required in all clusters, per example, Network Policies, LoadBalancers, Ingress, ...

I can see how some areas like sig-network or sig-node can benefit of having some sort of Conformance mechanism like this to validate specific features or functionalities that doesn't fit under the current Conformance umbrella

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-architecture" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-arch...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/f4d7bea1-5eed-43a2-9645-e5f50ee9bb3an%40googlegroups.com.

[jay vyas]

Thanks for bringing this up shane and antonio !!!!!!

 theres a whole lot of "granular production readiness/ conformance" questions floating around across all the SIGs nowadays..... it seems like moving to a higher level tool then e2e.test for giving people a report, as opposed to a pass/fail signal, is really what we want now....... 

ONE POSSIBLE SOLUTION:  in sig-windows, we aggregated sets of tags into the windows operational readiness specification to address this... https://github.com/kubernetes-sigs/windows-operational-readiness.   That way there could be a sense of more granular conformance then just the whole " guesbook works and so do EmptyDirs and APIs...." which we currently have.

- In sig node you run into things where certain container runtimes might do different things wrt things like CAdvisor, kubelet statistics, and even have different behaviours regarding the ability to support realtime CPU isolation and so on... and it would be nice if there were higher level ways to measure a node interms of its conformance to a particular specification i.e. supports realtime or supports GPUs or supports .... 

- in sig net we have: L7, networkpolicy, ingress/gateway - - - all totally pluggable and totally different across providers... 

- and  only a handfull  (6 or so? maybe more ? )  of the 300+ conformance tests (last i checked) rely specifically on kube-proxy functionality.... but we have many, many tests (like, functioning node port services, mutating services from one type to another, and so on...) which define a "Conformant" kube proxy impl.

 

Any other ideas on how we should solve this ?  I wonder if its possible to get Sigs to agree on a broad definition for granular conformance somehow ........

[Patrick Ohly]

jay vyas <jayunit1...@gmail.com> writes:
> Any other ideas on how we should solve this ? I wonder if its possible to
> get Sigs to agree on a broad definition for granular conformance
> somehow

At the technical level, Ginkgo test labels could be used to describe
which tests must pass for certain levels of conformance or specific
aspects of it. This can co-exist with the current approach of putting
special tags into the test names. See
https://github.com/kubernetes/kubernetes/pull/112894 for a
proof-of-concept. New categories then should only use labels.

But that's just the technical side of this. The bigger question is
around the definition of these conformance profiles.

--
Best Regards

Patrick Ohly
Cloud Software Architect