Are missing conformence tests for endpoints release blocking

[Mario Fahlandt]

Hello SIG Arch & SIG Release

after raising the topic at the SIG Release meeting at Tuesday 11th of February it was decided to open up a joined discussion between the SIGs.

@koksay @xmudrii and @mfahlandt took over the APIsnoop project starting january 1st 2025 and it came to our attention three endpoints made it in the Kubernetes 1.32 release without having tests: 
patchCoreV1NamespacedPodResize 1.32.0 not tested
readCoreV1NamespacedPodResize 1.32.0 not tested
replaceCoreV1NamespacedPodResize 1.32.0 not tested

https://apisnoop.cncf.io/conformance-progress/endpoints/1.32.0/?filter=promoted-without-tests

For the upcoming release of 1.33 we have currently 17 Endpoints without tests.

One topic from our side is: Who need to be informed about such findings? The Endpoint Owners? SIG Arch? SIG Release? All of them? 

And the Discussion that was requested is: If / When / at which amount are missing Tests Release blocking?

Cheers
Mario 

[Davanum Srinivas]

Mario,

Can we have a sig-release blocking job that will prevent us from making a release with such endpoints?

If so, the signal from such a job would end up pinging the SIG(s) and the PR owners who landed/promoted the endpoints. right?

thanks,

Dims

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-release" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-re...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-release/CAPMVczEbfVMQxabtuUSbdx8opJ99M7pEO%2Bi5toX3-NsXAA_AZQ%40mail.gmail.com.

--

Davanum Srinivas :: https://twitter.com/dims

[Jordan Liggitt]

A post-submit job that notices missing conformance tests sounds like a good idea. I think this has been suggested in the past.

In the case of the 1.32 pod resize endpoints, those aren't actually GA, they're part of the still-alpha InPlacePodVerticalScaling feature, so shouldn't have conformance tests yet.

You received this message because you are subscribed to the Google Groups "kubernetes-sig-architecture" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-arch...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/CANw6fcEt%2BZJLg2bbJ8E%3DNxKqyq9ZXm51VWhKBbur8-HpMCozLQ%40mail.gmail.com.

[Antonio Ojea]

post-submit or periodic? periodic seems better and easier to monitor

On Fri, 14 Feb 2025 at 21:49, 'Jordan Liggitt' via
kubernetes-sig-release <kubernetes-...@googlegroups.com>
wrote:

> To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-release/CAMBP-pJmVuQaBxyx0Dhpq0RX08w_NA3GNZFw0rzxMqEqA%3DdoxQ%40mail.gmail.com.

[Mario Fahlandt]

Both would be possible, but I agree periodic might be better.

We can basically query this file for the current upcoming release: https://github.com/kubernetes-sigs/apisnoop/blob/main/resources/coverage/conformance-progress.json 
curl -s https://raw.githubusercontent.com/kubernetes-sigs/apisnoop/refs/heads/main/resources/coverage/conformance-progress.json | jq '.[] | select(.release == "1.33.0") | .total.still_untested' Regarding the note from Jordan, we will take a look into this why the 3 endpoints then have been listed as required for test.