New "Invariant" E2E Tests

[Benjamin Elder]

Hi all, SIG Testing have implemented https://kep.k8s.io/5514 in https://github.com/kubernetes/kubernetes/pull/133394, this introduces special new e2e test results which validate some component metrics *after* the rest of the e2e suite. These checks involve a read-only fetch of apiserver metrics which should be quick and reliable.

This allows us to confirm that no declarative validation mismatches versus legacy validation or declarative validation panics occurred across the entire e2e suite in most e2e CI jobs. This extensive coverage is necessary in addition to unit tests, fuzzing, and integration tests because declarative validation will eventually touch all APIs and we need to ensure no regressions across all API fields.

We expect that there will be interest in adapting this functionality to more use cases, please reach out to the SIG Testing Leads to discuss, per the KEP we will review additional usage to ensure it meets the guidelines designed to ensure reliability laid out in the KEP.

If these new checks do fail for any reason, the failure message will contain standardized instructions about how to handle them, along with the required registered SIG and OWNERS to route bugs to, please do file a bug and assign the invariant owners + label the SIG from the failure message if you see failures like "[ReportAfterSuite] Invariant Metrics".

Additionally, for some time I will be watching for any issues with the framework and you can also CC me on any bugs, but I do not expect any, this has been extensively tested after lengthy discussions.

Thanks,
- Ben

[Antonio Ojea]

Thank you Ben for driving this, guaranteeing that certain parts of the
system are immutable and just don't change is a big win for
everyone, and testing for this kind of things is very challenging.

I like to think about a failure in these invariants as being like a go
panic() for a distributed system ... it's something that simply cannot
happen, and now we have an automated way to enforce it.

Awesome work!

On Fri, 3 Oct 2025 at 01:24, 'Benjamin Elder' via
kubernetes-sig-testing <kubernetes-...@googlegroups.com>
wrote:

> --
> You received this message because you are subscribed to the Google Groups "kubernetes-sig-testing" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-te...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-testing/CAOZRXm_B%3Daebj-7dczZ%3DeTgE5aGBoOGO8Xb7iLfFN8p8vbvgzg%40mail.gmail.com.

[Tim Hockin]

Thanks for following up on this idea - I think it will allow us to have a degree of confidence that we couldn't have before.

You received this message because you are subscribed to the Google Groups "dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev+uns...@kubernetes.io.
To view this discussion visit https://groups.google.com/a/kubernetes.io/d/msgid/dev/CABhP%3DtaLFG5jM-P4%3D79k%3D6cQbT%2BwWc0r-jnfBohZ%3DRZcW0KUxQ%40mail.gmail.com.