Ginkgo test filtering by labels

114 views
Skip to first unread message

Patrick Ohly

unread,
Jun 20, 2024, 3:00:52 AM (13 days ago) Jun 20
to d...@kubernetes.io, ons...@gmail.com
Hello!

For a while now all E2E tests in Kubernetes have been labeled (see "spec
labels" in https://onsi.github.io/ginkgo/#filtering-specs) in addition
to getting the traditional inline tags (like `[Slow]`).

After Onsi's recent enhancement of the `--label-filter` (thanks!)
and some related Kubernetes PRs we are now able to replace the
complicated regexp-based focus/skip combination for selecting tests with
a single --label-filter parameter that doesn't need regexps.

The old way of selecting tests through embedded tags in their name
continues to be supported, probably in perpetuity. But labels are
nicer. For example, https://github.com/kubernetes/test-infra/pull/32648
replaced

--focus="\[Serial\].*\[NodeFeature:SidecarContainers\]|\[NodeFeature:SidecarContainers\].*\[Serial\]" --skip="\[Flaky\]|\[Benchmark\]|\[NodeSpecialFeature:.+\]|\[NodeSpecialFeature\]|\[NodeFeature:Eviction\]"

with

--label-filter='(NodeConformance || !(NodeFeature: isEmpty)) && !Flaky && !Slow && !Serial'

This works on Kubernetes master which has the latest Ginkgo. Prow jobs
for older releases have to continue using the old approach.

The goal is to convert all Prow jobs to the new approach. Help with that
is welcome. If you are touching a job for some other reason, please
consider switching at the same time.

Once all jobs have been converted, tests that only depend on a feature
gate can be labeled with just `WithFeatureGate(<feature gate>)` which
automatically adds alpha/beta labels. Right now, they also need a
`WithFeature(<feature gate>)` to prevent running them in jobs which
still use the regexp-based skip mechanism.

For details, see
https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/e2e-tests.md#kinds-of-tests
and its recent update in
https://github.com/kubernetes/community/pull/7824.

--
Best Regards

Patrick Ohly
Cloud Software Architect
Reply all
Reply to author
Forward
0 new messages