When: Weekly on Wed, 9:45 – 10:15am
Notes: KubeVirt CI SIG meeting notes
Attendees: Dylan White, fossedihelm
Reminders:
we will create GitHub issues for tracking
GitHub issues and PRs
should be marked with /sig ci and /kind flake if applicable
should be marked with the target sig
Topics:
[urgent]
review and merge the quarantine PRs in 2 working days without SIG lgtm.
fix pending for auto-quarantine (TODO add link)
previous action items
state of existing issues: https://github.com/kubevirt/kubevirt/issues?q=is%3Aissue+is%3Aopen+label%3Akind%2Fflake+sort%3Aupdated-asc+label%3Asig%2Fci
[non-urgent]
[fossedihelm] 1.36 virtiofs quarantined test fix: https://github.com/kubevirt/kubevirt/pull/18017
real bug caused by kubernetes
[dhiller] three fix PRs for project-infra pending
Look at flakes
three storage tests: https://storage.googleapis.com/kubevirt-prow/reports/most-flaky-tests/kubevirt/kubevirt/index.html
flake stats - create issues accordingly
overall ( ∑=1031, 100.00% )
periodic-kubevirt-e2e-k8s-1.34-sig-compute ( ∑=224, 21.73% )
HostDevices issue still persists on 1.34
periodic-kubevirt-e2e-test-S390X ( ∑=157, 15.23% )
overall improving tendency
periodic-kubevirt-e2e-k8s-1.36-sig-compute ( ∑=107, 10.38% )
pr for virtiofs fix pending (see above)
pull-kubevirt-e2e-k8s-1.36-sig-compute ( ∑=82, 7.95% )
nothing obvious, looks like usual flake rate
periodic-kubevirt-e2e-k8s-1.35-sig-compute ( ∑=64, 6.21% )
nothing obvious, looks like usual flake rate
periodic-kubevirt-e2e-k8s-1.36-sig-storage ( ∑=53, 5.14% )
HotPlug volumes improving (fix went in https://github.com/kubevirt/kubevirt/pull/17825)
pull-kubevirt-e2e-k8s-1.36-sig-operator ( ∑=45, 4.36% )
pull-kubevirt-e2e-k8s-1.36-sig-storage ( ∑=37, 3.59% )
periodic-kubevirt-e2e-k8s-1.34-sig-storage ( ∑=33, 3.20% )
periodic-kubevirt-e2e-k8s-1.35-sig-storage ( ∑=30, 2.91% )
periodic-kubevirt-e2e-k8s-1.34-sig-monitoring ( ∑=29, 2.81% )
pull-kubevirt-e2e-k8s-1.35-sig-compute-migrations ( ∑=21, 2.04% )
periodic-kubevirt-e2e-k8s-1.35-sig-compute-migrations ( ∑=19, 1.84% )
pull-kubevirt-e2e-k8s-1.35-sig-operator ( ∑=15, 1.45% )
pull-kubevirt-e2e-k8s-1.35-sig-compute-serial ( ∑=11, 1.07% )
Last updated: 2026-06-03 07:42:25.778676256 +0000 UTC m=+15.106322934
Look at held tests:
dequarantine tests:
look at list of quarantined tests
Count: 14 tests in quarantine currently
check status, i.e. who is working on those
look at PRs that want to fix flakes
see whether we can dequarantine tests
misc topics
[fossedihelm] updating to go1.26: https://github.com/kubevirt/kubevirt/pull/17972
Does anyone want to step in?
briancarey commented: https://github.com/kubevirt/kubevirt/pull/17972#pullrequestreview-4400329227
Dylan to take a look
reference document for upgrading go: https://github.com/kubevirt/kubevirt/blob/main/docs/update-go-version.md
missing in the doc is upgrading of the toolchain, opportunity to add to the doc
1.26 is important precondition because we need to upgrade k8s deps which is built with 1.26, dra people i.e. want this
[nirdothan] let’s discuss the flaking tests lane being required https://redhat-internal.slack.com/archives/C01EX3K1FGE/p1780316410360699
[dhiller] if a test is not fit for the check-tests-for-flakes lane it can receive the no-flake-check decorator
// NoFlakeCheck decorates tests that are not compatible with the check-tests-for-flakes test lane.
// This should only be used for legitimate purposes, like on tests that have a flake-checker-friendly clone.
solutions
configure the lane to have the infra/reqs to make the test succeed
we also could create specific lanes that have the infrastructure so that those tests can be supported, i.e. specific sig-network flake-check lanes
suggestion: put the decorator in as a band aid in order to signal that the test is not supported yet
to be clear: that decorator should not be put on tests that are really flaky!
when applying the decorator we have to add the reason why it was applied, suggested to have a separate commit with an explaining commit message
we should make `make lint` fail if there’s no separate commit for applying that decorator to a test
structure suggestion: the only change in the commit is the application of the label and a comment, also the commit message as the word `reason` in it
note: similar mechanism enforced by sig-network Ordered decorator
another way would be to mark the PR with a specific label i.e. kind/no-flake-check-added, where the label color would be red, so that it is immediately visible
[dhiller] the optimal outcome would be that as many of the tests running on the “standard” required sig lanes which exist for all three kubernetes versions are also being covered on the check-tests-for-flake lane(s)
this should lead to more overall test stability on the required lanes which should be an achievement that every contributor benefits from
Action items
Daniel Hiller to take this to the mailing list and also community meeting
update/create issues with latest flakes spotted
communication
send meeting notes to kubevirt-dev, bcc sig people for spotted flakes (include meeting changes for upcoming instances)
Kind regards,
Daniel Hiller
He / Him / His
Principal Software Engineer, KubeVirt CI, OpenShift Virtualization
![]() |
Red Hat GmbH, Registered seat: Werner von Siemens Ring 12, D-85630 Grasbrunn, Germany Commercial register: Amtsgericht Muenchen/Munich, HRB 153243, Managing Directors: Ryan Barnhart, Charles Cachera, Avril Crosse O'Flaherty