2026-07-29 sig-ci quarantine catch-up - notes

[Daniel Hiller]

2026-07-29 sig-ci quarantine catch-up

When: Weekly on Wed, 9:45 – 10:15am

Notes: KubeVirt CI SIG meeting notes

Attendees: dywhite, dhiller, dollierp, fossedihelm

Reminders:

• we will create GitHub issues for tracking

• GitHub issues and PRs

• should be marked with /sig ci and /kind flake if applicable

• should be marked with the target sig

• kubevirt quarantine process documentation

Topics:

• [urgent]

• [dollierp] Merges are blocked by a s390x unit test failure

• Failing test was merged when the s390x clusters were down and lane disable

• PR fixing the issue needs attention from devs: PR#18611

• [dhiller] will investigate what we can improve related to the process

• why is a unit test platform dependent

• why are there architecture lanes for unit-tests?

• unit tests that are not platform independent are NOT unit tests!

• process should be that if a unit test is being optional or disabled, merging should stop

• [dollierp] Fix for cluster-up issue got merged

• Fix: https://github.com/kubevirt/kubevirtci/pull/1791: use fedora 43 QEMU

• Bump: https://github.com/kubevirt/kubevirt/pull/18605 (s390x was waived because irrelevant for this one)

• ci-health

• 

• 

• review and merge the quarantine PRs in 2 working days without SIG lgtm. 

• https://github.com/kubevirt/kubevirt/pulls?q=is%3Apr+is%3Aopen+quarantine+-label%3Ado-not-merge%2Fwork-in-progress+label%3Akind%2Fflake+-label%3Aneeds-rebase

• n/a

• previous action items

• state of existing issues: https://github.com/kubevirt/kubevirt/issues?q=is%3Aissue+is%3Aopen+label%3Akind%2Fflake+sort%3Aupdated-asc+label%3Asig%2Fci

• [non-urgent]

• n/a

• Look at flakes

• most flaky tests

• none on main

• [dollierp] TLS flake should be backported?

• https://storage.googleapis.com/kubevirt-prow/reports/most-flaky-tests/kubevirt/kubevirt/index.html#test_%5bsig-compute%5d%20Infrastructure%20tls%20configuration%20should%20enforce%20TLS%20configuration%20on%20virt-template%20components

• PR: https://github.com/kubevirt/kubevirt/pull/18341

• flake stats - create issues accordingly

• overall ( ∑=2071, 100.00% )

• periodic-kubevirt-e2e-k8s-1.34-sig-compute ( ∑=507, 24.48% )

• Automation needs to be updated to enable the Plugins feature gate in periodics

• https://github.com/kubevirt/kubevirt/blob/main/automation/test.sh#L128-L131 

• [dhiller] will look into how to enable this for periodics

• periodic-kubevirt-e2e-k8s-1.36-sig-compute ( ∑=502, 24.24% )

• Same as above

• periodic-kubevirt-e2e-k8s-1.35-sig-compute ( ∑=433, 20.91% )

• Same as above

• periodic-kubevirt-e2e-test-S390X ( ∑=326, 15.74% )

• pull-kubevirt-e2e-k8s-1.36-sig-compute-serial ( ∑=54, 2.61% )

• periodic-kubevirt-e2e-k8s-1.34-sig-monitoring ( ∑=40, 1.93% )

• pull-kubevirt-e2e-k8s-1.36-sig-compute-migrations ( ∑=35, 1.69% )

• periodic-kubevirt-e2e-kind-1.36-sev ( ∑=33, 1.59% )

Last updated: 2026-07-29 07:19:18.958732599 +0000 UTC m=+11.270396392

• Look at held tests:

• https://grafana.ci.kubevirt.io/d/uAoSeksSk/referee-retests?orgId=1&refresh=15m&from=now-1h&to=now

• https://github.com/kubevirt/kubevirt/pulls?q=is%3Apr+is%3Aopen+label%3Aapproved+label%3Algtm+-label%3Ado-not-merge%2Fhold+-label%3Aneeds-rebase

• dequarantine tests:

• look at list of quarantined tests

• Count: 13 tests in quarantine currently
  

• check status, i.e. who is working on those

• look at PRs that want to fix flakes

• [fossedihelm] there’s more, but those are missing the correct labels kind/flake & priority/critical-urgent

• see whether we can dequarantine tests

• n/a

• misc topics

• [dollierp] fossedihelm un-quarantine PR the check dequarantine is failing due to timeout

• [fossedihelm] failed on 468 iterations

• [fossedihelm] lgtmed the original fix for s390x

• [fossedihelm] qq: pushed PR wrt project-infra, is release tool references updated automatically?

• will check myself

Action items

• 
  
• 

  update/create issues with latest flakes spotted

• 

  communication

• 

  send meeting notes to kubevirt-dev, bcc sig people for spotted flakes (include meeting changes for upcoming instances)

--

Kind regards,

Daniel Hiller

He / Him / His

Principal Software Engineer, KubeVirt CI, OpenShift Virtualization

Red Hat

dhi...@redhat.com   

Red Hat GmbH, Registered seat: Werner von Siemens Ring 12, D-85630 Grasbrunn, Germany  
Commercial register: Amtsgericht Muenchen/Munich, HRB 153243,
Managing Directors: Ryan Barnhart, Charles Cachera, Avril Crosse O'Flaherty