2026-07-01 sig-ci quarantine catch-up - notes

[Daniel Hiller]

2026-07-01 sig-ci quarantine catch-up

When: Weekly on Wed, 9:45 – 10:15am

Notes: KubeVirt CI SIG meeting notes

Attendees: Dylan White, dhiller, dollierp, Orel Misan

Reminders:

• we will create GitHub issues for tracking

• GitHub issues and PRs

• should be marked with /sig ci and /kind flake if applicable

• should be marked with the target sig

• kubevirt quarantine process documentation

Topics:

• [urgent]

• [dhiller] sig-compute: two quarantined tests “Live Migration across namespaces” failing 100% since 16th June

• [xpivarc] awels should handle it

• [fossedihelm] there should be a PR already in the works for this

• This should be the fix: https://github.com/kubevirt/kubevirt/pull/18216

• [dhiller] additionally created issue https://github.com/kubevirt/kubevirt/issues/18302 to fix the labelling

• [orelmisan] unit tests are failing package TLS

• probably due to Felix PR?

• https://prow.ci.kubevirt.io/view/gs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/18095/pull-kubevirt-unit-test/2072158183824035840

• https://prow.ci.kubevirt.io/view/gs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/18095/pull-kubevirt-unit-test/2072110883865104384

• [dollierp] rather related to the golang bump than Felix

• [fossedihelm] rather resource issue on the postsubmit, which was fixed by Brians PR who applied increase to match those in the presubmit

• [dhiller] uploader https://github.com/kubevirt/kubevirt/pull/18264

• has to get merged before branch is cut

• ci-health

• 

• 

• review and merge the quarantine PRs in 2 working days without SIG lgtm. 

• https://github.com/kubevirt/kubevirt/pulls?q=is%3Apr+is%3Aopen+quarantine+-label%3Ado-not-merge%2Fwork-in-progress+label%3Akind%2Fflake+-label%3Aneeds-rebase

• n/a

• previous action items

• state of existing issues: https://github.com/kubevirt/kubevirt/issues?q=is%3Aissue+is%3Aopen+label%3Akind%2Fflake+sort%3Aupdated-asc+label%3Asig%2Fci

• n/a

• [non-urgent]

• Orel Misan Recent changes to SR-IOV lane

• needs acks from Edy or Orel Misan

• Look at flakes

• most flaky tests

• 
  

• flake stats - create issues accordingly

• overall ( ∑=1225, 100.00% )

• periodic-kubevirt-e2e-k8s-1.36-sig-compute-migrations ( ∑=122, 9.96% )

• sig-compute: two quarantined tests “Live Migration across namespaces” failing 100% since 16th June

• last successful run: https://prow.ci.kubevirt.io/view/gs/kubevirt-prow/logs/periodic-kubevirt-e2e-k8s-1.36-sig-compute-migrations/2066800526040764416

• first failing run: https://prow.ci.kubevirt.io/view/gs/kubevirt-prow/logs/periodic-kubevirt-e2e-k8s-1.36-sig-compute-migrations/2066891124525502464

• see discussion in urgent section

• periodic-kubevirt-e2e-k8s-1.36-sig-storage ( ∑=112, 9.14% )

• sig-storage: same as above are failing on sig-storage lane also?

• periodic-kubevirt-e2e-k8s-1.34-sig-storage ( ∑=106, 8.65% )

• sig-storage: same as above are failing on sig-storage lane also?

• periodic-kubevirt-e2e-k8s-1.35-sig-storage ( ∑=104, 8.49% )

• sig-storage: same as above are failing on sig-storage lane also?

• periodic-kubevirt-e2e-k8s-1.34-sig-compute ( ∑=90, 7.35% )

• USB Passthrough failing 50%

• periodic-kubevirt-e2e-k8s-1.35-sig-compute ( ∑=77, 6.29% )

• USB Passthrough failing 50%

• periodic-kubevirt-e2e-test-S390X ( ∑=74, 6.04% )

• is failing again now every time, investigation required

• periodic-kubevirt-e2e-k8s-1.36-sig-compute ( ∑=69, 5.63% )

• USB Passthrough failing 50%

• pull-kubevirt-e2e-k8s-1.36-sig-compute-migrations ( ∑=53, 4.33% )

• pull-kubevirt-e2e-k8s-1.36-sig-operator ( ∑=52, 4.24% )

• periodic-kubevirt-e2e-k8s-1.34-sig-operator ( ∑=44, 3.59% )

• pull-kubevirt-e2e-k8s-1.34-sig-operator ( ∑=44, 3.59% )

• periodic-kubevirt-e2e-k8s-1.34-sig-monitoring ( ∑=43, 3.51% )

• pull-kubevirt-e2e-kind-1.35-sig-compute-arm64 ( ∑=37, 3.02% )

• periodic-kubevirt-e2e-k8s-1.34-sig-performance-kwok ( ∑=28, 2.29% )

• periodic-kubevirt-e2e-k8s-1.34-sig-performance-kwok-100 ( ∑=28, 2.29% )

• pull-kubevirt-e2e-k8s-1.36-sig-compute ( ∑=22, 1.80% )

• pull-kubevirt-e2e-k8s-1.34-sig-monitoring ( ∑=18, 1.47% )

Last updated: 2026-07-01 06:52:10.330192382 +0000 UTC m=+12.820910847

• Look at held tests:

• https://grafana.ci.kubevirt.io/d/uAoSeksSk/referee-retests?orgId=1&refresh=15m&from=now-1h&to=now

• https://github.com/kubevirt/kubevirt/pulls?q=is%3Apr+is%3Aopen+label%3Aapproved+label%3Algtm+-label%3Ado-not-merge%2Fhold+-label%3Aneeds-rebase

• dequarantine tests:

• look at list of quarantined tests

• Count: 14 tests in quarantine currently
  

• check status, i.e. who is working on those

• look at PRs that want to fix flakes

• see whether we can dequarantine tests

• misc topics

• 
  

Action items

• 
  
• 

  update/create issues with latest flakes spotted

• 

  communication

• send meeting notes to kubevirt-dev, bcc sig people for spotted flakes (include meeting changes for upcoming instances)

--

Kind regards,

Daniel Hiller

He / Him / His

Principal Software Engineer, KubeVirt CI, OpenShift Virtualization

Red Hat

dhi...@redhat.com   

Red Hat GmbH, Registered seat: Werner von Siemens Ring 12, D-85630 Grasbrunn, Germany  
Commercial register: Amtsgericht Muenchen/Munich, HRB 153243,
Managing Directors: Ryan Barnhart, Charles Cachera, Avril Crosse O'Flaherty  

[Daniel Hiller]

issue for lane: https://github.com/kubevirt/kubevirt/issues/18306

--

-- 

Best,

Daniel