2025-11-05 sig-ci quarantine catch-up - notes

[Daniel Hiller]

2025-11-05 sig-ci quarantine catch-up

When: Weekly on Wed, 9:45 – 10:15am

Notes: KubeVirt CI SIG meeting notes

Attendees: dhiller, brianmcarey, fossedihelm, nirdothan

Topics:

• [urgent]

• [dh] retester bot issue

• retester was switched off yesterday

• PR to re-install : https://github.com/kubevirt/project-infra/pull/4486

• ci-health

• 

• 

• previous action items

• state of existing issues: https://github.com/kubevirt/kubevirt/issues?q=is%3Aissue+is%3Aopen+label%3Akind%2Fflake+sort%3Aupdated-asc+label%3Asig%2Fci

• [non-urgent]

• [dh] flaky sig-monitoring test quarantined: https://github.com/kubevirt/kubevirt/pull/16018

• Look at flakes

• flake stats - create issues accordingly

• count: 2687 failures overall

• periodic sig * np lanes (PR introducing) failing all the time:

• 629 https://testgrid.k8s.io/kubevirt-periodics#periodic-kubevirt-e2e-k8s-1.34-sig-compute-np&width=20

• 348: https://testgrid.k8s.io/kubevirt-periodics#periodic-kubevirt-e2e-k8s-1.34-sig-network-np&width=20

• 144: https://testgrid.k8s.io/kubevirt-periodics#periodic-kubevirt-e2e-k8s-1.34-sig-storage-np&width=20

• 94: https://testgrid.k8s.io/kubevirt-periodics#periodic-kubevirt-e2e-k8s-1.34-sig-compute-migrations-np&width=20

• [dh] imho we need to remove them from the report for now

• [ff] Agree

• [ff] Looking on the failures. The root cause is only one. nw policy one missing allowing route, already working on this

• [bc] to create a revert PR

• sig-monitoring presubmit lane is failing every time since 2025-11-28:

• https://testgrid.k8s.io/kubevirt-presubmits#pull-kubevirt-e2e-k8s-1.34-sig-monitoring&width=20

• sig-monitoring, any thoughts from your side on how to work on this issue?

• https://github.com/kubevirt/kubevirt/issues/16017

• other lanes:

• sig-compute

• 287 periodic-kubevirt-e2e-k8s-1.35-sig-compute: https://testgrid.k8s.io/kubevirt-periodics#periodic-kubevirt-e2e-k8s-1.35-sig-compute&width=20

• https://github.com/kubevirt/kubevirt/issues/15976

• https://storage.googleapis.com/kubevirt-prow/reports/flakefinder/kubevirt/kubevirt/flake-stats-14days-2025-11-05.html#%5bsig-compute%5dSecurityFeatures%20Check%20virt-launcher%20securityContext%20With%20selinuxLauncherType%20defined%20as%20spc_t%20%5btest_id%3a3787%5dShould%20honor%20custom%20SELinux%20type%20for%20virt-launcher

• 167 pull-kubevirt-e2e-k8s-1.33-sig-operator: https://testgrid.k8s.io/kubevirt-presubmits#pull-kubevirt-e2e-k8s-1.33-sig-operator&width=20

• https://github.com/kubevirt/kubevirt/issues/16036

• clustered failure: 41 https://prow.ci.kubevirt.io/view/gs/kubevirt-prow/pr-logs/directory/pull-kubevirt-e2e-k8s-1.33-sig-operator/1985296489965948928

• clustered failure: 42 https://prow.ci.kubevirt.io/view/gs/kubevirt-prow/pr-logs/directory/pull-kubevirt-e2e-k8s-1.33-sig-operator/1983885556538413056

• 104 periodic-kubevirt-e2e-k8s-1.33-sig-compute: https://testgrid.k8s.io/kubevirt-periodics#periodic-kubevirt-e2e-k8s-1.33-sig-compute&width=20

• largely contributed due to failing quarantined tests

• sig-storage

• 152: https://testgrid.k8s.io/kubevirt-presubmits#pull-kubevirt-e2e-k8s-1.32-sig-storage&width=20&exclude-non-failed-tests=

• https://github.com/kubevirt/kubevirt/issues/15974

• https://prow.ci.kubevirt.io/view/gs/kubevirt-prow/pr-logs/directory/pull-kubevirt-e2e-k8s-1.32-sig-storage/1981004772802564096

• 
  

• dequarantine tests:

• look at list of quarantined tests

• count: 17 tests in quarantine currently
  

• check status, i.e. who is working on those

• look at PRs that want to fix flakes

• sig-network is working on dequarantining ipv6 https://github.com/kubevirt/kubevirt/pull/15955

• see whether we can dequarantine tests 

• misc topics

• 
  

Action items

• 
  
• 

  update/create issues with latest flakes spotted

• 

  communication

• send meeting notes to kubevirt-dev, bcc sig people for spotted flakes (include meeting changes for upcoming instances)

--

Kind regards,

Daniel Hiller

He / Him / His

Principal Software Engineer, KubeVirt CI, OpenShift Virtualization

Red Hat

dhi...@redhat.com   

Red Hat GmbH, Registered seat: Werner von Siemens Ring 12, D-85630 Grasbrunn, Germany  
Commercial register: Amtsgericht Muenchen/Munich, HRB 153243,
Managing Directors: Ryan Barnhart, Charles Cachera, Avril Crosse O'Flaherty