2025-10-08 sig-ci quarantine catch-up - notes

[Daniel Hiller]

2025-10-08 sig-ci quarantine catch-up

When: Weekly on Wed, 9:45 – 10:15am

Attendees: dhiller, brianmcarey, fossedihelm, ananya, 

Topics:

• [urgent]

• 
  

• ci-health

• 

• 

• previous action items

• state of existing issues: https://github.com/kubevirt/kubevirt/issues?q=is%3Aissue+is%3Aopen+label%3Akind%2Fflake+sort%3Aupdated-asc+label%3Asig%2Fci

• [non-urgent]

• 
  

• Look at flakes

• flake stats - create issues accordingly

• count: 762 failures overall

• large amount of failures in the periodic jobs

• sig-storage clustered failure (206): https://prow.ci.kubevirt.io/view/gs/kubevirt-prow/logs/periodic-kubevirt-e2e-k8s-1.34-sig-storage/1974379146733686784

• https://github.com/kubevirt/kubevirt/issues/15839

• sig-compute 1.32 clustered failure (84): https://prow.ci.kubevirt.io/view/gs/kubevirt-prow/logs/periodic-kubevirt-e2e-k8s-1.32-sig-compute/1972096495574323200

• existing issue: https://github.com/kubevirt/kubevirt/issues/15791

• sig-compute 1.33 clustered failures

• (23) https://prow.ci.kubevirt.io/view/gs/kubevirt-prow/logs/periodic-kubevirt-e2e-k8s-1.33-sig-compute/1974046950093230080

• https://github.com/kubevirt/kubevirt/issues/15840

• (21) https://prow.ci.kubevirt.io/view/gs/kubevirt-prow/logs/periodic-kubevirt-e2e-k8s-1.33-sig-compute/1973805291170435072

• https://github.com/kubevirt/kubevirt/issues/15841

• significant amount of failures in presubmit sig-compute-serial (71):

• https://testgrid.k8s.io/kubevirt-presubmits#pull-kubevirt-e2e-k8s-1.34-sig-compute-serial&width=20

• need to check whether it’s only a noise due to lots of cancelled jobs

• as suspected largely caused due to cancelled jobs

• dequarantine tests:

• look at list of quarantined tests

• count: tests in quarantine currently
  (insert screenshot from ci-health)

• check status, i.e. who is working on those

• look at PRs that want to fix flakes

• see whether we can dequarantine tests 

• misc topics

• [bc] update to fedora 39 for fedora-with-test-tooling guests looks ready 

• https://github.com/kubevirt/kubevirt/pull/13322 

• [fossedihelm] q: do we have an update of 1.24 builder update?

• [bc] PR is there, has an lgtm, needs root approver approval

• https://github.com/kubevirt/kubevirt/pull/15784

• [xpivarc] reduce bazel wrt proto: https://github.com/kubevirt/kubevirt/pull/15837

Action items

• 

  [dhiller] https://github.com/kubevirt/kubevirt/pull/13322

• 

  update/create issues with latest flakes spotted

• 

  communication

• send meeting notes to kubevirt-dev, bcc sig people for spotted flakes (include meeting changes for upcoming instances)

--

Kind regards,

Daniel Hiller

He / Him / His

Principal Software Engineer, KubeVirt CI, OpenShift Virtualization

Red Hat

dhi...@redhat.com   

Red Hat GmbH, Registered seat: Werner von Siemens Ring 12, D-85630 Grasbrunn, Germany  
Commercial register: Amtsgericht Muenchen/Munich, HRB 153243,
Managing Directors: Ryan Barnhart, Charles Cachera, Avril Crosse O'Flaherty