2024-12-04 sig-ci quarantine catch-up - notes

[Daniel Hiller]

 2024-12-04 sig-ci quarantine catch-up

When: Weekly on Wed, 9:45 – 10:15am

Attendees: dhiller, ffossemo, omisan, brianmcarey

Topics:

• [urgent]

• [dh] strange check-tests-for-flakes error holding a PR: https://prow.ci.kubevirt.io/view/gs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/11833/pull-kubevirt-check-tests-for-flakes/1863922245907255296

• postcopy migration seems to have issues with the check-tests-for-flakes, Jed and Federico are investigating

• issue https://github.com/kubevirt/kubevirt/issues/13435

• previous action items

• state of existing issues: https://github.com/kubevirt/kubevirt/issues?q=is%3Aissue+is%3Aopen+label%3Akind%2Fflake+sort%3Aupdated-asc+label%3Asig%2Fci

• dh follow up on

•  https://github.com/kubevirt/kubevirt/issues/13316

• https://github.com/kubevirt/kubevirt/issues/13141

• [dh] q: do we want to remove the sig-storage-root lanes from our reports? IIUC there’s little to no interest from our side?

• another option would be to remove the lane?

• [non-urgent]

• new report: per-test-execution

• https://storage.googleapis.com/kubevirt-prow/reports/per-test-results/kubevirt/kubevirt/last-six-months/index.html

• [omisan] https://github.com/kubevirt/kubevirt/issues/13421 

• [ffossemo] we need to fix the issue and not ignore the warning that is emitted, since it proves that there’s flaky behaviour going on

• [ffossemo] to take a look

• [bc] maybe also caused by high load

• Look at flakes

• flake stats - create issues accordingly

• observability known flake

• [ffossemo] raised it during sig-observability

• arm lane has clustered failure: https://prow.ci.kubevirt.io/view/gcs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/13252/pull-kubevirt-e2e-arm64/1862481259553361920

• https://github.com/kubevirt/kubevirt/issues/13436

• sig-operator clustered failure: https://prow.ci.kubevirt.io/view/gcs/kubevirt-prow/pr-logs/pull/batch/pull-kubevirt-e2e-k8s-1.31-sig-operator/1863922560018681856

• https://github.com/kubevirt/kubevirt/issues/13364

• sig-compute-migrations clustered failure: https://prow.ci.kubevirt.io/view/gcs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/13333/pull-kubevirt-e2e-k8s-1.31-sig-compute-migrations/1863543034155634688

• https://github.com/kubevirt/kubevirt/issues/12932

• dequarantine tests:

• look at list of quarantined tests

• check status, i.e. who is working on those

• look at PRs that want to fix flakes

• see whether we can dequarantine tests 

Action items

• 
  
• 

  update/create issues with latest flakes spotted

• 

  communication

• send meeting notes to kubevirt-dev, bcc sig people for spotted flakes (include meeting changes for upcoming instances)

--

Kind regards,

Daniel Hiller

He / Him / His

Senior Software Engineer, KubeVirt CI, OpenShift Virtualization

Red Hat

dhi...@redhat.com   

Red Hat GmbH, Registered seat: Werner-von-Siemens-Ring 12, D-85630 Grasbrunn, Germany  
Commercial register: Amtsgericht München/Munich, HRB 153243,
Managing Directors: Ryan Barnhart, Charles Cachera, Michael O'Neill, Amy Ross