2026-07-06 sig-ci weekly meeting - notes

[Daniel Hiller]

2026-07-06 sig-ci weekly meeting

When: Weekly on Mon, 10:00 – 10:30am

Notes: KubeVirt CI SIG meeting notes

Attendees: Dylan White, dhiller, dollierp, nirdothan

Reminders:

• we will create GitHub issues for tracking

• GitHub issues and PRs

• should be marked with /sig ci and /kind flake if applicable

• should be marked with the target sig

Topics:

• [urgent]

• [dollierp] s390x clusters are down since Saturday around 8 AM (UTC)

• https://testgrid.k8s.io/kubevirt-project-infra-periodics

• stakeholders have been notified

• It seems to be a network problem on IBM site.

• pull request to disable s390x presubmit jobs: [project-infra] PR#5239

• [dhiller] merged PR by hand to disable the lanes, so that CI for kubevirt will recover eventually

• [dhiller] sig-compute: two quarantined tests “Live Migration across namespaces” failing 100% since 16th June

• check whether https://github.com/kubevirt/kubevirt/pull/18216 has fixed it

• unquarantine PR https://github.com/kubevirt/kubevirt/pull/18217

• delete source migrations seems to be good, target migration has had two failures in three days

• [fossedihelm] to follow up on this

• [fossedihelm] https://github.com/kubevirt/kubevirt/pull/18264 needs to be merged before the next bump Kubevirtci does(https://github.com/kubevirt/kubevirt/pull/18319).

• We need to backport the former, but not the latter AFAIK.

• [dhiller] merged by hand since CI is broken due to s390x anyway

• ci-health

• 

• 

• sig-ci failures: 95% external

• quick look at the e2e job failures:
  https://grafana.ci.kubevirt.io/d/efpTS3t4z/e2e-jobs-overview-v2

• 

• [timeboxed: 5 mins] revisit previous action items

• 
  

• existing issues opened in last 7 days: https://github.com/search?q=repo%3Akubevirt%2Fproject-infra+is%3Aissue++-label%3Akind%2Fenhancement&type=issues&state=open

• [dhiller] Hardcoded golang image tag in kubevirtci-presubmit-creator is not updated by image bumper

• [non-urgent]

• [fossedihelm] check-tests-for-flakes is not detecting changed tests properly in some corner cases: https://github.com/kubevirt/kubevirt/pull/18338

• If something is changed in a DescribeTable, only the last Entry is reported (duplicated the number of Entries).

• [dhiller] source code for detecting tests is located here: https://github.com/kubevirt/project-infra/blob/main/robots/ginkgo-tests/cmd/extract-changed.go

• [fossedihelm] Drop false positive reported: https://github.com/kubevirt/kubevirt/pull/18335

• [dhiller] will take a look

• [fossedihelm] https://github.com/kubevirt/project-infra/pull/5223

• [dhiller] will need to take a look at last time this was discussed

• [misc]

• Look at held tests:

• https://grafana.ci.kubevirt.io/d/uAoSeksSk/referee-retests?orgId=1&refresh=15m&from=now-1h&to=now

• is:pr is:open label:approved label:lgtm -label:do-not-merge/hold -label:needs-rebase 

• is:pr is:open quarantine -label:do-not-merge/work-in-progress label:kind/flake -label:needs-rebase 

• recently merged PRs authored by SIG CI

• 14 recently merged PRs authored by SIG CI (query: is:pr is:merged merged:>=2026-06-29 author:dhiller author:dollierp author:whitedyl org:kubevirt”)

• kubevirt/ci-health#162: fix: handle missing latest commit in PR history page gracefully (by @dhiller)

• kubevirt/project-infra#5224: fix(grafana-token): add --no-fork to git-pr.sh for private repo support (by @Whitedyl)

• kubevirt/project-infra#5221: feat(branch-protector): switch to GitHub App credentials (by @dollierp)

• kubevirt/ci-health#160: fix: use GCS API directly and retry on HTTP 500 (by @dhiller)

• kubevirt/project-infra#5210: fix(check-presubmits): add missing backslash before newline (by @dollierp)

• kubevirt/project-infra#5209: Add Whitedyl to ci-maintainers (by @Whitedyl)

• kubevirt/ci-health#159: fix: retry on transient 502/503 in Prow HTTP client (by @dhiller)

• kubevirt/kubevirt#18290: automation: enable fail fast mode when triggered by a Prow batch (by @dollierp)

• kubevirt/ci-health#158: feat: adapt Claude skills to be usable with Gemini (by @dhiller)

• kubevirt/ci-health#157: feat: add follow up suggestions for Claude skills (by @dhiller)

• kubevirt/ci-health#156: feat: add flake overview reports (by @dhiller)

• kubevirt/project-infra#5203: chore(jobs): consolidate PGP bot key preset (by @dollierp)

• kubevirt/project-infra#5200: feat(deck): switch to upstream manifests (by @dollierp)

• kubevirt/project-infra#5188: chore(kustomize): use kubectl instead of old standalone binary (by @dollierp)

Action items

• 
  
• 

  communication

• 

  send meeting notes to kubevirt-dev (include meeting changes for upcoming instances)

--

Kind regards,

Daniel Hiller

He / Him / His

Principal Software Engineer, KubeVirt CI, OpenShift Virtualization

Red Hat

dhi...@redhat.com   

Red Hat GmbH, Registered seat: Werner von Siemens Ring 12, D-85630 Grasbrunn, Germany  
Commercial register: Amtsgericht Muenchen/Munich, HRB 153243,
Managing Directors: Ryan Barnhart, Charles Cachera, Avril Crosse O'Flaherty  

[Alexander Wels]

On Mon, Jul 6, 2026 at 3:36 AM 'Daniel Hiller' via kubevirt-dev <kubevi...@googlegroups.com> wrote:

2026-07-06 sig-ci weekly meeting

When: Weekly on Mon, 10:00 – 10:30am

Notes: KubeVirt CI SIG meeting notes

Attendees: Dylan White, dhiller, dollierp, nirdothan

Reminders:

• we will create GitHub issues for tracking

• GitHub issues and PRs

• should be marked with /sig ci and /kind flake if applicable

• should be marked with the target sig

Topics:

• [urgent]

• [dollierp] s390x clusters are down since Saturday around 8 AM (UTC)

• https://testgrid.k8s.io/kubevirt-project-infra-periodics

• stakeholders have been notified

• It seems to be a network problem on IBM site.

• pull request to disable s390x presubmit jobs: [project-infra] PR#5239

• [dhiller] merged PR by hand to disable the lanes, so that CI for kubevirt will recover eventually

• [dhiller] sig-compute: two quarantined tests “Live Migration across namespaces” failing 100% since 16th June

• check whether https://github.com/kubevirt/kubevirt/pull/18216 has fixed it

• unquarantine PR https://github.com/kubevirt/kubevirt/pull/18217

• delete source migrations seems to be good, target migration has had two failures in three days

• [fossedihelm] to follow up on this

Yes, I found some issues on the target deletion side. 18217 has some fixes for that side. I'm running the de-quarantine test lane now. Last run did fail on 499 out of 500. But I think it found an actual issue. Next run should hopefully pass.

 

--
You received this message because you are subscribed to the Google Groups "kubevirt-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/kubevirt-dev/CAK%2BeyL5QCSR4ZK%2BAuJ%2Bh8%2BmmEu8FcCEQCzgpv8k7AJFdb8834g%40mail.gmail.com.