How to run upgrade test case for cert-manager 1.16

53 views
Skip to first unread message

Chao Yu

unread,
Dec 26, 2024, 1:55:04 AM12/26/24
to cert-manager-dev
HI Team

Could someone tell me are there any available upgrading test cases to run (eg e2e cases) when I update cert-manager from 1.15.3 to 1.16.1.

I can see many e2e test cases in https://github.com/cert-manager/cert-manager/tree/release-1.6/test/e2e. But if I update the cert-manager from 1.15.3 to 1.16.1 and test whether the existing issue/certifiate/secret still works or not in cert-manager 1.16.1 evn. What should I do?

thanks~
Chao

Maël Valais

unread,
Jan 7, 2025, 3:30:49 AMJan 7
to cert-manager-dev
Hey,

We periodically run a test that upgrades cert-manager from 1.15 to 1.16 to in [1]. Not much is tested after upgrading, though: we use this upgrade test as a safety net to catch things like broken Helm charts. But it won't check whether the issuers are still working.

Side note: I have just realized that the logs [2] indicate that the upgrade test upgrades from 1.9.2 to 1.16.2, not sure what's going on. 1.9.2 has long been deprecated, I think we should look into it.

Maël Valais

unread,
Jan 7, 2025, 4:34:44 AMJan 7
to cert-manager-dev
Hey again,

I misunderstood how test-upgrade is being run in Prow. Beyond the 1.9.2 issue I mentioned above, for which I submitted a fix in [1], this test only ever tests that it is possible to upgrade from the latest cert-manager tag (currently, 1.16.2) to the latest possibly unreleased commit on the release-1.16 branch. Although this upgrade test is still worthy, I didn't realize that we weren't testing common cross-minor-version upgrades such as 1.14.7 to v1.16.2 and 1.15.4 to 1.16.2.

To Chao: the `make test-upgrade` make target won't be helpful to you as it stands.
Reply all
Reply to author
Forward
0 new messages