tl;dr To avoid having our release automation diverging we should default to
uplifting changes to code involved in release automation that doesn't
affect the bits being shipped.
Over the lifetime of ESR52, the release process and surrounding automation
for non-ESR releases changed significantly (due to the taskcluster
migration). This required releaseduty to maintain very divergent
instructions for ESR releases ([1] vs [2]). Given the large amount of
changes involved in the final migration of buildbot, this was probably
unavoidable. While we don't anticipate major changes there are a number of
improvements in the pipeline[3][4][5] that will cause divergence.
We've also ran into issues (for example [1]) due to the build and release
automation not being kept up-to-date. We also want to keep our automation
tooling up-to-date with security fixes like [7].
We could just up-lift patches that have direct impact on the release
process, but as the ESR cycles progresses, it will become more difficult to
uplift changes. Instead I propose that we adopt a policy of uplifting
relevant changes to ESR60:
- taskgraph code (`taskcluster/taskgraph`)
- release automation tasks (`taskcluster/ci/release-*` mostly)
- testing/mozharness cleanups
As none of the changes impact the built artifacts, they should be low risk,
and make any changes that are critical to uplift less risky to uplift. I'm
excluding from this proposal changes to
- toolchain version updates
- test configuration
- new build configurations
My initial plan is as follows. It open to suggestions and to change as we
get experience with the process.
1. I will prepare a branch with changes to bring ESR60 up-to-date and post
that review.
2. I will run a staging release with the changes to verify that the changes
won’t cause issues when we go to do a release.
3. After each release cycle, I will repeat the above steps to pick up any
changes from the most recent release cycle. (After the initial uplift, this
should be fairly smooth)
There may be patches that require uplifting in a more timely manner
(security fixes or things that require coordination with out-of-tree
changes); they can continue to be handled in an ad-hoc manner. After a few
cycles, once this process has worked smoothly, we’ll look at migrating the
uplifts to be handled by releaseduty.
I don't have an exhaustive list of changes, but the following is a sampling
of changes (some of which have been uplifted, or there are already plans to
uplift).
- Mercurial Updates (Bug 1448438)
- Release Notification fixes (Bugs 1459185 and 1459116) [2]
- Converting actions to use hooks (Bug 1415868)
- Pretty release platforms (
https://bugzilla.mozilla.org/show_bug.cgi?id=1456234)
- Fetch tasks (
https://bugzilla.mozilla.org/show_bug.cgi?id=1460777)
- Publishing buildhub data (
https://bugzilla.mozilla.org/show_bug.cgi?id=1442306)
- Declarative artifacts (
https://bugzilla.mozilla.org/show_bug.cgi?id=1462393)
Additionally, I plan to forward port some fixes that were made directly to
esr60 in release automation to make this process easier.
-- Tom
[1]
https://github.com/mozilla-releng/releasewarrior-2.0/blob/master/docs/release-promotion/desktop/howto_58cycle_and_52esr_only.md
[2]
https://github.com/mozilla-releng/releasewarrior-2.0/blob/master/docs/release-promotion/desktop/howto.md
[3]
https://bugzilla.mozilla.org/show_bug.cgi?id=1415868
[4]
https://bugzilla.mozilla.org/show_bug.cgi?id=1462393
[5]
https://bugzilla.mozilla.org/show_bug.cgi?id=shipit-v2
[6]
https://bugzilla.mozilla.org/show_bug.cgi?id=1449350
https://bugzilla.mozilla.org/show_bug.cgi?id=1459391
https://bugzilla.mozilla.org/show_bug.cgi?id=1446296
[7]
https://bugzilla.mozilla.org/show_bug.cgi?id=1385978