Backport issue milestones

[Michael Munday]

Hi all,

The backport process consists of opening an issue against a release milestone (say Go 1.15.3) and sending a CL containing a backport of the fix to gerrit. Once this is done it is up to the release team how and when to merge the fix.

However, I have noticed that the release team commonly bumps the milestone of a backported issue - even if a fix is available - on the same day as the release is cut. Given that we cannot merge fixes into release branches ourselves it can be frustrating when a fix is available and no explanation is given as to why it wasn't merged. Backported issues are typically only created for critical bugs and sometimes there are users blocked waiting for a fix to be included in a release. These fixes don't always warrant the release-blocker tag (especially when a non 'first-class' port is involved) but are important to sections of the community nonetheless.

Is it possible to modify the process so that the release team bumps the milestones of backported issues a couple of days (or more) before it is actually released? Ideally with a message like this so that people following the issue get a notification:

The milestone for this issue is being bumped to the next minor release (currently Go 1.18.2). Go 1.18.1 will shortly be frozen in preparation for its release and new fixes are not being accepted. If a fix for this issue is available and you think that it is important it be included in Go 1.18.1 please respond to this message as soon as possible.

That way expectations are set and there is a (short) window to object to a fix being left out.

I'm open to other suggestions too. Even just a message to golang-dev saying that a minor release is about to be frozen so that we know to ping unmerged fixes would be helpful.

Thanks,

Michael

[Dmitri Shuralyov]

Hi Michael,

Thanks for starting this discussion. We want the backport process to be transparent and easy to understand—there should be as little mystery or uncertainty about who is responsible for what, or what's blocking. The more clear it is to everyone, the less inefficiency there is. We generally rely on the process being documented at https://golang.org/wiki/MinorReleases, but this thread shows we have ways to improve.

As background, we target the beginning of each month to issue minor releases. It's not a guarantee, as security releases pre-empt an upcoming minor release, or there may be no fixes to deliver in a given release, or other factors. We start a release when it is safe to do so, which means it is sometimes later than the target date.

The general goal is for minor releases to happen monthly, so that when a backport fix makes it to the release branch, it will be included in the upcoming minor release. That means the month leading up to the next minor release is a window of opportunity for backports to land on the release branches. We on the release team make progress on backport issues continuously throughout the month, and when a new month starts, we issue a minor release with all the fixes that are ready by then. If a backport is critical to include, the release-blocking label is used for that, but otherwise the fix will can only get included in the next minor release, if it is ready by then.

In this particular case, the backport issues 40693 (for 1.15) and 40694 (for 1.14) were approved, CLs were sent for both, trybots were passing, but there was no code review done on either CL. The CLs were not submitted by a release manager as it could not be submitted without a +2 code review, and there was no release-blocking label on the issue, so it didn't get included in the release at the beginning of this month.

The next minor release is targeting Oct 1 (only 20 days away from today), so as long as it gets code reviewed by then, it should get submitted and included since the backport issue is approved. As documented at https://golang.org/wiki/MinorReleases, only release managers can submit CLs to the release branches. However, everyone can and should help with creating the cherry pick CL, ensuring trybots have been started and are passing, and finding a code reviewer.

Please also note that issuing overall robust releases that everyone can use is a higher priority than delivering any particular backport fix more quickly. The risk of a problem goes up with the number of changes, so we will sometimes opt to move slowly on a backport issue if we need more signal that it is going to be safe to include. The main point is that the month before a minor release comes out is the best time to make progress. At the beginning of a new month, our goal is to issue a minor release as soon as it is safe to do so, possibly on the 1st of the month.

Thanks,

Dmitri

[Dmitri Shuralyov]

I think automatically updating the milestone of open backport issues consistently at the beginning of the month (rather than on the day of the release) is a good idea to consider. It would make it more visible and clear what we expect to happen.

I've opened golang.org/issue/41323 for tracking this. I expect this is something we will take into account as part of ongoing release automation work.