Go 1.17 release status
2,068 views
Skip to first unread message
Dmitri Shuralyov
May 10, 2021, 6:15:50 PM5/10/21
to golang-dev
Hello gophers!
We're now a week into the release freeze. As a reminder from golang.org/s/release (May 1 section):
This part of the release cycle is focused on improving the quality of the release,
by testing it and fixing bugs that are found. [...]
[...] By the end of the first month of the freeze, nearly all known bugs should have
been fixed or explicitly postponed (either to the next release or indefinitely).
(Please read the complete text for guidance on how the balance between
fixing bugs and the risk of introducing problems changes over time.)
As of right now, in the the Go 1.17 milestone there are:
- a total of 112 open issues (query)
- 39 issues with a release-blocker label, but not the "okay-after-beta1" label (query)
- The "okay-after-beta1" label means an issue is deemed safe to resolve before or after beta 1.
- These 39 issues block the Go 1.17 Beta 1 release. If they're all resolved before June 1, the Beta 1 release can happen ahead of schedule.
- 21 of these are documentation issues related to the Please help with Go 1.17 release notes thread
- 4 more issues with a release-blocker label, but also the "okay-after-beta1" label
- The Go 1.17 release cannot happen without some resolution to these issues, but they're not as time sensitive.
Please note there are approximately 3 weeks left in this month.
We will update this thread in two weeks, as we should have a better sense of release
status based on how quickly progress is made on the known release-blocking issues,
and what new issues are found via testing.
Thank you all,
Dmitri
Carlos Amedee
May 24, 2021, 5:48:42 PM5/24/21
to golang-dev
Hello gophers!
As we continue working toward releasing the Go 1.17 beta, we would like to provide an update on the release status:
The "okay-after-beta1" label means an issue is deemed safe to resolve before or after beta 1.
These 29 issues block the Go 1.17 Beta 1 release. If they're all resolved before June 1, the Beta 1 release can happen ahead of schedule.
18 of these are documentation issues related to the Please help with Go 1.17 release notes thread.
12 more issues with a release-blocker label, but also the "okay-after-beta1" label (query)
The Go 1.17 release cannot happen without some resolution to these issues, but they're not as time sensitive.
Please note, there is approximately 1 week left in this month. With 29 beta release blocking issues and three US working days left before the Memorial Day (US Holiday) weekend, we don't expect the beta release to happen on June 1st 2021. We will update this thread in a week with the status of the beta release.
Thank you,
Carlos
--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/CA%2BON-PELwLdO9mBeoOMPoaTd%3DtqyqD8-Q70v91P_EW6Q0mRPnA%40mail.gmail.com.
Dmitri Shuralyov
Jun 4, 2021, 8:13:45 PM6/4/21
to golang-dev
Hello gophers,
Thanks to your work, significant progress has been made. The latest status of the release is:
- A total of 62 open issues (query).
- 5 issues with a release-blocker label, but not the "okay-after-beta1" label (query).
- 4 of these issues are documentation issues related to the 12 remaining visible TODOs on https://tip.golang.org/doc/go1.17.
- The fifth issue is a problem in the misc/swig/callback test, and has a fix CL.
- 23 more issues with a release-blocker label, but also the "okay-after-beta1" label (query).
- The number has gone up as some release-blocking issues were determined not to need to block beta 1, and new issues were found. A goal of the beta release will be wider testing, which may uncover additional issues to fix before the final release.
This puts us very close to being able to release Go 1.17 Beta 1. We are currently targeting next week, pending the completion of the Go 1.17 release notes draft and the remaining beta release blocking issues.
Thanks,
Dmitri
Carlos Amedee
Jun 18, 2021, 3:56:35 PM6/18/21
to golang-dev
Hello Gophers!
We would like to thank everybody who contributed by helping to resolve enough issues to release the Go 1.17 beta. We are working toward releasing the Go 1.17 release candidate near the beginning of July.
There are a total of 53 open issues in the Go 1.17 milestone.
21 of those issues are release blockers.
If we are able to resolve these issues by Independence Day (US Holiday), we should be able to cut the first release candidate soon thereafter.
Thank you!
Carlos for the Go Team
Dmitri Shuralyov
Jul 2, 2021, 5:52:58 PM7/2/21
to golang-dev
Hello gophers,
By now, we are down to 41 open issues in the Go 1.17 milestone.
11 of those issues are release blockers.
Some of the release-blocking issues are recurring tasks that can be safely resolved
just before the final release. That leaves 6 non-recurring release-blocking issues.
There is more work that needs to be done on those issues to unblock RC 1,
but we are getting quite close. We'll be continuing that work next week,
though it will be a shorter week due to a US holiday.
Thanks,
Dmitri
Alex Rakoczy
Jul 15, 2021, 4:10:50 PM7/15/21
to golang-dev
Hello Gophers,
This week we saw the Go 1.17 RC1 release! Thank you to everyone who helped make this happen.
There are still 41 open issues in the Go 1.17 milestone, 1 of which is a non-recurring release blocker. The remaining issue should be resolved by the next proposal meeting.
Assuming no new issues are discovered, this release is looking very healthy.
Cheers,
Alex
Daniel Martí
Aug 9, 2021, 5:51:36 AM8/9/21
to golang-dev
Hi Alex, Dmitri, and others,
I was wondering if there were any news on Go 1.17, since the last email
was four weeks ago. I understand it's August, but I also assume there's
someone in charge of the release every summer :)
From what I can see here, these two bugs are holding up the release:
https://github.com/golang/go/issues/47543
https://github.com/golang/go/issues/47565
I really appreciate the added transparency over the past few releases.
At least now we can all see what's going on when a release is late by a
few weeks, thanks to these emails.
That said, I think the process might be due for another incremental
revamp. At least from my external point of view, I see these areas for
improvement:
1) Releases are still regularly late. Out of the last six major releases
(Go 1.11 to 1.16), the earliest was a week late (1.15 on August
11th), and the latest was a month late (1.13 on September 3rd).
2) Due to the above, trees tend to reopen at least 2-3 weeks late, which
can be longer if a release has complex "early changes" that get
priority. This means our release cycle is coser to a "two month
development, four month freeze" than the documented 3-3 month split -
at least for the majority not working on the set of "early changes".
3) Both points make the "release cycle" wiki page somewhat misleading.
The only way it's currently useful is if one reads it with the lens
of: ideally this is the calendar we follow, but in practice assume
the final release and tree reopening is a couple of weeks later.
4) That last point makes external contribution harder. Many companies
plan weeks ahead, e.g. sprints, so it's hard to allocate time for
upstreaming Go changes or testing Go betas/RCs if the work might be
blocked by anywhere from 1 to 4 weeks. For example, I had allocated
some time this week to upgrade many modules at work to Go 1.17.
It's hard for me to really see a pattern around why we tend to be a
couple of weeks behind schedule. At least in the last couple of
releases, we've released beta1 within a week, but the delay has quickly
built up with the RCs leading up to the final release. So it does seem
like something goes wrong whenever it's time to follow the RC calendar.
Perhaps one month between RC and final release is too optimistic?
Multiple release blocker issues tend to be found after each RC1,
when Google and others start using the new version in production.
Those issues tend to take at least a couple of weeks to iron out,
and extra RCs want 2-4 weeks before the final release.
If that sounds like a reasonable approximation of what's happening with
most releases, perhaps one way to mitigate this would be to aim for
beta1 and rc1 to be out a week ahead of schedule, giving the "production
testing" at least five weeks. I think that would be a reasonable change
to pull off, given that we've been stricter with "late merges".
Perhaps we'd need to move in an entirely different direction as well,
like pushing harder for Go users and companies to test betas and RCs
more aggressively. I'm sure that the team who works on releases is who
ultimately has the best view on the causes and solutions. My intuition
is that we'll want a mix of incremental steps.
Have a good summer!
On Thu, Jul 15, 2021 at 13:10:50 -0700, Alex Rakoczy wrote:
> Hello Gophers,
>
> This week we saw the Go 1.17 RC1 release! Thank you to everyone who helped
> make this happen.
>
> There are still 41 open issues in the Go 1.17 milestone
> <https://github.com/golang/go/issues?q=is%3Aopen+is%3Aissue+milestone%3AGo1.17>,
I was wondering if there were any news on Go 1.17, since the last email
was four weeks ago. I understand it's August, but I also assume there's
someone in charge of the release every summer :)
From what I can see here, these two bugs are holding up the release:
https://github.com/golang/go/issues/47543
https://github.com/golang/go/issues/47565
I really appreciate the added transparency over the past few releases.
At least now we can all see what's going on when a release is late by a
few weeks, thanks to these emails.
That said, I think the process might be due for another incremental
revamp. At least from my external point of view, I see these areas for
improvement:
1) Releases are still regularly late. Out of the last six major releases
(Go 1.11 to 1.16), the earliest was a week late (1.15 on August
11th), and the latest was a month late (1.13 on September 3rd).
2) Due to the above, trees tend to reopen at least 2-3 weeks late, which
can be longer if a release has complex "early changes" that get
priority. This means our release cycle is coser to a "two month
development, four month freeze" than the documented 3-3 month split -
at least for the majority not working on the set of "early changes".
3) Both points make the "release cycle" wiki page somewhat misleading.
The only way it's currently useful is if one reads it with the lens
of: ideally this is the calendar we follow, but in practice assume
the final release and tree reopening is a couple of weeks later.
4) That last point makes external contribution harder. Many companies
plan weeks ahead, e.g. sprints, so it's hard to allocate time for
upstreaming Go changes or testing Go betas/RCs if the work might be
blocked by anywhere from 1 to 4 weeks. For example, I had allocated
some time this week to upgrade many modules at work to Go 1.17.
It's hard for me to really see a pattern around why we tend to be a
couple of weeks behind schedule. At least in the last couple of
releases, we've released beta1 within a week, but the delay has quickly
built up with the RCs leading up to the final release. So it does seem
like something goes wrong whenever it's time to follow the RC calendar.
Perhaps one month between RC and final release is too optimistic?
Multiple release blocker issues tend to be found after each RC1,
when Google and others start using the new version in production.
Those issues tend to take at least a couple of weeks to iron out,
and extra RCs want 2-4 weeks before the final release.
If that sounds like a reasonable approximation of what's happening with
most releases, perhaps one way to mitigate this would be to aim for
beta1 and rc1 to be out a week ahead of schedule, giving the "production
testing" at least five weeks. I think that would be a reasonable change
to pull off, given that we've been stricter with "late merges".
Perhaps we'd need to move in an entirely different direction as well,
like pushing harder for Go users and companies to test betas and RCs
more aggressively. I'm sure that the team who works on releases is who
ultimately has the best view on the causes and solutions. My intuition
is that we'll want a mix of incremental steps.
Have a good summer!
On Thu, Jul 15, 2021 at 13:10:50 -0700, Alex Rakoczy wrote:
> Hello Gophers,
>
> This week we saw the Go 1.17 RC1 release! Thank you to everyone who helped
> make this happen.
>
> There are still 41 open issues in the Go 1.17 milestone
> 1 of which is a non-recurring release blocker
> <https://github.com/golang/go/issues?q=is%3Aopen+is%3Aissue+milestone%3AGo1.17+label%3Arelease-blocker>.
Alex Rakoczy
Aug 9, 2021, 3:35:24 PM8/9/21
to Daniel Martí, golang-dev
Hi Daniel,
First of all, thank you for your thoughtful email. I appreciate the
time and effort you put into your feedback.
I agree that it is hard to see a pattern of why we are typically one
or two weeks behind on a given release, as each release tends to come
with its own unique challenges.
A delayed release isn't all bad news. We're very happy to catch these
release-blocking issues before a final release, and will always delay
the release to ensure we fix any critical bugs.
This release is on track to be about one month after the first RC
(July 10) to an estimated August 16th, which gives two weeks of soak
time after RC2. In the case of this release, the first RC was delayed
in part due to taking a much needed break in the beginning of July,
announced as "quiet weeks". I don't foresee future quiet weeks
delaying us, however this one was special in that many people took a
much needed break from a stressful year.
As you know, the RC marks when we release Go to Google. We aim to cut
the release 4 weeks after the RC in order to ensure that there is
enough time for new software at Google to be deployed with the latest
version of Go. When we find an issue that needs to be fixed, it means
we need to cut a new RC and give that enough time to be deployed to
production as well.
Once the first RC was out, we ran into a challenging issue that was
difficult to predict or reproduce, https://golang.org/issue/47441.
It's hard to say how we could have handled this particular issue
better, as it took time to investigate, form a proper fix, then also
roll out to Google. While the issue has since been resolved, we're
still investigating this across the team.
I'm not sure that we'd want to extend our planned RC window, as I feel
that 4 weeks is an accurate amount of time that we'd like to dedicate
to RC testing.
I agree that we should keep improving our process, and have a few
ideas on how we can address each point.
For scheduling the tree opening, we could consider opening the tree
before the final release, as we have a branch created as part of the
RC. There's some challenges if we have to backport fixes to the
release branch, but it's not insurmountable. The biggest risk is
distracting from fixing RC issues, but I don't see anyone mistaking
priorities.
As far as finding tricky, critical bugs that only appear when running
under production workloads, we have several ideas: One is improving
our testing to better detect preemption related issues in the runtime.
We have some ideas bouncing around, but nothing concrete yet. Another
idea may involve releasing to Google earlier outside of the RC cycle,
which should help us discover these issues earlier in the cycle.
As you mentioned, we love the idea of Go users and companies testing
betas and RCs. The feedback is incredibly valuable, especially for
learning about edge cases in the runtime and the go command.
I think that improving our communication after the first RC, as well
as the possible process changes to tree-opening and internal releases,
will be a step in the right direction to improving this.
I’d love to hear your thoughts, and please let me know if I’ve failed
to address any of your points.
Thanks again!
Alex
First of all, thank you for your thoughtful email. I appreciate the
time and effort you put into your feedback.
I agree that it is hard to see a pattern of why we are typically one
or two weeks behind on a given release, as each release tends to come
with its own unique challenges.
A delayed release isn't all bad news. We're very happy to catch these
release-blocking issues before a final release, and will always delay
the release to ensure we fix any critical bugs.
This release is on track to be about one month after the first RC
(July 10) to an estimated August 16th, which gives two weeks of soak
time after RC2. In the case of this release, the first RC was delayed
in part due to taking a much needed break in the beginning of July,
announced as "quiet weeks". I don't foresee future quiet weeks
delaying us, however this one was special in that many people took a
much needed break from a stressful year.
As you know, the RC marks when we release Go to Google. We aim to cut
the release 4 weeks after the RC in order to ensure that there is
enough time for new software at Google to be deployed with the latest
version of Go. When we find an issue that needs to be fixed, it means
we need to cut a new RC and give that enough time to be deployed to
production as well.
Once the first RC was out, we ran into a challenging issue that was
difficult to predict or reproduce, https://golang.org/issue/47441.
It's hard to say how we could have handled this particular issue
better, as it took time to investigate, form a proper fix, then also
roll out to Google. While the issue has since been resolved, we're
still investigating this across the team.
I'm not sure that we'd want to extend our planned RC window, as I feel
that 4 weeks is an accurate amount of time that we'd like to dedicate
to RC testing.
I agree that we should keep improving our process, and have a few
ideas on how we can address each point.
For scheduling the tree opening, we could consider opening the tree
before the final release, as we have a branch created as part of the
RC. There's some challenges if we have to backport fixes to the
release branch, but it's not insurmountable. The biggest risk is
distracting from fixing RC issues, but I don't see anyone mistaking
priorities.
As far as finding tricky, critical bugs that only appear when running
under production workloads, we have several ideas: One is improving
our testing to better detect preemption related issues in the runtime.
We have some ideas bouncing around, but nothing concrete yet. Another
idea may involve releasing to Google earlier outside of the RC cycle,
which should help us discover these issues earlier in the cycle.
As you mentioned, we love the idea of Go users and companies testing
betas and RCs. The feedback is incredibly valuable, especially for
learning about edge cases in the runtime and the go command.
I think that improving our communication after the first RC, as well
as the possible process changes to tree-opening and internal releases,
will be a step in the right direction to improving this.
I’d love to hear your thoughts, and please let me know if I’ve failed
to address any of your points.
Thanks again!
Alex
> --
> You received this message because you are subscribed to the Google Groups "golang-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/20210809095123.htoomewvsmwyaug4%40carbon.localhost.
> You received this message because you are subscribed to the Google Groups "golang-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
Daniel Martí
Aug 9, 2021, 5:51:36 PM8/9/21
to Alex Rakoczy, golang-dev
Hi Alex, and thanks for the quick reply!
> As you know, the RC marks when we release Go to Google. We aim to cut
> the release 4 weeks after the RC in order to ensure that there is
> enough time for new software at Google to be deployed with the latest
> version of Go. When we find an issue that needs to be fixed, it means
> we need to cut a new RC and give that enough time to be deployed to
> production as well.
>
> Once the first RC was out, we ran into a challenging issue that was
> difficult to predict or reproduce, https://golang.org/issue/47441.
> It's hard to say how we could have handled this particular issue
> better, as it took time to investigate, form a proper fix, then also
> roll out to Google. While the issue has since been resolved, we're
> still investigating this across the team.
That makes sense. I think we have to work with the assumption that most
releses will have at least one challenging issue that doesn't get
spotted until production testing begins.
I'm particularly worried about upcoming releases, because generics will
bring far more invasive changes than we've seen for a while. And, as the
project grows, I also imagine we'll tend to merge more changes into each
release.
> For scheduling the tree opening, we could consider opening the tree
> before the final release, as we have a branch created as part of the
> RC. There's some challenges if we have to backport fixes to the
> release branch, but it's not insurmountable. The biggest risk is
> distracting from fixing RC issues, but I don't see anyone mistaking
> priorities.
That's a good idea, and I believe recent releases have already opened
the tree a few days before the final release. Though it's worth
remembering that it's just a minor mitigation to one of the symptoms.
> Another idea may involve releasing to Google earlier outside of the RC
> cycle, which should help us discover these issues earlier in the cycle.
That's what I was aiming for when I mentioned we could aim to release
rc1 earlier - to have an extra week or two to discover and iron out
last-minute tricky issues. I don't think the details matter as long as
we have the extra time to help prevent delays.
> As you mentioned, we love the idea of Go users and companies testing
> betas and RCs. The feedback is incredibly valuable, especially for
> learning about edge cases in the runtime and the go command.
>
> I think that improving our communication after the first RC, as well
> as the possible process changes to tree-opening and internal releases,
> will be a step in the right direction to improving this.
A big thumbs up to better communication. I think each release should
gain two "calls to action" to drive that involvement:
1) When beta1 is out, email golang-nuts and post a tweet telling people
about the main great new features in the release, asking users to
download the release and *run all their tests on it*.
I don't think it's fair to ask downstream users to start deploying
beta1 to production, but running their test suites (e.g. as an extra
job on CI, or once on a laptop) could already flag plenty of
regressions a whole two months before the final release is due.
2) When rc1 is out, send a similar email as before, and now asking
enterprises and personal users to start running it in one of their
live environments, ideally production.
And the reasons why they should do that will be made clear:
* Google has been running it on production for a week
* Early widespread testing will ensure a release on time
* A late release will make the next merge window smaller
The last two points, I think, will help convince many downstreams to do
their part. I know plenty of enterprise users who simply "can't wait"
for a particular new feature to release, like go:embed or generics.
We should make it clear that less testing contributes to a later release
and a smaller changelog in the following release.
Plus, for many downstreams it can be hard to contribute during the
freeze months, unless they have specific experience with a release bug
or its fix. The testing would be invaluable to the project, and allow
third parties to get involved by just investing a couple of hours.
I think both emails should also encourage enterprise downstreams to
reply in-thread with their experience, even if it's just to say "we've
been running it for two weeks and there are no problems". That should
help you gauge how many people are helping, and what rough portion of
them are running into issues.
I'll let you get back to the release work now :)
> As you know, the RC marks when we release Go to Google. We aim to cut
> the release 4 weeks after the RC in order to ensure that there is
> enough time for new software at Google to be deployed with the latest
> version of Go. When we find an issue that needs to be fixed, it means
> we need to cut a new RC and give that enough time to be deployed to
> production as well.
>
> Once the first RC was out, we ran into a challenging issue that was
> difficult to predict or reproduce, https://golang.org/issue/47441.
> It's hard to say how we could have handled this particular issue
> better, as it took time to investigate, form a proper fix, then also
> roll out to Google. While the issue has since been resolved, we're
> still investigating this across the team.
releses will have at least one challenging issue that doesn't get
spotted until production testing begins.
I'm particularly worried about upcoming releases, because generics will
bring far more invasive changes than we've seen for a while. And, as the
project grows, I also imagine we'll tend to merge more changes into each
release.
> For scheduling the tree opening, we could consider opening the tree
> before the final release, as we have a branch created as part of the
> RC. There's some challenges if we have to backport fixes to the
> release branch, but it's not insurmountable. The biggest risk is
> distracting from fixing RC issues, but I don't see anyone mistaking
> priorities.
the tree a few days before the final release. Though it's worth
remembering that it's just a minor mitigation to one of the symptoms.
> Another idea may involve releasing to Google earlier outside of the RC
> cycle, which should help us discover these issues earlier in the cycle.
rc1 earlier - to have an extra week or two to discover and iron out
last-minute tricky issues. I don't think the details matter as long as
we have the extra time to help prevent delays.
> As you mentioned, we love the idea of Go users and companies testing
> betas and RCs. The feedback is incredibly valuable, especially for
> learning about edge cases in the runtime and the go command.
>
> I think that improving our communication after the first RC, as well
> as the possible process changes to tree-opening and internal releases,
> will be a step in the right direction to improving this.
gain two "calls to action" to drive that involvement:
1) When beta1 is out, email golang-nuts and post a tweet telling people
about the main great new features in the release, asking users to
download the release and *run all their tests on it*.
I don't think it's fair to ask downstream users to start deploying
beta1 to production, but running their test suites (e.g. as an extra
job on CI, or once on a laptop) could already flag plenty of
regressions a whole two months before the final release is due.
2) When rc1 is out, send a similar email as before, and now asking
enterprises and personal users to start running it in one of their
live environments, ideally production.
And the reasons why they should do that will be made clear:
* Google has been running it on production for a week
* Early widespread testing will ensure a release on time
* A late release will make the next merge window smaller
The last two points, I think, will help convince many downstreams to do
their part. I know plenty of enterprise users who simply "can't wait"
for a particular new feature to release, like go:embed or generics.
We should make it clear that less testing contributes to a later release
and a smaller changelog in the following release.
Plus, for many downstreams it can be hard to contribute during the
freeze months, unless they have specific experience with a release bug
or its fix. The testing would be invaluable to the project, and allow
third parties to get involved by just investing a couple of hours.
I think both emails should also encourage enterprise downstreams to
reply in-thread with their experience, even if it's just to say "we've
been running it for two weeks and there are no problems". That should
help you gauge how many people are helping, and what rough portion of
them are running into issues.
I'll let you get back to the release work now :)
Carlos Amedee
Aug 11, 2021, 4:33:06 PM8/11/21
to golang-dev
Hello Gophers!
It's been almost two weeks since the Go 1.17 RC 2 release. Thank you all for testing it and reporting issues you've found. It is very helpful to ensure we issue a robust release. We are down to just 3 open issues in the Go1.17 milestone which are release blockers right now, and all of them are minor and manageable.
We agreed in a release meeting today that everything looks good thus far for us to target the coming Monday August 16th for the final Go 1.17 release.
As part of making the final release, we will also be opening the tree—only for early-in-cycle Go1.18 changes at first—and then generally to all Go 1.18 changes. We'll start the usual "Planning Go 1.18" thread with more information on the exact dates, and you can follow issue 47351 that tracks our progress to re-open the tree.
Thank you everyone for your help with this release.
Thanks,
Carlos
Reply all
Reply to author
Forward
0 new messages