Planning Go 1.15

1,813 views
Skip to first unread message

Dmitri Shuralyov

unread,
Feb 19, 2020, 2:02:44 PM2/19/20
to golang-dev

The Go 1.14 Release Candidate 1 has been in testing since Feb 5, and there are now no more release blocking issues that need to be investigated or resolved. We are planning to make the final Go 1.14 release early next week.


The Go 1.15 development tree will open soon on the master branch. As we have done with previous releases, we will first open the tree for changes that need to be made early in the cycle, and soon after that, more generally for all changes. We will provide information about the exact dates in this thread, please read on.


The release cycle document states “if a release is delayed, so is work on the next release.” The 1.14 release cycle is behind schedule, which cuts the development cycle of 1.15 shorter for everyone. This is unfortunate, but it was necessary to ensure known regressions in 1.14 could be investigated and resolved.


We on the Go release team (Alex, Carlos, and Dmitri) want to start taking steps now to reduce the chances of this pattern repeating itself in the upcoming 1.15 cycle. As part of that, we are making the following requests for all Go developers who will be submitting changes once the tree opens up:


  1. Risky Changes

    A change is risky if it can cause failures that are hard to diagnose (for example, changes to the runtime, GC, compiler, linker, TLS, other low-level component, or complex changes that need soak time under production workloads), or if it requires many CLs that are hard to revert (for example, large CLs or stacks of CLs).

    If you’re planning on working on a change that may be risky, please do the following:

    1. Unless the entire change is absolutely trivial to revert, protect the new code paths with a boolean flag, prefixed with “go115”, that can be used to quickly toggle back to the old implementation. It can be a simple bool constant, for example, const go115UseNewLinker = true. The flag *must be findable* by a simple grep for the string “go115”, so that we can find and clean up all such flags when we get to the Go 1.16 cycle.

    2. Consider how you would answer the following questions for your change: How risky is the change you're planning to make? How will you know if it is working as intended? How much production testing does it need for you to be confident it is working as intended? At what time should the keep/revert decision be made?

    3. Create a tracking issue with Go 1.15 milestone and release-blocker label using the golang.org/s/riskychange issue template. This issue will be used to track progress on the feature and make the final decision for Go 1.15.

  2. Revert Sooner

    When a change that landed on master results in a confirmed failure or unintended behavior change, we ask everyone to revert that change sooner, instead of letting it stay merged. For example, if we learn that a change in unspecified behavior of a function like url.Parse causes breakage, then it is better to roll back while thinking about how to address the issue.

    This helps ensure master is more stable, making it easier for people who are developing other changes.

  3. Freeze Exceptions

    Any exceptions to the freeze must be communicated to and explicitly approved by the Go release team before the freeze. If you’d like to request an exception, please start a thread on golang-dev. We will address any requests on a case-by-case basis with a strong preference for not permitting changes after the freeze.

    Changes that are not in line will be reverted on master and asked to be re-sent during next cycle.

  4. Check build.golang.org Before Submitting

    Before starting to submit your CL or a stack of CLs, please check that the tip is green at https://build.golang.org. If there is a widespread test failure, please wait, or better, help with efforts to diagnose and fix the failure.


We are making these requests with the goal of making the 1.15 cycle more predictable, and helping keep the tree in a healthier state throughout the development cycle. Thank you for the hard work everyone does to ensure each Go release is as good as it can be; it is really appreciated.


With the above requests in mind, we plan to open the tree for changes that should land early-in-cycle later today (Feb 19). We’ll send another email announcing that shortly. If your change can wait a few more days, please hold off until we open the tree generally on Friday (Feb 21).


In the meantime, please feel free to use this thread to share and discuss any work you’re planning to do or would like to do for Go 1.15.


(These are things you *PLAN TO DO YOURSELF*, not things you want other people to do.)

Dmitri Shuralyov

unread,
Feb 19, 2020, 2:29:31 PM2/19/20
to golang-dev
An update,

The Go 1.15 development tree is now open for early-in-cycle changes. If your change needs to merged early in the Go 1.15 cycle, please proceed carefully, keeping the requests we’ve made above in mind.

If your change does not need to land early in the cycle and can wait until Friday (Feb 21), please wait until then. We will update this thread once the Go 1.15 tree is open for all changes.

Thanks,
Dmitri for the Go release team

Josh Bleecher Snyder

unread,
Feb 19, 2020, 3:51:54 PM2/19/20
to Dmitri Shuralyov, golang-dev
I have a large, almost entirely reviewed stack of changes to the compiler’s rule generation code. Last change in the stack: 
https://go-review.googlesource.com/c/go/+/217014. It significantly reduces the time and memory required to build the compiler and shrinks the executable.

It doesn’t strictly speaking need to go in early, but I’m concerned about having to struggle through a bunch of unpleasant merge conflicts if it doesn’t. (Just rebasing it during the freeze hasn’t been fun.)

Is it OK for me to start getting it submitted now?

Thanks for your efforts to keep 1.15 on time!

-josh




--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/CA%2BON-PFgRYRHAMNpEcG1fRqmkZSxKfON6HWdiF3K0ntDh5hkkA%40mail.gmail.com.

Bryan C. Mills

unread,
Feb 19, 2020, 4:23:16 PM2/19/20
to golang-dev, Dmitri Shuralyov, Jay Conrod, mat...@golang.org
My first major order of business is to figure out if we can feasibly implement lazy module loading (https://golang.org/issue/36460).

After that, I plan to revisit module replacements and aliasing (https://golang.org/issue/26904) and ambiguous import resolution (https://golang.org/issue/27899).

David Chase

unread,
Feb 19, 2020, 4:26:13 PM2/19/20
to Josh Bleecher Snyder, Dmitri Shuralyov, golang-dev
I approve Josh's rulegen changes going in sooner rather than later, and I just also cleared it with Cherry.


On Wed, Feb 19, 2020 at 3:51 PM Josh Bleecher Snyder <josh...@gmail.com> wrote:

Daniel Martí

unread,
Feb 19, 2020, 5:17:01 PM2/19/20
to golang-dev
> In the meantime, please feel free to use this thread to share and discuss
> any work you’re planning to do or would like to do for Go 1.15.

I have quite a few encoding/json CLs from last cycle that have been
wandering around for over half a year, just because I can't find anyone
to give them a +2. I really need to push for those in the next month, or
otherwise they'll get delayed for another release.

I also have a WIP CL to rewrite go/parser with
cmd/compile/internal/syntax; see https://golang.org/cl/208637. I'll
continue the work during 1.15, but I expect it to remain an experiment
this cycle.

I've also been thinking about a rewrite of the json tokenizer ever since
I finished a round of json performance work in 1.13. I figure that, with
a proper refactor, we could see struct decoding being sped up by 10-20%.
I'm not sure how to go about this, though. Should I file a proposal?
Email the other three json owners? Just send a massive CL directly?
Ideas or input welcome.

I've also been filing a few ideas and proposals related to tooling in
the past few weeks, and intend to file a few more and continue
participating in https://github.com/golang/go/wiki/golang-tools.

Aside from the above, I intend to take it easy this cycle.

Akhil Indurti

unread,
Feb 20, 2020, 2:36:12 AM2/20/20
to golang-dev
I have a WIP CL for http://golang.org/issue/30951, namely https://go-review.googlesource.com/c/tools/+/197937/, that I'll try to update and land early-in-cycle.

Andrew Bonventre

unread,
Feb 20, 2020, 11:06:45 AM2/20/20
to David Chase, Josh Bleecher Snyder, Dmitri Shuralyov, golang-dev
For Josh’s rulegen changes, it is a perfect example of a change that should be gated by a constant per Dmitri’s mail above. I have no issue with it landing early provided it’s gated properly.

Josh Bleecher Snyder

unread,
Feb 20, 2020, 11:25:05 AM2/20/20
to Andrew Bonventre, David Chase, Dmitri Shuralyov, golang-dev
I considered that, but I don’t see a plausible way to do it.

The entire purpose of the change is to shrink the compiler. Gating it by a constant would significantly increase the size of the compiler, since we’d have to keep two copies of all the code around.

Plus it would create a code maintenance headache. It’d mean duplicating large sections of the compiler. (It’s the very reason that rebasing it is so painful.)

And for the majority of the changes, particularly the intricate/hard-to-revert ones, I have checked that the compiler generates the identical code before and after.

-josh

Than McIntosh

unread,
Feb 20, 2020, 11:37:59 AM2/20/20
to Josh Bleecher Snyder, Andrew Bonventre, David Chase, Dmitri Shuralyov, golang-dev
FYI, I'd like to add that we're in the same boat with respect to the linker modernization work currently underway.  There is no practical way to gate the new behavior under a flag unless we hold onto all of the old code that we're trying to delete.

About the only option we can think of at the moment to provide an on/off gate would be to simple "cp -r" the existing inker sources and install the old tool under "link.old", or something to this effect.

If we were to go that route, however, it means that from the time we create the fork until we remove the ability to gate on/off the modernized linker, we have to make bug fixes in two places, which seems like an unpleasant prospect.

Thanks, Than

David Chase

unread,
Feb 20, 2020, 11:46:05 AM2/20/20
to Josh Bleecher Snyder, Andrew Bonventre, Dmitri Shuralyov, golang-dev
I agree, this is not something that we can easily switch on and off.  We should just do it early and test the bleep out of it.

Austin Clements

unread,
Feb 20, 2020, 3:09:50 PM2/20/20
to Josh Bleecher Snyder, Andrew Bonventre, David Chase, Dmitri Shuralyov, golang-dev
I think Josh's changes are a great example of the limitations of gating. His changes are basically infeasible to gate, but I think that's okay: it just means we need to work harder to reduce their risk and consider their revert strategy. Two ways to do that come to mind: One is really good testing. In this case, Josh has confirmed that the compiler generates identical code before and after for most of the changes. This is a very strong risk mitigator in the compiler because the subtle bugs that make compiler changes often risky have to do with incorrect code generation. The other way to reduce risk is for failures to be obvious. In this case, it's a large CL stack, but large CL stacks are not hard to revert right after they go in. They become hard to revert as things later get built on them. Hence, if the nature of a change is that it would cause immediate, obvious failures, we can revert it right away, so I think that also significantly mitigates risk.

I'm absolutely for gating risky changes, but there may also be a danger of getting too caught up in mechanism. The point is to reduce risk and increase stability, so I think the most critical thing is that we consciously and intentionally reason through the risk/revert strategy, and use feature gates as a tool in that.

Austin Clements

unread,
Feb 20, 2020, 3:18:47 PM2/20/20
to Than McIntosh, Josh Bleecher Snyder, Andrew Bonventre, David Chase, Dmitri Shuralyov, golang-dev
I think the "cp -r" technique to gate the linker is actually pretty reasonable. You have a good point about bug fixes, but risk mitigation doesn't come without a cost. It's an interesting question, though: when we merge in the new linker, the intent is to switch to it then, so maybe we stop fixing bugs in the old linker at that point? These gating flags are *not* meant to be knobs, they're meant to be "in case of emergency, break glass."


Andrew Bonventre

unread,
Feb 20, 2020, 5:08:06 PM2/20/20
to Austin Clements, Than McIntosh, Josh Bleecher Snyder, David Chase, Dmitri Shuralyov, golang-dev
Our goal with these efforts is to get everyone (not singling you out, Josh) to consider risk mitigations more rigorously than they have in the past when landing their changes regardless of where we are in the cycle. Extensive testing of the compiler changes is one way to do this in lieu of a gate, however, I would ask that we be mindful when landing large stacks without gates to _not_ land follow-up CLs that would make reverting the stack much more difficult until after we’re sure the builders are green. It may take longer to land other CLs, but that’s also the point of this initial development period.

What I don’t want is for us to repeat what we’ve done in the past expecting different results with 1.15. We’ve tended to use the development period as an opportunity to introduce technical debt (“I’ll fix it during the freeze”) and we, the release team, will be pushing against that much more rigorously going forward.

Josh Bleecher Snyder

unread,
Feb 20, 2020, 6:22:36 PM2/20/20
to Andrew Bonventre, Austin Clements, Than McIntosh, David Chase, Dmitri Shuralyov, golang-dev
I want to second both of Austin's emails and add a few additional thoughts.

I suspect (but can't prove) that the most problematic cases for
release timeliness aren't the ones that immediately break a builder;
we can just revert those. And I suspect (but can't prove) that the
most problematic cases for release timeliness often aren't the
obviously high risk changes, in part because the authors of obviously
high risk changes tend to take extra time and care testing them and
providing mitigation strategies.

I suspect that the most problematic cases are those that seem safe
and/or very well tested, yet which cause subtle and/or rare bugs,
often due to unexpected interactions with other moving parts. And I'm
not sure that we have a great track record at predicting those (which
is not to say we shouldn't try). And I'm not sure that the existence
of gates in these cases necessarily buys us that much: We had a gate
for signal-based preemption, but still lost a bunch of time
discovering and working around a kernel bug. So it's not obvious to me
that an emphasis on gating is going to have a good ROI.

Here are two additional (possibly unpopular) suggestions for different
things that we could try this cycle. (I often wish mailing lists had
emoji response counters, but in this case, I'm probably better off
without!) Maybe they'll spark other ideas...

(1) We could switch to a different set of Gerrit permissions during
the freeze, on the first day, such that only a small handful of people
can +2 or submit changes. It'd no doubt be painful at first, but
taking the reins much more firmly during the freeze will strongly
encourage us to go into the freeze in better shape.

(2) Issue an alpha release the day the freeze begins, rain or shine.
One reason is that it would encourage getting API feedback and finding
obvious issues (like backwards incompatibility) sooner. The second and
more important reason is to signal that the tree needs to be stable
going into the freeze.

As to the new linker, will the output be byte-for-byte identical, or
are there other changes as well? If the output is identical, then
there's lots of additional benefit available with the "cp -r"
strategy: for a while, cmd/go could run both and report any
mismatches. I believe Russ did this during the c2go transition to good
effect, and I did it locally during my concurrent compilation work.

-josh

P.S. On a personal note, you may have noticed that I've been
submitting my changes a few at a time (depending on their exact
contents), to ensure that the dashboard is green before continuing.
This is my usual practice when I have lots of changes to submit.

Dave Cheney

unread,
Feb 20, 2020, 6:25:03 PM2/20/20
to Josh Bleecher Snyder, Andrew Bonventre, Austin Clements, Than McIntosh, David Chase, Dmitri Shuralyov, golang-dev
Strong emoji thumbs up for an alpha release the day of the freeze.

> On 21 Feb 2020, at 10:22, Josh Bleecher Snyder <josh...@gmail.com> wrote:
>
> I want to second both of Austin's emails and add a few additional thoughts.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/CAFAcib9%3Dp1G%3D-D%3D-GVkFEu-XATEkryU4vFxdTav0eTr1U0FZ-A%40mail.gmail.com.

David Riley

unread,
Feb 21, 2020, 1:01:44 AM2/21/20
to Austin Clements, Than McIntosh, Josh Bleecher Snyder, Andrew Bonventre, David Chase, Dmitri Shuralyov, golang-dev
> On Feb 20, 2020, at 3:18 PM, 'Austin Clements' via golang-dev <golan...@googlegroups.com> wrote:
>
> I think the "cp -r" technique to gate the linker is actually pretty reasonable. You have a good point about bug fixes, but risk mitigation doesn't come without a cost. It's an interesting question, though: when we merge in the new linker, the intent is to switch to it then, so maybe we stop fixing bugs in the old linker at that point? These gating flags are *not* meant to be knobs, they're meant to be "in case of emergency, break glass."

If the unit and regression tests don't catch catastrophic breakages like the ones feared here, why have them? This is what they're for, and the Go compiler has a pretty good set of tests as things go. At some point, especially for something like this, you need to be able to rely on the tooling built up over many years.

In any case, the solution is not to drag the bloated corpse of legacy code around. Make the change, get it well tested, and add new tests if anything new regresses.


- Dave


Bryan C. Mills

unread,
Feb 21, 2020, 11:27:31 AM2/21/20
to Josh Bleecher Snyder, Andrew Bonventre, Austin Clements, Than McIntosh, David Chase, Dmitri Shuralyov, golang-dev
On Thu, Feb 20, 2020 at 6:22 PM Josh Bleecher Snyder <josh...@gmail.com> wrote:
I want to second both of Austin's emails and add a few additional thoughts.

I suspect (but can't prove) that the most problematic cases for
release timeliness aren't the ones that immediately break a builder;
we can just revert those. And I suspect (but can't prove) that the
most problematic cases for release timeliness often aren't the
obviously high risk changes, in part because the authors of obviously
high risk changes tend to take extra time and care testing them and
providing mitigation strategies.

My impression from the past few releases is that the problematic cases fall into two categories:

1. Regressions that occur only in particular environments or under particular modes of use.
Some examples:
https://golang.org/issue/36752 (k8s performance regression with 1.14)
https://golang.org/issue/33850 (http.Transport leak when MaxConnsPerHost is set and connections are never broken)
• https://golang.org/issue/36557 (PowerRegisterSuspendResumeNotification failure under Docker on Windows)
• https://golang.org/issue/34094 ('go get' failed for GitLab subgroups due to go-import tags referring to nonexistent repos)

These are typically due to changes that optimized or fixed some particular case, and missed a regression for some other case due to either lack of awareness of the use-case or lack of test or benchmark coverage.
Those are high-risk changes, known to be high-risk, but hard to mitigate — both because of the long delay between when the regression is introduced and when it is detected, and because a simple revert would reintroduce some other problem.
But “extra time and care testing them” does not provide much mitigation for these: if we don't know about some particular use-case, or aren't aware that our test coverage for that use-case is lacking, then we also won't know to take extra care to test it.

But, we also have...

2. Correctness bugs that are detected early, but are not reverted (either because they are difficult to revert or difficult to diagnose without ongoing builder feedback, or because the feature is important to the release), and not addressed quickly because the owners need to prioritize fixes for other regressions that have also not been reverted.
Some examples:
• https://golang.org/issue/35775 was filed on Nov. 22, shortly after it was introduced (on Nov. 20), but the regression was not fixed until Jan. 9.
• https://golang.org/issue/35470 was detected on Nov. 8 (from builder failures starting on Nov. 5), but was not fixed until Nov. 26.
• https://golang.org/issue/35271 was filed on Oct. 31, but was not fixed until Nov. 27. In the meantime, its test failures masked a second regression (https://golang.org/issue/35461) on Nov. 7.

(Note that the 1.14 code freeze formally began on November 4.)


I suspect that the most problematic cases are those that seem safe
and/or very well tested, yet which cause subtle and/or rare bugs,
often due to unexpected interactions with other moving parts. And I'm
not sure that we have a great track record at predicting those (which
is not to say we shouldn't try). And I'm not sure that the existence
of gates in these cases necessarily buys us that much: We had a gate
for signal-based preemption, but still lost a bunch of time
discovering and working around a kernel bug. So it's not obvious to me
that an emphasis on gating is going to have a good ROI.

If we have true gates, and we plan for them ahead of time, then we can use those gates to disable a regression-inducing feature and release without it, rather than delaying the release until the feature can be fixed.

Or, we can at least disable the regression-inducing feature long enough to be able to test the remainder of the release as a stable unit.
For example: we could use the gates to eliminate known regressions immediately at the start of the freeze, and cut a beta that users could begin verifying in a production setting in order to find those subtle, environment-sensitive issues, rather than cutting a beta with known deadlocks and crashes that users may find too unstable to even try, or may find too flaky to bother diagnosing and reporting issues against.


Here are two additional (possibly unpopular) suggestions for different
things that we could try this cycle. (I often wish mailing lists had
emoji response counters, but in this case, I'm probably better off
without!) Maybe they'll spark other ideas...

(1) We could switch to a different set of Gerrit permissions during
the freeze, on the first day, such that only a small handful of people
can +2 or submit changes. It'd no doubt be painful at first, but
taking the reins much more firmly during the freeze will strongly
encourage us to go into the freeze in better shape.

(2) Issue an alpha release the day the freeze begins, rain or shine.
One reason is that it would encourage getting API feedback and finding
obvious issues (like backwards incompatibility) sooner. The second and
more important reason is to signal that the tree needs to be stable
going into the freeze.

In recent release cycles we have gotten very little feedback from beta releases — the folks who are willing to tolerate breakage tend to be running nightly builds, and the folks who prioritize stability wait until the release candidate or the release proper.
And folks who are working in the Go tree should already know that it needs to be stable going into the freeze. That's kind of the point of Dmitri's first message in this thread.

So I don't see much benefit to an additional alpha release. If anything, we need more users to actually try out the betas, and in order to do that I think we need some way to make those betas more stable.


David Chase

unread,
Feb 21, 2020, 11:38:07 AM2/21/20
to David Riley, Austin Clements, Than McIntosh, Josh Bleecher Snyder, Andrew Bonventre, Dmitri Shuralyov, golang-dev
We had several bad performance bugs get past the usual set of regression tests and benchmarks for 1.14, and we have an already-tested (by hindcasting 1.14 performance glitches -- I ran daily benchmarks, but we weren't monitoring them because we weren't sure the information would be useful) plan to reduce the risk of a repeat, but it would not have caught 100% and defense in depth is a good thing.  So I generally approve of the feature flags approach wherever it is not incredibly onerous (as it would be for Josh's compiler rules cleanup).

Josh Bleecher Snyder

unread,
Feb 21, 2020, 12:25:54 PM2/21/20
to Bryan C. Mills, Andrew Bonventre, Austin Clements, Than McIntosh, David Chase, Dmitri Shuralyov, golang-dev
Thanks for the concrete examples, Bryan. Those are very useful.

> And folks who are working in the Go tree should already know that it needs to be stable going into the freeze.

I fully agreed. But in practice, that culture doesn't seem
well-established. And I am a bit worried that we're trying to use
technical tools to fix a problem that is at least in part
non-technical.

-josh

Austin Clements

unread,
Feb 21, 2020, 12:36:06 PM2/21/20
to Josh Bleecher Snyder, Andrew Bonventre, Than McIntosh, David Chase, Dmitri Shuralyov, golang-dev
On Thu, Feb 20, 2020 at 6:22 PM Josh Bleecher Snyder <josh...@gmail.com> wrote:
I suspect that the most problematic cases are those that seem safe
and/or very well tested, yet which cause subtle and/or rare bugs,
often due to unexpected interactions with other moving parts. And I'm
not sure that we have a great track record at predicting those (which
is not to say we shouldn't try). And I'm not sure that the existence
of gates in these cases necessarily buys us that much: We had a gate
for signal-based preemption, but still lost a bunch of time
discovering and working around a kernel bug. So it's not obvious to me
that an emphasis on gating is going to have a good ROI.

I think the signal-based preemption bug was actually a great example of why gating can be useful and important. Even though the root cause ultimately wasn't asynchronous preemption, the fact that we could easily gate it off gave us a good starting point for debugging and was important in implicating signals in the root cause. Some bugs are just really hard! :) But I think having the gate on signal-based preemption actually helped significantly with debugging.

As to the new linker, will the output be byte-for-byte identical, or
are there other changes as well? If the output is identical, then
there's lots of additional benefit available with the "cp -r"
strategy: for a while, cmd/go could run both and report any
mismatches. I believe Russ did this during the c2go transition to good
effect, and I did it locally during my concurrent compilation work.

They're not likely to be byte-for-byte identical, though the differences are likely to be in the order of things where the order doesn't matter. Maybe we could build a smarter comparison tool, but I'm not sure.

-josh

P.S. On a personal note, you may have noticed that I've been
submitting my changes a few at a time (depending on their exact
contents), to ensure that the dashboard is green before continuing.
This is my usual practice when I have lots of changes to submit.

Thank you! I think this is a good policy.

Josh Bleecher Snyder

unread,
Feb 21, 2020, 12:38:38 PM2/21/20
to Austin Clements, Andrew Bonventre, Than McIntosh, David Chase, Dmitri Shuralyov, golang-dev
>> I suspect that the most problematic cases are those that seem safe
>> and/or very well tested, yet which cause subtle and/or rare bugs,
>> often due to unexpected interactions with other moving parts. And I'm
>> not sure that we have a great track record at predicting those (which
>> is not to say we shouldn't try). And I'm not sure that the existence
>> of gates in these cases necessarily buys us that much: We had a gate
>> for signal-based preemption, but still lost a bunch of time
>> discovering and working around a kernel bug. So it's not obvious to me
>> that an emphasis on gating is going to have a good ROI.
>
>
> I think the signal-based preemption bug was actually a great example of why gating can be useful and important. Even though the root cause ultimately wasn't asynchronous preemption, the fact that we could easily gate it off gave us a good starting point for debugging and was important in implicating signals in the root cause. Some bugs are just really hard! :) But I think having the gate on signal-based preemption actually helped significantly with debugging.

Yes, re-reading this thread I somehow came out as being anti-gating,
which is obviously a silly position. :)

-josh

Austin Clements

unread,
Feb 21, 2020, 12:40:56 PM2/21/20
to David Riley, Than McIntosh, Josh Bleecher Snyder, Andrew Bonventre, David Chase, Dmitri Shuralyov, golang-dev
On Fri, Feb 21, 2020 at 1:01 AM David Riley <frave...@gmail.com> wrote:
> On Feb 20, 2020, at 3:18 PM, 'Austin Clements' via golang-dev <golan...@googlegroups.com> wrote:
>
> I think the "cp -r" technique to gate the linker is actually pretty reasonable. You have a good point about bug fixes, but risk mitigation doesn't come without a cost. It's an interesting question, though: when we merge in the new linker, the intent is to switch to it then, so maybe we stop fixing bugs in the old linker at that point? These gating flags are *not* meant to be knobs, they're meant to be "in case of emergency, break glass."

If the unit and regression tests don't catch catastrophic breakages like the ones feared here, why have them?  This is what they're for, and the Go compiler has a pretty good set of tests as things go.  At some point, especially for something like this, you need to be able to rely on the tooling built up over many years.

In part, the linker just doesn't have great tests. It's also not about the catastrophic breakages. Those show up and we fix them. It's about the subtle breakages that show up down the road in some obscure configuration or situation. One of the reasons we don't like knobs is to limit these, but they still exist. In the linker in particular, these often show up when interacting with host objects, where we unavoidably inherit some of the complexity space of the host toolchain.

David Riley

unread,
Feb 21, 2020, 12:50:53 PM2/21/20
to David Chase, Austin Clements, Than McIntosh, Josh Bleecher Snyder, Andrew Bonventre, Dmitri Shuralyov, golang-dev
On Feb 21, 2020, at 11:37 AM, David Chase <drc...@google.com> wrote:
>
> We had several bad performance bugs get past the usual set of regression tests and benchmarks for 1.14, and we have an already-tested (by hindcasting 1.14 performance glitches -- I ran daily benchmarks, but we weren't monitoring them because we weren't sure the information would be useful) plan to reduce the risk of a repeat, but it would not have caught 100% and defense in depth is a good thing. So I generally approve of the feature flags approach wherever it is not incredibly onerous (as it would be for Josh's compiler rules cleanup).

I certainly don't disagree with the feature flags for small things. Especially if it's something regression testing won't catch (e.g. many performance regressions, which are likely to require specific context to trigger). Apologies if it sounded otherwise.


- Dave

Dmitri Shuralyov

unread,
Feb 21, 2020, 1:14:33 PM2/21/20
to David Riley, David Chase, Austin Clements, Than McIntosh, Josh Bleecher Snyder, Andrew Bonventre, golang-dev
Another update,

The Go 1.15 development tree is now open for general work. Please proceed carefully, keeping the requests we’ve made in the original email in mind.

All "wait-release" tags on Gerrit CLs have been removed (they're now "ex-wait-release").


Thanks,
Dmitri for the Go release team

Rémy Oudompheng

unread,
Feb 26, 2020, 5:27:50 AM2/26/20
to Dmitri Shuralyov, golang-dev
Hello,

My goal for 1.15 is to have the 3 strconv CLs ("ryu" patchset) in the tree quite early.

These are CL 170078, 170079, 170080.

Cheers,
Rémy

--

lab...@gmail.com

unread,
Mar 4, 2020, 2:51:59 PM3/4/20
to golang-dev
I have a few minor asm changes to add some power9 instructions and clean up some previous changes that are confusing. I also want to add an enhancement to the objdump as described here https://github.com/golang/go/issues/34372
I want to add a minor enhancement to the PCALIGN directive for ppc64/ppc64le. Due to a recent enhancement in this are, we can now allow larger alignments than when this was first implemented, so I want to add 32, and then use this directive in some asm files.
I'm looking at some updates to the asm for poly1305 on ppc64le to improve its performance by 10% or more. This is already asm but further improvements appear possible. (i.e., not new asm)
In addition to these, I am following some performance enhancements in the community that may have an affect on ppc64le and verify that they do.

I have a question about the ABI change that has been discussed for the past few releases related to parameter and return value passing in registers instead of storage. I have not seen that mentioned recently, is that still in plan? I see that as a big performance improvement on ppc64/ppc64le and my understanding is for arm64 too.

Austin Clements

unread,
Mar 16, 2020, 11:38:55 AM3/16/20
to lab...@gmail.com, golang-dev
On Wed, Mar 4, 2020 at 2:51 PM <lab...@gmail.com> wrote:
I have a question about the ABI change that has been discussed for the past few releases related to parameter and return value passing in registers instead of storage. I have not seen that mentioned recently, is that still in plan? I see that as a big performance improvement on ppc64/ppc64le and my understanding is for arm64 too.

That particular ABI change isn't likely to happen for 1.15. However, we are trying a smaller ABI change to flesh out the dual ABI system (which has been in place for a while now, but since the two ABIs are still the same, definitely has a few cut corners we need to fix) and are looking into a higher-level SSA call representation, which should make it easier to move the calling convention implementation into architecture lowering.

Daniel Theophanes

unread,
Mar 16, 2020, 11:40:36 PM3/16/20
to golang-dev
(These are things you *PLAN TO DO YOURSELF*, not things you want other people to do.)

I would like to get the following CL chain merged for database/sql for Go1.15. I need someone to review them.
 

Daniel Theophanes

unread,
Apr 4, 2020, 10:48:08 AM4/4/20
to golang-dev
I'm sorry to bother everyone, but could I also get a review on the rest of the CL chain?


The first two especially really need to make it into go1.15.
Thanks, -Daniel
Reply all
Reply to author
Forward
0 new messages