Planning Go 1.16

[Andrew Bonventre]

Hi folks,

The Go 1.16 development tree will open soon on the master branch. As we have done with previous releases, we will first open the tree for changes that need to be made early in the cycle, and soon after that, more generally for all changes. We will provide information about the exact dates in this thread, please read on.

As with Go 1.15, we are making the following requests for all Go developers who will be submitting changes once the tree opens up:

1. Risky Changes
   
   A change is risky if it can cause failures that are hard to diagnose (for example, changes to the runtime, GC, compiler, linker, TLS, other low-level component, or complex changes that need soak time under production workloads), or if it requires many CLs that are hard to revert (for example, large CLs or stacks of CLs).
   
   If you’re planning on working on a change that may be risky, please do the following:
   
   

1. Unless the entire change is absolutely trivial to revert, protect the new code paths with a boolean flag, prefixed with “go116”, that can be used to quickly toggle back to the old implementation. It can be a simple bool constant, for example, const go116UseEvenBetterLinker = true. The flag *must be findable* by a simple grep for the string “go116”, so that we can find and clean up all such flags when we get to the Go 1.17 cycle.

2. Consider how you would answer the following questions for your change: How risky is the change you're planning to make? How will you know if it is working as intended? How much production testing does it need for you to be confident it is working as intended? At what time should the keep/revert decision be made?

3. Create a tracking issue with Go 1.16 milestone and release-blocker label using the golang.org/s/riskychange issue template. This issue will be used to track progress on the feature and make the final decision for Go 1.16.
   
   

2. Revert Sooner
   
   When a change that landed on master results in a confirmed failure or unintended behavior change, we ask everyone to revert that change sooner, instead of letting it stay merged. For example, if we learn that a change in unspecified behavior of a function like url.Parse causes breakage, then it is better to roll back while thinking about how to address the issue.
   
   This helps ensure master is more stable, making it easier for people who are developing other changes.
   
   

3. Freeze Exceptions
   
   Any exceptions to the freeze must be communicated to and explicitly approved by the Go release team before the freeze. If you’d like to request an exception, please start a thread on golang-dev or file an issue in the issue tracker with “[freeze exception]” as a suffix (example). We will address any requests on a case-by-case basis with a strong preference for not permitting changes after the freeze.
   
   Changes that are not in line will be reverted on master and asked to be re-sent during the next cycle.
   
   

4. Check build.golang.org Before Submitting
   
   Before starting to submit your CL or a stack of CLs, please check that the tip is green at https://build.golang.org. If there is a widespread test failure, please wait, or better, help with efforts to diagnose and fix the failure.

Thank you for the hard work everyone does to ensure each Go release is as good as it can be; it is really appreciated.

With the above requests in mind, we plan to open the tree for changes that should land early-in-cycle tomorrow (2020-08-11). We’ll send another email announcing that shortly. If your change can wait a few more days, please hold off until we open the tree generally on Thursday (2020-08-13).

In the meantime, please feel free to use this thread to share and discuss any work you’re planning to do or would like to do for Go 1.16.

(These are things you *PLAN TO DO YOURSELF*, not things you want other people to do.)

[Andrew Bonventre]

As an update, we plan to open the tree tomorrow (Wednesday) for risky changes, with the tree generally open on Friday.

[Daniel Martí]

On Mon, Aug 10, 2020 at 20:27:26 -0400, Andrew Bonventre wrote:
> In the meantime, please feel free to use this thread to share and discuss
> any work you’re planning to do or would like to do for Go 1.16.

I didn't do anything too fancy in 1.15 beyond maintaining json and
participating in golang-tools; I'll continue doing those this cycle.

My json fixes for https://github.com/golang/go/issues/21092 and
https://github.com/golang/go/issues/31924 got reverted earlier this
cycle, so I need to take another look to see what we can do in 1.16.

I also have two somewhat old json CLs that I want to refresh and get
reviewed+merged this cycle:

https://go-review.googlesource.com/c/go/+/224079
https://go-review.googlesource.com/c/go/+/211457

Finally, from my email in the 1.15 planning thread:

On Wed, Feb 19, 2020 at 22:16:51 +0000, Daniel Martí wrote:
> I've also been thinking about a rewrite of the json tokenizer ever since
> I finished a round of json performance work in 1.13. I figure that, with
> a proper refactor, we could see struct decoding being sped up by 10-20%.
> I'm not sure how to go about this, though. Should I file a proposal?
> Email the other three json owners? Just send a massive CL directly?
> Ideas or input welcome.

I didn't put any time into that, but luckily Dave Cheney did in
https://dave.cheney.net/high-performance-json.html. I hope to convince
him to upstream his tokenizer into encoding/json in 1.16. It would be a
bit of work, but if done well we'd get a nice performance boost without
breaking any users.

If anyone wants to follow that json scanner/tokenizer stuff, follow
https://github.com/golang/go/issues/28923.

And lastly:

> I've also been filing a few ideas and proposals related to tooling in
> the past few weeks, and intend to file a few more and continue
> participating in https://github.com/golang/go/wiki/golang-tools.

This will continue. Here are some tooling issues that I intend to work
on, time permitting:

https://github.com/golang/go/issues/37281
https://github.com/golang/go/issues/32819
https://github.com/golang/go/issues/39484

That's all that's planned, for now. Reach out via the GitHub threads,
Slack, Twitter, or email if you'd like to participate in any.

[Chris Hines]

I would like to get a fix for https://github.com/golang/go/issues/38860 into 1.16. I believe that https://go-review.googlesource.com/c/go/+/232298/ may be sufficient. The fix involves runtime changes, so I guess that puts it into the risky category. It didn't make it into 1.15 because the issue was reported close to the freeze date and the fix was clearly too risky that late in the cycle.

Chris

[Meng Zhuo]

I'm porting thread sanitizer to mips64le arch.

Also I wish these arm64 SHA related  CLs merged into 1.16 too:

https://go-review.googlesource.com/c/go/+/180757  

https://go-review.googlesource.com/c/go/+/180057 

https://go-review.googlesource.com/c/go/+/180257 

[Rémy Oudompheng]

I would like to have the 3 CLs for strconv included in Go 1.16

https://go-review.googlesource.com/c/go/+/170080/2
https://go-review.googlesource.com/c/go/+/170079/2
https://go-review.googlesource.com/c/go/+/170078/2

Regards,
Rémy.

> --
> You received this message because you are subscribed to the Google Groups "golang-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/CAFPZzeSqYrzvDKjH18heJFHJ24XYk7tqidBmWMg8OAvwdTsh9Q%40mail.gmail.com.

[roger peppe]

In that area, and possibly removing the need for that entirely new tokenization API, I'd love to get some feedback on this issue that I raised that's had no discussion at all so far:

  https://github.com/golang/go/issues/40128

  

There's also this related one that has similarly seen no responses:

  https://github.com/golang/go/issues/40127

If they're accepted, I may have some time to spend implementing them, but I'm starting a new job soon, so we'd have to see.

  cheers,

    rog.

[Lynn Boger]

I have the following changes in mind for 1.16:

- I have a change to eliminate the prolog code to call morestack in functions where the only calls are to runtime panic functions, similar to what is done in amd64.

- I have a change to improve shifts that are done for array addressing within loops. There are some extended mnemonics and a new instruction in power9 (extswsli) which can be used in place of two shifts or a shift with and. This helps in several crypto functions (non-asm).

- There are a few existing asm functions that can be improved with some minor updates from crypto/elliptic, math/big/mulAddVWW and math/big/addMulVVW.

- I would like to follow up with the investigation on David Chase's CL to hoist constants out of loops. 

- I see there are changes going in which I'm guessing are to prepare for the new ABI. If there is any help needed with that regarding code changes or testing let me know.

[Andrew Bonventre]

The tree is now open for early-in-cycle changes only. We will open the tree more generally on Friday (August 14).

Happy hacking!

[Andrew Bonventre]

Also, as a note, if you have RELNOTE=yes changes, please update the 1.16 release notes as part of your change.

Thank you and sorry for the multiple emails back-to-back.

[Paul Murphy]

On Monday, August 10, 2020 at 7:27:46 PM UTC-5, Andrew Bonventre wrote:

In the meantime, please feel free to use this thread to share and discuss any work you’re planning to do or would like to do for Go 1.16.

 I would like to add support for a lightweight load/acquire operations, particularly for ppc64x.  The default full synchronization is considerably more expensive on ppc64 (e.g the fmt benchmarks show sync.(*Pool).pin consuming 20% time).

Similarly, there are number of minor improvements I would like to add to the ppc64 backend: generating subfic, and maddld on power9.  Within the assembler, adding support for some instructions for P9 which were missed (lxvx stxvx extswli).

[Tim Mohlmann]

This was submitted: https://go-review.googlesource.com/c/go/+/248337.

Related issue was marked early-in-cycle.

[Keith Randall]

The tree is still only open for early-in-cycle changes. I've noticed a few CLs going in that are not early-in-cycle, but are fixes that need to be backported to 1.15 or 1.14. I think that's totally OK, especially as it makes clean backports more likely. Maybe we should explicitly allow that exception for next time.

--

You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/7299790b-64f2-4597-b41d-f9a757cba41fn%40googlegroups.com.

[Dmitri Shuralyov]

Happy Friday gophers!

The tree is now open for general development. All "wait-release" tags on Gerrit CLs have been removed (they're now "ex-wait-release").

Please proceed cautiously, keeping the 4 requests we’ve made in the original email in mind. Thank you!

Dmitri for the Go release team

[Benny Siegert]

In this cycle, I would like to upstream the patches we have in pkgsrc to enable Go to work on netbsd-arm64.

What's more, I have a builder machine ready and installed, and I am just waiting for a Go builder key (netbsd-arm64-bsiegert maybe). I reached out repeatedly but my request was probably lost in the release preparations.

-- 

Benny

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/8eb2f73a-7971-432d-9061-91aaa116577dn%40googlegroups.com.

[Bryan C. Mills]

I'm making another run at lazy loading (https://golang.org/issue/36460), and replacement aliasing (https://golang.org/issue/26904) if time permits.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/CAN%2BFjHMDCewMxoz7OF04vKLqo-OrQz-t-YxgzjwKNS8CGG80hg%40mail.gmail.com.

[Chris Hines]

Question about contribution etiquette: What is the proper way to request further review and consideration of my CL for Go 1.16 (see quoted message below for links)?

I have reread the contribution guide and don't think I've missed any steps. If there is any work I still need to do, please let me know. It would be nice to get some feedback about where in the process the CL stands. Perhaps it is simply a matter of waiting for reviewer bandwidth, but I can't really tell. Considering the nature of the change it seems like one we would want to deal with earlier in the release cycle and I am worried that the clock is ticking.

Thanks,

Chris

[Matthew Dempsky]

On Tue, Sep 8, 2020 at 11:03 AM Chris Hines <ggr...@cs-guy.com> wrote:

Question about contribution etiquette: What is the proper way to request further review and consideration of my CL for Go 1.16 (see quoted message below for links)?

Typically commenting something like "ping" on the message or associated issue suffices.

[Chris Hines]

OK. I commented on the issue to that effect, here: https://github.com/golang/go/issues/38860#issuecomment-683883606. I guess I'm just waiting for bandwidth, but it is a bit frustrating not to have any idea when that might happen or if it's even on anyone's radar.

Chris

[Ian Lance Taylor]

On Tue, Sep 8, 2020 at 12:25 PM Chris Hines <ggr...@cs-guy.com> wrote:
>
> OK. I commented on the issue to that effect, here: https://github.com/golang/go/issues/38860#issuecomment-683883606. I guess I'm just waiting for bandwidth, but it is a bit frustrating not to have any idea when that might happen or if it's even on anyone's radar.

Sorry Chris, I do plan to look at that code again but I'm completely
swamped. I'll get to it.

I agree that it's hard to know whether the problem is reviewer
bandwidth or something else, since by definition if reviewers lack
bandwidth they aren't going around commenting on CLs saying "I don't
have bandwidth." Not sure what to do about that.

Ian

> On Tuesday, September 8, 2020 at 2:57:22 PM UTC-4, Matthew Dempsky wrote:
>>
>> On Tue, Sep 8, 2020 at 11:03 AM Chris Hines <ggr...@cs-guy.com> wrote:
>>>
>>> Question about contribution etiquette: What is the proper way to request further review and consideration of my CL for Go 1.16 (see quoted message below for links)?
>>
>>
>> Typically commenting something like "ping" on the message or associated issue suffices.
>

> --
> You received this message because you are subscribed to the Google Groups "golang-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/5d04e5af-249b-4208-b6cc-80156c1a97a1o%40googlegroups.com.

[Chris Hines]

Thanks for the reply, Ian. I look forward to your review when you have time.

I don't know what to do about reviewer bandwidth either. That said, I would offer as constructive criticism to the project that some kind of feedback about priority or expectations in a thread titled "Planning Go 1.16" on the developer's mailing list would help an infrequent contributor such as myself feel engaged in the process. Otherwise it's not clear to me that we're getting everything out of this thread that we could. I don't make a lot of contributions, but I've been on this mailing list long enough to expect the "Planning Go 1.xx" thread ritual each release cycle so I was excited to have something to add this time around. But it seems a bit like a black hole. It's not clear what, if any, coordination or planning is happening as a result of people posting to this planning thread.

Chris

On Tuesday, September 8, 2020 at 5:54:40 PM UTC-4, Ian Lance Taylor wrote:

On Tue, Sep 8, 2020 at 12:25 PM Chris Hines <ggr...@cs-guy.com> wrote:
>
> OK. I commented on the issue to that effect, here: https://github.com/golang/go/issues/38860#issuecomment-683883606. I guess I'm just waiting for bandwidth, but it is a bit frustrating not to have any idea when that might happen or if it's even on anyone's radar.

Sorry Chris, I do plan to look at that code again but I'm completely
swamped.  I'll get to it.

I agree that it's hard to know whether the problem is reviewer
bandwidth or something else, since by definition if reviewers lack
bandwidth they aren't going around commenting on CLs saying "I don't
have bandwidth."  Not sure what to do about that.

Ian


> On Tuesday, September 8, 2020 at 2:57:22 PM UTC-4, Matthew Dempsky wrote:
>>
>> On Tue, Sep 8, 2020 at 11:03 AM Chris Hines <ggr...@cs-guy.com> wrote:
>>>
>>> Question about contribution etiquette: What is the proper way to request further review and consideration of my CL for Go 1.16 (see quoted message below for links)?
>>
>>
>> Typically commenting something like "ping" on the message or associated issue suffices.
>
> --
> You received this message because you are subscribed to the Google Groups "golang-dev" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.

[Ian Lance Taylor]

On Tue, Sep 8, 2020 at 9:10 PM Chris Hines <ggr...@cs-guy.com> wrote:
>
> I don't know what to do about reviewer bandwidth either. That said, I would offer as constructive criticism to the project that some kind of feedback about priority or expectations in a thread titled "Planning Go 1.16" on the developer's mailing list would help an infrequent contributor such as myself feel engaged in the process. Otherwise it's not clear to me that we're getting everything out of this thread that we could. I don't make a lot of contributions, but I've been on this mailing list long enough to expect the "Planning Go 1.xx" thread ritual each release cycle so I was excited to have something to add this time around. But it seems a bit like a black hole. It's not clear what, if any, coordination or planning is happening as a result of people posting to this planning thread.

I think the goal of this thread, which happens at the start of every
release cycle, is to make sure that everybody is on the same page
regarding major work expected to occur during the cycle. Where people
are touching the same general area of the code, this leads into
coordination and/or planning. Otherwise, it does not.

I don't think the goal of the thread is to find reviewers for code,
which is a separate and ongoing problem, that applies to both large
and small changes.

Ian

[Chris Hines]

Let me clarify, I wasn't trying to suggest that this thread should help find reviewers in my "constructive criticism" comments. I shifted focus a bit from my earlier message asking about the best way to get reviewers. Sorry for the confusion.

What I think may be missing from this thread is some feedback about whether the work we are proposing to do for a release must be done by a certain stage or should wait until after another contribution was done, or something like that. The email that starts the thread focuses on those concerns, and then we all make our responses with what we want to work on, but then there doesn't seem to be any feedback after that. Concretely, my contribution may or may not be considered risky. I think it is mildly risky, but I am not the most qualified person to judge that. Depending on the answer maybe it should be merged sooner, or maybe it needs to be guarded with a feature flag. It seems like this thread is a good place to triage that sort of thing. Does it happen somewhere else? Who is responsible for that? I don't know.

Thanks,

Chris

[Ian Lance Taylor]

On Fri, Sep 18, 2020 at 1:56 PM Chris Hines <ggr...@cs-guy.com> wrote:
>
> Let me clarify, I wasn't trying to suggest that this thread should help find reviewers in my "constructive criticism" comments. I shifted focus a bit from my earlier message asking about the best way to get reviewers. Sorry for the confusion.
>
> What I think may be missing from this thread is some feedback about whether the work we are proposing to do for a release must be done by a certain stage or should wait until after another contribution was done, or something like that. The email that starts the thread focuses on those concerns, and then we all make our responses with what we want to work on, but then there doesn't seem to be any feedback after that. Concretely, my contribution may or may not be considered risky. I think it is mildly risky, but I am not the most qualified person to judge that. Depending on the answer maybe it should be merged sooner, or maybe it needs to be guarded with a feature flag. It seems like this thread is a good place to triage that sort of thing. Does it happen somewhere else? Who is responsible for that? I don't know.

Oh, I see. The expectation is that any conflicts in planned work will
appear to all concerned, and will be worked out. If there are no
conflicts, there is nothing further to discuss.

I don't think your proposed change needs to be under a feature flag or
anything like that. We just need to gain confidence that it won't
cause scheduling problems for other programs that don't use timers
heavily.

Ian

> On Monday, September 14, 2020 at 5:23:40 PM UTC-4 Ian Lance Taylor wrote:
>>
>> On Tue, Sep 8, 2020 at 9:10 PM Chris Hines <ggr...@cs-guy.com> wrote:
>> >
>> > I don't know what to do about reviewer bandwidth either. That said, I would offer as constructive criticism to the project that some kind of feedback about priority or expectations in a thread titled "Planning Go 1.16" on the developer's mailing list would help an infrequent contributor such as myself feel engaged in the process. Otherwise it's not clear to me that we're getting everything out of this thread that we could. I don't make a lot of contributions, but I've been on this mailing list long enough to expect the "Planning Go 1.xx" thread ritual each release cycle so I was excited to have something to add this time around. But it seems a bit like a black hole. It's not clear what, if any, coordination or planning is happening as a result of people posting to this planning thread.
>>
>> I think the goal of this thread, which happens at the start of every
>> release cycle, is to make sure that everybody is on the same page
>> regarding major work expected to occur during the cycle. Where people
>> are touching the same general area of the code, this leads into
>> coordination and/or planning. Otherwise, it does not.
>>
>> I don't think the goal of the thread is to find reviewers for code,
>> which is a separate and ongoing problem, that applies to both large
>> and small changes.
>>
>> Ian
>

> --
> You received this message because you are subscribed to the Google Groups "golang-dev" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/4c454d73-7f21-406c-9768-d21e6016c0c0n%40googlegroups.com.

[Austin Clements]

(Sorry I've been swamped and I'm just getting to this thread. I realize we're more than half way through the cycle at this point! How time flies.)

Here's a summary of some of the big things we're planning to land (or already have) for 1.16 on the runtime/compiler team at Google:

- Further improvements to the speed and RAM of the linker. This was almost entirely landed shortly after the tree opened. Across a range of benchmarks we see another 20–35% speed improvement and 5–15% RAM reduction beyond Go 1.15. There are a few more smaller improvements we might still land here. Thanks to Jeremy, Cherry, and Than for the amazing work on this.

- We are trying to land an initial version of a register-based calling convention. The high-level approach we're taking is described here: https://golang.org/design/40724-register-calling. There's a ton of work here but this is an "internal" ABI that we can continue to improve and iterate on, so we're aiming to land a minimum viable product for 1.16 on amd64. We already have a lot of potential improvements on the base design in mind and obviously plan to expand to all the architectures, but we think even the MVP will result in a 5% or more throughput improvement across the board on amd64. I have the detailed ABI spec mostly worked out. David's late call lowering (an important foundation for this and many other optimizations) is out for review and has started landing. We have a prototype of ABI bridges, which Than is getting into shape. Cherry's working out the details of traceback metadata. And Michael K. has a partly-working prototype of reflection calls.

- Cherry is working on native macOS ARM64 support.

- Michael K. is working on a far more flexible runtime metrics API that's under review now.

- Michael P. is in the process of landing lock-held assertions in the runtime.

On Mon, Aug 10, 2020 at 8:27 PM Andrew Bonventre <andy...@golang.org> wrote:

--

You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/CAFPZzeSqYrzvDKjH18heJFHJ24XYk7tqidBmWMg8OAvwdTsh9Q%40mail.gmail.com.

[Constantine Vassilev]

How to install Go 1.16 now?

[Benny Siegert]

It's not released yet. The best way to install it is to clone the git repository and build it from source.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/8832133a-94bd-4a04-9cad-4b707405a13cn%40googlegroups.com.

--

Benny

[Constantine Vassilev]

Are there instructions how to do that? The link to this git repository?

[Paul Jolly]

This is the first result on Google for "install go from source":

https://golang.org/doc/install/source

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/53f21709-c503-4d7b-a808-8345e598ca75n%40googlegroups.com.

[Constantine Vassilev]

Yes I can search. The question is how to install Go 1.16 from source. 

[Benny Siegert]

There is no Go 1.16 release yet.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/d7c8fa72-f766-46e0-96c4-480444ea9c31n%40googlegroups.com.

--

Benny

[Constantine Vassilev]

"The best way to install it is to clone the git repository and build it from source."

Where is this source?

[Brad Fitzpatrick]

There is no Go 1.16 yet. You can build the master git branch, though. Read that page's "Fetch the repository" section. 

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/d7c8fa72-f766-46e0-96c4-480444ea9c31n%40googlegroups.com.

[Constantine Vassilev]

I suppose this is the link to the master branch:

https://go.googlesource.com/go/+refs

https://go.googlesource.com/go/+/refs/heads/master

[Constantine Vassilev]

I made it work with status web site in the folder "www":

---

package main

import (

"embed"

"log"

"net/http"

)

//go:embed www

var src embed.FS

func main() {

// http://localhost:8080/www/

filehandler := http.FileServer(http.FS(src))

http.Handle("/", http.StripPrefix("/", filehandler))

// Run the web server.

log.Fatal(http.ListenAndServe(":8080", nil))

}

---

If use this link: http://localhost:8080/www/

the index.html inside is used to render the web site just fine.

How to make http://localhost:8080 to show the web site too?

On Saturday, November 28, 2020 at 11:07:50 AM UTC-8 Brad Fitzpatrick wrote: