Go 1.7 planning

[Russ Cox]

The Go 1.6 release candidate is looking good. Hopefully in a few weeks it will be out, and we'll open the tree for Go 1.7 development. That makes now a good time to talk about what we want to do in Go 1.7 as far as technical work.

Keith intends to merge the SSA-based compiler back end (amd64 only right now) into the main tree at the beginning of the Go 1.7 cycle. It works, and it generates code that on average runs about as fast as the current compiler. In many cases the SSA back end generates significantly better code than the old back end with little effect on performance: x86-64 chips (at least the server-class ones) are remarkably good at executing bad code quickly. We expect that the effect on non-x86 chips will be more pronounced, and there is still significant room for improvement in the x86 code quality, so we may yet see significant wins there. Unfortunately the SSA back end slows down the overall compilation by about 2x compared to using the old back end. This is not fundamental: we've spent almost all the development time focused on correctness and completeness, not on making it fast. The compiler work in the Go 1.7 cycle will focus on recovering much of that compile time loss.

Robert has a new, more compact export format for the compilers nearly ready. This should speed compilation a small amount and save a bit of disk space, and it fixes a few small problems in the old format. But most importantly it gives us a much cleaner base on which to make future changes to the export data, which will make it easier to do more interprocedural optimizations like escape analysis and inlining that work no matter where package boundaries are.

Rick and Austin have been working on further improvements to the garbage collector, especially in terms of throughput. That's a long effort and may not be ready for Go 1.7.

Another effort we'd like to do for Go 1.7 (but don't have concrete plans for yet) is collection and display of historical performance data, as a replacement for the old performance dashboard we had that no longer runs. The part that I believe is most important is the separation of collection from display. We want to make it possible for anyone to run a benchmarking system, just as anyone can run a builder, and have all that data feed into a database of some kind that anyone can fetch (perhaps as a Git repository) and analyze. It requires significant expertise to collect useful data and it requires significant but different expertise to analyze and display it well, and we think that separating these two parts will help let experts make progress on their halves independently.

In general I would encourage people once again to focus on cleanup and polish for Go 1.7. There are many parts of the tree that work but are not as good as they could be. For example we know that math/big is missing assembly support to varying degrees on some architectures. We also know there are many minor problems, missing examples, and so on that are significant collectively, even if not taken one at a time.

What are you planning to do in the Go 1.7 cycle? Or what GitHub issues do we not have marked with a Go 1.7 milestone yet but merit one, in your opinion?

Please reply to this email thread and we'll make a doc out of the responses.

Thanks.

Russ

[Brad Fitzpatrick]

 On Thu, Jan 28, 2016 at 7:15 AM, Russ Cox <r...@golang.org> wrote:

What are you planning to do in the Go 1.7 cycle? Or what GitHub issues do we not have marked with a Go 1.7 milestone yet but merit one, in your opinion?

I intend to do significantly less in Go 1.7 compared to Go 1.6. I have work I'd like to do on ACME, gRPC, GCP APIs, and other things not in the Go tree.

I will also be away at the end of the cycle.

If http2 bugs come up, those will be my main priority. I also have a few minor cleanups in mind to make the http2 code easier to follow and share more code between the client & server.

Unfortunately the SSA back end slows down the overall compilation by about 2x compared to using the old back end. This is not fundamental: we've spent almost all the development time focused on correctness and completeness, not on making it fast. The compiler work in the Go 1.7 cycle will focus on recovering much of that compile time loss.

This may be an unpopular opinion, but: perhaps we need to recover the 2x slowdown we already suffered from Go 1.4 to Go 1.5 before we get twice as slow yet again with the wishful thinking that we'll recover it. Go 1.5 and Go 1.6 are still painful compared to Go 1.4. I'd like to say we won't merge SSA until our compiler is fast again.

4x slower builds & tests than Go 1.4 will not be pleasant.

[David Chase]

There are three bugs related to escape analysis, two with CLs ready, one not.

1) https://go-review.googlesource.com/#/c/18041/1 -- explaining why stuff escapes. (-m -m)

2) https://go-review.googlesource.com/#/c/17581/ -- (slightly) better escape analysis for recursive calls

3) https://github.com/golang/go/issues/14018 -- we know the "real" type but only use the declared interface type, and punt on escape (and inlining)

#3 is upstream of escape analysis and could in theory iterate a few times against inlining, since each can enable more of the other.

There have been some ugly hacks in allocation-critical Go application code that this optimization would have obviated.

Current work is on SSA, reviewing CLs and working on making bounds check elimination more aggressive in hopes

that we'll ever see anything more than a mere 1% reduction in text size.  Early benchmarking suggests that it is not

terrible.  The code will need a round of serious reviewing.

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Robert Griesemer]

- I'm going to complete the new export format. It's a binary format, so there will also be a mechanism to display it in a human-readable form.

- I'm hoping to spend more time on cleaning up the compiler front-end. Making nodes smaller should help memory use and eventually performance, and also improve maintainability.

- There are several lose ends in math/big that require some careful attention. There's also some low-hanging fruit in some of the core assembly routines that should have significant impact across many math/big functions.

- Ongoing spec maintenance (mostly outstanding issues).

- There will be time required for bug fixes across existing libraries.

- There may be Google3 work (TBD).

- I will be away for up to one month, possibly a bit longer (wedding of my brother in Europe, summer vacation, GopherCon).

- gri

[Aram Hăvărneanu]

For 1.7 I will on working on SPARC64 and ARM64.

For SPARC64, I will finish the port and try to get it in in the first
half of the cycle.

For ARM64, I will do SSA, add fast assembly versions for things like
math/big, and generally do other optimizations.

--
Aram Hăvărneanu

[Michael Monashev]

Hi, Russ.

What about multy-valued channels?

my_chan:=make(chan (int, err), 100)
...
my_chan <- val, err
...
val, err <- my_chan

It can be useful to return values and error from goroutine. Like
'return' statement in function.

--

Michael

[Brad Fitzpatrick]

Language changes are out of scope.

[Matthew Dempsky]

On Thu, Jan 28, 2016 at 9:15 AM, Michael Monashev <softs...@gmail.com> wrote:

What about multy-valued channels?

These already exist.  The syntax for them is:

    myChan := make(chan struct{x int; err error}, 100)

    ...

    myChan <- struct{x int; err error}{val, err}

    ...

    s := <-myChan

    val, err = s.x, s.err

(If golang.org/issue/12854 is accepted/implemented, the send statement could be written as "myChan <- {val, err}".)

[Aram Hăvărneanu]

On Thu, Jan 28, 2016 at 6:15 PM, Michael Monashev <softs...@gmail.com> wrote:
> ...

This thread is for people working *on* Go. For feature requests and
proposals please go through the usual channels, not on this thread.
Thank you.

--
Aram Hăvărneanu

[Ian Lance Taylor]

On Thu, Jan 28, 2016 at 7:15 AM, Russ Cox <r...@golang.org> wrote:
>

> What are you planning to do in the Go 1.7 cycle? Or what GitHub issues do we
> not have marked with a Go 1.7 milestone yet but merit one, in your opinion?

I would like to provide guarantees that make it possible to use
finalizers with confidence (issue 13347).

I would like to decide how we are going to decide about the memory
model (issues 7948, 9442, 5045, plus various other communications).
People who care about memory models do not like it.

I would like to fix the object file format to make linking test
programs much more efficient but I doubt I will get to it.

Ian

[roger peppe]

I'm +1 on this. Although historically I've always used Go tip, with
sadness I usually use Go1.4.3 now because it saves me so
much time. As an example, in one code base that I work on,
if I touch a deep dependency
and rebuild a test binary, under 1.4.3 it takes about 50s,
but under 1.6 it takes >180s. That extra 2 minutes of thumb
twiddling feels painful. If the time difference goes up to 5 minutes,
I won't be happy.

As for development, I'd like to try to get encoding/xml fixed again.

cheers,
rog.

[Matthew Dempsky]

On Thu, Jan 28, 2016 at 9:29 AM, Robert Griesemer <g...@golang.org> wrote:

- I'm going to complete the new export format. It's a binary format, so there will also be a mechanism to display it in a human-readable form.

I'm interested in helping with this, and I've already prepared a patch series at https://github.com/golang/go/compare/master...mdempsky:binary-builtin to eliminate the compiler's internal uses of the legacy text format.  (Will move to Gerrit once Go 1.7 tree opens up.)

I'm also generally interested in helping to further cleanup the compiler and runtime to make them more Go-ey.  I already have a long list of random loose ends to look at, but I'm open to suggestions if anyone has them.

[Seb Binet]

Hi,

I'd like to go forward with the 'reflect.StructOf' proposal:

https://golang.org/cl/9251

And perhaps, work on the 'plugin' package.

-s

sent from my droid

--

[Brad Fitzpatrick]

On Thu, Jan 28, 2016 at 10:36 AM, Seb Binet <seb....@gmail.com> wrote:

Hi,

I'd like to go forward with the 'reflect.StructOf' proposal:

https://golang.org/cl/9251

And perhaps, work on the 'plugin' package.

What's the plugin package?

[Joe Tsai]

In 1.6, I made several changes to archive/tar to harden that library up. I would like to continue that work and fix up the remaining issues with that library (#9647, #9683, #11171, #12594, #13548). Most of the work is already done (github.com/dsnet/tar), and I have pretty much been merging the changes slowly back. Thanks to @bradfitz for reviewing most of the past changes; would you be interesting in reviewing the future ones too?

Also, I am interested in merging in a bzip2 encoder (#4828) and fixing minor bzip2 problems (#13941). I have a plan to submit the new code in two-phases. First phases will focus on an working encoder by re-using index/suffixarray for the BWT construction. It won't be particularly fast, but will be correct. The second phase will be improving the speed of the BWT construction; that may be a future Go version.

If time permits, I would love to merge in some of the performance optimizations I have for flate and bzip2 decompression rates, which improves performance by 1.5x to 3x. Though, I doubt I'll get to them this time around.

[Brad Fitzpatrick]

I can probably review the archive/tar stuff. Or dsymonds.

[klau...@gmail.com]

I plan to submit my performance and compression optimizations for deflate. They are compatible, and brings the native Go implementation very close to cgo-based ones in perms of performance. This should increase performance of zip, gzip, png and direct use of the deflate library.

[Caleb Spare]

I intend to work on some small encoding/json improvements (6492, 12529, and 12146) unless someone beats me to them.

If I have time, I also want to work on the testing features in 10149 and 4899. The latter is tricky and may prove out of reach for me, and a solution probably needs a formal proposal at this point.

[Dave Cheney]

I want to second Brad's call for a focus on recovering the speed of the current compiler before merging SSA. 

Taking a 2x hit in 1.5 was palatable for Go users with the proviso that it was temporary. To take another 2x would stretch that promise beyond the point I think most people will reasonably accept.

Thanks

Dave

[Keith Randall]

My original plan was to spend 20% extra compiler time for the SSA backend.  We would then see how much of that we would get back because of better generated code for the compiler itself.

We're at 100% extra only because no optimization at all has happened yet.  I still intend to get back down to that 20%.

Feel free to jump in and help out on the ssa branch.  Any contributions would be appreciated.

[Richard Gooch]

On Thursday, 28 January 2016 07:15:07 UTC-8, rsc wrote:

The Go 1.6 release candidate is looking good. Hopefully in a few weeks it will be out, and we'll open the tree for Go 1.7 development. That makes now a good time to talk about what we want to do in Go 1.7 as far as technical work.

 I'm interested in having a streaming interface added to encoding/gob and encoding/json so that there is no need to marshal/unmarshal data into/from a buffer before/after writing/reading. The current implementation is causing us some pain. If I can get in-principle agreement on this enhancement, I'll plan and scope out the work and either do it myself or assign one of my team to do it.

Regards,

Richard....

[Andrew Gerrand]

On 29 January 2016 at 08:55, Richard Gooch <rg+go...@safe-mbox.com> wrote:

 I'm interested in having a streaming interface added to encoding/gob and encoding/json so that there is no need to marshal/unmarshal data into/from a buffer before/after writing/reading. The current implementation is causing us some pain. If I can get in-principle agreement on this enhancement, I'll plan and scope out the work and either do it myself or assign one of my team to do it.

Have you filed an issue about this? https://golang.org/s/proposal-process

[Richard Gooch]

No. I just cried a little when I skimmed over it, before my eyes glazed over :-/
Hopefully a full design doc isn't needed?

Regards,

Richard....

[Andrew Gerrand]

The first step is to file an issue describing what you'd like to add.

After that we can determine whether a full proposal is required. (probably not)

--

[Matt Silverlock]

I'd like to see net/context + net/http integration as per https://groups.google.com/forum/#!topic/golang-dev/TerfinvPffw (an associated issue does not seem to exist for it).

[Richard Gooch]

On Thursday, 28 January 2016 14:11:42 UTC-8, Andrew Gerrand wrote:

The first step is to file an issue describing what you'd like to add.

After that we can determine whether a full proposal is required. (probably not)

 https://github.com/golang/go/issues/14140

Regards,

Richard....

[Ian Lance Taylor]

I assume (or I hope) that Seb is referring to the plugin package
described at https://golang.org/s/execmodes .

Ian

[areg.meli...@intel.com]

Our team is definitely going to contribute to SSA.

Also lots of IA specific optimizations both for RT and compiler.

[David Norton]

I agree with Brad, Rog, and Dave that compiler speed should be one of the priorities.  I see a 2.7x slow down going from 1.4 to 1.5.  Doubling that means staying on 1.4 for daily dev work for another cycle of Go.

GC performance:  +1 anything that helps that work and -1 anything that impedes it.  We ended up rolling back to 1.4 for official releases due to performance.  1.6 looks promising but still slower than 1.4.

On Thursday, January 28, 2016 at 10:15:07 AM UTC-5, rsc wrote:

The Go 1.6 release candidate is looking good. Hopefully in a few weeks it will be out, and we'll open the tree for Go 1.7 development. That makes now a good time to talk about what we want to do in Go 1.7 as far as technical work.

Keith intends to merge the SSA-based compiler back end (amd64 only right now) into the main tree at the beginning of the Go 1.7 cycle. It works, and it generates code that on average runs about as fast as the current compiler. In many cases the SSA back end generates significantly better code than the old back end with little effect on performance: x86-64 chips (at least the server-class ones) are remarkably good at executing bad code quickly. We expect that the effect on non-x86 chips will be more pronounced, and there is still significant room for improvement in the x86 code quality, so we may yet see significant wins there. Unfortunately the SSA back end slows down the overall compilation by about 2x compared to using the old back end. This is not fundamental: we've spent almost all the development time focused on correctness and completeness, not on making it fast. The compiler work in the Go 1.7 cycle will focus on recovering much of that compile time loss.

Robert has a new, more compact export format for the compilers nearly ready. This should speed compilation a small amount and save a bit of disk space, and it fixes a few small problems in the old format. But most importantly it gives us a much cleaner base on which to make future changes to the export data, which will make it easier to do more interprocedural optimizations like escape analysis and inlining that work no matter where package boundaries are.

Rick and Austin have been working on further improvements to the garbage collector, especially in terms of throughput. That's a long effort and may not be ready for Go 1.7.

Another effort we'd like to do for Go 1.7 (but don't have concrete plans for yet) is collection and display of historical performance data, as a replacement for the old performance dashboard we had that no longer runs. The part that I believe is most important is the separation of collection from display. We want to make it possible for anyone to run a benchmarking system, just as anyone can run a builder, and have all that data feed into a database of some kind that anyone can fetch (perhaps as a Git repository) and analyze. It requires significant expertise to collect useful data and it requires significant but different expertise to analyze and display it well, and we think that separating these two parts will help let experts make progress on their halves independently.

In general I would encourage people once again to focus on cleanup and polish for Go 1.7. There are many parts of the tree that work but are not as good as they could be. For example we know that math/big is missing assembly support to varying degrees on some architectures. We also know there are many minor problems, missing examples, and so on that are significant collectively, even if not taken one at a time.

What are you planning to do in the Go 1.7 cycle? Or what GitHub issues do we not have marked with a Go 1.7 milestone yet but merit one, in your opinion?

[Brad Fitzpatrick]

David, do you mean gc or GC? (Capitals means garbage collector, lower is go compiler)

If GC, I don't know which bug you're talking about.

[David Norton]

Brad, sorry, I should have been specific.  I did mean GC as in garbage collector.  Not a specific bug but just saying +1 to the throughput work already in progress and -1 to any other work that might slow their progress.

[minux]

I will bring external linking to linux/ppc64 and linux/mips64{,le}.

I will also upstream the disassembler into x/arch and then

integrate it into cmd/objdump. These should make linux/ppc64

port more complete.

Cherry and I also have plans for the linux/mips64 cgo support.

[ron minnich]

There is a riscv port in progress, although if we hit 1.7 we'd be amazed. There are also two more non-Unix OS ports in progress, and we're chasing behind ToT just as fast as we can.

ron

--

[Seb Binet]

yep, that's the one.
thanks for clearing that up and apologies for the vagueness of my first email.

-s

[Keith Randall]

A week of low-hanging-fruit fixes by a few of us and SSA has improved to about 40% slower than tip.  Progress...

[daniel...@gmail.com]

As a user of Go (currently using 1.4.3 because don't really need to upgrade right now, but certainly want to eventually. I love the compile speed, and that is actually one of the main reasons I haven't updated yet), I wanted to chime in and agree with this statement:

"This may be an unpopular opinion, but: perhaps we need to recover the 2x slowdown we already suffered from Go 1.4 to Go 1.5 before we get twice as slow yet again with the wishful thinking that we'll recover it. Go 1.5 and Go 1.6 are still painful compared to Go 1.4. I'd like to say we won't merge SSA until our compiler is fast again.

4x slower builds & tests than Go 1.4 will not be pleasant."

Totally something that makes sense to me. Before slowing it down further, improve, and introduce next step. Unless of course this new thing helps overall. (I have no idea how SSA works currently, and it might be double effort to improve before SSA is introduced, but as I said, no idea how SSA works)

However, if including things (SSA) that makes it 2x slower, wouldn't it be a point to possibly include it early, and work on improving it before final release? Meaning it won't be 2x worse at the release?

Regards,
Daniel

[jserv...@gmail.com]

I second Brad's call for a focus on recovering the speed of the current compiler. 

This is imperative to maintain Go’s web development demographic. 

According to the commander in chief, "most Go programmers come from languages like Python and Ruby" and here is why: "Python and Ruby programmers come to Go because they don't have to surrender much expressiveness, but gain performance and get to play with concurrency." (http://commandcenter.blogspot.com/2012/06/less-is-exponentially-more.html)

When developing a web application or a software prototype, people need a "tight feedback loop". 

Python and Ruby provide this near 0 second waiting time to see the results via interpretation. 

2 seconds is a feedback loop too long. 

Please do not drive these web developers back to use Python or Ruby. 

I am not sure why Go is a must to compile Go. If the 2x slowdown cannot be recouped due to a switch from C to Go, then switching back to C does not seem like a crazy idea to me. 

Even Swift uses C++ to compile itself. Can we be pragmatic and keep the developer (the end user) happy for what the language was designed for? 

Thanks,

JPro

[Steven Hartland]

I have to agree the significant slow down when moving away from the C compiler is causing a really noticeable slow down when developing golang apps, something I'd really like to see improved moving forward.

[Andrew Gerrand]

Again, a reminder this is a thread for people working on Go.

There's no need to "me too" Brad's comment.

It distracts from planning our next release.

We know that users want faster compile times, and it's among our priorities.

Thanks,

Andrew

[Bjørn Erik Pedersen]

>Again, a reminder this is a thread for people working on Go.

You would have to define *working on Go* for that to work.

For many people *using Go", the *me want* part fits like a glove into the "planning Go 1.7 subject".

bep

[Andrew Gerrand]

On 1 February 2016 at 12:12, Bjørn Erik Pedersen <bjorn.eri...@gmail.com> wrote:

>Again, a reminder this is a thread for people working on Go.

You would have to define *working on Go* for that to work.

It means people actively contributing to the core Go repositories.

Apologies if that wasn't clear from Russ' original post. 

Cheers,

Andrew

[Russ Cox]

For what it's worth, the message about compile speed has been received. Keith and David are now planning to spend February working on compile speed in the dev.ssa branch, looking toward a possible merge around March 1.

Russ

[matt]

Would also like to voice concerns of slowing down the compiler again, especially if you think there will be few benefits seen in performance in this cycle.

On my side I would like to help out on some of the ARM64 low level optimization work, and clean up efforts.

[Burcu Dogan]

I may have a tentative attempt to support Go on Brillo (https://developers.google.com/brillo/). The intent is to have GOOS=android builds run on Brillo devices if Brillo is not significantly different than the Android kernel. I don't have much context on how much work will be required or whether we need a GOOS=brillo port or not. Depending on the amount of work needs to be done, I may revoke my plans or just leave a proposal behind and move on.

On Thu, Jan 28, 2016 at 7:15 AM, Russ Cox <r...@golang.org> wrote:

The Go 1.6 release candidate is looking good. Hopefully in a few weeks it will be out, and we'll open the tree for Go 1.7 development. That makes now a good time to talk about what we want to do in Go 1.7 as far as technical work.

Keith intends to merge the SSA-based compiler back end (amd64 only right now) into the main tree at the beginning of the Go 1.7 cycle. It works, and it generates code that on average runs about as fast as the current compiler. In many cases the SSA back end generates significantly better code than the old back end with little effect on performance: x86-64 chips (at least the server-class ones) are remarkably good at executing bad code quickly. We expect that the effect on non-x86 chips will be more pronounced, and there is still significant room for improvement in the x86 code quality, so we may yet see significant wins there. Unfortunately the SSA back end slows down the overall compilation by about 2x compared to using the old back end. This is not fundamental: we've spent almost all the development time focused on correctness and completeness, not on making it fast. The compiler work in the Go 1.7 cycle will focus on recovering much of that compile time loss.

Robert has a new, more compact export format for the compilers nearly ready. This should speed compilation a small amount and save a bit of disk space, and it fixes a few small problems in the old format. But most importantly it gives us a much cleaner base on which to make future changes to the export data, which will make it easier to do more interprocedural optimizations like escape analysis and inlining that work no matter where package boundaries are.

Rick and Austin have been working on further improvements to the garbage collector, especially in terms of throughput. That's a long effort and may not be ready for Go 1.7.

Another effort we'd like to do for Go 1.7 (but don't have concrete plans for yet) is collection and display of historical performance data, as a replacement for the old performance dashboard we had that no longer runs. The part that I believe is most important is the separation of collection from display. We want to make it possible for anyone to run a benchmarking system, just as anyone can run a builder, and have all that data feed into a database of some kind that anyone can fetch (perhaps as a Git repository) and analyze. It requires significant expertise to collect useful data and it requires significant but different expertise to analyze and display it well, and we think that separating these two parts will help let experts make progress on their halves independently.

In general I would encourage people once again to focus on cleanup and polish for Go 1.7. There are many parts of the tree that work but are not as good as they could be. For example we know that math/big is missing assembly support to varying degrees on some architectures. We also know there are many minor problems, missing examples, and so on that are significant collectively, even if not taken one at a time.

What are you planning to do in the Go 1.7 cycle? Or what GitHub issues do we not have marked with a Go 1.7 milestone yet but merit one, in your opinion?

Please reply to this email thread and we'll make a doc out of the responses.

Thanks.

Russ

[Michael Hudson-Doyle]

On 29 January 2016 at 07:22, Ian Lance Taylor <ia...@golang.org> wrote:

> I would like to fix the object file format to make linking test
> programs much more efficient but I doubt I will get to it.

Do you have anything specific in mind beyond the sort of thing I
talked about in https://github.com/golang/go/issues/11123 ? I'd like
to get back to that, but I don't know if I'm going to have time for
it.

Cheers,
mwh

[Dave Cheney]

This is fantastic news Keith; we can have cake, and eat it it too!

[Nathan Youngman]

On Thursday, 28 January 2016 08:15:07 UTC-7, rsc wrote:

What are you planning to do in the Go 1.7 cycle? Or what GitHub issues do we not have marked with a Go 1.7 milestone yet but merit one, in your opinion?

I'd like to help out with x/crypto, particularly addressing this issue in the pkcs12 package.

https://github.com/golang/go/issues/14015

Additionally, I would like to improve the documentation for pkcs12 with additional examples.

While I'd like to see additional packages in x/crypto, such as pkcs7, I don't know if I'll learn enough about crypto to make that happen in this cycle.

Nathan.

[Russ Cox]

Thanks. Note that x/crypto is not subject to the usual release cycles.

Russ

[Edward Muller]

Any chance of getting any of this work in? https://github.com/cloudflare/go (see also: https://blog.cloudflare.com/go-crypto-bridging-the-performance-gap/)

On Thu, Jan 28, 2016 at 7:15 AM, Russ Cox <r...@golang.org> wrote:

The Go 1.6 release candidate is looking good. Hopefully in a few weeks it will be out, and we'll open the tree for Go 1.7 development. That makes now a good time to talk about what we want to do in Go 1.7 as far as technical work.

Keith intends to merge the SSA-based compiler back end (amd64 only right now) into the main tree at the beginning of the Go 1.7 cycle. It works, and it generates code that on average runs about as fast as the current compiler. In many cases the SSA back end generates significantly better code than the old back end with little effect on performance: x86-64 chips (at least the server-class ones) are remarkably good at executing bad code quickly. We expect that the effect on non-x86 chips will be more pronounced, and there is still significant room for improvement in the x86 code quality, so we may yet see significant wins there. Unfortunately the SSA back end slows down the overall compilation by about 2x compared to using the old back end. This is not fundamental: we've spent almost all the development time focused on correctness and completeness, not on making it fast. The compiler work in the Go 1.7 cycle will focus on recovering much of that compile time loss.

Robert has a new, more compact export format for the compilers nearly ready. This should speed compilation a small amount and save a bit of disk space, and it fixes a few small problems in the old format. But most importantly it gives us a much cleaner base on which to make future changes to the export data, which will make it easier to do more interprocedural optimizations like escape analysis and inlining that work no matter where package boundaries are.

Rick and Austin have been working on further improvements to the garbage collector, especially in terms of throughput. That's a long effort and may not be ready for Go 1.7.

Another effort we'd like to do for Go 1.7 (but don't have concrete plans for yet) is collection and display of historical performance data, as a replacement for the old performance dashboard we had that no longer runs. The part that I believe is most important is the separation of collection from display. We want to make it possible for anyone to run a benchmarking system, just as anyone can run a builder, and have all that data feed into a database of some kind that anyone can fetch (perhaps as a Git repository) and analyze. It requires significant expertise to collect useful data and it requires significant but different expertise to analyze and display it well, and we think that separating these two parts will help let experts make progress on their halves independently.

In general I would encourage people once again to focus on cleanup and polish for Go 1.7. There are many parts of the tree that work but are not as good as they could be. For example we know that math/big is missing assembly support to varying degrees on some architectures. We also know there are many minor problems, missing examples, and so on that are significant collectively, even if not taken one at a time.

What are you planning to do in the Go 1.7 cycle? Or what GitHub issues do we not have marked with a Go 1.7 milestone yet but merit one, in your opinion?

Please reply to this email thread and we'll make a doc out of the responses.

Thanks.

Russ

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Edward Muller
@freeformz

[Tylor Arndt]

Edward, see here: https://github.com/cloudflare/go/issues/5

[Henrik Johansson]

The google source references indicates merge. Has it been integrated?

[Edward Muller]

AFAICT from looking at a few different CLs this morning: yes.

https://go-review.googlesource.com/#/c/10484/

https://go-review.googlesource.com/#/c/9253/

https://go-review.googlesource.com/#/c/8968/

Not sure if there were others though.

[Nathan Youngman]

Thanks for clarifying. I wasn't entirely sure how the x/packages are handled in terms of release cycles, nor am I sure about API stability guarantees there.

Unfortunately it looks like a full implementation of pkcs7 or the latest Cryptographic Message Syntax (CMS)  RFC won't be possible without also modifying encoding/asn1 to support BER (or providing an alternate/internal asn1 package that does). 

That was already turned down: https://github.com/golang/go/issues/12267

I'm still open to helping out with pkcs12 however, as that is a package I am using quite a bit lately.

Nathan.

--

Nathan Youngman 
Email: he...@nathany.com
Web: https://nathany.com

[Mikio Hara]

On Fri, Jan 29, 2016 at 12:15 AM, Russ Cox <r...@golang.org> wrote:

> What are you planning to do in the Go 1.7 cycle? Or what GitHub issues do we
> not have marked with a Go 1.7 milestone yet but merit one, in your opinion?

I'd like to add support for FreeBSD 11 (#7849) during this cycle if
Mr. Taylor doesn't mind reviewing a series of tedious CLs. Also a few
cleanup CLs to net package for documentation.

[Dan Peterson]

On Thursday, January 28, 2016 at 11:15:07 AM UTC-4, rsc wrote:

What are you planning to do in the Go 1.7 cycle?

 

I'd like to continue helping where I can with the stub resolver, such as with #13295.

[t.br...@gmail.com]

On Thursday, January 28, 2016 at 11:16:52 PM UTC+1, Matt Silverlock wrote:

I'd like to see net/context + net/http integration as per https://groups.google.com/forum/#!topic/golang-dev/TerfinvPffw (an associated issue does not seem to exist for it).

I'm a bit late (coming here from Golang Weekly) but I think you meant to link to https://groups.google.com/forum/#!topic/golang-dev/cQs1z9LrJDU

[Bill O'Farrell]

We are ready to contribute our port to Linux on Z. We have a builder set up, and we could give Brad access if that would be helpful. Russ: could you give us some advice on structuring our CLs? What would be the best time frame for you?
Thanks... Bill

On Thursday, January 28, 2016 at 10:15:07 AM UTC-5, rsc wrote:

The Go 1.6 release candidate is looking good. Hopefully in a few weeks it will be out, and we'll open the tree for Go 1.7 development. That makes now a good time to talk about what we want to do in Go 1.7 as far as technical work.

Keith intends to merge the SSA-based compiler back end (amd64 only right now) into the main tree at the beginning of the Go 1.7 cycle. It works, and it generates code that on average runs about as fast as the current compiler. In many cases the SSA back end generates significantly better code than the old back end with little effect on performance: x86-64 chips (at least the server-class ones) are remarkably good at executing bad code quickly. We expect that the effect on non-x86 chips will be more pronounced, and there is still significant room for improvement in the x86 code quality, so we may yet see significant wins there. Unfortunately the SSA back end slows down the overall compilation by about 2x compared to using the old back end. This is not fundamental: we've spent almost all the development time focused on correctness and completeness, not on making it fast. The compiler work in the Go 1.7 cycle will focus on recovering much of that compile time loss.

Robert has a new, more compact export format for the compilers nearly ready. This should speed compilation a small amount and save a bit of disk space, and it fixes a few small problems in the old format. But most importantly it gives us a much cleaner base on which to make future changes to the export data, which will make it easier to do more interprocedural optimizations like escape analysis and inlining that work no matter where package boundaries are.

Rick and Austin have been working on further improvements to the garbage collector, especially in terms of throughput. That's a long effort and may not be ready for Go 1.7.

Another effort we'd like to do for Go 1.7 (but don't have concrete plans for yet) is collection and display of historical performance data, as a replacement for the old performance dashboard we had that no longer runs. The part that I believe is most important is the separation of collection from display. We want to make it possible for anyone to run a benchmarking system, just as anyone can run a builder, and have all that data feed into a database of some kind that anyone can fetch (perhaps as a Git repository) and analyze. It requires significant expertise to collect useful data and it requires significant but different expertise to analyze and display it well, and we think that separating these two parts will help let experts make progress on their halves independently.

In general I would encourage people once again to focus on cleanup and polish for Go 1.7. There are many parts of the tree that work but are not as good as they could be. For example we know that math/big is missing assembly support to varying degrees on some architectures. We also know there are many minor problems, missing examples, and so on that are significant collectively, even if not taken one at a time.

What are you planning to do in the Go 1.7 cycle? Or what GitHub issues do we not have marked with a Go 1.7 milestone yet but merit one, in your opinion?

[Derek Parker]

I would love to see improvements to the DWARF output, specifically addressing issue https://github.com/golang/go/issues/12899. If that could land in 1.7 I would be very happy. If the core team is too busy to dig into it due to other issues, I have enough domain around the code that generates it that I could submit a patch, especially if I could get my employer to sponsor some time to set aside for the work (which shouldn't be an issue).

[matwa...@gmail.com]

What about issue 11058 (building as c-shared libs for windows)? It would be a great manguage feature!
Thanks

[me...@smts.nl]

One of the great features of Go is the fast compilation time. Go 1.5 compilation is already slower as Go 1.4, so please do not make it again slower again. It would be great as Go 1.7 compilation times come close again to Go 1.4 compilation times. .

Op donderdag 28 januari 2016 16:15:07 UTC+1 schreef rsc:

[Christopher]

I volunteered to work on this, but I'm trying to get the c-archive build for Windows committed first. The code is done, I'm just working through the commit process. There's no reason why c-archive shouldn't go into 1.7. The c-shared piece requires some work to improve TLS support on Windows in addition to the simpler system linker calls. My plan is to get this into 1.7.

[chai2010]

Greate!

I can use Go replace C++ when Go 1.7 released!

2016年2月8日星期一，Christopher <nadias...@gmail.com> 写道：

--

You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

https://github.com/golang-china/gopl-zh

https://github.com/golang-china

https://github.com/chai2010

[Nathan Youngman]

Though not specific to the Go 1.7 cycle, I'd be happy to proofread any documentation CLs.

-cc he...@nathany.com

Not that I'm an expert at English, but I'm usually pretty good at spotting typos and the like.

There are ~25 open issues for the Go tour. I'd like to help out with that effort as well. Who's heading it up?

Nathan.

On Thursday, 28 January 2016 08:15:07 UTC-7, rsc wrote:

[lab...@linux.vnet.ibm.com]

Hi,

Do you have the disassembler working enough so I can try it out?

On Friday, January 29, 2016 at 11:12:30 AM UTC-6, minux wrote:

I will bring external linking to linux/ppc64 and linux/mips64{,le}.

I will also upstream the disassembler into x/arch and then

integrate it into cmd/objdump. These should make linux/ppc64

port more complete.

Cherry and I also have plans for the linux/mips64 cgo support.

[lab...@linux.vnet.ibm.com]

I plan to work on performance for golang on ppc64le as my time allows.  Starting with functions that are written in asm for other platforms but not on ppc64le.

[David Crawshaw]

I'm taking a look at http://golang.org/issue/6853

[Dmitry Vyukov]

I want to do 2 things:

1. Improve tracer performance for parallel programs. Ideally, tracer does not disturb program execution so that user observes real picture. But unfortunately for parallel workloads tracer introduces significant slowdown and skews traces. The plan is to replace the global traceseq variable used for sequence number assignment with a distributed scheme.

Hana wanted to tackle embedding of source:line info into trace files, which will make them self-contained.

With these two improvement tracer will become significantly more usable.

2. Update race runtime and fix few long-standing issues. First, there is a memory leak for long-running apps. Second, there is some potential for memory consumption reduction inside of C++ runtime (more or less switching to the same "per-P" scheme for caching that we use in runtime). Then, it would be nice to provide allocation stack for memory locations. The last part is less important.