Go 1.7 freeze begins

[Russ Cox]

The Go 1.7 freeze has begun.

Important pending CLs can still be reviewed but really need to be completed and merged by the end of the week, or else postponed.

The remaining bug fix work should be focused on regressions since Go 1.6, especially the kind of crash/unavoidable problem that we would issue a point release for.

Open Go 1.7 bugs for problems that are not new since Go 1.6 should in general be postponed (moved to Unplanned milestone): if Go 1.6 behaved that way, it's probably OK for Go 1.7 to continue to behave that way. That's even more true if Go 1.5 or earlier also behaved that way.

As noted in past emails to golang-dev and on golang.org/wiki/Go-Release-Cycle, the constraints above are stricter than in past cycles. An explicit goal is to ship the first beta on time, by May 31, instead of many weeks late as has been our past practice. (If the past pattern held, this release's first beta would be seven weeks late, or one week before the scheduled release date.)

Thanks.

Russ

[brainman]

> Open Go 1.7 bugs for problems that are not new since Go 1.6 should in general be postponed (moved to Unplanned milestone): if Go 1.6 behaved that way, it's probably OK for Go 1.7 to continue to behave that way. That's even more true if Go 1.5 or earlier also behaved that way.

I am worried that this policy will discourage us to fix old but hard to fix bugs. Bugs should not be judged on how old they are. Maybe old bug affects many users.

Alex

[Brad Fitzpatrick]

On Tue, May 3, 2016 at 4:48 PM, brainman <alex.b...@gmail.com> wrote:

> Open Go 1.7 bugs for problems that are not new since Go 1.6 should in general be postponed (moved to Unplanned milestone): if Go 1.6 behaved that way, it's probably OK for Go 1.7 to continue to behave that way. That's even more true if Go 1.5 or earlier also behaved that way.

I am worried that this policy will discourage us to fix old but hard to fix bugs. Bugs should not be judged on how old they are. Maybe old bug affects many users.

Let's judge it on a case-by-case basis. If you have a fix for an old bug affecting many users, we can discuss.

[Dave Cheney]

Hi Russ,

Can you comment on how the performance work on the compiler fits into
the freeze? Is the change window closed, or is there some scope for
reasonable tweaks?

As of this morning, compiling jujud from scratch tip was 58% slower than 1.4.3.

Thanks

Dave

> --
> You received this message because you are subscribed to the Google Groups
> "golang-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-dev+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Russ Cox]

Hi Dave,

On Tue, May 3, 2016 at 5:24 PM Dave Cheney <da...@cheney.net> wrote:

Can you comment on how the performance work on the compiler fits into
the freeze? Is the change window closed, or is there some scope for
reasonable tweaks?

The performance work is basically done. There might be room for 1-line CLs with significant effect this week, but nothing affecting many lines of code that has non-trivial risk of introducing bugs.

 

As of this morning, compiling jujud from scratch tip was 58% slower than 1.4.3.

I'm not sure whether you meant this as a declaration of success or a justification for more work. Compared to Go 1.6, that looks great. I'm sorry that we won't get it significantly closer to Go 1.4 - more to do for next round - but overall I'm thrilled with the improvements this cycle. Now is the time to start focusing on getting things release-ready, though, not further changes.

Russ

[Dave Cheney]

On Wed, May 4, 2016 at 8:29 AM, Russ Cox <r...@golang.org> wrote:
> Hi Dave,
>
> On Tue, May 3, 2016 at 5:24 PM Dave Cheney <da...@cheney.net> wrote:
>>
>> Can you comment on how the performance work on the compiler fits into
>> the freeze? Is the change window closed, or is there some scope for
>> reasonable tweaks?
>
>
> The performance work is basically done. There might be room for 1-line CLs
> with significant effect this week, but nothing affecting many lines of code
> that has non-trivial risk of introducing bugs.

Thank you for clarifying. I agree that is the safest choice, the time
for heroic refactoring is past.

>
>>
>> As of this morning, compiling jujud from scratch tip was 58% slower than
>> 1.4.3.
>
>
> I'm not sure whether you meant this as a declaration of success or a
> justification for more work. Compared to Go 1.6, that looks great. I'm sorry
> that we won't get it significantly closer to Go 1.4 - more to do for next
> round - but overall I'm thrilled with the improvements this cycle. Now is
> the time to start focusing on getting things release-ready, though, not
> further changes.
>

I'm sorry I wasn't clear. I'm both happy and frustrated with the
result of the 1.7 efforts. Getting the compiler from 178% to 58% of
1.4.3 is a great result, and the work that David Crawshaw and Michael
Hudson did on improving the linker makes incremental compilation and
the edit/compile/test loop significantly better than 1.6 (and even
1.4.3) for day to day development. Something that is not accurately
captured by this headline figure.

I'm frustrated because, with the exception of gri's importer change,
progress on improving compiler performance had basically flatlined
[1], it seems like all of the easy wins have been exhausted.

1. https://docs.google.com/spreadsheets/d/1mczKWp3DUuQvIAwZiORD29j5LRb96QZC4mZDn72gAE4/edit#gid=1771886866

Thanks

Dave

> Russ

[Josh Bleecher Snyder]

I'm frustrated because, with the exception of gri's importer change,
progress on improving compiler performance had basically flatlined
[1], it seems like all of the easy wins have been exhausted.

It also looked like that at the end of the Go 1.5 cycle. And I'm not sure I'd describe the efforts this cycle as "easy". :)

Just off the top of my head, for Go 1.8+:

* we can probably make regalloc generate much less garbage

* we can do some structural changes of some kinds of Nodes to generate fewer of them overall

* I believe mdempsky had ideas about getting rid of many ODCLFUNCs

* I am playing with improving inlining decisions, which I think could help compile times non-trivially

* I don't know whether Austin and Rick's planned changes to how we alloc and manage free lists have happened, but those could yield 5-10%, since the compiler is so alloc-heavy

* if we get rid of the old backend, Nodes might shrink more

* mdempsky might revive his caching of ptr-of and slice-of types

* maybe some of the discussed dwarf gen changes will reduce link time

* bringing SSA to non-x86 platforms will help them a lot I suspect

and I'm sure there is more I'm not thinking of—typing this on my phone. None of these are promises, obviously, and maybe the line will stay flat...but I'm not throwing in the towel quite yet. :)

Josh

P.S. Please keep sending those charts. It keeps it on people's minds and seeing how small improvements add up over time is a good motivator.

[Robert Griesemer]

On Tue, May 3, 2016 at 7:12 PM, Dave Cheney <da...@cheney.net> wrote:

I'm sorry I wasn't clear. I'm both happy and frustrated with the
result of the 1.7 efforts. Getting the compiler from 178% to 58% of
1.4.3 is a great result, and the work that David Crawshaw and Michael
Hudson did on improving the linker makes incremental compilation and
the edit/compile/test loop significantly better than 1.6 (and even
1.4.3) for day to day development. Something that is not accurately
captured by this headline figure.

I'm frustrated because, with the exception of gri's importer change,
progress on improving compiler performance had basically flatlined
[1], it seems like all of the easy wins have been exhausted.

I'm not too worried. For one, progress on compiler performance has actually progressed fairly steadily all the way up to the freeze. Secondly, the Go team is very keen on reducing compile times and it continues to be of significant focus going forward. Finally, the number of people working on the compiler has never been larger!

But let's not get side-tracked. As Russ has pointed out, we need to catch up on our delays with the releases, and who knows - if things work out well we have an option of releasing early, which in turn would open the door for a longer 1.8 dev cycle.

Finally, thanks so much for crediting my export format change, but I believe it contributed only little to the difference we see between 1.6 and now, at least as far as I can tell from my own measurements (quite frankly, I'm surprised about the impact it appears to have for jujud build times). Much of the improved compile times comes from better code, improved GC, and lots of cleanups by lots of people throughout. For instance, I saw a larger program of mine improve by 10% over the last month, w/o me doing anything.

And yes, thanks for those charts! It's another real-life benchmark, and we need more of those.

- gri

[Dave Cheney]

Hi Robert,

Yes, your export/import change did have a significant and measurable
impact on my benchmark. Which is fantastic, confusing, and frustrating
in that order as while many other changes that Josh, Matthew, khr,
myself, and others landed to the compiler, which looked good on
compilebench, they just didn't move the needle on this benchmark. I
worry that there is something that I just don't understand about
stages of this job and how they are interacting, but so far I haven't
been able to uncover a bottleneck or other smoking gun.

Thanks

Dave

[Brendan Tracey]

I'm excited about this release, my compile times are small but execution time matters a lot more SSA should be a step in the right direction. However, our benchmarks have been oscillating significantly throughout the cycle. Below is some data comparing 1.6 and tip (+082881d) on darwin/amd64.

The Idamax regression is issue 14995. Ddot currently looks like it's flat through the cycle, but as of the fix for 14511 some of the benchmarks were 20% faster. The Dgemm regressions are new to me as of this post. I can open up an issue, though the code is significantly more complicated, and so it'll be harder to determine the cause.

Hopefully some of these concerns will be addressed in 1.8. There are some large (though likely not low-hanging) fruit that could have a significant impact on numeric performance.

Thanks for all the work.

go get -u -t github.com/gonum/blas/native
cd $GOPATH/src/github.com/gonum/blas/native
go test -bench=Dgem -tags=noasm -count=5   
(or other regex)

DgemmSmSmSm-8             1.98µs ± 1%  1.93µs ± 2%     ~     (p=0.056 n=5+5)
DgemmMedMedMed-8           593µs ± 3%   690µs ±15%     ~     (p=0.095 n=5+5)
DgemmMedLgMed-8           4.49ms ±10%  5.24ms ± 3%  +16.85%  (p=0.008 n=5+5)
DgemmLgLgLg-8              393ms ± 3%   483ms ± 3%  +22.78%  (p=0.008 n=5+5)
DgemmLgSmLg-8             6.05ms ± 5%  6.91ms ± 3%  +14.22%  (p=0.008 n=5+5)
DgemmLgLgSm-8             4.67ms ± 5%  5.32ms ± 2%  +13.86%  (p=0.008 n=5+5)
DgemmHgHgSm-8              427ms ± 4%   391ms ± 6%   -8.32%  (p=0.016 n=5+5)
DgemmMedMedMedTNT-8        654µs ±12%   767µs ± 2%  +17.30%  (p=0.008 n=5+5)
DgemmMedMedMedNTT-8        624µs ± 9%   675µs ± 4%     ~     (p=0.095 n=5+5)
DgemmMedMedMedTT-8         835µs ± 1%   800µs ± 3%   -4.17%  (p=0.008 n=5+5)
DgemvSmSmNoTransInc1-8     215ns ± 1%   209ns ±16%     ~     (p=0.190 n=4+5)
DgemvSmSmNoTransIncN-8     243ns ± 1%   229ns ± 1%   -5.75%  (p=0.008 n=5+5)
DgemvSmSmTransInc1-8       218ns ± 0%   215ns ± 5%     ~     (p=0.127 n=5+5)
DgemvSmSmTransIncN-8       267ns ± 3%   240ns ± 2%  -10.12%  (p=0.008 n=5+5)
DgemvMedMedNoTransInc1-8  11.0µs ± 2%  10.7µs ± 1%   -2.95%  (p=0.008 n=5+5)
DgemvMedMedNoTransIncN-8  13.4µs ± 2%  13.0µs ± 4%     ~     (p=0.056 n=5+5)
DgemvMedMedTransInc1-8    13.3µs ± 4%  12.7µs ± 2%   -4.28%  (p=0.008 n=5+5)
DgemvMedMedTransIncN-8    16.0µs ± 1%  13.3µs ± 3%  -16.75%  (p=0.008 n=5+5)
DgemvLgLgNoTransInc1-8     971µs ± 3%   940µs ± 2%   -3.18%  (p=0.008 n=5+5)
DgemvLgLgNoTransIncN-8    1.21ms ± 3%  1.21ms ± 2%     ~     (p=0.841 n=5+5)
DgemvLgLgTransInc1-8      1.21ms ± 1%  1.22ms ± 3%     ~     (p=0.421 n=5+5)
DgemvLgLgTransIncN-8      1.50ms ± 2%  1.21ms ± 2%  -19.44%  (p=0.008 n=5+5)
DgemvLgSmNoTransInc1-8    18.5µs ± 2%  16.9µs ± 1%   -8.57%  (p=0.008 n=5+5)
DgemvLgSmNoTransIncN-8    20.8µs ± 1%  20.0µs ± 2%   -4.25%  (p=0.008 n=5+5)
DgemvLgSmTransInc1-8      17.4µs ± 2%  17.4µs ± 3%     ~     (p=0.690 n=5+5)
DgemvLgSmTransIncN-8      22.5µs ± 2%  20.0µs ± 1%  -11.19%  (p=0.008 n=5+5)
DgemvSmLgNoTransInc1-8    9.09µs ± 1%  8.98µs ± 2%     ~     (p=0.222 n=5+5)
DgemvSmLgNoTransIncN-8    11.7µs ± 1%  11.7µs ± 1%     ~     (p=0.548 n=5+5)
DgemvSmLgTransInc1-8      12.7µs ± 1%  12.2µs ± 1%   -3.86%  (p=0.008 n=5+5)
DgemvSmLgTransIncN-8      15.9µs ± 2%  12.6µs ± 2%  -20.54%  (p=0.008 n=5+5)
DdotSmallBothUnitary-8    21.0ns ± 1%  19.2ns ± 2%   -8.21%  (p=0.008 n=5+5)
DdotSmallIncUni-8         23.5ns ± 2%  24.7ns ± 1%   +5.37%  (p=0.008 n=5+5)
DdotSmallUniInc-8         23.2ns ± 4%  23.1ns ± 4%     ~     (p=1.000 n=5+5)
DdotSmallBothInc-8        23.4ns ± 2%  23.2ns ± 1%     ~     (p=0.143 n=5+5)
DdotMediumBothUnitary-8    919ns ± 3%   919ns ± 2%     ~     (p=0.968 n=5+5)
DdotMediumIncUni-8        1.22µs ± 1%  1.18µs ± 2%   -3.66%  (p=0.008 n=5+5)
DdotMediumUniInc-8        1.18µs ± 0%  1.17µs ± 1%     ~     (p=0.119 n=5+5)
DdotMediumBothInc-8       1.27µs ± 4%  1.22µs ± 3%   -4.55%  (p=0.008 n=5+5)
DdotLargeBothUnitary-8    94.0µs ± 2%  90.7µs ± 4%     ~     (p=0.056 n=5+5)
DdotLargeIncUni-8          194µs ± 6%   204µs ± 7%     ~     (p=0.151 n=5+5)
DdotLargeUniInc-8          128µs ± 1%   128µs ± 2%     ~     (p=0.548 n=5+5)
DdotLargeBothInc-8         256µs ± 2%   265µs ± 3%   +3.34%  (p=0.016 n=5+5)
DdotHugeBothUnitary-8     11.4ms ± 4%  11.3ms ± 2%     ~     (p=1.000 n=5+5)
DdotHugeIncUni-8          27.6ms ± 1%  28.0ms ± 2%     ~     (p=0.095 n=5+5)
DdotHugeUniInc-8          18.8ms ± 2%  19.4ms ± 2%   +3.32%  (p=0.016 n=5+5)
DdotHugeBothInc-8         35.1ms ± 2%  36.0ms ± 2%   +2.57%  (p=0.032 n=5+5)
IdamaxSmallUnitaryInc-8   31.7ns ± 7%  44.7ns ±12%  +41.19%  (p=0.008 n=5+5)
IdamaxSmallPosInc-8       27.8ns ±10%  40.2ns ±10%  +44.75%  (p=0.008 n=5+5)
IdamaxMediumUnitaryInc-8  1.53µs ± 4%  2.16µs ± 4%  +40.95%  (p=0.008 n=5+5)
IdamaxMediumPosInc-8      1.79µs ± 1%  2.59µs ± 7%  +44.16%  (p=0.008 n=5+5)
IdamaxLargeUnitaryInc-8    143µs ± 1%   214µs ± 3%  +50.24%  (p=0.008 n=5+5)
IdamaxLargePosInc-8        192µs ± 1%   264µs ± 5%  +37.53%  (p=0.008 n=5+5)
IdamaxHugeUnitaryInc-8    14.4ms ± 1%  20.9ms ± 3%  +45.16%  (p=0.008 n=5+5)
IdamaxHugePosInc-8        26.2ms ± 1%  30.7ms ± 2%  +16.93%  (p=0.008 n=5+5)

[Austin Clements]

On Tue, May 3, 2016 at 7:57 PM, Josh Bleecher Snyder <josh...@gmail.com> wrote:

* I don't know whether Austin and Rick's planned changes to how we alloc and manage free lists have happened, but those could yield 5-10%, since the compiler is so alloc-heavy

These landed in 56b54912, with a few follow-up performance fixes between there and a20fd1f6. I haven't benchmarked the effect on the compiler since I'm not set up with the compiler benchmarks. I'm not sure it'll be 5--10%, but i would hope it's in the 3--5% range.

[Dave Cheney]

Here is data point from a633d76.

The summary is the fixes that landed over night show a measurable
improvement in compilebench, something is preventing that translating
into an improvement in wall time on a full build of a large project
(gogs is ~100 packages, kube is 304 and juju is 512).

[Josh Bleecher Snyder]

> The summary is the fixes that landed over night show a measurable
> improvement in compilebench, something is preventing that translating
> into an improvement in wall time on a full build of a large project
> (gogs is ~100 packages, kube is 304 and juju is 512).

I'm not sure what's going on with gogs/kube/juju (maybe a work
scheduling problem?). However, I will note that there's a compilebench
bug that has been affecting the results reported in some commit
messages for MakeBash and binary sizes, for those like me who use the
-compile flag.

For example, https://github.com/golang/go/commit/bc1989f1154b5b8f235e7e4932d935e490d6e79e
reported in the commit message:

MakeBash 40.2s ±13% 39.8s ± 9% ~ (p=0.648 n=23+23)

Re-running without using -compile yields:

MakeBash 38.9s ± 1% 37.2s ± 2% -4.38% (p=0.000 n=24+25)

The bug is https://github.com/rsc/compilebench/issues/5 for those who
want to track it.

-josh