It would be interesting to see size of the created binary on the same graph.
--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Is the graph labeled correctly?
On Mon, Mar 7, 2016 at 11:22 AM, Dave Cheney <da...@cheney.net> wrote:
Hello,By a quirk of fate, the juju codebase has had to maintain compatibility with go 1.2 since that release. This afternoon I did an experiment compiling and linking our biggest command, 512 packages in total, with the latest release compiler from each minor revision.The raw results are here, along with a graph on the second tab. The test machine was a thinkpad x220, Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz, 8gb of ram, everything running out of a ramdisk on /tmp.
<image.png>
https://docs.google.com/spreadsheets/d/1mczKWp3DUuQvIAwZiORD29j5LRb96QZC4mZDn72gAE4/edit?usp=sharingI have heard rumours that others are also collecting benchmark data, but they may not have access to such a pathological data set as juju :) I intend to keep this data set current for as long as I have access to this machine.ThanksDave
Is the graph labeled correctly?
-rwxr-xr-x 1 dfc dfc 88174297 Mar 7 20:43 jujud
and comparing that to one just compiled with tip
-rwxrwxr-x 1 dfc dfc 79450868 Mar 7 12:44 /home/dfc/bin/jujud
Which looks like 10% reduction in binary size!
It's just what is in tip at the time. The only revisions that I believe has the ssa checks enabled are the ones post 1.6.
I intend to keep the chart updated over time.
The peek was go 1.5, things look to be getting better since then.
The line I'm tracking is real time, which grew by 2.5x after go 1.5.
I'll drop reporting the sys and user times on the graph, it is confusing.
If the SSA enabled compiler produces (smaller and) faster binaries, then I think the SSA enabled version from tip should be (re-)compiled by itself to see the best results...?
(In addition to disabling consistency checks)
version 1.4.3 | real | 00:00:43 | |
user | 00:01:39 | ||
sys | 00:00:17 | ||
version 1.5.3 | real | 00:02:19 | |
user | 00:07:05 | ||
sys | 00:00:24 | ||
version 1.6 | real | 00:02:09 | |
user | 00:06:35 | ||
sys | 00:00:20 | ||
rev 6ed1038 | real | 00:02:11 | |
user | 00:06:39 | ||
sys | 00:00:19 |
However I am misinterpreting the graph, there are numbers on it that are worse than I can reproduce in my own code base, which means juju is causing worse performance.
FYI turning off GC can actually decrease compilation times (especially user time):
$ go version
go version go1.6 linux/amd64
$ time go build -a std
real 0m6.569s
user 0m27.592s
sys 0m2.719s
$ time GOGC=off go build -a std
real 0m5.583s
user 0m12.872s
sys 0m3.597s
It looks something like the the attached SVG. Definitely GC-dominated,
but the graph gives up several interesting compiler functions.
--
Thanks Egon,On my list today was to modify my local cmd/go to output build times per package so I could sort them. I'll report back when I have done that.
> There is 1 million lines of non test code taking 1 minute of real time, and these packages are 100k total. Combined they take 20 seconds, or 1/3 of the build time for 1/10 the code. It would be great to see some profiles of those two packages if someone has time.
Not sure if you have seen it but I've sent some profiling results of compiling types package earlier to this thread.
>
> On Wednesday, March 9, 2016 at 10:21:40 AM UTC+13, Dave Cheney wrote:
>>
>> https://gist.github.com/davecheney/401ee7514927887683cb
>>
>> ^ tip (0c7ccbf6) fresh this morning
>>
>> go1.4 does not support -toolexec, so I'll bodge something in and do
>> the same test there.
>>
>> On Wed, Mar 9, 2016 at 7:42 AM, Dave Cheney <da...@cheney.net> wrote:
>> > /me slaps forehead
>> >
>> > go build -toolexec="/usr/bin/time -f '%e %U %S %C'"
>> > github.com/juju/juju/cmd/jujud
>> >
>> > with some post processing looks promising, I'll report back with the results.
>> >
>> > On Wed, Mar 9, 2016 at 7:34 AM, Russ Cox <r...@golang.org> wrote:
>> >> On Tue, Mar 8, 2016 at 3:27 PM, Dave Cheney <da...@cheney.net> wrote:
>> >>>
>> >>> Thanks Egon,
>> >>> On my list today was to modify my local cmd/go to output build times per
>> >>> package so I could sort them. I'll report back when I have done that.
>> >>
>> >>
>> >> go build -toolexec time, right? :-)
>> >>
>> >> Russ
>
The vim25/types package is odd, it contains 5200+ func init()'s, which is part of the autogenerated nature of the code (I'm thinking back to what bwk said about generated code being the meanest to compile).
linux/amd64 | darwin/amd64 | |
1.4.3 | 1.162 | 2.111 |
1.5.3 | 2.417 | 3.846 |
1.6 | 2.521 | 3.758 |
aa3650f | 1.711 | 2.346 |
Hello,By a quirk of fate, the juju codebase has had to maintain compatibility with go 1.2 since that release. This afternoon I did an experiment compiling and linking our biggest command, 512 packages in total, with the latest release compiler from each minor revision.The raw results are here, along with a graph on the second tab. The test machine was a thinkpad x220, Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz, 8gb of ram, everything running out of a ramdisk on /tmp.
https://docs.google.com/spreadsheets/d/1mczKWp3DUuQvIAwZiORD29j5LRb96QZC4mZDn72gAE4/edit?usp=sharingI have heard rumours that others are also collecting benchmark data, but they may not have access to such a pathological data set as juju :) I intend to keep this data set current for as long as I have access to this machine.ThanksDave
Playing with genpkg on tip:go run gen.go -n 3000 -noinitfn > x/types.go
time go build -gcflags='-d=ssa/check/off -memprofile=x.mprof' ./x
go tool pprof -alloc_space /Users/drchase/GoogleDrive/work/go-ssa/pkg/tool/darwin_amd64/compile x/x.mprofSo it happens that check is a pig and I haven't looked into that yet, but...
<snipped>
i.e., (sorry for the crap formatting) it's reference linking in ssa.go, and regalloc, by a lot.newValue is almost 100% linkForwardReferences -> resolveFwdRef -> lookupVarOutgoing -> (*Block)NewValue0A -> (*Func)NewValuelookupVarOutgoing (flat cost) is all at m[name] = v (i.e., forward references).buildInterferenceGraph is just the interference graph.I think that's good for a start.
--
lucky(~/devel/benchjuju) % grep juju/state.a\ /tmp/benchjuju.bb48b86-p*.txt | cut -f -6 -d ' '
/tmp/benchjuju.bb48b86-p2.txt:7.77 9.82 0.15 /home/dfc/go/pkg/tool/linux_amd64/compile -o $WORK/github.com/juju/juju/state.a
/tmp/benchjuju.bb48b86-p2.txt:8.61 10.78 0.24 /home/dfc/go/pkg/tool/linux_amd64/compile -o $WORK/github.com/juju/juju/state.a
/tmp/benchjuju.bb48b86-p2.txt:9.20 11.58 0.19 /home/dfc/go/pkg/tool/linux_amd64/compile -o $WORK/github.com/juju/juju/state.a
/tmp/benchjuju.bb48b86-p2.txt:9.08 11.37 0.20 /home/dfc/go/pkg/tool/linux_amd64/compile -o $WORK/github.com/juju/juju/state.a
/tmp/benchjuju.bb48b86-p2.txt:9.36 11.82 0.21 /home/dfc/go/pkg/tool/linux_amd64/compile -o $WORK/github.com/juju/juju/state.a
/tmp/benchjuju.bb48b86-p4.txt:11.74 12.20 0.24 /home/dfc/go/pkg/tool/linux_amd64/compile -o $WORK/github.com/juju/juju/state.a
/tmp/benchjuju.bb48b86-p4.txt:14.10 14.56 0.26 /home/dfc/go/pkg/tool/linux_amd64/compile -o $WORK/github.com/juju/juju/state.a
/tmp/benchjuju.bb48b86-p4.txt:14.70 14.83 0.31 /home/dfc/go/pkg/tool/linux_amd64/compile -o $WORK/github.com/juju/juju/state.a
/tmp/benchjuju.bb48b86-p4.txt:14.81 15.10 0.27 /home/dfc/go/pkg/tool/linux_amd64/compile -o $WORK/github.com/juju/juju/state.a
/tmp/benchjuju.bb48b86-p4.txt:14.94 15.21 0.31 /home/dfc/go/pkg/tool/linux_amd64/compile -o $WORK/github.com/juju/juju/state.a
Good news everyone! After a month's work the results are really starting to pay off. The following is a graph of real and user time relative to 1.4.3. Both real and user time is trending downwards. Real time is just above 200% relative to 1.4.3, and both real and user time are trending downwards at the same rate.
I'm using linux. I know that macho generation in the darwin linker is
a performance pain point.
--
OS X 10.11.4 i5-4258U CPU @ 2.40GHz 2 core
version real user sys real/1.4.3go 1.4.3 10.93 19.88 4.08 --go 1.5.3 22.32 54.68 4.87 2.04xgo 1.6 21.87 52.19 4.88 2.00xgo 1.7 d8ee180 16.98 40.72 4.08 1.55x
Linux 3.16 E5-1650v2 @ 3.50GHz 6 core
version real user sys real/1.4.3go 1.4.3 4.78 12.87 2.06 --go 1.5.3 10.83 44.82 2.51 2.27xgo 1.6 10.76 42.56 2.30 2.25xgo 1.7 d8ee180 8.77 33.64 2.02 1.83x
--
gogs has many build options. Could you please post the complete list
of steps you used to build gogs so I can turn that into a benchmark
suite.
I think this benchmark must be executed automatically, and included in CI.
--
There are some opinions about how feasible is https://github.com/gonum/plot to this job, on gonum-dev list.
Hello,By a quirk of fate, the juju codebase has had to maintain compatibility with go 1.2 since that release. This afternoon I did an experiment compiling and linking our biggest command, 512 packages in total, with the latest release compiler from each minor revision.The raw results are here, along with a graph on the second tab. The test machine was a thinkpad x220, Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz, 8gb of ram, everything running out of a ramdisk on /tmp.
https://docs.google.com/spreadsheets/d/1mczKWp3DUuQvIAwZiORD29j5LRb96QZC4mZDn72gAE4/edit?usp=sharingI have heard rumours that others are also collecting benchmark data, but they may not have access to such a pathological data set as juju :) I intend to keep this data set current for as long as I have access to this machine.ThanksDave
--