Garbage collection benchmark

591 views
Skip to first unread message

Brendan Tracey

unread,
Jan 3, 2017, 4:23:50 PM1/3/17
to golang-dev
Is the GC team interested in "real programs" that could be good benchmarks for GC, or do you have what you need?

I ask because I have a program that is GC limited. It took 6.8 seconds to run under go1.7.3, and now only takes 4.3 seconds to run under go1.8beta2. This seems like a much bigger improvement than expected from the release notes. Secondly, if I modify the program to save 2 slice allocations per iteration in the inner loop, the runtime drops to 2.5 seconds in both go versions. This seems like a very large difference, though I don't know if it's unexpected.

Details:
The program constructs a linear program based on certain logic, and then transforms my Go representation into a format that can be read by Gurobi (a commercial LP solver). The full program writes this representation to a file, though for the benchmarks above I disabled this write. The construction of the LP in Go format only takes about 1% of the runtime, though it does create a lot of persistent objects. In the conversion to Gurobi format, each constraint is turned into a []byte. The two slice allocations are temporary storage to convert the constraints between the two formats. The problem above has ~40,000 constraints and ~17,000 variables, and so the two slice allocations are each roughly "c := make([]float64, 17000)" (it's the same size for each iteration of the loop). I suspect that part of the problem is that there are a lot of long-living objects which make each GC take a while even though the number of new/dead objects is small. Secondly, I suspect that because the slices are large, they trigger GCs each iteration of the loop, even though the number of objects is small (if I remove the two allocations, but also make GOGC small, the running time is unaffected).

If it's useful to transform this into a benchmark I can.

r...@golang.org

unread,
Jan 3, 2017, 4:41:50 PM1/3/17
to golang-dev
Yes please.
The runtime team would look at it. The use case seems important.

Brendan Tracey

unread,
Jan 3, 2017, 7:53:52 PM1/3/17
to golang-dev
Oops, we got off list, but a reduced version of the problem that exhibits the behavior is at https://github.com/btracey/benchlp

Rick Hudson

unread,
Jan 3, 2017, 8:09:07 PM1/3/17
to Brendan Tracey, golang-dev
As far as licenses and such are concerned see https://golang.org/doc/contribute.html for contribution guidelines and copyright statements.


Brendan Tracey

unread,
Jan 3, 2017, 8:11:53 PM1/3/17
to Rick Hudson, golang-dev
Oh, did you want me to make a PR to Go? Is the correct location go/test/bench/garbage? Those are coded as main files instead of as Benchmarks, should I adjust my code?

Rick Hudson

unread,
Jan 4, 2017, 8:54:47 AM1/4/17
to Brendan Tracey, golang-dev
I'm just not in the position to read and/or understand your license. I just wanted to run your program a couple of times and understand how it was using the GC. I did not want to initiate a review process.

The fact that /test/bench/garbage is not in the Benchmark form is historical and is on my list of things to be fixed.


Brendan Tracey

unread,
Jan 4, 2017, 9:36:11 AM1/4/17
to Rick Hudson, golang-dev
Great. The package is go-gettable and runnable as a benchmark

go test -bench .

I only asked about the license because I didn’t know if Google had rules that applied even for testing small segments of code

Rick Hudson

unread,
Jan 4, 2017, 1:42:26 PM1/4/17
to Brendan Tracey, golang-dev
Thanks for all your work. As a general rule I don't download or read anything that is under a license.
Again thanks for the offer.


Brendan Tracey

unread,
Jan 4, 2017, 1:46:36 PM1/4/17
to Rick Hudson, golang-dev
I can remove the license. You’re the first I’ve heard with that policy, but I’ve encountered many people who refuse to use anything that doesn’t have an explicit license. I’m just trying to make it easy for you to use the code (and apparently failing).

Brendan Tracey

unread,
Jan 4, 2017, 1:50:28 PM1/4/17
to Rick Hudson, golang-dev

Rick Hudson

unread,
Jan 4, 2017, 4:09:24 PM1/4/17
to Brendan Tracey, golang-dev
Brendan, 
I was able to reproduce the speedups you mentioned but they do not seem to be the result of changes in the GC. Both 1.7 and 1.8alpha use about the same amount of their time in the GC. You can confirm by running with GODEBUG=gctrace=1 and comparing the output.
Thanks for you time.


Reply all
Reply to author
Forward
0 new messages