benchmarks

[Russ Cox]

Now that Go 1 has been out for a while, we should think about
expanding the official set of Go 1 benchmarks, which live in
test/bench/go1. These are the benchmarks we'll use to measure
performance improvements for Go 1.1.

Suggestions welcome. It is better to have tests of important aggregate
functionality (say, JSON encoding) than microbenchmarks.

Russ

[Brad Fitzpatrick]

I'd like to see an HTTP server benchmark included, with or without the client half too.

That tests a lot of parts, including the net package's event loop.

Or is that too high-level?

[Russ Cox]

On Tue, May 29, 2012 at 12:18 PM, Brad Fitzpatrick <brad...@google.com> wrote:
> I'd like to see an HTTP server benchmark included, with or without the
> client half too.
>
> That tests a lot of parts, including the net package's event loop.
>
> Or is that too high-level?

That seems fine, it just needs to be self-contained (cannot require ab
or siege or whatever), so that we can run it anywhere we can run Go.

Russ

[Brad Fitzpatrick]

Yup. There's already a self-contained benchmark like that in the tree.

[Lorenzo Stoakes]

I'd like to see a benchmark for heavy map usage, though to avoid it being a microbenchmark, it would need to be moulded into a realistic use of maps (I mention this since I recently coded up a puzzle in go which seemed surprisingly slow hash-wise).

--
Lorenzo Stoakes
http://www.codegrunt.co.uk

[Ingo Oeser]

On Tuesday, May 29, 2012 6:11:17 PM UTC+2, rsc wrote:

Suggestions welcome. It is better to have tests of important aggregate
functionality (say, JSON encoding) than microbenchmarks.

 

- channel throughput with MAXPROCS=1 relative to amount of channels and go routines 

+ Tests what happens if one or both of those rise a lot

+ Be able to find maximum of it on specific machine to support system sizing decisions

 

- channel throughput with MAXPROCS=8 or more to test the M:N scheduling performance.

+ base to study cacheline bouncing effects and enable work on avoiding it.

Since this doesn't involve the "gosched on syscall" logic, it will be quite system independent.

 

[Russ Cox]

On Tue, May 29, 2012 at 3:23 PM, Ingo Oeser <night...@googlemail.com> wrote:
> - channel throughput with MAXPROCS=1 relative to amount of channels and go
> routines
> + Tests what happens if one or both of those rise a lot
> + Be able to find maximum of it on specific machine to support system sizing
> decisions
>
> - channel throughput with MAXPROCS=8 or more to test the M:N scheduling
> performance.
> + base to study cacheline bouncing effects and enable work on avoiding it.
>
> Since this doesn't involve the "gosched on syscall" logic, it will be quite
> system independent.

The goal of the benchmark suite is to record real Go code whose
performance we care about, not contrived microbenchmarks. The
microbenchmarks will only change when the thing they are contrived to
measure changes. They are fine to have but they are not good
aggregate benchmarks. The latter are more interesting and harder to
find.

Russ

[Dave Cheney]

I think that gri's recent go/parser benchmark is useful as a real
world case. If you agree, would you like to see aCL adding to the
test/bench/go1 set ?

Cheers

Dave

[Russ Cox]

On Sat, Jun 2, 2012 at 11:46 PM, Dave Cheney <da...@cheney.net> wrote:
> I think that gri's recent go/parser benchmark is useful as a real
> world case. If you agree, would you like to see aCL adding to the
> test/bench/go1 set ?

Sure, but the test needs to be a single self-contained binary, so
please include the parser input as a string literal in its own file.
See jsondata_test.go although in this case there's not enough
data to worry about fancy encodings.

Russ

[Dave Cheney]

Sure thing.

On a related note some of those benchmarks consume too much memory to
be run on Arm platforms, do you have any suggestions on how they could
be adapted to deal with machines with piddly amounts of ram?

Cheers

Dave

[Russ Cox]

On Sat, Jun 2, 2012 at 11:59 PM, Dave Cheney <da...@cheney.net> wrote:
> On a related note some of those benchmarks consume too much memory to
> be run on Arm platforms, do you have any suggestions on how they could
> be adapted to deal with machines with piddly amounts of ram?

Which ones? Is it compilation or running that is the problem?

Russ

[Dave Cheney]

I was going to blame BinaryTree17 or Fannkuch11 but it turns out it's
the init routines for fasta and revcomp which bloats the heap out to
500mb before the tests start.

http://codereview.appspot.com/6272045 reduces the heap size to 51mb on
my host. What do you think ?

Cheers

Dave

[Russ Cox]

On Sun, Jun 3, 2012 at 12:31 AM, Dave Cheney <da...@cheney.net> wrote:
> I was going to blame BinaryTree17 or Fannkuch11 but it turns out it's
> the init routines for fasta and revcomp which bloats the heap out to
> 500mb before the tests start.
>
> http://codereview.appspot.com/6272045 reduces the heap size to 51mb on
> my host. What do you think ?

maybe instead of taking it out, it should be 25e5 if runtime.GOARCH=="arm"?