Please help me understand multi-threaded benchmark numbers..

[sre...@gmail.com]

Hi,

I am trying to benchmark a piece of code with multiple threads and was hoping you someone can help me understand these numbers..

Here is an Empty benchmark I wrote:

--
static void BM_Empty(benchmark::State& state) {
while (state.KeepRunning()) {
// Do nothing
}
}

BENCHMARK(BM_Empty)->Threads(1);
BENCHMARK(BM_Empty)->Threads(2);
BENCHMARK(BM_Empty)->Threads(4);

BENCHMARK(BM_Empty)->Threads(1)->UseRealTime();
BENCHMARK(BM_Empty)->Threads(2)->UseRealTime();
BENCHMARK(BM_Empty)->Threads(4)->UseRealTime();

Benchmark Time CPU Iterations
----------------------------------------------------------------------------
BM_Empty/threads:1 2 ns 2 ns 418132308
BM_Empty/threads:2 1 ns 2 ns 418901068
BM_Empty/threads:4 0 ns 2 ns 394432372
BM_Empty/real_time/threads:1 2 ns 2 ns 418057380
BM_Empty/real_time/threads:2 1 ns 2 ns 836633562
BM_Empty/real_time/threads:4 0 ns 2 ns 1621747004

--------

Questions
1) Are the numbers totals or averages? (i.e sum of all iterations across all threads or average iterations per thread)?

2) When I just use cpu time (i.e NOT use UseRealTime()), the number of iterations seem to remain more-or-less constant no matter how many threads I use (so . However, when I use walltime, the number of iterations seem to scale almost linearly with the number of threads.. I am not sure how to explain this.

3) Since "state.iterations()" would return me the number of iterations per thread, is there a way to get the total number of iterations across all threads?

thanks,
Sree

[sr...@google.com]

Nevermind, going through the code i got my questions answered

1) The number of iterations are total number of iterations across all threads
2) As per https://github.com/google/benchmark/blob/master/src/benchmark.cc#L59, looks like the total cpu time is divided across all threads. So UseRealTime() is what I should be using
3) "State" is per-thread. So state.iterations() is number of iterations per thread.

[Dominic Hamon]

Hi Sree

I'm glad you found your answers. Sorry I didn't get a chance to put together the answer sooner.

Dominic Hamon | Google

There are no bad ideas; only good ideas that go horribly wrong.

--
You received this message because you are subscribed to the Google Groups "benchmark-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to benchmark-discuss+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.