Should we pause state before calling clobberMemory

[Abhi Arora]

I'm currently in the process of benchmarking my database library and have provided a sample code snippet for reference.

BENCHMARK_DEFINE_F(ISqlConnectionBenchmark, benchmarkConnectionConstruction)(benchmark::State& state) {
    for (auto _ : state) {
         auto connection = ISqlConnectionBenchmark::createConnection();
         ::benchmark::DoNotOptimize(connection.get());

         // Skip measuring connection destruction
         connection.reset();
         connection = nullptr;
         ::benchmark::ClobberMemory();
    }
}

BENCHMARK_REGISTER_F(ISqlConnectionBenchmark, benchmarkConnectionConstruction);

To prevent optimization, I've employed the use of DoNotOptimize and ClobberMemory techniques. However, a team member has suggested that we should halt the timer before invoking the ClobberMemory function. This suggestion seems reasonable, especially if the ClobberMemory function is transformed into assembly code that effectively flushes all cache lines. I've conducted research on various examples and CPPCon presentations, but none have demonstrated such a practice.

Is there any relevant information or insight that I might be overlooking?

Regards, 
Abhi Arora

[dominic hamon]

hello

i think we caught up on the discord but to repeat here:  the pause/resume will add some overhead which may overwhelm faster benchmarks.  also, the concept of the library is for comparison of performance rather than absolutes (relative to different implementations, or relative over time).  as such, the clobbering will introduce some performance artefacts but they should be consistent across runs so the relative changes should capture the important bits.

nothing is stopping you from pausing/resuming but you may not get useful absolute numbers (they'll be more pessimistic than even the clobber).

dominic hamon | google

there are no bad ideas; only good ideas that go horribly wrong.

--
You received this message because you are subscribed to the Google Groups "benchmark-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to benchmark-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/benchmark-discuss/b0e3a5f2-ec9d-43c5-802d-b787d66646c3n%40googlegroups.com.