Generate benchmark binary

[Thomas Pelletier]

Hi,

I would like to be able to generate a binary for an existing benchmark, with the intent of using this binary with performance analysis tools (for example perf, cachegrind, etc). Is there an known method to achieve that?

The current workflow is to copy the code of the benchmark, put it in a new main() in a new module, and manually remove the bench loop, fixing imports, etc. I'd love to just point a tool at an existing benchmark and get a minimal binary to play with.

Initially I figured that using the binary generated by the test runner would be enough. Something like:

go test -c ./benchmark && perf stat ./benchmark.test -test.run=None -test.bench=Something -test.benchtime=1x -test.count=1

But it turns out the overhead of the harness is fairly significant for that kind of operation.

Second idea is to mimic go test but generate a minimal main function instead. This approach seems tricky as it actually compiles the test files and dependencies using the compiler API, which is not public. I guess one way around would be to fork golang/go and implement the tool in src/cmd/go, but that seems quite the lift to maintain.

Third idea is to manipulate the source directly. Given a module and a benchmark name, copy the module's code into a temporary directory, use go/parser on that module, look for the BenchmarkX() function, remove the references to testing.B, put that code into a new module, in a main() function, then point the actual go compiler to that temporary directory. This seems extremely brittle, full of corner cases.

Any other idea?

Thank you!
Thomas

[Thomas Pelletier]

If anyone finds this, here is a proof of concept: https://github.com/pelletier/go-bb.