Go 1.24 BLooper

[peterGo]

Go 1.24 introduces B.Loop() to replace B.N.

https://pkg.go.dev/tes...@go1.24rc2#B.Loop   
https://pkg.go.dev/tes...@go1.24rc2#B   

testing: add testing.B.Loop for iteration #61515   
https://github.com/golang/go/issues/61515   
As suggested by @rsc, b.Loop could be a clear signal to the compiler not to perform certain optimizations in the loop body that often signal to the compiler not to perform certain optimizations in the loop body.

There is a significant regression.

For example, here is an important benchmark that no longer works for a function that should be inlined (Strings_TrimNewline since strings.Lines() is not OS agnostic). b.Loop() does not optimize. b.N optimizes as is should.


$ go test blooper_test.go -run=! -bench=BenchmarkStrings
devel go1.24-3f4164f508 Mon Jan 20 09:25:11 2025 -0800
goos: linux
goarch: amd64
cpu: 12th Gen Intel(R) Core(TM) i5-1235U
noinline Strings_TrimNewline:
BenchmarkStringsBLoop-12     1000000000          1.044 ns/op
inline Strings_TrimNewline:
BenchmarkStringsBN-12       1000000000          0.6154 ns/op
$


blooper_test.go:

package main

import (
"fmt"
"runtime"
"testing"
)

// TrimNewline returns s without a trailing newline string.
// If s doesn't end with newline, s is returned unchanged.
// A newline string is LF ("\n") or CR+LF ("\r\n").
func Strings_TrimNewline(s string) string {
if len(s) > 0 && s[len(s)-1] == '\n' {
s = s[0 : len(s)-1]
if len(s) > 0 && s[len(s)-1] == '\r' {
s = s[0 : len(s)-1]
}
}
return s
}

var sinkString string

func BenchmarkStringsBLoop(b *testing.B) {
var sink string
for b.Loop() {
line := stringLine
sink = Strings_TrimNewline(line)
}
sinkString = sink
}

func BenchmarkStringsBN(b *testing.B) {
var sink string
for range b.N {
line := stringLine
sink = Strings_TrimNewline(line)
}
sinkString = sink
}

var (
nl         = "\r\n"
byteLine   = append(make([]byte, 256), nl...)
stringLine = string(byteLine)
)

func init() {
fmt.Println(runtime.Version())
}

[Michael Pratt]

There was a lot of discussion of this on the proposal, starting with (and well summarized by) https://go.dev/issue/61515#issuecomment-1646194700. If we don't inline, there will be cases like yours that wanted to benchmark with inlining but don't get it. If we do inline, there will be cases that are unintentionally inlined and simplified in unexpected ways. Ultimately, the decision was to err on the side of avoiding accidental inlining since that seems to give a better experience for folks not thinking about detailed compiler internals.

In fact, we can see evidence of dealing with the latter case (unintentional inlining) in your example, with the "stringLine" global variable. A naive implementation might just put something like `line := "\r\n\r\n\r\n"` directly in the loop body. Then once Strings_TrimNewline is inlined, since `line` is a constant, all of the len(s) operations would become constant. The compiler may even make the slicing constants and turn 100% of the function body into a constant (I don't think our compiler currently does so, but there is no reason it couldn't). This would make the benchmark very misleading. You've avoided this problem by forcing the string to load from a global, but the idea behind b.Loop avoiding optimizations is that we don't want users to have to think of these things.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/1c175710-6996-46b8-bb0a-00ff419fae40n%40googlegroups.com.