How to pause/restart timer in a benchmark loop?
791 views
Skip to first unread message
tapi...@gmail.com
Jul 7, 2021, 11:28:13 AM7/7/21
to golang-nuts
For example, I don't want the time consumed for "copy(data2, data)" being counted.
How to achieve this?
func BenchmarkFilter3(b *testing.B) {
data := buildOrginalData()
data2 := make([]int, len(data))
b.ResetTimer()
for i := 0; i < b.N; i++ {
copy(data2, data)
_ = Filter3(data2)
}
}
data := buildOrginalData()
data2 := make([]int, len(data))
b.ResetTimer()
for i := 0; i < b.N; i++ {
copy(data2, data)
_ = Filter3(data2)
}
}
tapi...@gmail.com
Jul 7, 2021, 11:29:30 AM7/7/21
to golang-nuts
This doesn't work:
func BenchmarkFilter3(b *testing.B) {
data := buildOrginalData()
data2 := make([]int, len(data))
b.ResetTimer()
for i := 0; i < b.N; i++ {
data := buildOrginalData()
data2 := make([]int, len(data))
b.ResetTimer()
for i := 0; i < b.N; i++ {
b.StopTimer()
copy(data2, data)
b.StartTimer()
_ = Filter3(data2)
}
}
copy(data2, data)
b.StartTimer()
_ = Filter3(data2)
}
}
peterGo
Jul 7, 2021, 12:22:28 PM7/7/21
to golang-nuts
$ go test copy_test.go -bench=. -benchmem
BenchmarkFilter3-4 358167 3209 ns/op 8192 B/op 1 allocs/op
$
}
func BenchmarkFilter3(b *testing.B) {
data := buildOrginalData()
}
b.ResetTimer()
for i := 0; i < b.N; i++ {
BenchmarkFilter3-4 358167 3209 ns/op 8192 B/op 1 allocs/op
$
package main
import "testing"
func Filter3(a []int) []int {
s := make([]int, len(a))
for i := range s {
s[i] += 42
}
return nil
}
func buildOrginalData() []int {
data := make([]int, 1024)
for i := range data {
data[i] = i
}
return data
import "testing"
func Filter3(a []int) []int {
s := make([]int, len(a))
for i := range s {
s[i] += 42
}
return nil
}
func buildOrginalData() []int {
data := make([]int, 1024)
for i := range data {
data[i] = i
}
return data
}
func BenchmarkFilter3(b *testing.B) {
data := buildOrginalData()
copies := make([][]int, b.N)
for i := range copies {
copies[i] = make([]int, len(data))
copy(copies[i], data)
for i := range copies {
copies[i] = make([]int, len(data))
copy(copies[i], data)
}
b.ResetTimer()
for i := 0; i < b.N; i++ {
_ = Filter3(copies[i])
}
}
}
}
Peter
On Wednesday, July 7, 2021 at 11:28:13 AM UTC-4 tapi...@gmail.com wrote:
jake...@gmail.com
Jul 7, 2021, 12:32:38 PM7/7/21
to golang-nuts
It would be helpful to give more information as to why you say "This doesn't work"?
But, I'm guessing that you are not seeing a decline in times when using StartTimer/StopTimer.
It is likely that is because the copy function is fast, and
frequent calls to StartTimer/StopTimer involve some error/overhead. So they are counteracting each other. Note that the documentation says the StopTimer can be used for "complex initialization", and I am guessing that a 1k copy does not count as complex. If you change the buffer size from 1024 to 1024*16 then you will see that the
StartTimer/StopTimer version is in fact faster.
PeterGo's solution works fine, as does bench marking a larger buffer. But if your goal is to compare
BenchmarkFilter3 and
BenchmarkFilter4, then you could reasonably just ignore the the overhead, since it will be the same for both functions.
tapi...@gmail.com
Jul 7, 2021, 8:01:54 PM7/7/21
to golang-nuts
On Wednesday, July 7, 2021 at 12:32:38 PM UTC-4 jake...@gmail.com wrote:
It would be helpful to give more information as to why you say "This doesn't work"?But, I'm guessing that you are not seeing a decline in times when using StartTimer/StopTimer.It is likely that is because the copy function is fast, and frequent calls to StartTimer/StopTimer involve some error/overhead. So they are counteracting each other. Note that the documentation says the StopTimer can be used for "complex initialization", and I am guessing that a 1k copy does not count as complex. If you change the buffer size from 1024 to 1024*16 then you will see that the StartTimer/StopTimer version is in fact faster.PeterGo's solution works fine, as does bench marking a larger buffer. But if your goal is to compare BenchmarkFilter3 and BenchmarkFilter4, then you could reasonably just ignore the the overhead, since it will be the same for both functions.
The StartTimer/StopTimer in my example doesn't work for the "go test" command hang there for ever.
Peter's solution exhausted my memory and hang my whole computer, and I have to restart it.
tapi...@gmail.com
Jul 7, 2021, 8:14:19 PM7/7/21
to golang-nuts
On Wednesday, July 7, 2021 at 8:01:54 PM UTC-4 tapi...@gmail.com wrote:
On Wednesday, July 7, 2021 at 12:32:38 PM UTC-4 jake...@gmail.com wrote:It would be helpful to give more information as to why you say "This doesn't work"?But, I'm guessing that you are not seeing a decline in times when using StartTimer/StopTimer.It is likely that is because the copy function is fast, and frequent calls to StartTimer/StopTimer involve some error/overhead. So they are counteracting each other. Note that the documentation says the StopTimer can be used for "complex initialization", and I am guessing that a 1k copy does not count as complex. If you change the buffer size from 1024 to 1024*16 then you will see that the StartTimer/StopTimer version is in fact faster.PeterGo's solution works fine, as does bench marking a larger buffer. But if your goal is to compare BenchmarkFilter3 and BenchmarkFilter4, then you could reasonably just ignore the the overhead, since it will be the same for both functions.The StartTimer/StopTimer in my example doesn't work for the "go test" command hang there for ever.
Sorry, It doesn't really hang. It just spends much more time than I expected.
And it looks the result ns/op values become larger, which is also unexpected.
peterGo
Jul 7, 2021, 9:43:11 PM7/7/21
to golang-nuts
On Wednesday, July 7, 2021 at 8:01:54 PM UTC-4 tapi...@gmail.com wrote:
Peter's solution exhausted my memory and hang my whole computer, and I have to restart it.
Adjust the number of iterations (b.N), to accomodate your limited memory, For example, for 8GB use 100,000,
$ go test filter_test.go -bench=Filter -benchmem -benchtime=100000x
Peter
Reply all
Reply to author
Forward
0 new messages