Compiler optimizations when returning a string

[Alexandre Cesaro]

I am quite unfamiliar with how compilers work but I think there might be some inconsistencies that could be fixed about functions that create and return a string.

Here is a memory benchmark: http://play.golang.org/p/Ba4fekX7Dt

BenchmarkBuildBytes creates and returns a byte slice.
BenchmarkBuildString creates a byte slice and then converts it to a string.
BenchmarkConvertString calls BenchmarkBuildBytes and converts it to a string.

Here are the results :

BenchmarkBuildBytes	    1024 B/op	       1 allocs/op
BenchmarkBuildString	    1024 B/op	       1 allocs/op
BenchmarkConvertString	    2048 B/op	       2 allocs/op

The results show that BenchmarkBuildString allocates only once like BenchmarkBuildBytes which is nice. However BenchmarkConvertString allocates twice.

First question: Could the compiler optimize BenchmarkConvertString so that it allocates only once?

With this optimization, packages could only provides a function returning a slice of bytes and the user could convert it to string without any additional allocation. Without it, efficient API needs to provide two versions of the functions, one that returns a slice of bytes and one that returns a string.
I feel that it could be possible for the compiler to optimize this since the byte slice is converted as soon as it is returned by the function. But again I am not very familiar with how compilers work.

Now here are the results when the length is set to 10,000 instead of 1,000:

BenchmarkBuildBytes	   10496 B/op	       1 allocs/op
BenchmarkBuildString	   26880 B/op	       2 allocs/op
BenchmarkConvertString	   20992 B/op	       2 allocs/op

Not only the optimization in BenchmarkBuildString no longer works but also even more memory is used than in BenchmarkConvertString.

Finally, here are the results when the length is set to 100,000:

BenchmarkBuildBytes	  106496 B/op	       1 allocs/op
BenchmarkBuildString	  212992 B/op	       2 allocs/op
BenchmarkConvertString	  212992 B/op	       2 allocs/op

Now the optimization in BenchmarkBuildString still does not work but at least it does not use more memory than BenchmarkConvertString.

Second question: Are these results normal or is there a bug? Could the optimization in BenchmarkBuildString stays whatever the slice length? If not, could it use at most the same amount of memory than BenchmarkConvertString?

[Josh Bleecher Snyder]

I believe that this is issue 6714.

Note that the second alloc in BenchmarkBuildString shows up when b is placed by the compiler on the heap instead of the stack. Making b large forces it onto the heap. So does changing 'const length = 1000' to 'var length 1000'.

-josh

[Russ Cox]

Yes, the compiler could optimize BenchmarkConvertString so that it only allocates once, but it would need to analyze buildBytes to be sure that the []byte being returned was not referred to by anything else in the program and would never be referred to again after the conversion. Given that assurance, the string(buildBytes()) could reuse the []byte. The compiler today does not do enough analysis to make that guarantee.

Russ

[Alexandre Cesaro]

Ok, thanks for your answer.

So the only weird thing that remains is when length is const 10,000. Why does BenchmarkBuildString uses more memory than BenchmarkConvertString? Is it a bug?

[Keith Randall]

In BenchmarkBuildString with len=10000, the byte array is still allocated on the stack.  The stack frame of buildString is too big so some additional stack must be allocated.  You're seeing the size of this stack, which is 16384 bytes.  (26880 = 10496 (10000 rounded up to size class) + 16384)

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.