prevent unnecessary escape to heap

264 views
Skip to first unread message

Steve Roth

unread,
Feb 4, 2021, 8:32:43 PM2/4/21
to golang-nuts
Hi, folks,

I could use some assistance getting rid of some unnecessary heap allocations.  I have code that needs to write individual bytes to an io.Writer.  (The Writer implementation given to my code is probably buffered, but my code shouldn't rely on a particular concrete type.) The relevant part of code looks like this:

func writeByte(b byte) {
var buf [1]byte
buf[0] = b
_, err = w.Write(buf[:])
}

Unfortunately, I find that every call of this function allocates a byte on the heap.  The compiler's escape analysis can't be sure that the Write method doesn't keep a copy of the pointer it's given, so "buf" escapes to the heap.

How can I implement a writeByte function, against an unknown io.Writer implementation, that doesn't allocate heap memory?

Thanks,
Steve

Jesse McNelis

unread,
Feb 4, 2021, 8:59:15 PM2/4/21
to Steve Roth, golang-nuts
On Fri, Feb 5, 2021 at 12:32 PM Steve Roth <st...@rothskeller.net> wrote:

How can I implement a writeByte function, against an unknown io.Writer implementation, that doesn't allocate heap memory?


As you've correctly stated, because the call to .Write() is via an interface the compiler can't tell whether any particular io.Writer will hold on to that slice.
So there isn't a way to do what you want.

But also, writing individual bytes to an io.Writer is likely to be a very slow thing to do so avoiding the allocation is the least of your worries.

bufio.Writer has a WriteByte() method precisely to avoid the allocation problem you're encountering and it also avoids the slowness of writing individual bytes to an io.Writer
https://golang.org/pkg/bufio/#Writer.WriteByte

- Jesse

Wagner Riffel

unread,
Feb 5, 2021, 9:38:27 AM2/5/21
to Steve Roth, golang-nuts
On Thu, Feb 4, 2021 at 10:32 PM Steve Roth <st...@rothskeller.net> wrote:
>
> How can I implement a writeByte function, against an unknown io.Writer implementation, that doesn't allocate heap memory?
>

The only way I'm aware of achieving this is proving to the compiler that
buf is safe to be kept by w by moving buf to the package block.

var buf [1]byte
func writeByte(b byte) {
Reply all
Reply to author
Forward
0 new messages