confused about sync.Pool

[Hoping White]

Hi, all

I have tested sync.Pool with the following code

package main

import (

"runtime"

"sync"

)

var count = 0

var pool = sync.Pool{

New: func() interface{} {

count++

buffer := make([]byte, 1024)

return buffer

},

}

func onFinalizer(data interface{}) {

buffer := data.(*interface{})

pool.Put(*buffer)

}

func test() {

data := pool.Get()

runtime.SetFinalizer(&data, onFinalizer)

}

func main() {

for {

for i := 0; i < 10000; i++ {

test()

runtime.Gosched()

}

println(count)

}

}

I expect that New function will be called limited times, but the result is

that the program will run until out-of-memory.

Did I use it improperly or something else happened?

[Ian Lance Taylor]

The Go garbage collector is more or less designed to let your program
get up to its working set. When the GC runs when you have N bytes
allocated, it sets its next run to occur when you have N*2 bytes
allocated. In your case, the GC can essentially never free anything.
Each time it tries, the finalizer brings the value back to life by
storing it in the pool. So the GC keeps increasing the points at
which it will run, until your program runs out of memory.

This is not a good way to use sync.Pool, not only because it doesn't
work, but because the whole point of sync.Pool is to take work away
from the GC. When you use a finalizer to add a value back to the
pool, you are missing the point of the pool. The GC is doing the work
it requires to find that the value can be freed and then run the
finalizer. The expensive part of GC is not the allocation; it's
finding the free memory. Using a finalizer to add a value back to
sync.Pool forces the GC to do the expensive work in order to save on
the cheap work. It doesn't make sense.

Ian

[Pablo Rozas-Larraondo]

I understand that the GC can never frees any byte slice, as they remain reachable inside the pool. But what's still not clear to me is why the Pool doesn't return these byte slices when it receives a Get() call.

As the memory grows the GC is called placing these objects in the Pool, why they are never returned and instead the Pool creates new objects?

Pablo

[Jesse McNelis]

On Wed, Dec 16, 2015 at 11:51 AM, Pablo Rozas-Larraondo
<p.rozas....@gmail.com> wrote:
> I understand that the GC can never frees any byte slice, as they remain
> reachable inside the pool. But what's still not clear to me is why the Pool
> doesn't return these byte slices when it receives a Get() call.
>
> As the memory grows the GC is called placing these objects in the Pool, why
> they are never returned and instead the Pool creates new objects?

New objects are created because the pool is empty when the Get() call is made.
The pool is empty because the finalizers haven't run yet because the
GC hasn't run yet.

At some point the GC is run, the objects get put back in the the pool
and are available for Get() again,
but eventually that pool runs out again and allocations start again
until the heap reaches the size of the next GC trigger.

Each loop through the heap gets bigger and the next GC trigger size
gets larger until you eventually run out of memory.

[Pablo Rozas Larraondo]

Very well explained, I understand it now. Thank you!