[go] sync: replace the incorrect Pool usage example
209 views
Skip to first unread message
Changkun Ou (Gerrit)
Jul 26, 2021, 12:25:08 PM7/26/21
to goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com
Changkun Ou has uploaded this change for review.
sync: replace the incorrect Pool usage example
This change suggests a more appropriate example of the usage of Pool.
The new example creates a request to a request handler with data in the
request body. As on the server-side, the handler reads the data, which
may be dynamically sized, then writes it back as response contents.
However, the use of Pool guarantees that the obtained buffer from the
pool is bounded, which suggests the more appropriate usage in contrast
to the previous example.
To emphasize the potential misuse of Pool, the change also removes the
documented but less suitable fmt usage, and states the potential issue
that may be caused by the misuse.
Fixes #23199
Change-Id: Id7fff8d728a0400b4984243c3d38ec19e7ed2eec
---
M src/sync/example_pool_test.go
M src/sync/pool.go
2 files changed, 64 insertions(+), 28 deletions(-)
diff --git a/src/sync/example_pool_test.go b/src/sync/example_pool_test.go
index 8288d41..f8dc3d5 100644
--- a/src/sync/example_pool_test.go
+++ b/src/sync/example_pool_test.go
@@ -6,40 +6,77 @@
import (
"bytes"
- "io"
- "os"
+ "fmt"
+ nethttp "net/http"
+ "net/http/httptest"
"sync"
- "time"
)
+const bufLen = 1 << 2
+
var bufPool = sync.Pool{
New: func() interface{} {
// The Pool's New function should generally only return pointer
// types, since a pointer can be put into the return interface
// value without an allocation:
- return new(bytes.Buffer)
+ return make([]byte, bufLen)
},
}
-// timeNow is a fake version of time.Now for tests.
-func timeNow() time.Time {
- return time.Unix(1136214245, 0)
-}
-
-func Log(w io.Writer, key, val string) {
- b := bufPool.Get().(*bytes.Buffer)
- b.Reset()
- // Replace this with time.Now() in a real logger.
- b.WriteString(timeNow().UTC().Format(time.RFC3339))
- b.WriteByte(' ')
- b.WriteString(key)
- b.WriteByte('=')
- b.WriteString(val)
- w.Write(b.Bytes())
- bufPool.Put(b)
-}
-
+// This example shows a usage of sync.Pool. The example creates
+// a request to a request handler with data in the request body.
+// As on the server-side, the handler reads the data, which may
+// have a dynamical size, then writes it back as response contents.
func ExamplePool() {
- Log(os.Stdout, "path", "/search?q=flowers")
- // Output: 2006-01-02T15:04:05Z path=/search?q=flowers
+ h := nethttp.HandlerFunc(func(w nethttp.ResponseWriter, r *nethttp.Request) {
+ for {
+ b := bufPool.Get().([]byte)
+
+ // Since sync.Pool reuses the items, thus the obtained memory
+ // may contain dirty data from previous use. Clear the
+ // buffer before using it.
+ for i := range b {
+ b[i] = 0
+ }
+
+ // According to the document of Pool, we must bound the size
+ // of the returned buffer to prevent unexpected large memory
+ // consumption. The r.Body.Read method does not grow the
+ // buffer, hence it is safe to put the buffer back and keep
+ // using the pool between requests. See the document of Pool
+ // for the potential issue if the buffer grows dynamically.
+ n, err := r.Body.Read(b)
+ if err != nil || n == 0 {
+ bufPool.Put(b)
+ return
+ }
+
+ w.Write(b[:n])
+ bufPool.Put(b)
+ }
+ })
+
+ // Simulate three concurrent requests.
+ done := make(chan string)
+ for i := 0; i < 3; i++ {
+ go func() {
+ r := httptest.NewRecorder()
+ data := []byte("A sync.Pool Example")
+ req, err := nethttp.NewRequest(nethttp.MethodPost, "/", bytes.NewReader(data))
+ if err != nil {
+ panic("failed to make a request")
+ }
+ h.ServeHTTP(r, req)
+ done <- r.Body.String()
+ }()
+ }
+
+ fmt.Println(<-done)
+ fmt.Println(<-done)
+ fmt.Println(<-done)
+
+ // Output:
+ // A sync.Pool Example
+ // A sync.Pool Example
+ // A sync.Pool Example
}
diff --git a/src/sync/pool.go b/src/sync/pool.go
index 1ae7012..eb6a66c 100644
--- a/src/sync/pool.go
+++ b/src/sync/pool.go
@@ -30,10 +30,9 @@
// clients of a package. Pool provides a way to amortize allocation overhead
// across many clients.
//
-// An example of good use of a Pool is in the fmt package, which maintains a
-// dynamically-sized store of temporary output buffers. The store scales under
-// load (when many goroutines are actively printing) and shrinks when
-// quiescent.
+// One the one hand, each item of the pool should consume the same memory cost
+// approximately. Specifically, items with dynamically sized buffers must
+// be bounded to prevent unexpected large memory consumption spikes.
//
// On the other hand, a free list maintained as part of a short-lived object is
// not a suitable use for a Pool, since the overhead does not amortize well in
To view, visit change 337393. To unsubscribe, or for help writing mail filters, visit settings.
Changkun Ou (Gerrit)
Jul 26, 2021, 12:30:17 PM7/26/21
to goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com
Attention is currently required from: Bryan C. Mills.
Changkun Ou uploaded patch set #2 to this change.
sync: replace the incorrect Pool usage example
This change suggests a more appropriate example of the usage of Pool.
The new example creates a request to a request handler with data in the
request body. As on the server-side, the handler reads the data, which
may be dynamically sized, then writes it back as response contents.
However, the use of Pool guarantees that the obtained buffer from the
pool is bounded, which suggests the more appropriate usage in contrast
to the previous example.
To emphasize the potential misuse of Pool, the change also removes the
documented but less suitable fmt usage, and states the potential issue
that may be caused by the misuse.
Fixes #23199
Change-Id: Id7fff8d728a0400b4984243c3d38ec19e7ed2eec
---
M src/sync/example_pool_test.go
M src/sync/pool.go
2 files changed, 64 insertions(+), 28 deletions(-)
To view, visit change 337393. To unsubscribe, or for help writing mail filters, visit settings.
Changkun Ou (Gerrit)
Aug 12, 2021, 8:01:53 AM8/12/21
to goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com
Attention is currently required from: Bryan C. Mills.
Changkun Ou uploaded patch set #3 to this change.
sync: replace the incorrect Pool usage example
This change suggests a more appropriate example of the usage of Pool.
The new example creates a request to a request handler with data in the
request body. As on the server-side, the handler reads the data, which
may be dynamically sized, then writes it back as response contents.
However, the use of Pool guarantees that the obtained buffer from the
pool is bounded, which suggests the more appropriate usage in contrast
to the previous example.
To emphasize the potential misuse of Pool, the change also removes the
documented but less suitable fmt usage, and states the potential issue
that may be caused by the misuse.
Fixes #23199
Change-Id: Id7fff8d728a0400b4984243c3d38ec19e7ed2eec
---
M src/sync/example_pool_test.go
M src/sync/pool.go
2 files changed, 64 insertions(+), 28 deletions(-)
To view, visit change 337393. To unsubscribe, or for help writing mail filters, visit settings.
Changkun Ou (Gerrit)
Aug 12, 2021, 8:29:45 AM8/12/21
to goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com
Attention is currently required from: Bryan C. Mills.
Changkun Ou uploaded patch set #4 to this change.
sync: replace the incorrect Pool usage example
This change suggests a more appropriate example of the usage of Pool.
The new example creates a request to a request handler with data in the
request body. As on the server-side, the handler reads the data, which
may be dynamically sized, then writes it back as response contents.
However, the use of Pool guarantees that the obtained buffer from the
pool is bounded, which suggests the more appropriate usage in contrast
to the previous example.
To emphasize the potential misuse of Pool, the change also removes the
documented but less suitable fmt usage, and states the potential issue
that may be caused by the misuse.
Fixes #23199
Change-Id: Id7fff8d728a0400b4984243c3d38ec19e7ed2eec
---
M src/sync/example_pool_test.go
M src/sync/pool.go
2 files changed, 66 insertions(+), 28 deletions(-)
To view, visit change 337393. To unsubscribe, or for help writing mail filters, visit settings.
Reply all
Reply to author
Forward
0 new messages