[go] sync: add an example for Map
Changkun Ou (Gerrit)
Changkun Ou has uploaded this change for review.
sync: add an example for Map
This change adds an example for sync.Map that demonstrates a potential
use case of the structure.
Fixes #20973
Change-Id: I317629d118fc4f2534e8298e988ec087af271168
---
M src/sync/example_test.go
1 file changed, 100 insertions(+), 0 deletions(-)
diff --git a/src/sync/example_test.go b/src/sync/example_test.go
index bdd3af6..e24b552 100644
--- a/src/sync/example_test.go
+++ b/src/sync/example_test.go
@@ -6,7 +6,9 @@
import (
"fmt"
+ "reflect"
"sync"
+ "time"
)
type httpPkg struct{}
@@ -57,3 +59,101 @@
// Output:
// Only once
}
+
+// This example broadcasts a message to all listeners periodically.
+// All listeners are managed in sync.Map as the listeners may join and
+// leave concurrently.
+func ExampleMap() {
+ var listeners sync.Map // map[string]chan interface{}
+
+ // The following goroutine ranges and broadcasts a message to all
+ // listeners every millisecond.
+ ready, done := make(chan bool), make(chan bool)
+ outs := make(chan interface{}, 42)
+ go func() {
+ ti := time.NewTicker(time.Millisecond)
+ ready <- true
+ for {
+ select {
+ case <-ti.C:
+ listeners.Range(func(id, v interface{}) bool {
+ ch, ok := v.(chan interface{})
+ if !ok { // Remove unexpected values.
+ fmt.Println("unexpected listeners.")
+ listeners.Delete(v)
+ return true
+ }
+
+ // Send the message in a separate goroutine, and prevent
+ // block on boardcasting process if the receiver is too busy.
+ go func() {
+ ch <- fmt.Sprintf("boardcast to %v", id)
+ }()
+ return true
+ })
+ case <-done:
+ close(outs)
+ return
+ }
+ }
+ }()
+
+ users := []struct {
+ id string
+ ch chan interface{}
+ }{
+ {"user-0", make(chan interface{}, 1)},
+ {"user-1", make(chan interface{}, 1)},
+ {"user-2", make(chan interface{}, 1)},
+ }
+
+ // Simulate users join concurrently. Each user will listen to the
+ // broadcast message exactly once. In practice, each user can
+
+ wg := sync.WaitGroup{}
+ wg.Add(len(users))
+ <-ready
+ for i := 0; i < len(users); i++ {
+ go func(i int) {
+ defer wg.Done()
+ u := users[i]
+ listeners.Store(u.id, u.ch)
+ outs <- <-u.ch
+ listeners.Delete(u.id)
+ }(i)
+ }
+ wg.Wait()
+ done <- true
+
+ // Checking results:
+ //
+ // Since each user receives the message exactly once. Hence the
+ // total number of broadcasted messages is 3.
+ //
+ // Because each user joins as a listener concurrently, all messages
+ // could be received in any sequential order.
+ msgs := []interface{}{}
+ for msg := range outs {
+ msgs = append(msgs, msg)
+ }
+ msgsOrders := [][]interface{}{
+ {"boardcast to user-0", "boardcast to user-1", "boardcast to user-2"},
+ {"boardcast to user-0", "boardcast to user-2", "boardcast to user-1"},
+ {"boardcast to user-1", "boardcast to user-2", "boardcast to user-0"},
+ {"boardcast to user-1", "boardcast to user-0", "boardcast to user-2"},
+ {"boardcast to user-2", "boardcast to user-1", "boardcast to user-0"},
+ {"boardcast to user-2", "boardcast to user-0", "boardcast to user-1"},
+ }
+
+ found := false
+ for _, o := range msgsOrders {
+ if reflect.DeepEqual(o, msgs) {
+ found = true
+ break
+ }
+ }
+ fmt.Println(len(msgs), found)
+
+ // Output:
+ // 3 true
+}
To view, visit change 337390. To unsubscribe, or for help writing mail filters, visit settings.
Joe Tsai (Gerrit)
Attention is currently required from: Bryan C. Mills, Ian Lance Taylor, Changkun Ou, Rob Pike.
1 comment:
Patchset:
I'm not sure if this is a good example for sync.Map.
sync.Map is optimized for concurrently accessed maps where: (1) when the entry for a given key is only ever written once but read many times, as in caches that only grow, or (2) when multiple goroutines read, write, and overwrite entries for disjoint sets of keys.
An example would be like a global configuration KV store where the settings rarely change.
The example here seems more suitable for a map+sync.RWMutex.
To view, visit change 337390. To unsubscribe, or for help writing mail filters, visit settings.
Changkun Ou (Gerrit)
Attention is currently required from: Bryan C. Mills, Ian Lance Taylor, Joe Tsai, Rob Pike.
1 comment:
Patchset:
I'm not sure if this is a good example for sync.Map. […]
Yes, I agree that the current sync.Map is optimized for the read-more-write-less scenarios. But it is arguable that sync.Map could be optimized even better, e.g. #21032, #21032, #21035.
The example may let the user stay longer such that the broadcast reads more often for reading the stored listeners than listeners join more frequently (write). Nevertheless, I think it remains a good example of demonstrating an appropriate example.
To view, visit change 337390. To unsubscribe, or for help writing mail filters, visit settings.
Changkun Ou (Gerrit)
Attention is currently required from: Bryan C. Mills, Ian Lance Taylor, Joe Tsai, Rob Pike.
Changkun Ou uploaded patch set #2 to this change.
sync: add an example for Map
This change adds an example for sync.Map that demonstrates a potential
use case of the structure.
Fixes #20973
Change-Id: I317629d118fc4f2534e8298e988ec087af271168
---
M src/sync/example_test.go
1 file changed, 100 insertions(+), 0 deletions(-)
To view, visit change 337390. To unsubscribe, or for help writing mail filters, visit settings.
Changkun Ou (Gerrit)
Attention is currently required from: Bryan C. Mills, Ian Lance Taylor, Joe Tsai, Rob Pike.
Changkun Ou uploaded patch set #3 to this change.
sync: add an example for Map
This change adds an example for sync.Map that demonstrates a potential
use case of the structure.
Fixes #20973
Change-Id: I317629d118fc4f2534e8298e988ec087af271168
---
M src/sync/example_test.go
1 file changed, 105 insertions(+), 0 deletions(-)
To view, visit change 337390. To unsubscribe, or for help writing mail filters, visit settings.
Bryan C. Mills (Gerrit)
Attention is currently required from: Ian Lance Taylor, Changkun Ou, Joe Tsai, Rob Pike.
Patch set 3:Code-Review -1
1 comment:
File src/sync/example_test.go:
// Send the message in a separate goroutine, and prevent
// block on boardcasting process if the receiver is too busy.
go func() {
ch <- fmt.Sprintf("boardcast to %v", id)
}()
This looks like a goroutine leak to me. What ensures that the listener is still receiving at this point? (`Range` does not block calls to `Delete`.)
To view, visit change 337390. To unsubscribe, or for help writing mail filters, visit settings.
Changkun Ou (Gerrit)
Attention is currently required from: Ian Lance Taylor, Changkun Ou, Joe Tsai, Rob Pike.
Changkun Ou uploaded patch set #4 to this change.
sync: add an example for Map
This change adds an example for sync.Map that demonstrates a potential
use case of the structure.
Fixes #20973
Change-Id: I317629d118fc4f2534e8298e988ec087af271168
---
M src/sync/example_test.go
1 file changed, 109 insertions(+), 0 deletions(-)
To view, visit change 337390. To unsubscribe, or for help writing mail filters, visit settings.
Changkun Ou (Gerrit)
Attention is currently required from: Bryan C. Mills, Ian Lance Taylor, Joe Tsai, Rob Pike.
1 comment:
File src/sync/example_test.go:
// Send the message in a separate goroutine, and prevent
// block on boardcasting process if the receiver is too busy.
go func() {
ch <- fmt.Sprintf("boardcast to %v", id)
}()
This looks like a goroutine leak to me. […]
That was great observation, thanks for spotting this. I just updated a new patch, and hope it solves this.
To view, visit change 337390. To unsubscribe, or for help writing mail filters, visit settings.
Rob Pike (Gerrit)
Attention is currently required from: Bryan C. Mills, Ian Lance Taylor, Changkun Ou, Joe Tsai.
1 comment:
File src/sync/example_test.go:
Patch Set #4, Line 93:
case ch <- fmt.Sprintf("boardcast to %v", id):broadcast
To view, visit change 337390. To unsubscribe, or for help writing mail filters, visit settings.
Changkun Ou (Gerrit)
Attention is currently required from: Bryan C. Mills, Ian Lance Taylor, Changkun Ou, Joe Tsai.
Changkun Ou uploaded patch set #5 to this change.
sync: add an example for Map
This change adds an example for sync.Map that demonstrates a potential
use case of the structure.
Fixes #20973
Change-Id: I317629d118fc4f2534e8298e988ec087af271168
---
M src/sync/example_test.go
1 file changed, 109 insertions(+), 0 deletions(-)
To view, visit change 337390. To unsubscribe, or for help writing mail filters, visit settings.
Changkun Ou (Gerrit)
Attention is currently required from: Bryan C. Mills, Ian Lance Taylor, Rob Pike, Joe Tsai.
1 comment:
File src/sync/example_test.go:
Patch Set #4, Line 93:
case ch <- fmt.Sprintf("boardcast to %v", id):broadcast
Done
To view, visit change 337390. To unsubscribe, or for help writing mail filters, visit settings.
Ian Lance Taylor (Gerrit)
Attention is currently required from: Bryan C. Mills, Changkun Ou, Joe Tsai.
9 comments:
File src/sync/example_test.go:
// Send the message in a separate goroutine, and prevent
// block on boardcasting process if the receiver is too busy.
go func() {
ch <- fmt.Sprintf("boardcast to %v", id)
}()
That was great observation, thanks for spotting this. […]
Done
File src/sync/example_test.go:
Patch Set #5, Line 63:
// This example demonstrates a goroutine broadcasts a message to allWill this comment appears with the example when it is displayed on pkg.go.dev?
Patch Set #5, Line 74:
outs := make(chan interface{}, 42)Why 42? In an ideal example every step should have a clear meaning. At first glance I don't think this channel needs to be buffered at all.
Patch Set #5, Line 89:
// Send the message in a separate goroutine, and prevent// Send the message in a separate goroutine so that we don't block the broadcasting goroutine if the receiver is busy.
Patch Set #5, Line 110:
{"user-0", make(chan interface{}, 1)},Is there a reason for these channels to be buffered?
Patch Set #5, Line 121:
wg := sync.WaitGroup{}A WaitGroup can't be copied, so although this usage is OK just write
var wg sync.WaitGroup
Patch Set #5, Line 123:
<-readyWhy bother with the ready channel? It doesn't seem necessary.
Patch Set #5, Line 128:
listeners.Store(u.id, u.ch)I think that you don't need u.ch, but can instead make the channel here in this goroutine. I think that would be better, as otherwise we might as well store the channels in the listeners map when we create them above.
Patch Set #5, Line 148:
msgsOrders := [][]interface{}{Rather than listing all the orders, just sort them.
To view, visit change 337390. To unsubscribe, or for help writing mail filters, visit settings.
Bryan C. Mills (Gerrit)
Attention is currently required from: Changkun Ou, Joe Tsai.
1 comment:
File src/sync/example_test.go:
Patch Set #5, Line 68:
func ExampleMap() {This example is still very complex — most of the real use-cases I've seen for sync.Map are for deduplication rather than concurrent leaving, joining, and broadcasting.
For those uses, we tend to see two patterns:
1. A Load (to eliminate allocations in the hot path), followed by some inexpensive computation or allocation, followed by a LoadOrStore that either saves the result or discards it.
2. A Load (as above), followed by some mechanism in the slow path to avoid duplicate work (perhaps either a LoadOrStore, as in encoding/json.typeEncoder, or a separate mutex as in reflect.FuncOf), followed by a Store to complete that work.
To me, those are the cases where a sync.Map is really clearly the right tool. This listener pattern is interesting, but (especially because Range immediately promotes the dirty-value map) it isn't obviously clearer to me than, say, an ordinary map stored in a 1-buffered channel or guarded by a sync.Mutex.
To view, visit change 337390. To unsubscribe, or for help writing mail filters, visit settings.
Changkun Ou (Gerrit)
Attention is currently required from: Changkun Ou, Joe Tsai.
Changkun Ou uploaded patch set #6 to this change.
sync: add an example for Map
This change adds an example for sync.Map that demonstrates a potential
use case of the structure.
Fixes #20973
Change-Id: I317629d118fc4f2534e8298e988ec087af271168
---
M src/sync/example_test.go
1 file changed, 92 insertions(+), 0 deletions(-)
To view, visit change 337390. To unsubscribe, or for help writing mail filters, visit settings.
Changkun Ou (Gerrit)
Attention is currently required from: Ian Lance Taylor, Joe Tsai.
8 comments:
File src/sync/example_test.go:
Patch Set #5, Line 63:
// This example demonstrates a goroutine broadcasts a message to allWill this comment appears with the example when it is displayed on pkg.go. […]
Yes, for example:
Patch Set #5, Line 74:
outs := make(chan interface{}, 42)
Why 42? In an ideal example every step should have a clear meaning. […]
You are right. It can be just unbuffered. The new patch removes the 42.
Patch Set #5, Line 89:
// Send the message in a separate goroutine, and prevent
// Send the message in a separate goroutine so that we don't block the broadcasting goroutine if the […]
Done
Patch Set #5, Line 110:
{"user-0", make(chan interface{}, 1)},Is there a reason for these channels to be buffered?
They don't have to be buffered. Removed in the new patch.
Patch Set #5, Line 121:
wg := sync.WaitGroup{}
A WaitGroup can't be copied, so although this usage is OK just write […]
Done
Patch Set #5, Line 123:
<-readyWhy bother with the ready channel? It doesn't seem necessary.
Removed in the new patch.
Patch Set #5, Line 128:
listeners.Store(u.id, u.ch)
I think that you don't need u.ch, but can instead make the channel here in this goroutine. […]
Done
Patch Set #5, Line 148:
msgsOrders := [][]interface{}{Rather than listing all the orders, just sort them.
Done
To view, visit change 337390. To unsubscribe, or for help writing mail filters, visit settings.
Changkun Ou (Gerrit)
Attention is currently required from: Bryan C. Mills, Ian Lance Taylor, Joe Tsai.
1 comment:
File src/sync/example_test.go:
Patch Set #5, Line 68:
func ExampleMap() {This example is still very complex — most of the real use-cases I've seen for sync. […]
I don't know if the mentioned two patterns can be written more straightforwardly, and the described two patterns seem to suggest there is no use of Delete. In the provided example (only in less than 50 lines), it is intended to show the two properties of sync.Map:
a. To demonstrate the concurrency-safe, a Store must happen concurrently with other Load. This means we must establish readers before the Store, and actively do Read from it.
b. To demonstrate that Read is more frequent than Write, we also have to call Load more frequent than Delete.
This example perfectly matches the above two intentions because:
a. A listener joins concurrently
b. Range does the Load more frequently than Delete.
Furthermore, in fact, the cgo.Handle is actual somewhat fundamentally similar to the current provided example.
To view, visit change 337390. To unsubscribe, or for help writing mail filters, visit settings.