Suggestions for dynamic per object lock

[Groups Discussion]

Hi all,

while refactoring an old application I found this code:

type concurrentChecks struct {

mutex        sync.Mutex

activeChecks map[string]int

}

// addCheck waits until the processing for an item with the same token is finished

func (c *concurrentChecks) addCheck(token string) {

for {

c.mutex.Lock()

if _, ok := c.activeChecks[token]; !ok {

c.activeChecks[token] = 1

c.mutex.Unlock()

return

}

c.mutex.Unlock()

time.Sleep(20 * time.Millisecond)

}

}

func (c *concurrentChecks) removeCheck(token string) {

c.mutex.Lock()

defer c.mutex.Unlock()

delete(c.activeChecks, token)

}

In this application there are different goroutines, each goroutine process some input data and extract a string token.  Different tokens must be processed in parallel, but if 2 or more goroutines get the same token they must be serialized. The above code is ugly but it works, replacing it using channels or a sync.Map seems not so easy. I would like to avoid something like this https://github.com/atedja/go-multilock/blob/master/multilock.go

Do you have suggestions to achieve the same result in a more idiomatic way?

thanks 

[Tamás Gulácsi]

Check out https://pkg.go.dev/golang.org/x/sync/singleflight !

[Robert Engels]

The code (and even similar using channels) is highly inefficient. You need to understand the expected collision rate because if it’s low (expected) paying the lock cost on every operation is wasteful. Better to partition the work based on the token (hash N) and avoid the lock operations.

You can also make it far more efficient by an atomic bitset with page locking.

> On Apr 4, 2020, at 9:26 AM, Tamás Gulácsi <tgula...@gmail.com> wrote:
>
> Check out https://pkg.go.dev/golang.org/x/sync/singleflight !
>
> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/aab893a3-7370-4682-8ba8-27efff919791%40googlegroups.com.

[Slawomir Pryczek]

I think what would work is declaring some number of mutexes, in array like 48. Then you can compute CRC32 of the token, and lock based on modulo. But yes, probably best would be to change underlying code and just partitioning the work by hash.

const concurrency = 48

type concurrentChecks struct {

	mutex [concurrency]sync.Mutex

}

func (c *concurrentChecks) addCheck(token string) {

	mutex_num := crc32.ChecksumIEEE([]byte(token)) % concurrency

	c.mutex[mutex_num].Lock()

}

func (c *concurrentChecks) removeCheck(token string) {

	mutex_num := crc32.ChecksumIEEE([]byte(token)) % concurrency

	c.mutex[mutex_num].Unlock()

}