go concurrency performance

[Robert Engels]

Hello,

I recently posted a project https://github.com/robaho/go-concurrency-test that tests the Go concurrency structures.

In general, they perform far slower when compared to equivalent Java structures.

I am bringing to your attention for 2 reasons, 1) I'd love to see them improve, 2) maybe I made some errors in my testing.

I realize this may not be a easily received topic, but I think it is helpful to advance the Go platform forward.

Here is the summary:

The Go language has significant room for improvement in terms of concurrent data structure performance, with the current implementations being far slower than the equivalent Java ones. This is no way it to claim that Java is superior to Go, as there are other features of Go (low GC pauses, lower memory footprint, no warm-up time, etc.) that make it a superior choice for many applications.

I welcome your feedback.

[Robert Engels]

The summary should read:

[Bryan C. Mills]

Several areas for improvement for sync.Map are known: see this search.

I notice that your LockCache implementation uses sync.RWMutex, for which the current implementation is also known not to scale well (#17973).

Finally, the ChannelCache implementation does twice as many goroutine switches as necessary: you're using a goroutine as a mutex, when it would be more efficient (and arguably clearer) to use the channel itself. (For detail, see the slides from my talk on Rethinking Classical Concurrency Patterns from GopherCon last week. The relevant pattern appears in slides 60-68.)

That said, I don't think your benchmarking methodology is measuring the properties you actually care about. You say that you're implementing a “shared cache”, but the benchmarks only access each key once per goroutine or thread, and access those keys in order (so that cache misses for each key will tend to correlate in time). That access pattern seems extremely atypical for all of the caches I've worked with. I would recommend benchmarking a real program against a real set of requests before you jump to any firm conclusions.

Moreover, you seem to be only benchmarking two threads / goroutines despite apparently having 8 logical cores on the test machine. If you're in a situation where performance matters, you should test with realistic resources too: is this really only a two-core problem in a production setting?

On Wed, Sep 5, 2018 at 11:18 AM Robert Engels <ren...@ix.netcom.com> wrote:

The summary should read:

The Go language has significant room for improvement in terms of concurrent data structure performance, with the current implementations being far slower than the equivalent Java ones. This is no way it to claim that Java is superior to Go, as there are other features of Go (low GC pauses, lower memory footprint, no warm-up time, etc.) that make it a superior choice for many applications.

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[robert engels]

I’ve updated the channel tests… I’ve read your slides, and reduced the number of channels to 2, but I don’t think your design applies (in the slides) - there still needs to be a Go routine processor. The cache is not a queue, so the Get cannot block waiting for another routine to Put - it is a cache. I think the request processor Go routine also more intuitively maps to the “share memory by communicating” design for more complex applications.

It did improve things somewhat but not significantly - if the design can be improved, maybe some more details would help.

[Jon Conradt]

This may be a better topic for golang-nuts.  It is a larger community and more focused on using (and tuning) real applications. This forum is more about the mechanics of the language and has a smaller audience.

Jon