Concurrent data structures in golang.

[Vlad Gruchik]

Go have useful concurrent model with goroutines. It is easy to create concurrent algorithm, but there are no cocurrent lock-free data structures. Maybe it is a good idea to create standart libraries with this data structures.

P.S. Are there any libraries with concurrent lock-free data structures?

[Shawn Milochik]

On Mon, Apr 18, 2016 at 4:15 AM, Vlad Gruchik <gruch...@gmail.com> wrote:

Go have useful concurrent model with goroutines. It is easy to create concurrent algorithm, but there are no cocurrent lock-free data structures. Maybe it is a good idea to create standart libraries with this data structures.

P.S. Are there any libraries with concurrent lock-free data structures?

This is not Go's philosophy. The Go team has repeatedly said things like:

"Less is exponentially more": http://commandcenter.blogspot.com/2012/06/less-is-exponentially-more.html

"Do less. Enable more." https://www.youtube.com/watch?v=XvZOdpd_9tc 

It is invariably people new to Go who experience the initial "Wow, this is awesome!" followed immediately by "Wait, why isn't that done for me in Go?" Please allow yourself the time to experience the language and its subtle beauty before you go trying to change it or resort to third-party libraries created by misguided people who were too impatient to understand Go.

There is no instant gratification. But there is more gratification lying under the surface than you can imagine right now.

[Ian Davis]

On Mon, Apr 18, 2016, at 12:15 PM, Vlad Gruchik wrote:

Go have useful concurrent model with goroutines. It is easy to create concurrent algorithm, but there are no cocurrent lock-free data structures. Maybe it is a good idea to create standart libraries with this data structures.

 

P.S. Are there any libraries with concurrent lock-free data structures?

 

Lock-free data structures are good for shared state that must be accessed by multiple concurrent processes. In many languages this is the only way to coordinate multi-threaded applications.

 

Go lets you do it that way that but it also subtly encourages you to rethink your design to avoid sharing. For many problems you'll find that your goroutines can maintain their own state and communicate changes via channels.

 

Ian

[Patrick Logan]

There are a variety of these kinds of data structures. I appreciate the core team does not spend time and energy determining which should be implemented in the standard library.

A selection of these would be nice as 3rd party packages. But they have their own set of trade offs and do not eliminate the need for coordination across goroutines. So far I've been successful using core data structures with channels in my public API, and the sync and sort packages for efficiency and implementation coordination.

[John Souvestre]

I agree.  Many of the common examples of Go’s concurrency (using channels and/or mutexes) don’t scale well enough for use in a 64+ core environment.  As such machines become common I’m hoping that there will be some standard library offerings.

 

Meanwhile, yes – there are some third-party libraries.

 

John

    John Souvestre - New Orleans LA

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[John Souvestre]

Ø  This is not Go's philosophy. The Go team has repeatedly said things like:

 

I would disagree.  I believe that they have said they didn’t see the need for lock-free data structures quite yet.  Indeed, if you wanted to keep things simple and have only one type then having lock-free would be better than having locking.  J

 

John

    John Souvestre - New Orleans LA

 

From: golan...@googlegroups.com [mailto:golan...@googlegroups.com] On Behalf Of Shawn Milochik
Sent: 2016 April 18, Mon 10:30
To: golang-nuts
Subject: Re: [go-nuts] Concurrent data structures in golang.

 

On Mon, Apr 18, 2016 at 4:15 AM, Vlad Gruchik <gruch...@gmail.com> wrote:

Go have useful concurrent model with goroutines. It is easy to create concurrent algorithm, but there are no cocurrent lock-free data structures. Maybe it is a good idea to create standart libraries with this data structures.

P.S. Are there any libraries with concurrent lock-free data structures?

 

 

This is not Go's philosophy. The Go team has repeatedly said things like:

 

"Less is exponentially more": http://commandcenter.blogspot.com/2012/06/less-is-exponentially-more.html

 

"Do less. Enable more." https://www.youtube.com/watch?v=XvZOdpd_9tc 

 

It is invariably people new to Go who experience the initial "Wow, this is awesome!" followed immediately by "Wait, why isn't that done for me in Go?" Please allow yourself the time to experience the language and its subtle beauty before you go trying to change it or resort to third-party libraries created by misguided people who were too impatient to understand Go.

 

There is no instant gratification. But there is more gratification lying under the surface than you can imagine right now.

 

 

--

[John Souvestre]

P.S.

 

Meanwhile, in addition to what’s already in sync/atomic, it would be nice to have CAS2/DCAS.  I believe that this would make implementing lock-free easier in many cases.

 

John

    John Souvestre - New Orleans LA

 

From: John Souvestre [mailto:Jo...@Souvestre.com]
Sent: 2016 April 18, Mon 11:34
To: golan...@googlegroups.com
Subject: RE: [go-nuts] Concurrent data structures in golang.

 

I agree.  Many of the common examples of Go’s concurrency (using channels and/or mutexes) don’t scale well enough for use in a 64+ core environment.  As such machines become common I’m hoping that there will be some standard library offerings.

 

Meanwhile, yes – there are some third-party libraries.

 

John

    John Souvestre - New Orleans LA

 

From: golan...@googlegroups.com [mailto:golan...@googlegroups.com] On Behalf Of Vlad Gruchik

Sent: 2016 April 18, Mon 06:15
To: golang-nuts

--

[Patrick Logan]

Do you have references for the 64+ core scenarios? I'm assuming utilizing these many cores well is going to depend mostly on the application needs and design, less on specific language mechanisms. For example I wouldn't expect my current application to scale up to that many cores as-is. 

I see Go's channels and sync package as a fairly "low to mid level" coordination mechanisms. The application design at a higher level need mechanisms above and beyond those. I'm not sure functional data structures will help more at that level _generally_. i.e. In either case, more design work will be necessary for specific application needs.

You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/5ZJjTrDZCaI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.

[John Souvestre]

There are plenty of examples in the literature of systems using locking which top out from 8 to 16 cores.

 

Check Dmitry’s site for lots of good info:  www.1024cores.net

 

John

    John Souvestre - New Orleans LA

 

[Patrick Logan]

Looks useful.  His introduction page and its link to the general-recipes page are a good starting point.

http://www.1024cores.net/home/scalable-architecture/introduction

These pages make clear why functional data structures are not enough, or not even much more desirable than anything else, without a deliberate high-level application design for effectively utilizing many-many cores.

[gardens...@email.com]

You are missing the point of a standard library. The language was made simple to increase productivity. A well rounded standard library would complement the language.

The elephant is the room is polymorphism - that issue has the go team stymied. Just having generic-like functionality within slices and maps seems to be handful. There is no doubt that the data types within container are useful, but giving those data types the same generic-like functionality is a daunting task. Its the reason those types use interfaces, and its the reason that there aren't more types within container.

I believe go:generate is one of the greatest building blocks within the go ecosystem, its a trivial yet extremely useful tool. there are other tools that do similar things but the best thing go:generate has going for it is that its part the standard ecosystem. go:generate has the potential to be a base for solving many of the 'generic' problems in go, and with that opening the way for container data types in go.

The standard library isn't lacking container data types because they aren't useful or aren't in demand. container data types are in the 'roll your own' category until some non-trivial issues are resolved.

[gardens...@email.com]

The elephant in the room is polymorphism, solve that, and you will be up the wahzoo in container data types with regards to go. who knows maybe even some could get upstream into go std. until then, container data types are in the 'roll your own' category.