What is the correct way to access/modify slice elements concurrently
849 views
Skip to first unread message
Kasun Vithanage
Nov 7, 2019, 11:47:03 PM11/7/19
to golang-nuts
Assume we have a type like follows
We would run both concurrently.
type Foo struct { Alive bool}
type Bar struct { Foos []*Foo}In two goroutines we are supposed to read and write the Alive variable. For example
func (b *Bar) CheckFoosAlive() {
for i := 0; i < len(b.Foos); i++ { if b.Foos[i].Alive { fmt.Println("Alive") } }}
func (b* Bar) MarkFooStatus(index int, status bool){ // after bound checking b.Foos[index].Alive = status}We would run both concurrently.
As we can clearly see putting a mutex for the entire Bar struct is seems a bit overhead for me. For example, if CheckFoosAlive is blocking and takes a lot of time, we don't want to lock until it's completed.
The whole idea is to reduce locking as much as possible to get access to an underlying element.
What is the best approach? (I replaced bools with uint32 and used atomic ops as for the current solution)
Would like to know your opinions
Regards,
Kasun
Volker Dobler
Nov 8, 2019, 3:54:02 AM11/8/19
to golang-nuts
On Friday, 8 November 2019 05:47:03 UTC+1, Kasun Vithanage wrote:
What is the best approach?
There is no single "best approach".
If two goroutines A and B access two different indices iA and iB you
do not need any synchronisation between them. If iA==iB you need
to protect reads from concurrent writes.
If you have far more reads than writes: Use a RWMutex, otherwise
a Mutex probably is fine. The trick here: Experiment and measure.
You can synchronise on the Bar-level or on the level of individual Foos.
Or you can group lets say N Foos together and synchronise on the
level of these groups. The trick here: Experiment and measure.
So it boils down to: What is your access pattern? Make several
experiments which simulate this pattern and measure. Then decide
on "the best".
V.
Robert Engels
Nov 8, 2019, 7:07:14 AM11/8/19
to Volker Dobler, golang-nuts
I think that is a bit unclear - even if they access different elements, if they ever access the same element even at different times , you need synchronization- it’s not only if the access the same element “concurrently”.
On Nov 8, 2019, at 2:54 AM, Volker Dobler <dr.volke...@gmail.com> wrote:
--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/e8fd2954-6b46-4337-8564-16e3f771098d%40googlegroups.com.
Marvin Renich
Nov 8, 2019, 8:26:15 AM11/8/19
to golang-nuts
* Kasun Vithanage <alan...@gmail.com> [191107 23:47]:
can be) a type that is handled by the atomic package, that is almost
certainly the best approach.
If the Foo struct is complicated, and you have lots of non-overlapping
access to b.Foos (with the occasional overlapping access), I strongly
suspect that putting a mutex in Foo and using that will give you the
best results.
As Volker said, try several different approaches, and measure with loads
approximating your real-world scenario. Alternatively, implement the
approach that you think is easiest to maintain (from a source code POV),
and test to see if the performance is acceptable under load. If it is,
don't bother trying to optimize.
...Marvin
> type Foo struct {
> Alive bool
> }
>
> type Bar struct {
> Foos []*Foo
> }
>
> Alive bool
> }
>
> type Bar struct {
> Foos []*Foo
> }
>
> func (b *Bar) CheckFoosAlive() {
> for i := 0; i < len(b.Foos); i++ {
> if b.Foos[i].Alive {
> fmt.Println("Alive")
> }
> }
> }
>
> func (b* Bar) MarkFooStatus(index int, status bool){
> // after bound checking
> b.Foos[index].Alive = status
> }
Volker's answer is very good, but for the simple case where Alive is (or
> for i := 0; i < len(b.Foos); i++ {
> if b.Foos[i].Alive {
> fmt.Println("Alive")
> }
> }
> }
>
> func (b* Bar) MarkFooStatus(index int, status bool){
> // after bound checking
> b.Foos[index].Alive = status
> }
can be) a type that is handled by the atomic package, that is almost
certainly the best approach.
If the Foo struct is complicated, and you have lots of non-overlapping
access to b.Foos (with the occasional overlapping access), I strongly
suspect that putting a mutex in Foo and using that will give you the
best results.
As Volker said, try several different approaches, and measure with loads
approximating your real-world scenario. Alternatively, implement the
approach that you think is easiest to maintain (from a source code POV),
and test to see if the performance is acceptable under load. If it is,
don't bother trying to optimize.
...Marvin
burak serdar
Nov 8, 2019, 8:42:51 AM11/8/19
to golang-nuts
On Fri, Nov 8, 2019 at 6:25 AM Marvin Renich <mr...@renich.org> wrote:
>
> * Kasun Vithanage <alan...@gmail.com> [191107 23:47]:
> > type Foo struct {
> > Alive bool
> > }
> >
> > type Bar struct {
> > Foos []*Foo
> > }
> >
> > func (b *Bar) CheckFoosAlive() {
> > for i := 0; i < len(b.Foos); i++ {
> > if b.Foos[i].Alive {
> > fmt.Println("Alive")
> > }
> > }
> > }
> >
> > func (b* Bar) MarkFooStatus(index int, status bool){
> > // after bound checking
> > b.Foos[index].Alive = status
> > }
>
> Volker's answer is very good, but for the simple case where Alive is (or
> can be) a type that is handled by the atomic package, that is almost
> certainly the best approach.
I was thinking about this. In general, it should be safe to replace
>
> * Kasun Vithanage <alan...@gmail.com> [191107 23:47]:
> > type Foo struct {
> > Alive bool
> > }
> >
> > type Bar struct {
> > Foos []*Foo
> > }
> >
> > func (b *Bar) CheckFoosAlive() {
> > for i := 0; i < len(b.Foos); i++ {
> > if b.Foos[i].Alive {
> > fmt.Println("Alive")
> > }
> > }
> > }
> >
> > func (b* Bar) MarkFooStatus(index int, status bool){
> > // after bound checking
> > b.Foos[index].Alive = status
> > }
>
> Volker's answer is very good, but for the simple case where Alive is (or
> can be) a type that is handled by the atomic package, that is almost
> certainly the best approach.
Lock()
read/write one value
Unlock()
with
AtomicRead/Write
Is that correct? The go memory model does not say anything about this.
>
> If the Foo struct is complicated, and you have lots of non-overlapping
> access to b.Foos (with the occasional overlapping access), I strongly
> suspect that putting a mutex in Foo and using that will give you the
> best results.
>
> As Volker said, try several different approaches, and measure with loads
> approximating your real-world scenario. Alternatively, implement the
> approach that you think is easiest to maintain (from a source code POV),
> and test to see if the performance is acceptable under load. If it is,
> don't bother trying to optimize.
>
> ...Marvin
>
> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/20191108132521.bkrbxj2lv7wpinuo%40basil.wdw.
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
Robert Engels
Nov 8, 2019, 8:55:06 AM11/8/19
to burak serdar, golang-nuts
Almost always, but it is very difficult as most times you are not really working with a single value (there are implied dependencies). These sort of solutions fall under “lock free” structures/algorithms - and they are fascinating to learn about.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAMV2RqoGKG8FAqxn-M_E9hN3%2BzqX1UABWmdHu9w6rneKCu%2BsuA%40mail.gmail.com.
Marvin Renich
Nov 8, 2019, 9:55:33 AM11/8/19
to golang-nuts
* burak serdar <bse...@computer.org> [191108 08:42]:
While others have argued that the GMM is not specific enough about this
case, I disagree. The atomic package says it is "useful for
implementing synchronization algorithms". This wording is, in my
opinion, sufficient to make two guarantees:
1. Values read and written with the atomic package will read and
write whole values without corruption.
2. If a memory write from one goroutine is observed by another
goroutine, this establishes a "happens before" relationship.
If these are not true, than the documentation in the atomic package is a
blatant lie. Thus, the memory model does not need any additional
verbiage to clarify this.
And while the next paragraph in the atomic documentation discourages
using it for synchronization, I think the doc'n is overly pessimistic.
Certainly for the simple case of avoiding races due to concurrent
write/read of a single value, the atomic operations are a very good
solution. If you can guarantee that all writes to a particular value
are done on a single goroutine, and all reads and writes are done using
the atomic package, than it is safe to have many other goroutines
reading that value, and using a mutex is overkill.
I do agree that channels and the sync package should be preferred over
implementing something similar with atomic, but don't shy away from
atomic for the simple cases that it does well.
...Marvin
> I was thinking about this. In general, it should be safe to replace
>
> Lock()
> read/write one value
> Unlock()
>
> with
>
> AtomicRead/Write
>
> Is that correct? The go memory model does not say anything about this.
Yes.
>
> Lock()
> read/write one value
> Unlock()
>
> with
>
> AtomicRead/Write
>
> Is that correct? The go memory model does not say anything about this.
While others have argued that the GMM is not specific enough about this
case, I disagree. The atomic package says it is "useful for
implementing synchronization algorithms". This wording is, in my
opinion, sufficient to make two guarantees:
1. Values read and written with the atomic package will read and
write whole values without corruption.
2. If a memory write from one goroutine is observed by another
goroutine, this establishes a "happens before" relationship.
If these are not true, than the documentation in the atomic package is a
blatant lie. Thus, the memory model does not need any additional
verbiage to clarify this.
And while the next paragraph in the atomic documentation discourages
using it for synchronization, I think the doc'n is overly pessimistic.
Certainly for the simple case of avoiding races due to concurrent
write/read of a single value, the atomic operations are a very good
solution. If you can guarantee that all writes to a particular value
are done on a single goroutine, and all reads and writes are done using
the atomic package, than it is safe to have many other goroutines
reading that value, and using a mutex is overkill.
I do agree that channels and the sync package should be preferred over
implementing something similar with atomic, but don't shy away from
atomic for the simple cases that it does well.
...Marvin
burak serdar
Nov 8, 2019, 10:20:23 AM11/8/19
to golang-nuts
barriers, so this should work. Afaik, an atomic read/write does not
yield, so busy-waiting using an atomic will probably not work. Maybe
some of these guarantees (and non-guarantees) can be explicitly stated
in the docs.
>
> ...Marvin
>
> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
Robert Engels
Nov 8, 2019, 11:41:47 AM11/8/19
to burak serdar, golang-nuts
See https://github.com/golang/go/issues/10958 for using atomics in tight/busy-wait loops.
Lars Seipel
Nov 9, 2019, 10:51:26 AM11/9/19
to Robert Engels, Volker Dobler, golang-nuts
On Fri, Nov 08, 2019 at 06:06:22AM -0600, Robert Engels wrote:
>I think that is a bit unclear - even if they access different elements, if they ever access the same element even at different times , you need synchronization- it’s not only if the access the same element “concurrently”.
No, that seems wrong. If, for any two distinct accesses A and B, either
>I think that is a bit unclear - even if they access different elements, if they ever access the same element even at different times , you need synchronization- it’s not only if the access the same element “concurrently”.
A happens before B or B happens before A (i.e. there is no concurrent
access), what would you need additional synchronization for? Access is
already serialized.
-ls
Robert Engels
Nov 9, 2019, 12:00:41 PM11/9/19
to Lars Seipel, Volker Dobler, golang-nuts
No. Because in the absence on a memory barrier the writes may not be flushed meaning you cannot reason about any value ever being changed. atomics provide the memory barrier, but the mm still does not specify a “happens before” relationship (but without this they are fairly useless).
> On Nov 9, 2019, at 9:51 AM, Lars Seipel <lars....@gmail.com> wrote:
> On Nov 9, 2019, at 9:51 AM, Lars Seipel <lars....@gmail.com> wrote:
Lars Seipel
Nov 10, 2019, 9:08:20 AM11/10/19
to Robert Engels, Volker Dobler, golang-nuts
On Sat, Nov 09, 2019 at 11:00:04AM -0600, Robert Engels wrote:
>No. Because in the absence on a memory barrier the writes may not be flushed meaning you cannot reason about any value ever being changed. atomics provide the memory barrier, but the mm still does not specify a “happens before” relationship (but without this they are fairly useless).
Visibility is implied by the definition of "no concurrent access".
>No. Because in the absence on a memory barrier the writes may not be flushed meaning you cannot reason about any value ever being changed. atomics provide the memory barrier, but the mm still does not specify a “happens before” relationship (but without this they are fairly useless).
This case is fully handled by the current memory model:
> Also, if e1 does not happen before e2 and does not happen after e2, then
> we say that e1 and e2 happen concurrently.
[…]
> That is, r is guaranteed to observe w if both of the following hold:
>
> 1. w happens before r.
> 2. Any other write to the shared variable v either happens before w or after r.
[https://golang.org/ref/mem#tmp_2]
Robert Engels
Nov 10, 2019, 9:28:21 AM11/10/19
to Lars Seipel, Volker Dobler, golang-nuts
You are reading the section on “single go routine”. That does not apply here as there are multiple go routines. See https://github.com/golang/go/issues/5045
This is still technical undefined by the memory model.
On Nov 10, 2019, at 8:08 AM, Lars Seipel <lars....@gmail.com> wrote:
--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/20191110140757.GB83577%40horsthansen.slrz.net.
Robert Engels
Nov 10, 2019, 9:32:00 AM11/10/19
to Lars Seipel, Volker Dobler, golang-nuts
Ignore the first part (error!) but the issue applies in terms of atomic.
Robert Engels
Nov 10, 2019, 9:33:39 AM11/10/19
to Lars Seipel, Volker Dobler, golang-nuts
What I meant was that if you read farther down in the synchronization section atomics are not discussed and the issue cited is related.
> On Nov 10, 2019, at 8:31 AM, Robert Engels <ren...@ix.netcom.com> wrote:
>
> Ignore the first part (error!) but the issue applies in terms of atomic.
> On Nov 10, 2019, at 8:31 AM, Robert Engels <ren...@ix.netcom.com> wrote:
>
> Ignore the first part (error!) but the issue applies in terms of atomic.
burak serdar
Nov 11, 2019, 9:03:06 PM11/11/19
to Lars Seipel, Robert Engels, Volker Dobler, golang-nuts
On Sun, Nov 10, 2019 at 7:08 AM Lars Seipel <lars....@gmail.com> wrote:
>
> On Sat, Nov 09, 2019 at 11:00:04AM -0600, Robert Engels wrote:
> >No. Because in the absence on a memory barrier the writes may not be flushed meaning you cannot reason about any value ever being changed. atomics provide the memory barrier, but the mm still does not specify a “happens before” relationship (but without this they are fairly useless).
>
> Visibility is implied by the definition of "no concurrent access".
>
> This case is fully handled by the current memory model:
You cannot define a "happens-before" relationship using only atomics
>
> On Sat, Nov 09, 2019 at 11:00:04AM -0600, Robert Engels wrote:
> >No. Because in the absence on a memory barrier the writes may not be flushed meaning you cannot reason about any value ever being changed. atomics provide the memory barrier, but the mm still does not specify a “happens before” relationship (but without this they are fairly useless).
>
> Visibility is implied by the definition of "no concurrent access".
>
> This case is fully handled by the current memory model:
with the current memory model. The happens-before relationship, the
way it is written, relies on one goroutine blocking until the other
comes to a point where "things happen" before the block is released.
There is no blocking with atomics, hence there is no point in time
where one goroutine can be sure of things happened in the other
goroutine.
The only guarantee with atomics is that when one goroutine reads a
value, it will read the last written value. There are no guarantees on
other values. According to the mm, things that happened before that
final write may not be observable to other goroutines.
>
> > Also, if e1 does not happen before e2 and does not happen after e2, then
> > we say that e1 and e2 happen concurrently.
>
> […]
>
> > That is, r is guaranteed to observe w if both of the following hold:
> >
> > 1. w happens before r.
> > 2. Any other write to the shared variable v either happens before w or after r.
> [https://golang.org/ref/mem#tmp_2]
>
robert engels
Nov 11, 2019, 9:06:02 PM11/11/19
to burak serdar, Lars Seipel, Volker Dobler, golang-nuts
Again, the issue for the mm is https://github.com/golang/go/issues/5045 but if you read the comments it appears the the de-facto standard is that the atomics do provide “happens before”, but it is not specified that this has to be the case.
Marvin Renich
Nov 12, 2019, 12:21:29 PM11/12/19
to golang-nuts
There are two different viewpoints you can take. Either the Go Memory
Model must stand alone, and any concurrency claims made by the language
and standard library must be based on the limited set of operations
defined in the GMM, or the GMM provides the definitions and provides a
substantial, but not complete, list of operations satisfying that
definition, and a standard library package (or language feature) may
claim to satisfy the GMM definitions by some unspecified internal means.
If you accept the second, than I believe the documentation of the
sync/atomic package is enough to allow atomic operations to be used to
establish a happens-before relationship.
* burak serdar <bse...@computer.org> [191111 21:03]:
with this.
Sort of, because any blocking that occurs takes place at the hardware
level (CPU and memory controller working out any cache and memory bus
contention).
> where one goroutine can be sure of things happened in the other
> goroutine.
>
> The only guarantee with atomics is that when one goroutine reads a
> value, it will read the last written value. There are no guarantees on
> other values. According to the mm, things that happened before that
> final write may not be observable to other goroutines.
The memory model is not defined in terms of blocking; in fact the only
mention of blocking is in the explanatory, non-normative text for
sync.Once. Blocking is a consequence of some operations in order to
obey the memory model; it is not the cause.
The memory model is defined in terms of happens-before relationships,
and says that certain operations create such a relationship.
My claim is that the GMM gives exactly three mutually exclusive choices:
Also, if e1 does not happen before e2 and does not happen after e2,
then we say that e1 and e2 happen concurrently.
There is no fourth choice, so if e1 and e2 do not happen concurrently,
then they have one of the happens-before relationships.
Most of the synchronization operations (e.g. channel reads and writes)
define that a happens-before relationship exists in a specific
direction.
The atomic package is different in that it specifies that two atomic
operations to the same memory location do not happen concurrently. By
logical inference, rather than explicit statement, there must be a
happens-before relationship, but the direction of that relationship is
not specified. You must use "pure logic" (in the mathematical sense),
if possible, to determine the direction.
I would be perfectly happy to have the memory model specify:
If an atomic read r [in one goroutine] can be proven to have observed
a specific atomic write w [from another goroutine], than the w
happens-before the r.
I like this even better:
Two atomic operations to the same variable v do not happen
concurrently, and thus a happens-before relationship exists between
them in an unspecified direction. Sometimes logic may be used to
prove the direction of the relationship.
If you adhere to the first viewpoint at the top of this message, than I
would say something like the above would be a mandatory addition to the
GMM document. However, I am perfectly happy with the viewpoint that the
GMM provides the definitions, and the list of operations satisfying
those definitions can be specified in the Language Specification and the
standard library documentation.
...Marvin
Model must stand alone, and any concurrency claims made by the language
and standard library must be based on the limited set of operations
defined in the GMM, or the GMM provides the definitions and provides a
substantial, but not complete, list of operations satisfying that
definition, and a standard library package (or language feature) may
claim to satisfy the GMM definitions by some unspecified internal means.
If you accept the second, than I believe the documentation of the
sync/atomic package is enough to allow atomic operations to be used to
establish a happens-before relationship.
* burak serdar <bse...@computer.org> [191111 21:03]:
> You cannot define a "happens-before" relationship using only atomics
^ Accepting the atomic documentation, I disagree with this.
> with the current memory model. The happens-before relationship, the
^ Either way, I completely disagree
with this.
> way it is written, relies on one goroutine blocking until the other
> comes to a point where "things happen" before the block is released.
> There is no blocking with atomics, hence there is no point in time
^ I (sort of) agree with this, ^ but not this.
> comes to a point where "things happen" before the block is released.
> There is no blocking with atomics, hence there is no point in time
Sort of, because any blocking that occurs takes place at the hardware
level (CPU and memory controller working out any cache and memory bus
contention).
> where one goroutine can be sure of things happened in the other
> goroutine.
>
> The only guarantee with atomics is that when one goroutine reads a
> value, it will read the last written value. There are no guarantees on
> other values. According to the mm, things that happened before that
> final write may not be observable to other goroutines.
mention of blocking is in the explanatory, non-normative text for
sync.Once. Blocking is a consequence of some operations in order to
obey the memory model; it is not the cause.
The memory model is defined in terms of happens-before relationships,
and says that certain operations create such a relationship.
My claim is that the GMM gives exactly three mutually exclusive choices:
Also, if e1 does not happen before e2 and does not happen after e2,
then we say that e1 and e2 happen concurrently.
then they have one of the happens-before relationships.
Most of the synchronization operations (e.g. channel reads and writes)
define that a happens-before relationship exists in a specific
direction.
The atomic package is different in that it specifies that two atomic
operations to the same memory location do not happen concurrently. By
logical inference, rather than explicit statement, there must be a
happens-before relationship, but the direction of that relationship is
not specified. You must use "pure logic" (in the mathematical sense),
if possible, to determine the direction.
I would be perfectly happy to have the memory model specify:
If an atomic read r [in one goroutine] can be proven to have observed
a specific atomic write w [from another goroutine], than the w
happens-before the r.
I like this even better:
Two atomic operations to the same variable v do not happen
concurrently, and thus a happens-before relationship exists between
them in an unspecified direction. Sometimes logic may be used to
prove the direction of the relationship.
If you adhere to the first viewpoint at the top of this message, than I
would say something like the above would be a mandatory addition to the
GMM document. However, I am perfectly happy with the viewpoint that the
GMM provides the definitions, and the list of operations satisfying
those definitions can be specified in the Language Specification and the
standard library documentation.
...Marvin
Marvin Renich
Nov 12, 2019, 12:31:15 PM11/12/19
to golang-nuts
[I almost missed this post because you did not reply to the thread.]
* Robert Engels <ren...@ix.netcom.com> [191108 11:41]:
This doesn't have anything to do with atomics. Atomic operations are
just one of many, many things you can do in a busy loop. There is
nothing about atomics that is any more relevant to this issue than the
increment statement, i++.
...Marvin
* Robert Engels <ren...@ix.netcom.com> [191108 11:41]:
This doesn't have anything to do with atomics. Atomic operations are
just one of many, many things you can do in a busy loop. There is
nothing about atomics that is any more relevant to this issue than the
increment statement, i++.
...Marvin
burak serdar
Nov 12, 2019, 12:45:12 PM11/12/19
to golang-nuts
On Tue, Nov 12, 2019 at 10:21 AM Marvin Renich <mr...@renich.org> wrote:
>
> There are two different viewpoints you can take. Either the Go Memory
> Model must stand alone, and any concurrency claims made by the language
> and standard library must be based on the limited set of operations
> defined in the GMM, or the GMM provides the definitions and provides a
> substantial, but not complete, list of operations satisfying that
> definition, and a standard library package (or language feature) may
> claim to satisfy the GMM definitions by some unspecified internal means.
>
> If you accept the second, than I believe the documentation of the
> sync/atomic package is enough to allow atomic operations to be used to
> establish a happens-before relationship.
Is there a guarantee that the compiler will not reorganize
>
> There are two different viewpoints you can take. Either the Go Memory
> Model must stand alone, and any concurrency claims made by the language
> and standard library must be based on the limited set of operations
> defined in the GMM, or the GMM provides the definitions and provides a
> substantial, but not complete, list of operations satisfying that
> definition, and a standard library package (or language feature) may
> claim to satisfy the GMM definitions by some unspecified internal means.
>
> If you accept the second, than I believe the documentation of the
> sync/atomic package is enough to allow atomic operations to be used to
> establish a happens-before relationship.
instructions around an atomic read/write? That is:
i++
k:=atomic.AddInt32(&j,1)
Is there a guarantee that the compiler won't rewrite this as:
k:=atomic.AddInt32(&j,1)
i++
> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/20191112172034.nkhmoaqjjrawc7wx%40basil.wdw.
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
Robert Engels
Nov 12, 2019, 12:59:13 PM11/12/19
to golang-nuts
The bug I referenced discusses the current problem with the MM specification. You are making assumptions that are not supported by the current MM, but as the bug points out, that is the current behavior.
Btw, there is no such thing as "concurrent access". There is a concept of "sharing" and the visibility of operations, and the resulting "memory consistency".
The current MM is very specific about which operations create a "happens before" relationship (an important property of memory consistency), atomic operations are not listed.
-----Original Message-----
>From: Marvin Renich <mr...@renich.org>
>Sent: Nov 12, 2019 11:20 AM
>To: golang-nuts <golan...@googlegroups.com>
Btw, there is no such thing as "concurrent access". There is a concept of "sharing" and the visibility of operations, and the resulting "memory consistency".
The current MM is very specific about which operations create a "happens before" relationship (an important property of memory consistency), atomic operations are not listed.
-----Original Message-----
>From: Marvin Renich <mr...@renich.org>
>Sent: Nov 12, 2019 11:20 AM
>To: golang-nuts <golan...@googlegroups.com>
>Subject: Re: [go-nuts] Re: What is the correct way to access/modify slice elements concurrently
>
>There are two different viewpoints you can take. Either the Go Memory
>Model must stand alone, and any concurrency claims made by the language
>and standard library must be based on the limited set of operations
>defined in the GMM, or the GMM provides the definitions and provides a
>substantial, but not complete, list of operations satisfying that
>definition, and a standard library package (or language feature) may
>claim to satisfy the GMM definitions by some unspecified internal means.
>
>If you accept the second, than I believe the documentation of the
>sync/atomic package is enough to allow atomic operations to be used to
>establish a happens-before relationship.
>
>
>There are two different viewpoints you can take. Either the Go Memory
>Model must stand alone, and any concurrency claims made by the language
>and standard library must be based on the limited set of operations
>defined in the GMM, or the GMM provides the definitions and provides a
>substantial, but not complete, list of operations satisfying that
>definition, and a standard library package (or language feature) may
>claim to satisfy the GMM definitions by some unspecified internal means.
>
>If you accept the second, than I believe the documentation of the
>sync/atomic package is enough to allow atomic operations to be used to
>establish a happens-before relationship.
>
Robert Engels
Nov 12, 2019, 1:00:45 PM11/12/19
to golang-nuts
This is not the issue I am referring to, I am referring to https://github.com/golang/go/issues/5045
-----Original Message-----
>From: Marvin Renich <mr...@renich.org>
>Sent: Nov 12, 2019 11:30 AM
>To: golang-nuts <golan...@googlegroups.com>
-----Original Message-----
>From: Marvin Renich <mr...@renich.org>
>Sent: Nov 12, 2019 11:30 AM
>To: golang-nuts <golan...@googlegroups.com>
>...Marvin
>
>--
>You received this message because you are subscribed to the Google Groups "golang-nuts" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
>To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/20191112173043.6mfa7iejihhkasos%40basil.wdw.
>
>--
>You received this message because you are subscribed to the Google Groups "golang-nuts" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
Marvin Renich
Nov 12, 2019, 1:42:40 PM11/12/19
to golang-nuts
[You keep sending replies that are not connected to the message to which
you are replying. RFC 822 (and subsequent RFCs) defines the In-Reply-To
header, which has been in use for more than 30 years. Most MUAs will
add this automatically. Failure to add the header really messes up MUAs
that display messages from the same thread together. Does your MUA not
add this header???]
* Robert Engels <ren...@ix.netcom.com> [191112 13:00]:
message:
>From: burak serdar <bse...@computer.org>
>Sent: Nov 8, 2019 9:19 AM
>To: golang-nuts <golan...@googlegroups.com>
>Subject: Re: [go-nuts] What is the correct way to access/modify slice elements concurrently
which has
Message-ID: <CAMV2RqoyJMC3XXTBKdcAitKUyvyzbykvrfZPgQcLw5=+Y+0...@mail.gmail.com>
What were you trying to say with your link to issue 10958?
...Marvin
you are replying. RFC 822 (and subsequent RFCs) defines the In-Reply-To
header, which has been in use for more than 30 years. Most MUAs will
add this automatically. Failure to add the header really messes up MUAs
that display messages from the same thread together. Does your MUA not
add this header???]
* Robert Engels <ren...@ix.netcom.com> [191112 13:00]:
> This is not the issue I am referring to, I am referring to https://github.com/golang/go/issues/5045
>
> -----Original Message-----
> >From: Marvin Renich <mr...@renich.org>
> >Sent: Nov 12, 2019 11:30 AM
> >To: golang-nuts <golan...@googlegroups.com>
> >Subject: Re: [go-nuts] What is the correct way to access/modify slice elements concurrently
> >
> >[I almost missed this post because you did not reply to the thread.]
> >
> >* Robert Engels <ren...@ix.netcom.com> [191108 11:41]:
> >> See https://github.com/golang/go/issues/10958 for using atomics in tight/busy-wait loops.
> >
> >This doesn't have anything to do with atomics. Atomic operations are
> >just one of many, many things you can do in a busy loop. There is
> >nothing about atomics that is any more relevant to this issue than the
> >increment statement, i++.
> >
> >...Marvin
Hmm. I am really confused. You sent the link I quoted in reply to this
>
> -----Original Message-----
> >From: Marvin Renich <mr...@renich.org>
> >Sent: Nov 12, 2019 11:30 AM
> >To: golang-nuts <golan...@googlegroups.com>
> >Subject: Re: [go-nuts] What is the correct way to access/modify slice elements concurrently
> >
> >[I almost missed this post because you did not reply to the thread.]
> >
> >* Robert Engels <ren...@ix.netcom.com> [191108 11:41]:
> >> See https://github.com/golang/go/issues/10958 for using atomics in tight/busy-wait loops.
> >
> >This doesn't have anything to do with atomics. Atomic operations are
> >just one of many, many things you can do in a busy loop. There is
> >nothing about atomics that is any more relevant to this issue than the
> >increment statement, i++.
> >
> >...Marvin
message:
>From: burak serdar <bse...@computer.org>
>Sent: Nov 8, 2019 9:19 AM
>To: golang-nuts <golan...@googlegroups.com>
>Subject: Re: [go-nuts] What is the correct way to access/modify slice elements concurrently
Message-ID: <CAMV2RqoyJMC3XXTBKdcAitKUyvyzbykvrfZPgQcLw5=+Y+0...@mail.gmail.com>
What were you trying to say with your link to issue 10958?
...Marvin
Robert Engels
Nov 12, 2019, 1:44:20 PM11/12/19
to golang-nuts
Sorry, at work so I need to use the 'web mail' interface and it doesn't appear to be including them when I reply.
>--
>You received this message because you are subscribed to the Google Groups "golang-nuts" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
>To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/20191112184159.2yep4f66wutu776l%40basil.wdw.
>You received this message because you are subscribed to the Google Groups "golang-nuts" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
Marvin Renich
Nov 12, 2019, 1:48:03 PM11/12/19
to golang-nuts
* Robert Engels <ren...@ix.netcom.com> [191112 13:44]:
...Marvin
> Sorry, at work so I need to use the 'web mail' interface and it
> doesn't appear to be including them when I reply.
Understood. I'll just try to follow the threading as best as I can.
> doesn't appear to be including them when I reply.
...Marvin
Marvin Renich
Nov 12, 2019, 2:25:17 PM11/12/19
to golang-nuts
* Robert Engels <ren...@ix.netcom.com> [191112 12:59]:
not the only possible POV.
> Btw, there is no such thing as "concurrent access". There is a concept
> of "sharing" and the visibility of operations, and the resulting
> "memory consistency".
I never used the term "concurrent access". The Go MM never uses the
terms "visibility" or "memory consistency". Are we talking about the
same document? I am talking about «https://golang.org/ref/mem».
The MM defines "happens before" and "happens concurrently". I might
have made a mistake in the terminology that I used, but I do not see it,
and you have not given me enough information to determine what mistake
you believe I have made.
> The current MM is very specific about which operations create a
> "happens before" relationship (an important property of memory
> consistency), atomic operations are not listed.
(same as my first paragraph above)
If the writers of the MM intended it to enumerate all possible language
and standard library features that implement a happens before
relationship, as opposed to defining the terminology and allowing the
language spec and std lib docn to specify that such-and-such a feature
establishes a happens-before relationship, then I agree that something
needs to be added to the MM to cover atomics.
...Marvin
> The bug I referenced discusses the current problem with the MM
> specification. You are making assumptions that are not supported by
> the current MM, but as the bug points out, that is the current
> behavior.
I can see that point of view, and I don't think it is incorrect, just
> specification. You are making assumptions that are not supported by
> the current MM, but as the bug points out, that is the current
> behavior.
not the only possible POV.
> Btw, there is no such thing as "concurrent access". There is a concept
> of "sharing" and the visibility of operations, and the resulting
> "memory consistency".
terms "visibility" or "memory consistency". Are we talking about the
same document? I am talking about «https://golang.org/ref/mem».
The MM defines "happens before" and "happens concurrently". I might
have made a mistake in the terminology that I used, but I do not see it,
and you have not given me enough information to determine what mistake
you believe I have made.
> The current MM is very specific about which operations create a
> "happens before" relationship (an important property of memory
> consistency), atomic operations are not listed.
If the writers of the MM intended it to enumerate all possible language
and standard library features that implement a happens before
relationship, as opposed to defining the terminology and allowing the
language spec and std lib docn to specify that such-and-such a feature
establishes a happens-before relationship, then I agree that something
needs to be added to the MM to cover atomics.
...Marvin
Marvin Renich
Nov 13, 2019, 11:28:47 AM11/13/19
to golang-nuts
* burak serdar <bse...@computer.org> [191112 12:45]:
that the language designers are in agreement that if atomics are not
covered by the Go Memory Model, they should be. From this, it seems
safe to me to treat them as if they provide a
does-not-happen-concurrently promise.
From that (and, independently, from the fact that atomics would
otherwise lose all their documented usefulness) it follows that you have
the same guarantee about reordering that you would have if you replaced
the atomic access with a mutex or channel operation. Everybody seems to
agree that the compiler can only reorder if such reordering not only
does not affect the behavior of the goroutine being reordered, but also
does not affect the behavior observed by other goroutines _where a
happens-before relationship between those goroutines_ promises such
behavior.
Unfortunately, I cannot find any wording in the GMM that prevents
reordering one goroutine when such reordering breaks a transitive
happens-before relationship with an operation in another goroutine.
The permission to reorder one goroutine is given before the term
"happens before" is defined. The GMM makes specific guarantees about
the non-transitive happens-before relationships between goroutines, and
gives examples that clearly demonstrate transitive intent, but that
transitive property is never explicitly stated.
In other words, the permission to reorder within one goroutine is never
limited to reordering happens-before operations that do not involve
other goroutines.
I believe this is a much more egregious omission in the GMM than not
mentioning atomics. It is clearly intended, otherwise the whole GMM is
completely useless.
...Marvin
> Is there a guarantee that the compiler will not reorganize
> instructions around an atomic read/write? That is:
>
> i++
> k:=atomic.AddInt32(&j,1)
>
> Is there a guarantee that the compiler won't rewrite this as:
>
> k:=atomic.AddInt32(&j,1)
> i++
First, from one of the issues to which Robert Engels pointed, it appears
> instructions around an atomic read/write? That is:
>
> i++
> k:=atomic.AddInt32(&j,1)
>
> Is there a guarantee that the compiler won't rewrite this as:
>
> k:=atomic.AddInt32(&j,1)
> i++
that the language designers are in agreement that if atomics are not
covered by the Go Memory Model, they should be. From this, it seems
safe to me to treat them as if they provide a
does-not-happen-concurrently promise.
From that (and, independently, from the fact that atomics would
otherwise lose all their documented usefulness) it follows that you have
the same guarantee about reordering that you would have if you replaced
the atomic access with a mutex or channel operation. Everybody seems to
agree that the compiler can only reorder if such reordering not only
does not affect the behavior of the goroutine being reordered, but also
does not affect the behavior observed by other goroutines _where a
happens-before relationship between those goroutines_ promises such
behavior.
Unfortunately, I cannot find any wording in the GMM that prevents
reordering one goroutine when such reordering breaks a transitive
happens-before relationship with an operation in another goroutine.
The permission to reorder one goroutine is given before the term
"happens before" is defined. The GMM makes specific guarantees about
the non-transitive happens-before relationships between goroutines, and
gives examples that clearly demonstrate transitive intent, but that
transitive property is never explicitly stated.
In other words, the permission to reorder within one goroutine is never
limited to reordering happens-before operations that do not involve
other goroutines.
I believe this is a much more egregious omission in the GMM than not
mentioning atomics. It is clearly intended, otherwise the whole GMM is
completely useless.
...Marvin
Reply all
Reply to author
Forward
0 new messages