Race detector question

[robert engels]

Hi,

I am working on a new project, and the race detector is reporting a race.

Essentially, the code is

var S []int

several go routines write new S values using a mutex

go routine Y reads S without grabbing a lock (it reads it initially under lock)

The semantics are such that Y can operate successfully with any valid value of S (e.g. could be stale). (essentially S is used with copy on write semantics)

The race detector reports this as a race.

I could change all reads of Y to use an atomic load, but I don’t think it should be necessary.

Is there any way to perform “lazy loads” in Go?

And a follow-up:

Is the race detector smart enough so that if a routines write to several vars (v1…n) and performs an atomic store to X, and another routine atomically reads X it can also non atomically read v1…n and it will see the stored values?

This has been the long standing issue with the Go memory model and “happens before”… but how does the race detector report this?

(Some background, the library functions fine under heavy concurrent stress tests - but the race detector says it is broken).

[robert engels]

Ignore the first part - got bit by the ol' append() and that slices are a struct not a pointer - so they can’t be atomically updated (need to use atomic.Value I guess) - so it was simpler to wrap in Lock/Unlock when accessing the slice and ensure a new slice was used during assignment.

The follow-up question still stands.

[Thomas Bushnell BSG]

You cannot make that assumption. It's not about what the race detector can detect.

Goroutine one:

  Writes non-synchronized X

  Writes atomic Y

  Writes non-synchronized Z with the value of X+Y

Goroutine two

  Reads atomic Y and sees the new value

Can goroutine two now read non-synchronized X and assume it sees the new value written by one? No, it cannot. There is no "happens before" relation connecting the two writes performed by goroutine one. Requirement one does not establish such a relationship. It only establishes that Z will be written with the correct sum of X and Y. There must be some sequential order within the context of goroutine one that sees the correct value; the compiler is free to swap the order of the writes X and Y.

If X were an atomic, then Requirement two would come into play. But because X and Z are not atomic, they play no role in Requirement two. Note that the description of atomic in the model says that writes to atomic values have the property you want. And since there is no before relationship established by any of the following text, this synchronization cannot be relied on.

Now you're asking whether the race detector ensures the synchronization property you're suggesting? The race detector doesn't ensure any synchronization properties; it detects bugs.

I think it is capable of detecting this one.

Thomas

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/8EC74417-C4AD-4490-9231-6E869EE72D93%40ix.netcom.com.

[burak serdar]

On Thu, Sep 15, 2022 at 8:03 AM 'Thomas Bushnell BSG' via golang-nuts <golan...@googlegroups.com> wrote:

You cannot make that assumption. It's not about what the race detector can detect.

Goroutine one:

  Writes non-synchronized X

  Writes atomic Y

  Writes non-synchronized Z with the value of X+Y

Goroutine two

  Reads atomic Y and sees the new value

The way I read the Go memory model, if Goroutine two sees the new value of Y, non-synchronizes writes to X by Goroutine 2 happened before Y, and thus, anything that happens after Y. This is based on:

"If a synchronizing read-like memory operation r observes a synchronizing write-like memory operation w (that is, if W(r) = w), then w is synchronized before r."

And:

"The happens before relation is defined as the transitive closure of the union of the sequenced before and synchronized before relations."

Because: 

  * The writes to non-synchronized X are sequenced before the atomic write to Y

  * The atomic read Y happened after atomic write to Y if it sees the new value

  * non-synchronized reads from X happen after that

So that should not be a race.

Am I reading this correctly? 

 

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CA%2BYjuxtd%2BpaU_BNxXDrMAN9v71r-Qhm9LcXcN2fTtjD_6oWw-Q%40mail.gmail.com.

[Robert Engels]

I think it needs to see the updated X - which agrees with burak. 

Reading Z is race. 

On Sep 15, 2022, at 9:24 AM, burak serdar <bse...@computer.org> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAMV2Rqr4vggPOWjiQg6qN0tJjhhXncKHLMCDwkqZTHBJJ7%3Dmug%40mail.gmail.com.

[Robert Engels]

To clarify, if the atomic read of Y sees the updated Y then a subsequent non-atomic read of X must see the updated X. This is a happens before relationship. 

The question was if the race detector understands this - I know - why not try it out…

On Sep 15, 2022, at 9:39 AM, Robert Engels <ren...@ix.netcom.com> wrote:



[robert engels]

Yea, the race detector is broken… it fails on the following code:

package main

import (
   "sync"
   "sync/atomic"
)

func main() {

   var x int32
   var y int32

   w := sync.WaitGroup{}
   w.Add(2)

   go func() {
      for {
         x = 1
         atomic.StoreInt32(&y, 1)
      }
      w.Done()
   }()
   go func() {
      for {
         if atomic.LoadInt32(&y) == 1 {
            if x != 1 {
               panic("should not happen")
            }
         }
      }
      w.Done()
   }()
   w.Wait()

}

The above code does not have a race, or Go doesn’t have “happens before” semantics with its atomics.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/93E2826E-1D85-48BE-BBD3-ED17F70C74F2%40ix.netcom.com.

[Thomas Bushnell BSG]

Happens before works just fine with atomics. But in your example, x is not an atomic. 

Thomas

[robert engels]

To clarify, this is not normally how this is used (for people not seeing the utility).

Typically there is a variable which is a pointer to a struct.

A new struct is created, members set, and then the reference pointer is atomically set to the address of the new struct.

A reader that reads the struct pointer atomically is guaranteed to see see the latest values in the struct (not some indeterminant values)

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/621ABC85-EFBF-4B9D-B94B-F27D6BDA7E85%40ix.netcom.com.

[robert engels]

This is not what “happens before” means - at least not in any other accepted concurrency designs.

See my second example as to why.

[Thomas Bushnell BSG]

I cannot speak to "other accepted concurrency designs" here. Simply that Go does not guarantee the operation you want it to, the memory model does not actually imply that it does, and the last sentence of the memory model is the most important one here: don't be clever.

[robert engels]

Please see the “sequentially consistent” here https://gcc.gnu.org/wiki/Atomic/GCCMM/AtomicSync

This is what most references to “happens before” imply. (and their example is my exact example).

[Thomas Bushnell BSG]

Well, what the Go memory model says is this:

"The memory operations in each goroutine must correspond to a correct sequential execution of that goroutine, given the values read from and written to memory. That execution must be consistent with the sequenced before relation, defined as the partial order requirements set out by the Go language specification for Go's control flow constructs as well as the order of evaluation for expressions."

What that means is, you follow the semantics of the language to determine what a sequential "everything in order" execution would do, and you keep track of the reads and writes which that does. The compiler can meet Requirement One provided the "values read from and written to memory" by "that goroutine" are the same, and they must be consistent with the "sequenced before" relation that is described in the following sections of the memory model.

The sequenced before relation established by atomics is this:

"The APIs in the sync/atomic package are collectively “atomic operations” that can be used to synchronize the execution of different goroutines. If the effect of an atomic operation A is observed by atomic operation B, then A is synchronized before B. All the atomic operations executed in a program behave as though executed in some sequentially consistent order."

This only applies if both A and B are atomic operations. 

[robert engels]

This is simply incorrect. The ‘issue’ about clarifying the memory has been about “happens before” since the beginning. The model was clarified. The race detector cannot cope with it.

I don’t think you fully understand what “happens before” means. Please read the article I referred to.

[burak serdar]

On Thu, Sep 15, 2022 at 9:11 AM Thomas Bushnell BSG <tbus...@google.com> wrote:

I cannot speak to "other accepted concurrency designs" here. Simply that Go does not guarantee the operation you want it to, the memory model does not actually imply that it does, and the last sentence of the memory model is the most important one here: don't be clever.

I believe that's what we are discussing here. The way I read it, the memory model does imply that a non-synchronized operation x sequenced before an operation y  that happened before z implies that all operations sequenced after z happens after x.

[Thomas Bushnell BSG]

On Thu, Sep 15, 2022 at 11:16 AM robert engels <ren...@ix.netcom.com> wrote:

This is simply incorrect. The ‘issue’ about clarifying the memory has been about “happens before” since the beginning. The model was clarified. The race detector cannot cope with it.

I don’t think you fully understand what “happens before” means. Please read the article I referred to.

The article you referred to is a casual description of a different language, neither the official GCC documentation itself nor the C++ standard. I'm not making representations about the semantics of C++ in any way.

I understand the model, the race-detector is implementing the model, and the model is as explicit as it can be that atomics create a "happens before" relationship only when both variables concerned are atomic. I don't see any reason to continue this conversation, so I'll leave off here.

Thomas

[robert engels]

Finally, you left out the most important clause from the “memory model” document:

"The preceding definition has the same semantics as C++’s sequentially consistent atomics and Java’s volatile variables.”

Both of these require the “happens before” semantics I describe - not what you are describing.

[Thomas Bushnell BSG]

On Thu, Sep 15, 2022 at 11:19 AM burak serdar <bse...@computer.org> wrote:

On Thu, Sep 15, 2022 at 9:11 AM Thomas Bushnell BSG <tbus...@google.com> wrote:

I cannot speak to "other accepted concurrency designs" here. Simply that Go does not guarantee the operation you want it to, the memory model does not actually imply that it does, and the last sentence of the memory model is the most important one here: don't be clever.

I believe that's what we are discussing here. The way I read it, the memory model does imply that a non-synchronized operation x sequenced before an operation y  that happened before z implies that all operations sequenced after z happens after x.

You may be misreading requirement one as implying that no reordering within a goroutine is ever permitted. It does not say that. What it says is that, within one goroutine, any reordering is permissible provided the actual value read and written by that goroutine are the correct ones.

Thomas

[burak serdar]

The memory model doesn't stop there though. In particular, this section:  https://go.dev/ref/mem#model

Thomas

[burak serdar]

I believe what the memory model implies is that no reordering is permitted to cross a memory barrier. An atomic operation has a memory barrier.

 

Thomas

[Dan Kortschak]

To extend this, I think the misconception here is the idea that atomics
are flow synchronisation operations in Go. They are not; there is
nothing in the code that makes a happens before relationship between
the write to x in the top goroutine and the read from x in the bottom.

On Thu, 2022-09-15 at 11:02 -0400, 'Thomas Bushnell BSG' via golang-

> > > > > > https://groups.google.com/d/msgid/golang-nuts/CA%2BYjuxtd%2BpaU_BNxXDrMAN9v71r-Qhm9LcXcN2fTtjD_6oWw-Q%40mail.gmail.com
> > > > > > .

> > > > >
> > > > > --
> > > > > You received this message because you are subscribed to the
> > > > > Google Groups "golang-nuts" group.
> > > > > To unsubscribe from this group and stop receiving emails from
> > > > > it, send an email to
> > > > > golang-nuts...@googlegroups.com.
> > > > > To view this discussion on the web visit

> > > > > https://groups.google.com/d/msgid/golang-nuts/CAMV2Rqr4vggPOWjiQg6qN0tJjhhXncKHLMCDwkqZTHBJJ7%3Dmug%40mail.gmail.com
> > > > > .
> > >

[robert engels]

After some analysis and discussion, the sample I provided earlier has a data race by definition - although in many cases these are benign and do no affect the correctness of the program. E.g. the program may be “sampling” and so any valid value read is “ok”. In detail, because of the loop, the writer could mutate the y value multiple times before the reader observed the change and read the updated value - thus the “race".

As to the “all accesses must be atomic” contention for “happens before” - this is not correct.The following is a valid and common use case of atomics even though multiple routines are modifying a variable non-atomically. The race detector does not report any issues. The CAS below creates a spin lock, but the technique can be more advanced with similar purpose - synchronization without locks/blocking - depending on the data structure.

The neat thing is how the race detector is able to detect that the following is “ok” - I look forward to understanding more on how that is accomplished.

(As a mea culpa - it was me forgetting that slices are a struct not a pointer - and so atomically manipulating them is not possible - at least not directly).

 

package main

import (
   "fmt"

   "sync"
   "sync/atomic"
)

func main() {

   var x int32
   var y int32

   w := sync.WaitGroup{}

   w.Add(3)

   go func() {
      for {
         if atomic.CompareAndSwapInt32(&x, 0, 1) {
            y = y + 1
            atomic.StoreInt32(&x, 0)

         }
      }
      w.Done()
   }()
   go func() {
      for {

         if atomic.CompareAndSwapInt32(&x, 0, 1) {
            y = y + 1
            atomic.StoreInt32(&x, 0)

         }
      }
      w.Done()
   }()
   go func() {
      for {

         if atomic.CompareAndSwapInt32(&x, 0, 1) {
            if y%1000000 == 0 {
               fmt.Print("matched")
            }
            atomic.StoreInt32(&x, 0)
         }
      }
      w.Done()
   }()
   w.Wait()
}

On Sep 15, 2022, at 10:59 AM, Robert Engels <ren...@ix.netcom.com> wrote:

I have not misunderstood it. Nor have I misunderstood the semantics with Javas volatile. One aspect of Go is broken - tbd - but most likely it is simply the race detector cannot support it (as my concurrent code functions as expected given the memory model). 

On Sep 15, 2022, at 10:22 AM, Thomas Bushnell BSG <tbus...@google.com> wrote:



The GCC Wiki you reference is speaking only of a case where all the variables concerned are in a class that specifies sequentially consistent ordering. You have also misunderstood the C++ semantics, which is understandable since the language is a disaster of bad specification.

Thomas

On Thu, Sep 15, 2022 at 11:20 AM robert engels <ren...@ix.netcom.com> wrote:

Finally, you left out the most important clause from the “memory model” document:

"The preceding definition has the same semantics as C++’s sequentially consistent atomics and Java’s volatile variables.”

Both of these require the “happens before” semantics I describe - not what you are describing.

On Sep 15, 2022, at 10:16 AM, robert engels <ren...@ix.netcom.com> wrote:

This is simply incorrect. The ‘issue’ about clarifying the memory has been about “happens before” since the beginning. The model was clarified. The race detector cannot cope with it.

I don’t think you fully understand what “happens before” means. Please read the article I referred to.

On Sep 15, 2022, at 10:11 AM, Thomas Bushnell BSG <tbus...@google.com> wrote:

I cannot speak to "other accepted concurrency designs" here. Simply that Go does not guarantee the operation you want it to, the memory model does not actually imply that it does, and the last sentence of the memory model is the most important one here: don't be clever.

-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAMV2RqoZ2J8hHY6mD_rNdJu7W4p0osW0E%2BUxx8pYnh815%3DFmTw%40mail.gmail.com.

[Jan Mercl]

On Fri, Sep 16, 2022 at 7:43 AM robert engels <ren...@ix.netcom.com> wrote:

> After some analysis and discussion, the sample I provided earlier has a data race by definition - although in many cases these are benign and do no affect the correctness of the program.

There's no such thing [in Go and many other languages] as a benign
data race: https://docs.google.com/document/d/1WAT22FtFdMt43i3O8YrnlQTNTzZ3IoJBtu3P2DNRytg/edit

> E.g. the program may be “sampling” and so any valid value read is “ok”.

Even when we forget that a data race can crash your program on certain
HW, the above quoted means you can replace the read with a PRNG, but
then, assuming your code is still correct, why bother with the read at
all? Because a racy read can produce not only some "old" value vs
"new" value with some probability. The racy read can produce any value
at all. Sure, at least some architectures have stronger guarantees and
they really produce "old" or "new" and nothing else, for certain sizes
and/or alignments of values. But not in the general case or for any
architecture that Go may support in the future.

tl;dr: You must always fix a data race, there are no exceptions.

[Jan Mercl]

To ren...@ix.netcom.com:

Please stop your silly bot or use a different email address. You're
using this email to participate in a discussion in a mail group. It's
pretty common to reply to the group and to the OP.

Feel free to send me an off-list message if _you_ think it's
necessary, but avoid a reply bot to spam my inbox, thank you.
Otherwise I propose the admin of this list to ban your email.

---------- Forwarded message ---------
From: <ren...@ix.netcom.com>
Date: Fri, Sep 16, 2022 at 9:09 AM
Subject: Re: Re: [go-nuts] Race detector question
To: Jan Mercl <0xj...@gmail.com>


I apologize for this automatic reply to your email.

To control spam, I now allow incoming messages only from senders I
have approved beforehand.

If you would like to be added to my list of approved senders, please
fill out the short request form (see link below). Once I approve you,
I will receive your original message in my inbox. You do not need to
resend your message. I apologize for this one-time inconvenience.

Click the link below to fill out the request:

https://webmail1.earthlink.net/newaddme?a=ren...@ix.netcom.com&id=11ed-358e-8d00da44-9696-00144ff9a1a5

[Robert Engels]

Normally when people reply to the list it only comes from the list - and the list is allow listed. You are the first person in years to say anything about this. Sorry. I don’t think there is much I can do except change email addresses to one. Y a different provider.

> On Sep 16, 2022, at 3:01 AM, Jan Mercl <0xj...@gmail.com> wrote:
>
> To ren...@ix.netcom.com:

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAA40n-Wdncw3FsWFDvR5k8noaR5An0aab0qS%2BeQxT05B8Mx3UA%40mail.gmail.com.

[Jan Mercl]

On Fri, Sep 16, 2022 at 4:23 PM Robert Engels <ren...@ix.netcom.com> wrote:

> Normally when people reply to the list it only comes from the list - and the list is allow listed.

All people in this thread sent their replies to both you and the list.

> You are the first person in years to say anything about this. Sorry. I don’t think there is much I can do except change email addresses to one. Y a different provider.

Not that I understand what you mean, but the only thing I want is that
your mailbot stops sending me emails. And no, I don't want to register
my email anywhere to get white listed.

Thanks.

[Robert Engels]

I don’t know what is happening. The message you just sent came from you not the list. This is not how it is supposed to work. That is why there is a problem.

> On Sep 16, 2022, at 9:30 AM, Jan Mercl <0xj...@gmail.com> wrote:

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAA40n-WE4jwW8ZisK1Or1s2HEq-mcOeDcwAvTHy%3DmUcKtV849A%40mail.gmail.com.

[Robert Engels]

That’s a very interesting read - thank you.

Are you sure that holds when using an atomic as a memory barrier - seems it would be very difficult for the compiler to know when it was safe to do that- it would need to understand the atomic framing.

> On Sep 16, 2022, at 2:10 AM, Jan Mercl <0xj...@gmail.com> wrote:

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAA40n-X5cK9eZpCRAi-yRO9X6s287KF6Dy%3Ds%2BL2%2BqM2CjW2cqQ%40mail.gmail.com.

[Jan Mercl]

On Fri, Sep 16, 2022 at 4:42 PM Robert Engels <ren...@ix.netcom.com> wrote:

> I don’t know what is happening. The message you just sent came from you not the list. This is not how it is supposed to work. That is why there is a problem.

FYI, I'm posting to the list from the GMail web client. This time I
have manually removed you from the recipients list.

That's all I know, sorry.

[Robert Engels]

Even that message came from you. It seems to me the list server is not configured properly - as the message should come from the list with the list email - and usually your name.

Again, really sorry but no one has reported this to me before.

> On Sep 16, 2022, at 9:46 AM, Jan Mercl <0xj...@gmail.com> wrote:

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAA40n-XK%2BMuLk5PaVsck3yJe1-gZCu1c6T_odv%2BmF0HeFB7D%3Dw%40mail.gmail.com.

[Keith Randall]

Atomic operations can establish happens-before relationships.

initially, y==0.

goroutine 1:

x = 99

atomic.Store(&y, 1)

goroutine 2:

if atomic.Load(&y) == 1 {

   println(x)

}

This program, if it prints anything, must print 99.

"x=99" and "atomic.Store(&y,1)" are ordered by the sequenced-before relationship.

"atomic.Load(&y)" and "println(x)" are likewise.

"atomic.Store(&y,1)" and "atomic.Load(&y)" are ordered by the synchronized-before relationship (if the latter in fact reads the value written by the former).

So according to the memory model, the x=99 statement happens-before the println(x) statement.

We use this pattern all the time in the runtime.

That said, I'm not sure how much of atomic.* the race detector understands. It would not surprise me that the race detector still thinks that there is a race.

@ThomasB, I think you're contending that the sequenced-before relationship doesn't order "x=99" and "atomic.Store(&y,1)". I disagree, those should be ordered by the sequenced-before relationship. The sequenced-before relationship is *almost* a total order on the execution of a single goroutine. The only exceptions are weird cases like the order of evaluation of arguments of a function call, which has some wiggle room in the spec. Certainly different statements are totally ordrered in the order in which they appear in the program execution.

[thepud...@gmail.com]

Hello Robert,

I suspect the race detector does not have a bug in this case.

Or at least, I suspect the issue in this example from Robert from upthead is the for loop on the write:
     https://go.dev/play/p/aCCvvBIrGXq

     > Yea, the race detector is broken… it fails on the following code:
     >

     > [snip]

     >    go func() {
     >       for {
     >          x = 1
     >          atomic.StoreInt32(&y, 1)
     >       }
     >       w.Done()
     >    }()
     >    go func() {
     >       for {
     >          if atomic.LoadInt32(&y) == 1 {
     >             if x != 1 {
     >                panic("should not happen")
     >             }
     >          }
     >       }
     >       w.Done()
     >    }()

     > [snip]

I think the first time through the first for loop, the write on 'x' is OK, but subsequent loops can have an illegal data race between writing and reading 'x'.

In other words, I suspect this modified version with the first for loop removed is OK, with the atomics causing synchronization between 'x' even though 'x' itself is not accessed atomically:
   https://go.dev/play/p/_3S2bb61LPZ

That at least passes '-race' (including with an additional outer loop added to give the race detector more opportunities to observe a data race).

That said, maybe I have misunderstood.

Informally, from the Go Memory Model blog post (https://research.swtch.com/gomm):

    > The atomics in particular should be documented to provide sequentially consistent behavior

    > that creates happens-before edges synchronizing the non-atomic code around them. 

    > This would match the default atomics provided by all other modern systems languages.

More formally, from sync/atomics doc:

    > In the terminology of the Go memory model, if the effect of an atomic operation A is observed 

    > by atomic operation B, then A “synchronizes before” B. Additionally, all the atomic operations 

    > executed in a program behave as though executed in some sequentially consistent order. [...]

and from Go Memory Model:

    > Requirement 1: The memory operations in each goroutine must correspond to a correct sequential 

    > execution of that goroutine, given the values read from and written to memory. That execution 

    > must be consistent with the "sequenced before" relation [...]
    >
    > Requirement 2: [...] The "synchronized before" relation is a partial order on synchronizing memory 

    > operations, derived from W. If a synchronizing read-like memory operation r observes a synchronizing 

    > write-like memory operation w (that is, if W(r) = w), then w is synchronized before r. [...]

    >
    > The "happens before" relation is defined as the transitive closure of the union of the sequenced 

    > before and synchronized before relations

Sorry if any of this is off base.

Best regards,

thepudds

[robert engels]

The loop is indeed the problem. The race detector cannot “see” that the value being set is constant - it only sees it as a subsequent write.

Thomas was correct that there is a data race. He was incorrect that the happens-before semantics of atomics don’t apply to non-atomic variables.

As I pointed out in the later example, you can use atomics to implement a spin lock on non-atomic variables and it works as expected and the race detector does not report an issue.

The paper cited on ‘benign data races” has some suspect contentions in regards to atomic usage (ie the compiler re-using a memory address as temp storage, because the writer could be changed to look like:

func code() {

     x := 1

     ready: = false

    go {

        a loop:

            … do stuff…

            if ready == false {

                  x = 2 // only mutate of x

                  atomic set ready = true

            }

        }

    go {

       atomic wait for ready == true 

       do something with x repeated…

    }

}

The above code would still be subject to a data race because the contention is that the compiler could use the storage for x in ‘do stuff’ which would corrupt the x value even though it was only written once. So I believe the compiler needs to avoid re-ordering and re-using storage in any methods where atomics are involved - but I don’t know this for certain. i.e. The compiler cannot know that x is the ‘exported’ value and so in the presence of atomics it cannot perform that optimization, and if can determine it - it wouldn’t “re use” it - so in either case it is safe.

   

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/20cacd2f-0fd7-43b5-ba65-74e9ba008569n%40googlegroups.com.

[stephen.t....@gmail.com]

On Thursday, September 15, 2022 at 4:01:59 AM UTC+1 ren...@ix.netcom.com wrote:

 

Is there any way to perform “lazy loads” in Go?

Not that I can see but it would be very useful in the case of a GUI running in a separate goroutine to the one creating/updating data being displayed by the GUI. It can be orchestrated with the sync primitives of course, but the developer could go in a few different directions and it's not clear to me which is the best option.

[Konstantin Khomoutov]

On Fri, Sep 16, 2022 at 01:26:05PM -0700, 'Keith Randall' via golang-nuts wrote:

> Atomic operations can establish happens-before relationships.
>
> initially, y==0.
>
> goroutine 1:
> x = 99
> atomic.Store(&y, 1)
>
> goroutine 2:
> if atomic.Load(&y) == 1 {
> println(x)
> }
>
> This program, if it prints anything, must print 99.
> "x=99" and "atomic.Store(&y,1)" are ordered by the sequenced-before
> relationship.
> "atomic.Load(&y)" and "println(x)" are likewise.
> "atomic.Store(&y,1)" and "atomic.Load(&y)" are ordered by the
> synchronized-before relationship (if the latter in fact reads the value
> written by the former).
> So according to the memory model, the x=99 statement happens-before the
> println(x) statement.
>
> We use this pattern all the time in the runtime.

But the subtle connection between the x and y in the goroutine 1 is not in any
way explicit, and as I understand, to provide the guarantees you've presented,
the compiler must consider any atomic write-like operation in the code of a
function to be a point at which all earlier stores must have their effects be
observable at any point in a running program which performs an atomic
read-like operation from the same memory location, right?
But I doubt there is any way for such fine-grained approach so basically any
atomic store must work like a full memory fence then.

Another question: if your code were read

| initially, y==0.
|
| goroutine 1:
| x = 42

| x = 99
| atomic.Store(&y, 1)

| x = 100

|
| goroutine 2:
| if atomic.Load(&y) == 1 {
| println(x)
| }

then would println have been guaranteed to still print 99 and not 100?

In other words, do I understand correctly that an atomic write-like operation
guarantees that all stores made by a goroutine before that operation must be
visible, but stores made afterwards may be visible or not visible?
Or, to put it differently, if the read from y in goroutine 2 sees a 1,
there's no way println(x) prints 42, but it may print 99 or 100.
..and also if x is not naturally aligned (I think it's impossible in Go) or
not "hardware atomic" - like a 64-bit integer on a 32-bit platform, -
println(x) could produce some weird value as a result of the torn write/torn
read which may happen when one core updates the x while another one reads it,
right?

[...]

[Ian Lance Taylor]

On Sun, Oct 30, 2022 at 10:16 AM Konstantin Khomoutov <kos...@bswap.ru> wrote:
>
> | initially, y==0.
> |
> | goroutine 1:
> | x = 42
> | x = 99
> | atomic.Store(&y, 1)
> | x = 100
> |
> | goroutine 2:
> | if atomic.Load(&y) == 1 {
> | println(x)
> | }
>
> then would println have been guaranteed to still print 99 and not 100?

No. (But it definitely wouldn't print 42.)

> In other words, do I understand correctly that an atomic write-like operation
> guarantees that all stores made by a goroutine before that operation must be
> visible, but stores made afterwards may be visible or not visible?

Yes.

> Or, to put it differently, if the read from y in goroutine 2 sees a 1,
> there's no way println(x) prints 42, but it may print 99 or 100.

Yes.

> ..and also if x is not naturally aligned (I think it's impossible in Go) or
> not "hardware atomic" - like a 64-bit integer on a 32-bit platform, -
> println(x) could produce some weird value as a result of the torn write/torn
> read which may happen when one core updates the x while another one reads it,
> right?

Well, this case is not supported in general, but I think the answer is yes.

Ian