Are there any atomic data types in Go?

[enormouspenguin]

Hi

By "atomic data types" I means that variables of those types could be read/written atomically without the need of importing sync/atomic package. Also, how to atomically read/write a variable of type channel?

[Jesse McNelis]

On 25 Feb 2015 18:09, "enormouspenguin" <kimkh...@gmail.com> wrote:
>
> Hi
>
> By "atomic data types" I means that variables of those types could be read/written atomically without the need of importing sync/atomic package. Also, how to atomically read/write a variable of type channel?
>

There are none.
All data types require additional syncronisation.

Use a lock to safely write to a variable of any type.

[enormouspenguin]

Do you know anyway I could atomically compare-and-swap a simple bool variable without the need of Lock or Channel because those are apparently heavy for my simple need?

[Dave Cheney]

Have you looked at the sync/atomic package ?

[enormouspenguin]

On Wednesday, February 25, 2015 at 3:33:36 PM UTC+7, Dave Cheney wrote:

Have you looked at the sync/atomic package ?

I searched but couldn't find a function to operate atomically on a bool variable. They mostly are for integer and pointer types, except the Value type (which looks like it could be applied to my case). My simplified case: is http://play.golang.org/p/4nETrNnSrY. Advises are desperately needed.

[Dmitry Vyukov]

Use uint32 with 0/1 values as the flag .

[Ian Davis]

You could just use an int as 1 or 0 to represent your bool and use atomic load/store functions from sync/atomic.

 

However that won't stop your code from being racy. The way you have written your function you check a boolean value called closed and then do some work if close was false.  However, even if you make an atomic read of the closed variable, another goroutine could also read it just before or after and then you would have two goroutines that think closed is false and both will perform the work.

 

You should use double checked locking like this:

 

http://play.golang.org/p/9xVlP6bT0X

 

Ian

[thwd]

That's a bit scary.

Just share a sync.Once instead of a bool:

http://play.golang.org/p/BuwPUENPq8

[enormouspenguin]

I see lots of advices recommending me to take the shorter (and safer) Mutex/Channel road. I am aware of the risk involve in using atomic operations but still I want to know what went wrong with this simple code: http://play.golang.org/p/4EwclgZT6f. Which is a patch to my previous code, using uint32 plus atomic compare-and-swap instead of naive bool (as suggested by Dmitry). It's so simple that my brain just refuse to see any flaw in it (though my heart tells me the opposite).

[Ian Davis]

 

That would work too provided closed isn't ever set back to false elsewhere in a more realistic program

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

 

[enormouspenguin]

On Thursday, February 26, 2015 at 12:36:31 AM UTC+7, Ian Davis wrote:

 

That would work too provided closed isn't ever set back to false elsewhere in a more realistic program

No, closed will never be set back to false anywhere else. The case I presented is an simplified excerpt of a real program I am working on.

[Sanjay]

Seems ok... but if you use sync.Once, then you have the further guarantee that after any call to close() returns, that it has completed execution exactly once. That is to say, unlike your code, on the second close, it won't just early return. It will wait until the first close() completes.

Sanjay

[enormouspenguin]

On Thursday, February 26, 2015 at 4:22:45 AM UTC+7, Sanjay wrote:

Seems ok... but if you use sync.Once, then you have the further guarantee that after any call to close() returns, that it has completed execution exactly once. That is to say, unlike your code, on the second close, it won't just early return. It will wait until the first close() completes.

sync.Once is extremely useful, but in my case, other concurrent calls to close() don't have to wait for the one that execute the clean up to finish. It just have to get out as soon as it detect that another goroutine is doing (or already done) the clean up. So I think it'll be more efficient to drop the mutex entirely and just rely on atomic.

But I still have one concern needed confirmation from pro Gophers.  In The Go Memory Model, I couldn't see any rule that prevent compiler (or CPU) from executing out of order and hoist any line of code inside the if out and above it. Thus making it all wrong. In fact sync/atomic package is barely mentioned once in the Advice section with no further explanation or rules declaration.

[Dave Cheney]

Could you describe your concern in code, not words ?

[enormouspenguin]

On Thursday, February 26, 2015 at 11:45:31 AM UTC+7, Dave Cheney wrote:

Could you describe your concern in code, not words ?

Yes, of course.

Here is the ideal code that should work if no reordering happen:

package main

import (

"fmt"

"sync/atomic"

)

var closed uint32

func close() { //Could be concurrently called by many goroutines

if atomic.CompareAndSwapUint32(&closed, 0, 1) {

//Do some cleanup

fmt.Println("Closed successfully!") //If implement correctly,

//this line should only be printed once

}

}

func main() {

go close()

close()

}

Here is the code that I imagined would be reordered:

package main

import (

"fmt"

"sync/atomic"

)

var closed uint32

func close() { //Could be concurrently called by many goroutines

//(*)

//Do some cleanup

fmt.Println("Closed successfully!") //If implement correctly,

//this line should only be printed once

//end of (*)

if atomic.CompareAndSwapUint32(&closed, 0, 1) {

//(*) should be here, but instead it's hoisted above by compiler or CPU (just my theory)

}

}

func main() {

go close()

close()

}

My concern is whether or not compiler (or CPU) is allowed to do that under the hood?

[Chris Hines]

I think the first two sentences in the "Happens Before" section answer your question:

Within a single goroutine, reads and writes must behave as if they executed in the order specified by the program. That is, compilers and processors may reorder the reads and writes executed within a single goroutine only when the reordering does not change the behavior within that goroutine as defined by the language specification.

 

[enormouspenguin]

On Thursday, February 26, 2015 at 12:35:51 PM UTC+7, Chris Hines wrote:

I think the first two sentences in the "Happens Before" section answer your question:

Within a single goroutine, reads and writes must behave as if they executed in the order specified by the program. That is, compilers and processors may reorder the reads and writes executed within a single goroutine only when the reordering does not change the behavior within that goroutine as defined by the language specification.

I totally missed that. Thank you for pointing it out.

Just curious, what is "the behavior [...] as defined by the language specification." in my case?

[Dave Cheney]

As Chris said

atomic.CompareAndSwapUint32(&closed, 0, 1)

is a memory fence. The compiler won't reorder the body of the condition above the condition itself, that would be very broken. The cpu won't reorder the instructions either because it is not permitted to reorder them before the memory fencing operation, hence the atomic.CAS serialises all goroutines who enter that block.

[enormouspenguin]

Do you mean just by calling atomic.CompareAndSwapUint32(&closed, 0, 1), I could actually issue a memory fence (barrier) instruction to CPU (even without the present of IF (branching) statement), therefore preventing reads/writes from reordering around it?

It make me feel like the CAS function is a hidden way (that we don't need to know) to indirectly access low level barrier instructions. It's just my opinion. I don't plan to exploit it (even if it's true) any time soon (or even at all) because it's too terrifyingly low level for me to even talk about it.

[Benjamin Measures]

On Thursday, 26 February 2015 16:05:30 UTC, enormouspenguin wrote:

Do you mean just by calling atomic.CompareAndSwapUint32(&closed, 0, 1), I could actually issue a memory fence (barrier) instruction to CPU (even without the present of IF (branching) statement), therefore preventing reads/writes from reordering around it?

On my reading it's just an implicit LOCK, which is different to an MFENCE. There's only a total ordering on other LOCKed instructions.

This makes sense, as atomic is only concerned with protected memory locations.

[John Souvestre]

Ø  Do you mean just by calling atomic.CompareAndSwapUint32(&closed, 0, 1), I could actually issue a memory fence (barrier) instruction to CPU (even without the present of IF (branching) statement), therefore preventing reads/writes from reordering around it?

 

It is my understanding that this is so, and it is so for all of the primitives in sync/atomic.  Check this thread from a couple of weeks ago:

 

    Compiler level memory barrier in Go i.e. asm volatile("" : : : "memory")

 

John

    John Souvestre - New Orleans LA

 

 

[atd...@gmail.com]

Instead of using a CAS, I would just use atomic Loads and Stores in your case. You can be sure that the Load and Store will remain in program order. The ordering of any other non atomic instructions between these two statements is nothing that you can be sure of.

[enormouspenguin]

I don't think it is sufficient to just use separate Load and Store in my case. What if 2 or more goroutines all receive the same value before one of those have a chance to change it? My constrain is that the code inside the IF must run only once, no matter how many times the function was called and how many goroutines the function was concurrently called from. If that scenario happen, the code would be called multiple times, hence violating the constrain. Am I missing something here?

[thwd]

On Friday, February 27, 2015 at 3:09:08 PM UTC+1, enormouspenguin wrote:

My constrain is that the code inside the IF must run only once, no matter how many times the function was called and how many goroutines the function was concurrently called from. 

sync.Once was designed with exactly this use-case in mind.

If you don't want close() to block all callers of (*sync.Once).Do(), then spawn a goroutine inside close() and return immediately.

Your overhead then becomes essentially 2 pushes and one pop. If that's too much then Go might not be the language for the job. 

- Tom

[atd...@gmail.com]

On Friday, February 27, 2015 at 2:09:08 PM UTC, enormouspenguin wrote:

I don't think it is sufficient to just use separate Load and Store in my case. What if 2 or more goroutines all receive the same value before one of those have a chance to change it? My constrain is that the code inside the IF must run only once, no matter how many times the function was called and how many goroutines the function was concurrently called from. If that scenario happen, the code would be called multiple times, hence violating the constrain. Am I missing something here?

Indeed, I missed that part, sorry.

The ordering issues are on memory operations so what you have done looks okay to me, but don't take my word for it, especially if it is a simplistic example.

Otherwise, atomic operations can indeed be reordered with plain loads/stores but it is a platform specific behaviour. In some cases, you might need to use additional fences.

I guess, unless you are really trying to squeeze some perf out, coarser grained synchronization will be easier to reason about. And even if you were to use a mere CAS, I would profile because it is not sure that it will always translate in that big of a win in your real-world case.