Golang 1.19+ preemption model on a preemptive linux kernel?

[Andrew Athan]

I'm having trouble finding definitive information about golang's preemption model when running with GOMAXPROCS>1 on a multi-core Intel with preemtive linux kernel.

As I understand it, in such a scenario, two goroutines may be scheduled by the go runtime onto two separate system threads on two separate CPUs.

Isn't it true then, that irrespective of whether the two goroutines are scheduled onto separate CPUs, the OS may preemptively interrupt either of those goroutines to let the other run? Also, that if they are indeed scheduled onto separate CPUs, that the memory accesses made by those goroutines are interleaved even if no individual goroutine is "preempted"?

I'm asking because in reviewing the code of some "lock free" concurrent data structures written in go, it appears the authors have made certain assumptions about race conditions (e.g.,  the code of on goroutine assumes that another goroutine won't perform a load/store between two function calls the first goroutine makes to check a boolean and store a value).

Given that goroutines are mappes onto system threads, and that in a preemptive kernel those threads may be preempted and/or irrespective of the preemptive kernel issue may run on separate cores, no assumptions should be made about the "atomicity" of multiple statements in a goroutine relative to other goroutines. Right?

Where can I get the most current and accurate information about this topic?

A.

[Ian Lance Taylor]

I'm not aware of any detailed documentation of the issues you are
looking at. So the most current and accurate information is the
source code. Or this mailing list.

You are correct that non-atomic memory operations made by different
goroutines can be freely interleaved. If there is any code in the
runtime that assumes otherwise, that code is badly broken. If you can
point to the specific code that you are concerned about, perhaps we
can allay your concerns, or perhaps we can fix the bug. Thanks.

Ian

[Robert Engels]

I have some examples of lock-free structures at github.com/robaho/go-concurrency-test

Most lock-free structures should not depend on OS scheduling. Some that use “cpu id “ to partition would. 

On Dec 5, 2022, at 12:02 PM, Andrew Athan <andr...@gmail.com> wrote:

I'm having trouble finding definitive information about golang's preemption model when running with GOMAXPROCS>1 on a multi-core Intel with preemtive linux kernel.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/0912663b-d4cb-41ee-a335-466c74b00d8cn%40googlegroups.com.

[Andrew Athan]

The statement I made is actually a bit stronger than the one you confirmed: Even if the goroutines are using atomic load/store, the preemtive nature of the os threads could easily interrupt one goroutine between an atomic load and its use of that value, giving another goroutine the opportunity to change the stored value at the atomic load location, potentially leading to a data race -- particularly when the loader is storing to a related but different location and/or without a CAS on the original loaded value.

To answer your question, no ... within the go runtime, I DO NOT claim to have found instances of that kind of load/check/store as separate operations within a function such that those operations are assumed to run atomically.

However, I've run into a couple of such instances in public repos. Thus, I wanted to confirm my thinking. For example, I initiated a discussion about same here:

https://github.com/alphadose/haxmap/issues/26

(I ran across haxmap while reviewing issues in the golang repo relative to sync.Map)

[Andrew Athan]

In brief then, in the presence of os threads running on a kernel that can preempt threads, it is incorrect to assume that any two statements within a goroutine will run atomically (both run, or not at all), irrespective of the simplicity of those statements, or whether any function calls involved are inlined, etc...

[robert engels]

I don’t think the analysis by alphadose in the issue you cite is correct. Go added async pre-emption (via signals) and I believe it can preempt a Go thread anywhere but in an internal critical section. Ian would know for sure. It is certainly not only at a function call boundary because that would cause the “infinite spin loop blocking GC” issue. Once a routine is preempted any available routine could run on the same processor/OS thread.

-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/0b44c5d7-7c45-4392-9046-eb56f8830a52n%40googlegroups.com.

[drc...@google.com]

From the POV of not-runtime-code, preemption can happen anywhere.  Certainly with GOMAXPROCS > 1, there is OS preemption, and Go's default goroutine preemption is now potentially preemptive in most parts of non-runtime functions.