Suggestion: Goroutine thread affinity

[Jsor]

I'm sure this has been suggested before, but Goroutine thread affinity is a feature that, while not strictly necessary, I think could improve the language. There are some C libraries that get very... cross when you try to access them from multiple threads. OpenGL, for instance, wants you to manage a single context per thread.

Now don't get me wrong, you can *manage* this now, but it can get rather kludgy. You have to lock the thread, create the context, never return from that goroutine or unlock the thread, and make absolutely sure that any goroutines you spawn from that goroutine don't touch the context and instead inform the main goroutine about what to change. Again, manageable (and because of channels, I'd argue it can be in some ways even easier than in C), but it can be difficult to work with if you really want to make use of concurrency.

My idea is an optional argument to the "go" keyword

go func, uint

If the second argument is passed in, the runtime will use it as an "id". The effects of doing this would be:

• On the first use of a given ID, the runtime will select a thread. That thread is the only thread goroutines with that affinity ID may be run on. That is, if I call "go Myfunc(), 1", then any other calls of the form "go <func>, 1" will be guaranteed to run on the same thread Myfunc() was run on -- regardless of whether or not Myfunc() has returned yet.
• However, goroutines with that affinity do not OWN (or "lock") said thread. Other goroutines may be run on that thread as well. Goroutines with a certain thread affinity may get precedence, though.
• If multiple IDs are in use, the runtime will attempt to spread them evenly among available threads. That is, for GOMAXPROCS=3, the first three IDs will be assigned to separate threads. If you add three more IDs, then each thread will have 2 IDs assigned to it, and so on.

If no ID is given (or an ID of 0?) then that goroutine may run on any thread, acting according to the current rules.

Some problems:

What if a Goroutine locks a thread that other goroutines have an affinity towards?

In my opinion, this simply means that no other goroutine with an affinity bound to that thread may run until the thread is unlocked or the goroutine returns. This does cause a potential trap, though: say you have two goroutines with different affinities, but bound to the same thread (because of the value of GOMAXPROCS and sheer bad luck). The user may erroneously assume that since they have different affinities, they must be on separate threads and thus it's okay to communicate via unbuffered channel between them. This will cause a deadlock. It is a problem, but in my opinion it's not too bad of a side effect (after all, I think you can currently make this happen if you use LockOSThread with GOMAXPROCS=1, and it's mostly just an extension of that problem). Explicit affinity would be sort of an "advanced feature" to begin with, so I think expecting users to be aware of these pitfalls is fair.

 

It could potentially be solved with some logic in the runtime that abdicates a locked thread to other goroutines with a different affinity ID if it's waiting on a channel, but that may be undesirable for various reasons.

What if you increase GOMAXPROCS after having used multiple affinity IDs? Will they get redistributed among the new threads?

No. Maybe I'm just colored by the problem I'm having, but it would be too dangerous to assume that any given ID can be sent to a different thread without consequences. However, further created IDs will be put on the new threads until they're even with the old ones.

What if you decrease GOMAXPROCS to an amount that would force the IDs to redistribute?

In my opinion, this should be a panic -- or at the very least the runtime should refuse to do it. Perhaps a function in the runtime package (runtime.RemoveAffinity(id uint) or something) could be available if the user is dead set on doing this.

 

What if your goroutine only needs to be on a certain thread sometimes but not others?

In my opinion, if you're trying to do that you're probably being overly fancy, and there are any number of workarounds that could suffice. But if this is deemed a "necessary" feature perhaps a function along the lines of runtime.SetCurrentGoroutineAffinity(id uint) could be made available.

I think this would also allow for people who really know what they're doing to request explicit parallelism (though extra features may be needed to really ensure that two different affinity IDs are not on the same thread as opposed to simply ensuring ones with the same ID do). Maybe in the future when "the scheduler improves" and GOMAXPROCS is removed the number of explicit thread affinities in the program can be used as a hint to runtime for how many threads to use.

The thing I like about the idea is that unless you're playing around with LockOSThread or GOMAXPROCS it's very conceptually simple to the user. Need all this stuff on the same thread for some reason? Make sure you put the same number when you call go. That's it. No messing around with LockOSThread and kludging your program to never exit that GoRoutine lest the next function call be assigned to a different thread. Hell, you don't even have to worry about what GOMAXPROCS is -- it will make sure every go call with the ID argument "1" runs on the same thread whether you have one thread or a billion running.

Thoughts? I know that if this does happen it's a very long time away since it would mess with the fundamental way the language works. But I figured bringing it up for discussion wouldn't hurt. Any big problems with it I didn't anticipate?

[Ian Lance Taylor]

On Wed, May 29, 2013 at 9:24 PM, Jsor <jrago...@gmail.com> wrote:
> I'm sure this has been suggested before, but Goroutine thread affinity is a
> feature that, while not strictly necessary, I think could improve the
> language. There are some C libraries that get very... cross when you try to
> access them from multiple threads. OpenGL, for instance, wants you to manage
> a single context per thread.

It's unfortunate that OpenGL has that restriction, but I don't think
we should change the language for that. We provide the tools needed
to work with OpenGL. Perhaps we can improve those tools. But not by
changing the language. In general I think that thread affinity should
be left to the runtime. Ideally the runtime can do something
reasonable in all cases. Where it can't, there are functions like
LockOSThread, as you mentioned.

Ian

[Peter Kleiweg]

Op donderdag 30 mei 2013 06:24:33 UTC+2 schreef Jsor het volgende:

If the second argument is passed in, the runtime will use it as an "id". The effects of doing this would be:

• On the first use of a given ID, the runtime will select a thread. That thread is the only thread goroutines with that affinity ID may be run on. That is, if I call "go Myfunc(), 1", then any other calls of the form "go <func>, 1" will be guaranteed to run on the same thread Myfunc() was run on -- regardless of whether or not Myfunc() has returned yet.
• However, goroutines with that affinity do not OWN (or "lock") said thread. Other goroutines may be run on that thread as well. Goroutines with a certain thread affinity may get precedence, though.
• If multiple IDs are in use, the runtime will attempt to spread them evenly among available threads. That is, for GOMAXPROCS=3, the first three IDs will be assigned to separate threads. If you add three more IDs, then each thread will have 2 IDs assigned to it, and so on.

GOMAXPROCS determines the number of processors used, not the number of threads. You can have many more threads.

 

Thoughts? I know that if this does happen it's a very long time away since it would mess with the fundamental way the language works. But I figured bringing it up for discussion wouldn't hurt. Any big problems with it I didn't anticipate?

What happens if you have locked a goroutine to a thread (with LockOSThread), and start a new goroutine from that thread? Is it in the same thread or in another? If the latter, then all you'd need is a variant function that locks the current goroutine, and all its decending goroutines, to the current thread.

[Rémy Oudompheng]

On 2013/5/30 Jsor <jrago...@gmail.com> wrote:
> I'm sure this has been suggested before, but Goroutine thread affinity is a
> feature that, while not strictly necessary, I think could improve the
> language. There are some C libraries that get very... cross when you try to
> access them from multiple threads. OpenGL, for instance, wants you to manage
> a single context per thread.
>
> Now don't get me wrong, you can *manage* this now, but it can get rather
> kludgy. You have to lock the thread, create the context, never return from
> that goroutine or unlock the thread, and make absolutely sure that any
> goroutines you spawn from that goroutine don't touch the context and instead
> inform the main goroutine about what to change. Again, manageable (and
> because of channels, I'd argue it can be in some ways even easier than in
> C), but it can be difficult to work with if you really want to make use of
> concurrency.
>
> My idea is an optional argument to the "go" keyword
>
> go func, uint
>
> If the second argument is passed in, the runtime will use it as an "id". The
> effects of doing this would be:
>
> On the first use of a given ID, the runtime will select a thread. That
> thread is the only thread goroutines with that affinity ID may be run on.
> That is, if I call "go Myfunc(), 1", then any other calls of the form "go
> <func>, 1" will be guaranteed to run on the same thread Myfunc() was run on
> -- regardless of whether or not Myfunc() has returned yet.

> [...]

>
> Thoughts? I know that if this does happen it's a very long time away since
> it would mess with the fundamental way the language works. But I figured
> bringing it up for discussion wouldn't hurt. Any big problems with it I
> didn't anticipate?

Goroutines are the basic building blck of Go concurrency. It doesn't
make sense for the Go language specification to mention threads,
either as larger building blocks (contrary to Go's spirit) or as an
optimization or system-specific constraint (implementation detail).
So what you say is not possible IMO.

Rémy.

[John Nagle]

On 5/30/2013 11:14 AM, R�my Oudompheng wrote:
> Goroutines are the basic building blck of Go concurrency. It doesn't
> make sense for the Go language specification to mention threads,
> either as larger building blocks (contrary to Go's spirit) or as an
> optimization or system-specific constraint (implementation detail).
> So what you say is not possible IMO.

Not quite. Goroutines are not preempted. Threads are.
If you have as many compute-bound goroutines as you have threads,
all other activity will be starved out.

This can come up in practice if you have a service that
does some long compute-bound tasks on request.

John Nagle

[Rémy Oudompheng]

On 2013/5/30 John Nagle <na...@animats.com> wrote:

The specification does not say that goroutines are not preempted. The
specification allows a program with concurrent CPU-intensive
goroutines to behave sanely. I'm not sure what you wanted to say.

Rémy.

[John Nagle]

On 5/30/2013 12:58 PM, R�my Oudompheng wrote:
> On 2013/5/30 John Nagle <na...@animats.com> wrote:

The specification doesn't say that programs are run "sanely".
The specification makes no time guarantees. Long stalls are
permitted by the spec, as long as the program completes
eventually.

The existing Go implementations do not preempt goroutines
within a thread. Control transfer occurs only at defined
points, such as locks or system calls. So starvation is
possible.

There's an argument for the Go scheduler always
keeping a spare thread around for short-term requests. If
no running goroutine has given up control in the last N
milliseconds, fork off a new thread, rather than waiting for
the compute-bound ones to finish. One thread per CPU
isn't a good rule if there are compute-bound tasks.

John Nagle

[Ian Lance Taylor]

On Thu, May 30, 2013 at 2:44 PM, John Nagle <na...@animats.com> wrote:
>
> The existing Go implementations do not preempt goroutines
> within a thread. Control transfer occurs only at defined
> points, such as locks or system calls. So starvation is
> possible.

FYI, there is a plan to address this issue. Whether this plan will be
fully implemented has not yet been decided.

https://groups.google.com/d/msg/golang-dev/vtrWpvf8nMA/PPs5Eqpk-VgJ

Ian

[John Nagle]

I kind of liked it without preemption. I'd suggest forking
off more threads on an as-needed basis rather than trying to be an OS.
Once you can preempt, you have to decide who preempts whom. That means
a scheduling policy, quantums, priorities - the usual stuff of an OS's
scheduler. Why repeat that work.

(Split-stack growth nostalgia: I wrote one of those in the
mid-1970s, adding multiple thread support to a Pascal compiler for
UNIVAC mainframes.)

John Nagle

[Jsor]

GOMAXPROCS determines the number of processors used, not the number of threads. You can have many more threads.

Ah, I guess I misunderstood how that works, sorry. Still, the basic idea of thread affinity isn't affected, just the specific implementation.

 

What happens if you have locked a goroutine to a thread (with LockOSThread), and start a new goroutine from that thread? Is it in the same thread or in another? If the latter, then all you'd need is a variant function that locks the current goroutine, and all its decending goroutines, to the current thread.

That could work. It's still not super clean (e.g. if you want the initial goroutine to be able to return), but it allows for concurrency in goroutines that need to be thread-sensitive.

Anyway, it's not going to happen. That's fine, it was just an idea. Personally, I like the idea of having a *little* bit of control over threading, and I don't think it undermines the language goals too much, but I suppose I'm outvoted. Though I do have a question: if I have a persistent goroutine that locks a thread and waits for input (e.g. a channel read) -- is it guaranteed that the program won't deadlock for that reason alone? It took me far less time to fix my rendering goroutine to lock to one thread than I thought, but I'm worried that on certain systems it may deadlock with me being unable to do anything about it simply because of the use of LockOSThread and waiting for channel input.

[Ian Lance Taylor]

On Thu, May 30, 2013 at 7:07 PM, Jsor <jrago...@gmail.com> wrote:

> Though I do have a question: if I have a persistent
> goroutine that locks a thread and waits for input (e.g. a channel read) --
> is it guaranteed that the program won't deadlock for that reason alone?

Yes.

Ian

[Dmitry Vyukov]

+1 

[Dmitry Vyukov]

On Fri, May 31, 2013 at 2:01 AM, John Nagle <na...@animats.com> wrote:

On 5/30/2013 2:51 PM, Ian Lance Taylor wrote:
> On Thu, May 30, 2013 at 2:44 PM, John Nagle <na...@animats.com> wrote:
>>
>>    The existing Go implementations do not preempt goroutines
>> within a thread.  Control transfer occurs only at defined
>> points, such as locks or system calls.  So starvation is
>> possible.
>
> FYI, there is a plan to address this issue.  Whether this plan will be
> fully implemented has not yet been decided.
>
> https://groups.google.com/d/msg/golang-dev/vtrWpvf8nMA/PPs5Eqpk-VgJ
>
> Ian

   I kind of liked it without preemption.  I'd suggest forking
off more threads on an as-needed basis rather than trying to be an OS.
Once you can preempt, you have to decide who preempts whom.  That means
a scheduling policy, quantums, priorities - the usual stuff of an OS's
scheduler.  Why repeat that work.

I does not work that way. We already retake control over scheduling from OS, it's mechanisms won't work anymore.

OS will preempt at wrong points, e.g. a thread that runs short running goroutines does not need to be preempted to run other Go runtime threads.

We will need to spawn potentially unbounded number of threads to guarantee fairness.

We will need to constantly park and unpark threads.

OS thread-processor affinity does not work for us as well, OS tries to provide affinity of threads to processors, but it's irrelevant for Go, because OS does not know what goroutines threads run.

OS NUMA affinity does not work either.

OS load balancing does not work either.

Once you've got into business of implementing your own scheduler, unfortunately you need to repeat all that work.