[go] runtime: fix coro interactions with thread-locked goroutines

0 views

Skip to first unread message

Michael Knyszek (Gerrit)

unread,

May 17, 2024, 3:46:16 PMMay 17

to goph...@pubsubhelper.golang.org, golang-...@googlegroups.com, Austin Clements, Go LUCI, David Chase

Michael Knyszek submitted the change

Change information

Commit message:

runtime: fix coro interactions with thread-locked goroutines

This change fixes problems with thread-locked goroutines using
newcoro/coroswitch/etc. Currently, the coro paths do not consider
thread-locked goroutines at all and can quickly result in broken
scheduler state or lost/leaked goroutines.

One possible fix to these issues is to fall back on goroutine+channel
semantics, but that turns out to be fairly complicated to implement and
results in significant performance cliffs. More complex thread-lock
state donation tricks also result in some fairly complicated state
tracking that doesn't seem worth it given the use-cases of iter.Pull
(and even then, there will be performance cliffs).

This change implements a much simpler, but more restrictive semantics.
In particular, thread-lock state is tied to the coro at the first call
to newcoro (i.e. iter.Pull). From then on, the invariant is that if the
coro has any thread-lock state *or* a goroutine calling into coroswitch
has any thread-lock state, that the full gamut of thread-lock state must
remain the same as it was when newcoro was called (the full gamut
meaning internal and external lock counts as well as the identity of the
thread that was locked to).

This semantics allows the common cases to be always fast, but comes with
a non-orthogonality caveat. Specifically, when iter.Pull is used in
conjunction with thread-locked goroutines, complex cases (passing next
between goroutines or passing yield between goroutines) are likely to
fail. Simple cases, where any number of iter.Pull iterators are used in
a straightforward way (nested, in series, etc.) from the same
goroutine, will work and will be guaranteed to be fast regardless of
thread-lock state.

This is a compromise for the near-term and we may consider lifting the
restrictions imposed by this CL in the future.

Fixes #65889.
Fixes #65946.