Go routine context

[Robert Engels]

Very interesting article came out recently. https://www.infoq.com/articles/java-virtual-threads/ and it has implications for the Go context discussion and the author makes a very good case as to why using the thread local to hold the context - rather than coloring every method in the chain is a better approach. If the “virtual thread aka Go routine” is extremely cheap to create you are far better off creating one per request than pooling - in fact pooling becomes an anti pattern. If you are creating one per request then the thread/routine becomes the context that is required. No need for a distinct Context to be passed to every method. 

[Ian Davis]

On Fri, 30 Sep 2022, at 3:32 PM, Robert Engels wrote:

Very interesting article came out recently. https://www.infoq.com/articles/java-virtual-threads/ and it has implications for the Go context discussion and the author makes a very good case as to why using the thread local to hold the context - rather than coloring every method in the chain is a better approach. If the “virtual thread aka Go routine” is extremely cheap to create you are far better off creating one per request than pooling - in fact pooling becomes an anti pattern. If you are creating one per request then the thread/routine becomes the context that is required. No need for a distinct Context to be passed to every method. 

I don't think it is usual to pool goroutines. Normal usage is to spin one up for each incoming request, which is the pattern used by Go's http server for example.

I skimmed the article but my knowledge of Java's thread local storage is limited. In Go it's common for requests to spawn subrequests in their own goroutines. With a context you would derive a child context, potentially with a shorter lifetime. What is the equivalent in thread local storage? Is there a way to access parent thread storage from a child (analogous to how a child context has access to all the values assigned to the parent)?

[Robert Engels]

Java has an InheritableThreadLocal for that very reason - that is automatically propagated and the child can subsequently override if needed. 

On Sep 30, 2022, at 9:43 AM, Ian Davis <m...@iandavis.com> wrote:



--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/b35520c7-87b2-4b83-be1b-2d2f4257d283%40app.fastmail.com.

[Ian Lance Taylor]

On Fri, Sep 30, 2022 at 7:32 AM Robert Engels <ren...@ix.netcom.com> wrote:
>
> Very interesting article came out recently. https://www.infoq.com/articles/java-virtual-threads/ and it has implications for the Go context discussion and the author makes a very good case as to why using the thread local to hold the context - rather than coloring every method in the chain is a better approach. If the “virtual thread aka Go routine” is extremely cheap to create you are far better off creating one per request than pooling - in fact pooling becomes an anti pattern. If you are creating one per request then the thread/routine becomes the context that is required. No need for a distinct Context to be passed to every method.

I didn't read the article (sorry).

In a network server a Go context is normally specific to, and shared
by, a group of goroutines acting on behalf of a single request. It is
also normal for a goroutine group to manage access to some resource,
in which case the context is passed in via a channel when invoking
some action on behalf of some request. Neither pattern is a natural
fit for a goroutine-local context.

Ian

[Rob Pike]

One of the critical decisions in Go was not defining names for goroutines. If we give threads/goroutines/coroutines (TGCs) names or other identifiable state, such as contexts, there arises a tendency to push everything into one TGC. We see what this causes with the graphics thread in most modern graphics libraries, especially when using a threading-capable language such as Go. You are restricted in what you can do on that thread, or you need to do some sort of bottlenecking dance to have the full language available and still honoring the requirements of a single graphics thread.

One way to see see what this means: Long ago, people talked of a "thread per request"  model, and honestly it was, or would have been, an improvement on standard practice at the time. But if you have cheap TGCs, there is no need to stop there: You can use multiple independently executing TGCs to handle a request, or share a TGC between requests for some part of the work (think database access, for example). You have the whole language available to you when programming a request, including the ability to use TGCs.

Like Ian, I have not read this paper, but I take it as a tenet that it is better to keep goroutines anonymous and state-free, and not to bind any particular calculation or data set to one thread of control as part of the programming model. If you want to do that, sure, go for it, but it's far too restrictive to demand it a priori and force it on others.

-rob

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAOyqgcWAfdc2Np2KA%2B2-U9Z5Hv7tdHGgJHWDTUg_6pbr%3D8jghg%40mail.gmail.com.

[Robert Engels]

I disagree. I think if you read the article you’ll understand why. You need to invert the context handling - the pattern you cite is exactly the pattern the author describes but when you create the routines on demand and the subordinate ones - the “thread” is the context and it removes the ugliness of coloring every function in the path with a context variable.

I think you’ll find the article interesting. It is certainly written by a CS “god” that knows what he’s talking about.

> On Sep 30, 2022, at 10:39 PM, Ian Lance Taylor <ia...@golang.org> wrote:

[Robert Engels]

Again, please read the paper. The arguments you make are refuted. The lack of routine context is a burden on the Go ecosystem and makes debugging highly concurrent Go systems far more difficult than similar systems in Java. 

On Sep 30, 2022, at 11:09 PM, Rob Pike <r...@golang.org> wrote:



[Robert Engels]

To clarify a point that not be obvious - all of the routines involved in handling the request are temporary and only live through the life of the request - which defacto makes their existence = context, so using a thread local is simply a shorthand for passing the context to every method invoked by the routine.

> On Sep 30, 2022, at 11:12 PM, Robert Engels <ren...@ix.netcom.com> wrote:
>
> I disagree. I think if you read the article you’ll understand why. You need to invert the context handling - the pattern you cite is exactly the pattern the author describes but when you create the routines on demand and the subordinate ones - the “thread” is the context and it removes the ugliness of coloring every function in the path with a context variable.

[Robert Engels]

Btw, the comments on the single threaded nature of gui systems is also not correct. Multiple attempts have been made to develop a concurrent gui - they have all failed miserably. A Google search on the subject will show lots of relevant research. Because a gui is interacted with by humans - not a machine - being single threaded maps more directly to the interaction model.

> On Sep 30, 2022, at 11:12 PM, Robert Engels <ren...@ix.netcom.com> wrote:
>

> I disagree. I think if you read the article you’ll understand why. You need to invert the context handling - the pattern you cite is exactly the pattern the author describes but when you create the routines on demand and the subordinate ones - the “thread” is the context and it removes the ugliness of coloring every function in the path with a context variable.

[Axel Wagner]

I've at least skimmed the article and I can't find any of the arguments you say are there.

For thread locals it says, if anything, that they should be avoided with virtual threads - at least for some uses (the ones that you'd use a sync.Pool for in Go). On coloring it only talks about the advantages of virtual threads over async/await, which, well most Gophers will agree with.

Apart from these, I can't find anything that I could reasonably connect to context.Context - the article seems almost exclusively an introduction to virtual threads and an explanation on how they differ from operating system threads. In particular, I don't see anything in this article which could address the arguments Ian mentioned.

It teases at more articles, about "Structured Concurrency" and "Extent local variables" - the latter sounds as if it *could* be what you talk about, but that article doesn't seem to exist yet.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAD38A60-EE4B-4A76-9F7B-66A9939874F5%40ix.netcom.com.

[Axel Wagner]

Oh, following the link about Structured Concurrency at the end brings you to https://openjdk.org/jeps/428

That *does* indeed seem to contain discussion about relevant topics. Perhaps that's the link you intended to post?

[Robert Engels]

I think the article I cited presents the argument. (I’m on mobile so a bit hamstrung). The section on thread locals doesn’t say to avoid them. It says to avoid them for caching - as the virtual thread should be temporary and not reused. Maybe the position is my own correlation (I’ll have to reread) but if the thread is temporary and only created to handle that request it is defacto the context for that request. I highly doubt I was smart enough to make that connection on my own :)

On Oct 1, 2022, at 12:50 AM, 'Axel Wagner' via golang-nuts <golan...@googlegroups.com> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAEkBMfEO-LVU_09itMwjdF-ciN_k9Dz_LthtWwgisMMz%2BQbMnw%40mail.gmail.com.

[Bakul Shah]

I did a quick read. Virtual threads seems much *closer* to goroutines and they suggest people "unlearn" the overuse of thread local storage (TLS)! The thread pooling they talk about is to avoid the much higher thread creation time -- not a problem in Go.

[Bakul Shah]

Quoting from this article: "Virtual threads (JEP 425) make it cost-effective to dedicate a thread to every such I/O operation, but managing the huge number of threads that can result remains a challenge."

My quick take: may be this is due to "old thread think" as virtual threads are so new in Java? Has managing large number of goroutines been a problem?

One use I can see for giving each thread a handle is for doing user level scheduling and in general treating them as first class objects. This may be a cost vs benefit issue. 

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAEkBMfEO-LVU_09itMwjdF-ciN_k9Dz_LthtWwgisMMz%2BQbMnw%40mail.gmail.com.

[Axel Wagner]

On Sat, Oct 1, 2022 at 8:06 AM Robert Engels <ren...@ix.netcom.com> wrote:

I think the article I cited presents the argument. (I’m on mobile so a bit hamstrung). The section on thread locals doesn’t say to avoid them. It says to avoid them for caching - as the virtual thread should be temporary and not reused.

Yes, that' why I said "if anything" - it certainly doesn't say "use them more", it says "stop using them for $things" (where $things is what we use sync.Pool for).

 

Maybe the position is my own correlation (I’ll have to reread) but if the thread is temporary and only created to handle that request it is defacto the context for that request.

I don't believe this is necessarily true. Note that `net/http` does this, but it still requires an explicit context for cancellation etc.

I've now read JEP 428 (Structured Concurrency) as well, which *is* a lot closer to talk about context.Context, in that it actually does address cancellation. In effect, it seems to propose a programming API that is very close to Go's errgroup, with a bit of enforcement from the runtime. Basically, they say that in Structured Concurrency, tasks should be organized into trees of tasks. A subtree of tasks is managed together - it is cancelled together, it fails together and before a task is finished, it must join all its subtasks. I find myself agreeing with all of this. Structured Concurrency is a very good programming model for Go as well and it's one I use it in my own programs as well. It really is how errgroup is used, with the exception that subtasks can cancel their parents as well.

The main difference to errgroup is that the StructuredTaskScope (roughly corresponding to a combination of *errgroup.Group and the context.Context) is stored in thread local storage and not passed explicitly. That's why I think it might be what you are referring to.

One thing to note is that the authors explicitly *don't* propose to add channels though, which is how they avoid dealing with what Ian said. I think as soon as you get into using channels, the idea of storing the StructuredTaskScope in thread local storage breaks down.

It's also noteworthy that this structured concurrency model is not the *only* programming model usable with virtual threads, just like not *all* goroutines have to be in an errgroup. And in fact, while I do like structured concurrency and use it as much as possible, there are cases where it makes sense *not* to use it. I think there is value in that flexibility, especially if we talk about a programming environment as a whole. The explicitness of context.Context is useful exactly in those cases where you stitch these different models together.

Overall, I don't think JEP 428 contradicts Go's usage of context.Context at all. I think most Gophers I know will find they agree with the arguments. I just don't think they really make an argument in favor of using thread local storage over passing an explicit context - all of these articles mostly just seem to assume that TLS is used and good.

In any case, I am happy to read the Java designers' thoughts on virtual threads and how they can improve programming.

[Robert Engels]

I read the structured concurrency paper and it indeed formalizes the ideas I derived from the original paper. If you review the examples it is implied that the context is supplied to workers via thread locals. The ideas presented in this paper define clearly what is lacking in the Go model - that the concurrency is unstructured and it leads to difficult to understand stack traces. 

If a thread only exists to service a “request” it defacto represents the context so if you have the facility to carry context related state with the thread it eliminates the need to explicitly pass it as method parameters. 

On Oct 1, 2022, at 1:19 AM, Bakul Shah <ba...@iitbombay.org> wrote:

Quoting from this article: "Virtual threads (JEP 425) make it cost-effective to dedicate a thread to every such I/O operation, but managing the huge number of threads that can result remains a challenge."

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/DD955473-8C00-40BA-A11E-DE94BC01F80E%40iitbombay.org.

[Axel Wagner]

On Sat, Oct 1, 2022 at 8:35 AM Robert Engels <ren...@ix.netcom.com> wrote:

I read the structured concurrency paper and it indeed formalizes the ideas I derived from the original paper. If you review the examples it is implied that the context is supplied to workers via thread locals. The ideas presented in this paper define clearly what is lacking in the Go model - that the concurrency is unstructured and it leads to difficult to understand stack traces. 

This is an extreme stretch to me. Note that Java's concurrency is also unstructured, with virtual threads as well.

In both cases, the language and runtime provide concurrency primitives which are then used and composed into higher level programming models. The StructuredTaskScope is one way in which Java supports structured concurrency on top of their unstructured concurrency model. errgroup.Group and similar APIs are ways in which Go supports structured concurrency on top of the unstructured concurrency model. There is no real contradiction here.

The one difference is that Java has TLS and Go doesn't. That's unambiguously true, but there are reasons for that. Those reasons aren't really addressed by anything I've read so far. And I think that's fine. I, personally, wouldn't go so far as to say it's *bad* that Java has TLS, just as little as I would say it's bad that Go doesn't have it. TLS has benefits and it has downsides and it is entirely reasonable for two programming languages to make different decisions about their inclusion.

Note that TLS (or GLS) and similar primitives *are* periodically talked about and I don't think it is categorically excluded to at least get primitives which address *some* of the uses. sync.Pool is one such example. However, Go tries to be careful about it and instead of adding general TLS, which can be used for things it would consider good *and* bad, rather talks about each individual use case built on top of it separately and tries to forge them into APIs which allow for the good and prevent the bad. A context.Context in TLS has so far been rejected because it seems the bad outweighs the good.

[Robert Engels]

I don’t disagree. I’ve written many concurrent systems in Java using queues (aka channels) and you end up including the context in the queued request then loading itmy into the TLS on the other side. 

The novel idea here is to avoid these queues (again channels) entirely and simply spawn threads directly needed to parallelize the sub requests. 

I think this may cause cache affinity issues (but I can see declarative ways of grouping sub tasks onto the same carrier thread). 

I respect your inclination to dismiss the ideas presented but to me it clearly describes what Go (and Java - unless you roll your own) is lacking - the ability to structure these highly concurrent systems to improve the traceability (which with thread per request was never an issue as the thread name matched the request Id - parallel sub tasks were harder but we’d still change the thread name to match the request it was working for). 

I write highly concurrent code in Go. The debugger is mostly worthless. Which is fine if you have enough test cases and such but when you interface to external systems under load… ugh. 

I suggest rereading the cited papers. He gives Go plenty of props for pushing lightweight threads. Give him the same respect and take an honest look to where Go could improve. He’s a very smart scientist. Leverage his knowledge. 

On Oct 1, 2022, at 1:47 AM, Axel Wagner <axel.wa...@googlemail.com> wrote:



[Axel Wagner]

I have now read JEP 429 Extent-local variables (ELV), which is probably the most salient for the discussion about context.Context.

ELV allow to "bind" values a variable in a way that is local to a (virtual) thread. Those values are AIUI not always inherited to child threads, but they *are* inherited when using StructuredTaskScope. ELV are immutable, but it is possible to create a new binding, forming a sort of "stack". Using x.get(), you can get the value bound in the topmost frame of that stack.

This, of course, is exactly what context.WithValue/context.Context.Value does: It creates a stack of immutable bindings of values to "keys" (corresponding to the ExtentLocal instances) in a stack. Those bindings then propagate down the stack, but not up.

The design document makes one argument in favor of TLS/ELV in favor of explicit argument passing:

Normally, data is shared between caller and callee by passing it as method arguments, but this is not viable for a Principal shared between the server component and the data access component because the server component calls untrusted user code first. We need a better way to share data from the server component to the data access component than wiring it into a cascade of untrusted method invocations.

This, of course, is not a concern for context.Value. The key can simply be an unexported type, preserving the property that a context.Context does not give access to "private" state.

So, in effect, we can find a rough correspondence between these new Java concurrency primitives and Go:

- virtual threads correspond to goroutines

- structured concurrency correspond to errgroup and the cancellation-aspect of context.Context

- extent-local variables correspond to the value-aspect of context.Context

The one difference remaining is that Go requires the explicit passing of a context.Context, while Java stores both of its aspects ultimately in TLS.

AIUI, the Java side does not allow ELV to be inherited or passed to other threads, unless they are created as a sub-task using the structured concurrency primitive. Making that possible is exactly what Ian points out above as the argument in favor of explicit context passing. Basically, while I find it agreeable that structured concurrency is a good model, I'm unconvinced that it's the *only* model or the only model that should have access to Go's equivalent of ELV. As long as we want that, we have to pass context explicitly, ISTM.

Note that the decision to tie ELV to structured concurrency is very intentional. The section "Problems with thread-local variables" mentions three problems with TLS: Unconstrained mutability, Unbounded lifetime and Expensive inheritance. Of these, the latter two are solved by tying the ELV to structured concurrency. In Go, they are solved by explicit passing of context.Context.

There is one argument in that design doc in favor of ELV as a language environment construct in favor of the context.Context design: Performance. As ELV are supported implicitly by the compiler and runtime (that's the "Extent" in ELV), they can be much cheaper to access, as they don't need to walk the entire context stack. Instead (AIUI) the language environment provides a separate context.Context stack for each ELV.

This is very similar to what I described a couple of years ago for local scoping in Go. Another way we might try to address this in the future is by collapsing context.Context transparently (i.e. have each context.Context own a map[any]any and store the values in there).

Again, I still don't see very strong arguments in favor of the Java ELV design over explicit passing. They explicitly decide not to allow the one thing that is argued in favor of argument passing - being able to inherit a context outside structured concurrency. I think that's a fine decision, I think restricting yourself to structured concurrency is a very defensible position to take - but I also don't think it's a decision Go necessarily has to follow.

[Axel Wagner]

On Sat, Oct 1, 2022 at 9:04 AM Robert Engels <ren...@ix.netcom.com> wrote:

I respect your inclination to dismiss the ideas presented

I did not dismiss anything. On the contrary, I believe they are good ideas and I said so repeatedly.

They are already present in Go, for the most part, and I'm happy they are.

[Robert Engels]

Sorry Axel if it was unclear. The message was directed in general at the responders in the list - some that responded while admitting they had not taken the time to read the paper. I appreciate the effort and reasoned rebuttal. 

On Oct 1, 2022, at 2:48 AM, Axel Wagner <axel.wa...@googlemail.com> wrote:



[Juliusz Chroboczek]

> Very interesting article came out recently.
> https://www.infoq.com/articles/java-virtual-threads/

Interesting article, thanks for the pointer.

> it has implications for the Go context discussion and the author makes
> a very good case as to why using the thread local to hold the
> context - rather than coloring every method in the chain is a better
> approach.

I didn't read that in the article, but then I may be missing something.
My takeaway is that:

- with M:N threds, one must be careful to not stash too much data in
thread local storage, since there may be large numbers of threads;

- with M:N threads, there's no need to do use thread pools; in the
absence of thread pools, thread-local storage is more useful, since
there's no risk of leaking thread-local data to a different task.

This implies to me that there is an interesting tradeoff, but does not
by itself argue against Go's design.

-- Juliusz

[Ian Lance Taylor]

On Fri, Sep 30, 2022 at 11:47 PM 'Axel Wagner' via golang-nuts
<golan...@googlegroups.com> wrote:
>
> Note that TLS (or GLS) and similar primitives *are* periodically talked about and I don't think it is categorically excluded to at least get primitives which address *some* of the uses. sync.Pool is one such example. However, Go tries to be careful about it and instead of adding general TLS, which can be used for things it would consider good *and* bad, rather talks about each individual use case built on top of it separately and tries to forge them into APIs which allow for the good and prevent the bad. A context.Context in TLS has so far been rejected because it seems the bad outweighs the good.

Another example of goroutine-local storage that we currently support
is runtime/pprof.Do, which adds labels to the current goroutine. This
seems OK as the labels are readonly and are inherited by goroutines
started with a go statement. The labels are recorded in profiles.

Ian

[robert engels]

I admit that I probably read into that based on the historical use of ThreadLocals to carry context in traditional ‘request is a thread’ systems. I think as Axel pointed out it is more explicit in the ’structured concurrency’ paper.

In Java the Thread has always carried context - it is used in security, logging, debugging, etc. The availability of millions of threads is not going to change that - as outlined in the first paper.

Anyway, I am glad you found it interesting.

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/87k05jpqae.fsf%40pirx.irif.fr.

[Axel Wagner]

To be clear, the reason we want context-passing to be explicit is precisely because Go's model is *not* "a request is a thread" and we find that desirable. Yes, `net/http` spawns a goroutine per request, but that goroutine can then spawn new ones, those might outlive the request and it can send values over channels for communication to complicate things further. We want to support that flexibility.

As I said, I like structured concurrency as a programming model and I try to encourage it. But I wouldn't want the language to lock us into that, because there *are* reasons to break out of it.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/3DC80551-BBBF-4B54-BEEB-E76DC0C05CC1%40ix.netcom.com.

[Anthony Martin]

Robert Engels <ren...@ix.netcom.com> once said:
> I think you’ll find the article interesting. It is certainly written
> by a CS “god” that knows what he’s talking about.

This is the same "god" that said:

"Everyone thinks that the concurrency model is Go’s secret
weapon, but I think their concurrency model is actually quite
error prone. That is, you have coroutines with a very basic
message-passing mechanism (channels). But in almost all cases,
the things on one side or the other of the channel are going to
have some shared mutable state guarded with locks."

Just another heliocentric distortion. Ra blah blah.

Cheers,
Anthony

[Dan Kortschak]

On Fri, 2022-09-30 at 09:32 -0500, Robert Engels wrote:
> Very interesting article came out

> recently. https://www.infoq.com/articles/java-virtual-threads/ and it

> has implications for the Go context discussion and the author makes a
> very good case as to why using the thread local to hold the context -
> rather than coloring every method in the chain is a better approach.

> If the “virtual thread aka Go routine” is extremely cheap to create
> you are far better off creating one per request than pooling - in
> fact pooling becomes an anti pattern. If you are creating one per
> request then the thread/routine becomes the context that is required.
> No need for a distinct Context to be passed to every method. 

The summary of the paper is pretty much the sentence "Virtual threads
have more in common with the user-mode threads found in other
languages, such as goroutines in Go or processes in Erlang -- but have
the advantage of being semantically identical to the threads we already
have." In this the only benefit that is there from the "advantage" is
due to a burden of maintaining compatibility with system threads in
Java code. Go obviously doesn't have that burden.

[Robert Engels]

I don’t see any arguments refuting the claim? Can you include a link so the analysis can be read in context?

> On Oct 1, 2022, at 4:13 PM, Anthony Martin <al...@pbrane.org> wrote:

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/YzitQt8310011cqe%40alice.

[Jan Mercl]

On Sun, Oct 2, 2022 at 1:37 AM Robert Engels <ren...@ix.netcom.com> wrote:

> I don’t see any arguments refuting the claim? Can you include a link so the analysis can be read in context?

The word "coroutines" does not ring a bell?

[w54n]

Not an entirely true statement, and a fairly misunderstanding of the subject.

[Robert Engels]

By many definitions Go routines and virtual threads are technically coroutines - versus a platform/OS thread. 

Again, I would like a link to the source of statement to evaluate it in context. 

On Oct 2, 2022, at 5:32 AM, w54n <w.v.me...@gmail.com> wrote:

Not an entirely true statement, and a fairly misunderstanding of the subject.

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/fbd63d49-df17-4150-9d48-6cad39a06381n%40googlegroups.com.

[Robert Engels]

Also, until fairly recently Go routines were non preemptive- making them even closer to a coroutine. I have no idea when this statement was made to judge that aspect as well. 

On Oct 2, 2022, at 6:15 AM, Robert Engels <ren...@ix.netcom.com> wrote:



[Dan Kortschak]

On Sun, 2022-10-02 at 06:15 -0500, Robert Engels wrote:
> Again, I would like a link to the source of statement to evaluate it
> in context. 

https://manningbooks.medium.com/interview-with-brian-goetz-7d6c47d05d63

[Jan Mercl]

On Sun, Oct 2, 2022 at 1:16 PM Robert Engels <ren...@ix.netcom.com> wrote:

> By many definitions Go routines and virtual threads are technically coroutines - versus a platform/OS thread.

Show one please.

Coroutines are normally a subject of non-preemptive multitasking,
threads are normally just the opposite. [0]

----
[0]: https://en.wikipedia.org/wiki/Coroutine

[Robert Engels]

Thank you. Great interview. As expected, everything he said was true - especially when you read the full segment - one in about 50 presented. No wonder the original poster did not want to link to the full article. I’ll reiterate - “CS god”.

I did a quick review of k8s - and it is almost equal usage of sync.Mutex and channels throughout the code. So in practicality - Go “do not share data” mantra is not easy to achieve in practice.

Go channels are simply two versions of a queue - a handoff or a buffered blocking queue. Java has many others.

Similarly, as Brian points out - Go has non-reentrant sync.Mutex. Java has multiple synchronization primitives to support a wider variety of use cases.

I encourage everyone to read the article - it’s an excellent interview (even if you disagree on the small reference made to Gos concurrency model).

> On Oct 2, 2022, at 6:21 AM, 'Dan Kortschak' via golang-nuts <golan...@googlegroups.com> wrote:

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/3d6ff1822f402e745cdc51776fcf6bfd0a112d75.camel%40kortschak.io.

[Robert Engels]

I already pointed that out.Go routines were non preemptive until recently. They are also still non preemptive when calling a system call without direct support, or CGo - it spins up another platform thread.

Java and other native thread languages don’t do this.

> On Oct 2, 2022, at 7:38 AM, Jan Mercl <0xj...@gmail.com> wrote:

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAA40n-XF7mF0--NbisAQoUHXEj-F1O_VUgxZ_nf1dJmipZTz3g%40mail.gmail.com.

[Jan Mercl]

On Sun, Oct 2, 2022 at 2:47 PM Robert Engels <ren...@ix.netcom.com> wrote:

> I already pointed that out.Go routines were non preemptive until recently. They are also still non preemptive when calling a system call without direct support, or CGo - it spins up another platform thread.

I'm talking about a principal difference that your "god" seems to not
understand. You are pointing out a particular past implementation
deficiency that has nothing to do with the principle.

<insert "We are not the same" meme here>

[Robert Engels]

They are still non-preemptive. Even the way preemption is implemented is cooperative.

Go routines are nearly identical to Java’s original “green threads” - which Java moved from to native threads - and now are going back with virtual threads.

Go routines (and virtual threads) are much closer to a coroutine than a thread - especially given that they ride on top of threads for their implementation.


> On Oct 2, 2022, at 7:53 AM, Jan Mercl <0xj...@gmail.com> wrote:

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAA40n-UnoN4tkSpz%3Dc-1igE3mqpQUbss3HPm7v0MVkt4LAYrWw%40mail.gmail.com.

[Robert Engels]

One other thing, if you don’t think he knows exactly how Go routines are implemented you are delusional.

> On Oct 2, 2022, at 7:53 AM, Jan Mercl <0xj...@gmail.com> wrote:
>

[Jan Mercl]

On Sun, Oct 2, 2022 at 3:04 PM Robert Engels <ren...@ix.netcom.com> wrote:

> One other thing, if you don’t think he knows exactly how Go routines are implemented you are delusional.

Maybe he should then fix the Wikipedia article I linked before. Good
luck with that.

PS: I assume you meant "goroutines" instead of "Go routines".

[Robert Engels]

Also, Go was non preemptive until early 2020. Go was released in 2009. Go has been non preemptive for almost it’s entire history.

> On Oct 2, 2022, at 7:53 AM, Jan Mercl <0xj...@gmail.com> wrote:
>

[Robert Engels]

Did you even bother to read the wiki article you linked? If you did you would see that Go is listed as one of the implementations of coroutines - Java is not.

Please learn more before you pollute other minds with your garbage.

> On Oct 2, 2022, at 8:13 AM, Jan Mercl <0xj...@gmail.com> wrote:

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAA40n-WXjyeW6uHkWvXJyRKxdSmvgbRObkS_%2BxTENSOCRW770A%40mail.gmail.com.

[Jan Mercl]

On Sun, Oct 2, 2022 at 3:00 PM Robert Engels <ren...@ix.netcom.com> wrote:

> They are still non-preemptive. Even the way preemption is implemented is cooperative.

Goroutines are preempted in a thread signal handler that invokes the
Go runtime scheduler that in turn saves the required goroutine state
and switches the execution context to a different goroutine, if
available.

Which part of that do you call "cooperative"?

[Robert Engels]

(In case other read the wiki - Go is listed under the “native support” section. Java and other languages have library support available for coroutines.

> On Oct 2, 2022, at 8:14 AM, Robert Engels <ren...@ix.netcom.com> wrote:
>
> Also, Go was non preemptive until early 2020. Go was released in 2009. Go has been non preemptive for almost it’s entire history.

[Jan Mercl]

On Sun, Oct 2, 2022 at 3:18 PM Robert Engels <ren...@ix.netcom.com> wrote:

> Did you even bother to read the wiki article you linked? If you did you would see that Go is listed as one of the implementations of coroutines - Java is not.

Here you go: https://groups.google.com/g/golang-nuts/c/SEBF93Wta2w/m/TB2P-LREvYcJ

> Please learn more before you pollute other minds with your garbage.

I don't bother to do so, but please report yourself to the CoC board, thank you.

[Axel Wagner]

Can we collectively stay calm and within the bounds of polite discourse? The overall tone of this thread is pretty aggressive.

I would also like to point out that this particular topic of discussion - the credentials of the author of the original article - seem fallacious to me from either side. They can neither be used to elevate arguments - that'd be an appeal to authority - nor can they be used to dismiss them - that'd be an ad-hominem.

The original article and the related design documents contain more than enough material to have a reasoned, substantive discussion, without litigating the character and credentials of their authors. I find them all pretty well-written and reasoned TBQH.

---

As far as that quote goes: I haven't read the full interview, but I tend to agree with the substance of that quote, personally. I myself have noticed that I rarely use channels and "real" CSP. I've also noticed especially novices to the language "overusing" channels, creating hard to understand and often buggy code. I think with discipline and adhering to a couple of rules can make them safe and lead to easy to understand code - but I find it often easier to "just slap a mutex on it", if you will.

I don't think it can be argued that Go's concurrency model is particularly safe on its own. Especially not with a language like Rust existing and gaining mainstream attention. I think that's fine and I still think goroutines and channels are a net benefit (it does make it significantly easier to add concurrency). think Go's overall simplicity still makes it easier to write understandable, mostly bug-free software, than many other languages I've tried.

I don't think any of this is an attack on the design of Go. I obviously love the language, despite these concerns. I just don't advertise its concurrency features as much to novices as I used to.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAA40n-WXjyeW6uHkWvXJyRKxdSmvgbRObkS_%2BxTENSOCRW770A%40mail.gmail.com.

[Robert Engels]

Well said. Btw, the appeal to authority fallacy does not apply to be persons that are generally accepted experts in their field. There should be no debate on whether Brian qualifies - his position alone nearly suffices. 

On Oct 2, 2022, at 8:41 AM, 'Axel Wagner' via golang-nuts <golan...@googlegroups.com> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAEkBMfE-eExBacz2c1OXtzAPU9NUsNzBBa135yiJFubgvE9bJA%40mail.gmail.com.

[Sven Anderson]

Ian Lance Taylor <ia...@golang.org> schrieb am Sa. 1. Okt. 2022 um 17:12:

Another example of goroutine-local storage that we currently support
is runtime/pprof.Do, which adds labels to the current goroutine.  This
seems OK as the labels are readonly and are inherited by goroutines
started with a go statement.  The labels are recorded in profiles.

Aren’t they rather write-only? It would be great, if they are write-once-read-many, so that you can legally use them in logs and debug output at any place.

I think it would be great to allow storing immutable context.Context for such cases.

As it happens, I wrote a small package, that does that „almost“ legally: 

https://pkg.go.dev/github.com/ansiwen/gctx

Cheers 

Sven

[Dan Kortschak]

On Mon, 2022-10-03 at 00:46 +0200, Sven Anderson wrote:
> As it happens, I wrote a small package, that does that „almost“
> legally: 
> https://pkg.go.dev/github.com/ansiwen/gctx

There is also github.com/kortschak/goroutine for getting goroutine IDs,
which can be used as the primitive for constructing your own
personalised foot gun.

[Ian Lance Taylor]

On Sun, Oct 2, 2022, 4:46 PM Sven Anderson <sv...@redhat.com> wrote:

Ian Lance Taylor <ia...@golang.org> schrieb am Sa. 1. Okt. 2022 um 17:12:

Another example of goroutine-local storage that we currently support
is runtime/pprof.Do, which adds labels to the current goroutine.  This
seems OK as the labels are readonly and are inherited by goroutines
started with a go statement.  The labels are recorded in profiles.

Aren’t they rather write-only? It would be great, if they are write-once-read-many, so that you can legally use them in logs and debug output at any place.

True, write only is a better description.

Ian