Locking goroutine to "main" thread

[buch...@gmail.com]

I've been experimenting with graphics programming using go+sdl2 for awhile now. I've always been uncertain how to deal with locking a goroutine to the main thread. MacOS requires interactions with the OS window to come from the main thread.

So far, I have something like this (pseudocode):

func main() {
  cmds := make(chan func())
  go func() {
    cmds <- func() {
      // this function runs on the main thread
      // and accesses the GPU via the global OpenGL state.

      gl.Draw()

    }

  }()
  runtime.LockOSThread()
  win := createWindow()

  for cmd := range cmds {
    cmd()
  }
}

I'd really love to get that LockOSThread call out of the main function and hide it away inside a library along with the other internal implementation details. Is that possible? As far as I can tell, it's not possible to ensure that a goroutine runs on the main thread without locking it directly from the main() function.

Thanks.

[Ian Lance Taylor]

On Sat, Dec 14, 2019 at 11:31 AM <buch...@gmail.com> wrote:
>
> I'd really love to get that LockOSThread call out of the main function and hide it away inside a library along with the other internal implementation details. Is that possible? As far as I can tell, it's not possible to ensure that a goroutine runs on the main thread without locking it directly from the main() function.

I don't know if this helps, but you can lock the main goroutine to a
thread by calling LockOSThread in an init function.

Ian

[buch...@gmail.com]

If I understand correctly, that would prevent me from starting any other goroutines. The following program deadlocks:

package main

import (
    "log"
    "runtime"
    "time"
)

func init() {
    runtime.LockOSThread()
}

func main() {
    go func() {
        for range time.After(time.Second) {
            log.Print("tick")
        }
    }()

    block := make(chan struct{})
    <-block
    log.Print("done")
}

[buch...@gmail.com]

Would it be reasonable to make a Go proposal that adds something like runtime.MainThread(), which would cause the calling goroutine to only ever be scheduled on the main thread? Unlike runtime.LockOSThread, it would not need to prevent other goroutines from running on the main thread (I think).

[buch...@gmail.com]

I found some existing discussion on this topic: https://github.com/golang/go/issues/14163

Not sure I fully understand the reasoning there. If there is a better way forward that doesn't involve adding a controversial function to the scheduling package, I'm all ears.

[Ian Lance Taylor]

On Sat, Dec 14, 2019 at 11:51 AM <buch...@gmail.com> wrote:
>
> If I understand correctly, that would prevent me from starting any other goroutines. The following program deadlocks:

That turns out not to be the case. You can start other goroutines.
Your program deadlocks because you called time.After when you meant to
call time.Tick.

Ian

> package main
>
> import (
> "log"
> "runtime"
> "time"
> )
>
> func init() {
> runtime.LockOSThread()
> }
>
> func main() {
> go func() {
> for range time.After(time.Second) {
> log.Print("tick")
> }
> }()
>
> block := make(chan struct{})
> <-block
> log.Print("done")
> }
>
>
>
>
> On Saturday, December 14, 2019 at 11:45:53 AM UTC-8, Ian Lance Taylor wrote:
>>
>> On Sat, Dec 14, 2019 at 11:31 AM <buch...@gmail.com> wrote:
>> >
>> > I'd really love to get that LockOSThread call out of the main function and hide it away inside a library along with the other internal implementation details. Is that possible? As far as I can tell, it's not possible to ensure that a goroutine runs on the main thread without locking it directly from the main() function.
>>
>> I don't know if this helps, but you can lock the main goroutine to a
>> thread by calling LockOSThread in an init function.
>>
>> Ian
>

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/e4ac1c49-0fbf-4f69-8dcc-64d91e7f3536%40googlegroups.com.

[Ian Davis]

On Sat, 14 Dec 2019, at 7:51 PM, buch...@gmail.com wrote:

If I understand correctly, that would prevent me from starting any other goroutines. The following program deadlocks:

I doesn't prevent you starting other goroutines but you need to ensure that all calls to the OpenGL context are performed on the thread you initialised it with (my understanding is that this is because OpenGL uses thread local storage). Typically you can do this by sending closures containing OpenGL calls via a channel to an executor running in a goroutine locked to the main thread. There is/was an example of this in the gomobile codebase. I can share more detail if you get stuck.

-- Ian

[Alex Buchanan]

Yep. My understanding of locking the OS thread from an init() was wrong. I think I have it working now. Thanks guys!

--

You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/zAB8Si7EMVM/unsubscribe.

To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/0adcdc5a-dd2c-4506-9110-a3ecabc508b9%40www.fastmail.com.

[Everton Marques]

See also:

https://github.com/faiface/mainthread

[buch...@gmail.com]

I've been getting by with a version of this that sends commands (closures) to a loop on the main thread:

https://github.com/buchanae/ink/blob/2af8781a960a0351b6b6b7ca23d81ae5c43535ec/win/window.go#L55

And here is where it pops those commands, and also integrates with the OS event loop:

https://github.com/buchanae/ink/blob/2af8781a960a0351b6b6b7ca23d81ae5c43535ec/win/window.go#L163

But, I'm still not satisfied. The solution ties together the scheduling (and code) of two separate event loops. In particular, I've found that my commands are delayed ~10ms – I think sdl.WaitEvent might be rate limited.

What I really want is for the two event loops (OS event loop vs app command loop) to be in two separate goroutines, both running on the main thread. That way, the OS event loop can react to infrequent window events (mouse clicks, etc) without burning lots of CPU, while my app command loop can react to commands instantly.

Does that make sense? Is this possible to implement without requiring a change to the Go runtime? Is it possible to change the Go runtime to allow multiple goroutines to be scheduled only to the main thread?

Thanks.

[Robert Engels]

Even if you could I don’t think you would want to do it this way. 

Have a go routine sleep on a channel. Post to the channel from the native code. 

Let your command loop run on any thread and synchronize via a channel the calls to/from native. 

The os event loop doesn’t need to run on main - it just needs to be locked to a thread - use a native thread - and post the os events to a channel.  

Probably easiest to export a simple Go postToEventChannel() and have the native use this. 

On Jan 3, 2020, at 2:18 PM, buch...@gmail.com wrote:



--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/e4e3a04d-0f76-4baf-9760-9992ef38d51f%40googlegroups.com.

[buch...@gmail.com]

I'm pretty sure the OS event loop is required to be on the main thread.

From the GLFW docs for glfwPollEvents:

"This function must only be called from the main thread."

From the SDL docs for SDL_WaitEvent:

"you can only call this function in the thread that initialized the video subsystem."

To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.

[robert engels]

You can definitely run the event loop for a process on a thread other than main. The main thread is the thread created by the OS to begin running the process - the UI thread is the one that initializes the Windowing system. Some OSs even support multiple UI threads (see https://docs.microsoft.com/en-us/cpp/parallel/multithreading-creating-user-interface-threads?view=vs-2019)

Go doesn’t do any Windowing system initialization by default - the main thread in Go (at least according to the referenced library) is the one that called the package init() - which may not even be the OS main thread since according to the Go docs "Package initialization—variable initialization and the invocation of init functions—happens in a single goroutine, sequentially, one package at a time.” - which makes no claim that this Go routine is running on the “main” OS thread.

Whatever thread you use to initialize GLFW is the UI thread (or main for GLFW).

This is pretty standard across windowing/graphics systems.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/cbbca787-1dec-4729-b91f-25a7b5725d80%40googlegroups.com.

[buch...@gmail.com]

Whether the UI thread is on the main thread or a separate thread, a single UI thread needs to process events from two sources: the OS window and from my application. Having one loop process both sources (without support from the Go runtime scheduler) is the part I'm struggling with.

My latest iteration is something like:

for {
  // PollEvent doesn't block/sleep

  for ev := sdl.PollEvent(); ev != nil; ev = sdl.PollEvent {
    // handle OS window events
  }
  select {
  case cmd := <-app.commands:
    // process application command

    cmd()
  case <-time.After(10*time.Millisecond):
    // only wait 10ms for application commands

  }
}

This is an improvement, but I'd still prefer help from the Go runtime. If the runtime supported locking two goroutines to the UI thread, I could split this loop into two separate loops and remove the time.After, I *think*.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/cbbca787-1dec-4729-b91f-25a7b5725d80%40googlegroups.com.

[robert engels]

You only need a single thread locked to the UI thread.

Use one Go routine locked to the UI thread. Put events onto a channel.

Have another Go routine read from the channel, and the app command channel using select.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/6a32824e-cd50-4fe6-9411-3ecb975718d5%40googlegroups.com.

[robert engels]

In reviewing your comments so more, I think you may be having trouble because you are initializing the graphics UI in the init() method. I think that is going to make things difficult. You are better off adding a StartUI() - which spawns a Go routine that handles all UI communicates (you could spawn this from the init() but I think that might make things harder.

You can sync the calls so only a single routine/thread will ever be created.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/F3F3EB37-66C4-43DC-9001-C5139D0BF3E2%40ix.netcom.com.

[robert engels]

Also, even simpler - just remove the time.Atfter case() and use a default: - but the problem is you will spin a single thread at 100% cpu - you really need to use a blocking sdl.WaitEvent()

On Jan 3, 2020, at 6:04 PM, robert engels <ren...@ix.netcom.com> wrote:

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/F3F3EB37-66C4-43DC-9001-C5139D0BF3E2%40ix.netcom.com.