Writing safe CGO calls and callbacks and how that affects system threads

[lu...@zeta.si]

Hello. I have a Go program that calls C functions in order to create a Gtk+ GUI. Because Gtk+ itself isn't thread-safe, I'm wondering if it's even possible to write GUI apps this way since there's no guarantee that the same thread will be used to call the API (even if I lock the main goroutine inside a OS thread). Am I correct?

Also, what happens if the called CGO function calls a Go function? Can it happen that the Go runtime will create a new thread, because the previous one is still locked? 

Greets,

Luka

[Ian Lance Taylor]

On Sun, Feb 12, 2017 at 1:47 AM, <lu...@zeta.si> wrote:
> Hello. I have a Go program that calls C functions in order to create a Gtk+
> GUI. Because Gtk+ itself isn't thread-safe, I'm wondering if it's even
> possible to write GUI apps this way since there's no guarantee that the same
> thread will be used to call the API (even if I lock the main goroutine
> inside a OS thread). Am I correct?

You have to arrange to make all Gtk calls from a single goroutine, and
you have to make that single goroutine call runtime.LockOSThread
before it makes any Gtk calls. Making all the calls from a single
goroutine typically means that that goroutine has to sit waiting on a
channel that other goroutines use to tell it what to do. This
approach should be reliable.

> Also, what happens if the called CGO function calls a Go function? Can it
> happen that the Go runtime will create a new thread, because the previous
> one is still locked?

If a Go function F1 calls a C function and the C function (running in
the same C thread) calls a Go function F2, F2 will run in the same
thread as F1. This is true whether or not F1 is locked to the thread.
In fact, for the duration of the C function, the goroutine will be
locked to the thread anyhow.

Ian

[Luka Napotnik]

Thanks for the reply. I've created a test program with function F1 that calls a C function F2. The function F2 then calls a Go function F3.

I've started the test once with GOMAXPROCS set to 1, and the second time without an env. (using Go 1.7). There are short pauses between calls and I've measured the number of system threads used by the process with ps -o nlwp.

I found that when running the test with GOMAXPROCS=1, a new thread is always spawned when the C function F2 calls a Go function, but not if the env. is not set. Why is that? I know GOMAXPROCS can be ignored in some cases, so I assumed the GO runtime needed an extra thread for executing function F3, after it was called from C.

Greets,

[Luka Napotnik]

I forgot to mention that the main goroutine in the test program is locked to a thread as I call runtime.LockOSThread() in init()

Greets,

[Ian Lance Taylor]

On Sun, Feb 12, 2017 at 9:57 PM, Luka Napotnik <lu...@zeta.si> wrote:
> Thanks for the reply. I've created a test program with function F1 that
> calls a C function F2. The function F2 then calls a Go function F3.
>
> I've started the test once with GOMAXPROCS set to 1, and the second time
> without an env. (using Go 1.7). There are short pauses between calls and
> I've measured the number of system threads used by the process with ps -o
> nlwp.
>
> I found that when running the test with GOMAXPROCS=1, a new thread is always
> spawned when the C function F2 calls a Go function, but not if the env. is
> not set. Why is that? I know GOMAXPROCS can be ignored in some cases, so I
> assumed the GO runtime needed an extra thread for executing function F3,
> after it was called from C.

I would guess that the new thread is being spawned to run the system
monitor or to run the garbage collector. But without seeing your
program I don't actually know.

You can check for yourself whether you are running in the same thread
easily enough by calling C.gettid.

Ian

[Luka Napotnik]

Thanks for the answers. I'll play around with C.gettid.

Greets,

[db0...@gmail.com]

On Monday, February 13, 2017 at 7:02:22 AM UTC+1, Luka Napotnik wrote:

I forgot to mention that the main goroutine in the test program is locked to a thread as I call runtime.LockOSThread() in init()

The Go Spec says "Package initialization—variable initialization and the invocation of init functions—happens in a single goroutine, sequentially, one package at a time. An init function may launch other goroutines, which can run concurrently with the initialization code.". The way I understand it, code in init() is not guaranteed to run in the same goroutine as main(), so you might be better of calling LockOSThread() from main() (or from whatever goroutine will do the GTK calls).

[Ian Lance Taylor]

I think you're right that the spec does not guarantee that the same
goroutine runs init functions and the main function. I'll just note
that in practice it is the same goroutine, for all the implementations
I know of.

Ian

[Luka Napotnik]

Ok, thanks for the clarification. 

--

Greets,

Luka Napotnik