Blocking in C functions called from go

[Pushkar]

I need to call some C functions from Go. I think I know how to proceed using cgo.
However, I was wondering how goroutines and blocking calls in the C functions work together.
So in the below example (pseudocode) will the goroutine be suspended when the pthread_lock call is waiting to acquire the lock?

I am guessing that the Go engine will detect that the OS thread on which the goroutine(s) are running is blocked so it will suspend them too.
Once the thread is runnable, the blocked goroutines will be scheduled to run.

For example:

int cfunc (int a) {
  pthread_lock(..);
  ++a;
  pthread_unlock();
}

func main() {
  for (i  := 0; i < 100; i++) {
    go cfunc(i)
  }
}

[Ian Lance Taylor]

On Fri, Oct 13, 2017 at 12:23 PM, 'Pushkar' via golang-nuts
<golan...@googlegroups.com> wrote:
> I need to call some C functions from Go. I think I know how to proceed using
> cgo.
> However, I was wondering how goroutines and blocking calls in the C
> functions work together.
> So in the below example (pseudocode) will the goroutine be suspended when
> the pthread_lock call is waiting to acquire the lock?

Yes.

> I am guessing that the Go engine will detect that the OS thread on which the
> goroutine(s) are running is blocked so it will suspend them too.
> Once the thread is runnable, the blocked goroutines will be scheduled to
> run.

Yes.

Ian

> For example:
>
> int cfunc (int a) {
> pthread_lock(..);
> ++a;
> pthread_unlock();
> }
>
> func main() {
> for (i := 0; i < 100; i++) {
> go cfunc(i)
> }
> }
>

> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Konstantin Khomoutov]

On Fri, Oct 13, 2017 at 12:23:42PM -0700, 'Pushkar' via golang-nuts wrote:

> I need to call some C functions from Go. I think I know how to proceed
> using cgo.
> However, I was wondering how goroutines and blocking calls in the C
> functions work together.
> So in the below example (pseudocode) will the goroutine be suspended when
> the pthread_lock call is waiting to acquire the lock?
>
> I am guessing that the Go engine will detect that the OS thread on which
> the goroutine(s) are running is blocked so it will suspend them too.
> Once the thread is runnable, the blocked goroutines will be scheduled to
> run.

To what Ian said, I would add only a single goroutine runs on a given
OS thread at any given time, and when a goroutine enters a cgo call, the
runtime makes sure that goroutine gets locked to the underlying OS
thread it's running on (as if the goroutine explicitly called
runtime.LockOSThread()). This makes sure both Go and C sides are
consistent about the thread their code executes on during the cgo call.

Another thing to keep in mind is that the Go runtime scheduler gives a
goroutine which blocks in a syscall or a cgo calls about 20 us (as of Go
1.9) to complete [1], and if it doesn't, the scheduler parks the
goroutine and may spawn another OS thread if there are runnable
goroutines -- to keep up with its promise to have runtime.GOMAXPROCS
goroutines running.

A corollary to this is that if you make many such blocking cgo calls
roughly in parallel, this might lead to spawning many OS threads.
This is not a bad thing in itself, but it's something to keep in mind.

1. https://github.com/golang/go/issues/21827#issuecomment-329103577

[Pushkar Pradhan]

Konstantin,
Thanks for the detailed explanation. That was helpful.

--

pushkar