GOMAXPROCS and LockOSThread

[Raph]

Does anyone know how using runtime.LockOSThread() affects the number of threads dedicated to goroutines as defined in runtime.GOMAXPROCS()?

There's a specific case where I need to lock a goroutine to a thread (OpenGL related). I wonder if that means remaining goroutines will multiplex onto GOMAXPROCS-1 threads, or will a new thread be pooled to maintain GOMAXPROCS for other goroutines?

It seems it would be more correct if only GOMAXPROCS-1 were left for other routines, but for n threads, what would happen if n+1 goroutines called LockOSThread()? So I suspect there's some logic that adds threads to the pool.

Does anybody know specifics about this?

Thanks!

[Ian Lance Taylor]

A goroutine that uses LockOSThread only counts against GOMAXPROCS
while it is actually running Go code. That is, GOMAXPROCS is the
number of goroutines that may run simultaneously. LockOSThread means
that a particular goroutine must run on a particular thread. They
seem related but they are really independent.

Ian

[Larry Clapp]

This is a subtlety that eluded me for a while.  GOMAXPROCS is the number of running threads, not the total number of threads.  You could have 30 threads, but if they're all blocked waiting on IO, none of them count against GOMAXPROCS.  (Right?)

So if I understand correctly: A goroutine locked to a thread only counts against GOMAXPROCS while it's running.  If it's waiting on IO (or a channel or not running for some other reason), then it doesn't.  But this is the same as any other goroutine.  Which is why being locked to a thread is (as you said) independent of GOMAXPROCS.

Finally, this implies (well, it says outright, if I understand correctly) that even if GOMAXPROCS == 1, you could have multiple threads, but only one should be running at any given time.  So if one goroutine does some blocking IO, and other goroutines are runnable, the runtime may well start a new thread to handle them, and then quiesce it (or whatever the appropriate term is) when the first one is runnable again.

(Right?)

-- Larry

[Raph]

I see - I suppose I was stuck thinking about a thread pool with a static number. So the underlying thread pool can/will grow and shrink depending on the state of the goroutines?

I wonder if a thread that is locked to a particular thread via LockOSThread has a disadvantage over non-locked goroutines in that it may have to wait longer for the particular thread than if it took any available one?

[Raph]

sorry, confusing typo. meant "goroutine", not "thread"

On Tuesday, September 11, 2012 8:09:04 PM UTC+1, Raph wrote:

I see - I suppose I was stuck thinking about a thread pool with a static number. So the underlying thread pool can/will grow and shrink depending on the state of the goroutines?

I wonder if a goroutine that is locked to a particular thread via LockOSThread has a disadvantage over non-locked goroutines in that it may have to wait longer for the particular thread than if it took any available one?

[Ian Lance Taylor]

On Tue, Sep 11, 2012 at 12:09 PM, Raph <gala...@gmx.de> wrote:
> I see - I suppose I was stuck thinking about a thread pool with a static
> number. So the underlying thread pool can/will grow and shrink depending on
> the state of the goroutines?

Yes.

> I wonder if a thread that is locked to a particular thread via LockOSThread
> has a disadvantage over non-locked goroutines in that it may have to wait
> longer for the particular thread than if it took any available one?

No, the locked thread will only be used for that one goroutine.

Ian

[Raph]

Thanks for the clear answers Ian!

Oh, and thanks for Go, it's been a fun ride so far :-)

[Sugu Sougoumarane]

You might have run into the same problem we encountered while using the mysql C API, which requires calling an init function for each thread.

Fortunately, the function was idempotent and cheap. So, we ended up wrapping every C call with another function that called init. It doesn't look pretty, but it works, and we didn't notice any performance degradation:

http://code.google.com/p/vitess/source/browse/go/mysql/mysql.go

It would be a nifty feature if we could provide thread initialization hooks for cgo. In the meantime, you can try and see if our approach will work for you.

[Larry Clapp]

Did you consider (or try) using syscall.Gettid() to identify the thread you're in so as to only call it once per thread?  (I Am Not A Go Guru; I am not saying this is a good idea, I'm just curious.  It might be a good idea. :)

-- Larry

[Sugu Sougoumarane]

Well. Keeping track of the thread ids for which you've called init might amount to the same work that mysql_init does, and you still have to cover all entry points. The best time would be at the time of thread creation, which only the runtime controls.

[Larry Clapp]

Agree on all counts.  I was just thinking it might perform better if you only did the thread initialization once per thread.  Hmm, but now upon rereading, I see the part about "and we didn't notice any performance degradation".

So never mind.  Thanks anyway.  :)

-- Larry