Is GOMAXPROCS used by the compiler or the runtime?

[tomwilde]

I've read that the GOMAXPROCS constant sets the maximum number of processors a Go program will attempt to use.

Is this constant used by the runtime, and needs to be set on the host machine executing the program, or the compiler?

[Aram Hăvărneanu]

> Is this constant used by the runtime, and needs to be set on the host
> machine executing the program, or the compiler?

Runtime.

--
Aram Hăvărneanu

[Kyle Lemons]

The runtime checks the environment variable.

[Graham Anderson]

You can set this at runtime

runtime.GOMAXPROCS(runtime.NumCPU())

Where NumCPU() is the number of logical cores/cpus available.

[Hotei]

Kyle,

Are you sure?  Looks like the latest runtime knows how many CPUs without an explict ENV setting.

 func main() {

fmt.Printf("ENV says GOMAXPROCS: :%s: \n", os.Getenv("GOMAXPROCS")

fmt.Printf("runtime says MAXPROCS = %d \n", runtime.NumCPU())

}

$ ./envprint

ENV says GOMAXPROCS: ::   <-- empty string

runtime says MAXPROCS = 6

[Matt Kane's Brain]

You want to print the result of runtime.GOMAXPROCS(-1).

$ GOMAXPROCS=72 go run cpu.go
ENV says GOMAXPROCS: :72:
runtime says MAXPROCS = 72

On Mon, Mar 12, 2012 at 22:57, Hotei <hote...@gmail.com> wrote:
> $ ./envprint
> ENV says GOMAXPROCS: ::   <-- empty string
> runtime says MAXPROCS = 6

--
matt kane's brain
http://hydrogenproject.com

[Hotei]

I always check the runtime value of NumCPU() before starting up a multi-processor program if I need to limit the goroutine count to match number of CPUs.  My problem is I also have one of those weird programs that runs slower in multiple-CPU mode and I have to fake GOMAXPROCS=1 if I want it to run full speed.  Most other programs run faster in multiple-CPU mode.  It's still a puzzle why.  I thought if I left GOMAXPROCS unset then NumCPU would default to 1 but apparently that is wrong.  I guess the moral of the story is if the program depends on a specific value then set it specifically  - like you did below.

[Hotei]

Just to confuse the situation a bit more, I tried to set GOMAXPROCS explicitly before running same program:

hotei@Odin:~/Desktop/MYGO/src/Simple/envprint$ GOMAXPROCS=1 ./envprint

ENV says GOMAXPROCS: :1:

runtime says MAXPROCS = 6

Ubuntu 11.04/64 bit,  Go weekly 2012-3-4

I can deal with it, but it's a bit strange.

[Kyle Lemons]

On Mon, Mar 12, 2012 at 7:57 PM, Hotei <hote...@gmail.com> wrote:

Kyle,

Are you sure?  Looks like the latest runtime knows how many CPUs without an explict ENV setting.

I may have misunderstood your question; the runtime knows how many CPUs you have, yes, but GOMAXPROCS defaults to 1 (see runtime.GOMAXPROCS()) unless overridden.

 

 func main() {

fmt.Printf("ENV says GOMAXPROCS: :%s: \n", os.Getenv("GOMAXPROCS")

Use %q ;-)

 

fmt.Printf("runtime says MAXPROCS = %d \n", runtime.NumCPU())

Also print runtime.GOMAXPROCS() and you'll see how many concurrent (active) goroutines it will actually schedule.

[Hotei]

After about an hour of experimentation here's what I found:  If I want to use a specific number of 'cores', I MUST set that number in the program with:

runtime.GOMAXPROCS(n)  

This seems to enforce a limit on the number of CPUs active, regardless of how many goroutines might be called to run on them.

Sensible values for n range from 1 up to the number of cores in my AMD CPU ( x 2 for Intel with active HT).  Setting it to 1 will make some programs faster, but most will benefit from higher numbers - providing more cores are available of course.

I can choose to set n using the GOMAXPROCS environment variable, but I have to do that explicitly by converting it to an int n and then calling the runtime.GOMAXPROCS(n) for it to "stick".

I can choose to let n "default", which on my system makes it use the value of runtime.NumCPU(). Others as noted in posts above have reported different results for their systems.

This applies to my Ubuntu 11.04/AMD weekly 2012-3-4 for sure and ? others.

For me the key is that "GOMAXPROCS" inside the program, the "GOMAXPROCS=n" environment variable, and "NumCPU()" can represent 3 distinct, potentially non-overlapping entities.

[⚛]

On Tuesday, March 13, 2012 7:24:52 PM UTC+1, Hotei wrote:

After about an hour of experimentation here's what I found:  If I want to use a specific number of 'cores', I MUST set that number in the program with:

runtime.GOMAXPROCS(n)  

This seems to enforce a limit on the number of CPUs active, regardless of how many goroutines might be called to run on them.

Sensible values for n range from 1 up to the number of cores in my AMD CPU ( x 2 for Intel with active HT).  Setting it to 1 will make some programs faster, but most will benefit from higher numbers - providing more cores are available of course.

I can choose to set n using the GOMAXPROCS environment variable, but I have to do that explicitly by converting it to an int n and then calling the runtime.GOMAXPROCS(n) for it to "stick".

You do not need to call runtime.GOMAXPROCS(n) yourself when the GOMAXPROCS environment variable is set. The environment variable is read & converted to an integer automatically by Go run-time at program initialization.

 

I can choose to let n "default", which on my system makes it use the value of runtime.NumCPU(). Others as noted in posts above have reported different results for their systems.

If there is no GOMAXPROCS variable in the environment, the default value on all systems is 1. The source code is at http://r60.golang.org/src/pkg/runtime/proc.c, line 195.

 

This applies to my Ubuntu 11.04/AMD weekly 2012-3-4 for sure and ? others.

If you are counting OS threads, note that Go run-time may create more OS threads than the value returned by runtime.GOMAXPROCS(-1).

[Kyle Lemons]

This seems to enforce a limit on the number of CPUs active, regardless of how many goroutines might be called to run on them.

Yes, that would be what the documentation says.

 

Setting it to 1 will make some programs faster, but most will benefit from higher numbers - providing more cores are available of course.

That is certainly not the case at this point.

 

I can choose to set n using the GOMAXPROCS environment variable, but I have to do that explicitly by converting it to an int n and then calling the runtime.GOMAXPROCS(n) for it to "stick".

There is something else at work here, because it definitely works as an environment variable.

 

I can choose to let n "default", which on my system makes it use the value of runtime.NumCPU(). Others as noted in posts above have reported different results for their systems.

If there is no GOMAXPROCS variable in the environment, the default value on all systems is 1. The source code is at http://r60.golang.org/src/pkg/runtime/proc.c, line 195.

As atom says; this is because your application must be in the subset that will benefit from thread-level parallelism with the current scheduler.  The reverse is not the case.

 

This applies to my Ubuntu 11.04/AMD weekly 2012-3-4 for sure and ? others.

If you are counting OS threads, note that Go run-time may create more OS threads than the value returned by runtime.GOMAXPROCS(-1).

In particular, blocking syscalls can have their own thread beyond the "running" threads (which are bounded by GOMAXPROCS).

 

For me the key is that "GOMAXPROCS" inside the program, the "GOMAXPROCS=n" environment variable, and "NumCPU()" can represent 3 distinct, potentially non-overlapping entities.

The latter has no relation to either of the former, it's simply an informational function.

[Bobby Powers]

My informal understanding has always been that GOMAXPROCS is the
number of operating system threads that the go runtime will multiplex
goroutines onto. In addition, the go runtime may have a number of
other threads, related to cgo, or polling, or other potentially
blocking things. Maybe that helps