Get the number of CPUs
Stefan Pantos
I want a way of finding out the number of CPU's on a system which
will work under Linux, Unix and windows. Does anyone know of a way to do
this?
Thanks,
Stefan
David Butenhof
> I want a way of finding out the number of CPU's on a system which
> will work under Linux, Unix and windows. Does anyone know of a way to do
> this?
Um, yeah. Read the specs for the particular hardware box you're using. That
will work with any hardware and OS! ;-)
Other than that... you're completely out of luck, because there's no
standard interface for this information. It's AVAILABLE, in some form, on
every platform that supports multiprocessors, but there are a wide range of
different mechanisms.
Believe it or not, we spent months in POSIX trying to come up with standard
interfaces for this sort of information, and couldn't come to any
consensus. Every interface that exists is either too simple or too
complicated for many systems, and we simply couldn't get a workable
compromise. For example, beware that on a NUMA system, knowing "how many"
CPUs you have is useless unless you know WHICH cpus you can touch and HOW
they're connected to each other. (Actually, it's often WORSE than useless
because your simplistic attempts at using the misleading information will
result in far worse performance than letting the OS "do its thing".)
--
/--------------------[ David.B...@hp.com ]--------------------\
| Hewlett-Packard Company Tru64 UNIX & VMS Thread Architect |
| My book: http://www.awl.com/cseng/titles/0-201-63392-2/ |
\----[ http://homepage.mac.com/dbutenhof/Threads/Threads.html ]---/
Stefan Pantos
> Stefan Pantos wrote:
>
>> I want a way of finding out the number of CPU's on a system which
>> will work under Linux, Unix and windows. Does anyone know of a way to
>> do this?
>
> Um, yeah. Read the specs for the particular hardware box you're using.
> That will work with any hardware and OS! ;-)
>
> Other than that... you're completely out of luck, because there's no
> standard interface for this information. It's AVAILABLE, in some form,
> on every platform that supports multiprocessors, but there are a wide
> range of different mechanisms.
>
> Believe it or not, we spent months in POSIX trying to come up with
> standard interfaces for this sort of information, and couldn't come
> to any consensus. Every interface that exists is either too simple or
> too complicated for many systems, and we simply couldn't get a
> workable compromise. For example, beware that on a NUMA system,
> knowing "how many" CPUs you have is useless unless you know WHICH
> cpus you can touch and HOW they're connected to each other. (Actually,
> it's often WORSE than useless because your simplistic attempts at
> using the misleading information will result in far worse performance
> than letting the OS "do its thing".)
>
I take it that there will never be a way in POSIX to find this
information out. Thats realy annoying because I don't want to take up
system resources up by having more than one thread when on a UP but
would like it to take advantage of the CPUs of an MP.
I was hoping that there was some clever round about way of doing it.
Something like doing a test set of threads to find it out but I doubt
that that would work.
You don't happen to be able to get which processor a thread is running.
Can you?
Thanks,
Stefan
Casper H.S. Dik
>You don't happen to be able to get which processor a thread is running.
>Can you?
I'm sure that no interface will ever be offered to return that information
either; it's fairly useless: by the time you see the answer it may
no longer be valid.
Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.
David Butenhof
> I take it that there will never be a way in POSIX to find this
> information out. Thats realy annoying because I don't want to take up
> system resources up by having more than one thread when on a UP but
> would like it to take advantage of the CPUs of an MP.
Well, "never" is a long time. Still, I wouldn't count on anything happening
soon.
> I was hoping that there was some clever round about way of doing it.
> Something like doing a test set of threads to find it out but I doubt
> that that would work.
>
> You don't happen to be able to get which processor a thread is running.
> Can you?
Oh, of course you can. It's just that every system does it differently. And
beware, since kernel calls are convenient scheduling points and generally
it's possible to end up finding out on which CPU you HAD been running
before you got rescheduled because you asked. ;-)
This isn't an effective way to scope out the system configuration, even if
there were a portable way to ask. And, again, you won't find out anything
about NUMA memory configuration that way.
The best way to do high-performance shared-memory parallelism is to use
OpenMP interfaces (usually in FORTRAN, but many implementations support C
and even C++), which handles the parallelism configuration and distribution
for you. For low or coarse-grained parallelism, or if you're not seriously
compute-bound, you're usually better off letting the OS handle all this for
you anyway.
Eric Sosman
>
> In <400f...@usenet01.boi.hp.com> David Butenhof wrote:
> > Stefan Pantos wrote:
> >
> > Believe it or not, we spent months in POSIX trying to come up with
> > standard interfaces for this sort of information, and couldn't come
> > to any consensus. Every interface that exists is either too simple or
> > too complicated for many systems, and we simply couldn't get a
> > workable compromise. For example, beware that on a NUMA system,
> > knowing "how many" CPUs you have is useless unless you know WHICH
> > cpus you can touch and HOW they're connected to each other. (Actually,
> > it's often WORSE than useless because your simplistic attempts at
> > using the misleading information will result in far worse performance
> > than letting the OS "do its thing".)
> >
>
> I take it that there will never be a way in POSIX to find this
> information out. Thats realy annoying because I don't want to take up
> system resources up by having more than one thread when on a UP but
> would like it to take advantage of the CPUs of an MP.
One thing you seem to have overlooked in David Butenhof's
reply is this bit:
[...] knowing "how many" CPUs you have is useless
unless you know WHICH cpus you can touch [...]
Many systems provide ways to restrict which CPUs a process
can use, and some can even apply different restrictions to
different threads within a single process. So if you query
the system and discover that twenty-four CPUs are installed
and twenty of those are currently on-line, you still don't
know how many CPUs are available TO YOU.
David Butenhof
And, in case it wasn't clear, all this is precisely why the POSIX
discussions got bogged down. There are endless complicated details to
design (much less build) an interface that actually allows you to write
portable high-performance code that cares about this stuff. If your code
doesn't care, then don't complicate things by making it pretend that it
cares. And if it DOES care, then your code is inherently nonportable ANYWAY
because no conceivable (or buildable) interface will allow you to specify
(and the OS to understand) all of the real constraints and relationships
that must matter to it.
Should there be a standard interface for "number of CPUs" anyway, since the
concept is universal? Perhaps. But anyone who uses it for anything other
than 'printf("There seem to be %d CPUs available", number_of_cpus);' is
deluding themselves. It's just not worth standardizing by itself. And as
soon as you try to make the next step of standardizing something useful to
do with the information (e.g., binding to a CPU), you're back into the
unplumbable depths of shared resource management.
Our judgement (this was something like 8 years ago, but the situation hasn't
changed and I think it's unlikely to change) was that any interface we
could actually get through the standards process, and that would be
implementable on real systems, would add complication to the application
and implementation and yet had no hope of solving the real problem.
The interface you REALLY want is pthread_shall_i_create_another_thread(),
returning a boolean TRUE or FALSE depending on whether application resource
demand and system scheduling constraints would allow the addition of
another thread to improve application throughput and/or performance.
Homework: please sketch out a practical implementation in pseudocode. (Then
define "universe" and construct 3 examples. ;-) )
those who know me have no need of my name
>I don't want to take up system resources up by having more than one thread
>when on a UP but would like it to take advantage of the CPUs of an MP.
here's your real desire -- acceptable performance. you were looking at
counting cpu's then starting threads using that as some sort of factor.
instead consider instrumenting your code so that i can obtain some measure
of it's own performance (remembering that this is not static), using that
to adjust. sometimes that's not workable either, but it's about the best
chance that doesn't involve hand tuning the code then mothering the system
as it runs.
--
a signature
Randy Howard
says...
> I take it that there will never be a way in POSIX to find this
> information out. Thats realy annoying because I don't want to take up
> system resources up by having more than one thread when on a UP but
> would like it to take advantage of the CPUs of an MP.
How would you like it to behave on a hyperthreaded machine? That sort
of depends on what type of work is being done in the thread(s) does it
not?
> I was hoping that there was some clever round about way of doing it.
There probably is one for just about every target platform you can
imagine. The issue is that it's not portable, and you'll be writing
a lot of conditional platform-dependent code. Even then, you'll have
to decide how to interpret it "generically" upstream, or write even
more conditional code.
> You don't happen to be able to get which processor a thread is running.
Why? It might change, unless you are using a system which allows threads
to be set (I.e. affinity mask) to certain processor(s).
--
Randy Howard
2reply remove FOOBAR