How to bind a POSIX thread to a specific CPU?
Ioannis E. Venetis
I am developing a multithreaded application using POSIX threads and I
would like to bind each thread to a different processor. I found that
there is the pthread_use_only_cpu() function which does what I need, but
it is available only under Tru64 5.1A and later.
Unfortunately, the system I am using has Tru64 5.1 and this function is
not available. Is there another way to do this? I read some things about
RADs (Resource Affinity Domains), but I am not certain whether they can
be used to achieve what I like to do. To be honest, I am quite confused
about what RADs are and how they are used and I would like to have some
more information about them. Are RADs managed only by administrators?
Can I create RADs from inside my application and bind the threads to
specific processors in a RAD? If so, how is this done? Is there a
pointer to documantation that explains step-by-step the usage of RADs?
Or am I on a totally wrong path to implement binding of threads on
processors?
Thank you in advance for your help,
Ioannis
miki....@email.si
i...@hpclab.ceid.upatras.gr says...
Ioannis E. Venetis
miki....@email.si wrote:
> Hm, I remember somthing like pthread_set_concurency()... see man page
Hello,
Thank you for answering, but it seems that pthread_setconcurrency() does
not do anything on Tru64 5.1. According to the man page it is there only
for Single UNIX Specification, Version 2 source code compatibility and
has no other effect when called.
Moreover, this function just gives a hint to the threads implementation
libary how many kernel-threads (if the implementation uses both kernel
and user-level threads) to have concurrently active. It does not specify
where each kernel-thread will run.
There must be another way.
Best regards,
Ioannis
miki....@email.si
i...@hpclab.ceid.upatras.gr says...
Im not sure, if you can control with POSIX library, where each thread
would go. As I remember, you always have at least two "thread"
libraries, one which is standard POSIX, and maintain its own scheduler,
and the native one, which actually depends on a System kernel
implementation of multithreading.
I suggest to get some document of Tru64 internals (implementation of a
processes/thread).
pthread_set_concurency on a Solaris OS sets number of physical thread
active at any time.
Lets say you have 1000 threads, set_concurency set to 5, and 2 CPU's.
From the POSIX view, you have control only to 5 physical threads.
The same way as NFS daemons, or APACHE workers.
And those 5 would be scheduled to 2 CPU's from the kernel.
But when creating PTHREAD, you have one option (I don't remember which
one), to create physical thread (kernel thread).
But you will probably not be able to send thread to specified CPU.
Im not much into TRU64 unix, but I believe that any multiprocessor
machine has ability to form CPU groups, and dedicate processes to
them.
Hope it helps a little bit..
Loic Domaigne
I have cross-posted to comp.programming.threads, since there are people
hanging there known to be proficient in both (POSIX) threads and Tru64.
Loic.
David Butenhof
"RAD" is the term used by Tru64 UNIX for a "resource locality" on a NUMA
(non-uniform memory) system -- Alpha "Wildfire" and "Marvel" hardware
platforms. On a uniprocessor or traditional SMP system there's only one
RAD. They're hardware-oriented, so there's no way to create or change
them from software; only to control the allocation of resources among them.
As for binding... the first question for Ioannis would be "why would you
want to do that"? The unusual name "use_only_cpu" was carefully chosen;
binding is rarely an advantage to the application, and far more likely
to be harmful. (When you bind to a CPU you're still competing with other
threads for access to that CPU, but you're unable to use another that's
completely idle.) There ARE cases where binding is useful or even
critical; but far more often binding is used by people who not only have
no real idea what they're doing (or why) but who also lack the resources
and knowledge to perform and understand the measurements that would
prove they'd made a mistake.
Anyone who really needs binding on Tru64 UNIX (especially in a NUMA
environment) ought to be running V4.1B anyway, not V5.1; for a lot of
reasons.
If you really can't or won't upgrade, you'll need to figure out the
unsupported and largely undocumented depths of the Mach core functions
in Tru64 UNIX. It is possible, but it's not trivial.
--
Dave Butenhof, David.B...@hp.com
HP Utility Pricing software, POSIX thread consultant
Manageability Solutions Lab (MSL), Hewlett-Packard Company
110 Spit Brook Road, ZK2/3-Q18, Nashua, NH 03062
Dragan Cvetkovic
> Anyone who really needs binding on Tru64 UNIX (especially in a NUMA
> environment) ought to be running V4.1B anyway, not V5.1; for a lot of
> reasons.
>
> If you really can't or won't upgrade,
[snip]
David, is the above a typo (V4.1B instead of V5.1B), or does True64 have
backward numbering schema? :-)
Dragan
--
Dragan Cvetkovic,
To be or not to be is true. G. Boole No it isn't. L. E. J. Brouwer
!!! Sender/From address is bogus. Use reply-to one !!!
loic...@gmx.net
> > Anyone who really needs binding on Tru64 UNIX (especially in a NUMA
> > environment) ought to be running V4.1B anyway, not V5.1; for a lot of
> > reasons.
> >
> > If you really can't or won't upgrade,
>
> [snip]
>
> David, is the above a typo (V4.1B instead of V5.1B), or does True64 have
> backward numbering schema? :-)
speaking of typo, Tru64 is surely a "true UNIX", but doesn't have an
_e_.
Dave might perhaps tell us the history behind this name ;-)
Loic.
Dragan Cvetkovic
> Salut Dragan,
>
>> > Anyone who really needs binding on Tru64 UNIX (especially in a NUMA
>> > environment) ought to be running V4.1B anyway, not V5.1; for a lot of
>> > reasons.
>> >
>> > If you really can't or won't upgrade,
>>
>> [snip]
>>
>> David, is the above a typo (V4.1B instead of V5.1B), or does True64 have
>> backward numbering schema? :-)
>
> speaking of typo, Tru64 is surely a "true UNIX", but doesn't have an
Yes, noticed that. After I posted the article. I'll have to go fetch my
first morning coffee.
David Butenhof
> David Butenhof <david.b...@hp.com> writes:
>
>>Anyone who really needs binding on Tru64 UNIX (especially in a NUMA
>>environment) ought to be running V4.1B anyway, not V5.1; for a lot of
>>reasons.
>>
>>If you really can't or won't upgrade,
>
> [snip]
>
> David, is the above a typo (V4.1B instead of V5.1B), or does True64 have
> backward numbering schema? :-)
Not so much a "typo" as a reality distortion effect induced by
substantially insufficient caffeine concentration in the time interval
closely predating construction of the message in question...
;-)
Ah, yes; that ought to be "V5.1B".
David Butenhof
Sure. We had DEC OSF/1, and life wasn't so bad, but nobody really cared
about "OSF/1" anymore, and the Digital Marketers didn't like anyone
using the "DEC" abbreviation, so it was changed to "Digital UNIX". And
that was pretty good.
But that's a problem for marketing people. A name that MAKES SENSE? I
mean, ANYONE, ANYONE at ALL, can look at that name and figure out that
it refers to a UNIX operating system produced by Digital. What's the
point of a name that MEANS something? Besides, we were on the forefront
of 64-bit computing. Not just complicated 64-bit extensions to a 32-bit
OS, but a real, ah, true, 64-bit computing environment. But then, "True
64 [bit] UNIX" would have the same problems as "Digital UNIX". Someone
might look at it and actually understand what you're talking about
without an interpreter... and that would be bad, right? "Security
through obscurity" may not actually WORK, but it sure is simple... and
if we just misspell the thing a lot of people won't figure out what it
means. And heck, even if you don't know what it means, "Tru64" sounds
kinda cool, right?
Of course, that's a slight misrepresentation of their actual thought
processes. (No, no; we shan't delve into the issue of whether the
misrepresentation is in their favor or not.) Anyway, a name that's not
an actual word (or phrase) really does make a safer and more useful
trademark. (As opposed to common and ordinary everyday words like "VAX",
which turned out to conflict with the trademark of a British vacuum
cleaner company. Although they were able to resolve that dispute with an
agreement that VAX computers remain high quality, reliable, and classy,
so that nobody would be tempted to infringe the vacuum cleaner
producers' registered slogan "VAX sucks".)
loic...@gmx.net
>
> Yes, noticed that. After I posted the article. I'll have to go fetch my
> first morning coffee.
Incidently, you did the same mistake last week on c.u.p.
Guess that you're trying unconsciously to undo marketing's encryption
;-)
Cheers,
Loic.
Dragan Cvetkovic
Well, in my defence, I do know how to spell (and use) creat(2). :-)
Bye, Dragan
Ioannis E. Venetis
Thank you for answering and giving your opinions.
> "RAD" is the term used by Tru64 UNIX for a "resource locality" on a NUMA
> (non-uniform memory) system -- Alpha "Wildfire" and "Marvel" hardware
> platforms. On a uniprocessor or traditional SMP system there's only one
> RAD. They're hardware-oriented, so there's no way to create or change
> them from software; only to control the allocation of resources among them.
This clarifies the general idea behind a RAD and is a good starting
point to understand things better.
> As for binding... the first question for Ioannis would be "why would you
> want to do that"? The unusual name "use_only_cpu" was carefully chosen;
> binding is rarely an advantage to the application, and far more likely
> to be harmful. (When you bind to a CPU you're still competing with other
> threads for access to that CPU, but you're unable to use another that's
> completely idle.) There ARE cases where binding is useful or even
> critical; but far more often binding is used by people who not only have
> no real idea what they're doing (or why) but who also lack the resources
> and knowledge to perform and understand the measurements that would
> prove they'd made a mistake.
I am aware of the complications of binding kernel-threads to CPUs on a
multiprocessor system and I have some experience on these issues
(although certainly not the experience you have). However, (as you
already mentioned) I believe that there are cases where binding a
kernel-thread to a CPU has advantages. Moreover, the application will
not bind kernel-threads to CPUs all the time, but only if the user
requests it. Therefore, the user might deside if it is an advantage to
him/her or not, depending on the execution time in each case.
I have successfully used binding on other platforms (mainly Intel SMP
systems, with and without HyperThreading running Linux) and I believe
that applications that exclusively use the system and are CPU bound have
in general better performance in this case. The same application has
been tested on such systems and had better performance with binding.
Therefore, I would like to test it also on Tru64.
Of course a NUMA architecture is quite different from those Intel
systems and this is one more reason why I would like to experiment with
this. I am just doing some research and experimenting is not bad in this
case.
> Anyone who really needs binding on Tru64 UNIX (especially in a NUMA
> environment) ought to be running V4.1B anyway, not V5.1; for a lot of
> reasons.
Unfortunately, I am only using the system and can't upgrade the OS.
> If you really can't or won't upgrade, you'll need to figure out the
> unsupported and largely undocumented depths of the Mach core functions
> in Tru64 UNIX. It is possible, but it's not trivial.
These are bad news (at least for me :-) ). Anyway, it seems that I will
have to live without this feature in my application under Tru64 UNIX or
wait for an OS upgrade.
Best regards,
Ioannis