posix threads, pthread_yeld
fabio
i tried pthread_yield() and i saw that it doesn't work to me, thread
will not switch to another.. i think that maybe it doesn't work
for the standard pthread scheduling type (not real time), so if i
change it, is it supposed to work?
Thx!
Faz
Ian Collins
Was there another thread ready to run? How did you test, posing some
code would help.
--
Ian Collins.
loic...@gmx.net
As asked by Ian, could you be a bit more specific?
The following program runs as expected on my particular implementation
(Fedora core 3 / NPTL), even if the default time sharing scheduling is
used:
#include <pthread.h>
#include <unistd.h>
#include <stdio.h>
void*
thread(void* name)
{
for (;;) {
printf ("Thread %s is running\n", (char*) name);
pthread_yield();
}
}
int
main()
{
pthread_t t1,t2;
pthread_create(&t1, NULL, thread, "1");
pthread_create(&t2, NULL, thread, "2");
sleep (3);
return 0;
}
HTH,
Loic.
Thomas Maier-Komor
WTF is pthread_yield? Are you talking about sched_yield()? Is
pthread_yield a a Linux invention?
I cannot find pthread_yield on the opengroup's page
(http://www.opengroup.org/onlinepubs/009695399)...
Tom
David Schwartz
"Thomas Maier-Komor" <maie...@lpr.e-technik.tu-muenchen.de> wrote in
message news:dstp3p$9ef$1...@wsc10.lrz-muenchen.de...
> WTF is pthread_yield? Are you talking about sched_yield()? Is
> pthread_yield a a Linux invention?
A few UNIX-type OSes have a 'pthread_yield' function that does roughly
the same thing as 'sched_yield'. Its use should be strongly discouraged
(which was probably what you were trying to do, just a bit too subtly)
because it is neither standard nor standardized. It may not be available and
even if it is, it might not do what you expected it to do.
DS
Dave Butenhof
pthread_yield() was the standard POSIX function to yield to another
thread within the same process during the DRAFTS of the document that
eventually became POSIX 1003.1c-1995.
It wasn't until we merged the separate text with the base document
(1003.1b-1993 at the time) and rationalized that for the presence of
threads that we realized the text of pthread_yield() and sched_yield()
were now identical. This seemed silly and pointless, and we deleted
pthread_yield() from the document.
That is, pthread_yield() was never standard, but it existed in drafts at
least through something like draft 7 or 8 (I'm home and don't have my
draft archives to check). So if you have an implementation of draft 4
(DCE threads), draft 5 (someone did it, I think), or draft 7 (HP-UX
prior to 10.30), OR if you have a true POSIX threads implementation
that's trying to be friendly and accommodate older code (e.g., ported
from draft 4/5/7) it might implement pthread_yield() as an "extension".
In any case, pthread_yield() needn't do anything at all EXCEPT when your
threads are using POSIX realtime scheduling policies (SCHED_RR,
SCHED_FIFO) on a uniprocessor. In any other situation, consider
pthread_yield() to be nothing more than a "scheduler hint" saying
essentially "Oh mighty scheduler, shoulds't thou thinkest in thine
wisdom that thy might taketh from me my processor in the near future, I
beseech thou to do it now."
And it may do it, or it may shrug its little virtual shoulders and go on
about its business. You don't know, your code can't tell, and it doesn't
make any difference. Or rather -- if it DOES make any difference, reset
your expectations and figure out what you really meant to do instead.
Really, if you're depending on pthread_yield() to do anything, you've
probably got a logic problem. Scheduling is not synchronization; it's
just scheduling. There are a few cases where a yield may improve
performance slightly, but they're rare and the benefit is generally
pretty small at best. (And mostly when the code does really nasty things
like hold a mutex across most of a tight loop -- a yield between the
unlock and re-lock can help other threads make progress. But you
shouldn't lock in that pattern, and if you do you shouldn't depend on
other threads making progress. ;-) )
Thomas Maier-Komor
Thanks for the historical background. I really just wanted to say "don't
use it, it is a non-standard function", but historical background
information like this is always interesting, and it makes some issues
and differences between systems easier to understand...
Cheers,
Tom
Chris Vine
> In any case, pthread_yield() needn't do anything at all EXCEPT when your
> threads are using POSIX realtime scheduling policies (SCHED_RR,
> SCHED_FIFO) on a uniprocessor. In any other situation, consider
> pthread_yield() to be nothing more than a "scheduler hint" saying
> essentially "Oh mighty scheduler, shoulds't thou thinkest in thine
> wisdom that thy might taketh from me my processor in the near future, I
> beseech thou to do it now."
Come on now. The scheduler would get completely confused. The correct text
to send it would be: "Oh mighty scheduler, shouldest thou think in thine
wisdom that thou mightest take from me my processor in the near future, I
beseech thee to do it now".
Chris
--
To reply by e-mail remove the --nospam-- in the address.
dick.dunbar
and to provide a proper response for the question, we really have to
know what OS this is being run on.
My experiments with Solaris 8 and below, all showed a very VERY unfair
scheduler.
It's like the old Smothers Brothers schtick ....
Dick: "Take it Tommy"
Tom: "No!"
That issue completely disappeared with Solaris 10.
David Schwartz
"dick.dunbar" <dick....@gmail.com> wrote in message
news:1140054188.5...@g43g2000cwa.googlegroups.com...
> My experiments with Solaris 8 and below, all showed a very VERY unfair
> scheduler.
Unfair to processes or unfair to threads? If the former, that's not
good. If the latter, who cares how fair it is--performance is more
important. You can code fairness if you need it, you can't add back in
performance.
DS
dick.dunbar
Unfair to thread scheduling. It is important to performance that
threads make progress. If one thread is allowed to dominate, through
no fault of it's own, you will produce a very erratic responding
application.
You cannot code fairness if the scheduer is inherently unfair.
You can be a nice guy all day long ... to the deteriment of your own
progress.
Windows is a classic case that scheduling is the application's
responsibiilty.
We (me) don't expect unix platforms to act that way.
Your remark caught me off-guard ... maybe I missed something, but the
statement that an application can control fairness is just wrong in my
experience. You're at the mercy of the underlying OS. I spend a lot
of time writing toy programs to punish systems and see how they behave.
The Solaris situation was exposed when our app didn't run well ...
responsively ... and that's the only performance I care about.
I'll put it this way ... if your app doesn't perform, and you buy
faster hardware to meet expectations ... and the performance degrades
instead of improves ...
You have one of two situations:
- The app is coded incorrectly for threads
- The underlying OS is not doing a good job, leading to mutex locks and
all sorts of crazy abherent behaviour.
Joe Seigh
>>who cares how fair it is--performance is more important.
>
>
> Unfair to thread scheduling. It is important to performance that
> threads make progress. If one thread is allowed to dominate, through
> no fault of it's own, you will produce a very erratic responding
> application.
>
> You cannot code fairness if the scheduer is inherently unfair.
> You can be a nice guy all day long ... to the deteriment of your own
> progress.
> Windows is a classic case that scheduling is the application's
> responsibiilty.
> We (me) don't expect unix platforms to act that way.
>
> Your remark caught me off-guard ... maybe I missed something, but the
> statement that an application can control fairness is just wrong in my
> experience. You're at the mercy of the underlying OS. I spend a lot
> of time writing toy programs to punish systems and see how they behave.
> The Solaris situation was exposed when our app didn't run well ...
> responsively ... and that's the only performance I care about.
SCHED_OTHER doesn't claim to be fair. It allows for better
performance as long as you keep system utilization below a
certain level.
SCHED_FIFO and SCHED_RR have better forward progress guarantees
but at an overall cost of reduced performance.
There are also factors that dramatically increase context switching
overhead such as paging or lock contention. If this happens the
system usually doesn't recover since the work load has to be
reduced considerably below the the point performance started to
degrade at. In fact considerably below normal system load. Almost
to the point of quiescing the application. Most people never try
this and restarting the application is the usual response.
--
Joe Seigh
When you get lemons, you make lemonade.
When you get hardware, you make software.
dick.dunbar
> performance as long as you keep system utilization below a
> certain level.
> SCHED_FIFO and SCHED_RR have better forward progress guarantees
> but at an overall cost of reduced performance.
Those are pthread hints. The OS does not have to take hints.
It's my believe Solaris 8 was buggy ... I reported the bug ... it was
fixed in Solaris 10.
At the end of the day, the OS is scheduling threads ... full-stop.
It doesn't matter whether those threads are packaged in different
processes,
or in the same process. Threads are the scheduling entity for a cpu,
not a process.
Chris Friesen
>>SCHED_OTHER doesn't claim to be fair. It allows for better
>>performance as long as you keep system utilization below a
>>certain level.
>
>
>>SCHED_FIFO and SCHED_RR have better forward progress guarantees
>>but at an overall cost of reduced performance.
>
>
> Those are pthread hints. The OS does not have to take hints.
Huh? Those are scheduling policies, not hints. Their behaviour is well
documented. If you are in SCHED_RR and call sched_yield(), you *will*
be put at the end of the runqueue after all other tasks with the same
static priority.
sched_yield() with SCHED_OTHER is a bit trickier, but linux at least has
decided that there is only one static priority for SCHED_OTHER (zero),
so sched_yield() puts you after all other runnable tasks.
> At the end of the day, the OS is scheduling threads ... full-stop.
> It doesn't matter whether those threads are packaged in different
> processes,
> or in the same process. Threads are the scheduling entity for a cpu,
> not a process.
No. I can prove it by counterexample, as linux schedules "tasks", which
may or may not be grouped into "thread groups". When "thread groups"
share certain attributes (and they don't necessarily have to) then the
component tasks behave like POSIX threads.
With SCHED_OTHER the scheduler can decide to vary timeslice, dynamically
adjust task priority, use task selection heuristics, etc., all in the
name of throughput.
With the soft realtime policies, predictability is more important than
performance.
Chris
dick.dunbar
> be put at the end of the runqueue after all other tasks with the same
> static priority.
Absent bugs, we agree; that's what it's for.
Recall that we're also talking several iterations on the pthreads
standards, so there was lots of opportunity for vendors to fall
behind in multiple versions of the threads support.
> linux schedules "tasks", which may or may not be grouped into "thread groups".
Fair enough. I was thinking of a real operating system. :-) Just
kidding, I like linux
> With the soft realtime policies,
Here come the standards again. DavidB can keep my honest on the
history,
but posix groups were split between the "threads" people and the "real
time"
people, with some overlap.
When pthreads was ready, the real-time folks tried to add their API's
into the mix. They got rebuffed by some vendors ... and the RT
features
were added as "Optional". You could claim pthreads conformance,
without implementing the real-time facilities.
It's one of the reasons we have this odd-ball functions "sched_yield",
when every threads programmer is looking for the "pthreads_" version of
that.
Happily, the vendor planets are lining up, and most of the major
players
have added the real-time features. You have to read the vendor docs
carefully here ... all they were required to do was implement a stub
function, so the linker wouldn't complain about the app, and to put in
some magic flags in their header files, and run-time descriptions on
whether the function was actually supported.
History lessons aside, I still claim that what is running on a physical
cpu is a thread, in a modern OS. How they are grouped and dispatched
is another matter.
We also agree that the standards writers have given us the proper
thought models to implement what we intend ... and many modern
apps are expoiting real-time features, including NASA scarfing
LandRover data from space.
Dave Butenhof
>> If you are in SCHED_RR and call sched_yield(), you *will*
>> With the soft realtime policies,
>
> Here come the standards again. DavidB can keep my honest on the history, but
> posix groups were split between the "threads" people and the "real >
time" people, with some overlap.
In truth there were at least 3 easily identifiable "camps", but it's
also important to remember that the realtime people were the HOSTS. That
is, the thread people saw an opportunity in some realtime needs to get
threads into POSIX. All of the POSIX thread development was in the
context of the "1003.4" (realtime) working group.
The realtime people were finishing the realtime amendment, and they had
a substantial investment in the priority based scheduling policies. It
was natural for them to want those in the threads amendment as well.
> When pthreads was ready, the real-time folks tried to add their API's
> into the mix. They got rebuffed by some vendors ... and the RT
> features were added as "Optional". You could claim pthreads conformance,
> without implementing the real-time facilities.
No, the realtime interfaces were there all along. In fact the original
reference document original proposed had somewhat similar scheduling
interfaces -- priority based scheduling is hardly unique to POSIX. (And
in fact the specific POSIX base rule is to standardize "existing
practice", NOT to invent.)
It's hard to make a multiprocessor thread realtime scheduler that's also
fast; it requires a lot of synchronization that the scheduler might not
otherwise need. Realtime scheduling is an option in 1003.1b as well,
(for processes); NOT just for threads. The realtime people never
intended to force everyone to write realtime code; they only wanted to
be assured that THEY could.
> It's one of the reasons we have this odd-ball functions "sched_yield",
> when every threads programmer is looking for the "pthreads_" version of
> that.
Oddly, I just re-explained this last week in this discussion thread; but
perhaps you missed it. There WAS a pthread_yield(). But there was also
sched_yield from the previous realtime (process) amendment. Most of the
development and balloting of the threads amendment took place on a
completely separate document. But when we got close to finalization we
went through the important exercise of creating a full POSIX document
with threads integrated. One of the trickier bits of this merge was
figuring out where in the base standard "process" should be changed to
"thread". (We're still finding places where it should have been changed
but wasn't... or even where it was changed but shouldn't have been.)
In particular, on review we noticed that because it was critical that a
POSIX system with thread support schedules threads, not processes, the
descriptions of pthread_yield() and sched_yield() were now identical
except that sched_yield existed and had meaning without threads. So
there was clearly no point in adding pthread_yield(), and it was removed
from the standard.
That people "expect" to find pthread_yield() is largely the fault of
vendors who "jumped the gun" and implemented a non-standard draft
interface. They shouldn't have done this, and it's not the fault of
POSIX or the thread working group that the final form is not the same as
those drafts. (Which were explicitly published "for review and comment
only", and distributed only to a limited audience who had a reason and
desire to review it.)
> Happily, the vendor planets are lining up, and most of the major
> players
> have added the real-time features. You have to read the vendor docs
> carefully here ... all they were required to do was implement a stub
> function, so the linker wouldn't complain about the app, and to put in
> some magic flags in their header files, and run-time descriptions on
> whether the function was actually supported.
If realtime features aren't implemented, they're return ENOSYS; and
there are also sysconf() options and <unistd.h> symbols to detect this
at runtime or even compile time. It's not that hard, and it's not
particularly "tricky".
What was annoying was certain vendors who thought it'd be better to
CLAIM to support realtime but not bother to make the functions actually
work. (Making it difficult for developers or programs to determine
reliably whether the functions would result in any usable effect.) I
believe those implementations are becoming lost in the mists of time,
and I won't name names.
> History lessons aside, I still claim that what is running on a physical
> cpu is a thread, in a modern OS. How they are grouped and dispatched
> is another matter.
>
> We also agree that the standards writers have given us the proper
> thought models to implement what we intend ... and many modern
> apps are expoiting real-time features, including NASA scarfing
> LandRover data from space.
Priority scheduling is only a rough approximation to what anyone really
wants. It's just simpler and cheaper to implement (and use) than
anything better; and better solutions tend not to be widely deployed at
this time. (And remember POSIX standarizes "existing practice".)
Managing prioritized resources is extraordinarily difficult even in a
relatively contained embedded application environment, and nearly
impossible otherwise.
It can be used productively by people who are careful enough and
determined. Don't mistake that for "correct" or "general". ;-)
dick.dunbar
Dave, I read everything you write in this newsgroup.
To paraphrase George Patton: " I read your damn book, you elegant
bastard"
Also faithfully follow DS and JS.