Why are pthreads horrible?
Bryan Ischo
implementation, I have run across many sentiments expressed by various
people that "pthreads are horrible". I'm trying to understand what
causes people to say this. As an example, on the Linux NTPL page
(http://www-124.ibm.com/developerworks/oss/pthreads/) Linus Torvalds
is quoted as saying:
"pthreads are horrible, and Linux has a very different model, and
there was no glue between the two."
I personally think the pthreads API is great. I think the
'attributes' concept is a little clumsy but that is such a minor part
of it all. I find that the pthreads API seems to boil threading down
to the very minimal set of operations needed to write effective and
correct threaded code. I don't understand the bad feelings towards
pthreads (although I am finding that the implementations often leave
alot to be desired, but I don't think that's the fault of the pthreads
API itself).
Is it that people who don't like pthreads just don't like threaded
programming? Do they not like the fact that it provides such bare
minimum threading constructs and expect you to write higher-level
abstractions (semaphores, etc) yourself? I happen to think that an
API which is minimal is best especially when it comes to such a very,
very complex subject such as threading, and when the API has to be
ported across so many different platforms.
I personally find Linux's clone-based threads to be "horrible" but I
guess Linus would disagree :) ...
Thanks!
Bryan
Sean Burke
bji...@ischo.com (Bryan Ischo) writes:
I think Linus' objection was that pthreads violated the "layering"
that we normal enforce in unix software. Clone-based threads were
good from this point of view, because they required nothing in the
way of support from the kernel, and the kernel could preserve a
comparatively simple view of the user process. The kernel did not
need to care whether LinuxThreads or some other threading system
was in use.
Implementing NPTL pthreads did require significant revisions to
the kernel. A fundamental principle of unix is to keep the interface
between kernel and userspace small and manageable, so the kernel
team had justifiable qualms about it.
-SEan
Loic Domaigne
> Hi all. Another question. In researching the Linux pthreads
> implementation, I have run across many sentiments expressed by various
> people that "pthreads are horrible". I'm trying to understand what
> causes people to say this. As an example, on the Linux NTPL page
> (http://www-124.ibm.com/developerworks/oss/pthreads/) Linus Torvalds
> is quoted as saying:
> "pthreads are horrible, and Linux has a very different model, and
> there was no glue between the two."
Linus and some of his major co-developpers have in the past critized
Pthreads (I mean the POSIX Standard) as "brain damaged pile of crap",
something like that. Never really understood that particular position,
that wasn't never technically justified BTW...
Linus fought like hell to avoid the introduction of everything related
to threads at the kernel level... For him, all that "crap" belonged to
user space and the sole kernel support needed was clone(). That was
his position back to 1996 or something.
It just took about 6 years to admit that this point of view wasn't
acceptable, and yes, we need other kinds of support in the kernel.
In 2002, Ulrich Drepper, Ingo Molnar and co. began to develop the
needed kernel supports and library for a clean Pthreads support.
Which is now that famous NPTL.
And the funny in that story is that NPTL has introduced many big
advances in the kernel, first ahead likely the O(1) scheduler. In
older 2.4.x time, the time to schedule N tasks were linear in N.
Now with the O(1), the task scheduling is constant, independantly to
the number of tasks to schedule...
> I personally think the pthreads API is great.
So do many persons here on c.p.t ;-)
> I think the 'attributes' concept is a little clumsy but that is
> such a minor part of it all. I find that the pthreads API seems
> to boil threading down to the very minimal set of operations needed
> to write effective and correct threaded code.
At the API level, they are quite uniform. But, yes, the options
supported/not supported vary from platform to platform, because the
standard didn't wanted to put "stress on existing implementations"...
So far, I never got real troubles (other posters may have a quite
different viewpoint on this).
> I don't understand the bad feelings towards pthreads (although I
> am finding that the implementations often leave alot to be desired,
> but I don't think that's the fault of the pthreads API itself).
I believe that Linus flunched his exam on Pthreads. Or maybe his
girlfriend left him for a guy in the Posix 1003.1c standard commitee,
something like that ;-)
> I personally find Linux's clone-based threads to be "horrible" but I
> guess Linus would disagree :) ...
I find the concept of clone() great, because it offers an orthogonal
design. You can fine tune what you want to share with your clone:
Virtual Memory, set of file descriptors, signal handler... That's a
nice idea.
The thing that was missing was that a process in a set of clones
("threads"), and that all those clones are peers (no parent-child
relationship)...
Oh BTW, if you are interested in looking at OS that tooks exactly
the opposite direction from Linux (namely "Everything is threads"),
then you should look at the design of Mach.
Regards,
Loic.
--
Article posté via l'accès Usenet http://www.mes-news.com
Accès par Nnrp ou Web
Loic Domaigne
> I think Linus' objection was that pthreads violated the "layering"
> that we normal enforce in unix software.
That's an interesting view I wasn't aware of... But not likely if
Linus really said:
<copy>
Linus Torvalds: Look at Next Generation POSIX Threads (NGPT) for the future
of threads, he advised. "pthreads are horrible, and Linux has a very
different model, and there was no glue between the two." NGPT could be that
glue.
</copy>
AFAIK, NGPT is a M:N implementation, and the effective way to do it
is to use upcall mechanism. That mechanism brokes effectively the
classical layering...
However, I'm not entirely sure that this holds for a 1:1 model... And
I would tend to say no... (and well, a well known Pthreads authority
that hangs regularly on c.p.t. is invited to quote here ;-)
Sure, I greatly appreciate all what Linus did. But I still believe
that he was not totally right regarding the Threading aspect.
Of course, Pthreads have some deficiency like:
- current working directory.
- fork() and threads.
- etc.
But, yeah... who's perfect? Things get progressively fixed...
My 2cents,
Loic Domaigne
> > I think Linus' objection was that pthreads violated the "layering"
> > that we normal enforce in unix software.
> That's an interesting view I wasn't aware of...
[snip]
> However, I'm not entirely sure that this holds for a 1:1 model... And
> I would tend to say no... (and well, a well known Pthreads authority
> that hangs regularly on c.p.t. is invited to quote here ;-)
Ok, reread quickly the relevant parts in Vahalia. Except the well known
places where traditional (non threaded) Unix process doesn't fit well
with multi-threaded process, I do not really see any issue about the
layering you are referring to.
For instance, it seems Digital Unix had a nice design that extended
the process abstraction in order to include thread abstraction [ guess
it is more or less still present in Tru64 ]. And it doesn't seem to
break that layering stuff.
Regards,
Sean Burke
Loic Domaigne <loic...@gmx.net> writes:
> Hi Sean,
>
> > I think Linus' objection was that pthreads violated the "layering"
> > that we normal enforce in unix software.
>
> That's an interesting view I wasn't aware of... But not likely if
> Linus really said:
>
> <copy>
> Linus Torvalds: Look at Next Generation POSIX Threads (NGPT) for the future
> of threads, he advised. "pthreads are horrible, and Linux has a very
> different model, and there was no glue between the two." NGPT could be that
> glue.
> </copy>
I'm surprised that Linus said that. My mistake for presuming
to speak on his behalf.
> AFAIK, NGPT is a M:N implementation, and the effective way to do it
> is to use upcall mechanism. That mechanism brokes effectively the
> classical layering...
>
> However, I'm not entirely sure that this holds for a 1:1 model... And
> I would tend to say no... (and well, a well known Pthreads authority
> that hangs regularly on c.p.t. is invited to quote here ;-)
My guess is that the NPTL and the futex mechanism found favor
because the kernel interface is quite small and very general.
> Sure, I greatly appreciate all what Linus did. But I still believe
> that he was not totally right regarding the Threading aspect.
>
> Of course, Pthreads have some deficiency like:
>
> - current working directory.
> - fork() and threads.
> - etc.
>
> But, yeah... who's perfect? Things get progressively fixed...
Yes, they do. It's amazing how much easier it has
become to write portable threaded code. Now if Apple
would only get their issues corrected...
-SEan
Sean Burke
Loic Domaigne <loic...@gmx.net> writes:
> Hello again,
>
> > > I think Linus' objection was that pthreads violated the "layering"
> > > that we normal enforce in unix software.
>
> > That's an interesting view I wasn't aware of...
>
> [snip]
>
> > However, I'm not entirely sure that this holds for a 1:1 model... And
> > I would tend to say no... (and well, a well known Pthreads authority
> > that hangs regularly on c.p.t. is invited to quote here ;-)
>
> Ok, reread quickly the relevant parts in Vahalia. Except the well known
> places where traditional (non threaded) Unix process doesn't fit well
> with multi-threaded process, I do not really see any issue about the
> layering you are referring to.
>
> For instance, it seems Digital Unix had a nice design that extended
> the process abstraction in order to include thread abstraction [ guess
> it is more or less still present in Tru64 ]. And it doesn't seem to
> break that layering stuff.
Yes, I didn't mean to imply that POSIX threads were
some sort of corruption of classical unix. Just that
in the unix world, we understand the value of layered
systems with clean interfaces between layers.
So, because POSIX thread support required extensions
to the kernel/userspace interface, it came in for
added scrutiny. By contrast, changes in areas of the
kernel that aren't visible in userspace have a much
lower bar for acceptance.
-SEan
David Schwartz
"Bryan Ischo" <bji...@ischo.com> wrote in message
news:d976b79f.04031...@posting.google.com...
> "pthreads are horrible, and Linux has a very different model, and
> there was no glue between the two."
My recollection was that at the time, there was a discussion of what
kernel hackery would be needed to support 'features' of the pthreads
standard that are, at best, questionable. Most of these 'features' are ways
that certain operations affect the entire process, like 'seteuid'.
DS
David Butenhof
Ultimately, the disparity comes from the fact that clone() was a poor
implementation of its theoretical goals. In the limit, and I believe in
intent, clone accomplishes the same thing as the mach scheduling model,
which is to separate "execution entities" from "process context" in a way
that, among other goals, allows combining several execution entities within
the same process context. Clone tries to support all the substates in
between the extremems of "traditional UNIX process" and "fully threaded
process" with many execution entities and one shared "process context" --
in my mind that's wildly overgeneralized, as most of those intermediate
combinations have little or no real value. The Mach model is far simpler
and more straightforward: all the "process context" is in the Mach TASK
object, to which you can attach any number of execution entities (threads).
A "traditional process" is simply one thread attached to one task.
Pragmatically, the problem is that until NPTL, clone just didn't work right.
I don't know whether it does now, or if they're still hacking around the
weaknesses. With a correct and complete implementation of clone, however,
all the "process state" (pid, uid, etc) resides in one common place shared
by all the execution entities. Any process state needs to be changed in
only one place, no matter how many execution entities are connected.
--
/--------------------[ David.B...@hp.com ]--------------------\
| Hewlett-Packard Company Tru64 UNIX & VMS Thread Architect |
| My book: http://www.awl.com/cseng/titles/0-201-63392-2/ |
\----[ http://homepage.mac.com/dbutenhof/Threads/Threads.html ]---/
David Schwartz
"David Butenhof" <David.B...@hp.com> wrote in message
news:4059...@usenet01.boi.hp.com...
>> My recollection was that at the time, there was a discussion of what
>> kernel hackery would be needed to support 'features' of the pthreads
>> standard that are, at best, questionable. Most of these 'features' are
>> ways that certain operations affect the entire process, like 'seteuid'.
> Pragmatically, the problem is that until NPTL, clone just didn't work
> right.
> I don't know whether it does now, or if they're still hacking around the
> weaknesses. With a correct and complete implementation of clone, however,
> all the "process state" (pid, uid, etc) resides in one common place shared
> by all the execution entities. Any process state needs to be changed in
> only one place, no matter how many execution entities are connected.
While I agree with you, it's possible that you're missing my point. Yes,
it's not hard to implement things such that 'seteuid' works for the entire
process, all you have to do is make sure all the threads have only one
structure of some kind and that this structure contains the effective user
ID.
Linus' point, I think, was that this was bad design. Each thread should
have its own euid. Consider a program like Samba, for example.
Linus' philosophy was to make it easy to do things the right way, not to
add kernel features to make it easy to do things the wrong way, standard or
no. He felt that in this case as well as others, the pthreads standard only
specified that these operations were process wide to support implementations
that couldn't do things the way that he felt was right.
DS
Loic Domaigne
> While I agree with you, it's possible that you're missing my point.
> Yes, it's not hard to implement things such that 'seteuid' works for
> the entire process, all you have to do is make sure all the threads
> have only one structure of some kind and that this structure contains
> the effective user ID.
> Linus' point, I think, was that this was bad design.
> Each thread should have its own euid. Consider a program like Samba,
> for example.
And that's a perfectly viable point. Actually, I have no problem with
an implementation that doesn't follow or implement brain-damaged features
of a standard because "it's the standard". I have however troubles with
people that rejects everything because there is a few weakness or
unaesthetic points. We can't reasonably see everything in term of black
and white (hmm, well, or I should say 0 and 1 ;-)
Taking the euid example. I believe each thread could have its own euid
+ a shared euid for the process. And Linux could have offer a (non standard)
version seteuid that change only the TS euid. You could go further, and
stick to the standard if you like (for portability reason from other
platforms), by providing the standard seteuid that would manipulate the
process euid.
(Ok, that's just my feeling. Thinks might be in reality much harder. That's
always easy to do theoritical programming ;-)
> Linus' philosophy was to make it easy to do things the right way,
> not to add kernel features to make it easy to do things the wrong way,
> standard or no.
> He felt that in this case as well as others, the pthreads standard only
> specified that these operations were process wide to support
> implementations that couldn't do things the way that he felt was right.
And that's OK. I find Linus intend with clone() an interesting approach to
threads. But I feel that he missed what his clone() could have offered
to Linux if he would have been so bullheaded regarding (P)threads.
I mean, how come that I had to wait 2003 to have a correct signalling
semantic? To have pshared mutex / Posix sema? ...
But yes, maybe I just have too high expectations. Frankly, I enjoy Linux
for years now! ... But I still love other unix as well ( since, so far
there's no OS monogamy ;-)
Jeremy Allison
> Linus' point, I think, was that this was bad design. Each thread should
> have its own euid. Consider a program like Samba, for example.
And he was right about that :-). There are some horrible things Samba has
to do to work correctly on POSIX. Don't get me started on the braindamage
that is the locking model.... :-) :-).
Jeremy Allison,
Samba Team.
Casper H.S. Dik
>David Schwartz wrote:
As a security person I balk at the semantics of multiple effective
uids per thread; note also that is would be completely undoable with
a two-level thread model and that it is also extremely hard to combine
with something like signal handlers. (The signal handler runs with
which effective uid)?
And questions like "what is the representative euid" (i.e., surely
users shouldn't be able to kill smbd just because there's a thread running
with their euid)
POSIX threads are the perfect abstractions for threading: they only
abstract the execution context but keep the rest of the process model
intact.
A number of system calls, however, make little sense in threaded processes:
chroot(), chdir(), set*id()
but saying that geteuid() can return different values is almost like saying
that file descriptors returned by open() shouldn't be shared between
threads.
Ah yes, the Samba locking model is a mess :-)
Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.
Loic Domaigne
JA: Jeremy Allison
DS: David Schwartz
DS> >> Linus' point, I think, was that this was bad design. Each thread
DS> >> should have its own euid. Consider a program like Samba, for
DS> >> example.
JA> > And he was right about that :-). There are some horrible things
JA> > Samba has to do to work correctly on POSIX. Don't get me started on
JA> > the braindamage that is the locking model.... :-) :-).
CS> As a security person I balk at the semantics of multiple effective
CS> uids per thread; note also that is would be completely undoable with
CS> a two-level thread model and that it is also extremely hard to combine
CS> with something like signal handlers. (The signal handler runs with
CS> which effective uid)?
[snip]
Ok, set*id() might be not the right example...
CS> A number of system calls, however, make little sense in threaded
CS> processes:
CS>
CS> chroot(), chdir(), set*id()
For chdir(), I'm not sure (?) IIRC, there has been a discussion on
austin-group-future regarding per-thread CWD. Wouldn't it make any sense
to have possibly different CWD per thread ??
I can't see any reason why not, but gee, I'm definitively not an expert
in that area!
Independantly, I believe that Linus intend was right. But I still
believe too that he missed - for a long time - the fundamental point...
As expressed *rightly* by D.R. Butenhof:
<quote>
Libpthread isn't just an add-on library; it's a part of the system, just
like libc. Inseparable and deeply intertwined. An extension of the kernel
into user space, really.
</quote>
Linus forced the separation to its zenith, namely one syscall: clone().
With the result we know...
Patrick TJ McPhee
Loic Domaigne <loic...@gmx.net> wrote:
% For chdir(), I'm not sure (?) IIRC, there has been a discussion on
% austin-group-future regarding per-thread CWD. Wouldn't it make any sense
% to have possibly different CWD per thread ??
% I can't see any reason why not, but gee, I'm definitively not an expert
% in that area!
It seems to me the objective with threads is to have a light-weight
execution context which reduces the overhead of parallel computing. If you
want to start adding crap into the threads, you ought to have a reason
for it. The question is not `why shouldn't such and such be included
as part of a thread?', but `why should it?'.
I think it's easy to underestimate the difficulty of developing a standard
like posix, and it's especially easy to sit on the side-lines saying
`I'm not doing that bit because I think it's dumb'. I don't think it's
especially helpful to do that.
--
Patrick TJ McPhee
East York Canada
pt...@interlog.com
Christoph Hellwig
> It seems to me the objective with threads is to have a light-weight
> execution context which reduces the overhead of parallel computing. If you
> want to start adding crap into the threads, you ought to have a reason
> for it. The question is not `why shouldn't such and such be included
> as part of a thread?', but `why should it?'.
I think you're pretty much missing the point. The plan9 rfork model
after which Linux's clone() or IRIX's sproc() are (more or less)
modelled don't try to follow the "lets create multiple execution
contexts inside a process" model, but are rather designed to allow
multiple processes to share certain attributes.
So it's not a new entinity inside a process but rather a generalization
of the tradition UNIX process scheme, see the "Parallel programming" in
http://www.cs.bell-labs.com/sys/doc/9.html for some more details.
In Linux for example vfork() and fork() are just wrappers around the
clone infrastructure. and clone() can be used to create new processes
that don't share the filesystem namespace with the parent, etc..
Now this is a very different (and in my and Linus' optinion) nicer model
than the "Threads" concept. But outside Plan9 it has been only used
very little. IRIX at least has a bunch of helper routines for
synchronization & co around sprocs and SGI-own software makes quite a
a lot use of it, in Linux OTOH it's missing (so Linus' interested in
that model seems to be a little half-assed at least) so there's very
little use, I only know of a handfull programs exploiting various clone
features.
So while rfork() might or might not be the nicer model it has failed in
wild, and I think Linus finally realized that. After all Linux isn't
about some academic daydream but has always been software that was
supposed to be useable, so it was time to finally bite the bullet and
properly support what the marketplace wants.
Patrick TJ McPhee
Christoph Hellwig <h...@engr.sgi.com> wrote:
% On Sat, Mar 20, 2004 at 11:00:22PM +0100, Patrick TJ McPhee wrote:
% > It seems to me the objective with threads is to have a light-weight
% > execution context which reduces the overhead of parallel computing.
[...]
% I think you're pretty much missing the point.
I don't. I do think you quoted the wrong part of my message. This was
addressed at the question `why shouldn't there be a per-thread
working directory?'
wrt clone(), I don't have a problem with its objectives and I don't
have an opinion as to whether it's better or worse than the thread
model. I do think that standardisation is useful. I recognise that there
are costs to adhering to standards, and I won't hold it against anyone
for not doing it, but I reserve the right to say that refusing to support
a widely promulgated standard is not especially helpful, and that refusing
to do so simply because you think you have a better way of doing things is
vainglorious, and it's the wrong reason for making that decision.
Loic Domaigne
> % For chdir(), I'm not sure (?) IIRC, there has been a discussion on
> % austin-group-future regarding per-thread CWD. Wouldn't it make any sense
> % to have possibly different CWD per thread ??
> % I can't see any reason why not, but gee, I'm definitively not an expert
> % in that area!
> It seems to me the objective with threads is to have a light-weight
> execution context which reduces the overhead of parallel computing. If you
> want to start adding crap into the threads, you ought to have a reason
> for it. The question is not `why shouldn't such and such be included
> as part of a thread?', but `why should it?'.
I agree with that. The point discussed between some posters and me is that
*all* things proposed by a standard are necessarily fine from a technical
standpoint. Most are (especially for standard like Posix and SUS that are
worked out by fine people), but not all are.
I guess, being constructive isn't necessary something bad: things that
might be improved should be improved whenever possible. This approach
offers the opportunity for an existing standard to mature...
> I think it's easy to underestimate the difficulty of developing a standard
> like posix,
True. But, you should admit that such difficulties arise among others
because many parties - that might have already an implementation - want
their solutions becoming the standard...
... And so, this turns sometimes into many battles where "compromises" have
to be sought, hurting sometimes the solution's quality...
> and it's especially easy to sit on the side-lines saying `I'm not doing
> that bit because I think it's dumb'.
Yeah, I see what you mean. Seems to be BTW a standard way of thinking by
some major LK developpers...
Though I dislike that attitude, I dislike equally the one that follows
blindly "a standard" ( or "a fashion" or whatever mass people assumes
to be "the right way of doing X" ).
And no - I'm not a reactionary or something ;-)
> I don't think it's especially helpful to do that.
It depends. If you ask me to skydive without parachute, I won't, even
if a standard says that's OK to do so ;-)
Loic Domaigne
> > It seems to me the objective with threads is to have a light-weight
> > execution context which reduces the overhead of parallel computing.
[snip]
> I agree with that. The point discussed between some posters and me is that
> *all* things proposed by a standard are necessarily fine from a technical
Err...
r / are necessarily fine / are *not* necessarily fine
David Butenhof
> "David Butenhof" <David.B...@hp.com> wrote in message
> news:4059...@usenet01.boi.hp.com...
>
>> Pragmatically, the problem is that until NPTL, clone just didn't work
>> right.
>> I don't know whether it does now, or if they're still hacking around the
>> weaknesses. With a correct and complete implementation of clone, however,
>> all the "process state" (pid, uid, etc) resides in one common place
>> shared by all the execution entities. Any process state needs to be
>> changed in only one place, no matter how many execution entities are
>> connected.
>
> While I agree with you, it's possible that you're missing my point.
> Yes,
> it's not hard to implement things such that 'seteuid' works for the entire
> process, all you have to do is make sure all the threads have only one
> structure of some kind and that this structure contains the effective user
> ID.
>
> Linus' point, I think, was that this was bad design. Each thread
> should have its own euid. Consider a program like Samba, for example.
>
> Linus' philosophy was to make it easy to do things the right way, not
> to add kernel features to make it easy to do things the wrong way,
> standard or no.
No, Linus did things to make HIS particular private version of "the truth"
easy, and refused to make any concessions for the way anyone else felt the
system ought to work -- standards or not. This is not an issue of "right"
or "wrong", but of standard or proprietary. (Though if I was going to argue
for a philosophical "right" that included multiple UIDs per process, I
wouldn't try to carry along the baggage of UNIX-style interfaces that
presume a single security domain for all shared resources!)
POSIX (and in fact all normal variants of "UNIX") do not have the system
interfaces and are not designed to effectively support programming within
an environment containing shared resources with separate UIDs. Where do
signals go? What about file descriptors? None of this makes sense without
designing a substantially different system interface model. The POSIX
thread model was designed to fit within the existing POSIX API with minimal
changes, and it works. (Admittedly, not without a few rough edges in areas
where we would still have needed to completely redesign the system to fix
it correctly; like fork.) If you want multiple identities, you're really
better off using them with the security and error isolation provided by
full processes, to avoid leakage of any problems from one identity to
another.
There are a great many applications that can benefit from the POSIX model of
concurrency and sharing, and they can benefit equally across a wide range
of implementations. There are certainly some applications that can benefit
from a different model; and nobody ever said that Linux, or Solaris, or
anyone else who saw a need couldn't try to provide such features. However,
to implement only "extensions" incompatible with the portable standard is
to display a proprietary arrogance on par with Microsoft. I know there are
many who think this arrogance (in Linux or in Microsoft) is "cool", but I'm
not one of them. When Linux finally "gave in" and allowed suppport for
standards, it gained at least the beginnings of a long-missing legitimacy.
At the very least, I think they were being intellectually dishonest by
"fixing" basic and pervasive aspects of the system to suit themselves while
superficially borrowing the bulk of the API (even though the two aspects
aren't completely consistent) in order to allow them to "capture" existing
applications more effectively.
Don't get me wrong; they don't claim to be "POSIX", and there should be no
real obligation to do things the POSIX way if they don't want to. I just
disagree (quite strongly) with their decisions and reasons in this area and
find it "curious" that so many people blindly went along with them. (But
then, I fail to comprehend why anyone likes Windows, either.) And, on the
other hand, there's absolutely no reason they (or you) should care what I
think... or waste your time arguing about it. ;-)
Giancarlo Niccolai
> POSIX (and in fact all normal variants of "UNIX") do not have the system
> interfaces and are not designed to effectively support programming within
> an environment containing shared resources with separate UIDs. Where do
> signals go? What about file descriptors? None of this makes sense without
> designing a substantially different system interface model. The POSIX
> thread model was designed to fit within the existing POSIX API with
> minimal changes, and it works. (Admittedly, not without a few rough edges
> in areas where we would still have needed to completely redesign the
> system to fix it correctly; like fork.)
Well, I may be wrong, but this is how I see it: threading is about a (set
of) processor(s) running many part of A PROCESS (a SINGLE process) at the
same time. Multitasking is about running different processes at the same
time. Honestly, I feel a shiver when I hear that mixing the two things
could do good; we should recreate a part of the process environment thread
specific? Which part? And so, why not all? and what then would be the
difference between threads and processes? Shared memory with a different
process environment? Uhm... but we may have shared memory under different
processes too, right now! So, is multitasking the perfect multithreading?
Well, I don't think so.
Ok for fork() implementation not being "clear" or even brilliant when
threads are in. Ok for signal handling "feeling" a little strange under
multithreading. Ok for the fact that maybe (only maybe) a
thread-from-process to thread-from-process (windows like) communication
could be interesting (but then what about networking or pipes?).
But one thing is a thread, one thing is a process, and IMHO, threads must
contend & coordinate to manage not just shared resources & memory, but also
process context; its in their nature.
Giancarlo.
Steve Watt
Giancarlo Niccolai <No...@nothing.org> wrote:
>David Butenhof wrote:
>
>> POSIX (and in fact all normal variants of "UNIX") do not have the system
>> interfaces and are not designed to effectively support programming within
>> an environment containing shared resources with separate UIDs. Where do
>> signals go? What about file descriptors? None of this makes sense without
>> designing a substantially different system interface model. The POSIX
>> thread model was designed to fit within the existing POSIX API with
>> minimal changes, and it works. (Admittedly, not without a few rough edges
>> in areas where we would still have needed to completely redesign the
>> system to fix it correctly; like fork.)
>
>Well, I may be wrong, but this is how I see it: threading is about a (set
>of) processor(s) running many part of A PROCESS (a SINGLE process) at the
>same time. Multitasking is about running different processes at the same
>time. Honestly, I feel a shiver when I hear that mixing the two things
>could do good; we should recreate a part of the process environment thread
>specific? Which part? And so, why not all? and what then would be the
>difference between threads and processes? Shared memory with a different
>process environment? Uhm... but we may have shared memory under different
>processes too, right now! So, is multitasking the perfect multithreading?
No, I think you've got the abstraction picture slightly wrong. Under
current POSIX, a process is not an executable thing. It's a box into
which you place executable things, and those executable things are
called threads.
The box also has a bunch of other resources: File descriptors, a
memory map, and a protection domain.
For convenience, when a process is created, a thread is also created.
It is relatively easy to imagine a system where a process is created
with no threads, and the thread that creates[1] the process also creates
its thread(s). But that's not the model UNIX has used in the past, so
it's not the model that POSIX specifies.
>Ok for fork() implementation not being "clear" or even brilliant when
>threads are in. Ok for signal handling "feeling" a little strange under
>multithreading. Ok for the fact that maybe (only maybe) a
>thread-from-process to thread-from-process (windows like) communication
>could be interesting (but then what about networking or pipes?).
>But one thing is a thread, one thing is a process, and IMHO, threads must
>contend & coordinate to manage not just shared resources & memory, but also
>process context; its in their nature.
Threads must *cooperate* to manage the process state, not contend. Very
much like threads must cooperate to manage file descriptors (you can't
close an fd that another thread is using).
If you have reason to feel your threads are "competing" for anything,
you are probably not solving the overall problem in the most effective
way.
[1] Note that threads create processes, threads call fork(), threads
do everything. A process can't do anything -- it's not executable.
In anthropomorphic terms, thtreads are alive, processes are not.
--
Steve Watt KD6GGD PP-ASEL-IA ICBM: 121W 56' 57.8" / 37N 20' 14.9"
Internet: steve @ Watt.COM Whois: SW32
Free time? There's no such thing. It just comes in varying prices...
David Schwartz
"Steve Watt" <st...@nospam.Watt.COM> wrote in message
news:Hv5rH...@Watt.COM...
> Threads must *cooperate* to manage the process state, not contend. Very
> much like threads must cooperate to manage file descriptors (you can't
> close an fd that another thread is using).
Right, but the model can make it easy or hard for them to cooperate. For
example, a model in which two threads that called 'socket' at the same time
might get the same descriptor would make it much harder for threads to
cooperate.
POSIX's decision, for example, to make the working directory shared by
all threads makes it much harder for threads to cooperate in the most common
cases. Ditto for the effective user ID.
This type of oversharing makes it much harder than it needs to be to
write thread safe code.
DS
Giancarlo Niccolai
>>But one thing is a thread, one thing is a process, and IMHO, threads must
>>contend & coordinate to manage not just shared resources & memory, but
>>also process context; its in their nature.
>
> Threads must *cooperate* to manage the process state, not contend. Very
> much like threads must cooperate to manage file descriptors (you can't
> close an fd that another thread is using).
Don't bend my words. *contend & coordinate* (which is very clear in the
above test) IS *cooperate*.
>
> If you have reason to feel your threads are "competing" for anything,
> you are probably not solving the overall problem in the most effective
> way.
Cooperate IS (partially) competing (which is partially contending). Is also
*coordinate*. Cooperate = compete + coordinate.
>
> [1] Note that threads create processes, threads call fork(), threads
> do everything. A process can't do anything -- it's not executable.
> In anthropomorphic terms, thtreads are alive, processes are not.
In multithreading. I was talking of multithreading versus multitasking. In a
parallel-task system WITHOUT threads, processes ARE alive (in
anthropomorphic terms). In posix pthread threading abstraction they are a
hood, but I was not referring to the process as it is for a thread, but to
the process when threading is not in.
Am I really so bad in expressing myself? If not fulgid, I at least hope to
be understandable.
Giancarlo.
David Butenhof
> POSIX's decision, for example, to make the working directory shared by
> all threads makes it much harder for threads to cooperate in the most
> common cases. Ditto for the effective user ID.
Working directory is a convenience, not a necessity. And while it's an
important convenience at the shell level, it's not that big a deal for a
program to use fully qualified paths. Basically, nobody made any good
argument that it was worth forcing all threads to carry this around and add
it to the context switching overhead.
EUID is a different matter. That's a substantial change in file management,
which would require a lot of work to support. It would mean either making
fds thread-private, or adding protection checks to every fd (and file
stream) access. After all, we can't allow a file created with only owner
access to be written by a thread with a different EUID, merely because it
happens to live in the same process. That's added expense, added
development, substantial new confusion. This wasn't something anyone on the
original committee was interested in persuing. It wasn't "minimal" or
essential, and there were far more important things to deal with.
Certainly it would be useful for some applications. There are a number of
systems that actually do support per-thread "personna" -- OpenVMS, for
example. Such an extension could certainly be proposed for standardization;
but, you know, in the 9 years since 1003.1c-1995 was completed, nobody's
ever actually taken the time to formally propose such a project. Huh.
Interesting, eh?
> This type of oversharing makes it much harder than it needs to be to
> write thread safe code.
No, the absence of a specialized feature you'd like makes it harder than
you'd prefer to write certain kinds of code using threads within a process.
That's not the same thing. "Oversharing" is in the eye of the beholder. You
could easily find a number of people eager to tell you that the
complication and overhead of context switching this extra state, and the
extra permission checks involved, would impose unacceptable overhead on
THEIR code that doesn't need this feature. That doesn't make you wrong to
want it, but this sort of griping isn't particularly realistic or
productive.
Jeremy Allison
> POSIX threads are the perfect abstractions for threading: they only
> abstract the execution context but keep the rest of the process model
> intact.
>
> A number of system calls, however, make little sense in threaded processes:
>
> chroot(), chdir(), set*id()
>
> but saying that geteuid() can return different values is almost like saying
> that file descriptors returned by open() shouldn't be shared between
> threads.
smbd doesn't use threading in that way - it would be somewhat horrid and unstable
due to some of the issues you've raised.
> Ah yes, the Samba locking model is a mess :-)
Well it's the Microsoft locking model really :-). I actually like the POSIX
fcntl locking model except for one thing that kills it - that's the fact
that a close() deletes all locks on the file on all fd's currently open in
the process, not just the closing fd.
For whoever originally coded that.... death is too good for them :-).
Jeremy.
Casper H.S. Dik
>Well it's the Microsoft locking model really :-). I actually like the POSIX
>fcntl locking model except for one thing that kills it - that's the fact
>that a close() deletes all locks on the file on all fd's currently open in
>the process, not just the closing fd.
Yeah, that's something I've never been able to wrap my head around;
what where they thinking when they specified that. It probably came
from New Jersey.