[9fans] Threads: Sewing badges of honor onto a Kernel

1024 views
Skip to first unread message

dbai...@ameritech.net

unread,
Feb 26, 2004, 11:55:39 PM2/26/04
to

A few of us had a discussion this evening on the state of Linux and
bloat in the Operating System development community. This isn't an
event isolated to this evening, however tonight had a bit of spice.

Andrey gleamed this bit of interesting text on the O'Reilly[1] website.
Let's have a look at a snippet, shall we?

Under the "Kernel Space and User Space" heading:

| ... Another example of this happened in the Plan 9 operating system.
| They had this really cool system call to do a better process fork--a
| simple way for a program to split itself into two and continue processing
| along both forks. This new fork, which Plan 9 called R-Fork (and SGI later
| called S-Proc) essentially creates two separate process spaces that share
| an address space. This is helpful for threading especially.

| Linux does this too with its clone system call, but it was implemented
| properly. However, with the SGI and Plan9 routines they decided that
| programs with two branches can share the same address space but use
| separate stacks. Normally when you use the same address in both threads,
| you get the same memory location. But you have a stack segment that is
| specific, so if you use a stack-based memory address you actually get
| two different memory locations that can share a stack pointer without
| overriding the other stack.

| While this is a clever feat, the downside is that the overhead in maintaining
| the stacks makes this in practice really stupid to do. They found out too
| late that the performance went to hell. Since they had programs which used
| the interface they could not fix it. Instead they had to introduce an additional
| properly-written interface so that they could do what was wise with the stack
| space. ...

This commentary from Linus Torvalds on rfork[2] proclaims that a single, unified
stack space for multiple threads is not only beneficial but imperative for proper
thread propagation. What we are wondering is ... why?

What scenario would make a unified stack segment beneficial in a multi-threaded
execution environment. Is this something that should be an optional parameter
on-the-fly in a call to rfork() ? Does clone() in Linux allow you to decide, or, does
it enforce the unified stack segment? Doesn't unification create collisions within
scopes of process namespaces through recursion and non-leaf code vectors?

Since we couldn't figure out the logic behind this Lunix tantra, we thought
trolling the public would be a slick idea. Who has an opinion on (or a defense
for ;-)) this statement?

Also, who is the "They" that "found out too late that the performance went to
hell".

Curious minds are interested.

Don (north_)

References:

[1] http://www.oreilly.com/catalog/opensources/books/linus.html
[2] fork(2)

andrey mirtchovski

unread,
Feb 27, 2004, 12:07:41 AM2/27/04
to
the url is wrong -- s/books/book/

also, you omitted the conclusion from the chapter, which is (imho) the best:

| In fifteen years, I expect somebody else to come along and say, hey, I can
| do everything that Linux can do but I can be lean and mean about it because
| my system won't have twenty years of baggage holding it back. They'll say
| Linux was designed for the 386 and the new CPUs are doing the really
| interesting things differently. Let's drop this old Linux stuff. This is
| essentially what I did when creating Linux. And in the future, they'll be
| able to look at our code, and use our interfaces, and provide binary
| compatibility, and if all that happens I'll be happy.

andrey "just fanning the flames" mirtchovski


Scott Schwartz

unread,
Feb 27, 2004, 12:06:50 AM2/27/04
to
Beware that this mailing list doesn't permit postings
with more than a few cc-s.

To: 9f...@cse.psu.edu
cc: torv...@osdl.org, sio...@beefed.org, fa...@beefed.org

dbai...@ameritech.net

unread,
Feb 27, 2004, 12:13:51 AM2/27/04
to

> also, you omitted the conclusion from the chapter, which is (imho) the best:

I didn't want to paste *too* much into the email ;-)

dbai...@ameritech.net

unread,
Feb 27, 2004, 12:14:43 AM2/27/04
to
> the url is wrong -- s/books/book/

Ah, yep. Can't paste over VNC. Thanks, Andrey

full URL is:
http://www.oreilly.com/catalog/opensources/book/linus.html

Don (north_)

Rob Pike

unread,
Feb 27, 2004, 12:27:48 AM2/27/04
to
this is a peculiar distortion of a peculiar episode. the original
issue linus
objected to, and rather rudely if i remember right, is that if the
stacks are
distinct, there can be confusing bugs if you pass pointers around that
accidentally happen to refer to stack addresses. in practice, we never
do this because the thread library makes it all but impossible. however
this issue really rankled linus,

on the other hand, having a separate piece of address space with
different contents is a powerful idea. it's called something like
per-thread space, and it's really convenient for many implementation
details in threading. but the feature is (or was; i really don't know
the
situation any more) hard to provide on linux and in fact the clone
interface used to be really nasty because the two processes returned
from the syscall with two processes sharing the same stack. i
understand
something has been done to make the code after a clone on linux
easier to write, but when linus was dyspeptic about rfork, the linux
dance was really tricky.

so it's just a detail, a design choice with advantages and disadvantages
either way. win some, lose some. yet somehow our code
is 'stupid' and linux is 'proper', so i guess the snarkiness endures.

-rob

Rob Pike

unread,
Feb 27, 2004, 12:29:31 AM2/27/04
to
in case that wasn't clear enough, i have no idea what linus is talking
about when he says we had 'overhead' that made it 'really stupid'
'in practice'.

-rob

dbai...@ameritech.net

unread,
Feb 27, 2004, 12:43:51 AM2/27/04
to

> so it's just a detail, a design choice with advantages and disadvantages
> either way. win some, lose some. yet somehow our code
> is 'stupid' and linux is 'proper', so i guess the snarkiness endures.

What I am mainly confused about, is... We're talking about user
context threading. In user context threading we've got the abstractions
of a BSS, Data, Text, Stack, (etc in some cases) to maintain a sense
of program-persistant-object-classification. As a coder, we know
and can specify (for the most part) where these "objects" will end
up.

So the question to me is, why would you *want* a unified stack?
The only state in which this might be relevant is threads maintained
in ASM where data is passed back and forth via Stack and manipulated
via Registers. Otherwise, things just don't make much sense. In a
land of higher level languages, this semantic seems irrational.

This is mainly what I love about Plan 9's threading implementation, that
it unifies memory in the heap, and not at the stack. I agree that this
creates a powerful tool not only in efficiency, but also in reliability.

I see Linus' logic as more of a negation of both efficiency and reliability.

I'm not interested in starting a flame war, but I *am* interested in
understanding the logic, because I just can't see it clearly from my
point of view.

Don (north_)

ron minnich

unread,
Feb 27, 2004, 12:45:35 AM2/27/04
to
I did the original rfork for freebsd, and I wrote it so I split the stacks
too. It was lovely, since you just rforked and all was shared save the
stack -- no assembly required. When they did the committed version, after
fork, all was shared. This sucked. It was quite a mess to fix up your
stack so you weren't walking on each other's stack -- required assembly
goo to make it go. I hated it and stopped using it.

So I like plan 9 rfork quite a bit. I think Linus/Linux are wrong.

ron

ron minnich

unread,
Feb 27, 2004, 12:48:58 AM2/27/04
to
On Fri, 27 Feb 2004 dbai...@ameritech.net wrote:

> So the question to me is, why would you *want* a unified stack?

well, you don't. It makes programming a nightmare and you need assembly
goo to seperate the stacks after a fork, else you will be stepping on
each others' stack. As the freebsd users of their official version of
rfork() learned, that meant you hit segv walls very quickly. Ouch.

> I see Linus' logic as more of a negation of both efficiency and reliability.

I don't even understand it.

ron

George Michaelson

unread,
Feb 27, 2004, 12:51:40 AM2/27/04
to

what happens on MP systems?

-George

Rob Pike

unread,
Feb 27, 2004, 12:55:49 AM2/27/04
to
i agree with you. i think the model, although done for reasons of
expediency, has some advantages. the split stack provides a
unique and powerful memory abstraction: same address, different
value. (c.f. the up and mp registers in the kernel.)

> I see Linus' logic as more of a negation of both efficiency and
> reliability.

i don't understand his logic.

-rob

ron minnich

unread,
Feb 27, 2004, 1:00:50 AM2/27/04
to
On Fri, 27 Feb 2004, George Michaelson wrote:

> what happens on MP systems?
>


not sure what you're driving at.

ron

George Michaelson

unread,
Feb 27, 2004, 1:08:50 AM2/27/04
to

on an MP system, the forked process/thread has chances to be run on a distinct
CPU underneath. I would have thought the complexity of managing a shared stack
was magnified in that circumstance.

process migration -wise?

-George

dbai...@ameritech.net

unread,
Feb 27, 2004, 1:09:02 AM2/27/04
to

> > I see Linus' logic as more of a negation of both efficiency and
> > reliability.

> i don't understand his logic.

Well, I interpreted his logic that each process, itself, should be the
manager of how data is abstracted into each thread. Further, that
the kernel should not impose an interface on such userspace
processes. When I think in terms of future scalability and dynamic
code design, this *seems* to be a good thing.

However, over time, we've seen that the lack of a conformed
interface has prompted the design of such an interface: pthreads,
etc. So, in the end, the necessity for POSIX compliance (or what ever
compliance you prefer) is perceived as paramount.

Though, this could have been avoided and deemed unnecessary if
these kernels had only implemented an interface in the first place.

I believe the shortsightedness of not imposing a strict type created
too many possibilities. This creates a bigger problem when dealing
with libraries imported into a system with no defined interface, and
you end up with the desire for one.

So, in the end, Plan 9 looks like the better solution. The foresight was
there to see a necessity for conformity where all possible permutations
of a given set tended to yield outcomes meeting at the same axis:
thus an interface.

I duno, that's my two cents, anyway. But, that's dependant on me
getting Linus' logic correct, which is why he was CC'd.

Don (north_)

Linus Torvalds

unread,
Feb 27, 2004, 1:15:50 AM2/27/04
to

On Thu, 26 Feb 2004 dbai...@ameritech.net wrote:
>
> This commentary from Linus Torvalds on rfork[2] proclaims that a single, unified
> stack space for multiple threads is not only beneficial but imperative for proper
> thread propagation. What we are wondering is ... why?

(1) There are valid reasons to pass pointers between threads, and yes,
they can be pointers to thread stack areas.

(2) No existing common hardware can do "partial TLB flushes" efficiently.
The main performance advantage of threads is that you don't need to flush
the VM state on thread switches. If you have partial VM area sharing, you
lose that whole point.

(3) Implementation sucks. Irix and Plan-9 both get it wrong, and they
_pay_ for it. Heavily. The Linux code is just better.

Linus

Scott Schwartz

unread,
Feb 27, 2004, 1:20:45 AM2/27/04
to
Al Viro once told me that if Plan 9 had shared libraries, rfork wouldn't
work as well. Apparently they're concerned about page tables when there
are lots of different combinations of sharing going on.

dbai...@ameritech.net

unread,
Feb 27, 2004, 1:38:59 AM2/27/04
to

> (1) There are valid reasons to pass pointers between threads, and yes,
> they can be pointers to thread stack areas.

Do you have code from existing Linux implementation that exemplifies
this scenario? What are your opinions on the benefits? If there is existing
code that exemplifies this scenario, how often does it propagate? Is it
often enough to denounce a standard interface in the kernel?

Also, what are your comments, Linus, on shared library execution altering
the scope of an interface from kernel context to user context? Even if there
isn't a unified stack, there is still an interface defined by such thread projects
as POSIX threads, etc. Doesn't that just create a standard interface in user-
land that could be sped up, if moved to kernel context?

Don (north_)

Linus Torvalds

unread,
Feb 27, 2004, 1:45:49 AM2/27/04
to

On Fri, 27 Feb 2004 dbai...@ameritech.net wrote:
>

> Do you have code from existing Linux implementation that exemplifies
> this scenario?

Anything that accesses any process arguments would access the stack of the
original thread. That's the most obvious one. But synchronization with
completion events on the stack is also perfectly feasible (the kernel does
exactly that internally, for example - and user mode could do the same).

Linus

dbai...@ameritech.net

unread,
Feb 27, 2004, 1:54:38 AM2/27/04
to

> Anything that accesses any process arguments would access the stack of the
> original thread.

I'm not sure I can agree here. Arguments are not meant for the scope of an
entire process, but for the initialization of a program's state. In a program that
parses a fully qualified file name path derived from command line arguments,
the initial state should only open a file descriptor and pass that descriptor to
a child thread while performing other duties. This simple example depicts
how a process' arguments should be restricted to initialization. After all, if
the process can't open the file, there is no need to thread.

It seems that allowing such a misuse of resource would be more of a
rationalization for allowing someone's bad programming, rather than a
strong argument against defined thread interface.

Can you elaborate on your synchronization example, please?

Don (north_)

Donald Brownlee

unread,
Feb 27, 2004, 1:57:39 AM2/27/04
to

Linus Torvalds wrote:


>
> (1) There are valid reasons to pass pointers between threads, and yes,
> they can be pointers to thread stack areas.
>

OK. If one of the threads is a "debugger" thread and another
is a "thread to be debugged," then it would be helpful if
the "debugger" thread can access the stack of the "thread to be debugged."

Why? So it can muck with the stack, say, to alter a return
address. Whatever!

But to inspect or modify the stack of another thread requires
some knowledge of that other thread's state. Like, is it
stopped?

D.

Linus Torvalds

unread,
Feb 27, 2004, 2:00:56 AM2/27/04
to

On Fri, 27 Feb 2004 dbai...@ameritech.net wrote:
>

> I'm not sure I can agree here. Arguments are not meant for the scope of an
> entire process, but for the initialization of a program's state.

Wrong. You have zero basis for your assertion, and it's simply bogus.

Would you say that the process environment[] array is also only applicable
to the first thread?

But it doesn't matter. Regardless, threads should see each others stacks.

> Can you elaborate on your synchronization example, please?

There are tons of examples in the linux kernel. Search for any use of
"struct completion" - most of them are allocatied on a threads stack
(which is obviously distinct between different threads), and other threads
and interrupts will use that data structure to wake up the thread.

But hey, I'm not interested in trying to convince you. Feel free to think
I'm wrong.

Linus

Lucio De Re

unread,
Feb 27, 2004, 2:08:46 AM2/27/04
to
On Fri, Feb 27, 2004 at 01:48:09AM -0500, dbai...@ameritech.net wrote:
>
> > Anything that accesses any process arguments would access the stack of the
> > original thread.
>
> I'm not sure I can agree here. Arguments are not meant for the scope of an
> entire process, but for the initialization of a program's state. In a program that
> [ ... ]

Not only. Presumably, the stack is duplicated on rfork(2) which means
that reading the arguments is OK, writing them would be a problem.

But just because the arguments can be changed does not mean that it
can't be deprecated in the case of threads.

++L

PS: I do think this particular one is a quibble, though. My choice
would be along the lines of: "having made choice "A", is there a
mechanism to implement choice "B" from within it?" If there is,
then only efficiency criteria can be used to select "A" over "B".
If there isn't, then either the reverse applies in which case "B"
is the obvious contender, or the two are distinct solutions, each
requiring implementation.

Rob Pike

unread,
Feb 27, 2004, 2:13:50 AM2/27/04
to
the argument about TLB flush times is interesting.

> But it doesn't matter. Regardless, threads should see each others
> stacks.

and on plan 9, they do. they just can't see each other's *stack
segments*
within their own address space.

there seems to be confusion on this point. plan 9's kernel splits the
stack segment after a fork, but in the normal state, the processes run
with the sp in shared memory. the marvelous properties of the
same-address-different-contents split stack segment is used only
during the fiddly bits of process manipulation and to store per-process
data.

how does linux store per-process data in user space?

-rob

dbai...@ameritech.net

unread,
Feb 27, 2004, 2:14:20 AM2/27/04
to

> Wrong. You have zero basis for your assertion, and it's simply bogus.

Clean, logical programming isn't a sound basis? Or is it only not a sound
basis because you've decided so? Saying that I have zero basis for my
assertion is a bit bogus, in it self, don't you think? I CC'd you my
original email to 9fans because I was hoping to understand your logic
behind the rfork() comments, not to start a flame war.

> Would you say that the process environment[] array is also only applicable
> to the first thread?

Yes. That's why we have env(3) in Plan 9. So each thread can alter and
read the environment, globally, in a process.

> There are tons of examples in the linux kernel. Search for any use of
> "struct completion" - most of them are allocatied on a threads stack
> (which is obviously distinct between different threads), and other threads
> and interrupts will use that data structure to wake up the thread.

Okay, but is there a reason why it *needs* to be on the stack? What's the
rationale behind this usage.

> But hey, I'm not interested in trying to convince you. Feel free to think
> I'm wrong.

Well, I'm hoping to determine what really is the best thing for OS design,
as a whole. It isn't in my interest to determine who is right or wrong, only
what makes sense universally. If we are really all here to try and design
code that works the best it can, that's all that matters. Ego is irrelevant and
only gets in the way of those discoveries that can really help us materialize
the better code.

Even if there are no more comments on this thread, I still learned a lot
by asking the questions. If that stirrs up some contraversy, so be it. I'd
rather learn and ask questions than stay silent and experience nothing.

Don (north_)

Charles Forsyth

unread,
Feb 27, 2004, 2:16:46 AM2/27/04
to
>>the argument about TLB flush times is interesting.

it might have been if Linux hadn't gone through an O(n) scheduler before it did that context switch.

Lucio De Re

unread,
Feb 27, 2004, 2:41:41 AM2/27/04
to
On Fri, Feb 27, 2004 at 02:06:57AM -0500, dbai...@ameritech.net wrote:
>
> > Would you say that the process environment[] array is also only applicable
> > to the first thread?
>
> Yes. That's why we have env(3) in Plan 9. So each thread can alter and
> read the environment, globally, in a process.
>
Is Torvalds really saying that environment[] is held _in_ the stack?!
No wonder he was reluctant to copy it! Specially when using "bash".
But I must be mistaken.

++L

a...@9srv.net

unread,
Feb 27, 2004, 2:42:41 AM2/27/04
to
// It isn't in my interest to determine who is right or wrong,
// only what makes sense universally.

let me help, then: nothing. nothing makes sense "universally".

Linux is Linux. Plan 9 is Plan 9. the two systems have very different
design goals. that's not to defend linux's fork design or any other
particular decision, but asking (by implication) why linux doesn't
just do something like env(3) misses those goals.

personally, i'm less interested in arguing point 1 from linus's post
(there might be valid reasons to do this sometimes), which seems at
least plausable, and much more interested in the other two. on the
second, i think clarification is in order: that's the main advantage
of threads over what, processes? and on that last one (the bit on
performance and code quality) i'm *really* curious what he's talking
about. do we have a particularly heavy fork? i'm confused. saying
"The Linux code is just better." just *begs* for support.

oh, and it's "Plan 9", or sometimes "plan9", but never "Plan-9".
thanks.

Linus Torvalds

unread,
Feb 27, 2004, 2:42:52 AM2/27/04
to


On Fri, 27 Feb 2004 dbai...@ameritech.net wrote:
>

> Okay, but is there a reason why it *needs* to be on the stack? What's the
> rationale behind this usage.

The rationale is that it's incredibly more sane, and it's the logical
place to put something that (a) needs to be allocated thread-specific and
(b) doesn't need any special allocator.

In short, it's an automatic variable.

You are arguing against automatic variables in C. You apparently do so for
some incredibly broken religious reason, and hey, I simply don't care.

> Well, I'm hoping to determine what really is the best thing for OS design,
> as a whole.

I can guarantee you that the broken behaviour of SGI sproc and plan-9
rfork i sa major pain in the ass for VM management.

I'm obviously very biased, but I claim that the Linux VM is the best VM
out there. Bar none. It's flexible, it's clearly very portable to every
single relevant hardware base out there (and quite a few that aren't
relevant), and it's efficient.

And at the same thing, the Linux VM (from an architecture standpoint) is
simple. There are lots of fundamentally hard problems in teh VM stuff, but
they tend to be things that _are_ fundamentally hard for other reasons (ie
page replacement algorithms in the presense of many different caches that
all fight for memory). But the actual virtual mapping side is very simple
indeed.

And the plan-9/irix thing isn't. It's an abomination.

And there are real _technical_ reasons why it's an abomination:

- it means that you cannot share hardware page tables between threads
unless you have special hardware (ie it is either fundamentally
unportable, or it is fundamentally inefficient)
- it means that you have to give different TLB contexts to threads,
causing inefficient TLB usage. See above.
- it means that you need to keep track of separate lists of "shared" and
"private" VM mapping lists, and locking of your VM data structures is a
complete nightmare.
- it almost certainly means a lot of special cases, since on the magic
hardware that does have segmented page tables and where you want to
share the right segments, you now have magic hardware-dependent limits
for which areas can be shared and which can be private.

But yes, I'm biased.

Linus

Fco.J.Ballesteros

unread,
Feb 27, 2004, 2:48:46 AM2/27/04
to
> And at the same thing, the Linux VM (from an architecture standpoint) is
> simple.

Do you notice that you had to say 'from an architecture standpoint' ?
I don't think it's simple at all. I agree it seems to be more efficient
(although I don't have measures to support this), but it's
contorted code, at best.

Regarding what's broken or not, it mostly depends on bugs, and bugs
depend mostly on complexity.

dbai...@ameritech.net

unread,
Feb 27, 2004, 2:52:44 AM2/27/04
to

> The rationale is that it's incredibly more sane, and it's the logical
> place to put something that (a) needs to be allocated thread-specific and
> (b) doesn't need any special allocator.

You've just proven my point. Thread specific. Being Thread specific, it
is data that is reserved to the scope of a single thread. Nothing more.
If you want more scope there are many more usages of memory that
are better utilized.

I'm not arguing against automatic variables. I'm arguing against using
automatic variables anywhere but the scope of stack in which they are
defined.

I don't know where you're getting religiousness, but, there's nothing
wrong with trying to do the best for all of your users. That's why
companies research their target audience, so they can improve a
product according to the most global needs of a given group.

Don (north_)

Linus Torvalds

unread,
Feb 27, 2004, 2:53:40 AM2/27/04
to

On Fri, 27 Feb 2004, Lucio De Re wrote:
>
> Is Torvalds really saying that environment[] is held _in_ the stack?!
> No wonder he was reluctant to copy it! Specially when using "bash".
> But I must be mistaken.

Take a look if you don't believe me.

There is a _lot_ of state on the stack. That's how C works.

It's perfectly valid behaviour to do something like this:

myfunction()
{
mytype myvariable;

pthread_create(.. &myvariable);
...
pthread_join(..);
}

where you give the thread you created its initial state on your own stack.
You obviously must keep it there until the thread is done with it (either
by waiting for the whole thread as in the above example, or by using some
other synchronization mechanism), but the point is that C programmers are
used to a fundamentally "flat" environment without segments etc.

And what a private stack is, is _nothing_ more than a segment.

And I have not _ever_ met a good C programmer that liked segments.

So in a C/UNIX-like environment, private stacks are wrong. You could
imagine _other_ environments where they might be valid, but even those
other environments would not invalidate my points about efficiency and
simplicity.

Linus

dbai...@ameritech.net

unread,
Feb 27, 2004, 2:55:41 AM2/27/04
to

> personally, i'm less interested in arguing point 1 from linus's post
> (there might be valid reasons to do this sometimes), which seems at
> least plausable, and much more interested in the other two. on the
> second, i think clarification is in order: that's the main advantage
> of threads over what, processes? and on that last one (the bit on
> performance and code quality) i'm *really* curious what he's talking
> about. do we have a particularly heavy fork? i'm confused. saying
> "The Linux code is just better." just *begs* for support.

I wasn't interested in #2 because #2 didn't make much sense to me.
I wasn't interested in #3 because I had no desire to discuss things
resembling personal attacks.

Don (north_)

Rob Pike

unread,
Feb 27, 2004, 3:01:52 AM2/27/04
to

> So in a C/UNIX-like environment, private stacks are wrong. You could
> imagine _other_ environments where they might be valid, but even those
> other environments would not invalidate my points about efficiency and
> simplicity.

as i said before, the stacks are not private. you're right, that's a
bad thing.
that's why they're not private.

the segment called 'stack' is private, but that's a different thing.
i stress: stack != stack segment. stack is where your sp is; stack
segment is a figment of the VM system.

i ask again: how does linux create per-thread storage?

the way the plan 9 thread library works is so different from linux's
that they're hard to compare. program design in the two worlds is
radically different. so your claim of 'better' is curious to me. by
'better'
you seem to mean 'faster' and 'cleaner'. faster at least can be
measured.
you speak with certainty. have you seen performance comparisons?
i haven't, although it wouldn't surprise me to learn that there are
useful
programs for which linux outperforms plan 9, and vice versa of course.

-rob

Linus Torvalds

unread,
Feb 27, 2004, 3:04:46 AM2/27/04
to

On Fri, 27 Feb 2004 dbai...@ameritech.net wrote:
>

> > The rationale is that it's incredibly more sane, and it's the logical
> > place to put something that (a) needs to be allocated thread-specific and
> > (b) doesn't need any special allocator.
>
> You've just proven my point. Thread specific. Being Thread specific, it
> is data that is reserved to the scope of a single thread. Nothing more.
> If you want more scope there are many more usages of memory that
> are better utilized.

NO!

A "per-thread allocation" does NOT MEAN that other threads should not
access it. It measn that the ALLOCATION is thread-private, not that the
USE is thread-private.

per-thread allocations are quite common, and critical. If you have global
state, you need to protect them with locks, and you need to have nasty
global allocators.

One common per-thread allocation is the "I want to wait for an event". The
data is clearly for that one thread, and using a global allocator would be
WRONG. Not to mention inefficient.

But once the data has been allocated, other threads are what will actually
use the data to wake the original thread up. So while it wants a
per-thread allocator, it simply wouldn't _work_ if other threads couldn't
access the data.

That's what a "completion structure" is in the kernel. It's all the data
necessary to let a thread wait for something to complete. Another thread
will do "complete(xxx)", where "xxx" is that per-thread data.

You don't like it. Fine. I don't care. You're myopic, and have an agenda
to push, so you want to tell others that "you can't do that, it's against
my agenda".

While I'm telling you that people _do_ do that, and that it makes sense,
and if you didn't have blinders on, you'd see that.

Linus

Lucio De Re

unread,
Feb 27, 2004, 3:06:42 AM2/27/04
to
On Thu, Feb 26, 2004 at 11:57:52PM -0800, Linus Torvalds wrote:
>
> On Fri, 27 Feb 2004, Lucio De Re wrote:
> >
> > Is Torvalds really saying that environment[] is held _in_ the stack?!
> > No wonder he was reluctant to copy it! Specially when using "bash".
> > But I must be mistaken.
>
> Take a look if you don't believe me.
>
> There is a _lot_ of state on the stack. That's how C works.
>
Hm. Yes, C allows it. And your point is valid that an arbitrary
data segment may justifiably be copied. But the stack is traditionally
the repository of special, let's call it "holy" information and
having a thread stomping all over it when another thread gets to
find out the hard way is not a thrilling prospect.

After all, there is nothing to prevent a coder from altering the
return address (traditionally a stack entry) in one thread and
totally wrecking the behaviour of another on return, right?

I still think the acid test lies with whether one version can model
the other.

++L

PS: and I really hope that the arguments and environment variables
are stored elsewhere and only the pointers appear on the stack :-)
I mean, even MS-DOS got that bit "sane", to use a very loaded word
previously used by Linus in this discussion.

Scott Schwartz

unread,
Feb 27, 2004, 3:07:33 AM2/27/04
to
I suspect Linus isn't subscribed to 9fans, so we'll have to cc him.

That sharing is achieved by the thread library, not by the rfork
system call, though, right?

dbai...@ameritech.net

unread,
Feb 27, 2004, 3:11:17 AM2/27/04
to
> You don't like it. Fine. I don't care. You're myopic, and have an agenda
> to push, so you want to tell others that "you can't do that, it's against
> my agenda".

I'm not saying you can't, I'm just asking why, and you're not giving
me any sensible reason for it besides "because people do". That isn't
logical to me, and that's got nothing to do with any mysterious
agenda. I'm just interested in the theory.

> While I'm telling you that people _do_ do that, and that it makes sense,
> and if you didn't have blinders on, you'd see that.

Ok, so people do *do* that. That's fine, but that doesn't make it
correct, or sensible. I just wanted some semblance of logic, not
flames that equate to "I do it because I can". I can hop on one leg,
too, but that doesn't make hopping on one leg efficient.

Don (north_)

Lucio De Re

unread,
Feb 27, 2004, 3:12:43 AM2/27/04
to
On Fri, Feb 27, 2004 at 12:08:38AM -0800, Linus Torvalds wrote:

>
> On Fri, 27 Feb 2004 dbai...@ameritech.net wrote:
> >
> > > The rationale is that it's incredibly more sane, and it's the logical
> > > place to put something that (a) needs to be allocated thread-specific and
> > > (b) doesn't need any special allocator.
> >
> > You've just proven my point. Thread specific. Being Thread specific, it
> > is data that is reserved to the scope of a single thread. Nothing more.
> > If you want more scope there are many more usages of memory that
> > are better utilized.
>
> NO!
>
> A "per-thread allocation" does NOT MEAN that other threads should not
> access it. It measn that the ALLOCATION is thread-private, not that the
> USE is thread-private.
>
Wait a minute! If the stack is not thread private, what is? I
think this answers my question: stack duplication allows per-thread
private space, whereas stack sharing doesn't. This is a one-way
path. Unless you drop into register access, where it's the platform
that decides whether registers are duplicated or not.

But maybe they should also be shared, to be totally consistent.

Of course, I may be talking out of turn, but I really don't see
how threads can have private space if the stack isn't private.

++L

Rob Pike

unread,
Feb 27, 2004, 3:16:41 AM2/27/04
to

On Feb 27, 2004, at 12:06 AM, Scott Schwartz wrote:

> I suspect Linus isn't subscribed to 9fans, so we'll have to cc him.

sorry about that.

>
> Rob writes:
> | the argument about TLB flush times is interesting.
> |
> | > But it doesn't matter. Regardless, threads should see each others
> | > stacks.
> |
> | and on plan 9, they do. they just can't see each other's *stack
> | segments*
> | within their own address space.
> |
> | there seems to be confusion on this point. plan 9's kernel splits
> the
> | stack segment after a fork, but in the normal state, the processes
> run
> | with the sp in shared memory. the marvelous properties of the
> | same-address-different-contents split stack segment is used only
> | during the fiddly bits of process manipulation and to store
> per-process
> | data.
>
> That sharing is achieved by the thread library, not by the rfork
> system call, though, right?

right. but where it's implemented is in some sense just a detail, as my
other
message, incuded below, implies.

another way of looking at it is that the kernel provide some nice
primitives upon which to build a thread library. the plan 9 kernel
leaves out a lot of stuff that you're supposed to do in the kernel
for threads, but we were exploring options.

>
> | how does linux store per-process data in user space?
> |
> | -rob

here's my other message, with the cc: this time:

Rob Pike

unread,
Feb 27, 2004, 3:18:49 AM2/27/04
to

On Feb 27, 2004, at 12:11 AM, Lucio De Re wrote:

> Of course, I may be talking out of turn, but I really don't see
> how threads can have private space if the stack isn't private.

well, perhaps the stack isn't the only place to do it, but it's
certainly an easy one, and one that makes the syscall interface
to fork easy to implement in a threaded environment: longjmp
to the private stack, fork, adjust, longjmp back.

-rob

Geoff Collyer

unread,
Feb 27, 2004, 3:20:36 AM2/27/04
to
I think we're talking past each other due to different terminologies.
Linus seems to use `thread' to mean `a process sharing address space
(other than normal text segment sharing)', whereas in Plan 9, that's
just a process; some share address space, some don't. A Plan 9 thread
is entirely a user-mode creation of the Plan 9 thread library, which
doesn't implement POSIX threads.

Lucio De Re

unread,
Feb 27, 2004, 3:32:46 AM2/27/04
to
But I can't think of even one possible alternative. After all, the
stack is the only storage being duplicated (ignoring registers) so
where does one keep pointers to the private space?

++L

boyd, rounin

unread,
Feb 27, 2004, 3:46:40 AM2/27/04
to
> > So the question to me is, why would you *want* a unified stack?
>
> well, you don't. It makes programming a nightmare and you need assembly
> goo to seperate the stacks after a fork, ...

yup

boyd, rounin

unread,
Feb 27, 2004, 3:50:37 AM2/27/04
to
> > what happens on MP systems?
> >
> not sure what you're driving at.

i do. they have forgotten their lister and the old (now quick):

s = splfoo();

/* critical region */

splx(s);

usually simplifies things, unlike the lunacy you have to code
on linix >= 2.2 -- i've coded it; it made me sick.

Linus Torvalds

unread,
Feb 27, 2004, 3:53:36 AM2/27/04
to

On Fri, 27 Feb 2004, Rob Pike wrote:
>
> > So in a C/UNIX-like environment, private stacks are wrong. You could
> > imagine _other_ environments where they might be valid, but even those
> > other environments would not invalidate my points about efficiency and
> > simplicity.
>
> as i said before, the stacks are not private. you're right, that's a
> bad thing.
> that's why they're not private.
>
> the segment called 'stack' is private, but that's a different thing.
> i stress: stack != stack segment. stack is where your sp is; stack
> segment is a figment of the VM system.

Well, in another email I already said that "private stack" and "segments"
are really exactly the same thing - some people think of segments as
paging things, others think of them in the x86 sense, but in the end it
all comes down to the fact that a "stack address" ends up making sense
only within a specific context (and that context can sometimes be
partially visible to other threads by using explicit segment registers or
other magic, like special instructions that can take another address
space).

And private/segmented stacks are bad.

They are bad exactly because they magically make automatic variables
fundamentally different from other variables. And they really have no
reason they should be different.

There is absolutely nothing wrong with having a thread take the address of
some automatic variable, and then just pass that address off to another
routine. And if that other routine decides that it is going to create a
hundred threads to solve the problem that the variable described in
parallell, then that should JUST WORK. Anything else would be EVIL.

Having a pointer that sometimes works, and sometimes doesn't, based on who
uses it - that's just crazy talk.

> i ask again: how does linux create per-thread storage?

The same way it creates any other storage: with mmap() and brk(). You just
malloc the thing, and you pass in the new stack as an argument to the
thread creation mechanism (which linux calls "clone()", just to be
different).

And because that storage is just storage, things like the one I described
above "just work". If you pass another thread a pointer to your stack, the
other thread can happily manipulate it, and never even needs to know that
it's an automatic variable somewhere else.

And sure, you can shoot yourself in the foot that way. You can pass off a
pointer to another thread, and then return from the function without
synchronizing with that other thread properly, and now the other thread
will scribble all over your stack. But that's really nothing different
than using "alloca()" and passing off that to something that remembers the
address.

> the way the plan 9 thread library works is so different from linux's
> that they're hard to compare. program design in the two worlds is
> radically different. so your claim of 'better' is curious to me. by
> 'better' you seem to mean 'faster' and 'cleaner'. faster at least can
> be measured.

To me, the final decision on "better" tends to be a fairly wide issue.
Performance is part of it - especially infrastructure that everybody
depends on should always strive to at least _allow_ good performance, even
if not everybody ends up caring.

But the concept, to me, is more important. Basically, I do not see how you
can really have a portable and even _remotely_ efficient partial VM
sharing. And if you can't have it, then you shouldn't design the
interfaces around it.

> you speak with certainty. have you seen performance comparisons? i
> haven't, although it wouldn't surprise me to learn that there are useful
> programs for which linux outperforms plan 9, and vice versa of course.

When it comes to threads, I only see three interesting performance
metrics: how fast can you create them, how fast can you synchronize them
(both "join" and locking), and how well do you switch between them.

The locking is pretty much OS-independent, since fast locking has to be
done in user space anyway (with just the contention case falling back to
the OS, and if your app cares about performance it hopefully won't have
much contention).

So we're left with create, tear-down and switch. All of which are
_fundamentally_ faster if you just have a "share everything" model.
Create and tear-down are just increment/decrement a reference counter
(there's a spinlock involved too). Task switch is a no-op from a VM
standpoint (except we have a per-thread lazy TLB invalidate that will
trigger).

In contrast, partial sharing is a major pain. You definitely don't just do
a reference count increment for your VM.

Linus

boyd, rounin

unread,
Feb 27, 2004, 3:54:05 AM2/27/04
to
> (3) Implementation sucks. Irix and Plan-9 both get it wrong, and they
> _pay_ for it. Heavily. The Linux code is just better.

5M SLOC for the kernel? look at list.h

better? mon cul.

boyd, rounin

unread,
Feb 27, 2004, 3:57:48 AM2/27/04
to
> Can you elaborate on your synchronization example, please?

FOTFLMAO :)

nice one, don.

boyd, rounin

unread,
Feb 27, 2004, 4:07:06 AM2/27/04
to
> But yes, I'm biased.

no, you're wrong.

boyd, rounin

unread,
Feb 27, 2004, 4:06:43 AM2/27/04
to
> it might have been if Linux hadn't gone through an O(n) scheduler before
it did that context switch.

yes, the comments in the 2.4 scheduler are immediate:

danger, danger, will robinson.

boyd, rounin

unread,
Feb 27, 2004, 4:08:47 AM2/27/04
to
> And I have not _ever_ met a good C programmer that liked segments.

err, the PDP-11?

Linus Torvalds

unread,
Feb 27, 2004, 4:26:37 AM2/27/04
to

Well, what Linux has is really what I privately call a "context of
execution".

The "clone()" system call in Linux just creates a new such "context of
execution", and you can choose to arbitrarily share pretty much any OS
state, by just saying which state you want to share in a bitmap. In
addition to the bitmap there are a few pointers you pass around, the full
required state is actually

clone_flags: bitmap of how to create the new context of execution
newsp: stack pointer of new context
parent tidptr: pointer (in the parent) to the thread ID information
child tidptr: pointer (in the child) to the thread ID information
tls pointer: pointer to TLS (thread-local-storage) for the context

but not all of them are necessarily used (ie if you don't want to set TLS
or TID information, those pointers are obviously unused).

The bits you can control the context copy with are:

CSIGNAL /* signal mask to be sent at exit */

CLONE_VM /* set if VM shared between processes */
CLONE_FS /* set if fs info shared between processes */
CLONE_FILES /* set if open files shared between processes */
CLONE_SIGHAND /* set if signal handlers and blocked signals shared */
CLONE_IDLETASK /* set if new pid should be 0 (kernel only)*/
CLONE_PTRACE /* set if we want to let tracing continue on the child too */
CLONE_VFORK /* set if the parent wants the child to wake it up on mm_release */
CLONE_PARENT /* set if we want to have the same parent as the cloner */
CLONE_THREAD /* Same thread group? */
CLONE_NEWNS /* New namespace group? */
CLONE_SYSVSEM /* share system V SEM_UNDO semantics */
CLONE_SETTLS /* create a new TLS for the child */
CLONE_PARENT_SETTID /* set the TID in the parent */
CLONE_CHILD_CLEARTID /* clear the TID in the child */
CLONE_DETACHED /* Unused, ignored */
CLONE_UNTRACED /* set if the tracing process can't force CLONE_PTRACE on this clone */
CLONE_CHILD_SETTID /* set the TID in the child */
CLONE_STOPPED /* Start in stopped state */

(CSIGNAL isn't a bit - it's the low 8 bits, and it specifies the signal
you want to send to your parent when you die).

So a "fork()" is literally really just a "clone(SIGCHLD)". We're saying
that we don't want to share anything, and that we want to send a SIGCHLD
at exit.

Setting the CLONE_VM bit says that the VM gets shared. That means that
instead of copying the page tables, we just copy the pointer to the
"struct mm_struct", which describes everything in the VM, and we increment
its reference count.

There is no "partial copy". If you say that you want to share the VM, you
get the WHOLE VM. Or you will get a totally private VM. Similarly, i fyou
say that you want to share the file descriptors (CLONE_FILES), they will
all be shared: one context doing an "open()" will have that fd be valid in
all other contexts that share it.

(The difference between CLONE_FILES and a regular fork() is that a
CLONE_FILES will increment just _one_ reference count: the reference count
for the whole array of pointers to files. In contrast, a fork-like
non-shared case will create a whole new array of pointers to files, and
then for each file increment the pointer for that file).

What most "unix people" call threads is somethign that is created with
pretty much all flags set - we share pretty much everything except for the
register state and the kernel stack between the contexts. And when I say
"share", I really mean share: most of the bits end up being copying a
kernel pointer and incrementing the reference count for that object.

Some of the bits are "administrative": the VFORK bit isn't about sharing,
it's about the parent waiting until the child releases the VM back to it
(btw, that uses a "completion" structure on the parents stack). Similarly,
the SETTID/CLEARTID bits are about writing the TID ("thread ID" as opposed
to "process ID") to the VM space atomically with the creation (or in the
case of CLEARTID, teardown) of the thread. That ends up helping the thread
management (from user space) a _lot_.

(Tangential to this discussion is the TLS or "thread local storage" bit -
some architecture-specific way of indicating a small thread-specific
storage area. It's not the stack, it's just a regular allocation, and
different architectures have different ways of pointing to it. Usually
there's some architected register set aside for it).

And you can mix and match things. You can literally create a new context
that shares the file descriptors (so that one process doing an "open()"
will open files in the other one), but doesn't share the VM space.

Although some of them are interdependent - CLONE_THREAD (which is really
just "all signal state" despite the name - it has nothing to do with VM
per se) depends on CLONE_SIGHAND (which is just the set of signal handlers
associated with the context), which in turn depends on CLONE_VM (because
it doesn't make sense to be able to take a signal in different contexts
unless they share the same VM).

This has gotten fairly far off the notion of stacks and VM.. But I hope
it's clear to everybody that I heartily agree with rfork-like
functionality. It's just segmented/private stacks I can't understand.

Linus

boyd, rounin

unread,
Feb 27, 2004, 4:35:46 AM2/27/04
to
> The bits you can control the context copy with are:

what? only ~20. surely you need some more?

Linus Torvalds

unread,
Feb 27, 2004, 4:41:50 AM2/27/04
to

Think it through. You should _not_ duplicate the stack (because that
wreaks havoc with your TLB and normal usage), so what do you have left?

Once you've eliminated the impossible, what you have left, however
improbable, is the truth.

Registers.

Why are you ignoring registers? That's what you _should_ use.

For example, inside the Linux kernel, we tend to use the stack POINTER as
the thread-local state. When we allocate a new context of execution, we
allocate (depending on architecture) 8kB of memory, and it's aligned so
that if the architecture doesn't have any other registers free, we can get
at the "thread_info" structure by just doing bit masking on the stack
pointer.

That ends up being quite powerful - and it's cheap too, exactly because it
is a register, and thus fast to access. The stack itself is by no means
private - other threads can access the stack.

In fact, we used to put the whole "struct task_struct" (which is the thing
that defines a context of execution in Linux) that way, but it ends up
doing nasty things to caches when important global data structures are all
aligned on powers-of-two boundaries, so we ended up getting rid of that.

In user space, that doesn't tend to work too well, because the stack isn't
as well bounded as in the kernel, but most architectures either have lots
of registers (and then one is just used for the thread-local pointer) or
even an architected register that user space can read but not write.

One of the most problematic architectures is the x86, which doesn't have
lots of general-purpose registers (so using one of them to point to TLS
would be bad), and doesn't have any nice architected register either.
There we ended up using a segment register, however much I hate them.

We could have just made a trivial system call ("get the thread-local
pointer" from the kernel stack), but obviously there are performance
issues here.

In short: there is absolutely no reason to make the stack be private. The
only thing you need for thread-local-storage is literally just one
register, to indirect through. And it can be a fairly strange one at that,
ie it doesn't need to be able to hold a full 32-bit value.

Linus

boyd, rounin

unread,
Feb 27, 2004, 4:47:47 AM2/27/04
to
> We could have just made a trivial system call ("get the thread-local
> pointer" from the kernel stack), but obviously there are performance
> issues here.

just one? surely another 32 or 64 ...

Linus Torvalds

unread,
Feb 27, 2004, 4:48:41 AM2/27/04
to

On Fri, 27 Feb 2004, boyd, rounin wrote:
>
> > The bits you can control the context copy with are:
>
> what? only ~20. surely you need some more?

So far, no. It's been growing over the years, but not quickly.

Realize that you don't want to even control this with a very fine
granularity. Every single new thing that you allow being copied or shared
ends up being more state that you have to reference-count and keep track
of. You want to have as little as possible of that kind of state.

So keep them to big fundamental things. The VM. The file table. The signal
state. The namespace. (And those f*ing horrible SysV IPC things).

Splitting it up more would just hurt. Just out of interest, what else
could you possibly want, and why?

After all, it's not like the kernel keeps track of all that much else than
VM, files and signals.

Linus

Lucio De Re

unread,
Feb 27, 2004, 4:54:42 AM2/27/04
to
On Fri, Feb 27, 2004 at 01:46:27AM -0800, Linus Torvalds wrote:
>
> Why are you ignoring registers? That's what you _should_ use.
>
Because they are not in the base language? Because they impair
portability?

Because in my C code, I can't instantiate them as variables without
running the risk of my colleagues doing the same in a conflicting
manner?

Just out of curiosity, how do I get to the private space without
a lock? It's been a long time since I studied these things and lots
of water has flown under bridges, so I could be missing something.

++L

Linus Torvalds

unread,
Feb 27, 2004, 4:55:43 AM2/27/04
to

On Fri, 27 Feb 2004, boyd, rounin wrote:
>

Why?

You only need one thread-local pointer. You just put all your stuff offset
from that one. Realize that the kernel doesn't do anything at all with
that value - it's purely a random value that the user can set, and the
kernel doesn't even need to know that it is a pointer. You could use it as
an index into other pointers if you wanted to.

So anything but one doesn't make any sense. Where would you stop?

Am I missing something here? Linux certainly happily makes do with just
one TLS pointer.

Linus

Charles Forsyth

unread,
Feb 27, 2004, 5:00:59 AM2/27/04
to
>>Just out of curiosity, how do I get to the private space without
>>a lock? It's been a long time since I studied these things and lots
>>of water has flown under bridges, so I could be missing something.

static __inline Proc *getup(void) {
Proc *p;
__asm__( "movl %%esp, %%eax\n\t"
: "=a" (p)
);
return *(Proc **)((unsigned long)p & ~(KSTACK - 1));
};

where the process that makes the new process's stack puts a pointer to
the private space (a Proc* in this case) at the base of that stack, and where KSTACK is
a power-of-2. i don't think it can be accessed portably in Linux,
but you don't need a lock.

Linus Torvalds

unread,
Feb 27, 2004, 5:06:37 AM2/27/04
to

On Fri, 27 Feb 2004, Lucio De Re wrote:

> On Fri, Feb 27, 2004 at 01:46:27AM -0800, Linus Torvalds wrote:
> >
> > Why are you ignoring registers? That's what you _should_ use.
>
> Because they are not in the base language? Because they impair
> portability?

You literally need _one_ operation: you need the operation of "give me the
TLS pointer" (well, your thread setup code obviously needs a way to set
the pointer when creating a thread too).

The rest _is_ in the language - although if you want nice syntax, you
sometimes want more explicit language support.

Have you ever looked at those system header files? They contain a lot of
magic cruft that is specific to your compiler and your particular
architecture.

> Because in my C code, I can't instantiate them as variables without
> running the risk of my colleagues doing the same in a conflicting
> manner?
>
> Just out of curiosity, how do I get to the private space without
> a lock? It's been a long time since I studied these things and lots
> of water has flown under bridges, so I could be missing something.

In the kernel, the x86 implementation of the thread-local pointer is
literally:

/* how to get the thread information struct from C */
static inline struct thread_info *current_thread_info(void)
{
struct thread_info *ti;
__asm__("andl %%esp,%0; ":"=r" (ti) : "0" (~(THREAD_SIZE - 1)));
return ti;
}

That's it. It compiles to _two_ instructions, eg something like

movl $-8192,%eax
andl %esp,%eax

and it's all done. You literally _cannot_ do it faster or smaller. No
locking, no memory accesses, no TLB games, no NOTHING.

On some other architectures, you have

register struct thread_info *__current_thread_info __asm__("$8");
#define current_thread_info() __current_thread_info

which just tells the compiler that the thread_info pointer is in hardware
register 8. And you're done. The compiler will just automatically use %r8
whenever you do a "current_thread_info()".

In user space, there are similar things going on. These days there is more
compiler support to make it easier to create these things, but it all
boils down to having a thread-local pointer somewhere.

Linus

Lucio De Re

unread,
Feb 27, 2004, 5:08:34 AM2/27/04
to
On Fri, Feb 27, 2004 at 10:00:32AM +0000, Charles Forsyth wrote:
>
> >>Just out of curiosity, how do I get to the private space without
> >>a lock? It's been a long time since I studied these things and lots
> >>of water has flown under bridges, so I could be missing something.
>
> static __inline Proc *getup(void) {
> Proc *p;
> __asm__( "movl %%esp, %%eax\n\t"
> : "=a" (p)
> );
> return *(Proc **)((unsigned long)p & ~(KSTACK - 1));
> };
>
> where the process that makes the new process's stack puts a pointer to
> the private space (a Proc* in this case) at the base of that stack, and where KSTACK is
> a power-of-2. i don't think it can be accessed portably in Linux,
> but you don't need a lock.

Yeow! And where do I put the returned address, so that no other thread
can stomp on it? While I attempt to use it?

++L

Douglas A. Gwyn

unread,
Feb 27, 2004, 5:11:25 AM2/27/04
to
dbai...@ameritech.net wrote:
> Under the "Kernel Space and User Space" heading:
[very confused description omitted]
> | While this is a clever feat, the downside is that the overhead in maintaining
> | the stacks makes this in practice really stupid to do. They found out too
> | late that the performance went to hell. Since they had programs which used
> | the interface they could not fix it. Instead they had to introduce an additional
> | properly-written interface so that they could do what was wise with the stack
> | space. ...
I don't think any of that is correct.

Lucio De Re

unread,
Feb 27, 2004, 5:14:51 AM2/27/04
to
On Fri, Feb 27, 2004 at 02:11:27AM -0800, Linus Torvalds wrote:
>
> On Fri, 27 Feb 2004, Lucio De Re wrote:
>
> > On Fri, Feb 27, 2004 at 01:46:27AM -0800, Linus Torvalds wrote:
> > >
> > > Why are you ignoring registers? That's what you _should_ use.
> >
> > Because they are not in the base language? Because they impair
> > portability?
>
> You literally need _one_ operation: you need the operation of "give me the
> TLS pointer" (well, your thread setup code obviously needs a way to set
> the pointer when creating a thread too).
>
Yes, but...

> > Because in my C code, I can't instantiate them as variables without
> > running the risk of my colleagues doing the same in a conflicting
> > manner?
> >

Unless I'm missing the wood for the trees, this returns a pointer.
Where do I put _it_ so no other thread, choosing to do likewise,
decides to stomp all over it?

++L

Charles Forsyth

unread,
Feb 27, 2004, 5:14:54 AM2/27/04
to
>>Yeow! And where do I put the returned address, so that no other thread
>>can stomp on it? While I attempt to use it?

the process is running by then on its own stack, so the result is either in
a register (EAX say) or in a temporary on the stack but either way, it's private.

how does it get to its own stack in the first place? that's the bit
that's been causing some of the fuss here. at least for a few years,
a variant of `clone' is available that takes a pointer to the new top-of-stack for the
new process, so the creating process allocates a stack somewhere somehow
and passes that to clone. thus the new process is running on a private
stack once it starts.

Lucio De Re

unread,
Feb 27, 2004, 5:25:41 AM2/27/04