newbie question
<- Chameleon ->
thanks
<- Chameleon ->
>
> thanks
>
Oh! I mean in C language of course!...
Florian Groรe-Coosmann
>>Are local variables (inside functions) which are not static, thread safe?
>
Yes. Every thing on the stack is "private" to the current thread.
Cheers, Florian
David Schwartz
"<- Chameleon ->" <cham...@hotmail.NOSPAM.com> wrote in message
news:bcbpjo$f$1...@nic.grnet.gr...
It depends what you mean by thread safe.
DS
David Schwartz
"Florian Groรe-Coosmann" <flo...@grosse-coosmann.germany> wrote in message
news:bcbrho$421$1...@online.de...
All memory is shared between threads, including the stacks. While each
thread has its own stack, each thread's "own stack" is shared with all the
other threads. There is no magic thread safety to local variables. You have
to use the same locks and guards to ensure visibility as with any other
variable.
A long time ago, there were threading APIs that really did make each
thread's stack private. But I don't believe any APIs your likely to use
today do that. WIN32 doesn't. POSIX threads doesn't.
DS
SenderX
> to use the same locks and guards to ensure visibility as with any other
> variable.
Lock();
LocalCopy = *pSharedData;
Unlock();
LocalCopy is now 100% safe. No locks needed to access it.
--
The designer of the SMP and HyperThread friendly, AppCore library.
Alexander Terekhov
SenderX wrote:
[...]
> Lock();
>
> LocalCopy = *pSharedData;
>
> Unlock();
>
> LocalCopy is now 100% safe. No locks needed to access it.
And now you presume that your "LocalCopy" and *pSharedData are fully
isolated (no risk of word-tearing). But neither Windows [IIRC] nor
POSIX [I know it for sure] provide you with such guarantee. In fact,
you just can't have "100% safe" portable threaded program. Impossible.
The entire world (revised Java aside) is broken. Unfortunately.
regards,
alexander.
Eric Sosman
>
> > There is no magic thread safety to local variables. You have
> > to use the same locks and guards to ensure visibility as with any other
> > variable.
>
> Lock();
>
> LocalCopy = *pSharedData;
>
> Unlock();
>
> LocalCopy is now 100% safe. No locks needed to access it.
pGlobal = &LocalCopy;
LocalCopy is now UNsafe, and locks ARE needed to access it.
SenderX
So this pseudo-code would be unsafe?
pSharedData /* Shared */
LocalCompare, LocalNew /* Local */
/* Read, Copy */
Lock();
LocalNew = LocalCompare = *pSharedData;
Unlock();
/* Modify, " without " a lock */
LocalNew.lVar1 += 3;
LocalNew.lVar2 = 7;
LocalNew.lVar3 -= 1;
LocalNew.dwMask |= MASK_ITEM1;
/* Try and update */
Lock();
if ( *pSharedData == LocalCompare )
{
*pSharedData = LocalNew;
}
Unlock();
Where would this crash?
SenderX
Why would I point a global pointer, to a struct that resides on a threads
stack?
Pseudo:
struct Data
{
LONG lVar1;
LONG lVar2;
LONG lVar3;
LONG lVar4;
};
Data *pGlobal;
Data *pSharedData = malloc( Data );
A Thread:
Data LocalCopy;
Lock();
LocalCopy = *pSharedData;
Unlock();
/* Why would I want to do this? */
pGlobal = &LocalCopy;
When LocalCopy goes out of scope, pGlobal will point to garbage.
Alexander Terekhov
SenderX wrote:
[...]
> /* Why would I want to do this? */
> pGlobal = &LocalCopy;
Why? Well, for example:
void wait( int value ) {
mutex::guard guard( m_mutex );
if ( value == m_value ) {
waiter this_waiter( m_queue );
do { m_condvar.wait( guard ); }
while ( !this_waiter.is_reset() );
}
}
regards,
alexander.
SenderX
> > pGlobal = &LocalCopy;
>
> Why? Well, for example:
Well, I concede that point. But, you can still work on local copies ( atomic
copy ) of global state, without locking the global.
If you do indeed pass a pointer from a local object residing on a threads
stack, to a global pointer... It will need to sync.
I guess the answer to the OP, is that it depends on what you are doing with
your local stack state.
By the way, your test semaphore looks cool. Mine has been doing great as
well, it almost totally avoids the kernel under very-heavy load.
Have you measured how many times your futex skips the kernel? It should be a
lot.
;)
David Schwartz
"SenderX" <x...@xxx.xxx> wrote in message
news:iKpGa.24861$YZ2.15233@rwcrnsc53...
> Let me clarify here...
>
> So this pseudo-code would be unsafe?
>
> pSharedData /* Shared */
> LocalCompare, LocalNew /* Local */
> /* Read, Copy */
> Lock();
> LocalNew = LocalCompare = *pSharedData;
> Unlock();
>
> /* Modify, " without " a lock */
> LocalNew.lVar1 += 3;
> LocalNew.lVar2 = 7;
> LocalNew.lVar3 -= 1;
>
> LocalNew.dwMask |= MASK_ITEM1;
> /* Try and update */
> Lock();
>
> if ( *pSharedData == LocalCompare )
> {
> *pSharedData = LocalNew;
> }
>
> Unlock();
> Where would this crash?
If *pSharedData and LocalNew are both 8-bits and happen to be inside the
same 32-bit doubleword on a platform where all memory operations must take
place as 32-bits.
DS
David Schwartz
"SenderX" <x...@xxx.xxx> wrote in message
> > There is no magic thread safety to local variables. You have
> > to use the same locks and guards to ensure visibility as with any other
> > variable.
> Lock();
>
> LocalCopy = *pSharedData;
>
> Unlock();
>
> LocalCopy is now 100% safe. No locks needed to access it.
To access it assuming only this thread accesses it. But it has no magic
thread safety. You will have just as many problems with this 'LocalCopy' as
with 'pSharedData' if more than one thread tries to access it at a time.
Everything is thread safe if only one thread touches it.
DS
David Schwartz
"SenderX" <x...@xxx.xxx> wrote in message
> I guess the answer to the OP, is that it depends on what you are doing
with
> your local stack state.
Exactly! A variable itself cannot be thread safe, it's just memory like
any other memory. It's how you access it (whether you use proper locks or
atomic instructions when there's a possibility of concurrent accesses or
memory visibility issues).
DS
SenderX
magic
> thread safety. You will have just as many problems with this 'LocalCopy'
as
> with 'pSharedData' if more than one thread tries to access it at a time.
I would NOT let other threads touch local stack data. That is bad mojo man.
LocalCopy is safe to the thread that owns the stack it resides on. Period.
SenderX
> > LocalNew.lVar1 += 3;
> > LocalNew.lVar2 = 7;
> > LocalNew.lVar3 -= 1;
There way over 8-bits... Anyway, that code is safe.
LocalNew and LocalCompare are not visible to any other thread, but the owner
of the local stack that they reside on. So, what is going to modify them?
Florian Groรe-Coosmann
David Schwartz schrieb:
> "Florian Groรe-Coosmann" <flo...@grosse-coosmann.germany> wrote in message
> news:bcbrho$421$1...@online.de...
>
>
>><- Chameleon -> schrieb:
>>
>>>>Are local variables (inside functions) which are not static, thread
> safe?
>
>>Yes. Every thing on the stack is "private" to the current thread.
>>
>>Cheers, Florian
>
>
> All memory is shared between threads, including the stacks. While each
> thread has its own stack, each thread's "own stack" is shared with all the
> other threads. There is no magic thread safety to local variables. You have
> to use the same locks and guards to ensure visibility as with any other
> variable.
>
> A long time ago, there were threading APIs that really did make each
> thread's stack private. But I don't believe any APIs your likely to use
> today do that. WIN32 doesn't. POSIX threads doesn't.
This is the nearly exact definition, but you either have to export the
address of a stack object to let "other" manipulate it (other may be
the system of another thread) or the other thread does a wild pointer
use.
But this doesn't seem the reason behind the question to me. Chameleon
has done the question for common use, I think. If an answer exceeds
more than one or two lines, do you think a newbie will understand it
at all? "private" in quotes was intended to be flexible enough for
both the beginner's as the expert's use.
Cheers, Florian
David Schwartz
"Florian Groรe-Coosmann" <flo...@grosse-coosmann.germany> wrote in message
> This is the nearly exact definition, but you either have to export the
> address of a stack object to let "other" manipulate it (other may be
> the system of another thread) or the other thread does a wild pointer
> use.
> But this doesn't seem the reason behind the question to me. Chameleon
> has done the question for common use, I think. If an answer exceeds
> more than one or two lines, do you think a newbie will understand it
> at all? "private" in quotes was intended to be flexible enough for
> both the beginner's as the expert's use.
If a newbie doesn't understand what 'thread safe' means, he can't write
multithreaded code and expect it to work. There is no correct short answer.
DS
David Schwartz
"SenderX" <x...@xxx.xxx> wrote in message
> > To access it assuming only this thread accesses it. But it has no
> > magic
> > thread safety. You will have just as many problems with this 'LocalCopy'
> > as
> > with 'pSharedData' if more than one thread tries to access it at a time.
> I would NOT let other threads touch local stack data. That is bad mojo
man.
Then how would you implement, say, a parallel section to multiply a
matrix by a constant? Copy the constant someplace else? Why?
> LocalCopy is safe to the thread that owns the stack it resides on. Period.
Bullshit. No area of memory is any safer than any other.
DS
Patrick TJ McPhee
David Schwartz <dav...@webmaster.com> wrote:
% If a newbie doesn't understand what 'thread safe' means, he can't write
% multithreaded code and expect it to work. There is no correct short answer.
But there are some completely moronic long answers.
--
Patrick TJ McPhee
East York Canada
pt...@interlog.com
SenderX
So, all access to memory... Shared or Local... Must be protected by a lock?
lol.
SenderX
So, your code would be as follows:
VOID SomeFunc()
{
DATA Local1;
Lock();
Local1.Val1 = 2;
Local2.Var2 = 3;
Unlock();
}
?
David Schwartz
"SenderX" <x...@xxx.xxx> wrote in message
> > Bullshit. No area of memory is any safer than any other.
> So, all access to memory... Shared or Local... Must be protected by a
lock?
>
> lol.
I didn't say that. No area of memory "must be protected by a lock".
Every area of memory must be protected by a lock if two threads might access
it concurrently.
DS
David Schwartz
"SenderX" <x...@xxx.xxx> wrote in message
> > Bullshit. No area of memory is any safer than any other.
>
> So, your code would be as follows:
>
>
> VOID SomeFunc()
> {
> DATA Local1;
>
> Lock();
>
> Local1.Val1 = 2;
>
> Local2.Var2 = 3;
>
> Unlock();
> }
No more than yours would be:
int c, d;
int main(void)
{
c=2;
d=3;
}
Local variables and global variables are equally thread safe. They both
require locks if you can't provide exclusion or visbility another way.
DS
David Butenhof
> Are local variables (inside functions) which are not static, thread safe?
Um, as you've no doubt noticed, answers you get in this newsgroup may not
always be simple.
The simple answer, given what you almost certainly mean by the question, is
"yes".
That is, given
int foo () {
int i;
(do something with 'i' that DOES NOT involve taking its address and
storing that address where another thread may see it)
}
Then the use of 'i' in foo() is thread-safe.
--
/--------------------[ David.B...@hp.com ]--------------------\
| Hewlett-Packard Company Tru64 UNIX & VMS Thread Architect |
| My book: http://www.awl.com/cseng/titles/0-201-63392-2/ |
\----[ http://homepage.mac.com/dbutenhof/Threads/Threads.html ]---/
Sergey P. Derevyago
> > Are local variables (inside functions) which are not static, thread safe?
> Um, as you've no doubt noticed, answers you get in this newsgroup may not
> always be simple.
> The simple answer, given what you almost certainly mean by the question, is
> "yes".
>
> That is, given
>
> int foo () {
> int i;
>
> (do something with 'i' that DOES NOT involve taking its address and
> storing that address where another thread may see it)
> }
>
> Then the use of 'i' in foo() is thread-safe.
>
void g(char* ch);
void f()
{
char a[2];
// ...
a[0]='"'; // fine, but...
// ...
g(&ac[1]); // ... from this point the use of a[0] might not be thread-safe
// while we've passed the address of a[1]
}
--
With all respect, Sergey. http://cpp3.virtualave.net/
mailto : ders at skeptik.net
Alexander Terekhov
David Butenhof wrote:
[...]
> Then the use of 'i' in foo() is thread-safe.
Again, NOTHING is {portably} "thread-safe" as long as the term "memory
location" remains NOT defined. POSIX is hopelessly broken.
regards,
alexander.
P.S. Hi ders. ;-)
David Butenhof
> David Butenhof wrote:
> [...]
>> Then the use of 'i' in foo() is thread-safe.
>
> Again, NOTHING is {portably} "thread-safe" as long as the term "memory
> location" remains NOT defined. POSIX is hopelessly broken.
A local variable is either on stack or in a register. Registers are
inherently thread-private, trivially "thread-safe".
A POSIX thread stack is "almost always" a distinct set of pages of memory
that are not shared implicitly. Memory granularity isn't an issue. While an
implementation that didn't do guard pages could, in theory, combine
multiple stacks into a page that'd be silly and pointless. And if they
shared a cache line (the only realistic way you'd have access conflicts
between threads that "thought" they were dealing with private data), it'd
be horrendously inefficient -- and absolutely unsustainable.
So as long as you don't make a stack address known to another thread, you're
safe. Not, it is true, "by law", but by any rational or reasonable
practice.
Now, Sergey's response touches on a more subtle point: that variable A might
become "unsafe" because the address of "distinct" variable B has been
exposed to other threads. He chose to use elements of an array, but the
effect is more devestatingly subtle if you're actually dealing with
separate scalar variables that happen to be allocated adjacently. In
theory, they need not even be DECLARED adjacent to each other, because the
compiler is not required to allocate in the order variables are declared
(though most do and some real ancient C code would break if it didn't).
All this is true, and a million other things besides. The point is that all
this discussion is overcomplicating a simple generic answer to a simple
generic question.
If I'd made the first response I would have added a paragraph AFTER the
simple answer to make it clear that the simple answer was, well, "simple".
Since I followed over 20 rambling argumentative posts about the
complications, there was no need for anything but a simple summary.
And, Alexander, as far as being "hopelessly broken", that's a silly
overstatement. Yes, POSIX is not complete. It can't be, because it's not a
language or binary interface specification. It is intended to provide tools
to enable the possibility of portable applications; but it cannot hope to
make all possible applications portable.
Memory granularity is not a simple issue. There are many hardware, language,
and code generation issues that were entirely beyond the scope of the
standard. (Don't think we didn't try.) Even the formal memory model
wouldn't have dealt with this; there are too many factors. We even
considered defining particular types for which we could require that a
compiler generate atomic references -- but that would be too limiting for
real-world code development. (Nor were we convinced we could make any
compiler writers buy into the idea.) We couldn't invent language syntax
(though junk ideas like a "shared" attribute were tossed around... which
would have had even worse effects on real-world code). We could have tried
to concisely DEFINE the problem -- but that doesn't help portability,
either; only talking about the problems. The terms we can meaningfully use
in normative or even informative text within POSIX are really too limiting.
The final summary is that these problems don't affect most code. You MAY run
into problems when ADJACENT "memory locations" are shared without identical
synchronization. You're unlikely to run into problems if your "data units"
are allocated from heap, because heap data has to be aligned on a
sufficiently large granularity to support safe access. (Consider this a
critical "quality of implementation" issue.) If you declare global or
static data that will be shared, make sure you align and separate "sharing
units". Don't allocate from heap a single "block" from which you intend to
allocate multiple "sharing units". Don't EVER give out your home stack
address to a stranger. ;-)
Alexander Terekhov
David Butenhof wrote:
[...]
> And, Alexander, as far as being "hopelessly broken", that's a silly
> overstatement.
First off, I wouldn't file a defect report if it were not broken. And,
as for "hopelessly", well, that's because that DR was rejected with a
rather silly sort-of commentary about "atomic operations".
> Yes, POSIX is not complete.
That's BIG understatement. The use of NOT DEFINED term makes it BROKEN.
Well, speaking of 4.10 (and memory isolation aside for a moment), what
do you think about the following "would be addition" to it:
The informal semantics of acquire/release memory synchronization
operations can be defined as: (1) before an acquire access or an
ordinary access is allowed to perform with respect to any other
processor, all preceding acquire accesses must be performed (in the
program order), and (2) before a release access is allowed to perform
with respect to any other processor, all preceding acquire accesses,
release accesses, and all ordinary accesses must be performed. An act
of acquiring mutex ownership can be viewed as performing an acquire
operation. An act of releasing mutex ownership can be viewed as
performing a release operation. An acquire operation prevents all
subsequent load and store accesses from being performed before the
acquire operation -- it can be viewed as store or load operation that
is performed in conjunction with the "hoist-load" and the "hoist-store"
reordering constraints (barriers) applied with respect to the
subsequent load and store accesses. A release operation prevents all
preceding load and store accesses from being performed after the
release operation -- it can be viewed as store or load operation that
is performed in conjunction with the "sink-load" and the "sink-store"
reordering constraints/barriers applied with respect to the preceding
load and store accesses. An act of acquiring a read lock on a read-
write lock can be viewed as an operation that is performed in
conjunction with the "hoist-load" barrier only; without "hoist-store"
barrier that is needed for an acquire operation on a mutex. An act of
releasing a read lock on a read-write lock can be viewed as an
operation that is performed in conjunction with the "sink-load"
barrier only; without "sink-store" barrier that is needed for a
release operation on a mutex.
<?>
TIA.
regards,
alexander.
--
"With SCO's announcement of upping the ante in its lawsuit against IBM
(now seeking up to $50 billion in damages), the world is in agreement
that SCO owns the world of comedy. As such, SCO is rightfully claiming
that all jokes, humor, and laugher is derived from their lawsuit with
IBM and belongs to SCO's IP. Furthermore, SCO has sent out letters to
over 1,500 well know comics, comedians, and talk show hosts indicating
that they may be infringing on SCO'c IP and may be subject to damage
claims by SCO. As all humor belongs to SCO, SCO has also sent a letter
to R. Dangerfield demanding that he stop being funny or he may face
legal action."
-- unknown
Dragan Cvetkovic
> David Butenhof wrote:
>
> > Yes, POSIX is not complete.
>
> That's BIG understatement. The use of NOT DEFINED term makes it BROKEN.
Err, completeness and consistency are two different notions. And as we know
since Goedel, you can't have both ...
Bye, Dragan
--
Dragan Cvetkovic,
To be or not to be is true. G. Boole No it isn't. L. E. J. Brouwer
David Butenhof
>
> David Butenhof wrote:
> [...]
>> And, Alexander, as far as being "hopelessly broken", that's a silly
>> overstatement.
>
> First off, I wouldn't file a defect report if it were not broken. And,
> as for "hopelessly", well, that's because that DR was rejected with a
> rather silly sort-of commentary about "atomic operations".
While I believe that you probably wouldn't have filed a defect report if you
didn't BELIEVE it was broken, but I hope you'll admit that the fact you
filed a defect report doesn't constitute PROOF that the standard is broken.
>> Yes, POSIX is not complete.
>
> That's BIG understatement. The use of NOT DEFINED term makes it BROKEN.
No, it doesn't. There are many things that don't need to be defined, and
your statement is so wildly overgeneral as to be meaningless.
> Well, speaking of 4.10 (and memory isolation aside for a moment), what
> do you think about the following "would be addition" to it:
What do I think of it? It's overblown, rambling, introduces new terms and
concepts with no definition and which furthermore have little or any
relationship to the issue you're presumably attempting to address. And, at
best, when you've gotten past all this, you've only expanded and
complicated the memory visibility guarantees already provided for mutexes.
You have NOT dealt at all with the subject of this particular little rathole
in which we find ourselves, which is the "false sharing" of adjacent memory
locations with separate synchronization protocol.
I think your enormous paragraph could perhaps be substantially edited into a
more detailed non-formal description of the existing mutex memory
visibility rules. I'm not convinced that doing so would provide any great
benefit to users or implementors. In fact, it's not only less clear, but
also longer and more complicated than the Leslie Lamport/Garret Swart
formal memory model that was rejected from an early draft of the standard
as "too complicated".
> --
> "With SCO's announcement of upping the ante in its lawsuit against IBM
> (now seeking up to $50 billion in damages), the world is in agreement
> that SCO owns the world of comedy. As such, SCO is rightfully claiming
> that all jokes, humor, and laugher is derived from their lawsuit with
> IBM and belongs to SCO's IP. Furthermore, SCO has sent out letters to
> over 1,500 well know comics, comedians, and talk show hosts indicating
> that they may be infringing on SCO'c IP and may be subject to damage
> claims by SCO. As all humor belongs to SCO, SCO has also sent a letter
> to R. Dangerfield demanding that he stop being funny or he may face
> legal action."
> -- unknown
Now THAT, I like. ;-)
Alexander Terekhov
David Butenhof wrote:
[...]
> While I believe that you probably wouldn't have filed a defect report if you
> didn't BELIEVE it was broken, but I hope you'll admit that the fact you
> filed a defect report doesn't constitute PROOF that the standard is broken.
Sure it's broken. Not saying what is "memory location", the XBD/4.10,
basically, doesn't say anything at all.
[...]
> > Well, speaking of 4.10 (and memory isolation aside for a moment), what
> > do you think about the following "would be addition" to it:
>
> What do I think of it? It's overblown, rambling, introduces new terms and
> concepts with no definition and which furthermore have little or any
> relationship to the issue you're presumably attempting to address. And, at
> best, when you've gotten past all this, you've only expanded and
> complicated the memory visibility guarantees already provided for mutexes.
I need it for atomic<>-like C-stuff with "built-in" memory barriers.
http://www.terekhov.de/pthread_refcount_t/experimental/refcount.cpp
and things like DCSI/DCCI (in addition to non-blocking ref.counting).
>
> You have NOT dealt at all with the subject of this particular little rathole
> in which we find ourselves, which is the "false sharing" of adjacent memory
> locations with separate synchronization protocol.
http://groups.google.com/groups?selm=3DB98DED.E9F10490%40web.de
(Subject: Re: Memory isolation)
>
> I think your enormous paragraph could perhaps be substantially edited into a
> more detailed non-formal description of the existing mutex memory
> visibility rules. I'm not convinced that doing so would provide any great
> benefit to users or implementors. In fact, it's not only less clear, but
> also longer and more complicated than the Leslie Lamport/Garret Swart
> formal memory model that was rejected from an early draft of the standard
> as "too complicated".
Do you have a link? Please.
[... SCO ...]
> Now THAT, I like. ;-)
http://messages.yahoo.com/bbs?.mm=FN&action=m&board=1600684464&tid=cald&sid=1600684464&mid=15132
<quote>
New SCO Press Release
by: martin_lvnv
For Immediate Release:
SCO has determined through further research that even IBM mainframes
use propriety SCO methods and concepts without a license. Since almost
all computer software derives from unix, and mainframes use software,
we have determined that IBM mainframes use SCO IP. SCO hereby declares
them all "illegal" and demands they be unplugged.
IBM mainframes use "usernames" and "passwords", concepts that were
proprietary to SCO since they were in used the oldest version of SYSV
unix. We were surprised to find IBM has been using these concepts
without license since before SCO was even incorporated. Misuse dates
back to some of IBMs earliest computers in the 1950s.
"We knew IBM had been transferring SCO technology from sys V Unix to
Linux" stated SCO CEO Daryl MacBride "but we didn't realize they were
transferring the technology to MVS, IBMs mainframe operating system
too". SCO will be asking for damages going all the way back to the
dawn of the computer era in the 1950s. It seems all of IBMs success
over the decades has been from stealing SCO IP, so we're going to sue
them for every dime they ever made. And then double it for punitive
damages.
Asked to comment on the new allegations, an IBM spokesman responded
"Blow Me".
In other news SCO announces the licensing of the "username" and
"password" concepts to Microsoft. A Microsoft spokesman admitted:
"We thought we stole it from IBM or Apple or acquired in from some
company we crushed. As soon as we determined it was SCO's
legitimate IP, we licensed these important concepts".
Including Microsofts one time license fee of $84 million dollars,
SCO announced they turned a small profit this quarter.
</quote>
regards,
alexander.
Joseph Seigh
Alexander Terekhov wrote:
>
> David Butenhof wrote:
> [...]
> > > Well, speaking of 4.10 (and memory isolation aside for a moment), what
> > > do you think about the following "would be addition" to it:
> >
> > What do I think of it? It's overblown, rambling, introduces new terms and
> > concepts with no definition and which furthermore have little or any
> > relationship to the issue you're presumably attempting to address. And, at
> > best, when you've gotten past all this, you've only expanded and
> > complicated the memory visibility guarantees already provided for mutexes.
>
> I need it for atomic<>-like C-stuff with "built-in" memory barriers.
>
> http://www.terekhov.de/pthread_refcount_t/experimental/refcount.cpp
>
> and things like DCSI/DCCI (in addition to non-blocking ref.counting).
I told you that you should have stuck with the definitions that tied into mutex
visibility rules. :)
> >
> > I think your enormous paragraph could perhaps be substantially edited into a
> > more detailed non-formal description of the existing mutex memory
> > visibility rules. I'm not convinced that doing so would provide any great
> > benefit to users or implementors. In fact, it's not only less clear, but
> > also longer and more complicated than the Leslie Lamport/Garret Swart
> > formal memory model that was rejected from an early draft of the standard
> > as "too complicated".
>
> Do you have a link? Please.
>
ditto if there is one.
> [... SCO ...]
>
> > Now THAT, I like. ;-)
>
> http://messages.yahoo.com/bbs?.mm=FN&action=m&board=1600684464&tid=cald&sid=1600684464&mid=15132
>
> <quote>
>
> New SCO Press Release
> by: martin_lvnv
...
Actually, I like the fact that SCO is suing IBM for something (RCU) that IBM invented in the
80's even before Sequent came up with it.
Joe Seigh
Alexander Terekhov
Joseph Seigh wrote:
[...]
> Actually, I like the fact that SCO is suing IBM for something (RCU)
> that IBM invented in the 80's even before Sequent came up with it.
Yeah.
http://slashdot.org/comments.pl?sid=67903&cid=6222502
regards,
alexander.
Joseph Seigh
Hmm... I just noticed that the paper on Tornado by the K42 guys mentions
the 4,809,168 patent but seems to think it only applies to uniprocessors
despite the fact that the term multiprocessing is used multiple times.
Joe Seigh
David Butenhof
> David Butenhof wrote:
> [...]
>> While I believe that you probably wouldn't have filed a defect report if
>> you didn't BELIEVE it was broken, but I hope you'll admit that the fact
>> you filed a defect report doesn't constitute PROOF that the standard is
>> broken.
>
> Sure it's broken. Not saying what is "memory location", the XBD/4.10,
> basically, doesn't say anything at all.
YOU haven't said anything about "memory location", either. That was my
point: your so-called "correction" has nothing to do with this issue even
if it was correct or complete on its own. You're talking simply about
visibility and ordering, not granularity.
If you can find a clear and reasonably bulletproof way to describe realistic
necessary granularity limits for conforming implementations across a wide
range of hardware platforms, without making language changes or
"reinterpreting" existing language standard text, please feel free to write
and propose. If you're interested in language changes, that's fine, but in
that case you take it up with the language standard committees, not POSIX.
Note that the original POSIX memory model included a special type with size
and alignment that allowed it to be treated as "a separate variable as far
as meeting the [No races on m] condition is concerned", which was called
(and the name provides a clue as to how old this is) rt_thread_coherency_t.
And a macro, as part of the C binding, called rt_thread_align() to declare
a union of an rt_thread_coherency_t array and some user type such that the
alignment and size would provide isolation. This does place a burden on the
compiler to provide sufficient alignment guarantees. (If the compiler
"naturally aligns" int/long as on most RISC machines, and if that alignment
is sufficient for isolation, then we're all set -- otherwise
rt_thread_align() would have required special non-standard compiler
support.)
There was some concern about this, from several sides. For one, because it
might demand special compiler support, which POSIX didn't have the power to
grant. (Now it's easy to say that any compiler failing to support threaded
programming is dead in the water -- back THEN it was threaded programming
that was dead in the water if compilers didn't support it, so we had to
keep a light touch.) For another, of course, it demanded application
programmer diligence in using these awkward constructs everywhere it
mattered, and that doesn't look good. ;-)
> [...]
>> > Well, speaking of 4.10 (and memory isolation aside for a moment), what
>> > do you think about the following "would be addition" to it:
>>
>> What do I think of it? It's overblown, rambling, introduces new terms and
>> concepts with no definition and which furthermore have little or any
>> relationship to the issue you're presumably attempting to address. And,
>> at best, when you've gotten past all this, you've only expanded and
>> complicated the memory visibility guarantees already provided for
>> mutexes.
>
> I need it for atomic<>-like C-stuff with "built-in" memory barriers.
>
> http://www.terekhov.de/pthread_refcount_t/experimental/refcount.cpp
>
> and things like DCSI/DCCI (in addition to non-blocking ref.counting).
Yes, I understand that. However, you haven't said a thing that enables any
of those to be done portably. You're merely talking about MUTEX memory
visibility using different and more obscure terms than the current memory
model, without adding any real value. You haven't even defined the terms
you like to use for barrier semantics.
Alexander, the text you wrote is not even close to being something that
could be considered for bringing into a standard. It's not even close
enough to tempt someone to finish the job for you.
>> You have NOT dealt at all with the subject of this particular little
>> rathole in which we find ourselves, which is the "false sharing" of
>> adjacent memory locations with separate synchronization protocol.
>
> http://groups.google.com/groups?selm=3DB98DED.E9F10490%40web.de
> (Subject: Re: Memory isolation)
>
>> I think your enormous paragraph could perhaps be substantially edited
>> into a more detailed non-formal description of the existing mutex memory
>> visibility rules. I'm not convinced that doing so would provide any great
>> benefit to users or implementors. In fact, it's not only less clear, but
>> also longer and more complicated than the Leslie Lamport/Garret Swart
>> formal memory model that was rejected from an early draft of the standard
>> as "too complicated".
>
> Do you have a link? Please.
POSIX never put drafts online, and the memory model existed only in really,
really early drafts. Nor did we work much by email back then; most
everything was printed material circulated at physical meetings.
I do have a copy in my archives, and I could type it in some day if I get
motivated -- but the complete section is 6 pages, and I'd have to be fairly
thoroughly motivated. ;-)
There's really nothing magical in there. It's pretty "common sense" stuff.
It COULD easily be extended to describe barrier operations, or even
rewritten be be based on barrier and atomic ops with mutexes layered on top
of that. Someday perhaps I'll give that a try.
> [... SCO ...]
>
>> Now THAT, I like. ;-)
>
>
http://messages.yahoo.com/bbs?.mm=FN&action=m&board=1600684464&tid=cald&sid=1600684464&mid=15132
Now THAT one is getting close enough to some of the serious reports that it
leaves me thinking "maybe we shouldn't let them see this" rather than "ha
ha". ;-)
The big motivation for the formation of OSF was that AT&T's revised
post-breakup System V licensing (no longer under telecom monopoly
restrictions) was not only far more expensive but also far more
restrictive. I never saw a copy, but I did hear rumors that it included
clauses implying (or at least that many potential licensees feared could be
interpreted as saying) that AT&T would own at least some if not all IP
developed by licensees from or on that code base. This is clearly the basis
of some of SCO's claims, as well as "humor" like this article. In contrast,
the earlier and cheaper perpetual licenses on which existing "family" OSs
like BSD, AIX, ULTRIX were based were also supposedly less restrictive: the
idea of OSF was to build a new modern OS out of something protected by the
old license; which ended up being BSD hosted on a Mach kernel (though AIX
had also been considered).
Alexander Terekhov
David Butenhof wrote:
>
> Alexander Terekhov wrote:
>
> > David Butenhof wrote:
> > [...]
> >> While I believe that you probably wouldn't have filed a defect report if
> >> you didn't BELIEVE it was broken, but I hope you'll admit that the fact
> >> you filed a defect report doesn't constitute PROOF that the standard is
> >> broken.
> >
> > Sure it's broken. Not saying what is "memory location", the XBD/4.10,
> > basically, doesn't say anything at all.
>
> YOU haven't said anything about "memory location", either. That was my
> point: your so-called "correction" has nothing to do with this issue even
> if it was correct or complete on its own. You're talking simply about
> visibility and ordering, not granularity.
I never said that it was meant to be a "correction". I said "would be
addition".
>
> If you can find a clear and reasonably bulletproof way to describe realistic
> necessary granularity limits for conforming implementations across a wide
> range of hardware platforms, without making language changes or
> "reinterpreting" existing language standard text, please feel free to write
> and propose. If you're interested in language changes, that's fine, but in
> that case you take it up with the language standard committees, not POSIX.
http://terekhov.de/your-next-book
http://groups.google.com/groups?threadm=3EEB1397.89F8DF7C%40web.de
(Subject: Re: thread safety)
>
> Note that the original POSIX memory model included a special type with size
> and alignment that allowed it to be treated as "a separate variable as far
> as meeting the [No races on m] condition is concerned", which was called
> (and the name provides a clue as to how old this is) rt_thread_coherency_t.
> And a macro, as part of the C binding, called rt_thread_align() to declare
> a union of an rt_thread_coherency_t array and some user type such that the
> alignment and size would provide isolation. This does place a burden on the
> compiler to provide sufficient alignment guarantees.
http://groups.google.com/groups?selm=3DE63165.904EF6B9%40web.de
(Subject: Re: Memory isolation)
[...]
> http://messages.yahoo.com/bbs?.mm=FN&action=m&board=1600684464&tid=cald&sid=1600684464&mid=15132
>
> Now THAT one is getting close enough to some of the serious reports that it
> leaves me thinking "maybe we shouldn't let them see this" rather than "ha
> ha". ;-)
>
> The big motivation for the formation of OSF was that AT&T's revised
> post-breakup System V licensing (no longer under telecom monopoly
> restrictions) was not only far more expensive but also far more
> restrictive. I never saw a copy, but I did hear rumors that it included
> clauses implying (or at least that many potential licensees feared could be
> interpreted as saying) that AT&T would own at least some if not all IP
> developed by licensees from or on that code base.
Yup. That's section 2.01 in the "Exhibit A":
http://www.sco.com/scosource/ExhibitA.qxd.pdf
> This is clearly the basis
> of some of SCO's claims, as well as "humor" like this article. In contrast,
However, don't miss THIS from the "Exhibit C":
http://www.sco.com/scosource/ExhibitC.qxd.pdf
<quote>
Regarding Section 2.01, we agree that modifications and
derivative works prepared by or for you are owned by you.
However, ownership of any portion or portions of SOFTWARE
PRODUCTS included in any such modification or derivative
work remains with us.
</quote>
"SOFTWARE PRODUCTS" is the original System V code. Well, of course,
SCO will surely claim that things like "=" and "+" continue to belong
to SCO... and that "x += y" is nothing but the intentionally obscured
(and "stolen") SCOs "x = x + y" stuff, I guess.
regards,
alexander.
David Butenhof
> David Butenhof wrote:
>>
>> Alexander Terekhov wrote:
>>
>> > David Butenhof wrote:
>> > [...]
>> >> While I believe that you probably wouldn't have filed a defect report
>> >> if you didn't BELIEVE it was broken, but I hope you'll admit that the
>> >> fact you filed a defect report doesn't constitute PROOF that the
>> >> standard is broken.
>> >
>> > Sure it's broken. Not saying what is "memory location", the XBD/4.10,
>> > basically, doesn't say anything at all.
>>
>> YOU haven't said anything about "memory location", either. That was my
>> point: your so-called "correction" has nothing to do with this issue even
>> if it was correct or complete on its own. You're talking simply about
>> visibility and ordering, not granularity.
>
> I never said that it was meant to be a "correction". I said "would be
> addition".
It's not an "addition", because it doesn't ADD anything. It merely changes
what's already there, restating the existing model in terms of vague and
undefined concepts.
Bringing barriers/fences into the standard doesn't ADD anything unless the
change provides a way to DO something you couldn't already do. You're
leaving mutexes as the only defined OPERATIONS while adding convoluted,
vague, and unnecessary "background" to explain them.
>> If you can find a clear and reasonably bulletproof way to describe
>> realistic necessary granularity limits for conforming implementations
>> across a wide range of hardware platforms, without making language
>> changes or "reinterpreting" existing language standard text, please feel
>> free to write and propose. If you're interested in language changes,
>> that's fine, but in that case you take it up with the language standard
>> committees, not POSIX.
>
> http://terekhov.de/your-next-book
> http://groups.google.com/groups?threadm=3EEB1397.89F8DF7C%40web.de
> (Subject: Re: thread safety)
So you've mentioned it to someone involved with languages? Um, OK. Did that
accomplish anything?
>> Note that the original POSIX memory model included a special type with
>> size and alignment that allowed it to be treated as "a separate variable
>> as far as meeting the [No races on m] condition is concerned", which was
>> called (and the name provides a clue as to how old this is)
>> rt_thread_coherency_t. And a macro, as part of the C binding, called
>> rt_thread_align() to declare a union of an rt_thread_coherency_t array
>> and some user type such that the alignment and size would provide
>> isolation. This does place a burden on the compiler to provide sufficient
>> alignment guarantees.
>
> http://groups.google.com/groups?selm=3DE63165.904EF6B9%40web.de
> (Subject: Re: Memory isolation)
Yeah, unions are awkward in a lot of ways. The intent was to try to come up
with SOMETHING that could be used IN C (C++ was entirely out of scope) to
address the problem. The union solution wasn't considered acceptable, and
there was nothing else POSIX could say to help -- which is why the standard
says nothing at all. Nobody's arguing that the lack of specification is
good. However, nobody could come up with a useful and practical solution
within the scope of POSIX.
> However, don't miss THIS from the "Exhibit C":
>
> http://www.sco.com/scosource/ExhibitC.qxd.pdf
>
> <quote>
>
> Regarding Section 2.01, we agree that modifications and
> derivative works prepared by or for you are owned by you.
> However, ownership of any portion or portions of SOFTWARE
> PRODUCTS included in any such modification or derivative
> work remains with us.
>
> </quote>
>
> "SOFTWARE PRODUCTS" is the original System V code. Well, of course,
> SCO will surely claim that things like "=" and "+" continue to belong
> to SCO... and that "x += y" is nothing but the intentionally obscured
> (and "stolen") SCOs "x = x + y" stuff, I guess.
Well, OBVIOUSLY! ;-)
Alexander Terekhov
Joseph Seigh wrote:
[...]
> Hmm... I just noticed that the paper on Tornado by the K42 guys mentions
> the 4,809,168 patent but seems to think it only applies to uniprocessors
> despite the fact that the term multiprocessing is used multiple times.
Joe, forget it. SCO is the "true" owner.
http://www.sco.com/ibmlawsuit/amendedcomplaintjune16.html
<quote>
139. IBM has even gone so far as to publish the DYNIX/ptx copyright as
part of the source code and documentation contribution of UNIX-derived
RCU technology it has improperly made available to the open source
community. The following copyright attribution is found in Linux
kernel 2.4.x:
Copyright (c) International Business Machines Corp., 2001 This
program is free software; you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version. This program is distributed in
the hope that it will be useful, but WITHOUT ANY WARRANTY; without
even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE. See the GNU General Public License for more
details. You should have received a copy of the GNU General Public
License along with this program; if not, write to the Free
Software Foundation, Inc., 59 Temple Place - Suite 330, Boston,
MA 02111-1307, USA. Author: Dipankar Sarma (Based on a Dynix/ptx
implementation by Paul Mckenney.
140. This publication of the RCU copyright is an example of IBM's
blatant disregard of SCO's rights to control the use of the Software
Product, including derivative works and modifications thereof,
pursuant to ยง2.05 of the Sequent Software Agreement.
</quote>
regards,
alexander.
Joseph Seigh
Alexander Terekhov wrote:
>
> Joseph Seigh wrote:
> [...]
> > Hmm... I just noticed that the paper on Tornado by the K42 guys mentions
> > the 4,809,168 patent but seems to think it only applies to uniprocessors
> > despite the fact that the term multiprocessing is used multiple times.
>
> Joe, forget it. SCO is the "true" owner.
>
> http://www.sco.com/ibmlawsuit/amendedcomplaintjune16.html
>
...
Except that it (RCU) was a mainframe method, not a Unix method, that was
transfered to Linux. SCO contractual rights, if any, only cover transfer
of Unix methods.
Joe Seigh
Joseph Seigh
Alexander Terekhov wrote:
>
> Joseph Seigh wrote:
> [...]
> > Hmm... I just noticed that the paper on Tornado by the K42 guys mentions
> > the 4,809,168 patent but seems to think it only applies to uniprocessors
> > despite the fact that the term multiprocessing is used multiple times.
>
> Joe, forget it. SCO is the "true" owner.
>
> http://www.sco.com/ibmlawsuit/amendedcomplaintjune16.html
>
Alexander Terekhov
Joseph Seigh wrote:
[...]
> Except that it (RCU) was a mainframe method, not a Unix method, that was
> transfered to Linux. SCO contractual rights, if any, only cover transfer
> of Unix methods.
But all mainframes technologies also belong to SCO (as The UNIX
Technology derivative works). They simply forgot to revoke z/OS
(OS/390, etc.) license(s). They'll do it next week.
http://www.opengroup.org/csq/view.mhtml?RID=ibm%2FXU1%2F2
regards,
alexander.
--
"In 1997, Darl McBride, now SCO's chief executive, sued his then
employer, IKON Office Solutions, and won a settlement that he
says was worth multiple millions. (IKON acknowledges the
settlement but disputes the amount.)"
Alexander Terekhov
David Butenhof wrote:
[...]
> It's not an "addition", because it doesn't ADD anything.
Sure it does add something. In fact, it was supposed to replace bits in
the section "1/11" of my ``big mess'' document:
http://www.terekhov.de/pthread_refcount_t/draft-edits.txt
> It merely changes
> what's already there, restating the existing model in terms of vague and
> undefined concepts.
You may argue that the language is bad. That's fine. But it does
introduce some "concepts" that would allow to have "less constrained"
operations. That's the point.
>
> Bringing barriers/fences into the standard doesn't ADD anything unless the
> change provides a way to DO something you couldn't already do. You're
> leaving mutexes as the only defined OPERATIONS while adding convoluted,
> vague, and unnecessary "background" to explain them.
See the doc above. It does have more operations. atomic<>-stuff would
add even more.
>
> >> If you can find a clear and reasonably bulletproof way to describe
> >> realistic necessary granularity limits for conforming implementations
> >> across a wide range of hardware platforms, without making language
> >> changes or "reinterpreting" existing language standard text, please feel
> >> free to write and propose. If you're interested in language changes,
> >> that's fine, but in that case you take it up with the language standard
> >> committees, not POSIX.
> >
> > http://terekhov.de/your-next-book
> > http://groups.google.com/groups?threadm=3EEB1397.89F8DF7C%40web.de
> > (Subject: Re: thread safety)
>
> So you've mentioned it to someone involved with languages? Um, OK. Did that
> accomplish anything?
Well. Not much. Thus far. ;-)
regards,
alexander.
David Butenhof
> David Butenhof wrote:
> [...]
>> It's not an "addition", because it doesn't ADD anything.
>
> Sure it does add something. In fact, it was supposed to replace bits in
> the section "1/11" of my ``big mess'' document:
No, it was clearly INTENDED to add something, but it fails to accomplish
this goal.
> You may argue that the language is bad. That's fine. But it does
> introduce some "concepts" that would allow to have "less constrained"
> operations. That's the point.
No, the point is that it doesn't "introduce" anything. In fact, the biggest
problem is that it casually REFERENCES concepts that are NOT introduced, or
even properly defined.
>> Bringing barriers/fences into the standard doesn't ADD anything unless
>> the change provides a way to DO something you couldn't already do. You're
>> leaving mutexes as the only defined OPERATIONS while adding convoluted,
>> vague, and unnecessary "background" to explain them.
>
> See the doc above. It does have more operations. atomic<>-stuff would
> add even more.
Perhaps, but the text you quoted neither provides new operations nor any
basis or definitions on which new operations might be added later. It does
nothing but REDEFINE the existing mutex behavior in convoluted terms that
ADD nothing.
Alexander Terekhov
David Butenhof wrote:
[...]
> > See the doc above. It does have more operations. atomic<>-stuff would
> > add even more.
>
> Perhaps, but the text you quoted neither provides new operations nor any
> basis or definitions on which new operations might be added later. It does
> nothing but REDEFINE the existing mutex behavior in convoluted terms that
> ADD nothing.
"... An act of acquiring a read lock on a read-
write lock can be viewed as an operation that is performed in
conjunction with the "hoist-load" barrier only; without "hoist-store"
barrier that is needed for an acquire operation on a mutex. An act of
releasing a read lock on a read-write lock can be viewed as an
operation that is performed in conjunction with the "sink-load"
barrier only; without "sink-store" barrier that is needed for a
release operation on a mutex."
And, you know, the fact POSIX threading API doesn't have rdunlock()
and wrunlock() operations [separate calls] makes me kinda wonder WHY
"someone" was definitely NOT thinking in the direction of reducing
the amount of reordering constraints imposed on the implementation
[i.e. compiler, in the first place; things like IPA aside] to the
absolute minimum that's actually needed when the current threading
API was created/voted. You keep telling me about the mutex... think
of a read-write lock for a second, please.
regards,
alexander.
David Butenhof
> David Butenhof wrote:
> [...]
>> > See the doc above. It does have more operations. atomic<>-stuff would
>> > add even more.
>>
>> Perhaps, but the text you quoted neither provides new operations nor any
>> basis or definitions on which new operations might be added later. It
>> does nothing but REDEFINE the existing mutex behavior in convoluted terms
>> that ADD nothing.
>
> "... An act of acquiring a read lock on a read-
> write lock can be viewed as an operation that is performed in
> conjunction with the "hoist-load" barrier only; without "hoist-store"
> barrier that is needed for an acquire operation on a mutex. An act of
> releasing a read lock on a read-write lock can be viewed as an
> operation that is performed in conjunction with the "sink-load"
> barrier only; without "sink-store" barrier that is needed for a
> release operation on a mutex."
I thought I already commented on this -- perhaps it was a similar discussion
in another context. Anway, although the APPLICATION code that HOLDS a
read-write lock for read access doesn't need any guarantees on the ordering
of any writes it makes while holding the lock, that's only part of the
stroy.
The IMPLEMENTATION code that acquires and releases the read-write lock
usually DOES need such guarantee -- because it has to communicate with
other readers and with current and future writers. That is, it needs to
maintain the rwlock state, and may need to queue itself to wait for an
active writer. It also needs to be able to see any waiting writers that
block while the read-lock is active.
Now, there's no reason that such implementation requirements need be
extended explicitly to application code, and of course there's always the
chance that some implementation might be able to manage all the necessary
state using some single atomic queue (for example) such that no barriers
are necessary. However, this requirement already HAS been so written, and
to RETRACT it is potentially a big deal. This is a lot like the
pthread_cond_signal/broadcast... but even worse because POSIX has no real
definitions or background for the sort of subtle distinctions you're trying
to make.
Which brings us back to the main issue, that your text is sloppy and
rambling, and throws around new terms without providing any concrete
definitions. POSIX does not have barriers, and introducing them is going to
take a lot more work than just throwing some references into the middle of
a paragraph on the synchronization memory model.
> And, you know, the fact POSIX threading API doesn't have rdunlock()
> and wrunlock() operations [separate calls] makes me kinda wonder WHY
> "someone" was definitely NOT thinking in the direction of reducing
> the amount of reordering constraints imposed on the implementation
> [i.e. compiler, in the first place; things like IPA aside] to the
> absolute minimum that's actually needed when the current threading
> API was created/voted. You keep telling me about the mutex... think
> of a read-write lock for a second, please.
Believe me, I've thought about it for a lot more than a second.
As for the combined unlock function, all I can say is that I always thought
that was a bad idea too. Not just for potential efficiency, but also
because it restricts error checking. That is, if someone accidentally calls
pthread_rwlock_wrunlock() twice, you can cause it to fail. But readers
generally aren't identified, merely counted, so if a writer calls
pthread_rwlock_unlock() twice, and on the second call there's one or more
readers, the call can do nothing but "successfully" unlock for read. The
alternative would be to keep a list of all active readers; but that gets
messy quickly when you have to account for a thread possibly holding
multiple read locks at the same time.
You can take this as evidence that POSIX doesn't always listen to me. ;-)