recursive mutexes
red floyd
Does POSIX define what happens when a pthread currently holding a mutex
locks the same mutex?
i.e.
void f()
{
pthread_mutex_lock(&some_mutex);
pthread_mutex_lock(&some_mutex);
}
The Linux (MDK9.1) implementation of pthreads has a non-portable mutex
attribute letting you specify what behavior you want in said situation.
I'd like to try to write portable code, though.
j...@invalid.address
> Does POSIX define what happens when a pthread currently holding a
> mutex locks the same mutex?
Yes, see
http://www.opengroup.org/onlinepubs/009695399/functions/pthread_mutex_lock.html
Joe
--
We can't all be heroes because someone has to sit on the curb and
clap as they go by.
- Will Rogers
Casper H.S. Dik
>i.e.
>void f()
>{
> pthread_mutex_lock(&some_mutex);
> pthread_mutex_lock(&some_mutex);
>}
Portable:
pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE);
Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.
red floyd
> red floyd <no....@here.dude> writes:
>
>
>
>>Does POSIX define what happens when a pthread currently holding a mutex
>>locks the same mutex?
>
>
>>i.e.
>
>
>>void f()
>>{
>> pthread_mutex_lock(&some_mutex);
>> pthread_mutex_lock(&some_mutex);
>>}
>
>
>>The Linux (MDK9.1) implementation of pthreads has a non-portable mutex
>>attribute letting you specify what behavior you want in said situation.
>
>
>>I'd like to try to write portable code, though.
>
>
> Portable:
>
> pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE);
>
> Casper
OK, thanks Joe and Casper. I guess my systems implementation is older,
and thus the recursive mutex hadn't yet been standardized or something.
So I'll simply #define something like this
#ifdef USE_NON_PORTABLE_RECURSIVE_MUTEX_
#define RECURSIVE_MUTEX_ATTR PTHREAD_MUTEX_RECURSIVE_NP /* MDK9.1 */
#else
#define RECURSIVE_MUTEX_ATTR PTHREAD_MUTEX_RECURSIVE /* other */
#endif
and use RECURSIVE_MUTEX_ATTR, so that I don't have to worry about it in
the future.
Alexander Terekhov
red floyd wrote: ...
Don't use recursive mutexes. It's akin to sex with used condoms. ;-)
regards,
alexander.
Loic Domaigne
> Don't use recursive mutexes. It's akin to sex with used condoms. ;-)
what's going on, Alexander? Are you now using now sentences that everybody can
understand?
I am not used to that ;-)
Cheers,
Loic.
--
Article posté via l'accès Usenet http://www.mes-news.com
Accès par Nnrp ou Web
SenderX
YIKES!
red floyd
May I ask why?
David Schwartz
> May I ask why?
There are two possibilities:
1) You don't know the mutex is being used recursively. In this case, the
recursive mutex will hide a serious problem.
2) You know the mutex is being recursively. In this case, just don't
lock it since you know it's already locked anyway.
The problem is that recursive mutexes hide something from you and that
something is extremely important. Consider code like this:
A) Lock mutex
B) Unlock mutex
C) Do something, assuming the mutex is unlocked
What happens if the mutex is recursive?
The only value of a recursive mutex is that it allows you to write a
function that works properly whether or not a mutex is locked. But the
caller must know that the mutex is locked anyway, so why not just have two
versions of the function? (With the one you call without a lock possibly
just being a wrapper that grabs the lock and calls the other function.)
It's just too hard and dangerous to write sensible code that works with
a mutex that might or might not start out with that mutex locked and with no
way to tell which. And the only value of recursive mutexes is that they let
you do this.
DS
SenderX
with
> a mutex that might or might not start out with that mutex locked and with
no
> way to tell which. And the only value of recursive mutexes is that they
let
> you do this.
There good for hashed locks.
David Schwartz
I can't imagine how. The only way they might help is if you have code
that grabs more than one object and they might 'happen to' have the same
lock protecting them. But if your code does this, they might also happen to
have different locks protecting them. In this case, you must enforce your
lock ordering rules or else you can deadlock.
In other words, if one thread can grab object A then object B, grabbing
lock 1 followed by lock 2, another thread could grab object C followed by
object D, grabbing lock 2 followed by lock 1. Then you deadlock.
So if you're going to use hashed locks, you may either hold two locks at
once or you may not. If you may not, recursive locks don't help.
If you may, then you must check to see what locks they refer to so that
you acquire them in the right order. While you're doing this, it's just as
easy to check if they're the same lock, and if so only acquire it once. So
recursive locks don't help you in this case.
Worse, recursive locks might lull you into thinking you don't have to
check what locks you are acquiring, and then you are vulnerable to lock
order reversal and deadlocks.
DS
Giancarlo Niccolai
Other than Mr. Schwartz reply (which I consider absolutely good), I may add
also that recursive mutexes are slower (they must check if they are locked
or not) and brings you to program less carefully about shared resource
usage. If you stop to think twice about that, you'll realize that when you
want to lock a resource, you just want to know that you are the owner of
that, and not that you've been entitled to be the owner N times...
The only reason I can see for recursive mutexes is to call recursive
functions, where the locks are so complex that writing a wrapper is not a
viable way, or to create "onion peel" like libraries where the lower levels
are completely obscure and abstracted to the higher ones, and there is
sometimes the need to "extend" the mutex locking also to some function in
the higher levels; but in both the cases there are probably better ways to
deal with the problem with a non recursive mutex, with a little more
cleaner design effort.
Giancarlo.
Martin James
> It's just too hard and dangerous to write sensible code that works
with
> a mutex that might or might not start out with that mutex locked and with
no
> way to tell which. And the only value of recursive mutexes is that they
let
> you do this.
>
Uhh.. OK I have a class protected by an internal mutex. I wish to derive
from the class & add some extra private data & methods in my descendant. I
want the private data in the child to be protected by the mutex, so I obtain
it in the child. The methods in the child call the inherited methods of the
parent, which attempt to get the mutex.again & deadlock the single caller.
Now what? I can add protected 'unlocked' methods in the parent class that
are called from public methods in the parent that get the mutex first. The
child must then ensure that it only calls the unlocked methods that don't
get the mutex. Is this messy & dangerous compared with recursive mutexes or
what?
No, I will not use two mutexes :)
Rgds,
Martin
David Schwartz
>> It's just too hard and dangerous to write sensible code that
>> works with a mutex that might or might not start out with that mutex
>> locked and with no way to tell which. And the only value of
>> recursive mutexes is that they let you do this.
> Uhh.. OK I have a class protected by an internal mutex. I wish to
> derive from the class & add some extra private data & methods in my
> descendant. I want the private data in the child to be protected by
> the mutex, so I obtain it in the child. The methods in the child
> call the inherited methods of the parent, which attempt to get the
> mutex.again & deadlock the single caller.
Right, that would be bad.
> Now what? I can add protected 'unlocked' methods in the parent
> class that are called from public methods in the parent that get the
> mutex first. The child must then ensure that it only calls the
> unlocked methods that don't get the mutex. Is this messy & dangerous
> compared with recursive mutexes or what?
No, it's cleaner and safer by *far*. What happens when someone adds some
code to the class that looks like this:
LockMutex();
while (ObjectIsNotReady())
{
UnlockMutex();
BlockOnEvent();
LockMutex();
}
DoStuff();
UnlockMutex();
Now, all of a sudden, that 'UnlockMutex()' is a nop, and the loop will
spin forever. The problem is that the functions in the inner class can
*only* look exactly like this:
LockMutex();
DoStuff();
UnlockMutex();
Otherwise, the functionality in derived classes may change what code is
run holding the mutex and what code is run without it, and that's very, very
scary.
> No, I will not use two mutexes :)
That may or may not make more sense.
DS
Martin James
> No, it's cleaner and safer by *far*. What happens when someone adds
some
> code to the class that looks like this:
>
> LockMutex();
> while (ObjectIsNotReady())
> {
> UnlockMutex();
> BlockOnEvent();
> LockMutex();
> }
> DoStuff();
> UnlockMutex();
>
> Now, all of a sudden, that 'UnlockMutex()' is a nop, and the loop will
> spin forever.
I guess it's a moot point whether a deadlock or livelock is spotted first :)
The problem is that the functions in the inner class can
> *only* look exactly like this:
>
> LockMutex();
> DoStuff();
> UnlockMutex();
Anther moot point is whether the chances of a developer adding dubious code
that attmpts to unlock a mutex inside an encapsulated object is any greater
or less than a developer accidentally calling the locking, rather than the
lockfree, method of a parent.
> Otherwise, the functionality in derived classes may change what code
is
> run holding the mutex and what code is run without it, and that's very,
very
> scary.
Once acquired, always acquired, that's what I say :)
I have to agree that code can be written to screw up almost anything. I'm
quite happy to put up with the onerous restriction of:
> LockMutex();
> DoStuff();
> UnlockMutex();
in order to avoid the complications of locking and unlocking methods in the
same class.
> > No, I will not use two mutexes :)
>
> That may or may not make more sense.
No thanks. Four classes down, I'm acquiring four mutexes & some developer
will find a way of acquiring them out-of-order.
Rgds,
Martin
David Schwartz
> No thanks. Four classes down, I'm acquiring four mutexes & some
> developer will find a way of acquiring them out-of-order.
Recursive mutexes make the risk of this *much* greater and *much* harder
to detect. What happens when some member of your class hierarchy calls
another function that needs to acquire a mutex?
DS
Uenal Mutlu
> red floyd wrote:
>
> > May I ask why?
>
> There are two possibilities:
>
> 1) You don't know the mutex is being used recursively. In this case, the
> recursive mutex will hide a serious problem.
>
> 2) You know the mutex is being recursively. In this case, just don't
> lock it since you know it's already locked anyway.
>
> The problem is that recursive mutexes hide something from you and that
> something is extremely important. Consider code like this:
>
> A) Lock mutex
> B) Unlock mutex
> C) Do something, assuming the mutex is unlocked
>
> What happens if the mutex is recursive?
Recursive locks work only for the same (current) thread.
The code above is IMO wrong, obviously it should be:
A) Lock mutex
B) Do something
C) Unlock mutex
and this is ok too (here the recursive feature is in action):
A) Lock mutex
B) Do something
C) Lock mutex
D) Do something2
E) Unlock mutex
F) Unlock mutex
C and E are redundant, however application code (the current thread)
cannot always know that it alread has the lock, though it could test for it.
But testing is unnessary because calling such a Lock() will already do
the testing implicitly, so then there is no need for an explicit testing in
applic code, it just would make code longer and slow down the performance.
To clarify:
there is Lock() without recursive feature,
and there is Lock() with recursive feature.
And: such lock objects are initialized at creation.
> The only value of a recursive mutex is that it allows you to write a
> function that works properly whether or not a mutex is locked. But the
> caller must know that the mutex is locked anyway, so why not just have two
usually the calling thread must not know that; it simply can issue a Lock()
request of its own. If it is already locked by this thread then recursion
is in effect and the lock will be granted very fast.
> versions of the function? (With the one you call without a lock possibly
> just being a wrapper that grabs the lock and calls the other function.)
>
> It's just too hard and dangerous to write sensible code that works with
> a mutex that might or might not start out with that mutex locked and with no
> way to tell which. And the only value of recursive mutexes is that they let
> you do this.
Lock objects are initialized at creation, they don't have a random state.
Recursive locks are very practical. It saves coding and prevents from self-deadlocking.
David Schwartz
"Uenal Mutlu" <52000108...@t-online.de> wrote in message
news:d66ogb$2fe$00$1...@news.t-online.com...
> C and E are redundant, however application code (the current thread)
> cannot always know that it alread has the lock, though it could test for
> it.
Yes, it can and must know that it already has the lock. A function that
operates on X should be defined as being called with X locked, or with X
unlocked. It could even take a boolean that indicates whether the caller has
locked the object or not, though this is rarely the right thing to do.
Simply put, the only thing recursive mutexes gives you is the ability to
write code that deals with X that can be called whether or not X is locked.
This is not only almost never useful, but it's almost always dangerous.
DS
Måns Rullgård
> "David Schwartz" wrote
>> red floyd wrote:
>>
>> > May I ask why?
>>
>> There are two possibilities:
>>
>> 1) You don't know the mutex is being used recursively. In this case, the
>> recursive mutex will hide a serious problem.
>>
>> 2) You know the mutex is being recursively. In this case, just don't
>> lock it since you know it's already locked anyway.
>>
>> The problem is that recursive mutexes hide something from you and that
>> something is extremely important. Consider code like this:
>>
>> A) Lock mutex
>> B) Unlock mutex
>> C) Do something, assuming the mutex is unlocked
>>
>> What happens if the mutex is recursive?
>
> Recursive locks work only for the same (current) thread.
I sure hope so.
> The code above is IMO wrong, obviously it should be:
> A) Lock mutex
> B) Do something
> C) Unlock mutex
David meant what he said. What if the something is signaling another
thread to go ahead with something that locks the mutex, and then wait
for that other thread to signal you back? Deadlock.
> and this is ok too (here the recursive feature is in action):
> A) Lock mutex
> B) Do something
> C) Lock mutex
> D) Do something2
> E) Unlock mutex
> F) Unlock mutex
>
> C and E are redundant, however application code (the current thread)
> cannot always know that it alread has the lock, though it could test
> for it.
Bad application design. A properly designed application will always
know what to expect.
>> It's just too hard and dangerous to write sensible code that
>> works with a mutex that might or might not start out with that
>> mutex locked and with no way to tell which. And the only value of
>> recursive mutexes is that they let you do this.
>
> Lock objects are initialized at creation, they don't have a random
> state. Recursive locks are very practical. It saves coding and
> prevents from self-deadlocking.
In all the years I've been programming, I have never used a recursive
mutex (except in Java, but that's another story).
--
Måns Rullgård
m...@inprovide.com
Uenal Mutlu
>
> "Uenal Mutlu" wrote
>
> > C and E are redundant, however application code (the current thread)
> > cannot always know that it alread has the lock, though it could test for
> > it.
>
> Yes, it can and must know that it already has the lock. A function that
> operates on X should be defined as being called with X locked, or with X
> unlocked. It could even take a boolean that indicates whether the caller has
> locked the object or not, though this is rarely the right thing to do.
You are unnecessarily complicating things and make the program
slower by doing these testings. These tests are not necessary if
your design is good.
> Simply put, the only thing recursive mutexes gives you is the ability to
> write code that deals with X that can be called whether or not X is locked.
Not true. It is the caller's job to call X only after having the lock.
It is not X's job to check whether the object was locked.
Do you see the difference, and the consequence, and what it means for performance?
> This is not only almost never useful, but it's almost always dangerous.
Recursive locking has less dangers than locking without recursive feature.
Proof: using recursive locking you never can block or deadlock yourself,
but using a locking method without recursive feature you can very easily deadlock yourself.
In the latter case even just blocking (ie. waiting for the lock) means deadlock!
Don't you see that?
Uenal Mutlu
> "Uenal Mutlu" writes:
>
> > "David Schwartz" wrote
> >> red floyd wrote:
> >>
> >> > May I ask why?
> >>
> >> There are two possibilities:
> >>
> >> 1) You don't know the mutex is being used recursively. In this case, the
> >> recursive mutex will hide a serious problem.
> >>
> >> 2) You know the mutex is being recursively. In this case, just don't
> >> lock it since you know it's already locked anyway.
> >>
> >> The problem is that recursive mutexes hide something from you and that
> >> something is extremely important. Consider code like this:
> >>
> >> A) Lock mutex
> >> B) Unlock mutex
> >> C) Do something, assuming the mutex is unlocked
> >>
> >> What happens if the mutex is recursive?
> >
> > Recursive locks work only for the same (current) thread.
>
> I sure hope so.
>
> > The code above is IMO wrong, obviously it should be:
> > A) Lock mutex
> > B) Do something
> > C) Unlock mutex
>
> David meant what he said. What if the something is signaling another
> thread to go ahead with something that locks the mutex, and then wait
> for that other thread to signal you back? Deadlock.
This has nothing to do with recursive locking per se, does it? I mean: the same
would happen also without recursive locking, wouldn't it?
And, apart from that I don't think this way. In my thinking each thread
knows itself only and tries to lock the shared object(s) before changing
its/their content. We are talking of locking some shared objects here, don't we?
You maybe should give a practical example in pseudocode for what you mean.
> > and this is ok too (here the recursive feature is in action):
> > A) Lock mutex
> > B) Do something
> > C) Lock mutex
> > D) Do something2
> > E) Unlock mutex
> > F) Unlock mutex
> >
> > C and E are redundant, however application code (the current thread)
> > cannot always know that it alread has the lock, though it could test
> > for it.
>
> Bad application design. A properly designed application will always
> know what to expect.
This is a shortsighted view. What do you think recursive locking is intended for?
Recursive locking has nearly no overhead if the implementation of lock()
and unlock() were properly done.
They have many many advantages.
> >> It's just too hard and dangerous to write sensible code that
> >> works with a mutex that might or might not start out with that
> >> mutex locked and with no way to tell which. And the only value of
> >> recursive mutexes is that they let you do this.
> >
> > Lock objects are initialized at creation, they don't have a random
> > state. Recursive locks are very practical. It saves coding and
> > prevents from self-deadlocking.
>
> In all the years I've been programming, I have never used a recursive
> mutex (except in Java, but that's another story).
Then you must have overlooked their real value.
Peter Dimov
> "David Schwartz" wrote
>> Yes, it can and must know that it already has the lock. A
>> function that operates on X should be defined as being called with X
>> locked, or with X unlocked. It could even take a boolean that
>> indicates whether the caller has locked the object or not, though
>> this is rarely the right thing to do.
>
> You are unnecessarily complicating things and make the program
> slower by doing these testings.
No, he's in fact making the program faster.
Uenal Mutlu
> "Uenal Mutlu" writes:
>
> >> >> you and that something is extremely important. Consider code
> >> >> like this:
> >> >>
> >> >> A) Lock mutex
> >> >> B) Unlock mutex
> >> >> C) Do something, assuming the mutex is unlocked
> >> >>
> >> >> What happens if the mutex is recursive?
> >> >
> >> > Recursive locks work only for the same (current) thread.
> >>
> >> I sure hope so.
> >>
> >> > The code above is IMO wrong, obviously it should be:
> >> > A) Lock mutex
> >> > B) Do something
> >> > C) Unlock mutex
> >>
> >> David meant what he said. What if the something is signaling another
> >> thread to go ahead with something that locks the mutex, and then wait
> >> for that other thread to signal you back? Deadlock.
> >
> > mean: the same would happen also without recursive locking, wouldn't
> > it?
>
> other thread will be able to acquire it. If the mutex is recursive,
> you can't know which level you just unlocked.
You are expecting a specific thread will get the released lock?
This can't work.
> > And, apart from that I don't think this way. In my thinking each
> > thread knows itself only and tries to lock the shared object(s)
> > before changing its/their content. We are talking of locking some
> > shared objects here, don't we?
>
> If "each thread knows itself only", how can we be talking about shared
> objects at all?
Because we are talking of threads and not processes...
> > You maybe should give a practical example in pseudocode for what you
> > mean.
>
> I don't think that's needed.
It sure would clarify what you mean.
> >> > and this is ok too (here the recursive feature is in action):
> >> > A) Lock mutex
> >> > B) Do something
> >> > C) Lock mutex
> >> > D) Do something2
> >> > E) Unlock mutex
> >> > F) Unlock mutex
> >> >
> >> > C and E are redundant, however application code (the current thread)
> >> > cannot always know that it alread has the lock, though it could test
> >> > for it.
> >>
> >> Bad application design. A properly designed application will always
> >> know what to expect.
> >
> > This is a shortsighted view. What do you think recursive locking is
> > intended for?
>
> They are for lazy programmers unwilling to learn proper design.
I would say this applies exactly to yourself.
> > Recursive locking has nearly no overhead if the implementation of lock()
> > and unlock() were properly done.
>
> I'm not worried about overhead. I'm worried about writing buggy code.
Then continue worrying. Using recursive locking is safer than using
one which doesn't have such recursive feature.
> > They have many many advantages.
>
> Prove it. Show me one problem that can't be solved better without them.
Since you don't believe me it's your turn to prove that recursive locking
is more dangerous (your saying) than using no recursive locking.
My point is: recursive locking is much safer than non-recursive locking.
> >> >> It's just too hard and dangerous to write sensible code that
> >> >> works with a mutex that might or might not start out with that
> >> >> mutex locked and with no way to tell which. And the only value of
> >> >> recursive mutexes is that they let you do this.
> >> >
> >> > Lock objects are initialized at creation, they don't have a random
> >> > state. Recursive locks are very practical. It saves coding and
> >> > prevents from self-deadlocking.
> >>
> >> In all the years I've been programming, I have never used a recursive
> >> mutex (except in Java, but that's another story).
> >
> > Then you must have overlooked their real value.
>
> Or I realized their real danger.
I doubt it because there is no danger in using recursive locking over
non-recursive locking. OTOH recursive locking has advantages over
non-recursive locking. So what? If you doubt this then prove it.
(Follow-up set to comp.programming.threads where it belongs to)
Uenal Mutlu
Are you sure? Let's see:
void X::f()
{
Lock();
for (int i = 0; i < 1000; i++)
f();
Unlock();
}
void X::g()
{
if (!IsLocked())
return;
//...do..something...
}
Are you saying the above version of g() is faster than this one: ?
void X::g()
{
//...do..something...
}
Peter Dimov
> "Peter Dimov" wrote
>> Uenal Mutlu wrote:
>>> "David Schwartz" wrote
>>>> Yes, it can and must know that it already has the lock. A
>>>> function that operates on X should be defined as being called with
>>>> X locked, or with X unlocked. It could even take a boolean that
>>>> indicates whether the caller has locked the object or not, though
>>>> this is rarely the right thing to do.
>>>
>>> You are unnecessarily complicating things and make the program
>>> slower by doing these testings.
>>
>> No, he's in fact making the program faster.
>
> Are you sure? Let's see:
>
> void X::f()
> {
> Lock();
> for (int i = 0; i < 1000; i++)
> f();
Stack overflow or deadlock here, you probably wanted to call g().
> Unlock();
> }
>
> void X::g()
> {
> if (!IsLocked())
> return;
> //...do..something...
> }
>
> Are you saying the above version of g() is faster than this one: ?
> void X::g()
> {
> //...do..something...
> }
Of course not. Why would you want to test anything if you could just use the
above? We're talking about recursive mutexes, remember?
void X::f()
{
Lock();
for (int i = 0; i < 1000; i++)
{
g();
}
Unlock();
}
void X::g()
{
Lock();
//...do..something...
Unlock();
}
compared to
void X::f()
{
Lock();
for (int i = 0; i < 1000; i++)
{
g( false );
}
Unlock();
}
void X::g( bool lock )
{
if( lock ) Lock();
//...do..something...
if( lock ) Unlock();
}
Let's overlook the fact that the correct version is of course:
void X::f()
{
Lock();
for (int i = 0; i < 1000; i++)
{
g_unlocked();
}
Unlock();
}
void X::g_unlocked()
{
//...do..something...
}
void X::g()
{
Lock();
g_unlocked();
Unlock();
}
Uenal Mutlu
Besides being slower, the first version is also buggy. And I don't know
what he or you want do if the object is not already locked. I guess you will do the following:
void X::g()
{
bool fILockedItHere = false;
if (!IsLocked())
{
Lock();
fILockedItHere = true;
}
//...do..something...
if (fILockedItHere)
Unlock();
}
But, this is not thread safe!!! :-) Do you know why?
The only consequence is: my method is the only right one, believe me! :-)
Uenal Mutlu
> Uenal Mutlu wrote:
> > "Peter Dimov" wrote
> >> Uenal Mutlu wrote:
> >>> "David Schwartz" wrote
> >>>> Yes, it can and must know that it already has the lock. A
> >>>> function that operates on X should be defined as being called with
> >>>> X locked, or with X unlocked. It could even take a boolean that
> >>>> indicates whether the caller has locked the object or not, though
> >>>> this is rarely the right thing to do.
> >>>
> >>> You are unnecessarily complicating things and make the program
> >>> slower by doing these testings.
> >>
> >> No, he's in fact making the program faster.
> >
> > Are you sure? Let's see:
> >
> > void X::f()
> > {
> > Lock();
> > for (int i = 0; i < 1000; i++)
> > f();
>
> Stack overflow or deadlock here, you probably wanted to call g().
:-) Of course g() was meant.
> > Unlock();
> > }
> >
> > void X::g()
> > {
> > if (!IsLocked())
> > return;
> > //...do..something...
> > }
> >
> > Are you saying the above version of g() is faster than this one: ?
> > void X::g()
> > {
> > //...do..something...
> > }
>
> Of course not. Why would you want to test anything if you could just use the
> above? We're talking about recursive mutexes, remember?
No I haven't. See below
> void X::f()
> {
> Lock();
>
> for (int i = 0; i < 1000; i++)
> {
> g();
> }
>
> Unlock();
> }
>
> void X::g()
> {
> Lock();
>
> //...do..something...
>
> Unlock();
> }
The above solution works only if your Lock() understands recursive locking.
Otherwise a deadlock will happen.
> compared to
>
> void X::f()
> {
> Lock();
>
> for (int i = 0; i < 1000; i++)
> {
> g( false );
> }
>
> Unlock();
> }
>
> void X::g( bool lock )
> {
> if( lock ) Lock();
>
> //...do..something...
>
> if( lock ) Unlock();
> }
I don't like the above because one usually uses a locker
class like this one to automate the unlocking:
Locker
{
Locker(mutex& Am) : m(Am)
{
m.Lock();
}
~Locker()
{
m.Unlock();
}
private:
mutex& m;
};
void X::f()
{
Locker(m);
for (int i = 0; i < 1000; i++)
g();
}
void X::g()
{
//...do..something...
}
> Let's overlook the fact that the correct version is of course:
>
> void X::f()
> {
> Lock();
>
>
> for (int i = 0; i < 1000; i++)
> {
> g_unlocked();
> }
>
> Unlock();
> }
>
>
> void X::g_unlocked()
> {
> //...do..something...
> }
>
> void X::g()
> {
> Lock();
> g_unlocked();
> Unlock();
> }
You don't need 2 versions of g() if you use recursive locking.
The overhead of recursive locking is neglectable because
it's just incrementing a counter in Lock() and decrementing it in Unlock().
Giancarlo Niccolai
>
>>
>> In all the years I've been programming, I have never used a recursive
>> mutex (except in Java, but that's another story).
>
> Then you must have overlooked their real value.
Sorry if I jump in late in the discussion, but I believed that David was
enough to fix this mental bug that you have encountered.
IMHO usage of recursive mutexes is generally an immediate and incontestable
proof of poor design.
It's not a matter of multithreading theoremes, it's a matter of parallel
activity design. Coordination is an high level activity in any parallel
process, be it coordination between firm divisions, production chains,
information processing or simply programming.
You don't want a magazine clerk of the production function to coordinate
with a magazine clerk of the provisions. The production manager will raise
the phone and talk with the provision manager (or the other way around)
when they need coordination. And believe me, they both know when they take
up the phone (lock) and when they put it down (unlock), and it's unlikely
that they can raise the phone twice without lowering it first.
Same for threads. When threads need coordination, a well designed system
will put this coordination in a place that each thread can perfectly
manage, and will make sure that coordination will 1) take less time and
computational power as possible and 2) nothing else will be done during
coordination step.
Locking a mutex means your agents are phoning each other. It's not polite to
have listener(s) hanging because you must do something while phoning them,
even if this something is somehow a recursive function call.
Locking for everything else except coordination (that is, inter-thread
communication) is bad design. Not just bad programming, but bad
understanding of parallel processing logic.
Bests,
Giancarlo Niccolai.
David Schwartz
> Recursive locking has less dangers than locking without recursive feature.
> Proof: using recursive locking you never can block or deadlock yourself,
> but using a locking method without recursive feature you can very easily
> deadlock
> yourself.
> In the latter case even just blocking (ie. waiting for the lock) means
> deadlock!
> Don't you see that?
Okay, suppose I write a function that supposed to only be called if X is
not locked. Calling it with X locked is a *bug*. Now suppose I make a
mistake -- I am only human. I call the function by mistake with X locked.
The function goes to lock X itself. I *want* this to break because it
indicates a *bug*. It is *bad* if this just magically works because then the
bug will not get discovered.
DS
David Schwartz
>> > The code above is IMO wrong, obviously it should be:
>> > A) Lock mutex
>> > B) Do something
>> > C) Unlock mutex
>> David meant what he said. What if the something is signaling another
>> thread to go ahead with something that locks the mutex, and then wait
>> for that other thread to signal you back? Deadlock.
> This has nothing to do with recursive locking per se, does it?
Yes, it does.
> I mean: the same
> would happen also without recursive locking, wouldn't it?
No. Because with locks that are not recursive, an 'unlock' function
actually unlocks something.
> And, apart from that I don't think this way. In my thinking each thread
> knows itself only and tries to lock the shared object(s) before changing
> its/their content. We are talking of locking some shared objects here,
> don't we?
> You maybe should give a practical example in pseudocode for what you mean.
Okay, let me put it as simple as possible. There are some things you can
only do when you hold a lock, like manipulate shared data. There are some
things you can only do when you don't hold a lock, like block or wait for
shared data to change. Thus code that manipulates shared data needs to know
whether it holds a lock or not.
> This is a shortsighted view. What do you think recursive locking is
> intended for?
> Recursive locking has nearly no overhead if the implementation of lock()
> and unlock() were properly done.
> They have many many advantages.
The *only* advantage is that you can write code that acquires a
particular lock without knowing whether it already holds that same lock.
However, code has to know what locks it holds *anyway* in order to be
developed sanely.
DS
Måns Rullgård
In other words, recursive mutexes are nothing but a way of papering
over design flaws in an application.
--
Måns Rullgård
m...@inprovide.com
David Schwartz
> Since you don't believe me it's your turn to prove that recursive locking
> is more dangerous (your saying) than using no recursive locking.
> My point is: recursive locking is much safer than non-recursive locking.
We've already done this over and over. Consider:
x.Lock();
DoSomeStuff();
x.Unlock();
DoSomeStuffThatTakesALongTime();
If the lock for x is not recursive, we know that we can safely take a
long time without stalling other threads that might want the x lock. If the
lock is recursive, we might unknowingly hold the x lock while we do the
stuff that takes a long time.
Here's another one:
x.Lock();
while (x.IsReservedByAnotherThread())
{
x.Unlock();
DoOtherStuffSinceXIsNotReady();
}
x.DoStuff();
x.Unlock();
This code will deadlock if the x mutex is recursive. The other thread
can never clear its reservation because this thread might still hold the x
mutex through the entire 'while' loop.
We really mean what we're saying. Really, really. Recursive mutexes are
really bad and they really do hide serious bugs.
You could write either of the two code sections above and *never* detect
the problem because it may only result in performance issues during your
tests. But in another environment where the functions erroneously held with
locks take longer, the result could be catastrophic.
DS
David Schwartz
> Are you sure? Let's see:
>
> void X::f()
> {
> Lock();
> for (int i = 0; i < 1000; i++)
> f();
> Unlock();
> }
>
> void X::g()
> {
> if (!IsLocked())
> return;
> //...do..something...
> }
>
> Are you saying the above version of g() is faster than this one: ?
> void X::g()
> {
> //...do..something...
> }
I never argued the performance issue. But here's a more realistic
example:
Without recursive mutexes:
protected:
f_locked(void)
{ // call with mutex locked
DoStuff();
}
public:
f(void)
{ // call with mutex unlocked
Lock();
f_locked();
Unlock();
}
many_f(void)
{ // call with mutex unlocked
Lock();
for(int i=0; i<1000; i++) f_locked();
Unlock();
}
With recursive mutex:
public:
f(void)
{ // call with mutex in any state
Lock();
DoStuff();
Unlock();
}
many_f(void)
{ // call with mutex in any state
Lock();
for(int i=0; i<1000; i++) f();
Unlock();
}
Which do you think is faster?
DS
doug
S'not true. You can't just increment/decrement a counter in
Lock()/Unlock() - these operations require memory barriers or they won't
work properly across multiple threads and CPUs. With recursive locks, too.
You keep arguing about silly things, proclaiming people to be wrong (e.g.
"see below..." - where!?!), and never providing evidence (only faulty code).
When someone questions you, you answer rather indignantly. Each post sounds
as though you've read the next chapter in your threading book and want to
share.
Anyways, this is interesting. More technical than alt.guitar.bass, but just
as much flamebait, if better disguised!
With baited breathe,
Doug
doug
This is exactly what I mean.
Uenal Mutlu
>
> "Uenal Mutlu" wrote
> > "Peter Dimov" wrote
> >> Uenal Mutlu wrote:
> >> > "Peter Dimov" wrote
> >> >> Uenal Mutlu wrote:
> >> >>> "David Schwartz" wrote
> >> >>>> Yes, it can and must know that it already has the lock. A
> >> >>>> function that operates on X should be defined as being called with
> >> >>>> X locked, or with X unlocked. It could even take a boolean that
> >> >>>> indicates whether the caller has locked the object or not, though
> >> >>>> this is rarely the right thing to do.
> >> >>>
> >> >>> You are unnecessarily complicating things and make the program
> >> >>> slower by doing these testings.
>
> > The overhead of recursive locking is neglectable because
> > it's just incrementing a counter in Lock() and decrementing it in
> > Unlock().
>
> S'not true. You can't just increment/decrement a counter in
> Lock()/Unlock() - these operations require memory barriers or they won't
> work properly across multiple threads and CPUs. With recursive locks, too.
You seem to not understand what I'm talking about.
Do you understand this fact: "Recursive locking is defined for the current lock owning thread only"?
Since you already have the lock you can do what ever you want since nobody
else can change anything, but you can do! You simply increment it in Lock()
and decrement it in Unlock(). Because it is safe because you already have it locked.
Uenal Mutlu
>
> I never argued the performance issue. But here's a more realistic
> example:
The example was in response to Peter's posting who wrote "No, he's in fact making the program faster."
To your question regarding the speed of recursive vs. non-recursive locks:
True, recursive locking is slower than non-recursive locking.
But recursive locking simplifies coding (no need to have a locked and unlocked
version of a function) and makes the application safer wrt self-deadlocking.
And my point is: if the recursive-locking is a fast implementation, then the
overhead of recursive locking is minimally (inc and dec). And if this is true (and it is)
then why make life complicated by not using them since they have the above
positive properties.
And, IMO it is not a good design to test in f() whether the object
was locked or not. I prefer the following "agreement" or "contract":
It is the caller's job to call f() only after having the lock.
It is not f()'s job to check whether the object was locked.
This too simplies coding, and it leads to faster programs.
Uenal Mutlu
>
> "Uenal Mutlu" wrote
>
> > is more dangerous (your saying) than using no recursive locking.
> > My point is: recursive locking is much safer than non-recursive locking.
>
> We've already done this over and over. Consider:
>
> x.Lock();
> DoSomeStuff();
> x.Unlock();
> DoSomeStuffThatTakesALongTime();
>
> If the lock for x is not recursive, we know that we can safely take a
> long time without stalling other threads that might want the x lock. If the
> lock is recursive, we might unknowingly hold the x lock while we do the
> stuff that takes a long time.
You are putting operations into 2 categories:
1) operations which can be done only if the object is locked
2) operations which can be done only if the object is not locked
This is unneccessary complication, and it's dangerous.
My point of view is: all operations on a shared object can be done only
in a locked state (#1 above). So one has to forget #2 and never assume to do #2.
An object can be modified only in a locked state, otherwise wait or
give up your time-slice back to the schedular.
The result is: simplication, safety, speed.
> Here's another one:
>
> x.Lock();
> while (x.IsReservedByAnotherThread())
> {
> x.Unlock();
> DoOtherStuffSinceXIsNotReady();
> }
> x.DoStuff();
> x.Unlock();
>
> This code will deadlock if the x mutex is recursive. The other thread
> can never clear its reservation because this thread might still hold the x
> mutex through the entire 'while' loop.
The same would happen with non-recursive locking too, wouldn't it?
Besides this, it is a bad and buggy design. You never should lock an
object in a scope and unlock it in a different scope.
> We really mean what we're saying. Really, really. Recursive mutexes are
> really bad and they really do hide serious bugs.
This is simply not true. Recursive locking is a superset of non-recursive locking.
Everthing possible in non-recursive locking is possible in recursive-locking too,
except deadlocking himself. So then how can recursive-locking be more dangerous
than non-recursive locking? This is simple basic logic.
Chris Thomasson
You do not seem to understand what "you" are talking about.
Chris Thomasson
> sounds as though you've read the next chapter in your threading book and
> want to share.
It sure does. He seems a bit inexperienced...
doug
> You seem to not understand what I'm talking about.
> Do you understand this fact: "Recursive locking is defined for the current
> lock owning thread only"?
> Since you already have the lock you can do what ever you want since nobody
> else can change anything, but you can do! You simply increment it in
> Lock()
> and decrement it in Unlock(). Because it is safe because you already have
> it locked.
>
>
Think carefully. How is this possible?
It is possible if you store 'have I got this lock?' in thread local storage.
So yes, by keeping a counter here, you don't need a memory barrier in
recursive Lock()/Unlock() *. However, when you find that you don't have the
semaphore, you still need synchronisation between threads so that the sema4
is acquired safely.
What you've ended up doing here is accessing TLS for *every* acquire - thus
making the common case of 'semaphore not acquired' slower.
(* not true on certain SMP architectures. Without a memory barrier in
Lock()/Unlock(), a thread rescheduled on a different CPU may mis-read the
recursive counter (depending on how TLS is implemented).)
Time to read your next chapter...
doug
"Chris Thomasson" <_no_damn_spam_cristom@_no_damn_comcast.net_spam> wrote in
message news:2pOdndaD_5x...@comcast.com...
It's good that he gave it a stab, but I don't understand his reluctance to
read up on existing 'well-known' practises in this area and his
unwillingness to listen to others.
For the current code I'm working on, it would take me several hours, perhaps
days, to type in the lock information his tool requires. That's not to
mention the fact that threads are created dynamically, and their behaviour
is unpredictable (I see nothing in his tool that allows you to grab this
lock OR that lock, depending on some state.). All he's done is specialise
hierarchical locking for simplified scenarios.
Uenal Mutlu
>
> > You seem to not understand what I'm talking about.
> > Do you understand this fact: "Recursive locking is defined for the current
> > lock owning thread only"?
> > Since you already have the lock you can do what ever you want since nobody
> > else can change anything, but you can do! You simply increment it in
> > Lock()
> > and decrement it in Unlock(). Because it is safe because you already have
> > it locked.
> >
> Think carefully. How is this possible?
It is really basic stuff. Ask yourself how you would extend a non-recursive
locking method to make it recursive? Recursivity starts with the 2nd Lock()
call on the same object within the same thread, true? So, after the 1st you
already have the lock, don't you? From then on just use a simple unprotected counter.
It is intended for the current _lock owning thread_ only. It is irrelevant for all
other threads because they don't have the lock.
Let me know if this makes sense to you.
> It is possible if you store 'have I got this lock?' in thread local storage.
> So yes, by keeping a counter here, you don't need a memory barrier in
> recursive Lock()/Unlock() *. However, when you find that you don't have the
> semaphore, you still need synchronisation between threads so that the sema4
> is acquired safely.
> What you've ended up doing here is accessing TLS for *every* acquire - thus
> making the common case of 'semaphore not acquired' slower.
>
> (* not true on certain SMP architectures. Without a memory barrier in
> Lock()/Unlock(), a thread rescheduled on a different CPU may mis-read the
> recursive counter (depending on how TLS is implemented).)
I think I now know what the reason of these misunderstandings are:
You and some others seem to see a lock as a property of a thread,
and by this you do "lock the thread". I on the other hand see a lock as
a property of the shared object.
I don't use any TLS, I use just the usual sharing mechanism of threads;
ie. all objects in a program can be accessed by all threads.
By this, I put locks on objects, not threads.
Each such shared object has its own mutex, and not one mutex per thread,
and of course also not one mutex for all threads (which IMO is nonsense).
Sergei Organov
> "David Schwartz" wrote
> > I never argued the performance issue. But here's a more realistic example:
>
> The example was in response to Peter's posting who wrote "No, he's in fact
> making the program faster."
>
> To your question regarding the speed of recursive vs. non-recursive
> locks: True, recursive locking is slower than non-recursive locking.
> But recursive locking simplifies coding (no need to have a locked and
> unlocked version of a function) and makes the application safer wrt
> self-deadlocking.
Yes, it simplifies coding in the sense that it makes it easier to write
"working" program that is in fact incorrect. Please try to understand
what others already told you: dead lock could be a good thing as getting
it early and correct the problem is much better than hiding the problem
behind recursive locks.
If you in fact don't care about correctness of your programs, it's
better to get rid of all the locking altogether, -- absolutely safe wrt
deadlocks, simple, and pretty fast program you will get ;)
--
Sergei.
Uenal Mutlu
>
> "Chris Thomasson" wrote in
> >> When someone questions you, you answer rather indignantly. Each post
> >> sounds as though you've read the next chapter in your threading book and
> >> want to share.
> >
> > It sure does. He seems a bit inexperienced...
> >
> It's good that he gave it a stab, but I don't understand his reluctance to
> read up on existing 'well-known' practises in this area and his
> unwillingness to listen to others.
> For the current code I'm working on, it would take me several hours, perhaps
> days, to type in the lock information his tool requires. That's not to
> mention the fact that threads are created dynamically, and their behaviour
> is unpredictable (I see nothing in his tool that allows you to grab this
> lock OR that lock, depending on some state.). All he's done is specialise
> hierarchical locking for simplified scenarios.
It is more an educational tool. It can be extended to eliminate step 1 (entering
the list of objects). This list then could be filled from the thread objects.
I will add this later.
If you put your input data into a text file then you can feed the file by using
input redirection on the command line. ie. DeadlockDetect <myinput.txt
Uenal Mutlu
> "Uenal Mutlu"
> > "David Schwartz" wrote
> > > I never argued the performance issue. But here's a more realistic example:
> >
> > The example was in response to Peter's posting who wrote "No, he's in fact
> > making the program faster."
> >
> > To your question regarding the speed of recursive vs. non-recursive
> > locks: True, recursive locking is slower than non-recursive locking.
> > But recursive locking simplifies coding (no need to have a locked and
> > unlocked version of a function) and makes the application safer wrt
> > self-deadlocking.
>
> Yes, it simplifies coding in the sense that it makes it easier to write
> "working" program that is in fact incorrect. Please try to understand
> what others already told you: dead lock could be a good thing as getting
> it early and correct the problem is much better than hiding the problem
> behind recursive locks.
But you forget the fact that recursive locks have the property to
not deadlock the thread. Then it is not valid to say that because
I called Lock() twice that this leads to deadlock, no! I'm willingly
making use of its recursivity feature because it simplifies many things,
and most importantly it is safer than non-recursive locking, though
it has a neglectable overhead. If I use a _recursive locking method_
then I'm of course aware of these facts.
> If you in fact don't care about correctness of your programs, it's
I do care very much of the correctness of my program. It has highest
priority, second is performance, third is simplicity/maintainability.
> better to get rid of all the locking altogether, -- absolutely safe wrt
> deadlocks, simple, and pretty fast program you will get ;)
:-)
Unfortunately, impossible to not use locking in nowadays projects.
Locking has become very important and will be in future.
Lockless shared access is practically seen impossible to do if
there are at least two threads sharing the same resource.
Uenal Mutlu
>
> For the current code I'm working on, it would take me several hours, perhaps
> days, to type in the lock information his tool requires.
If you strictly follow the deadlock theorem I posted here, then you can
be assured that no deadlock will happen.
> That's not to mention the fact that threads are created dynamically, and their behaviour
> is unpredictable (I see nothing in his tool that allows you to grab this
> lock OR that lock, depending on some state.).
It is intended for locks put on individual shared objects. Ie. it is not
for "locking threads" or something that.
It is irrelevant when the threads are created. It is important what the
threadproc does: ie. in which order it locks the objects.
The requirement is: you must create a list of all your shared objects (the order
is irrelevant), then in each threadproc pick the objects you want to use
just in the same order as they are on the list.
> All he's done is specialise hierarchical locking for simplified scenarios.
Not true. The deadlock theorem I formulated is an important generalization.