Recursive lock?
axter
Also, does the POSIX standard support recursive lock via
pthread_mutex_lock?
In another thread, someone stated that a code requirement was bad
because it required recursive lock, so I'm trying to understand his/her
position.
I believe recursive lock means you can call the lock multiple times
from the same thread and it won't block your thread.
I'm not sure if it means you need to call unlock the same amount of
times you called lock.
Jarek Kucypera
> pthread_mutex_lock?
AFAIR yes, you must specify it in mutex attributes
> I believe recursive lock means you can call the lock multiple times
> from the same thread and it won't block your thread.
Your belief is right ;)
> I'm not sure if it means you need to call unlock the same amount of
> times you called lock.
Yes, you do need to balance locks with unlocks.
> and is it bad, and if so why?
I don't think it is (bad). I haven't followed the discussion above,
but in general case the gratest benefit is (IMO), it supports splitting
your code into a number of independent blocks (routines, methods,
objects, whatever) that can be smoothly reused.
J.K.
David Butenhof
> What exactly is meant by recursive lock, and is it bad, and if so why?
You got it right below, so I'll follow up with the "bad and why" at the end.
> Also, does the POSIX standard support recursive lock via
> pthread_mutex_lock?
Yes; you use the same functions, after initialization with an attributes
object specifying the recursive mutex type.
Beware; although you can use a recursive mutex for pthread_cond_wait(),
this will only UNLOCK, not necessarily RELEASE the mutex. (That is, if
your thread holds the lock at a "depth" greater than 1, you've only
decreased the counter by one.) This is good, because holding a lot
implies that you have unstable data predicates being protected by the
lock. (Otherwise you would have UNLOCKED so that other threads can make
concurrent progress on the shared data.)
> In another thread, someone stated that a code requirement was bad
> because it required recursive lock, so I'm trying to understand his/her
> position.
>
> I believe recursive lock means you can call the lock multiple times
> from the same thread and it won't block your thread.
> I'm not sure if it means you need to call unlock the same amount of
> times you called lock.
Yes, because the idea is that each of the recursive lock regions is
independent. If they KNEW about each other, you wouldn't need a
recursive mutex because you could keep track yourself more efficiently.
So as long as any of your code still has unstable shared data
predicates, the lock must remain held by the thread. Only when ALL lock
regions have announced their predicates are stable (by unlocking) can
the lock be RELEASED to allow access by other threads.
This is the evil of recursive mutexes. They sound convenient and "safe".
But concurrency is the "bread and butter", the "raison d'etre" of
threading. The longer you hold a mutex (also known as "bottleneck"), the
more you restrict (or prevent) concurrency.
If your predicates are stable, you should unlock. If you're locking
mutexes recursively, you're saying that you don't know (or don't care)
whether your predicates are stable. You're not understanding, or
managing, your concurrency. And that means you're not writing a clean
and efficient threaded program.
This is not to say that all threaded programs need to be "clean and
efficient". Sometimes people use threads simply as a program structuring
device -- a way to separate call stacks, just a portable way to write
co-routines. In that case (like our recently prolific contributer Uenal
Mutlu), you may not care about concurrency, or even performance. It'll
cost you many wasted context switches (especially if your threads are
timesliced, because timeslices will frequently happen while your thread
is holding a lock, and other threads will just need to block until the
owner runs again to complete and unlock).
This seems to me a bit like using a coffee mug to pound nails, but it's
your mug and your nails; and if that's what makes you happy it's OK by
me. ;-)
--
Dave Butenhof, David.B...@hp.com
HP Utility Pricing software, POSIX thread consultant
Manageability Solutions Lab (MSL), Hewlett-Packard Company
110 Spit Brook Road, ZK2/3-Q18, Nashua, NH 03062
doug
"David Butenhof" <david.b...@hp.com> wrote in message
news:xy%ie.5692$p76....@news.cpqcorp.net...
This is the part Uenal doesn't get. (He also doesn't understand that
recursive locks need synchronisation between threads on SMP systems, even
the case where you already own the lock (because you have to find out).
[Actually, you can use TLS for this, but then you make the fast-path slower,
so what's the point?])
Michel
> 8-< snipped discussion about recursive locks
Thanks for your as usual insightful reply, maybe its time for a
"Recursive locks considered harmful" (to paraphrase Dijkstra)
paper/article signed Mr Butenhof ;)
Anyways I have a question for the developers on the recursive locks is
evil camp and the win32 platform. Have you made non recursirve wrappers
for the recursive CriticalSection and mutex type?
/Michel
Dragan Cvetkovic
> David Butenhof wrote:
> > 8-< snipped discussion about recursive locks
>
> Thanks for your as usual insightful reply, maybe its time for a "Recursive
> locks considered harmful" (to paraphrase Dijkstra) paper/article signed Mr
> Butenhof ;)
No, make that a chapter in _the_ book :-)
Dragan
--
Dragan Cvetkovic,
To be or not to be is true. G. Boole No it isn't. L. E. J. Brouwer
!!! Sender/From address is bogus. Use reply-to one !!!
doug
"Michel" <michel...@swipnet.se> wrote in message
news:7N4je.24739$d5.1...@newsb.telia.net...
Yeah, we wrap them up. We do this because we write our code to an
'isolation layer' api. We then write an isolation layer for each operating
system, mapping the api calls to the primitives supported on that machine.
It's not the fastest way to do it, but it's portable, and in debug builds we
can add in code for things like:
- double acquires
- invalid frees
- high contention (lots of threads waiting for some lock)
- long holds (a thread holds a lock for a long time)
- hierarchy checking
Means we don't have to touch the app code when we port.
Doug
Michel
> Yeah, we wrap them up. We do this because we write our code to an
> 'isolation layer' api. We then write an isolation layer for each operating
> system, mapping the api calls to the primitives supported on that machine.
>
> It's not the fastest way to do it, but it's portable, and in debug builds we
> can add in code for things like:
> - double acquires
> - invalid frees
> - high contention (lots of threads waiting for some lock)
> - long holds (a thread holds a lock for a long time)
> - hierarchy checking
>
> Means we don't have to touch the app code when we port.
>
Event though I work primarily in win32 land I wrap and isolate all apis
as well, for a lot of the reasons you mention, and to get a
C++/exception safe interface. Actually i don't see why It would hamper
performance or be as fast as native, at least not when diagnostics is
compiled away. But what typically does the wrapper do on double acquire
(assert, throw) and what differences is there in debug and release builds?
/Michel
doug
> doug wrote:
>> Yeah, we wrap them up. We do this because we write our code to an
>> 'isolation layer' api. We then write an isolation layer for each
>> operating system, mapping the api calls to the primitives supported on
>> that machine.
>>
>> It's not the fastest way to do it, but it's portable, and in debug builds
>> we can add in code for things like:
>> - double acquires
>> - invalid frees
>> - high contention (lots of threads waiting for some lock)
>> - long holds (a thread holds a lock for a long time)
>> - hierarchy checking
>>
>> Means we don't have to touch the app code when we port.
>>
>
> Event though I work primarily in win32 land I wrap and isolate all apis as
> well, for a lot of the reasons you mention, and to get a C++/exception
> safe interface. Actually i don't see why It would hamper performance or be
> as fast as native, at least not when diagnostics is compiled away.
absolutely true - my mishtake!
> But what typically does the wrapper do on double acquire (assert, throw)
We assert and bring the server down. We get a stack trace of all threads
and a core dump (on suitable OSs), and on a diags build we get information
on who is holding what. (plus a load of other stuff.)
We view things like this as coding errors, so kill the product asap, so
there's no chance of missing it.
> and what differences is there in debug and release builds?
Well, there's a lot more code running when you do these debug checks. You
have to lock around these checks too. All this adds up a rather different
profile when you're running the code. It's good, but it's not rock solid -
there's still a chance that a bug doesn't show up in debug testing due to
the code differences, and doesn't show up in release builds until you ship
it. Lovely.
This type of bug would typically not be a double acuqire, invalid free, or
hierarchy violdation - *as long as you cover every single code path in your
debug testing*. Rather, it'd be a performance or scalability problem.
Static analysis would solve all this, but is usually impossible for the
places where it's really needed. For today.
>
> /Michel
Phil Frisbie, Jr.
> Anyways I have a question for the developers on the recursive locks is
> evil camp and the win32 platform. Have you made non recursirve wrappers
> for the recursive CriticalSection and mutex type?
Yes, I have my own C wrapper for common thread functions wrapped around native
Windows threads and pthreads for ease of portability. It only supports a
non-recursive mutex.
I was bit hard with recursive mutexes a few years ago. Code that worked fine on
Linux and Windows would lock up on Solaris. Until that Solaris report I did not
know I was calling a mutex recursively, so I modified my mutex code to be more
strict on error checking and found it! I simply had to refactor two functions to
fix it.
--
Phil Frisbie, Jr.
Hawk Software
http://www.hawksoft.com
axter
FYI:
I'm currently working on a C++ synchronized smart pointer, that allows
thread safe access to an object.
See following link:
http://code.axter.com/sync_ptr.h
http://code.axter.com/sync_ctrl.h
Now, my code is currently using recursive lock, but I plan to change it
to non-recursive by adding a referencing counting object to the RefLock
class.
This should allow any one to use this wrapper class to make an instance
of any object thread safe, by automatically synchronizing access to the
pointer.
If you use a reference counting object, it shouldn't be that hard to
avoid recursive locking by linking the reference counting object with
the mutex.
axter
Thanks for the good detailed explanation.
I was trying to determine if I was using recursive locking in my code,
and if so, I wanted to know if I needed to change it.
>From your explanation, and others, my code does have recursive locking.
Peter Koch Larsen
"Michel" <michel...@swipnet.se> skrev i en meddelelse
news:7N4je.24739$d5.1...@newsb.telia.net...
I have my "light" mutex class implemented as a critical section:
class lightweight_mutex
{
bool islocked;
CRITICAL_SECTION cs;
....
void acquire()
{
acquire_critical_section(cs);
assert(!locked);
locked = true;
}
};
just pseudo-code... i do not remember the exact code. The check disappears
in release-mode.
/Peter
Michel
>>Event though I work primarily in win32 land I wrap and isolate all apis as
>>well, for a lot of the reasons you mention, and to get a C++/exception
>>safe interface. Actually i don't see why It would hamper performance or be
>>as fast as native, at least not when diagnostics is compiled away.
>
>
> absolutely true - my mishtake!
>
>
>>But what typically does the wrapper do on double acquire (assert, throw)
>
>
> We assert and bring the server down. We get a stack trace of all threads
> and a core dump (on suitable OSs), and on a diags build we get information
> on who is holding what. (plus a load of other stuff.)
>
> We view things like this as coding errors, so kill the product asap, so
> there's no chance of missing it.
This sounds like a reasonable approach, and the one that spring to my
mind as well.
Another question is how you identify the locks/sections in the output
log files?
> Well, there's a lot more code running when you do these debug checks. You
> have to lock around these checks too. All this adds up a rather different
> profile when you're running the code. It's good, but it's not rock solid -
> there's still a chance that a bug doesn't show up in debug testing due to
> the code differences, and doesn't show up in release builds until you ship
> it. Lovely.
Amen, I think a lots of us has been up late nights to find those kinds
of bugs.
> This type of bug would typically not be a double acuqire, invalid free, or
> hierarchy violdation - *as long as you cover every single code path in your
> debug testing*. Rather, it'd be a performance or scalability problem.
I most often see race conditions crop up in release builds, but as
experience builds up these should be rarer.
/Michel
doug
> doug wrote:
>>>Event though I work primarily in win32 land I wrap and isolate all apis
>>>as well, for a lot of the reasons you mention, and to get a
>>>C++/exception safe interface. Actually i don't see why It would hamper
>>>performance or be as fast as native, at least not when diagnostics is
>>>compiled away.
>>
>>
>> absolutely true - my mishtake!
>>
>>
>>>But what typically does the wrapper do on double acquire (assert, throw)
>>
>>
>> We assert and bring the server down. We get a stack trace of all threads
>> and a core dump (on suitable OSs), and on a diags build we get
>> information on who is holding what. (plus a load of other stuff.)
>>
>> We view things like this as coding errors, so kill the product asap, so
>> there's no chance of missing it.
>
> This sounds like a reasonable approach, and the one that spring to my mind
> as well.
>
> Another question is how you identify the locks/sections in the output log
> files?
It's done as a macro, to you have access to __FILE__ and __LINE__
preprocessor stuff. We then have small macros in our text editors to 'jump'
to the location in question with a keypress.
It sometimes amazes me how much effort we put into infrastructure like this.
I reckon about half of our time goes on codecheck stuff like this,
regressible unit test frameworks, text editor macros, etc.