Library requirement for mutex destroy
Noob
Hello,
I'm using a proprietary library (no source code) which requires the user
to provide 4 mutex-related functions (create, delete, lock, unlock).
AFAICT, if I were using a POSIX OS, I could make trivial wrappers around
pthread_mutex_init
pthread_mutex_destroy
pthread_mutex_lock
pthread_mutex_unlock
But this library has specific requirements for the delete function:
If delete is called on a locked mutex, the function must return an error
code to signal the error to the function's caller.
However, the documentation for pthread_mutex_destroy explicitly states:
http://www.opengroup.org/onlinepubs/009695399/functions/pthread_mutex_init.html
"""
It shall be safe to destroy an initialized mutex that is unlocked.
Attempting to destroy a locked mutex results in undefined behavior.
"""
This means that my delete function can't just call pthread_mutex_destroy
because I would invoke UB if the mutex is locked, and I wouldn't be able
to return the requested error code, right?
So... Does this mean I have to keep track whether the mutex is locked?
(It would appear so.)
Or perhaps I could mess with the OS code? (shudder)
I'd like to tell the library author that it is unreasonable to impose
overhead on every lock and unlock operation just to catch an obvious (?)
programming error on delete. What do you think?
N.B. the library requires mutex with recursive semantic.
Regards.
David Schwartz
> But this library has specific requirements for the delete function:
>
> If delete is called on a locked mutex, the function must return an error
> code to signal the error to the function's caller.
That is, IMO, fundamentally busted behavior, unless it only means
locked by the calling thread.
> This means that my delete function can't just call pthread_mutex_destroy
> because I would invoke UB if the mutex is locked, and I wouldn't be able
> to return the requested error code, right?
>
> So... Does this mean I have to keep track whether the mutex is locked?
> (It would appear so.)
> Or perhaps I could mess with the OS code? (shudder)
>
> I'd like to tell the library author that it is unreasonable to impose
> overhead on every lock and unlock operation just to catch an obvious (?)
> programming error on delete. What do you think?
>
> N.B. the library requires mutex with recursive semantic.
Since the mutex is recursive, one of two things are the case:
1) The requirement only applies if the calling thread holds the lock
on the mutex, in which case you have to keep track of the lock count
anyway. (Otherwise, how would you know what to do in the unlock
operation?) The library is being reasonable, though quirky.
2) The requirement applies even if another thread holds the mutex.
This is a real pain to implement, and the requirement is fundamentally
busted.
DS
Noob
> Noob wrote:
>
>> But this library has specific requirements for the delete function:
>>
>> If delete is called on a locked mutex, the function must return an error
>> code to signal the error to the function's caller.
>
> That is, IMO, fundamentally busted behavior, unless it only means
> locked by the calling thread.
Could you expand on /why/ you think it is busted behavior?
(I need ammo to try to convince the library authors.)
The exact requirements are:
1) Return an error when attempting to delete "a mutex currently owned by
another thread".
2) Return an error when attempting to delete "a mutex currently owned by
the calling thread".
NB: by "owned" I assume they mean "locked".
Since a locked mutex is so either by the calling thread, or by another
thread (duh!) it seems the two requirements boil down to:
Return an error (and, I assume, leave the mutex alone) when attempting
to delete a locked mutex.
>> This means that my delete function can't just call pthread_mutex_destroy
>> because I would invoke UB if the mutex is locked, and I wouldn't be able
>> to return the requested error code, right?
>>
>> So... Does this mean I have to keep track whether the mutex is locked?
>> (It would appear so.)
>> Or perhaps I could mess with the OS code? (shudder)
>>
>> I'd like to tell the library author that it is unreasonable to impose
>> overhead on every lock and unlock operation just to catch an obvious (?)
>> programming error on delete. What do you think?
>>
>> N.B. the library requires mutex with recursive semantic.
>
> Since the mutex is recursive, one of two things are the case:
>
> 1) The requirement only applies if the calling thread holds the lock
> on the mutex, in which case you have to keep track of the lock count
> anyway. (Otherwise, how would you know what to do in the unlock
> operation?) The library is being reasonable, though quirky.
The OS I'm using provides recursive mutex. Therefore, I don't think I
have to deal with lock counts, as the OS takes care of that.
Unless I need to know in delete whether the mutex is locked.
Trying to delete a locked mutex sounds like a serious programmer error.
Mutex destruction (and creation) should be done in well-defined parts of
the application, when one can be sure the mutex is not in use anymore,
right?
> 2) The requirement applies even if another thread holds the mutex.
> This is a real pain to implement, and the requirement is fundamentally
> busted.
According to the library's spec, delete may be called at any time, by
any thread, and it is the user's responsibility to return an error and
leave the mutex untouched if the mutex is currently locked...
Regards.
Eric Sosman
> David Schwartz wrote:
>
>> Noob wrote:
>>
>>> But this library has specific requirements for the delete function:
>>>
>>> If delete is called on a locked mutex, the function must return an error
>>> code to signal the error to the function's caller.
>> That is, IMO, fundamentally busted behavior, unless it only means
>> locked by the calling thread.
>
> Could you expand on /why/ you think it is busted behavior?
> (I need ammo to try to convince the library authors.)
>
> The exact requirements are:
>
> 1) Return an error when attempting to delete "a mutex currently owned by
> another thread".
This is broken, period. Suppose thread T1 tries to delete the
mutex and discovers (somehow) that it is locked by T2. T1 leaves
the mutex alone, the deletion attempt returns an error, and all is
well.
But suppose T1 comes along at a moment when the mutex is unlocked,
and succeeds in deleting it. Two clock cycles later, T2 tries to lock
the destroyed mutex, and the fertilizer hits the fan. The fact that T1
found the mutex unlocked is no proof that the mutex is "unused," or
that T2 might not attempt to use it again.
If T1 deletes a mutex that T2 might still want to use, the
application is broken. If T1 can find the mutex locked by T2, it
must be the case that T2 still wants to use it. Therefore the mere
possibility that delete-while-locked-elsewhere could occur is proof
that the design is broken.
> 2) Return an error when attempting to delete "a mutex currently owned by
> the calling thread".
This isn't quite as clear-cut, but it still seems dubious.
> [...]
> Trying to delete a locked mutex sounds like a serious programmer error.
Yes, but the point is more general: Trying to delete an "in use"
mutex is a serious programmer error. A mutex in the locked state is
obviously in use, but it is a mistake to reason that a mutex in the
unlocked state is not in use.
> According to the library's spec, delete may be called at any time, by
> any thread, and it is the user's responsibility to return an error and
> leave the mutex untouched if the mutex is currently locked...
Broken. The danger isn't when deletion fails because the mutex is
locked, but when deletion succeeds because the mutex is unlocked -- but
is still in use.
Måns Rullgård
> On Oct 28, 6:27�am, Noob <r...@127.0.0.1> wrote:
>
>> But this library has specific requirements for the delete function:
>>
>> If delete is called on a locked mutex, the function must return an error
>> code to signal the error to the function's caller.
>
> That is, IMO, fundamentally busted behavior, unless it only means
> locked by the calling thread.
I agree. I wouldn't go anywhere near that library. Find another one
written by someone with half a brain.
That said, you could use pthread_mutex_trylock() to find out whether
or not the muxtex is locked.
--
M�ns Rullg�rd
ma...@mansr.com
Noob
> Noob wrote:
>
>> David Schwartz wrote:
>>
>>> Noob wrote:
>>>
>>>> But this library has specific requirements for the delete function:
>>>>
>>>> If delete is called on a locked mutex, the function must return an
>>>> error code to signal the error to the function's caller.
>>>
>>> That is, IMO, fundamentally busted behavior, unless it only means
>>> locked by the calling thread.
>>
>> Could you expand on /why/ you think it is busted behavior?
>> (I need ammo to try to convince the library authors.)
>>
>> The exact requirements are:
>>
>> 1) Return an error when attempting to delete "a mutex currently owned by
>> another thread".
>
> This is broken, period. Suppose thread T1 tries to delete the
> mutex and discovers (somehow) that it is locked by T2. T1 leaves
> the mutex alone, the deletion attempt returns an error, and all is
> well.
Except that one is left to wonder why T1 ever thought it would be a good
idea to delete the mutex...
> But suppose T1 comes along at a moment when the mutex is unlocked,
> and succeeds in deleting it. Two clock cycles later, T2 tries to lock
> the destroyed mutex, and the fertilizer hits the fan.
The library also requires "return an error when attempting to lock an
uninitialized mutex".
I don't know what they do in that library... Perhaps this explains why
they do not provide the source code.
> The fact that T1
> found the mutex unlocked is no proof that the mutex is "unused," or
> that T2 might not attempt to use it again.
Obviously.
> If T1 deletes a mutex that T2 might still want to use, the
> application is broken. If T1 can find the mutex locked by T2, it
> must be the case that T2 still wants to use it. Therefore the mere
> possibility that delete-while-locked-elsewhere could occur is proof
> that the design is broken.
>
>> 2) Return an error when attempting to delete "a mutex currently owned by
>> the calling thread".
>
> This isn't quite as clear-cut, but it still seems dubious.
>
>> [...]
>> Trying to delete a locked mutex sounds like a serious programmer error.
>
> Yes, but the point is more general: Trying to delete an "in use"
> mutex is a serious programmer error. A mutex in the locked state is
> obviously in use, but it is a mistake to reason that a mutex in the
> unlocked state is not in use.
Are there situations, algorithms, where many mutexes are created then
deleted (in a loop). It seems to me that mutexes are typically long
lived objects, initialized at application start-up, and destroyed
shortly before termination.
>> According to the library's spec, delete may be called at any time, by
>> any thread, and it is the user's responsibility to return an error and
>> leave the mutex untouched if the mutex is currently locked...
>
> Broken. The danger isn't when deletion fails because the mutex is
> locked, but when deletion succeeds because the mutex is unlocked -- but
> is still in use.
Sigh.
Regards.
Noob
> David Schwartz wrote:
>
>> Noob wrote:
>>
>>> But this library has specific requirements for the delete function:
>>>
>>> If delete is called on a locked mutex, the function must return an error
>>> code to signal the error to the function's caller.
>>
>> That is, IMO, fundamentally busted behavior, unless it only means
>> locked by the calling thread.
>
> I agree. I wouldn't go anywhere near that library. Find another one
> written by someone with half a brain.
I have no such freedom. I must use that proprietary library.
It is a MAFIAA requirement (DVB Conditional Access).
> That said, you could use pthread_mutex_trylock() to find out whether
> or not the mutex is locked.
Any way I slice it, their requirement seems to impose high overhead
on lock and unlock operations for no gain whatsoever.
Regards.
Dave Butenhof
> [ Cross-posted to c.p.t and c.u.p ]
>
> Hello,
>
> I'm using a proprietary library (no source code) which requires the user
> to provide 4 mutex-related functions (create, delete, lock, unlock).
>
> AFAICT, if I were using a POSIX OS, I could make trivial wrappers around
>
> pthread_mutex_init
> pthread_mutex_destroy
> pthread_mutex_lock
> pthread_mutex_unlock
>
> But this library has specific requirements for the delete function:
>
> If delete is called on a locked mutex, the function must return an error
> code to signal the error to the function's caller.
>
> However, the documentation for pthread_mutex_destroy explicitly states:
> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_mutex_init.html
>
> """
> It shall be safe to destroy an initialized mutex that is unlocked.
> Attempting to destroy a locked mutex results in undefined behavior.
> """
>
> This means that my delete function can't just call pthread_mutex_destroy
> because I would invoke UB if the mutex is locked, and I wouldn't be able
> to return the requested error code, right?
Strictly speaking, what this means is that POSIX doesn't *require* an
implementation to know whether the mutex is already locked.
That's due to concerns over the performance of tracking mutex ownership
on some platforms... as well as (as you mention below) concerns that all
implementations be penalized by extra code that only helps broken
applications. Correct (production level) code doesn't destroy a mutex to
which any other thread might hold or request a reference. And as already
mentioned in a reply, once you get to that point you're lost anyway
because there are plenty of errors you won't be able to catch.
> So... Does this mean I have to keep track whether the mutex is locked?
> (It would appear so.)
>
> Or perhaps I could mess with the OS code? (shudder)
>
> I'd like to tell the library author that it is unreasonable to impose
> overhead on every lock and unlock operation just to catch an obvious (?)
> programming error on delete. What do you think?
>
> N.B. the library requires mutex with recursive semantic.
This requirement probably simplifies things. You really want a recursive
POSIX mutex (mutex type PTHREAD_MUTEX_RECURSIVE), and you have to track
mutex ownership to do that. Regardless of formal requirements, it's
highly unlikely a platform implementing recursive mutexes wouldn't
report EBUSY on an attempt to unlock a locked mutex.
But, again, your requirement is beyond the scope of the API standard and
into implementation defined territory. You need to check the
documentation for the platform(s) you care about to see what they'll
guarantee, or "roll your own".
If you're interested in a particular platform, this should be simple. If
you're interested in general portability, you'll pretty much need to
assume the answer is "roll your own" or "declare defeat and give up".
Eric Sosman
> Eric Sosman wrote:
>
>> Noob wrote:
>>
>>> David Schwartz wrote:
>>>
>>>> Noob wrote:
>>>>
>>>>> But this library has specific requirements for the delete function:
>>>>>
>>>>> If delete is called on a locked mutex, the function must return an
>>>>> error code to signal the error to the function's caller.
>>>> That is, IMO, fundamentally busted behavior, unless it only means
>>>> locked by the calling thread.
>>> Could you expand on /why/ you think it is busted behavior?
>>> (I need ammo to try to convince the library authors.)
>>>
>>> The exact requirements are:
>>>
>>> 1) Return an error when attempting to delete "a mutex currently owned by
>>> another thread".
>> This is broken, period. Suppose thread T1 tries to delete the
>> mutex and discovers (somehow) that it is locked by T2. T1 leaves
>> the mutex alone, the deletion attempt returns an error, and all is
>> well.
>
> Except that one is left to wonder why T1 ever thought it would be a good
> idea to delete the mutex...
Right: This can only happen if there's something fundamentally
wrong with the T1/T2 program itself. The program is still using
something that it thinks it doesn't need; put differently, the
program believes that `X & !X' is true. A library that tries to
assist the program in such beliefs is at best quixotic.
>> But suppose T1 comes along at a moment when the mutex is unlocked,
>> and succeeds in deleting it. Two clock cycles later, T2 tries to lock
>> the destroyed mutex, and the fertilizer hits the fan.
>
> The library also requires "return an error when attempting to lock an
> uninitialized mutex".
Does it also require an error for attempting to lock a chunk of
memory containing random garbage that just sort of happens to look
like a mutex? What other miracles does this library demand?
No, look, seriously: malloc some memory, initialize a mutex M1 in
it, and let T1 and T2 use it for a while. Now T1 decides (erroneously)
that M1 is no longer needed, so it deletes it and frees the memory.
Now call malloc again, get the same chunk of memory back, and create a
brand-new mutex M2 at the same address where the now-defunct M1 used
to live. When T2 tries to lock M1, supplying an address that points to
the perfectly valid M2, you're supposed to be able to detect it?
Lotsa luck ...
> Are there situations, algorithms, where many mutexes are created then
> deleted (in a loop). It seems to me that mutexes are typically long
> lived objects, initialized at application start-up, and destroyed
> shortly before termination.
In an object-oriented design you might have a mutex inside each
instance of some class or other, protecting some of the instance's
own internals. If you create and destroy lots of instances of the
class, you'll create and destroy a lot of mutexes. (Mutices?)
This is pretty common in classes that represent containers or
collections or things of that sort, that tend to be fairly long-lived.
But even a short-lived object might need a mutex if it needs to preserve
an invariant involving two or more fields, or a field that can't be
updated atomically. You might have a Color class, for example, that
allows you to get/set any of RGB, HSV, CMYK, ... and needs to keep
all the representations in agreement; such a class might well have a
private mutex to serialize the getters and setters.
The good news on that sort of mutex, I guess, is that it's often
"internal" to the class, not accessible to J. Random Outsider. It
lives and dies with the class instance, so whenever it's safe to
delete the object it's also safe to delete the object's private mutex.
Scott Lurndal
My first thought too, use _trylock in the delete function. However, that
won't work for recursive mutexes, because it won't fail if the current thread
has the mutex.
scott
Måns Rullgård
Then don't use recursive mutexes. Using recursing mutexes is also a
strong indicator of design flaws.
Scott Lurndal
Talk to the vendor of the proprietary library used by the OP. They're
the ones that require recursive mutexes - I don't believe they should be
used either.
scott
Chris M. Thomasson
Man! This sucks!! Anyway, I believe you could meet all the requirements,
however, it's not going to be pretty. Here is what I hacked together for you
so far:
http://cpt.pastebin.com/f540aaef5
This is crowbar proof except for the case when a thread unlocks a mutex that
it did not previously locked. This can be handled, but it will add even more
overhead. All other cases are handled gracefully. Please take a look at the
sample application. It should compile for you. I create N threads which sit
in a loop and basically create, lock, unlock and destroy a global mutex.
Conflicts are resolved by using a reference count and external hash lock
table.
Please study the code and tell me if it works for that nasty library you
have to work with.
David Schwartz
> >> If delete is called on a locked mutex, the function must return an error
> >> code to signal the error to the function's caller.
> > That is, IMO, fundamentally busted behavior, unless it only means
> > locked by the calling thread.
> Could you expand on /why/ you think it is busted behavior?
> (I need ammo to try to convince the library authors.)
Either they rely on this behavior or they don't. If they don't rely on
it, why do they make you implement it? If they do rely on it, what
happens when one thread destroys a mutex a split-second before another
thread acquires it?
The only way to make that sensible is to insist you detect an attempt
to lock a destroyed mutex. However, think about what this means. It
means you can never, ever reuse a mutex's identifier. It means a mutex
cannot be a pointer to a structure that is freed when that mutex
ceases to exist.
Basically, it means the library plans to use a mutex among threads
that cannot even agree on the lifespan of the mutex, and that's
nonsensical because, among other reasons, they have to agree on the
lifespan of what the data protects anyway.
> Since a locked mutex is so either by the calling thread, or by another
> thread (duh!) it seems the two requirements boil down to:
>
> Return an error (and, I assume, leave the mutex alone) when attempting
> to delete a locked mutex.
Even if it's locked by the calling thread?
> > Since the mutex is recursive, one of two things are the case:
> > 1) The requirement only applies if the calling thread holds the lock
> > on the mutex, in which case you have to keep track of the lock count
> > anyway. (Otherwise, how would you know what to do in the unlock
> > operation?) The library is being reasonable, though quirky.
> The OS I'm using provides recursive mutex. Therefore, I don't think I
> have to deal with lock counts, as the OS takes care of that.
It's not his fault you're using mutexes that don't provide his
semantics when his semantics are reasonable. If he had a requirement
that was specific to a mutex held by the calling thread for a
recursive mutex, the semantics would be reasonable (ignoring the
silliness of recursive mutexes to begin with) though odd. Your best
bet would be to use non-recusive mutexes (if available, otherwise, use
recursive mutexes but don't use them recusively) and implement the
recursion yourself.
It's not unusual to need different recursion semantics and have to
implement your own recursion even when the platform supplies recursive
mutexes. For example, someone might insist on reader/writer mutexes
where the write lock was recursive. That's not totally unreasonable,
but likely your platform doesn't provide them, so you would have to do
code it yours.
> Trying to delete a locked mutex sounds like a serious programmer error.
If it's locked by another thread, it must be. If it's locked by the
calling thread, it could be defined either way by the mutex semantics.
Some typical mutexes either say it will succeed and delete the mutex
(sort of unlocking it), some say it is undefined. Requiring specific
behavior in this case is a somewhat bad thing because it means custom
code will be needed in an operation that is best handled by the
platform's optimized code.
As you see, asking for special semantics when another thread holds the
mutex is fundamentally busted.
> Mutex destruction (and creation) should be done in well-defined parts of
> the application, when one can be sure the mutex is not in use anymore,
> right?
Well, it depends on the way you use mutexes. You can have mutexes
associated with objects dynamically created and destroyed all over the
place. But this will not work out sanely unless an "outer
synchronization" protects the public visibility of objects.
Create object with mutex, acquire global mutex, make object visible to
other threads. Acquire global mutex, find object, lock object, remove
object from collection, unlock global mutex, unlock object, delete
object. That kind of thing.
> > 2) The requirement applies even if another thread holds the mutex.
> > This is a real pain to implement, and the requirement is fundamentally
> > busted.
> According to the library's spec, delete may be called at any time, by
> any thread, and it is the user's responsibility to return an error and
> leave the mutex untouched if the mutex is currently locked...
This will require giving each mutex a 64-bit identifier that is looked
up in a global hash table protected by a global mutex. Mutex
identifiers can never be reused. YUCK!
DS
Eric Sosman
> [...]
> I have no such freedom. I must use that proprietary library.
> It is a MAFIAA requirement (DVB Conditional Access).
The only MAFIAA references I find are to the merger of
the Motion Picture Association of America (MPAA) with the
Recording Industry Association of America (RIAA) to form the
Music And Film Industry Association of America (MAFIAA). The
merger was announced on April First, 2006. The date and the
acronym suggest some amount of spoofery ...
And DVB must be either the Democratic Voice of Burma or
something to do with Victoria Beckham, aka Posh Spice.
--
Eric Sosman
eso...@ieee-dot-org.invalid
Noob
> Noob wrote:
>
>> I have no such freedom. I must use that proprietary library.
>> It is a MAFIAA requirement (DVB Conditional Access).
>
> The only MAFIAA references I find are to the merger of
> the Motion Picture Association of America (MPAA) with the
> Recording Industry Association of America (RIAA) to form the
> Music And Film Industry Association of America (MAFIAA).
Right. http://en.wiktionary.org/wiki/MAFIAA
> The merger was announced on April First, 2006. The date and
> the acronym suggest some amount of spoofery ...
One might call it a caricature ;-)
> And DVB must be either the Democratic Voice of Burma or
> something to do with Victoria Beckham, aka Posh Spice.
Sorry for the acronym soup.
http://en.wikipedia.org/wiki/Digital_Video_Broadcasting
http://en.wikipedia.org/wiki/Conditional_access
Noob
> Man! This sucks!! Anyway, I believe you could meet all the requirements,
> however, it's not going to be pretty. Here is what I hacked together for
> you so far:
>
> http://cpt.pastebin.com/f540aaef5
Thanks Chris, I'll take a hard look at it.
Please note that, although I was discussing this topic in light of a
POSIX OS, the proprietary OS I'm using (ST OS21) does not conform to
POSIX, AFAIK.
Rainer Weikusat
[...]
> This is crowbar proof except for the case when a thread unlocks a mutex that
> it did not previously locked.
The behaviour of such an operation is either undefined ('normal' and
'default' mutexes) or it is an error ('errorcheck' and 'rescursive'
mutexes).
Måns Rullgård
I truly feel sorry you. The sensible thing to do in your situation is
to walk away and find something proper to work on.
Noob
> Noob wrote:
>
>> Please note that, although I was discussing this topic in light of a
>> POSIX OS, the proprietary OS I'm using (ST OS21) does not conform to
>> POSIX, AFAIK.
>
> I truly feel sorry you.
Because I have to work with ST's platform? :-)
You have some experience with their platforms, don't you?
> The sensible thing to do in your situation is
> to walk away and find something proper to work on.
You think I should change jobs? :-)
Would you believe me if I told you ST refuses to provide any support
whatsoever because they consider we are too low volume?
Sigh^2
Rainer Weikusat
> On Oct 28, 8:04�am, Noob <r...@127.0.0.1> wrote:
>> >> If delete is called on a locked mutex, the function must return an error
>> >> code to signal the error to the function's caller.
>
>> > That is, IMO, fundamentally busted behavior, unless it only means
>> > locked by the calling thread.
>
>> Could you expand on /why/ you think it is busted behavior?
>> (I need ammo to try to convince the library authors.)
[...]
> Basically, it means the library plans to use a mutex among threads
> that cannot even agree on the lifespan of the mutex, and that's
> nonsensical because, among other reasons, they have to agree on the
> lifespan of what the data protects anyway.
A more probable conjecture would be that the library uses the
delete-operation to detect if a mutex has meanwhile been unlocked
without blocking on it and another conjecture based on that and a
short search of the as-always-inaccessible relevant documentation[*]
would be that (at least some versions of) Windows behaves in this way.
[*] In this particular case, determining if this might be so
had required to manually read through the descriptons of no
less than 16,000 defined 'system error codes' which, as far as
I could determine, are not explicitly associated with the
functions which might return them in any way.
Måns Rullgård
> M�ns Rullg�rd wrote:
>
>> Noob wrote:
>>
>>> Please note that, although I was discussing this topic in light of a
>>> POSIX OS, the proprietary OS I'm using (ST OS21) does not conform to
>>> POSIX, AFAIK.
>>
>> I truly feel sorry you.
>
> Because I have to work with ST's platform? :-)
>
> You have some experience with their platforms, don't you?
As little as I could get away with, but yes, I've used them.
>> The sensible thing to do in your situation is
>> to walk away and find something proper to work on.
>
> You think I should change jobs? :-)
>
> Would you believe me if I told you ST refuses to provide any support
> whatsoever because they consider we are too low volume?
I have no problem believing that.
Chris M. Thomasson
> Chris M. Thomasson wrote:
>
>> Man! This sucks!! Anyway, I believe you could meet all the requirements,
>> however, it's not going to be pretty. Here is what I hacked together for
>> you so far:
>>
>> http://cpt.pastebin.com/f540aaef5
>
Here is updated code which handles the case in which a thread tries to
unlock a mutex that is locked by another thread:
http://cpt.pastebin.com/f3044e915
It also fixes a race-condition I overlooked. The unlocking needs to be done
under the protection of the hash lock. The previous code did not do this,
which would allow a thread A to destroy the mutex just before a thread B
unlocked it.
Sorry about that non-sense!
Anyway, I cannot seem to find any more issues.
> Please note that, although I was discussing this topic in light of a
> POSIX OS, the proprietary OS I'm using (ST OS21) does not conform to
> POSIX, AFAIK.
I am not sure as I am not really familiar with at operating system.
Chris M. Thomasson
http://groups.google.com/group/comp.programming.threads/msg/40f8d124cd038419
GRRR!!! I need coffee!
;^(....
Chris M. Thomasson
news:87eiome...@fever.mssgmbh.com...
Okay, I forgot about that because I very rarely use recursive mutexs; thank
you! Here is updated code which takes this into account:
Chris M. Thomasson
"Noob" <ro...@127.0.0.1> wrote in message news:hcbsu6$ocf$1...@aioe.org...
> Chris M. Thomasson wrote:
>
>> Man! This sucks!! Anyway, I believe you could meet all the requirements,
>> however, it's not going to be pretty. Here is what I hacked together for
>> you so far:
>>
>> http://cpt.pastebin.com/f540aaef5
>
> Thanks Chris, I'll take a hard look at it.
Here is updated code which handles the case in which a thread tries to
unlock a mutex that is locked by another thread:
http://cpt.pastebin.com/f3044e915
It also fixes a race-condition I overlooked. The unlocking needs to be done
under the protection of the hash lock. The previous code did not do this,
which would allow a thread A to destroy the mutex just before a thread B
unlocked it.
Sorry about that non-sense!
Anyway, I cannot seem to find any more issues.
> Please note that, although I was discussing this topic in light of a
> POSIX OS, the proprietary OS I'm using (ST OS21) does not conform to
> POSIX, AFAIK.
I am not sure as I am not really familiar with at operating system.
Chris M. Thomasson
news:hccn64$fi5$2...@news.eternal-september.org...
>
> "Noob" <ro...@127.0.0.1> wrote in message news:hcbsu6$ocf$1...@aioe.org...
>> Chris M. Thomasson wrote:
>>
>>> Man! This sucks!! Anyway, I believe you could meet all the requirements,
>>> however, it's not going to be pretty. Here is what I hacked together for
>>> you so far:
>>>
>>> http://cpt.pastebin.com/f540aaef5
>>
>> Thanks Chris, I'll take a hard look at it.
>
> Here is updated code which handles the case in which a thread tries to
> unlock a mutex that is locked by another thread:
>
>
> http://cpt.pastebin.com/f3044e915
>
>
> It also fixes a race-condition I overlooked. The unlocking needs to be
> done under the protection of the hash lock. The previous code did not do
> this, which would allow a thread A to destroy the mutex just before a
> thread B unlocked it.
>
> Sorry about that non-sense!
>
>
> Anyway, I cannot seem to find any more issues.
I have implemented the algorithm in Relacy Race Detector:
http://relacy.pastebin.com/f227e4245
Relacy likes it! So, this scheme works. BTW, what does the interface look
like for the ST OS21 operating system? If it's compatible, then you have a
complete solution which will conform to the libraries rules.
David Schwartz
> A more probable conjecture would be that the library uses the
> delete-operation to detect if a mutex has meanwhile been unlocked
> without blocking on it and another conjecture based on that and a
> short search of the as-always-inaccessible relevant documentation[*]
> would be that (at least some versions of) Windows behaves in this way.
I don't see how that could make sense. What if the mutex has meanwhile
been unlocked but another thread is about to lock it? How do you
guarantee that this other thread doesn't lock some other mutex?
(Unless you add all the other crufty requirements, like that a mutex
handle cannot be a pointer and that they cannot be re-used.)
DS
Rainer Weikusat
> On Oct 29, 5:42�am, Rainer Weikusat <rweiku...@mssgmbh.com> wrote:
>
>> A more probable conjecture would be that the library uses the
>> delete-operation to detect if a mutex has meanwhile been unlocked
>> without blocking on it and another conjecture based on that and a
>> short search of the as-always-inaccessible relevant documentation[*]
>> would be that (at least some versions of) Windows behaves in this way.
>
> I don't see how that could make sense. What if the mutex has meanwhile
> been unlocked but another thread is about to lock it? How do you
> guarantee that this other thread doesn't lock some other mutex?
Because there are is one thread with a mutex or maybe n threads with
mutexes and one 'master thread' 'watching' them (or something similar
with clearly separated 'kinds of threads'). That's a possible
arrangement where the delete-semantics would make (some) sense. Other
arrangements are conceivable but people usually write code to
accomplish something definite and not because they are out of their
senses (often both to a degree, actually -- once something has
been implemented, it is all too clear how it should have been done
instead :->, but 'it works' is a very practical quality).
Chris M. Thomasson
news:6b9a4262-aaab-4ea4...@g22g2000prf.googlegroups.com...
On Oct 28, 8:04 am, Noob <r...@127.0.0.1> wrote:
> > >> If delete is called on a locked mutex, the function must return an
> > >> error
> > >> code to signal the error to the function's caller.
> > > That is, IMO, fundamentally busted behavior, unless it only means
> > > locked by the calling thread.
> > Could you expand on /why/ you think it is busted behavior?
> > (I need ammo to try to convince the library authors.)
> Either they rely on this behavior or they don't. If they don't rely on
> it, why do they make you implement it? If they do rely on it, what
> happens when one thread destroys a mutex a split-second before another
> thread acquires it?
> The only way to make that sensible is to insist you detect an attempt
> to lock a destroyed mutex. However, think about what this means. It
> means you can never, ever reuse a mutex's identifier. It means a mutex
> cannot be a pointer to a structure that is freed when that mutex
> ceases to exist.
Indeed. The hack I posted also requires that the memory which makes up a
mutex cannot be freed. However, one can definetely use reference counting
with strong thread saftey to handle this senerio:
_________________________________________________________
struct foo
{
mutex m_lock;
// [...];
};
static global_ptr<foo> g_foo;
void threads()
{
for (;;)
{
local_ptr<foo> l_foo(g_foo);
if (! l_foo)
{
l_foo = new foo;
l_foo = g_foo.swap(l_foo);
}
l_foo->m_lock.lock();
l_foo->m_lock.unlock();
g_foo = NULL;
}
}
_________________________________________________________
`foo' objects are allowed to be dynamically created and destroyed on the
fly. Threads don't actually need to care about lifetime management because
the reference count handles all of that for them. Although, I have to say
that a library which has rules that basically demand using this type of
reference counting is pretty darn crappy.
frege
> [ Cross-posted to c.p.t and c.u.p ]
>
> Hello,
>
> I'm using a proprietary library (no source code) which requires the user
> to provide 4 mutex-related functions (create, delete, lock, unlock).
>
> AFAICT, if I were using a POSIX OS, I could make trivial wrappers around
>
> pthread_mutex_init
> pthread_mutex_destroy
> pthread_mutex_lock
> pthread_mutex_unlock
>
>
> code to signal the error to the function's caller.
>
>
> """
> It shall be safe to destroy an initialized mutex that is unlocked.
> Attempting to destroy a locked mutex results in undefined behavior.
> """
>
> This means that my delete function can't just call pthread_mutex_destroy
> because I would invoke UB if the mutex is locked, and I wouldn't be able
> to return the requested error code, right?
>
> (It would appear so.)
>
> Or perhaps I could mess with the OS code? (shudder)
>
> I'd like to tell the library author that it is unreasonable to impose
> overhead on every lock and unlock operation just to catch an obvious (?)
> programming error on delete. What do you think?
>
> N.B. the library requires mutex with recursive semantic.
>
I basically agree with everyone else - the specs are stupid. But my 2
cents anyhow:
1. just don't implement it, but say you did. ie have delete never
return an error. The error would only ever been seen in broken code
anyhow.
or
2. ref count it:
class ByTheSpecMutex
{
boost::shared_ptr<RealMutex> pmutex;
public:
error_code delete()
{
if (pmutex->count() > 1)
return in_use_error;
...
}
};
Tony
Rainer Weikusat
> On Oct 28, 9:27�am, Noob <r...@127.0.0.1> wrote:
>> [ Cross-posted to c.p.t and c.u.p ]
[...]
>> But this library has specific requirements for the delete function:
>>
>> If delete is called on a locked mutex, the function must return an error
>> code to signal the error to the function's caller.
[...]
> I basically agree with everyone else - the specs are stupid. But my 2
> cents anyhow:
>
> 1. just don't implement it, but say you did. ie have delete never
> return an error. The error would only ever been seen in broken code
> anyhow.
I consider it likely that the library was implemented by someone
familiar with (and only with) the Windows threading interfaces,
because these happen to have a couple of the properties David Schwartz
mentionend, especially, that it is impossible to declare 'mutex'
objects residing in memory accessible to all and therefore,
indiscriminately usable by all threads. A Windows mutex is a
system-managed object and can only be used by threads which happen to
have a copy of the handle. Further, there doesn't seem to be a trylock
operation and I am fairly convinced that the CloseHandle function
will detect an attempt to release a locked mutex and notify the caller
of that (I could be wrong on all the details, though), since this
would be in line with the 'child-safety precaution at any cost'-mindset
which is so common among everything sprung from Redmond. Because of
this, I assume that the library uses 'delete' to emulate 'trylock' and
if this was true, your suggestion above would certainly not be a
helpful one.
Eric Sosman
You may be right about the Windows origin of the nonsense
specification, but I still cannot understand how a delete
could be used in a trylock emulation. If you delete the mutex
and are told "No, it's locked," fine: Your pseudo-trylock can
report that the mutex was locked and hence not acquirable.[*]
But what if the deletion succeeds? You then know that the
mutex wasn't locked, but what good does it do you? You can't
now lock it, because it no longer exists. (Any any other
thread that was about to lock it -- or delete it -- is in
deep trouble, too.)
[*] Actually, not even this much is true. The O.P. said
the mutexes were supposed to be recursively lockable, so a
mutex that is locked and hence not deletable might have been
locked by the same thread, and hence acquirable despite the
deletion failure.
--
Eric Sosman
eso...@ieee-dot-org.invalid
Rainer Weikusat
> Rainer Weikusat wrote:
>> frege <gottlo...@gmail.com> writes:
>>> On Oct 28, 9:27 am, Noob <r...@127.0.0.1> wrote:
>>>> [ Cross-posted to c.p.t and c.u.p ]
>>
>> [...]
>>
>>>> But this library has specific requirements for the delete function:
>>>>
>>>> If delete is called on a locked mutex, the function must return an error
>>>> code to signal the error to the function's caller.
[...]
>> Further, there doesn't seem to be a trylock
>> operation and I am fairly convinced that the CloseHandle function
>> will detect an attempt to release a locked mutex and notify the caller
>> of that
[...]
>> I assume that the library uses 'delete' to emulate 'trylock'
[...]
> You may be right about the Windows origin of the nonsense
> specification, but I still cannot understand how a delete
> could be used in a trylock emulation. If you delete the mutex
> and are told "No, it's locked," fine: Your pseudo-trylock can
> report that the mutex was locked and hence not acquirable.[*]
> But what if the deletion succeeds? You then know that the
> mutex wasn't locked, but what good does it do you?
> You can't now lock it, because it no longer exists.
If it is now impossible to lock the mutex which no longer
exists, the intent was quite obviously not to lock the mutex, only to
determine that a particular thread has unlocked a particular mutex,
presumably, after having completed some related task.
> (Any any other thread that was about to lock it -- or delete it --
> is in deep trouble, too.)
As obviously, no such 'other thread' may exist.
> [*] Actually, not even this much is true. The O.P. said
> the mutexes were supposed to be recursively lockable, so a
> mutex that is locked and hence not deletable might have been
> locked by the same thread, and hence acquirable despite the
> deletion failure.
Something which obviously doesn't make sense might simply be
wrong. As I wrote in the first paragraph: Since 'destruction of the
mutex' is a side effect of such a probe-operation, acquiring said
mutex can't be the objective. Determining information about a
particular mutex (and consequently, the state of a particular thread)
could.
David Schwartz
> If it is now impossible to lock the mutex which no longer
> exists, the intent was quite obviously not to lock the mutex, only to
> determine that a particular thread has unlocked a particular mutex,
> presumably, after having completed some related task.
Or before having started some related task. And if you have some kind
of synchronization other than the mutex that tells you the thread has
started the task, why wouldn't that same method tell you when it had
finished that task?
My bet is someone just took whatever ratbag assortment of feature
there platform's mutexes happened to have, or that they thought their
platform had due to misunderstandings and confusions of various sorts,
and asked for that.
DS
Eric Sosman
> Eric Sosman <eso...@ieee-dot-org.invalid> writes:
>
>> Rainer Weikusat wrote:
>>> [...]
>>> I assume that the library uses 'delete' to emulate 'trylock'
>
> [...]
>
>> You may be right about the Windows origin of the nonsense
>> specification, but I still cannot understand how a delete
>> could be used in a trylock emulation. If you delete the mutex
>> and are told "No, it's locked," fine: Your pseudo-trylock can
>> report that the mutex was locked and hence not acquirable.[*]
>> But what if the deletion succeeds? You then know that the
>> mutex wasn't locked, but what good does it do you?
>> You can't now lock it, because it no longer exists.
>
> If it is now impossible to lock the mutex which no longer
> exists, the intent was quite obviously not to lock the mutex, only to
> determine that a particular thread has unlocked a particular mutex,
> presumably, after having completed some related task.
In what sense does that "emulate `trylock'?"
>> (Any any other thread that was about to lock it -- or delete it --
>> is in deep trouble, too.)
>
> As obviously, no such 'other thread' may exist.
If it is possible that the delete operation finds the
mutex locked, it follows that some thread somewhere has a
reference to the mutex. (Or "had" a reference to the mutex,
locked it, and then leaked the reference, in which case the
mutex can never be unlocked or deleted by anybody, ever.)
If a thread other than the deleter has a reference to
the mutex, that thread might use the reference to attempt a
lock or delete or other operation (if there are any in this
library). There is no way inherent in the mutex to prevent
the other thread from making use of its reference. There
are other ways (program conventions or protocols) to do so,
but if it is possible for delete to find the mutex locked,
it is clear that no such protocols are operating.
If the deleting thread is the only holder of a reference
and finds the mutex locked, either the deleter itself holds
the lock or the the thread that locked it has leaked its
reference as described above. If the deleter holds the lock
it is just barely possible that the operation might make sense,
but it can't be a trylock emulation (because it never acquires
the mutex, not even when the mutex is available).
> Something which obviously doesn't make sense might simply be
> wrong. As I wrote in the first paragraph: Since 'destruction of the
> mutex' is a side effect of such a probe-operation, acquiring said
> mutex can't be the objective.
That is, it's *not* a trylock emulation.
> Determining information about a
> particular mutex (and consequently, the state of a particular thread)
> could.
Determining what the state of affairs looked like one CPU
cycle ago might be interesting for statistical purposes, but
for nothing else. A destructive determination isn't good even
for that much.
--
Eric Sosman
eso...@ieee-dot-org.invalid
Måns Rullgård
Rainer Weikusat
> Rainer Weikusat wrote:
>> Eric Sosman <eso...@ieee-dot-org.invalid> writes:
>>
>>> Rainer Weikusat wrote:
>>>> [...]
>>>> I assume that the library uses 'delete' to emulate 'trylock'
>>
>> [...]
>>
>>> You may be right about the Windows origin of the nonsense
>>> specification, but I still cannot understand how a delete
>>> could be used in a trylock emulation. If you delete the mutex
>>> and are told "No, it's locked," fine: Your pseudo-trylock can
>>> report that the mutex was locked and hence not acquirable.[*]
>>> But what if the deletion succeeds? You then know that the
>>> mutex wasn't locked, but what good does it do you?
>>> You can't now lock it, because it no longer exists.
>>
>> If it is now impossible to lock the mutex which no longer
>> exists, the intent was quite obviously not to lock the mutex, only to
>> determine that a particular thread has unlocked a particular mutex,
>> presumably, after having completed some related task.
>
> In what sense does that "emulate `trylock'?"
In the sense that it enables to check if a mutex is presently locked
without blocking on it. Since I already wrote this in my first posting
on this topic, I thought the restricted meaning was clear enough,
especially taking into account that a delete operation cannot ever
lock a mutex.
[...]
>> Determining information about a
>> particular mutex (and consequently, the state of a particular thread)
>> could.
>
> Determining what the state of affairs looked like one CPU
> cycle ago might be interesting for statistical purposes, but
> for nothing else.
You have no information beyond what was posted about this library and
therefore, the assumption that no such 'program conventions or
protocols' will be in use seems unjustified: It is again one whose
sole purpose is to construct an example which doesn't make sense,
based on incomplete factual information.
Eric Sosman
> Eric Sosman <eso...@ieee-dot-org.invalid> writes:
>> Rainer Weikusat wrote:
>>> Eric Sosman <eso...@ieee-dot-org.invalid> writes:
>>>
>>>> Rainer Weikusat wrote:
>>>>> [...]
>>>>> I assume that the library uses 'delete' to emulate 'trylock'
>>> [...]
>>>
>>> If it is now impossible to lock the mutex which no longer
>> In what sense does that "emulate `trylock'?"
>
> In the sense that it enables to check if a mutex is presently locked
Ah! Not an emulation of trylock, then, but of tryl.
--
Eric Sosman
eso...@ieee-dot-org.invalid
Chris M. Thomasson
> frege <gottlo...@gmail.com> writes:
>> On Oct 28, 9:27 am, Noob <r...@127.0.0.1> wrote:
>>> [ Cross-posted to c.p.t and c.u.p ]
>
> [...]
>
>>> But this library has specific requirements for the delete function:
>>>
>>> If delete is called on a locked mutex, the function must return an error
>>> code to signal the error to the function's caller.
>
> [...]
>
>> I basically agree with everyone else - the specs are stupid. But my 2
>> cents anyhow:
>>
>> 1. just don't implement it, but say you did. ie have delete never
>> return an error. The error would only ever been seen in broken code
>> anyhow.
>
> I consider it likely that the library was implemented by someone
> familiar with (and only with) the Windows threading interfaces,
> because these happen to have a couple of the properties David Schwartz
> mentionend, especially, that it is impossible to declare 'mutex'
> objects residing in memory accessible to all and therefore,
> indiscriminately usable by all threads. A Windows mutex is a
> system-managed object and can only be used by threads which happen to
> have a copy of the handle. Further, there doesn't seem to be a trylock
> operation
There is a try-lock for Windows mutexs:
WaitForSingleObject(..., 0);
> and I am fairly convinced that the CloseHandle function
> will detect an attempt to release a locked mutex and notify the caller
> of that (I could be wrong on all the details, though), since this
> would be in line with the 'child-safety precaution at any cost'-mindset
> which is so common among everything sprung from Redmond.
No. Windows mutexs are robust. Therefore if a process dies while it hold's
one the next locker will receive `WAIT_ABANDONED'. This is essential for
creating robust lock-based inter-process synchronization schemes.
Chris M. Thomasson
[...]
One other point. Windows named mutexs are reference counted using a
inter-process strongly-thread safe counting algorithm. This is why any
thread can acquire a reference to a named mutex that it did not previously
own a reference to.
David Schwartz
> One other point. Windows named mutexs are reference counted using a
> inter-process strongly-thread safe counting algorithm. This is why any
> thread can acquire a reference to a named mutex that it did not previously
> own a reference to.
All named mutexes are this way. The "name" is, pretty much by
definition, an index into a synchronized global table.
DS
Rainer Weikusat
[...]
>> and I am fairly convinced that the CloseHandle function
>> will detect an attempt to release a locked mutex and notify the caller
>> of that (I could be wrong on all the details, though), since this
>> would be in line with the 'child-safety precaution at any cost'-mindset
>> which is so common among everything sprung from Redmond.
>
> No. Windows mutexs are robust. Therefore if a process dies while it hold's
> one the next locker will receive `WAIT_ABANDONED'. This is essential for
> creating robust lock-based inter-process synchronization schemes.
This doesn't seem to have any relation to the text I wrote.