Using a mutex within the data structure that I want to protect
Mark Hobley
purpose of making thread safe code.
Is it ok, to include the mutex datatype within the data structure that I wish
to protect? For example:
struct buffertable {
struct buffer *addr;
int sz;
unsigned int currentdefault;
unsigned int chunksize;
unsigned int tablechunk;
pthread_mutex_t buffertablemx; <--- Can I put the mutex here?
};
extern struct buffertable buffertable;
In the above case I want to protect the values of the buffertable using the
buffertablemx mutex.
Can I also protect dynamically created structured arrays by having a mutex
as part of the array definition?
struct buffer {
char *addr;
unsigned int sz;
unsigned int ptr;
unsigned int tail;
pthread_mutex_t buffermx; <--- Can I put the mutex here?
};
buffertable.addr = malloc(tablesize * sizeof(struct buffer));
^
|
I plan to have a mutex for dynamically allocated structures. This will enable
me to apply a mutex to just one particular element of the array without
affecting other elements. In this case, it is convenient to be able to bundle
the mutex record in with the data structure, because sometimes the tablesize
will be increased via a realloc call, and this would mean that new mutexes
are required to work with the new elements.
I am using C89, if that matters.
--
Mark Hobley
Linux User: #370818 http://markhobley.yi.org/
Rainer Weikusat
> I have some data structures that I want to protect using a mutex for the
> purpose of making thread safe code.
>
> Is it ok, to include the mutex datatype within the data structure that I wish
> to protect? For example:
>
> struct buffertable {
> struct buffer *addr;
> int sz;
> unsigned int currentdefault;
> unsigned int chunksize;
> unsigned int tablechunk;
> pthread_mutex_t buffertablemx; <--- Can I put the mutex here?
> };
Eh ... "why not"?
Eric Sosman
> I have some data structures that I want to protect using a mutex for the
> purpose of making thread safe code.
>
> Is it ok, to include the mutex datatype within the data structure that I wish
> to protect? For example:
>
> struct buffertable {
> struct buffer *addr;
> int sz;
> unsigned int currentdefault;
> unsigned int chunksize;
> unsigned int tablechunk;
> pthread_mutex_t buffertablemx;<--- Can I put the mutex here?
> };
>
> extern struct buffertable buffertable;
>
> In the above case I want to protect the values of the buffertable using the
> buffertablemx mutex.
>
> Can I also protect dynamically created structured arrays by having a mutex
> as part of the array definition?
>
> struct buffer {
> char *addr;
> unsigned int sz;
> unsigned int ptr;
> unsigned int tail;
> pthread_mutex_t buffermx;<--- Can I put the mutex here?
> };
Sure. Good place for it, I'd say.
> buffertable.addr = malloc(tablesize * sizeof(struct buffer));
> ^
> |
> I plan to have a mutex for dynamically allocated structures. This will enable
> me to apply a mutex to just one particular element of the array without
> affecting other elements. In this case, it is convenient to be able to bundle
> the mutex record in with the data structure, because sometimes the tablesize
> will be increased via a realloc call, and this would mean that new mutexes
> are required to work with the new elements.
No, don't do that. If realloc() relocates the memory area
by copying its contents, you'll wind up with copies of the original
mutexes. Copying a mutex doesn't give you a usable mutex, but
a piece of useless junk. Also, any thread that now tries to lock
or unlock a mutex in the freed area is in deep trouble ...
To make it work, you would need to do something awful like
- make sure all the mutexes in the region are unlocked, and
that no other thread will try to lock any of them,
- pthread_mutex_destroy() all the mutexes,
- realloc(),
- pthread_mutex_init() all the mutexes in the region (and
any new ones, too),
- allow the rest of the program to start using the newly-
initialized mutexes.
So: If you must move your data structures around in memory,
don't put mutexes (or semaphores, or condition variables) in them
even though that would ordinarily be a good spot. Personally, I'd
take a second look at why you want to move things around; maybe
it's not really necessary.
--
Eric Sosman
eso...@ieee-dot-org.invalid
Jens Thoms Toerring
> I have some data structures that I want to protect using a mutex for the
> purpose of making thread safe code.
> Is it ok, to include the mutex datatype within the data structure that I wish
> to protect? For example:
> struct buffertable {
> struct buffer *addr;
> int sz;
> unsigned int currentdefault;
> unsigned int chunksize;
> unsigned int tablechunk;
> pthread_mutex_t buffertablemx; <--- Can I put the mutex here?
> };
That's completely legal. 'pthread_mutex_t' is just another type,
just with a fancy name. You can put basically everything into a
structure that the compiler can figure out how much space (and
what alignment) it needs.
> extern struct buffertable buffertable;
> In the above case I want to protect the values of the buffertable using the
> buffertablemx mutex.
> Can I also protect dynamically created structured arrays by having a mutex
> as part of the array definition?
> struct buffer {
> char *addr;
> unsigned int sz;
> unsigned int ptr;
> unsigned int tail;
> pthread_mutex_t buffermx; <--- Can I put the mutex here?
> };
> buffertable.addr = malloc(tablesize * sizeof(struct buffer));
> ^
> |
> I plan to have a mutex for dynamically allocated structures. This
> will enable me to apply a mutex to just one particular element of
> the array without affecting other elements. In this case, it is
> convenient to be able to bundle the mutex record in with the data
> structure, because sometimes the tablesize will be increased via a
> realloc call, and this would mean that new mutexes are required to
> work with the new elements.
Also that is legal - if you can put a 'pthread_mutex_t' in a
structure you can put it into an array of structures, be it
allocated statically or dynamically.
What I don't like is your lying to the compiler about the type
the 'addr' member is obviously going to point to. What keeps
you from using the real type, i.e 'struct buffer *', as the
type of the 'addr' member? Using a 'char *' (or 'void *' which
would look a bit more natural since at least shows it's not
about a char array but some opaque data type) only will force
you to use ugly casts all over the place. It's completely ok
to write
struct buffer {
struct buffer *addr;
unsigned int sz;
unsigned int ptr;
unsigned int tail;
pthread_mutex_t buffermx; <--- Can I put the mutex here?
};
The compiler sees that enough room for a pointer to astructure
is required, the exact layout of the structure pointed to is
not needed at this point. And then you also could write
buffertable.addr = malloc(tablesize * sizeof *buffertable.addr);
which avoids having to rewrite anything here if you decide to
change the name of 'struct buffer' sometime later.
> I am using C89, if that matters.
Not really.
Regards, Jens
--
\ Jens Thoms Toerring ___ j...@toerring.de
\__________________________ http://toerring.de
Jens Thoms Toerring
> In comp.unix.programmer Mark Hobley <markh...@hotpop.donottypethisbit.com> wrote:
> > Can I also protect dynamically created structured arrays by having a mutex
> > as part of the array definition?
> > struct buffer {
> > char *addr;
> > unsigned int sz;
> > unsigned int ptr;
> > unsigned int tail;
> > pthread_mutex_t buffermx; <--- Can I put the mutex here?
> > };
> > buffertable.addr = malloc(tablesize * sizeof(struct buffer));
> > ^
> > |
> > I plan to have a mutex for dynamically allocated structures. This
> > will enable me to apply a mutex to just one particular element of
> > the array without affecting other elements. In this case, it is
> > convenient to be able to bundle the mutex record in with the data
> > structure, because sometimes the tablesize will be increased via a
> > realloc call, and this would mean that new mutexes are required to
> > work with the new elements.
> Also that is legal - if you can put a 'pthread_mutex_t' in a
> structure you can put it into an array of structures, be it
> allocated statically or dynamically.
Please don't listen to me but to Eric Sosman, I missed the bit
with the reallocation and only considered a dynamically allo-
cated array of such structures!
Rick Jones
> > char *addr;
> > unsigned int sz;
> > unsigned int ptr;
> > unsigned int tail;
> > pthread_mutex_t buffermx;<--- Can I put the mutex here?
> > };
> Sure. Good place for it, I'd say.
Depends - how many struct buffers will there be and how many will be
"active" at one time? It could make more sense to hash the
structure's address into a table of mutexes - more than one struct
buffer would be "protected" by a given mutex, but if there are lots
and lots and lots of struct buffers it might save a great deal of
memory.
Personally, if embedding I'd be inclined to put the mutex at the
beginning of the structure, on the premis that it has the most
stringent alignment requirements and I like to avoid "holes" in data
structures.
rick jones
--
The computing industry isn't as much a game of "Follow The Leader" as
it is one of "Ring Around the Rosy" or perhaps "Duck Duck Goose."
- Rick Jones
these opinions are mine, all mine; HP might not want them anyway... :)
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...
Rick Jones
> To make it work, you would need to do something awful like
> - make sure all the mutexes in the region are unlocked, and
> that no other thread will try to lock any of them,
That implies a higher-level lock yes? Perhaps a global read-write
lock where normally a read lock is taken, but when one wants to mess
with the structures one grabs the read/write as a write.
BTW, for the OP - something similar would be required if one used a
hash table of mutexes - in that case though you would walk through the
hash table locking all the mutexes, then you can realloc the
structure(s) protected by those mutexes, then you can unlock all the
mutexes and let the other threads do their thing. Or you could nest
under a read/write lock - but it means having a read/write lock's
pathlength added to each access.
rick jones
--
Wisdom Teeth are impacted, people are affected by the effects of events.
Barry Margolin
What makes you think that buffer::addr points to a struct buffer and not
a character string? This is an array element, why do you think it's an
element of a linked list?
--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***
Mark Hobley
> So: If you must move your data structures around in memory,
> don't put mutexes (or semaphores, or condition variables) in them
> even though that would ordinarily be a good spot. Personally, I'd
> take a second look at why you want to move things around; maybe
> it's not really necessary.
I am writing a library to handle dynamic buffers. The initializer provides a
dynamic table that can track a default number of buffers. In most cases this
will be sufficient. However, in some cases, it may be that an application
requests more buffers than the table will hold. In this case, the library will
resize the table (using realloc) and add an additional number of entries to
satisfy the demand for extra buffers.
http://markhobley.yi.org/mlibrary/libbuffers/index.html
Mark.
Jens Thoms Toerring
> In article <82pg9e...@mid.uni-berlin.de>,
> j...@toerring.de (Jens Thoms Toerring) wrote:
> > In comp.unix.programmer Mark Hobley <markh...@hotpop.donottypethisbit.com>
> > wrote:
> > > buffertable.addr = malloc(tablesize * sizeof(struct buffer));
> >
> > What I don't like is your lying to the compiler about the type
> > the 'addr' member is obviously going to point to. What keeps
> > you from using the real type, i.e 'struct buffer *', as the
> > type of the 'addr' member? Using a 'char *' (or 'void *' which
> > would look a bit more natural since at least shows it's not
> > about a char array but some opaque data type) only will force
> > you to use ugly casts all over the place. It's completely ok
> > to write
> What makes you think that buffer::addr points to a struct buffer and not
> a character string? This is an array element, why do you think it's an
> element of a linked list?
My impression isn't that 'addr' is meant as part of a linked list
but as a pointer to an array of structures - the way the alloca-
tion is done would seem to me to be very strange if really meant
for an array of chars as the type of 'addr' (char *) is expres-
sing. What would one do with a char array of a length that's a
multiple of the length of a structure, given that the amount of
memory allocated can depend on the compiler and the system being
used? But, of course, I could be wrong and there's some hiddens
sense behind this I am not discerning.
Casper H.S. Dik
"struct buffer" and the "char *addr" should be allocated in two
different bits of memory; "struct buffer" shouldn't be reallocated but
reallocating *addr should work as long as it doesn't include more
mutexes.
Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.
Mark Hobley
> My impression isn't that 'addr' is meant as part of a linked list
> but as a pointer to an array of structures - the way the alloca-
> tion is done would seem to me to be very strange if really meant
> for an array of chars as the type of 'addr' (char *) is expres-
> sing. What would one do with a char array of a length that's a
> multiple of the length of a structure, given that the amount of
> memory allocated can depend on the compiler and the system being
> used?
Right. They are separate entities. The buffertable is the master record. This
tells us where the buffer information table (of type buffer) is allocated in
memory. The buffer information table can be dynamically expanded. The buffer
information table contains records of dynamically allocated string
manipulation buffers (of type char). The string manipulation buffers can be
resized dynamically using chunks. The buffer information table can be resized
to hold additional records in the event that more buffers are requested than
the table can currently hold.
Master Record ---> Buffer Information Table
(buffertable) (buffer)
Entry 1 ---> String manipulation buffer
(char[])
Entry 2 ---> String manipulation buffer
...
Entry N ---> String manipulation buffer
http://markhobley.yi.org/mlibrary/libbuffers/index.html
Mark.
Eric Sosman
> In comp.unix.programmer Eric Sosman<eso...@ieee-dot-org.invalid> wrote:
>> To make it work, you would need to do something awful like
>
>> - make sure all the mutexes in the region are unlocked, and
>> that no other thread will try to lock any of them,
>
> That implies a higher-level lock yes? Perhaps a global read-write
> lock where normally a read lock is taken, but when one wants to mess
> with the structures one grabs the read/write as a write.
Something along those lines, yes. The higher-level lock
protects the array that holds the individual mutexes. Somebody
who wants to lock one of those mutexes does
pthread_rwlock_rdlock(&global_lock);
pthread_mutex_lock(&array[n].mutex);
...
pthread_mutex_unlock(&array[n].mutex);
pthread_rwlock_unlock(&global_lock);
(order is important). To reallocate the whole array one does
pthread_rwlock_wrlock(&global_lock);
for (i = 0; i < count; ++i)
pthread_mutex_destroy(&array[i].mutex);
count = new_count;
array = realloc(array, count * sizeof *array);
for (i = 0; i < count; ++i)
pthread_mutex_init(&array[i].mutex, &attrs);
pthread_rwlock_unlock(&global_lock);
(just a sketch; error-checking omitted for brevity). Seems to
me the phrase "something awful" is apt for this sort of thing ...
Maybe there's a slicker way to do it with semaphores? I'm
not very good at semaphores.
> BTW, for the OP - something similar would be required if one used a
> hash table of mutexes - in that case though you would walk through the
> hash table locking all the mutexes, then you can realloc the
> structure(s) protected by those mutexes, then you can unlock all the
> mutexes and let the other threads do their thing. [...]
Right. In this case, the mutexes themselves never move,
just the things they protect.
--
Eric Sosman
eso...@ieee-dot-org.invalid
Mark Hobley
> That implies a higher-level lock yes? Perhaps a global read-write
> lock where normally a read lock is taken, but when one wants to mess
> with the structures one grabs the read/write as a write.
The buffertable lock is the higher level lock. That means that the
buffertable is being manipulated.
I haven't actually implement locks on the buffers themselves, but I am thinking
that I could add support for multiple processes writing to or reading from a
single buffer (packet fashion). I don't need this yet, but I thought this might
be required in future.
If need be, I could just apply a single mutex for all buffers. This would
simplify the code, but this would mean that threads would have to wait, even
if the buffer that they are using is not being used elsewhere, although I
uess that the delays would only be minimal.
Another option could be to use linked lists for the mutexes, but I think the
overhead for this too great for the benefits that it provides. In all
probability there will probably only be one thread using the buffer library at
any given time.
Is there a way of applying a shared lock to a mutex? I don't necessarily need
an exclusive lock for reading processes. Reading processes could apply
a shared lock, preventing a write process from obtaining an exclusive lock
and updating the buffer until the reading process has finished.
Mark.
> Or you could nest
> under a read/write lock - but it means having a read/write lock's
> pathlength added to each access.
I don't know what that means. If it means that I am manipulating pathnames,
then the buffer library is required to be implemented before any string
manipulation code, because the string manipulation code will make use of the
libbuffers library in order to obtain a workspace to manipulate the strings.
Mark.
Rainer Weikusat
> In comp.unix.programmer Rick Jones <rick....@hp.com> wrote:
[...]
> Is there a way of applying a shared lock to a mutex? I don't necessarily need
> an exclusive lock for reading processes. Reading processes could apply
> a shared lock, preventing a write process from obtaining an exclusive lock
> and updating the buffer until the reading process has finished.
Yes. This is a called an rwlock, the type begin pthread_rwlock_t and
using the ptrhead_rwlock_* functions.
>> Or you could nest
>> under a read/write lock - but it means having a read/write lock's
>> pathlength added to each access.
>
> I don't know what that means.
It means that the code path (sequence of sucessive IP/PC values)
necessary to perform an access would include the rwlock code.
Barry Margolin
I think you're confusing buffer::addr with buffertable::addr. The
latter *is* declared as struct buffer*, and that's what he's assigning
in the above statement.
Moi
> In article <82qj54...@mid.uni-berlin.de>,
> j...@toerring.de (Jens Thoms Toerring) wrote:
>
>> In comp.unix.programmer Barry Margolin <bar...@alum.mit.edu> wrote:
>> > In article <82pg9e...@mid.uni-berlin.de>,
>> > j...@toerring.de (Jens Thoms Toerring) wrote:
>>
>> > > In comp.unix.programmer Mark Hobley
>> > > <markh...@hotpop.donottypethisbit.com> wrote:
>> > > > buffertable.addr = malloc(tablesize * sizeof(struct buffer));
>
> I think you're confusing buffer::addr with buffertable::addr. The
> latter *is* declared as struct buffer*, and that's what he's assigning
> in the above statement.
IMHO the OP could avoid confusing himself by choosing different names
for the two elements, e.g. "buff" and "data" (maybe with a 'p' somewhere ;-)
HTH,
AvK
Mark Hobley
> IMHO the OP could avoid confusing himself by choosing different names
> for the two elements, e.g. "buff" and "data" (maybe with a 'p' somewhere ;-)
I am open to suggestions.
Currently "buffertable" is the master record that points to the buffer
information table. The buffer information table contains records of type
"buffer", which point to the dynamically allocated buffers.
I am thinking of renaming buffertable to buffermaster to indicate that it is
a master record. I didn't do this initially because I thought that it
could potentially cause confusion with master/slave naming conventions.
I am interesting in hearing what conventions other people would have used.
Mark.
Moi
On second thought, the master->buff thing points to an array of buffs,
so I would either call it .array, or .buffs.
For the mutex-field, I would use the same name, but probably shorter,
maybe .mutex, or .lock or something.
Also, to avoid realloc copying your mutexes, you could have the
buffmaster maintain an array of pointers to the actual buffs, instead of
a pointer to an array of buffs.
so it would become:
struct buffer {
mutex_t mutex;
unsigned size;
unsigned head, tail;
char *data;
};
struct buffermaster {
mutex_t mutex;
unsigned count;
unsigned used;
...
struct buffer **ptrs;
};
But, of course it is all a matter of personal taste...
HTH,
AvK