pthread_once_t in dynamically allocated (and freed) memory?

[da...@boostpro.com]

Hi,

I want to dynamically allocate memory, then once, on demand, thread-safely initialize part of that memory, and potentially free the memory. Then later I want to repeat the process with what is potentially the same memory block (and thus the same address of pthread_once_t) that has been returned from the allocator. Does this

a. "just work" as long as it is set up with PTHREAD_ONCE_INIT, or
b. do I need to do insert some kind of fence to ensure that all threads see the re-initialized memory, or
c. never work; don't do that!
d. something else I haven't thought of?

I ask because all the examples I can find store the pthread_once_t in static storage and never re-initialize it.

Thanks in advance,
Dave

[Kaz Kylheku]

On 2018-02-25, da...@boostpro.com <da...@boostpro.com> wrote:
> Hi,
>
> I want to dynamically allocate memory, then once, on demand, thread-safely initialize part of that memory, and potentially free the memory. Then later I want to repeat the process with what is potentially the same memory block (and thus the same address of pthread_once_t) that has been returned from the allocator. Does this
>
> a. "just work" as long as it is set up with PTHREAD_ONCE_INIT, or
> b. do I need to do insert some kind of fence to ensure that all threads see the re-initialized memory, or
> c. never work; don't do that!
> d. something else I haven't thought of?

It must work.

Firstly, POSIX says that PTHREAD_ONCE_INIT is a constant.
Thus it may be used in assignment. This is important because, by
contrast, the PTHREAD_MUTEX_INITIALIZER is described as a macro
that can be used for initialization, and not a constant.

Secondly, there is a storage restriction on the phread_once_t
control variable: namely, it may not be defined in automatic storage
(i.e. non-static local variable). There is no restriction against
pthread_once_t being in dynamic storage.

> I ask because all the examples I can find store the pthread_once_t in
> static storage and never re-initialize it.

However, if that is in a shared library, it's dynamic anyway. The
dynamic case is effectively exercised any time a shared library is
dlopen'ed that has a file scope pthread_once_t.

[da...@boostpro.com]

On Sunday, February 25, 2018 at 6:29:49 PM UTC-8, Kaz Kylheku wrote:

> On 2018-02-25, Dave Abrahams <dave-at-boostpro.com> wrote:
> > Hi,
> >
> > I want to dynamically allocate memory, then once, on demand, thread-safely initialize part of that memory, and potentially free the memory. Then later I want to repeat the process with what is potentially the same memory block (and thus the same address of pthread_once_t) that has been returned from the allocator. Does this
> >
> > a. "just work" as long as it is set up with PTHREAD_ONCE_INIT, or
> > b. do I need to do insert some kind of fence to ensure that all threads see the re-initialized memory, or
> > c. never work; don't do that!
> > d. something else I haven't thought of?
>
> It must work.

Well, I realize that's what the docs seem to imply, but…

> Firstly, POSIX says that PTHREAD_ONCE_INIT is a constant.
> Thus it may be used in assignment. This is important because, by
> contrast, the PTHREAD_MUTEX_INITIALIZER is described as a macro
> that can be used for initialization, and not a constant.

Sure, but when I assign it, I am doing nothing to ensure that assignment is visible to all threads before they call pthread_once on it. Therefore it seemed plausible that another core's cache still contains some other value for that memory location, and a thread running on that core and calling pthread_once would see the uninitialized value instead of PTHREAD_ONCE_INIT, and skip calling the initialization function, which would be a problem…

> Secondly, there is a storage restriction on the phread_once_t
> control variable: namely, it may not be defined in automatic storage
> (i.e. non-static local variable). There is no restriction against
> pthread_once_t being in dynamic storage.
>
> > I ask because all the examples I can find store the pthread_once_t in
> > static storage and never re-initialize it.
>
> However, if that is in a shared library, it's dynamic anyway. The
> dynamic case is effectively exercised any time a shared library is
> dlopen'ed that has a file scope pthread_once_t.

Hm, you make a good point. But you know how tricky races are; I wouldn't count on all that exercise to have flushed out a bug if this issue wasn't totally considered.

[Kaz Kylheku]

On 2018-02-26, da...@boostpro.com <da...@boostpro.com> wrote:
> On Sunday, February 25, 2018 at 6:29:49 PM UTC-8, Kaz Kylheku wrote:
>> On 2018-02-25, Dave Abrahams <dave-at-boostpro.com> wrote:
>> > Hi,
>> >
>> > I want to dynamically allocate memory, then once, on demand, thread-safely initialize part of that memory, and potentially free the memory. Then later I want to repeat the process with what is potentially the same memory block (and thus the same address of pthread_once_t) that has been returned from the allocator. Does this
>> >
>> > a. "just work" as long as it is set up with PTHREAD_ONCE_INIT, or
>> > b. do I need to do insert some kind of fence to ensure that all threads see the re-initialized memory, or
>> > c. never work; don't do that!
>> > d. something else I haven't thought of?
>>
>> It must work.
>
> Well, I realize that's what the docs seem to imply, but…

The docs clearly require it; if it doesn't work, it's
an implementation bug.

>> Firstly, POSIX says that PTHREAD_ONCE_INIT is a constant.
>> Thus it may be used in assignment. This is important because, by
>> contrast, the PTHREAD_MUTEX_INITIALIZER is described as a macro
>> that can be used for initialization, and not a constant.
>
> Sure, but when I assign it, I am doing nothing to ensure that assignment is visible to all threads before they call pthread_once on it. Therefore it seemed plausible that another core's cache still contains some other value for that memory location, and a thread running on that core and calling pthread_once would see the uninitialized value instead of PTHREAD_ONCE_INIT, and skip calling the initialization function, which would be a problem…

This depends on how the threads acquire the pointer to this entire
object that contains the "pthread_once_t".

Suppose that we dynamically allocate "struct point { int x, y; }"
and initialize x and y, then pass this on to other threads.

How do we ensure that the memory is synchronized so that threads
see the stable values of x and y?

Whatever approaches works for these x and y is applicable to the
initialization of a pthread_once_t.

If we just allocate an object, initialize it, put its pointer
into an unprotected shared variable which another thread notices
has become non-null and uses it, and do not execute any memory
barriers, then we have a problem on some machines.

If some synchronization is involved then we should be okay. E.g.
consumer is waiting on a condition variable for the pointer to
materialize and then uses it.

static obj *ptr; // assumed initially null

// producer
new_object->once = PTHREAD_ONCE_INIT;
pthread_mutex_lock(&mut);
ptr = new_object;
pthread_mutex_unlock(&mut);
pthread_cond_broadcast(&cond);

// consumer
pthread_mutex_lock(&mut);
while (ptr == 0)
pthread_cond_wait(&cond, &mut);
pthread_mutex_unlock(&mut);
pthread_once(&ptr->once, init_fun);

Here, the memory synchronizing properties of the mutex ensure
that all the assignments done prior to the producer's
pthread_mutex_unlock call are visible at the time the consumer
acquires the mutex and finds ptr not to be null.

The consumer cannot find a non-null ptr, such that
ptr->once has not yet been initialized.

If you don't use synchronization like this when introducing the pointer
to the other threads, then you're on your own; but it's not really an
issue of the pthread_once, but rather an issue of the the reordering of
the loads and stores of "ptr" and "ptr->once".

>> Secondly, there is a storage restriction on the phread_once_t
>> control variable: namely, it may not be defined in automatic storage
>> (i.e. non-static local variable). There is no restriction against
>> pthread_once_t being in dynamic storage.
>>
>> > I ask because all the examples I can find store the pthread_once_t in
>> > static storage and never re-initialize it.
>>
>> However, if that is in a shared library, it's dynamic anyway. The
>> dynamic case is effectively exercised any time a shared library is
>> dlopen'ed that has a file scope pthread_once_t.
>
> Hm, you make a good point. But you know how tricky races are; I
> wouldn't count on all that exercise to have flushed out a bug if this
> issue wasn't totally considered.

But, come to think of it, is this situation really protected in all
circumstances?

Suppose thread A obtains a handle with dlopen, passes it unsafely to
thread B (e.g. just puts it into a shared location where B picks it up,
no mutexes).

Is it guaranteed that B can use dlsym, etc, to read the stable value of
a global variable variable?

I think, if so, it's only because some mutexes are incidentally used
inside dlopen, like to insert the library into some global list
of loaded libraries.

[I apologize for this topical interruption and now return you to the
regular deluge of drivel by the local troll.]