How to create and use thread pools for tasks?

[Juha Nieminen]

If I understand correctly, this should be possible in C++11 (and later),
but I'm not sure how it's done. std::condition_variable is probably
involved, but I don't know how it's used for this.

So what I want to do is your typical thread pooling system, like this:

- At the start of the program, create n background threads.

- These background threads idle until the main thread gives them a task
to calculate, at which point the thread performs the task, then
informs the main thread that it's done, giving it the result, and then
starts idling again until it gets a new task (or, perhaps, if feasible,
the main thread may immediately give it a new task as a response.)

- The main thread gives tasks for those threads to calculate. To do this,
it does the following:
* First it gives all the background threads tasks to calculate.
* Then it waits for any thread to be done with its task.
* When a thread informs it that it's done, the main thread handles the
result, and gives that thread a new task.

- When all the tasks are done, all the background threads are simply left
idling, as new tasks may appear in the future. If that's the case, then
the previous step is repeated.

Could someone give a quick tutorial of how this is done?

[Paavo Helde]

In C++ there are lambdas, std::async, std::promise and std::future.
std::async may use or not use a thread pool internally, as far as I have
understood. Googling suggests probably not. On the other hand, creating
a new thread for each task would mean significant overhead if the tasks
are small.

The tasks to be computed are best constructed by lambdas. The lambda
syntax allows to prescribe easily which captured variables are copied
and which are referenced, this is important for thread-safety.

If you want to create your own explicit thread pool, this is IMO best
organized by a message queue. The needed jobs are posted in the queue. N
worker threads are all waiting on the queue, when a new job arrives one
of them will extract it and process it, then goes back to wait on the
queue. The queue can be easily built on top of a
std::deque<std::function>, std::mutex and std::condition_variable. I'm
sure there are also non-blocking variants possible for very high
performance needs.

The thread pool would replace the std::async part; other parts could
still use the same components (lambdas, std::promise, std::future).
cppreference.com has some examples. My own thread pool implementations
predate c++11, so I'm not so familiar with std::promise and std::future,
but in a new implementation I would probably try to use them.

[Jorgen Grahn]

On Tue, 2020-07-28, Paavo Helde wrote:
> 28.07.2020 11:54 Juha Nieminen kirjutas:
>> If I understand correctly, this should be possible in C++11 (and later),
>> but I'm not sure how it's done. std::condition_variable is probably
>> involved, but I don't know how it's used for this.
>>
>> So what I want to do is your typical thread pooling system, like this:

...

>> Could someone give a quick tutorial of how this is done?

...

> If you want to create your own explicit thread pool, this is IMO best
> organized by a message queue. The needed jobs are posted in the queue. N
> worker threads are all waiting on the queue, when a new job arrives one
> of them will extract it and process it, then goes back to wait on the
> queue. The queue can be easily built on top of a
> std::deque<std::function>, std::mutex and std::condition_variable.

It would have been nice if there was a std message queue, not just the
building blocks needed. Not because it's hard to write one, but to
encourage that way of communicating with threads. It's not the /only/
way, but a good default mechanism.

Not that I'm an expert on threads or anything.

/Jorgen

--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .

[Juha Nieminen]

Paavo Helde <ees...@osa.pri.ee> wrote:
> If you want to create your own explicit thread pool, this is IMO best
> organized by a message queue.

That gives me the idea for the project I'm thinking that, indeed, maybe
it's easier if the threads themselves just find out by themselves what
is the next task to perform (in this case they all do the same kind task,
just with different parameters).

The only thing I need to figure out is to have the threads idle until
the main thread has written the required parameters to a common struct
and then notify all the threads to start taking tasks from there.

Then I also need to figure out how to make the threads notify the
main thread when all the tasks are done.

I wonder if this should be done with std::future or with
std::condition_variable.

[Paavo Helde]

28.07.2020 21:23 Juha Nieminen kirjutas:
> Paavo Helde <ees...@osa.pri.ee> wrote:
>> If you want to create your own explicit thread pool, this is IMO best
>> organized by a message queue.
>
> That gives me the idea for the project I'm thinking that, indeed, maybe
> it's easier if the threads themselves just find out by themselves what
> is the next task to perform (in this case they all do the same kind task,
> just with different parameters).

Well, that's what the queue is about. The worker thread finds out its
task by extracting an entry from the queue.

> The only thing I need to figure out is to have the threads idle until
> the main thread has written the required parameters to a common struct
> and then notify all the threads to start taking tasks from there.

For each task, the main thread would fill out a struct with parameters,
pushes it into the queue under a mutec lock and call
conditionvar.notify_one().

Each worker thread would lock the queue mutex and call
conditionvar.wait() (which unlocks the mutex behind the scenes while
waiting). When the wait call returns, the worker thread checks if there
is something in the queue (as there can be spurious wakeups), if so,
pops a task from the queue and releases the mutex. That's it.

>
> Then I also need to figure out how to make the threads notify the
> main thread when all the tasks are done.
>
> I wonder if this should be done with std::future or with
> std::condition_variable.

This can be done by another queue (containing another mutex and another
condition variable). When ready, each thread would push the result to
the result queue and notify its condition variable. The main thread
waits on this condition variable, when notified, extracts the result
from the queue, puts it into the right place and increments the result
count. When all results are received, it stops waiting.

Your aim is to minimize the time spent under mutex locks when
pushing/popping queues. This can be achieved by using move or swap in
those operations. Queue-based thread synchronization is great in that
only these cheap move/swap operations need to be protected, and
otherwise the threads can run by their own without any need to worry
about MT safety or synchronization.

[Chris M. Thomasson]

For some reason the following message might be of interest to you. This
was before c++11 came out.

https://groups.google.com/forum/#!original/comp.lang.c++/nKgbzzJ5nXU/Aqgcj1NOIHEJ


Here is the response from Scott Meyer's:

https://groups.google.com/forum/#!original/comp.lang.c++/nKgbzzJ5nXU/VqzeTsbFWioJ

[Chris M. Thomasson]

On 7/28/2020 1:54 AM, Juha Nieminen wrote:

Read all of:

https://groups.google.com/d/topic/comp.lang.c++/ENpo61_GLaw/discussion

crude xample code:

https://pastebin.com/raw/GA9pJvKN

Might help you out a bit?