Fwd: Run background tasks in foreground pool on platforms without background thread priority

[François Doray]

+scheduler-dev Which might be the right list for this TaskScheduler discussion.

---------- Forwarded message ---------
From: François Doray <fdo...@google.com>
Date: Tue, May 22, 2018 at 11:49 AM
Subject: Re: Run background tasks in foreground pool on platforms without background thread priority
To: task-sche...@chromium.org <task-sche...@chromium.org>

Conclusion from 1:1 meeting: The maximum number of background tasks that can run concurrently should be half the size of the smallest pool. This ensures that background tasks won't flood a pool and limits the number of background tasks that can run concurrently, which is desirable in general.

A question remains: If a background task blocks in a ScopedBlockingCall, should the maximum number of background tasks that can run concurrently be incremented to compensate? I think we don't have a choice, if we want to support background tasks that block forever (e.g. code) or that synchronously wait on each other. This however allows this problematic situation:

1. Pool capacity = 2. Max background tasks = 1.
2. Background task A starts running and enters ScopedBlockingCall(WILL_BLOCK). Pool capacity = 3. Max background tasks = 2.
3. Background task B starts running
4. Background task A exits its ScopedBlockingCall. Pool capacity = 2. Max background tasks = 1.
5. Since we can't preempt running tasks, there are now 2 background tasks using the 2 threads in the pool, despite the max background tasks being 1.

Should there also be a rule that the pool capacity must be greater than the current number of running background tasks?

On Tue, May 22, 2018 at 9:52 AM François Doray <fdo...@google.com> wrote:

I've uploaded a CL that redirects background tasks to the foreground pool if the platform doesn't support background thread priority. The only thing that's missing is setting a cap on the number of background tasks that can run concurrently. This cap would prevent background tasks from using all threads in a pool.

How should we set this cap?

1. Max concurrent background tasks in TaskScheduler = new Finch-configurable constant.
2. Max concurrent background tasks in TaskScheduler = sum of the sizes of the background and background-blocking pools.
• Caveat: On iOS, 2+2=4 > 3, allows background tasks to flood a foreground pool.
3. Max concurrent background tasks in foreground pool and foreground-blocking pool controlled by 2 separate Finch-configurable constants (max per pool).
• Caveat: The infrastructure to support this is not implemented.

[Gabriel Charette]

On Tue, May 22, 2018 at 11:49 AM 'François Doray' via task-scheduler-dev <task-sche...@chromium.org> wrote:

+scheduler-dev Which might be the right list for this TaskScheduler discussion.

Yes, task-scheduler-dev@ is dead.

 

---------- Forwarded message ---------
From: François Doray <fdo...@google.com>
Date: Tue, May 22, 2018 at 11:49 AM
Subject: Re: Run background tasks in foreground pool on platforms without background thread priority
To: task-sche...@chromium.org <task-sche...@chromium.org>

Conclusion from 1:1 meeting: The maximum number of background tasks that can run concurrently should be half the size of the smallest pool. This ensures that background tasks won't flood a pool and limits the number of background tasks that can run concurrently, which is desirable in general.

A question remains: If a background task blocks in a ScopedBlockingCall, should the maximum number of background tasks that can run concurrently be incremented to compensate? I think we don't have a choice, if we want to support background tasks that block forever (e.g. code) or that synchronously wait on each other. This however allows this problematic situation:

1. Pool capacity = 2. Max background tasks = 1.
2. Background task A starts running and enters ScopedBlockingCall(WILL_BLOCK). Pool capacity = 3. Max background tasks = 2.
3. Background task B starts running
4. Background task A exits its ScopedBlockingCall. Pool capacity = 2. Max background tasks = 1.
5. Since we can't preempt running tasks, there are now 2 background tasks using the 2 threads in the pool, despite the max background tasks being 1.

Should there also be a rule that the pool capacity must be greater than the current number of running background tasks?

Interesting, that's fine if background tasks using ScopedBlockingCalls don't do much other work after exiting the ScopedBlockingCall. I think that's generally true, shall we make it a rule?

I guess a hard rule that the capacity must be greater than the number of running background tasks is also interesting, I'm just not sure this is much of a problem in practice. If we're not in a ScopedBlockingCall we're technically already fully subscribing the machine and we should finish that background task ASAP instead of oversubscribing the machine with another thread (the kernel will not know which one is a "background task" and will be slow at both -- the whole point of joining the pools on these platforms is the lack of kernel thread priority).

Thanks for thinking this through.

On Tue, May 22, 2018 at 9:52 AM François Doray <fdo...@google.com> wrote:

I've uploaded a CL that redirects background tasks to the foreground pool if the platform doesn't support background thread priority. The only thing that's missing is setting a cap on the number of background tasks that can run concurrently. This cap would prevent background tasks from using all threads in a pool.

How should we set this cap?

1. Max concurrent background tasks in TaskScheduler = new Finch-configurable constant.
2. Max concurrent background tasks in TaskScheduler = sum of the sizes of the background and background-blocking pools.
• Caveat: On iOS, 2+2=4 > 3, allows background tasks to flood a foreground pool.
3. Max concurrent background tasks in foreground pool and foreground-blocking pool controlled by 2 separate Finch-configurable constants (max per pool).
• Caveat: The infrastructure to support this is not implemented.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/task-scheduler-dev/CAGD3t5E2%3Dyfy1RWujB_uUwph-Fd0b0Do8TZymKrx9vB77aBmKQ%40mail.gmail.com.

[François Doray]

On Tue, May 22, 2018 at 12:00 PM Gabriel Charette <g...@chromium.org> wrote:

On Tue, May 22, 2018 at 11:49 AM 'François Doray' via task-scheduler-dev <task-sche...@chromium.org> wrote:

+scheduler-dev Which might be the right list for this TaskScheduler discussion.

Yes, task-scheduler-dev@ is dead.

 

---------- Forwarded message ---------
From: François Doray <fdo...@google.com>
Date: Tue, May 22, 2018 at 11:49 AM
Subject: Re: Run background tasks in foreground pool on platforms without background thread priority
To: task-sche...@chromium.org <task-sche...@chromium.org>

Conclusion from 1:1 meeting: The maximum number of background tasks that can run concurrently should be half the size of the smallest pool. This ensures that background tasks won't flood a pool and limits the number of background tasks that can run concurrently, which is desirable in general.

A question remains: If a background task blocks in a ScopedBlockingCall, should the maximum number of background tasks that can run concurrently be incremented to compensate? I think we don't have a choice, if we want to support background tasks that block forever (e.g. code) or that synchronously wait on each other. This however allows this problematic situation:

1. Pool capacity = 2. Max background tasks = 1.
2. Background task A starts running and enters ScopedBlockingCall(WILL_BLOCK). Pool capacity = 3. Max background tasks = 2.
3. Background task B starts running
4. Background task A exits its ScopedBlockingCall. Pool capacity = 2. Max background tasks = 1.
5. Since we can't preempt running tasks, there are now 2 background tasks using the 2 threads in the pool, despite the max background tasks being 1.

Should there also be a rule that the pool capacity must be greater than the current number of running background tasks?

Interesting, that's fine if background tasks using ScopedBlockingCalls don't do much other work after exiting the ScopedBlockingCall. I think that's generally true, shall we make it a rule?

I guess a hard rule that the capacity must be greater than the number of running background tasks is also interesting, I'm just not sure this is much of a problem in practice. If we're not in a ScopedBlockingCall we're technically already fully subscribing the machine and we should finish that background task ASAP instead of oversubscribing the machine with another thread (the kernel will not know which one is a "background task" and will be slow at both -- the whole point of joining the pools on these platforms is the lack of kernel thread priority).

Do you agree with this?

1. We need to increase the max number of background tasks when a background task blocks, to avoid deadlocks and starvation of background tasks.
2. We won't do anything to handle background tasks that continue to run after exiting a ScopedBlocking (potentially causing a pool to be flooded with background tasks), because it shouldn't happen too often.

Instead of having pool sizes in SchedulerWorkerPoolImpl and max background tasks in TaskTracker, should we just have pools that can grow indefinitely and these settings in TaskTracker:

• Max tasks not in a blocking scope (background+foreground).
• Max background tasks in a blocking scope.
• The number of tasks in a blocking scope is always 256.
• That might not work on Android.

[Gabriel Charette]

On Tue, May 22, 2018 at 12:40 PM 'François Doray' via task-scheduler-dev <task-sche...@chromium.org> wrote:

On Tue, May 22, 2018 at 12:00 PM Gabriel Charette <g...@chromium.org> wrote:

On Tue, May 22, 2018 at 11:49 AM 'François Doray' via task-scheduler-dev <task-sche...@chromium.org> wrote:

+scheduler-dev Which might be the right list for this TaskScheduler discussion.

Yes, task-scheduler-dev@ is dead.

 

---------- Forwarded message ---------
From: François Doray <fdo...@google.com>
Date: Tue, May 22, 2018 at 11:49 AM
Subject: Re: Run background tasks in foreground pool on platforms without background thread priority
To: task-sche...@chromium.org <task-sche...@chromium.org>

Conclusion from 1:1 meeting: The maximum number of background tasks that can run concurrently should be half the size of the smallest pool. This ensures that background tasks won't flood a pool and limits the number of background tasks that can run concurrently, which is desirable in general.

A question remains: If a background task blocks in a ScopedBlockingCall, should the maximum number of background tasks that can run concurrently be incremented to compensate? I think we don't have a choice, if we want to support background tasks that block forever (e.g. code) or that synchronously wait on each other. This however allows this problematic situation:

1. Pool capacity = 2. Max background tasks = 1.
2. Background task A starts running and enters ScopedBlockingCall(WILL_BLOCK). Pool capacity = 3. Max background tasks = 2.
3. Background task B starts running
4. Background task A exits its ScopedBlockingCall. Pool capacity = 2. Max background tasks = 1.
5. Since we can't preempt running tasks, there are now 2 background tasks using the 2 threads in the pool, despite the max background tasks being 1.

Should there also be a rule that the pool capacity must be greater than the current number of running background tasks?

Interesting, that's fine if background tasks using ScopedBlockingCalls don't do much other work after exiting the ScopedBlockingCall. I think that's generally true, shall we make it a rule?

I guess a hard rule that the capacity must be greater than the number of running background tasks is also interesting, I'm just not sure this is much of a problem in practice. If we're not in a ScopedBlockingCall we're technically already fully subscribing the machine and we should finish that background task ASAP instead of oversubscribing the machine with another thread (the kernel will not know which one is a "background task" and will be slow at both -- the whole point of joining the pools on these platforms is the lack of kernel thread priority).

Do you agree with this?

1. We need to increase the max number of background tasks when a background task blocks, to avoid deadlocks and starvation of background tasks.
2. We won't do anything to handle background tasks that continue to run after exiting a ScopedBlocking (potentially causing a pool to be flooded with background tasks), because it shouldn't happen too often.

Yes

 

Instead of having pool sizes in SchedulerWorkerPoolImpl and max background tasks in TaskTracker, should we just have pools that can grow indefinitely and these settings in TaskTracker:

• Max tasks not in a blocking scope (background+foreground).
• Max background tasks in a blocking scope.
• The number of tasks in a blocking scope is always 256.
• That might not work on Android.

Hmmm, I don't think we should mix MayBlock() and non-MayBlock() tasks. Or am I confused because we're now conflating MayBlock() and ScopedBlockingCall? WDYM by "blocking scope"

 

Thanks for thinking this through.

On Tue, May 22, 2018 at 9:52 AM François Doray <fdo...@google.com> wrote:

I've uploaded a CL that redirects background tasks to the foreground pool if the platform doesn't support background thread priority. The only thing that's missing is setting a cap on the number of background tasks that can run concurrently. This cap would prevent background tasks from using all threads in a pool.

How should we set this cap?

1. Max concurrent background tasks in TaskScheduler = new Finch-configurable constant.
2. Max concurrent background tasks in TaskScheduler = sum of the sizes of the background and background-blocking pools.
• Caveat: On iOS, 2+2=4 > 3, allows background tasks to flood a foreground pool.
3. Max concurrent background tasks in foreground pool and foreground-blocking pool controlled by 2 separate Finch-configurable constants (max per pool).
• Caveat: The infrastructure to support this is not implemented.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/task-scheduler-dev/CAGD3t5E2%3Dyfy1RWujB_uUwph-Fd0b0Do8TZymKrx9vB77aBmKQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/task-scheduler-dev/CAGD3t5G__DoTgGPF7J%2BbSb-QFS8tdWxYPS7Ts16YuAcA0v4Xuw%40mail.gmail.com.