Posting a shutdown-blocking task to the current sequence

[Ken Rockot]

I have some code which uses the current SequencedTaskRunnerHandle to implement behavior similar to PostTaskAndReply, i.e. it does async work and then replies to the caller on the original calling sequence.

In this case however, it is imperative that the reply task block shutdown. I have not found any way to accomplish this with our current scheduling APIs. Am I missing something?

For more context, the reply callback here will always have a base::SequenceBound<T> argument whose target TaskRunner is shutdown-blocking. This means that if shutdown completes before the reply task can execute, the SequencedBound will be destroyed after shutdown has completed. This will result in it attempting to post a shutdown-blocking task (to run ~T) after doing so is no longer allowed.

[Gabriel Charette]

Le mar. 16 févr. 2021 20 h 33, 'Ken Rockot' via scheduler-dev <schedu...@chromium.org> a écrit :

I have some code which uses the current SequencedTaskRunnerHandle to implement behavior similar to PostTaskAndReply, i.e. it does async work and then replies to the caller on the original calling sequence.

In this case however, it is imperative that the reply task block shutdown. I have not found any way to accomplish this with our current scheduling APIs. Am I missing something?

If you post a BLOCK_SHUTDOWN task (e.g. your reply) from within another BLOCK_SHUTDOWN task, it is guaranteed to extend shutdown.

Caveats :

1) ShutdownBehavior is associated with a task runner, not an individual task. This is to avoid having non blocking tasks that surprisingly become blocking when sequenced ahead of blocking tasks.

2) ShutdownBehavior is only supported by ThreadPool task runners (via base::TaskTraits). This is because BrowserThread::UI never runs again once shutdown is initiated (I've thought of changing that but it's been like this since Chrome v1) and BrowserThread::IO is Thread::Join()'ed on shutdown so effectively BLOCK_SHUTDOWN already.

That being said if the task runner you get from SequencedTaskRunnerHandle is BLOCK_SHUTDOWN, so will your reply be.

For more context, the reply callback here will always have a base::SequenceBound<T> argument whose target TaskRunner is shutdown-blocking. This means that if shutdown completes before the reply task can execute, the SequencedBound will be destroyed after shutdown has completed. This will result in it attempting to post a shutdown-blocking task (to run ~T) after doing so is no longer allowed.

Now that's quite interesting. SequenceBound is one of the first widely used constructs that implicitly posts on destruction... This breaks the assertions built in to ThreadPool that try to prevent posting BLOCK_SHUTDOWN after Shutdown() as this catches ordering mistakes that would surprisingly result in a BLOCK_SHUTDOWN task not running.

This is the first time I hear about this limitation caused by ~SequenceBound. Do you actually need to run ~T? Maybe we could add Leaky traits to SequenceBound?

+Daniel Cheng ^^?

--
You received this message because you are subscribed to the Google Groups "scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheduler-de...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/scheduler-dev/CA%2BapAgFk44DbkwpnMViNWbxZHZuFbcix_cBOaTUuwHP9adGqDA%40mail.gmail.com.

[Ken Rockot]

On Wed, Feb 17, 2021 at 4:57 AM Gabriel Charette <g...@chromium.org> wrote:

Le mar. 16 févr. 2021 20 h 33, 'Ken Rockot' via scheduler-dev <schedu...@chromium.org> a écrit :

I have some code which uses the current SequencedTaskRunnerHandle to implement behavior similar to PostTaskAndReply, i.e. it does async work and then replies to the caller on the original calling sequence.

In this case however, it is imperative that the reply task block shutdown. I have not found any way to accomplish this with our current scheduling APIs. Am I missing something?

If you post a BLOCK_SHUTDOWN task (e.g. your reply) from within another BLOCK_SHUTDOWN task, it is guaranteed to extend shutdown.

Caveats :

1) ShutdownBehavior is associated with a task runner, not an individual task. This is to avoid having non blocking tasks that surprisingly become blocking when sequenced ahead of blocking tasks.

2) ShutdownBehavior is only supported by ThreadPool task runners (via base::TaskTraits). This is because BrowserThread::UI never runs again once shutdown is initiated (I've thought of changing that but it's been like this since Chrome v1) and BrowserThread::IO is Thread::Join()'ed on shutdown so effectively BLOCK_SHUTDOWN already.

That being said if the task runner you get from SequencedTaskRunnerHandle is BLOCK_SHUTDOWN, so will your reply be.

Right, unfortunately in this case the SequencedTaskRunnerHandle is acquired at an arbitrary point before the initial shutdown-blocking task is posted.

For more context, the reply callback here will always have a base::SequenceBound<T> argument whose target TaskRunner is shutdown-blocking. This means that if shutdown completes before the reply task can execute, the SequencedBound will be destroyed after shutdown has completed. This will result in it attempting to post a shutdown-blocking task (to run ~T) after doing so is no longer allowed.

Now that's quite interesting. SequenceBound is one of the first widely used constructs that implicitly posts on destruction... This breaks the assertions built in to ThreadPool that try to prevent posting BLOCK_SHUTDOWN after Shutdown() as this catches ordering mistakes that would surprisingly result in a BLOCK_SHUTDOWN task not running.

This is the first time I hear about this limitation caused by ~SequenceBound. Do you actually need to run ~T? Maybe we could add Leaky traits to SequenceBound?

+Daniel Cheng ^^?

Yeah, a leaky SequenceBound is sufficient in this case. I've already made this one leaky by wrapping it with a sometimes-leaked unique_ptr, but I was hoping for a cleaner approach. :)

I suppose the rationale here is that if you allow distinct per-task TaskShutdownBehavior, you will occasionally need to violate (or complicate) the sequencing guarantees of the API?

[Wez]

So is it the case that the TaskRunner to which the PostTaskAndReply task is being posted is _not_ shutdown-blocking? Only the reply TaskRunner is?

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/scheduler-dev/CA%2BapAgE%2BjgHBHnDcJ6Buz3QT7GL3oGbshT7%3DcFw8fViFXRWM7A%40mail.gmail.com.

[Gabriel Charette]

Le mer. 17 févr. 2021 10 h 59, 'Ken Rockot' via scheduler-dev <schedu...@chromium.org> a écrit :

On Wed, Feb 17, 2021 at 4:57 AM Gabriel Charette <g...@chromium.org> wrote:

Le mar. 16 févr. 2021 20 h 33, 'Ken Rockot' via scheduler-dev <schedu...@chromium.org> a écrit :

I have some code which uses the current SequencedTaskRunnerHandle to implement behavior similar to PostTaskAndReply, i.e. it does async work and then replies to the caller on the original calling sequence.

In this case however, it is imperative that the reply task block shutdown. I have not found any way to accomplish this with our current scheduling APIs. Am I missing something?

If you post a BLOCK_SHUTDOWN task (e.g. your reply) from within another BLOCK_SHUTDOWN task, it is guaranteed to extend shutdown.

Caveats :

1) ShutdownBehavior is associated with a task runner, not an individual task. This is to avoid having non blocking tasks that surprisingly become blocking when sequenced ahead of blocking tasks.

2) ShutdownBehavior is only supported by ThreadPool task runners (via base::TaskTraits). This is because BrowserThread::UI never runs again once shutdown is initiated (I've thought of changing that but it's been like this since Chrome v1) and BrowserThread::IO is Thread::Join()'ed on shutdown so effectively BLOCK_SHUTDOWN already.

That being said if the task runner you get from SequencedTaskRunnerHandle is BLOCK_SHUTDOWN, so will your reply be.

Right, unfortunately in this case the SequencedTaskRunnerHandle is acquired at an arbitrary point before the initial shutdown-blocking task is posted.

Timing shouldn't matter. The task runner is always or never BLOCK_SHUTDOWN.

For more context, the reply callback here will always have a base::SequenceBound<T> argument whose target TaskRunner is shutdown-blocking. This means that if shutdown completes before the reply task can execute, the SequencedBound will be destroyed after shutdown has completed. This will result in it attempting to post a shutdown-blocking task (to run ~T) after doing so is no longer allowed.

Now that's quite interesting. SequenceBound is one of the first widely used constructs that implicitly posts on destruction... This breaks the assertions built in to ThreadPool that try to prevent posting BLOCK_SHUTDOWN after Shutdown() as this catches ordering mistakes that would surprisingly result in a BLOCK_SHUTDOWN task not running.

This is the first time I hear about this limitation caused by ~SequenceBound. Do you actually need to run ~T? Maybe we could add Leaky traits to SequenceBound?

+Daniel Cheng ^^?

Yeah, a leaky SequenceBound is sufficient in this case. I've already made this one leaky by wrapping it with a sometimes-leaked unique_ptr, but I was hoping for a cleaner approach. :)

I suppose the rationale here is that if you allow distinct per-task TaskShutdownBehavior, you will occasionally need to violate (or complicate) the sequencing guarantees of the API?

--
You received this message because you are subscribed to the Google Groups "scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheduler-de...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/scheduler-dev/CA%2BapAgFk44DbkwpnMViNWbxZHZuFbcix_cBOaTUuwHP9adGqDA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheduler-de...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/scheduler-dev/CA%2BapAgE%2BjgHBHnDcJ6Buz3QT7GL3oGbshT7%3DcFw8fViFXRWM7A%40mail.gmail.com.

[Gabriel Charette]

Le mer. 17 févr. 2021 12 h 31, Wez <w...@chromium.org> a écrit :

So is it the case that the TaskRunner to which the PostTaskAndReply task is being posted is _not_ shutdown-blocking? Only the reply TaskRunner is?

My understanding is both are shutdown-blocking. Otherwise it definitely won't work.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/scheduler-dev/CALekkJdq41okvwa-Si6rrsja1WvEXbjS-CCDYoGW2121q3q%3DEg%40mail.gmail.com.

[Ken Rockot]

On Wed, Feb 17, 2021 at 10:53 AM Gabriel Charette <g...@chromium.org> wrote:

Le mer. 17 févr. 2021 10 h 59, 'Ken Rockot' via scheduler-dev <schedu...@chromium.org> a écrit :

On Wed, Feb 17, 2021 at 4:57 AM Gabriel Charette <g...@chromium.org> wrote:

Le mar. 16 févr. 2021 20 h 33, 'Ken Rockot' via scheduler-dev <schedu...@chromium.org> a écrit :

I have some code which uses the current SequencedTaskRunnerHandle to implement behavior similar to PostTaskAndReply, i.e. it does async work and then replies to the caller on the original calling sequence.

In this case however, it is imperative that the reply task block shutdown. I have not found any way to accomplish this with our current scheduling APIs. Am I missing something?

If you post a BLOCK_SHUTDOWN task (e.g. your reply) from within another BLOCK_SHUTDOWN task, it is guaranteed to extend shutdown.

Caveats :

1) ShutdownBehavior is associated with a task runner, not an individual task. This is to avoid having non blocking tasks that surprisingly become blocking when sequenced ahead of blocking tasks.

2) ShutdownBehavior is only supported by ThreadPool task runners (via base::TaskTraits). This is because BrowserThread::UI never runs again once shutdown is initiated (I've thought of changing that but it's been like this since Chrome v1) and BrowserThread::IO is Thread::Join()'ed on shutdown so effectively BLOCK_SHUTDOWN already.

That being said if the task runner you get from SequencedTaskRunnerHandle is BLOCK_SHUTDOWN, so will your reply be.

Right, unfortunately in this case the SequencedTaskRunnerHandle is acquired at an arbitrary point before the initial shutdown-blocking task is posted.

Timing shouldn't matter. The task runner is always or never BLOCK_SHUTDOWN.

To be clear, I mean it's at the caller's discretion where the call is made from. It could be from any task execution context, not necessarily from the same blocking task runner that is used to post the first task.

Similar to PostTaskAndReply -- the reply runs wherever PostTaskAndReply is called.

[Ken Rockot]

On Wed, Feb 17, 2021 at 10:53 AM Gabriel Charette <g...@chromium.org> wrote:

Le mer. 17 févr. 2021 12 h 31, Wez <w...@chromium.org> a écrit :

So is it the case that the TaskRunner to which the PostTaskAndReply task is being posted is _not_ shutdown-blocking? Only the reply TaskRunner is?

My understanding is both are shutdown-blocking. Otherwise it definitely won't work.

Looking at the implementation:

- the PostTask task runner is whatever you ask for - if you provide BLOCK_SHUTDOWN, it will block shutdown

- the reply task runner just SequencedTaskRunnerHandle::Get(), at the time PostTaskAndReply is called

So in general the two task runners are determined independently, and therefore so is the shutdown-blocking behavior of each task. I don't see any guarantee that both are either shutdown-blocking or not.

[Gabriel Charette]

Le mer. 17 févr. 2021 14 h 54, 'Ken Rockot' via scheduler-dev <schedu...@chromium.org> a écrit :

On Wed, Feb 17, 2021 at 10:53 AM Gabriel Charette <g...@chromium.org> wrote:

Le mer. 17 févr. 2021 12 h 31, Wez <w...@chromium.org> a écrit :

So is it the case that the TaskRunner to which the PostTaskAndReply task is being posted is _not_ shutdown-blocking? Only the reply TaskRunner is?

My understanding is both are shutdown-blocking. Otherwise it definitely won't work.

Looking at the implementation:

- the PostTask task runner is whatever you ask for - if you provide BLOCK_SHUTDOWN, it will block shutdown

- the reply task runner just SequencedTaskRunnerHandle::Get(), at the time PostTaskAndReply is called

So in general the two task runners are determined independently, and therefore so is the shutdown-blocking behavior of each task. I don't see any guarantee that both are either shutdown-blocking or not.

Indeed, I thought you meant your impl was PostTaskAndReply-like. Not literally PostTaskAndReply. You indeed won't be able to apply a TaskShutdownBehavior on an arbitrary reply sequence.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/scheduler-dev/CA%2BapAgHtgfYSi9K%2BrAsM3C0YhQorbB%3Dc3oprbUivQE_md4cUow%40mail.gmail.com.

[Wez]

It seems that we need a special-case for requests to delete objects on shutdown-blocking sequences during/after shutdown - perhaps that's what you meant by a "leaky" trait, Gab?

[Gabriel Charette]

On Thu, Feb 18, 2021 at 4:50 AM Wez <w...@chromium.org> wrote:

It seems that we need a special-case for requests to delete objects on shutdown-blocking sequences during/after shutdown - perhaps that's what you meant by a "leaky" trait, Gab?

I literally meant "leaking" because that's our favored model on shutdown already (ref. base::NoDestructor). i.e. if it's too late to post, just leak T*.

We could also have special logic to ~T off-sequence when the sequence is dead I guess (it's technically still sequenced because the owning sequence is gone, so the UI thread is the "worker thread" running ~T). But that sounds more complicated than leaking (need to teach SequenceChecker about this fake extension of the original sequence). Hence why I'm asking if LeakyTraits would be enough (like LeakyLazyInstanceTraits).

 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/scheduler-dev/CALekkJde%2BQ0R1hBtyeG2HC43Hc5YXASLWyknSrXRtGhxSAu9bQ%40mail.gmail.com.