WithIO()

5 views
Skip to first unread message

François Doray

unread,
Dec 13, 2016, 5:05:01 PM12/13/16
to task-sche...@chromium.org
Yesterday, we discussed replacing WithFileIO() with WithIO(IOType io_type = IOType::DISK_IO). IOType would initially be { DISK_IO, PIPE_IO, NETWORK_IO, DEVICE_IO }. Making our traits more granular opens up opportunities for future improvements. On the other hand, it makes the migration more complex (we have to figure out what each file descriptor represents), is more error-prone (no assertions to verify our traits) and it makes post_task.h call sites more verbose.

I would like to bring a few new arguments in favor of having simpler traits (maybe a generic .MayBlock() instead of .WithWait() + .WithIO(IOType io_type = IOType::DISK_IO)):
  • Developers may make bad assumptions after reading code that uses incorrect traits.
  • A pipe may be used for the same purpose as a WaitableEvent code. If code is refactored to use a WaitableEvent instead of a pipe, will all post task call sites be updated? Generic traits have less chances of becoming obsolete.
  • It's easier to say that a system/library call blocks than to figure precisely what it does (e.g. crypto code, ::LogonUser code). There is less confusion with .MayBlock().
  • It might be harder to use the post_task.h API directly rather than receiving a TaskRunner if we have to provide granular traits (e.g. net::FileStream is used with pipes, sockets and real files!).
I know that we can simplify our traits later if we end up not using all of them. However, that means that time will have been wasted to annotate tasks with granular traits and that Chrome developers will have to re-learn how to use the post_task.h API.

Thoughts?

Robert Liao

unread,
Dec 13, 2016, 5:08:15 PM12/13/16
to François Doray, task-sche...@chromium.org
we have to figure out what each file descriptor represents
This is why the traits I prescribed are based off of API usage instead of what the descriptor actually represents.

From the chat:
The original three (at least what I thought)
WithWait = synchronization primitives used
WithFileIO = sync file APIs used
WithIO= any other sync IO used

Developers already have to consider the consequences of their API calls, so they should have a good idea on what sort of operations the API does.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-dev+unsub...@chromium.org.
To post to this group, send email to task-scheduler-dev@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/task-scheduler-dev/CAGD3t5Hxi%3D2JOAgWppc0aCmnV60rq7de6_AMjUy%2BOdiUbZsKxA%40mail.gmail.com.

François Doray

unread,
Dec 13, 2016, 5:25:37 PM12/13/16
to Robert Liao, task-sche...@chromium.org


On Tue, Dec 13, 2016, 5:08 PM Robert Liao <rob...@chromium.org> wrote:
we have to figure out what each file descriptor represents
This is why the traits I prescribed are based off of API usage instead of what the descriptor actually represents.

From the chat:
The original three (at least what I thought)
WithWait = synchronization primitives used
WithFileIO = sync file APIs used
WithIO= any other sync IO used

These three traits are indeed simpler than .WithWait() + .WithIO(IOType). In this world, would you use . WithIO() for a blocking ::LogonUser() call? What about joining a process that does IO (recovery_component_installer.cc)?


Developers already have to consider the consequences of their API calls, so they should have a good idea on what sort of operations the API does.

On Tue, Dec 13, 2016 at 2:04 PM, François Doray <fdo...@chromium.org> wrote:
Yesterday, we discussed replacing WithFileIO() with WithIO(IOType io_type = IOType::DISK_IO). IOType would initially be { DISK_IO, PIPE_IO, NETWORK_IO, DEVICE_IO }. Making our traits more granular opens up opportunities for future improvements. On the other hand, it makes the migration more complex (we have to figure out what each file descriptor represents), is more error-prone (no assertions to verify our traits) and it makes post_task.h call sites more verbose.

I would like to bring a few new arguments in favor of having simpler traits (maybe a generic .MayBlock() instead of .WithWait() + .WithIO(IOType io_type = IOType::DISK_IO)):
  • Developers may make bad assumptions after reading code that uses incorrect traits.
  • A pipe may be used for the same purpose as a WaitableEvent code. If code is refactored to use a WaitableEvent instead of a pipe, will all post task call sites be updated? Generic traits have less chances of becoming obsolete.
  • It's easier to say that a system/library call blocks than to figure precisely what it does (e.g. crypto code, ::LogonUser code). There is less confusion with .MayBlock().
  • It might be harder to use the post_task.h API directly rather than receiving a TaskRunner if we have to provide granular traits (e.g. net::FileStream is used with pipes, sockets and real files!).
I know that we can simplify our traits later if we end up not using all of them. However, that means that time will have been wasted to annotate tasks with granular traits and that Chrome developers will have to re-learn how to use the post_task.h API.

Thoughts?

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/task-scheduler-dev/CAGD3t5Hxi%3D2JOAgWppc0aCmnV60rq7de6_AMjUy%2BOdiUbZsKxA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/task-scheduler-dev/CAK7A45VpprCkD%3DVJMRaPw4w1tDS6_B0ucEswf2ui62FA37TuiA%40mail.gmail.com.

Robert Liao

unread,
Dec 13, 2016, 5:41:01 PM12/13/16
to François Doray, task-sche...@chromium.org
::LogonUser isn't a sync primitive and isn't a sync file API call, so it goes into WithIO.

Joining a process is a sync primitive call (ends up being WaitFor[Single|Multiple]Object[s], so that's WithWait.

On Tue, Dec 13, 2016 at 2:25 PM, François Doray <fdo...@chromium.org> wrote:


On Tue, Dec 13, 2016, 5:08 PM Robert Liao <rob...@chromium.org> wrote:
we have to figure out what each file descriptor represents
This is why the traits I prescribed are based off of API usage instead of what the descriptor actually represents.

From the chat:
The original three (at least what I thought)
WithWait = synchronization primitives used
WithFileIO = sync file APIs used
WithIO= any other sync IO used

These three traits are indeed simpler than .WithWait() + .WithIO(IOType). In this world, would you use . WithIO() for a blocking ::LogonUser() call? What about joining a process that does IO (recovery_component_installer.cc)?


Developers already have to consider the consequences of their API calls, so they should have a good idea on what sort of operations the API does.

On Tue, Dec 13, 2016 at 2:04 PM, François Doray <fdo...@chromium.org> wrote:
Yesterday, we discussed replacing WithFileIO() with WithIO(IOType io_type = IOType::DISK_IO). IOType would initially be { DISK_IO, PIPE_IO, NETWORK_IO, DEVICE_IO }. Making our traits more granular opens up opportunities for future improvements. On the other hand, it makes the migration more complex (we have to figure out what each file descriptor represents), is more error-prone (no assertions to verify our traits) and it makes post_task.h call sites more verbose.

I would like to bring a few new arguments in favor of having simpler traits (maybe a generic .MayBlock() instead of .WithWait() + .WithIO(IOType io_type = IOType::DISK_IO)):
  • Developers may make bad assumptions after reading code that uses incorrect traits.
  • A pipe may be used for the same purpose as a WaitableEvent code. If code is refactored to use a WaitableEvent instead of a pipe, will all post task call sites be updated? Generic traits have less chances of becoming obsolete.
  • It's easier to say that a system/library call blocks than to figure precisely what it does (e.g. crypto code, ::LogonUser code). There is less confusion with .MayBlock().
  • It might be harder to use the post_task.h API directly rather than receiving a TaskRunner if we have to provide granular traits (e.g. net::FileStream is used with pipes, sockets and real files!).
I know that we can simplify our traits later if we end up not using all of them. However, that means that time will have been wasted to annotate tasks with granular traits and that Chrome developers will have to re-learn how to use the post_task.h API.

Thoughts?

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-dev+unsub...@chromium.org.
To post to this group, send email to task-scheduler-dev@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/task-scheduler-dev/CAGD3t5Hxi%3D2JOAgWppc0aCmnV60rq7de6_AMjUy%2BOdiUbZsKxA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-dev+unsub...@chromium.org.
To post to this group, send email to task-scheduler-dev@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/task-scheduler-dev/CAK7A45VpprCkD%3DVJMRaPw4w1tDS6_B0ucEswf2ui62FA37TuiA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-dev+unsub...@chromium.org.
To post to this group, send email to task-scheduler-dev@chromium.org.
 To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/task-scheduler-dev/CAGD3t5F_7_ARPuZkYsESNT-XW%3DUGV-anjH32PaPvCuOMjCvV7Q%40mail.gmail.com.

Gabriel Charette

unread,
Dec 14, 2016, 12:04:52 PM12/14/16
to Robert Liao, François Doray, task-sche...@chromium.org
But Francois' point I think is that this labelling is error-prone. If we're fine with having a "blocking allowed" pool (instead of IO), then it doesn't matter why it blocks?

And ultimately we may want to handle the blockage ourselves (or defer it to the kernel on platforms that allow it). 

To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/task-scheduler-dev/CAGD3t5Hxi%3D2JOAgWppc0aCmnV60rq7de6_AMjUy%2BOdiUbZsKxA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/task-scheduler-dev/CAK7A45VpprCkD%3DVJMRaPw4w1tDS6_B0ucEswf2ui62FA37TuiA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/task-scheduler-dev/CAGD3t5F_7_ARPuZkYsESNT-XW%3DUGV-anjH32PaPvCuOMjCvV7Q%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/task-scheduler-dev/CAK7A45XqEqfFiowNh3RdsaL59bJUwCQmqR-krsDc-yGG%2B5%3DqJw%40mail.gmail.com.

Robert Liao

unread,
Dec 14, 2016, 12:56:23 PM12/14/16
to Gabriel Charette, François Doray, task-sche...@chromium.org
Most, if not all labeling will have some level of error to it. Even with LogonUser, someone may forget to label that with MayBlock and no one would be the wiser. 

Without static time analysis (some of which is theoretically undecidable) and with APIs that go straight to the OS (LogonUser), the best we can do is to make the guidance easy (if you call these APIs, you should do this), and provide feedback when possible (asserts, review).

Taking a step back, the main question we have to answer is do we have aspirations to handle FileIO differently from the other types of IO that can go through the system.

If the answer is yes, we should label these today as they are reasonably easy to determine (uses the file API).
If the answer is no, then I would agree, MayBlock is sufficient for our uses.

The main thing I would like to avoid is a global comb through of the codebase when we determine that a class of tasks would better be classified separately. That would require manual analysis on our part to label each call site.

I'm less concerned about removing a classification once we've determined it's not useful as removing a label is, in most cases, automatic.

To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-dev+unsub...@chromium.org.
To post to this group, send email to task-scheduler-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-dev+unsub...@chromium.org.
To post to this group, send email to task-scheduler-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-dev+unsub...@chromium.org.
To post to this group, send email to task-scheduler-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-dev+unsub...@chromium.org.
To post to this group, send email to task-scheduler-dev@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/task-scheduler-dev/CAK7A45XqEqfFiowNh3RdsaL59bJUwCQmqR-krsDc-yGG%2B5%3DqJw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-dev+unsub...@chromium.org.
To post to this group, send email to task-scheduler-dev@chromium.org.
 To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/task-scheduler-dev/CAJTZ7LK2w14pZudPSx%2BnvbwWUukgZ1eVdWEp2PDtqTNpP3jbgA%40mail.gmail.com.

Gabriel Charette

unread,
Dec 14, 2016, 1:06:37 PM12/14/16
to Robert Liao, Gabriel Charette, François Doray, task-sche...@chromium.org
I'd imagine you need .MayBlock() to pass AssertIO/WaitAllowed().

While I agree that removing traits later is easier: we should make sure our API is concise enough that developers enjoy using it (will be key to ensure help during migration).

Keeping .WithFileIO() means we also need to keep a separate .WithIO() (for AssertIOAllowed()).

In which case it's not clear whether a user of those is supposed to add .WithWait().

In the normal case you probably only want one.

But what if you're doing multiple types of IO or IO+wait in a single call..? And that's where it gets confusing (even more so when assigning traits to a TaskRunner).

The good side is we're starting with WorkerPool which has the most egregious tasks in Chrome I think. So perhaps it's okay to make those callsites a bit complex to have expressibility at the average callsite in the rest of the migration.

But overall if we can get away with .MayBlock() I think it'd greatly simplify our API. Let's loop in base/ OWNERS into a summary?

To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.
--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/task-scheduler-dev/CAK7A45WYQB7SxeRWAKpQek-7hCqU%3DY6BSXAu8sxU1__M-ZSoDA%40mail.gmail.com.

Robert Liao

unread,
Dec 14, 2016, 1:23:38 PM12/14/16
to Gabriel Charette, François Doray, task-sche...@chromium.org
Right, but LogonUser doesn't AssertIO/WaitAllowed, and so you wouldn't catch that. That's a direct Windows call.

If the guidance is simple, then figuring out WithFileIO/WithIO/WithWait should be pretty easy.

WithWait = Sync Primitives used
WithFileIO = Sync File APIs used
WithIO = Any other sync IO used

Three items is pretty concise and it's relatively fast to make a decision on which trait to use.

Did I use a lock? I need to use WithWait.
Is there a file API call here? Use WithFileIO.
Hrm... this does IO but isn't a file API call. I'll use WithIO.

I agree that when you mix the traits, things get a little more interesting, but it's no different than base's Wait/IO asserts today. You need to have a TaskTrait that has both WithWait and With[File]IO.

To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-dev+unsub...@chromium.org.
To post to this group, send email to task-scheduler-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-dev+unsub...@chromium.org.
To post to this group, send email to task-scheduler-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-dev+unsub...@chromium.org.
To post to this group, send email to task-scheduler-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-dev+unsub...@chromium.org.
To post to this group, send email to task-scheduler-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-dev+unsub...@chromium.org.
To post to this group, send email to task-scheduler-dev@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/task-scheduler-dev/CAJTZ7LK2w14pZudPSx%2BnvbwWUukgZ1eVdWEp2PDtqTNpP3jbgA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-dev+unsub...@chromium.org.
To post to this group, send email to task-scheduler-dev@chromium.org.

François Doray

unread,
Dec 14, 2016, 1:55:49 PM12/14/16
to Robert Liao, Gabriel Charette, task-sche...@chromium.org
Still playing the devil's advocate :)

On Wed, Dec 14, 2016 at 1:23 PM Robert Liao <rob...@chromium.org> wrote:
Right, but LogonUser doesn't AssertIO/WaitAllowed, and so you wouldn't catch that. That's a direct Windows call.

If the guidance is simple, then figuring out WithFileIO/WithIO/WithWait should be pretty easy.

WithWait = Sync Primitives used
WithFileIO = Sync File APIs used
WithIO = Any other sync IO used

If I make a synchronous mojo call, do I have to ask myself how it's implemented (hint: it is a post task to the IO thread + a wait on a sync primitive - but it could have been a write + read in a pipe), or do I just say "I'm not using a sync primitive or a sync file API directly"? What if mojo changes? Where is the boundary between "my code" and "someone's else code"? With .MayBlock(), there is no confusion.
 

Three items is pretty concise and it's relatively fast to make a decision on which trait to use.

Did I use a lock? I need to use WithWait.
Is there a file API call here? Use WithFileIO.
Hrm... this does IO but isn't a file API call. I'll use WithIO.

I agree that when you mix the traits, things get a little more interesting, but it's no different than base's Wait/IO asserts today. You need to have a TaskTrait that has both WithWait and With[File]IO.

On Wed, Dec 14, 2016 at 10:06 AM, Gabriel Charette <g...@chromium.org> wrote:
I'd imagine you need .MayBlock() to pass AssertIO/WaitAllowed().

While I agree that removing traits later is easier: we should make sure our API is concise enough that developers enjoy using it (will be key to ensure help during migration).

Keeping .WithFileIO() means we also need to keep a separate .WithIO() (for AssertIOAllowed()).

In which case it's not clear whether a user of those is supposed to add .WithWait().

In the normal case you probably only want one.

But what if you're doing multiple types of IO or IO+wait in a single call..? And that's where it gets confusing (even more so when assigning traits to a TaskRunner).

The good side is we're starting with WorkerPool which has the most egregious tasks in Chrome I think. So perhaps it's okay to make those callsites a bit complex to have expressibility at the average callsite in the rest of the migration.

But overall if we can get away with .MayBlock() I think it'd greatly simplify our API. Let's loop in base/ OWNERS into a summary?

On Wed, Dec 14, 2016 at 12:56 PM Robert Liao <rob...@chromium.org> wrote:
Most, if not all labeling will have some level of error to it. Even with LogonUser, someone may forget to label that with MayBlock and no one would be the wiser. 

Without static time analysis (some of which is theoretically undecidable) and with APIs that go straight to the OS (LogonUser), the best we can do is to make the guidance easy (if you call these APIs, you should do this), and provide feedback when possible (asserts, review).

Taking a step back, the main question we have to answer is do we have aspirations to handle FileIO differently from the other types of IO that can go through the system.

I don't see how we would ever handle different "blocking" traits differently. I believe that the goal is always to have |num_cores| tasks running on the CPU. To do that, we need to know when tasks may be blocked, but it doesn't make a difference why they are blocked.
 

If the answer is yes, we should label these today as they are reasonably easy to determine (uses the file API).
If the answer is no, then I would agree, MayBlock is sufficient for our uses.

The main thing I would like to avoid is a global comb through of the codebase when we determine that a class of tasks would better be classified separately. That would require manual analysis on our part to label each call site.

I agree. But there is a cost to have complex traits now.
 

I'm less concerned about removing a classification once we've determined it's not useful as removing a label is, in most cases, automatic.

On Wed, Dec 14, 2016 at 9:04 AM, Gabriel Charette <g...@chromium.org> wrote:
But Francois' point I think is that this labelling is error-prone. If we're fine with having a "blocking allowed" pool (instead of IO), then it doesn't matter why it blocks?

And ultimately we may want to handle the blockage ourselves (or defer it to the kernel on platforms that allow it). 

Detecting blockage without relying on traits would be great. We could use platform-specific API (Windows thread pool, GCD) and/or annotations (ScopedThreadMayBlock - would be used in blocking base/ functions and developers could use it when they block without going through base).
 

To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/task-scheduler-dev/CAJTZ7LK2w14pZudPSx%2BnvbwWUukgZ1eVdWEp2PDtqTNpP3jbgA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.

Gabriel Charette

unread,
Dec 14, 2016, 2:01:33 PM12/14/16
to François Doray, Robert Liao, Gabriel Charette, task-sche...@chromium.org
I say we do this: Francois summarizes pros/cons discussed here in a doc and we run it by base/ OWNERS.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/task-scheduler-dev/CAGD3t5HX_VdqSc_KAsmv0SNxhaQMTvrQSuqzk2NjpmXZ%3DQ%3DSsA%40mail.gmail.com.

Robert Liao

unread,
Dec 14, 2016, 2:19:55 PM12/14/16
to Gabriel Charette, François Doray, task-sche...@chromium.org
I say we do this: Francois summarizes pros/cons discussed here in a doc and we run it by base/ OWNERS.
Sure. We should, however, make this opinionated given that this determines where we go with the scheduler.

 If I make a synchronous mojo call, do I have to ask myself how it's implemented (hint: it is a post task to the IO thread + a wait on a sync primitive - but it could have been a write + read in a pipe), or do I just say "I'm not using a sync primitive or a sync file API directly"? What if mojo changes? Where is the boundary between "my code" and "someone's else code"? With .MayBlock(), there is no confusion.
If you are making a synchronous mojo call, you have to assume worst case (the service could be running outside of my process space, and there's potentially no way for the caller to figure that out). This means you have to assume IO.

I don't see how we would ever handle different "blocking" traits differently. I believe that the goal is always to have |num_cores| tasks running on the CPU. To do that, we need to know when tasks may be blocked, but it doesn't make a difference why they are blocked.
I agree that determining the number of threads is wait invariant (doesn't matter why you're waiting).
The main thing we can do here is reorder some of the IO so that we don't saturate the disk. A long time ago, we discussed separating File IO into different queues per disk (yes, this would require changing the base API). Consider a machine with two disks, A and B. If we have lots of work going towards A and a one-off task going towards B, we could decide to throttle A (to be a better citizen) and let B proceed.

Because we didn't want to change the file APIs and base already had a notion of IO, FileIO was born. 
 
I agree. But there is a cost to have complex traits now.
While MayBlock is simpler, considering IO and FileIO is only two more flags. Developers already have to consider priority in this system, so two more bits doesn't add much overall complexity to the system. I would argue the additional cost is marginal. We already have to assess if a call blocks. To do that, we have to determine why it blocks and that means figuring out the source of the blocking call.


To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-dev+unsub...@chromium.org.
To post to this group, send email to task-scheduler-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-dev+unsub...@chromium.org.
To post to this group, send email to task-scheduler-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-dev+unsub...@chromium.org.
To post to this group, send email to task-scheduler-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-dev+unsub...@chromium.org.
To post to this group, send email to task-scheduler-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-dev+unsub...@chromium.org.
To post to this group, send email to task-scheduler-dev@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/task-scheduler-dev/CAJTZ7LK2w14pZudPSx%2BnvbwWUukgZ1eVdWEp2PDtqTNpP3jbgA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-dev+unsub...@chromium.org.
To post to this group, send email to task-scheduler-dev@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-dev+unsub...@chromium.org.
To post to this group, send email to task-scheduler-dev@chromium.org.

Gabriel Charette

unread,
Dec 14, 2016, 2:45:15 PM12/14/16
to Robert Liao, Gabriel Charette, François Doray, task-sche...@chromium.org
On Wed, Dec 14, 2016 at 2:19 PM Robert Liao <rob...@chromium.org> wrote:
I say we do this: Francois summarizes pros/cons discussed here in a doc and we run it by base/ OWNERS.
Sure. We should, however, make this opinionated given that this determines where we go with the scheduler.

Yes, let's agree on doc before sending.
 

 If I make a synchronous mojo call, do I have to ask myself how it's implemented (hint: it is a post task to the IO thread + a wait on a sync primitive - but it could have been a write + read in a pipe), or do I just say "I'm not using a sync primitive or a sync file API directly"? What if mojo changes? Where is the boundary between "my code" and "someone's else code"? With .MayBlock(), there is no confusion.
If you are making a synchronous mojo call, you have to assume worst case (the service could be running outside of my process space, and there's potentially no way for the caller to figure that out). This means you have to assume IO.

I don't see how we would ever handle different "blocking" traits differently. I believe that the goal is always to have |num_cores| tasks running on the CPU. To do that, we need to know when tasks may be blocked, but it doesn't make a difference why they are blocked.
I agree that determining the number of threads is wait invariant (doesn't matter why you're waiting).
The main thing we can do here is reorder some of the IO so that we don't saturate the disk. A long time ago, we discussed separating File IO into different queues per disk (yes, this would require changing the base API). Consider a machine with two disks, A and B. If we have lots of work going towards A and a one-off task going towards B, we could decide to throttle A (to be a better citizen) and let B proceed.

I don't recall ever wanting to do that? Optimizing for machines with multiple disks doesn't seem useful for the average user (especially that Chrome would pretty much never have its own data split across two disks).

And I think we agree that we should send all the I/O we can to a given disk and let the kernel figure it out based on priority of calling threads, so throttling I/O shouldn't be a goal?
 
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/task-scheduler-dev/CAJTZ7LK2w14pZudPSx%2BnvbwWUukgZ1eVdWEp2PDtqTNpP3jbgA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.

Robert Liao

unread,
Dec 14, 2016, 3:23:00 PM12/14/16
to Gabriel Charette, François Doray, task-sche...@chromium.org
And I think we agree that we should send all the I/O we can to a given disk and let the kernel figure it out based on priority of calling threads, so throttling I/O shouldn't be a goal?
This was before we discovered that Windows preferred sending all at once and we were discussing Grand Central Dispatch style IO.

Gabriel Charette

unread,
Dec 14, 2016, 3:27:08 PM12/14/16
to Robert Liao, Gabriel Charette, François Doray, task-sche...@chromium.org
On Wed, Dec 14, 2016 at 3:23 PM Robert Liao <rob...@chromium.org> wrote:
And I think we agree that we should send all the I/O we can to a given disk and let the kernel figure it out based on priority of calling threads, so throttling I/O shouldn't be a goal?
This was before we discovered that Windows preferred sending all at once and we were discussing Grand Central Dispatch style IO.

Oh you mean async I/O all over chrome? I remember we mentioned that'd be great but beyond the scope of LL.
 

--
You received this message because you are subscribed to the Google Groups "task-scheduler-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to task-scheduler-...@chromium.org.
To post to this group, send email to task-sche...@chromium.org.

François Doray

unread,
Dec 15, 2016, 11:26:00 AM12/15/16
to Gabriel Charette, Robert Liao, task-sche...@chromium.org
Reply all
Reply to author
Forward
0 new messages