Remove sync APIs from SharedWorker

[Jake Archibald]

Was looking at how to spec which APIs we want in service worker's global scopes. The simplest thing is to take the items that are exposed on Worker. Unfortunately that gets us all the *Sync filesystem APIs and perhaps eventually the sync IDB API.

We don't want those in ServiceWorker because it's a performance vampire, but I don't think we particularly want them in SharedWorker either. If we must have sync equivalents to async methods (and I'm not entirely sure why), can we restrict them to DedicatedWorkerScope?

Suggested this over at http://lists.w3.org/Archives/Public/public-script-coord/2014JulSep/0120.html, apparently Mozilla suggested this a long time ago but we were against it, anyone know why?

[PhistucK]

Jonas wrote "showed no interest". To me, it looks like Google was not against it, but it just never responded.

(My guess is that it would have simply meant more work since the code already landed)

Of course, I never read the original thread, so I am just guessing. :)

☆PhistucK

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Ian Hickson]

On Tue, 29 Jul 2014, 'Jake Archibald' via blink-dev wrote:
>
> We don't want those in ServiceWorker because it's a performance vampire,
> but I don't think we particularly want them in SharedWorker either.

Why would you not want them in SharedWorker?

The only way you can talk to a SharedWorker is via an asynchronous API (a
MessagePort). It's black-box indetectible to the consumer how the shared
worker does its work. For example, if the author doesn't want to be
running things in parallel, the author could queue up all the requests and
service them one after the other using async APIs, or they could just
service them using sync APIs, and either way the result will be the same.
Whether this is sane or not depends on the precise situation. One can
easily imagine situations where the whole point of the shared worker is to
have all the client's activities serialised.

If you ban sync APIs in shared workers, all you're going to do is make
people spawn a dedicated worker for their shared worker, then send it all
the ports, and have it do all the work while the shared worker sits idle.

--
Ian Hickson U+1047E )\._.,--....,'``. fL
http://ln.hixie.ch/ U+263A /, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'

[Elliott Sprehn]

On Tue, Jul 29, 2014 at 11:41 AM, Ian Hickson <i...@hixie.ch> wrote:

On Tue, 29 Jul 2014, 'Jake Archibald' via blink-dev wrote:
>
> We don't want those in ServiceWorker because it's a performance vampire,
> but I don't think we particularly want them in SharedWorker either.

I agree, we should remove the sync APIs from SharedWorker (and eventually Worker too).

Why would you not want them in SharedWorker?

The only way you can talk to a SharedWorker is via an asynchronous API (a
MessagePort). It's black-box indetectible to the consumer how the shared
worker does its work. For example, if the author doesn't want to be
running things in parallel, the author could queue up all the requests and
service them one after the other using async APIs, or they could just
service them using sync APIs, and either way the result will be the same.
Whether this is sane or not depends on the precise situation. One can
easily imagine situations where the whole point of the shared worker is to
have all the client's activities serialised.

If you ban sync APIs in shared workers, all you're going to do is make
people spawn a dedicated worker for their shared worker, then send it all
the ports, and have it do all the work while the shared worker sits idle.

Worker shouldn't support sync APIs either. There should be no facility to have a hung thread on IO. It probably even makes sense to kill workers that never yield to their event loop after long periods (many seconds).

- E

[Adam Barth]

What about compute-only workers?  For example, imagine a worker that is computing all the primes.  It seems like it ought to be able to run forever computing and posting its results back to its creator.

Adam

[Ian Hickson]

Indeed. Both being able to do sync I/O and being able to just spend
forever computing data were two of the main uses cases for Workers in the
first place.

[Elliott Sprehn]

On Tue, Jul 29, 2014 at 3:05 PM, Ian Hickson <i...@hixie.ch> wrote:

On Tue, 29 Jul 2014, Adam Barth wrote:
> On Tue Jul 29 2014 at 2:58:55 PM Elliott Sprehn <esp...@chromium.org>
> wrote:
> >

> > Worker shouldn't support sync APIs either. There should be no facility
> > to have a hung thread on IO. It probably even makes sense to kill
> > workers that never yield to their event loop after long periods (many
> > seconds).
>
> What about compute-only workers?  For example, imagine a worker that is
> computing all the primes.  It seems like it ought to be able to run
> forever computing and posting its results back to its creator.

Indeed. Both being able to do sync I/O and being able to just spend
forever computing data were two of the main uses cases for Workers in the
first place.

What about when I'm done computing primes and want to ask it to politely shutdown? Or when I want to ask it to skip ahead to a known prime and start from there?

Given that you can't poll the message queue for postMessage you must periodically yield to get control messages.

You're basically saying "Why can't the parser thread in blink hang forever parsing tokens off the network.", and the answer is simple, "what if the user switched tabs and wants to start parsing a new document, or we need to interleave or pause the work?"

There's no reasonable reason to have a thread that hangs forever and cannot accept control messages.

- E

 

[Elliott Sprehn]

On Tue, Jul 29, 2014 at 3:39 PM, Elliott Sprehn <esp...@chromium.org> wrote:

On Tue, Jul 29, 2014 at 3:05 PM, Ian Hickson <i...@hixie.ch> wrote:

On Tue, 29 Jul 2014, Adam Barth wrote:
> On Tue Jul 29 2014 at 2:58:55 PM Elliott Sprehn <esp...@chromium.org>
> wrote:
> >

> > Worker shouldn't support sync APIs either. There should be no facility
> > to have a hung thread on IO. It probably even makes sense to kill
> > workers that never yield to their event loop after long periods (many
> > seconds).
>
> What about compute-only workers?  For example, imagine a worker that is
> computing all the primes.  It seems like it ought to be able to run
> forever computing and posting its results back to its creator.

Indeed. Both being able to do sync I/O and being able to just spend
forever computing data were two of the main uses cases for Workers in the
first place.

What about when I'm done computing primes and want to ask it to politely shutdown? Or when I want to ask it to skip ahead to a known prime and start from there?

Another example is that it's computing primes too fast, how do I ask it to throttle itself and slow down?

You never want a runaway thread blasting data back at you in the form of postMessage as fast as possible with no way to ask it to slow down (ex. don't send me data right now, I'm in the middle of animation). Sure you can terminate() it, but that's a huge hammer and kills it entirely. All you needed was a throttle.

- E 

[Ian Hickson]

On Tue, 29 Jul 2014, Elliott Sprehn wrote:
>
> What about when I'm done computing primes and want to ask it to politely
> shutdown? Or when I want to ask it to skip ahead to a known prime and
> start from there?

Then you write your worker to yield regularly.

Not everyone needs that. For example, maybe you spawn a few threads to do
the map part of a map-reduce, and you just want the data ASAP, and brute-
force termination is sufficient as a way to terminate the workers.

FWIW, I'm not saying you're wrong here. Maybe we shouldn't provide a way
for Web authors to write code that doesn't yield or that uses synchronous
APIs. All I'm saying is that the original use cases for Workers included
being able to do those things. That's why we designed the sync APIs to put
in workers, and that's why workers have a "terminate()" method. If we want
to change direction now, we certainly can. Send a proposal to the WHATWG
list and see what the other vendors think.

[William Chan (陈智昌)]

Let me first caveat this email by saying I don't care too much, and I defer to others. Feel free to ignore this email. It's just some ranting about my personal opinion, and I don't feel too strongly about it, despite the length of the email.

I'm going to interpret your comment as a fully generalized "all sync APIs are bad" statement. Let me know if you had an implied caveat/context that I'm missing and thus I'm arguing something that you aren't asserting. That said,

Asynchronous programming is a PITA. We do it in Chromium and Blink because, for various reasons, we _have_ to. There are many tradeoffs between sync and async programming models, but I don't agree that we should remove all sync APIs in non-critical-to-responsiveness threads (I fully support killing off with prejudice sync APIs in threads like the main ServiceWorker thread and the UI thread). There are many valid reasons to use blocking calls (e.g. it's simpler to understand), and I believe we should allow it.

Back in my server programming days, I was a proud member of the Church of Thread per Request programming using blocking calls. And if I were writing a server today, I'd probably use Go to get this done. Now, that's just my personal religious belief, and you don't have to agree, and furthermore, I don't care because well, that's just, like, your opinion, man. But I think the web platform should allow me to practice my personal religious beliefs, at least to the degree they're not absurdly dangerous nuclear weapons like sync XHR on the UI thread.

To your example of not wanting a runaway thread to be able to post an unlimited number of messages...you know what? Chromium allows this today, and it's just fine. When a runaway thread does this, it's a bug and we fix it and maybe we force the developer at fault to wear the Cone of Shame. But we don't disallow all sync APIs in all our threads, only on the jank critical threads. And I support adding APIs in the web platform to do something like we do with ThreadRestrictions, or Android does with StrictMode to allow developers to restrict sync APIs on their Workers if desired.

Anyway, that's just my two cents. Which is admittedly not worth very much. Cheers.

- E 

[Jake Archibald]

On Tue, Jul 29, 2014 at 7:41 PM, Ian Hickson <i...@hixie.ch> wrote:

On Tue, 29 Jul 2014, 'Jake Archibald' via blink-dev wrote:
>
> We don't want those in ServiceWorker because it's a performance vampire,
> but I don't think we particularly want them in SharedWorker either.

Why would you not want them in SharedWorker?

If you're doing sync things within your SharedWorker you're going to block messages coming into the SharedWorker. If it's desirable to handle the entirety of each message & potential response in series, promises make that really simple:

var queue = Promise.resolve();

this.onconnect = function(event) {

  var port = event.ports[0];

  port.onmessage = function() {

    queue = queue.then(function() {

      // do stuff

    }).catch(function(err) {

      // handle errors

    });

  }

};

In reality you'd use the queue only for things that need to be serial & go parallel for the rest.

[Jake Archibald]

On Wed, Jul 30, 2014 at 5:14 AM, William Chan (陈智昌) <will...@chromium.org> wrote:

Asynchronous programming is a PITA.

Then we should fix that, and ES7 async functions go a long way.

var response = fetchSync(url);

var json = response.body.asJSONSync();

…is simple but blocking.

(async function() {

  var response = await fetch(url);

  var json = await response.body.asJSON();

}());

…is simple & non-blocking.

I can see the case for heavy sync computation in dedicated workers, although I don't think we need sync filesystem APIs there to aid that, and as Elliot points out you need to make sure you're yielding regularly.

[James Robinson]

No matter how much you sugar it asynchronous programming is significantly harder to understand and debug in some situations than synchronous straight-line code.  We can't write straight-line code for the most part in the web platform since code runs on what is effectively the UI thread, but that doesn't mean the concept is fundamentally sound.  In environments and languages where synchronous calls do not block the UI it's a great model.  Web workers are the closest thing the web platform has to such an environment.

If the purpose of a worker is to perform file I/O and to do nothing else, there's no need to for it to yield regularly for other operations.  What else could it do other than the task it is dedicated to?

- James

[Anne van Kesteren]

On Wed, Jul 30, 2014 at 8:27 PM, 'James Robinson' via blink-dev
<blin...@chromium.org> wrote:
> If the purpose of a worker is to perform file I/O and to do nothing else,
> there's no need to for it to yield regularly for other operations. What
> else could it do other than the task it is dedicated to?

End user changes his mind and wants to load a different file. Does not
seem great he first has to wait for a file he doesn't want to load.


--
http://annevankesteren.nl/

[Elliott Sprehn]

On Wed, Jul 30, 2014 at 11:34 AM, Anne van Kesteren <ann...@annevk.nl> wrote:

On Wed, Jul 30, 2014 at 8:27 PM, 'James Robinson' via blink-dev
<blin...@chromium.org> wrote:
> If the purpose of a worker is to perform file I/O and to do nothing else,
> there's no need to for it to yield regularly for other operations.  What
> else could it do other than the task it is dedicated to?

It could abort that task at a sane place when the user cancels the job. If you're writing out data to disk calling terminate() might leave the resulting files or app's databases in a corrupted state. Instead if you're reacting to control messages you can abort at a sensible place.

End user changes his mind and wants to load a different file. Does not
seem great he first has to wait for a file he doesn't want to load.

Indeed. Or what if the file is on a samba share that just went offline and it blocks for many minutes now? All file operations are now blocked because your IO thread is hung. The model where the IO thread can block like this is the reason that your linux machine completely locks up whenever a file share becomes unavailable.

I don't think there's ever a justification for blocking and not responding to any kind of control messages. That's not how you build reliable software.

See also:

http://docs.oracle.com/javase/1.5.0/docs/guide/misc/threadPrimitiveDeprecation.html

If we're serious about allowing the hung threads programming model we need to add Worker.interrupt(), InterruptException, and WorkerGlobalScope#isInterrupted() so you can at least do something sensible.

- E

[William Chan (陈智昌)]

I don't understand why people are contriving more complicated use cases. Why is it unacceptable to say "I don't care about that scenario"? Why must my application care about people who use samba shares? I just want to serve 5TB.

[Ian Hickson]

On Wed, 30 Jul 2014, 'Jake Archibald' via blink-dev wrote:

> On Tue, Jul 29, 2014 at 7:41 PM, Ian Hickson <i...@hixie.ch> wrote:
> > On Tue, 29 Jul 2014, 'Jake Archibald' via blink-dev wrote:
> > >
> > > We don't want those in ServiceWorker because it's a performance

> > > vampire, but I don't think we particularly want them in SharedWorker
> > > either.
> >

> > Why would you not want them in SharedWorker?
>

> If you're doing sync things within your SharedWorker you're going to
> block messages coming into the SharedWorker.

Yes. For many shared workers, that's _the entire point_. For example, if
the point of your shared worker is to have a defined serialised order of
operations on a client-side database, so that the underlying data is
guaranteed to remain consistent.

> If it's desirable to handle the entirety of each message & potential
> response in series, promises make that really simple:
>
> var queue = Promise.resolve();
>
> this.onconnect = function(event) {
> var port = event.ports[0];
> port.onmessage = function() {
> queue = queue.then(function() {
> // do stuff
> }).catch(function(err) {
> // handle errors
> });
> }
> };

No offense, but your idea of "really simple" is not the same as mine. IMHO
the code above is borderline unreadable. Compare it to:

this.onconnect = function(event) {
var port = event.ports[0];
port.onmessage = function() {

try {
// do stuff
} catch (err) {
// handle errors
}
};

}


On Wed, 30 Jul 2014, Jake Archibald wrote:
>
> var response = fetchSync(url);
> var json = response.body.asJSONSync();
>
> …is simple but blocking.
>
> (async function() {
> var response = await fetch(url);
> var json = await response.body.asJSON();
> }());
>
> …is simple & non-blocking.

That's not "simple". It's not at all obvious what "await" does or what
"async function" means. We also know that authors have huge trouble
understanding lambdas in general, especially lambdas that are immediately
invoked like this.

> I can see the case for heavy sync computation in dedicated workers,
> although I don't think we need sync filesystem APIs there to aid that,
> and as Elliot points out you need to make sure you're yielding
> regularly.

You don't necessarily ever need to yield.

On Wed, 30 Jul 2014, James Robinson wrote:
>
> No matter how much you sugar it asynchronous programming is
> significantly harder to understand and debug in some situations than
> synchronous straight-line code.

Hear hear.

> If the purpose of a worker is to perform file I/O and to do nothing
> else, there's no need to for it to yield regularly for other operations.
> What else could it do other than the task it is dedicated to?

Exactly.

On Wed, 30 Jul 2014, Anne van Kesteren wrote:
>
> End user changes his mind and wants to load a different file.

What if it's not possible for the end user to want a different file? For
example, the file is a game's high score table.

On Wed, 30 Jul 2014, Elliott Sprehn wrote:
>
> Indeed. Or what if the file is on a samba share that just went offline
> and it blocks for many minutes now? All file operations are now blocked
> because your IO thread is hung. The model where the IO thread can block
> like this is the reason that your linux machine completely locks up
> whenever a file share becomes unavailable.

That's why we shouldn't have sync I/O on the UI thread, sure. But if your
I/O is all from the same NFS-backed database, and one call blocks, all
subsequent calls are going to block too, so async doesn't help.

> If we're serious about allowing the hung threads programming model we
> need to add Worker.interrupt(), InterruptException, and
> WorkerGlobalScope#isInterrupted() so you can at least do something
> sensible.

I think a message peeking mechanism would be better than an interrupt, but
sure, adding features to make this kind of thing more powerful is
definitely something we should consider. I recommend bringing up such
proposals on wha...@whatwg.org or filing them at: http://whatwg.org/newbug

[Elliott Sprehn]

On Wed, Jul 30, 2014 at 12:36 PM, William Chan (陈智昌) <will...@chromium.org> wrote:

I don't understand why people are contriving more complicated use cases. Why is it unacceptable to say "I don't care about that scenario"?

I don't care about security scenarios either [1], yet you're forcing me to use https to use service workers. This is no different, and it's extremely frustrating that we're willing to impose all kinds of restrictions based on security but not willing to keep you from writing code that's unreliable and prone to data corruption.

Why must my application care about people who use samba shares? I just want to serve 5TB.

In the real world, in reliable software, these are things you need to think about.

[1] Not really.

[James Robinson]

Chrome works by doing synchronous IO on a set of dedicated threads.  Are you saying that chrome is unreliable software, or that the things Chrome is doing are things that you don't think the web platform should allow?

[Elliott Sprehn]

On Wed, Jul 30, 2014 at 1:25 PM, James Robinson <jam...@google.com> wrote:

Chrome works by doing synchronous IO on a set of dedicated threads.  Are you saying that chrome is unreliable software, or that the things Chrome is doing are things that you don't think the web platform should allow?

What does Chrome do if I quit while it's in the middle of saving my profile data?

[James Robinson]

On Wed, Jul 30, 2014 at 1:31 PM, Elliott Sprehn <esp...@chromium.org> wrote:

On Wed, Jul 30, 2014 at 1:25 PM, James Robinson <jam...@google.com> wrote:

Chrome works by doing synchronous IO on a set of dedicated threads.  Are you saying that chrome is unreliable software, or that the things Chrome is doing are things that you don't think the web platform should allow?

What does Chrome do if I quit while it's in the middle of saving my profile data?

Chrome attempts to write the profile out to disk on shutdown blocking exit of the process until it completes.  There are ways this can fail (I'm sure you've seen the infobar stating that Chrome detected it did not shut down cleanly on the last run) but this seems to be getting pretty off-topic.

- James

[William Chan (陈智昌)]

Ninja uses blocking file writes: https://github.com/martine/ninja/blob/master/src/disk_interface.cc#L197.

GNU cat uses blocking file I/O too: http://git.savannah.gnu.org/cgit/coreutils.git/tree/src/cat.c#n164

I'm satisfied with their unreliability.

On Wed, Jul 30, 2014 at 1:25 PM, James Robinson <jam...@google.com> wrote:

[Elliott Sprehn]

I don't think that's off topic. The worker has no way to block termination, so if you need to ask it to stop there's nothing you can do except terminate() and have corrupted data.

Also does Chrome's IO thread have a way to ask it to stop writing or do we kill it? Specifically if it's writing a 100MB file to a slow disk and then we want to it to stop, do we pthread_kill() the IO thread?

- E 

[Matthew Menke]

Despite its name, Chrome's IO thread does not, in general, do synchronous IO.  When it's blocked, we can't talk to the network, or even talk to renderers, since it's used for IPCs as well.  The FILE thread, on the other hand...

[Jonas Sicking]

On Wed, Jul 30, 2014 at 11:27 AM, 'James Robinson' via blink-dev
<blin...@chromium.org> wrote:
> No matter how much you sugar it asynchronous programming is significantly
> harder to understand and debug in some situations than synchronous
> straight-line code. We can't write straight-line code for the most part in
> the web platform since code runs on what is effectively the UI thread, but
> that doesn't mean the concept is fundamentally sound. In environments and
> languages where synchronous calls do not block the UI it's a great model.
> Web workers are the closest thing the web platform has to such an
> environment.
>
> If the purpose of a worker is to perform file I/O and to do nothing else,
> there's no need to for it to yield regularly for other operations. What
> else could it do other than the task it is dedicated to?

Note that while this thread has strayed to cover various philosophical
topics, the original proposal here is not to remove all sync APIs from
all workers.

The proposal is just to remove sync APIs from SharedWorkers. Dedicated
Workers will continue to have access to sync XHR, FileReaderSync as
well as various Blink-specific APIs (mainly filesystem I believe).

I think it's generally agreed these days that doing sync IO from the
UI thread is a bad idea. SharedWorkers share *some* of the same
problems that sync IO on the UI thread has. This is because
SharedWorkers are intended to do work for several windows. So if a
SharedWorker uses sync IO to service a request from one particular
Window, then it means that it will remain unresponsive to service
requests from other Windows.

I'm personally undecided on the debate if sync IO offer inherent
ergonomic benefits, or if we can get async programming to be pleasant
enough using syntax sugar in JS as well as APIs like Promises and
async iterators.

When it comes to IO, it sounds like Gecko and Blink often do the same
thing. Use the main thread to kick off IO work. But do the actual IO a
background thread using synchronous IO APIs which keeps the code clean
and easy to understand.

However it's definitely the case that once we add an API to the web
platform, it's *really* hard to get rid of. This is especially true
for capabilities that can't be polyfilled or that is very convenient
for developers to use. Sync IO falls into both those categories.

So if we decide to keep sync IO in SharedWorkers now, it's likely to
be there forever, even if we a couple of years from now realize that
it does more bad than good.

So my preferred solution is to experiment with sync APIs in dedicated
Workers, but leave them out of other contexts. This way we can improve
the ergonomics of async APIs, while still allowing webdevelopers to do
what both Gecko and Blink do: kick off worker threads which do the
heavy lifting. Including using sync IO in those worker threads if they
feel like that gives the best performance/ergonomics tradeoffs.

Part of our strategy towards that goal in Gecko has been to make
creating dedicated Workers as cheap of an operation as we can. They
always run in-process with the page/worker that created them, and the
JS-engine team has worked on ensuring that setting up the Worker JS
environment is as cheap as possible.

/ Jonas

[Jonas Sicking]

On Wed, Jul 30, 2014 at 1:51 PM, Elliott Sprehn <esp...@chromium.org> wrote:
> On Wed, Jul 30, 2014 at 1:42 PM, James Robinson <jam...@google.com> wrote:
>>
>> On Wed, Jul 30, 2014 at 1:31 PM, Elliott Sprehn <esp...@chromium.org>
>> wrote:
>>>
>>> On Wed, Jul 30, 2014 at 1:25 PM, James Robinson <jam...@google.com>
>>> wrote:
>>>>
>>>> Chrome works by doing synchronous IO on a set of dedicated threads. Are
>>>> you saying that chrome is unreliable software, or that the things Chrome is
>>>> doing are things that you don't think the web platform should allow?
>>>>
>>>
>>> What does Chrome do if I quit while it's in the middle of saving my
>>> profile data?
>>
>>
>> Chrome attempts to write the profile out to disk on shutdown blocking exit
>> of the process until it completes. There are ways this can fail (I'm sure
>> you've seen the infobar stating that Chrome detected it did not shut down
>> cleanly on the last run) but this seems to be getting pretty off-topic.
>>
>
> I don't think that's off topic. The worker has no way to block termination,
> so if you need to ask it to stop there's nothing you can do except
> terminate() and have corrupted data.

As part of creating an improved FileSystem API, we should definitely
look at adding support for atomic writes. I.e. writes where either the
entire write succeeds, or where none of the write happens.

The way this is normally done in other platforms is to write a new
file somewhere on the same disk, then rename the new file to the
existing file's filename.

Both the existing filesystem API proposals enable this, but require
that developers do all the work themselves. Which is hard both because
it's more lines of code, and it's hard because it's non-obvious that
that's how to do an atomic write.

Adding something like Directory.atomicWrite(filename, blob) seems like
a very useful thing.

/ Jonas

[Victor Costan]

On Mon, Aug 4, 2014 at 8:52 PM, Jonas Sicking <jo...@sicking.cc> wrote:

Both the existing filesystem API proposals enable this, but require

that developers do all the work themselves. Which is hard both because
it's more lines of code, and it's hard because it's non-obvious that
that's how to do an atomic write.

Adding something like Directory.atomicWrite(filename, blob) seems like
a very useful thing.

Wouldn't it be better to have writes be atomic by default, and have a way to opt-out when performance demands it? Premature optimization being the root of all evil and all that...

 

[William Chan (陈智昌)]

I'm favorably inclined towards this suggestion.

[Jonas Sicking]

We could make Directory.createFile atomic. But making partial writes
(i.e. FileHandleWritable.write) atomic would be incredibly expensive.
Writing 10 bytes in the middle of a 10MB file would cause 20MB of file
IO. I don't think it's a premature optimization to avoid that.

But I think this is getting off-topic.

/ Jonas

[Darin Fisher]

Big fat +1

[Darin Fisher]

By the way, I think one of the concerns I've heard expressed about synchronous APIs is that they are different than their asynchronous counterparts. One way to address that concern is to only have asynchronous APIs but marry them with an API to wait on a Promise to be satisfied. Such an API would only be available on select threads. Then, folks could write "friendly" fread/fwrite style APIs as polyfills.

-Darin

[Elliott Sprehn]

On Thu, Aug 7, 2014 at 2:40 PM, Darin Fisher <da...@chromium.org> wrote:

By the way, I think one of the concerns I've heard expressed about synchronous APIs is that they are different than their asynchronous counterparts. One way to address that concern is to only have asynchronous APIs but marry them with an API to wait on a Promise to be satisfied. Such an API would only be available on select threads. Then, folks could write "friendly" fread/fwrite style APIs as polyfills.

+1, that sounds like a great idea. Can we do that with "await"?

function* fwrite(fd, buff) {

  var writer = new FileWriter(fd);

  await writer.write(buff);

  return;

}

[Jake Archibald]

Await only gives you the appearance of synchronicity within the async function itself.

Although I don't see why can't have a blockUntil method that behaves in a similar way:

var val = blockUntil(promise);

…which returns the resolved value or throws the rejection value.

[William Chan (陈智昌)]

Just for clarity's sake, what does the proposed blockUntil(promise) do? Specifically, does it pump the event loop?

[Darin Fisher]

No, I would hope not. Otherwise, it will not be a sufficient replacement for existing synchronous APIs.

-Darin

[Jonas Sicking]

On Thu, Aug 7, 2014 at 2:40 PM, Darin Fisher <da...@chromium.org> wrote:
> By the way, I think one of the concerns I've heard expressed about synchronous APIs is that they are different than their asynchronous counterparts. One way to address that concern is to only have asynchronous APIs but marry them with an API to wait on a Promise to be satisfied. Such an API would only be available on select threads. Then, folks could write "friendly" fread/fwrite style APIs as polyfills.

I did some thinking about this some time back. I think it's a super
interesting idea. Back before we had Promises it unfortunately seemed
impossibly hard both to spec and implement, but it's possible that
Promises changes things enough that it's doable.

One goal that I'd like to stick to is to make it at least "very hard"
to create a deadlock or race.

If we simply introduce a blockUntil(promise) function, it becomes very
easy to create a deadlock. For example start two IDB write
transactions against the same objectStore, then block until the second
one finishes. The first transaction won't ever commit since it'll wait
to dispatch success events before committing, but since the thread is
blocked it'll wait forever. Same thing with the filesystem API is we
add support for atomic read-modify-write operations.

Or have two worker threads send messages to each other and block on
getting a response.

Part of the solution here could be to have a BlockablePromise subclass
which is returned from various APIs and only allow blockUntil() to
work with BlockablePromise. We could even allow
BlockablePromise.race() and BlockablePromise.all() which would return
a new BlockablePromise, but fail if it was passed anything but
BlockablePromises.

However that still wouldn't solve the IDB/filesystem problem above
since presumably we'd want to make IDB/filesystem generally return
BlockablePromises.

/ Jonas

[Darin Fisher]

Hmm, I'm not so sure we need to worry about dead locks. We just need to ensure that the blocking calls get interrupted when the worker threads should shutdown. Developer tools can help developers understand when they made mistakes.

-Darin

[Jonas Sicking]

On Tue, Aug 12, 2014 at 11:02 AM, Darin Fisher <da...@chromium.org> wrote:
> Hmm, I'm not so sure we need to worry about dead locks. We just need to
> ensure that the blocking calls get interrupted when the worker threads
> should shutdown. Developer tools can help developers understand when they
> made mistakes.

This sounds a bit optimistic to me given how subtle the issue is, and
how hard it is to debug given that it's an intermittent condition.

But I'm definitely not sure. However I don't think we need to come to
an agreement about this in this thead.

The question at hand is if we should keep APIs like "sync XHR", "sync
FileReader" and "sync Filesystem" in shared workers, or just have them
in dedicated workers for now.

/ Jonas

[Darin Fisher]

On Tue, Aug 12, 2014 at 2:24 PM, Jonas Sicking <jo...@sicking.cc> wrote:

On Tue, Aug 12, 2014 at 11:02 AM, Darin Fisher <da...@chromium.org> wrote:
> Hmm, I'm not so sure we need to worry about dead locks. We just need to
> ensure that the blocking calls get interrupted when the worker threads
> should shutdown. Developer tools can help developers understand when they
> made mistakes.

This sounds a bit optimistic to me given how subtle the issue is, and
how hard it is to debug given that it's an intermittent condition.

This is the solution we use for child processes in Chrome. Synchronous IPCs are all incorruptible by shutdown events. I think it has proven fairly robust and simple.

 

But I'm definitely not sure. However I don't think we need to come to
an agreement about this in this thead.

The question at hand is if we should keep APIs like "sync XHR", "sync
FileReader" and "sync Filesystem" in shared workers, or just have them
in dedicated workers for now.

/ Jonas

I would support getting rid of all synchronous APIs and replacing them with this BlockUntil(promise) API. I don't see a problem with exposing this API to shared workers. If developers wish to misuse it, then I don't see why we should prohibit them from doing so.

I don't think we should withhold capabilities from developers out of fear that they will be misused. Rather we should empower framework developers to build great frameworks that guide most developers to use the platform properly.

(There are exceptions of course. It is unfortunate that the main thread is shared by multiple applications, so one misbehaving application can negatively impact others.)

-Darin

[Jonas Sicking]

On Tue, Aug 12, 2014 at 8:46 PM, Darin Fisher <da...@chromium.org> wrote:
>
>
>
> On Tue, Aug 12, 2014 at 2:24 PM, Jonas Sicking <jo...@sicking.cc> wrote:
>>
>> On Tue, Aug 12, 2014 at 11:02 AM, Darin Fisher <da...@chromium.org> wrote:
>> > Hmm, I'm not so sure we need to worry about dead locks. We just need to
>> > ensure that the blocking calls get interrupted when the worker threads
>> > should shutdown. Developer tools can help developers understand when
>> > they
>> > made mistakes.
>>
>> This sounds a bit optimistic to me given how subtle the issue is, and
>> how hard it is to debug given that it's an intermittent condition.
>
>
> This is the solution we use for child processes in Chrome. Synchronous IPCs
> are all incorruptible by shutdown events. I think it has proven fairly
> robust and simple.
>
>
>>
>>
>> But I'm definitely not sure. However I don't think we need to come to
>> an agreement about this in this thead.
>>
>> The question at hand is if we should keep APIs like "sync XHR", "sync
>> FileReader" and "sync Filesystem" in shared workers, or just have them
>> in dedicated workers for now.
>>
>> / Jonas
>
>
> I would support getting rid of all synchronous APIs and replacing them with
> this BlockUntil(promise) API. I don't see a problem with exposing this API
> to shared workers. If developers wish to misuse it, then I don't see why we
> should prohibit them from doing so.

I'm not sure that getting rid of existing sync APIs in dedicated
workers is going to be web compatible at this point.

Given how little support there is for Shared Workers (only Chrome and
Opera until very recently when Firefox also got support) I think we
have more freedom to make changes there.

So I think the question is if we can get rid of these APIs now, or if
we should wait until we have blockUntil (or something like it) in
place. The risk of waiting is of course that dependency on these APIs
might increase.

/ Jonas

[Darin Fisher]

Sure, I didn't mean to suggest it would be. I just meant that I would support

trying to do so ;-)

 

Given how little support there is for Shared Workers (only Chrome and
Opera until very recently when Firefox also got support) I think we
have more freedom to make changes there.

Chrome may not. We'll see.

 

So I think the question is if we can get rid of these APIs now, or if
we should wait until we have blockUntil (or something like it) in
place. The risk of waiting is of course that dependency on these APIs
might increase.

I think we can certainly begin exploring deprecation. We'd need to

understand current usage, etc.

The idea with blockUntil is that we could use that as the solution for

future desires for synchronous APIs on workers. On the compat

front, if need be, blockUntil could also help us polyfill existing

synchronous APIs. On the Chrome side we are developing tech

that would allow us to implement various APIs entirely in JS on top

of the rest of the platform, when that makes sense. This could be

such a candidate.

-Darin