Intent to Implement and Ship: ES Modules for dedicated workers (‘module’ type option)

[Hiroki Nakagawa]

Contact emails

Impl: nhi...@chromium.org, kou...@chromium.org, hiro...@chromium.org

Spec: dom...@chromium.org

Explainer / Summary

‘module’ type in WorkerOptions enables to load module scripts into workers.

const worker = new Worker(‘worker.js’, { type: ‘module’ });

A worker constructed with this option can use “static” import, “dynamic” import(), and “export” statements.

// worker-library.js

export function doSomething() { … };

export const answer = 42;

// worker.js

import * as module from “./worker-library.js”

module.doSomething();

console.log(module.answer);  // 42

Spec-wise, this type option is available for any worker types (dedicated, shared, and service workers), but this Intent-to-Ship targets only dedicated workers for now.

Spec

‘module’ type option:

https://html.spec.whatwg.org/multipage/workers.html#workeroptions

Module script loading (see the step 13):

https://html.spec.whatwg.org/multipage/workers.html#worker-processing-model

Link to Origin Trial feedback summary

N/A

Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.

Demo link

N/A

Debuggability

The DevTools supports module scripts on dedicated workers. Developers can debug and inspect them like classic scripts.

Risks

Interoperability and Compatibility

Low.

• Module scripts via <script> tag have already been shipped in major browsers.

• The module worker spec has been discussed with other browser vendors and reached consensus.

• There is a plenty of web-platform-tests for module scripts via <script> tag and module workers.

Edge: Positive (showed an interest in the initial spec PR)

Firefox: Positive (have been involved in discussions since the initial spec PR. impl issue)

Safari: No signals (impl issue)

Web developers: Positive (starred by 34 users as of June 14, 2018)

Ergonomics

> Are there any other platform APIs this feature will frequently be used in tandem with?

> Could the default usage of this API make it hard for Chrome to maintain good performance (i.e. synchronous return, must run on a certain thread, guaranteed return timing)?

No.

Activation

Easy: Developers just need to write module scripts and create a dedicated worker with the type option. Also, they can reuse existing module scripts used for <script> tag.

Is this feature fully tested by web-platform-tests? Link to test suite results from wpt.fyi.

Yes (https://wpt.fyi/results/workers/modules)

Entry on the feature dashboard

https://www.chromestatus.com/feature/5761300827209728

[Ben Kelly]

On Fri, Jun 15, 2018 at 1:22 AM, Hiroki Nakagawa <nhi...@chromium.org> wrote:

Risks

Interoperability and Compatibility

Low.

• Module scripts via <script> tag have already been shipped in major browsers.

• The module worker spec has been discussed with other browser vendors and reached consensus.

• There is a plenty of web-platform-tests for module scripts via <script> tag and module workers.

Please be aware that there are currently some interop issues in chrome's dedicated worker support that it would be nice to note to bake into a new design/implementation.

In particular, last I checked chrome dedicated worker script loads are still made as subresource loads from their parent document.  This is has been noted by web devs as an incompatibility:

https://bugs.chromium.org/p/chromium/issues/detail?id=828777 (which was duped to:)

https://bugs.chromium.org/p/chromium/issues/detail?id=731604

I guess I have some concerns that if chrome ships worker module with this same broken behavior as the sole initial implementation that developers will begin depending on it.

I think there are also issues with dedicated workers in not reflecting the final URL in `self.location` when the script is redirected.  I don't have an issue for that handy, though.

Thanks.

Ben

[Ben Kelly]

On Fri, Jun 15, 2018 at 9:44 AM, Ben Kelly <bke...@mozilla.com> wrote:

I think there are also issues with dedicated workers in not reflecting the final URL in `self.location` when the script is redirected.  I don't have an issue for that handy, though.

The worker URL-after-redirect might be a wider issue.  I filed a spec issue to discuss:

https://github.com/whatwg/html/issues/3760

Sorry if this is a tangent for this thread.  It just feels there are some interop loose ends in workers at the moment that could be more problematic if adoption accelerates with modules, etc.

Ben

[Hiroki Nakagawa]

Hi Ben,

Thank you for your feedback.

2018-06-15 22:44 GMT+09:00 Ben Kelly <bke...@mozilla.com>:

On Fri, Jun 15, 2018 at 1:22 AM, Hiroki Nakagawa <nhi...@chromium.org> wrote:

Risks

Interoperability and Compatibility

Low.

• Module scripts via <script> tag have already been shipped in major browsers.

• The module worker spec has been discussed with other browser vendors and reached consensus.

• There is a plenty of web-platform-tests for module scripts via <script> tag and module workers.

Please be aware that there are currently some interop issues in chrome's dedicated worker support that it would be nice to note to bake into a new design/implementation.

In particular, last I checked chrome dedicated worker script loads are still made as subresource loads from their parent document.  This is has been noted by web devs as an incompatibility:

https://bugs.chromium.org/p/chromium/issues/detail?id=828777 (which was duped to:)

https://bugs.chromium.org/p/chromium/issues/detail?id=731604

Yeah, this is on our TODO list. This has been blocked on service worker servicification as it will change how service worker intercepts requests.

 

I guess I have some concerns that if chrome ships worker module with this same broken behavior as the sole initial implementation that developers will begin depending on it.

Your concerns are legitimate. We intend to fix it as soon as the servicification is complete, but probably it will take a little more time before we can really start writing the CL… How much do you feel this is blocking the module worker?

 

I think there are also issues with dedicated workers in not reflecting the final URL in `self.location` when the script is redirected.  I don't have an issue for that handy, though.

Thanks.

Ben

Thank you for filing the spec issue. We’re going to fix the both redirect case and service worker case before enabling the module worker.

 

Ben

[Ben Kelly]

On Mon, Jun 18, 2018 at 9:08 AM, Hiroki Nakagawa <nhi...@chromium.org> wrote:

2018-06-15 22:44 GMT+09:00 Ben Kelly <bke...@mozilla.com>:

https://bugs.chromium.org/p/chromium/issues/detail?id=828777 (which was duped to:)

https://bugs.chromium.org/p/chromium/issues/detail?id=731604

Yeah, this is on our TODO list. This has been blocked on service worker servicification as it will change how service worker intercepts requests.

 

I guess I have some concerns that if chrome ships worker module with this same broken behavior as the sole initial implementation that developers will begin depending on it.

Your concerns are legitimate. We intend to fix it as soon as the servicification is complete, but probably it will take a little more time before we can really start writing the CL… How much do you feel this is blocking the module worker?

Its difficult for me to say exactly how blocking this should be.  I can clarify my concerns so your internal review can consider them better, though:

1. As I mentioned in my previous email the dedicated worker loading as a subresource of the owning document has already been an interop issue.  The autocad folks ran into it and it could have been a high profile case of bad interop.  Its hard to tie this directly to module support, but I think the potential to diverge increases with such a large feature change added on top of existing incompatible loading behavior.

2. There has been back-and-forth at the spec level about whether dedicated workers should be subresource requests or not.  In particular I feel like this came up a lot when worker modules were spec'd.  I worry that if modules are added on top of this behavior it might allow problems in the spec to go unnoticed longer.  In my experience a lot of spec issues get found and sorted out during implementation.

Would it be possible to write a WPT test that verifies what we think is the correct behavior as part of your effort?  That at least can be reviewed and show where the current implementation diverges.

Also, would it be possible to test against the s13n configuration of service workers to see if the behavior is correct there before shipping modules?

Not sure what other risk mitigations are possible.

 

https://github.com/whatwg/html/issues/3760

Thank you for filing the spec issue. We’re going to fix the both redirect case and service worker case before enabling the module worker.

Thanks!

Ben

[Hiroki Nakagawa]

2018-06-19 6:09 GMT+09:00 Ben Kelly <bke...@mozilla.com>:

On Mon, Jun 18, 2018 at 9:08 AM, Hiroki Nakagawa <nhi...@chromium.org> wrote:

2018-06-15 22:44 GMT+09:00 Ben Kelly <bke...@mozilla.com>:

https://bugs.chromium.org/p/chromium/issues/detail?id=828777 (which was duped to:)

https://bugs.chromium.org/p/chromium/issues/detail?id=731604

Yeah, this is on our TODO list. This has been blocked on service worker servicification as it will change how service worker intercepts requests.

 

I guess I have some concerns that if chrome ships worker module with this same broken behavior as the sole initial implementation that developers will begin depending on it.

Your concerns are legitimate. We intend to fix it as soon as the servicification is complete, but probably it will take a little more time before we can really start writing the CL… How much do you feel this is blocking the module worker?

Its difficult for me to say exactly how blocking this should be.  I can clarify my concerns so your internal review can consider them better, though:

1. As I mentioned in my previous email the dedicated worker loading as a subresource of the owning document has already been an interop issue.  The autocad folks ran into it and it could have been a high profile case of bad interop.  Its hard to tie this directly to module support, but I think the potential to diverge increases with such a large feature change added on top of existing incompatible loading behavior.

2. There has been back-and-forth at the spec level about whether dedicated workers should be subresource requests or not.  In particular I feel like this came up a lot when worker modules were spec'd.  I worry that if modules are added on top of this behavior it might allow problems in the spec to go unnoticed longer.  In my experience a lot of spec issues get found and sorted out during implementation.

Thank you for the clarification. That's helpful in coordinating the issues. I'll send updates to this thread once a consensus is built in the worker/loading team.

 

Would it be possible to write a WPT test that verifies what we think is the correct behavior as part of your effort?  That at least can be reviewed and show where the current implementation diverges.

There're several WPTs for service worker interception for dedicated workers:

- https://wpt.fyi/results/service-workers/service-worker/dedicated-worker-service-worker-interception.https.html

- https://wpt.fyi/results/service-workers/service-worker/worker-interception.https.html

- https://wpt.fyi/results/service-workers/service-worker/worker-interception-redirect.https.html

Also, there're LayoutTests that check Chrome's behavior deviated from the spec:

- https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/http/tests/serviceworker/dedicated-worker-service-worker-interception.html

 

Also, would it be possible to test against the s13n configuration of service workers to see if the behavior is correct there before shipping modules?

Yes, it's feasible.

[Ben Kelly]

On Tue, Jun 19, 2018 at 9:45 AM, Hiroki Nakagawa <nhi...@chromium.org> wrote:

2018-06-19 6:09 GMT+09:00 Ben Kelly <bke...@mozilla.com>:

Would it be possible to write a WPT test that verifies what we think is the correct behavior as part of your effort?  That at least can be reviewed and show where the current implementation diverges.

There're several WPTs for service worker interception for dedicated workers:

- https://wpt.fyi/results/service-workers/service-worker/dedicated-worker-service-worker-interception.https.html

- https://wpt.fyi/results/service-workers/service-worker/worker-interception.https.html

- https://wpt.fyi/results/service-workers/service-worker/worker-interception-redirect.https.html

Excellent.  Thank you!

Ben

[Chris Harrelson]

LGTM1

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2B1UsbRTMcKg710mYYx4MC%3Dj_7%2BZdJ%2BavZ5mGbYkBKSR%3DepmPQ%40mail.gmail.com.

[Rick Byers]

The API owners discussed this a bit today. We're interested to better understand the trade-off of blocking this feature on the interop issues Ben raised.

Ben, are you satisfied that the web platform tests adequately cover the interop issues you're worried about? Or do you feel more investment in the tests is needed to reflect the scope of the real-world compat problems you're worried about? The anecdoate of Autocad hitting the issue and having to do work to avoid a significant compat problem is compelling to me - thanks for weighing in!

Hiroki-san, do you have a rough idea of when network servicification will be sufficiently complete to be able to validate that the issue is fixed (relevant WPTs pass)? I'm trying to understand what Ben's proposal to block shipping this feature on fixing the issue would cost us practice. Obviously people are definitely anxious for module support in workers.

Thanks,

   Rick

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOMQ%2Bw-Oh5attnE6kspA2UWcSjJkLzrsqUqjdErEhCWgtcTqow%40mail.gmail.com.

[Ben Kelly]

On Thu, Jun 21, 2018 at 1:24 PM, Rick Byers <rby...@chromium.org> wrote:

The API owners discussed this a bit today. We're interested to better understand the trade-off of blocking this feature on the interop issues Ben raised.

Ben, are you satisfied that the web platform tests adequately cover the interop issues you're worried about? Or do you feel more investment in the tests is needed to reflect the scope of the real-world compat problems you're worried about? The anecdoate of Autocad hitting the issue and having to do work to avoid a significant compat problem is compelling to me - thanks for weighing in!

Yes and maybe. :-)

I think its clear the implementation team is aware of the problem and won't code themselves into a corner.  So that is good.

The real unknown, though, is how many sites will adopt module workers and also want to do fancy things with service worker clients.  They seem more likely to inadvertently become dependent on the loading behavior.  I guess if chromium is willing to ship the service worker fix even if it breaks these sites, then that is ok.  Obviously the sooner the service worker fix can ship the better.  (Is there any estimate for that?)

Thanks for taking this situation into consideration.

Ben

[Matt Falkenhagen]

On Fri, Jun 22, 2018 at 2:24 AM, Rick Byers <rby...@chromium.org> wrote:

The API owners discussed this a bit today. We're interested to better understand the trade-off of blocking this feature on the interop issues Ben raised.

Ben, are you satisfied that the web platform tests adequately cover the interop issues you're worried about? Or do you feel more investment in the tests is needed to reflect the scope of the real-world compat problems you're worried about? The anecdoate of Autocad hitting the issue and having to do work to avoid a significant compat problem is compelling to me - thanks for weighing in!

Hiroki-san, do you have a rough idea of when network servicification will be sufficiently complete to be able to validate that the issue is fixed (relevant WPTs pass)? I'm trying to understand what Ben's proposal to block shipping this feature on fixing the issue would cost us practice. Obviously people are definitely anxious for module support in workers.

We intend to fix service worker interception for dedicated workers. However, service worker interception is changing a lot with the network servification work, so to avoid complications I’ve been reluctant to fix it until after that work is done.

Note that servicification alone won’t fix the issue; we still have more work to do on top of it.

The good news is service worker interception for shared workers (which Chrome today mostly matches the spec for) also has to change a lot for servicification. Once that is done, it should be relatively straightforward to change dedicated workers to use the same code as shared workers.

NetS13nServiceWorker (the service worker bits of network service) is currently in an experiment on Canary. I’d estimate (hopefully) Q3 for being enabled by default.

 

Thanks,

   Rick

On Thu, Jun 21, 2018 at 11:08 AM Chris Harrelson <chri...@chromium.org> wrote:

LGTM1

On Tue, Jun 19, 2018 at 7:10 AM Ben Kelly <bke...@mozilla.com> wrote:

On Tue, Jun 19, 2018 at 9:45 AM, Hiroki Nakagawa <nhi...@chromium.org> wrote:

2018-06-19 6:09 GMT+09:00 Ben Kelly <bke...@mozilla.com>:

Would it be possible to write a WPT test that verifies what we think is the correct behavior as part of your effort?  That at least can be reviewed and show where the current implementation diverges.

There're several WPTs for service worker interception for dedicated workers:

- https://wpt.fyi/results/service-workers/service-worker/dedicated-worker-service-worker-interception.https.html

- https://wpt.fyi/results/service-workers/service-worker/worker-interception.https.html

- https://wpt.fyi/results/service-workers/service-worker/worker-interception-redirect.https.html

Excellent.  Thank you!

Ben

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2B1UsbRTMcKg710mYYx4MC%3Dj_7%2BZdJ%2BavZ5mGbYkBKSR%3DepmPQ%40mail.gmail.com.

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOMQ%2Bw-Oh5attnE6kspA2UWcSjJkLzrsqUqjdErEhCWgtcTqow%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "worker-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to worker-dev+unsubscribe@chromium.org.
To post to this group, send email to worke...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/worker-dev/CAFUtAY_9EpzBuE%3Djx8MBWaGSzY-5DvppTjjAk7sDMfqYto%2BnJA%40mail.gmail.com.

[Hiroki Nakagawa]

Hi,

I'm sorry for the late update.

We, the blink-worker team, carefully discussed the risk of the compatibility issues raised in this thread, and decided to put this intent-to-ship on hold while we're working on addressing them.

We'll send the intent-to-ship again once the risks are mitigated enough to ship the feature.

Thank you for your feedback.

[Hiroki Nakagawa]

Hi blinkers,

Let me resume this intent-to-ship thread.

This I2S was suspended because of concerns about the interop issues on workers as follows:

  (A) Worker’s URL (e.g., ‘self.location’) is not the final response URL, and

  (B) Dedicated Workers are not "service worker client".

To be clear, these issues are derived from the general worker / loading infra in Blink, and not directly relevant to module dedicated workers.

The response URL issue (A) was already fixed. For the "service worker client" issue (B), we’ve actively been working on this as a part of the PlzDedicatedWorker project, and confirmed it will make the WPTs listed above pass, that is still behind the flag because of a few corner cases though. We’re going to send a separate I2S for the issue once it’s ready (maybe end of 2019Q4 or 2020Q1).

As above, the risks would relatively be mitigated.

Updates from the original I2S

- Added more tests for module dedicated workers, for example, for referrer policy and mixed contents.

- Web developers are getting more interested in the feature. Now the crbug issue has 69 stars as of October 30, 2019.

Thanks,

Hiroki

2018年8月1日(水) 16:28 Hiroki Nakagawa <nhi...@chromium.org>:

Thanks,

   Rick

LGTM1

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2B1UsbRTMcKg710mYYx4MC%3Dj_7%2BZdJ%2BavZ5mGbYkBKSR%3DepmPQ%40mail.gmail.com.

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOMQ%2Bw-Oh5attnE6kspA2UWcSjJkLzrsqUqjdErEhCWgtcTqow%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "worker-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to worker-dev+...@chromium.org.

To post to this group, send email to worke...@chromium.org.

[Ben Kelly]

Sounds good to me.  Thanks for all the progress on the interop issues!

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABj5diOa4thfUDTfXbnBQ7Rujr8iEHU%2B_HFcjoVXWWTqEQX0HA%40mail.gmail.com.

[Chris Harrelson]

Given those updates, LGTM1. Glad to hear the second interop issue will be fixed soon after shipping.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAK7rkMiDF2SO0vW0rEsRqK5W4iv6ES309eWFudFiG5Z%3DdKGa0A%40mail.gmail.com.

[TAMURA, Kent]

LGTM2

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOMQ%2Bw-SChdgGwFNk8rC_1V2p%3D%3D7xD9jSdcaDoZxYQ6yFmrsSg%40mail.gmail.com.

--

TAMURA Kent
Software Engineer, Google

[Yoav Weiss]

On Wed, Nov 6, 2019 at 8:08 PM TAMURA, Kent <tk...@chromium.org> wrote:

LGTM2

On Fri, Nov 1, 2019 at 3:56 AM Chris Harrelson <chri...@chromium.org> wrote:

Given those updates, LGTM1. Glad to hear the second interop issue will be fixed soon after shipping.

On Wed, Oct 30, 2019 at 6:31 AM Ben Kelly <wande...@chromium.org> wrote:

Sounds good to me.  Thanks for all the progress on the interop issues!

On Wed, Oct 30, 2019 at 3:11 AM Hiroki Nakagawa <nhi...@chromium.org> wrote:

Hi blinkers,

Let me resume this intent-to-ship thread.

This I2S was suspended because of concerns about the interop issues on workers as follows:

  (A) Worker’s URL (e.g., ‘self.location’) is not the final response URL, and

  (B) Dedicated Workers are not "service worker client".

To be clear, these issues are derived from the general worker / loading infra in Blink, and not directly relevant to module dedicated workers.

The response URL issue (A) was already fixed. For the "service worker client" issue (B), we’ve actively been working on this as a part of the PlzDedicatedWorker project, and confirmed it will make the WPTs listed above pass, that is still behind the flag because of a few corner cases though. We’re going to send a separate I2S for the issue once it’s ready (maybe end of 2019Q4 or 2020Q1).

So this potential interop issue will be out there for ~1 milestone and then fixed? Have any other browser shipped this already? Are web developers supposed to somehow modify their code once B lands?

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGH7WqEasGV3m-rEcvxAvUKki3L34uCJ1fH%3DqK-VBr58fxHtBg%40mail.gmail.com.

[Ben Kelly]

On Thu, Nov 7, 2019 at 2:38 PM Yoav Weiss <yo...@yoav.ws> wrote:

On Wed, Nov 6, 2019 at 8:08 PM TAMURA, Kent <tk...@chromium.org> wrote:

LGTM2

On Fri, Nov 1, 2019 at 3:56 AM Chris Harrelson <chri...@chromium.org> wrote:

Given those updates, LGTM1. Glad to hear the second interop issue will be fixed soon after shipping.

On Wed, Oct 30, 2019 at 6:31 AM Ben Kelly <wande...@chromium.org> wrote:

Sounds good to me.  Thanks for all the progress on the interop issues!

On Wed, Oct 30, 2019 at 3:11 AM Hiroki Nakagawa <nhi...@chromium.org> wrote:

Hi blinkers,

Let me resume this intent-to-ship thread.

This I2S was suspended because of concerns about the interop issues on workers as follows:

  (A) Worker’s URL (e.g., ‘self.location’) is not the final response URL, and

  (B) Dedicated Workers are not "service worker client".

To be clear, these issues are derived from the general worker / loading infra in Blink, and not directly relevant to module dedicated workers.

The response URL issue (A) was already fixed. For the "service worker client" issue (B), we’ve actively been working on this as a part of the PlzDedicatedWorker project, and confirmed it will make the WPTs listed above pass, that is still behind the flag because of a few corner cases though. We’re going to send a separate I2S for the issue once it’s ready (maybe end of 2019Q4 or 2020Q1).

So this potential interop issue will be out there for ~1 milestone and then fixed? Have any other browser shipped this already? Are web developers supposed to somehow modify their code once B lands?

To clarify, this service worker client issue has been an interop issue for a while.  Its not directly related to the es modules for dedicated workers, though.

I previously raised the concern that if es modules increased worker usage than these existing interop issues could get worse.  The worker team graciously delayed shipping es modules while they worked on the interop issues, but they are not directly connected.

I'm personally happy with the current situation since it seems unlikely es modules will change things significantly in one milestone that would make the other interop problems worse.

Anyway, I hope that helps.

[Yoav Weiss]

Thanks for clarifying!

LGTM3

[Hiroki Nakagawa]

Thank you all! This feature is now enabled by default on ToT (M80). I updated the chrome status entry.

2019年11月8日(金) 5:05 Yoav Weiss <yo...@yoav.ws>: