Intent to Ship: WebAssembly PostMessage

[bradn...@chromium.org]

Contact emails

mtr...@chromium.org, bradn...@chromium.org, natt...@chromium.org

Spec

https://github.com/WebAssembly/design/pull/1074

(Clarifying existing informal understanding: "structured clone for postMessage works for Wasm modules)

TAG Review: https://github.com/w3ctag/design-reviews/issues/167

Summary

Extends WebAssembly to support PostMessage of WebAssembly.Module objects to Web Workers. To clarify, this is scoped to just Web Workers (same process, different thread), and not extended to cross-process scenarios (such as cross-origin postMessage, or shared web workers).

Link to “Intent to Implement” blink-dev discussion

Follow on to WebAssembly intent to implement:

https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/6iqiskBdDQE

As well as WebAssembly structured cloning intent to implement:

https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/Dogpn1hpnhw

Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes

Demo links

https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/http/tests/wasm/wasm_remote_postMessage_test.https.html

Debuggability

Leverages existing debugging support for JavaScript APIs.

Interoperability and Compatibility Risk

This feature is was originally intended to be included in launch of WebAssembly. It is supported by Firefox, and other engines have expressed the intention to include it.

Edge: Not yet implemented, in progress.

Firefox: Launched.

Safari: Implementation in progress.

Web developers: Emscripten developers likely to use this pattern once available for Web Workers.

Of all the tests for web-exposed behavior, are any not in web-platform-tests? Please explain and link to bugs.

The API is covered here:

https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/external/wpt/wasm/

OWP launch tracking bug:

https://bugs.chromium.org/p/chromium/issues/detail?id=746486&desc=2

Entry on the feature dashboard

Subset of:

https://www.chromestatus.com/features/5453022515691520

[Chris Harrelson]

Hi,

The pull request mentioned in this intent appears to not be merged yet. Is that correct? Did Firefox launch the API without a spec?

Chris

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/64e8cef2-4541-4de5-9329-26156d3a2a4d%40chromium.org.

[Bradley Nelson]

You are correct that this PR has not yet been merged. :-/

I would characterize the PR as clarifying and narrowing the more general assertion made current in JS.md that modules are structured clonable:

https://github.com/WebAssembly/design/blob/master/JS.md#structured-clone-of-a-webassemblymodule

Firefox shipped postMessage under that broader framing.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Rick Byers]

To what extent are the web-exposed aspects of the new more detailed specification text covered by tests?  I see this test covers the basic postMessage scenario, but from a quick look I didn't see anything that tests, for example, the cases in the spec PR that throw DataCloneError.  Are you aware of any observable details where Chrome and Firefox behavior differ today?

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/64e8cef2-4541-4de5-9329-26156d3a2a4d%40chromium.org.

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAG%3DZA4fkhotPZvzV%2B4kOwHyc%3DgJgAZ%3DgzqZba9tpa7Ay5jn2GQ%40mail.gmail.com.

[Domenic Denicola]

For an example of some exhaustive tests of a related area, namely SharedArrayBuffers, see https://github.com/w3c/web-platform-tests/tree/master/html/infrastructure/safe-passing-of-structured-data/shared-array-buffers . See also the test plan we used in assembling those tests, in https://github.com/w3c/web-platform-tests/pull/5003 .

 

In an ideal world, we’d get similar coverage for WebAssembly.Module and WebAssembly.Memory.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/64e8cef2-4541-4de5-9329-26156d3a2a4d%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAG%3DZA4fkhotPZvzV%2B4kOwHyc%3DgJgAZ%3DgzqZba9tpa7Ay5jn2GQ%40mail.gmail.com.

 

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAFUtAY9V4CSg29KtifaPcB3NWuNy26tdpx7TKbhPmxPY3pfA1w%40mail.gmail.com.

[Philip Jägenstedt]

This intent is still unresolved, let's fix that!

What is the ETA on getting https://github.com/WebAssembly/design/pull/1074 merged? If there is something blocking it that really has no bearing on eventual interop, then going ahead with it open is certainly an option, but what's the situation?

In terms of tests, does http://wpt.fyi/wasm/wasm_serialization_tests.html cover it all? Firefox is already passing those tests, so at a glance the testing story already looks to be in order.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/DM2PR0501MB8594E1E7569B2201C5BE074DFBE0%40DM2PR0501MB859.namprd05.prod.outlook.com.

[bradn...@chromium.org]

Hi All,

Bringing this back to life, as we've now got some customers that are interested in sharing WebAssembly compilation between Workers (even though we don't yet have SABs turned back on).

The structured cloning aspect got lost before we issued our fPWD, but is being resolved for the next draft (by Dan Ehrenberg who know's this space far better than I do):

https://github.com/WebAssembly/spec/pull/711

Assuming that lands in the Editors draft soon, how would folks feel about switching this on in M66 (it's been sitting ready behind a flag for some time now)?

Thanks!

-BradN

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/64e8cef2-4541-4de5-9329-26156d3a2a4d%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAG%3DZA4fkhotPZvzV%2B4kOwHyc%3DgJgAZ%3DgzqZba9tpa7Ay5jn2GQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Marijn Kruisselbrink]

On Wed, Feb 21, 2018 at 7:46 PM, <bradn...@chromium.org> wrote:

Hi All,

Bringing this back to life, as we've now got some customers that are interested in sharing WebAssembly compilation between Workers (even though we don't yet have SABs turned back on).

The structured cloning aspect got lost before we issued our fPWD, but is being resolved for the next draft (by Dan Ehrenberg who know's this space far better than I do):

https://github.com/WebAssembly/spec/pull/711

Assuming that lands in the Editors draft soon, how would folks feel about switching this on in M66 (it's been sitting ready behind a flag for some time now)?

Is it actually sitting ready being a flag, or is just some version of the feature that might or might not match the spec sitting there. I.e. currently there are test failures such as external/wpt/wasm/wasm_service_worker_test.https.html (caused by https://crbug.com/798572), but from looking at that PR it seems like the spec might be changing to no longer care about agent clusters? But if that is the case I'm not sure the implementation correctly handles cross-process postMessage, which before with the same-agent-cluster restriction wouldn't be possible, but with the new spec seem like they should be possible?

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/36d19f1b-1245-490f-8671-871dbcaee22b%40chromium.org.

[Alex Russell]

Hey Brad,

Per the TAG review of the WASM API last year, it was brought up that removing modules from their "resourceness" is odd. Can the needs of the folks trying to share modules across workers use the Cache Storage API to round-trip them? Cache Storage doesn't require special care to maintain the Origin Model (it's scoped already), and enables transparent optimization of modules at global addresses.

Regards

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2BOSsVYFGPmFhL68VCFrQ-%3DktHrhF5%2BD35UmL4D8NFUH0Me01Q%40mail.gmail.com.

[bradn...@chromium.org]

Hi Alex,

Towards, the goal of more heavily tying wasm modules originating from the network to their URI, we've now shipped WebAssembly.compileStreaming + instantiateStreaming.

InstatiateStreaming is now used by Emscripten and other code generators when browser support is available.

We also have begun work on ES6 modules integration, and the proposal now has a champion.

However, as I thought we'd explored last year, providing a robust signal to allow code sharing between Workers is essential to being able to offer a reliable threading primitive for the platform.

The browser network and code cache has the potential to fulfill this role in the limit, but as a practical matter, current caches would require a substantial re-architecting in each major engine to make this work well.

More straightforward URI based deduping could possibly be made to work by keeping a per renderer cache, but would likely have to punch through a variety of layering assumptions.

Code sharing rendezvousing by URI with weaker guarantees is something I'd like to see us implement this year, particularly for cross-origin code sharing, in tandem with cached-while-revalidate being recast to include code compilation as part of "revalidate", however, I'm doubtful that this will be robust enough any time soon to use for thread creation.

Offering serialization to Workers, in contrast, seems to offer a crisp way to signal developer intent (handing code between entities), and doesn't seem to have downside I'm aware of, other than maybe redundant functionality.

I had the impression the TAG had given its blessing to this API surface?

https://github.com/w3ctag/design-reviews/issues/167

Or have I misinterpreted that issue?

Thanks!

-BradN

[Ben L. Titzer]

We need a decision on shipping this. As per Brad's email, it looks like everything is ready to go, we just need an appropriate LGTM from a web platform owner.

Thanks!

-BradN

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/64e8cef2-4541-4de5-9329-26156d3a2a4d%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAG%3DZA4fkhotPZvzV%2B4kOwHyc%3DgJgAZ%3DgzqZba9tpa7Ay5jn2GQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAFUtAY9V4CSg29KtifaPcB3NWuNy26tdpx7TKbhPmxPY3pfA1w%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/DM2PR0501MB8594E1E7569B2201C5BE074DFBE0%40DM2PR0501MB859.namprd05.prod.outlook.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/36d19f1b-1245-490f-8671-871dbcaee22b%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Marijn Kruisselbrink]

On Mon, Jun 11, 2018 at 2:51 AM Ben L. Titzer <tit...@google.com> wrote:

We need a decision on shipping this. As per Brad's email, it looks like everything is ready to go, we just need an appropriate LGTM from a web platform owner.

What is the current situation with tests for this? As far as I can tell the situation is similar to what it was back in February, and we're still failing tests such as external/wpt/wasm/wasm_service_worker_test.https.html. I don't know if that is because the test doesn't match the spec or if it is because our implementation is broken, but it might make sense for somebody to try to make sure tests, spec and implementation all align.

[Rick Byers]

I'm sorry, it looks like we dropped the ball on this one (the tracking system we're using for intents now doesn't go back this far!).  I need to do some research myself to be able to weigh in on this with any value, but I'll make sure it's first on the agenda for the API owner meeting Thursday so that we have a response to you after that by the latest.

[Anne van Kesteren]

On Mon, Jun 11, 2018 at 2:51 AM, 'Ben L. Titzer' via blink-dev
<blin...@chromium.org> wrote:
> We need a decision on shipping this. As per Brad's email, it looks like
> everything is ready to go, we just need an appropriate LGTM from a web
> platform owner.

Unless I'm missing something,
https://github.com/WebAssembly/spec/pull/711 still isn't merged and
the issue with SharedArrayBuffer objects raised privately at some
point (not a security issue as far as I could tell), which would
probably also apply here as I noted in that PR, still hasn't been made
public by the relevant parties.

It'd be really great if that could finally be dealt with in some manner.


--
https://annevankesteren.nl/

[Ben Smith]

OK, it sounds like there are some implementation issues (the WPT tests mentioned above), as well as some spec issues (the PR above) that need to be resolved.

In addition there was some concern from Alex about using postMessage at all for this purpose, though some folks have told me that they've discussed this with him offline.

I'll look into these issues and follow up on this thread.

[Ben Smith]

Here's the current status: at the WebAssembly CG, we decided to include structured clone of modules in the v1 spec, without IndexedDB support.  The spec PR for this landed yesterday.

I looked into the single WPT failure too, which has this associated bug. The basic issue AIUI is that the current implementation will share modules between the main thread and the service worker, even though they are in different agent clusters. Should this bug block shipping?

[Michael Hablich]

+Domenic. 

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/7e6e2cf8-3f36-423b-8f22-47a5dcce3a6b%40chromium.org.

[Domenic Denicola]

On Thu, Jun 28, 2018 at 3:39 AM Michael Hablich <hab...@chromium.org> wrote:

+Domenic. 

On Thu, Jun 28, 2018 at 3:38 AM Ben Smith <bi...@chromium.org> wrote:

Here's the current status: at the WebAssembly CG, we decided to include structured clone of modules in the v1 spec, without IndexedDB support.  The spec PR for this landed yesterday.

I looked into the single WPT failure too, which has this associated bug. The basic issue AIUI is that the current implementation will share modules between the main thread and the service worker, even though they are in different agent clusters. Should this bug block shipping?

From my perspective, it seems a bit dangerous to ship with this bug: we could get into a situation where web developers write code to share modules between the main thread and the service worker, that works in Chrome, but fails to work in other browsers, producing only-in-Chrome sites. Worse, if such code becomes widespread, it could prevent us from future re-architecturing of our own main thread/service worker split.

Hopefully the API owners can weigh in with a more official answer; they have a lot of experience weighing these sorts of things.

 

[Marijn Kruisselbrink]

Note that this bug isn't limited to service workers, that just happens to be the case that has test coverage. The same issues also occurs with shared workers. And it's not consistent either, since the behavior depends on if chrome happens to decide to put the shared/service worker in the same process as the window/worker messaging to it. If source and destination are in the same process a WASM module postmessage will succeed, if they're in different processes it will fail.

(that also means that there is a much harder to trigger and thus much more theoretical bug with window-to-window messaging as well. If you can somehow get two "unrelated" same origin windows in the same process (should be pretty plausible with site isolation), and get a MessageChannel setup between them (might require posting a port from one to the other via a shared worker), those two windows will also be able to send WASM modules to each other, even though the spec says they are not in the same agent cluster).

[Domenic Denicola]

On Thu, Jun 28, 2018 at 12:20 PM Marijn Kruisselbrink <m...@chromium.org> wrote:

Note that this bug isn't limited to service workers, that just happens to be the case that has test coverage.

On that note, it may be worth comparing the test coverage for WebAssembly.Module serialization, with that for SharedArrayBuffer. Anne and I spent a decent amount of time working through a big checklist to try to make sure we could test the contours of the agent cluster formalism in some depth.

[Ben Smith]

Just wanted to update the thread here. It took a while, but I've made some progress on this front, adding checks for messages containing SharedArrayBuffers and WebAssembly Modules, and sending a "messageerror" event if they aren't in the same agent cluster: https://chromium-review.googlesource.com/1130505.

This CL doesn't cover everything yet; it still needs to handle the window-to-window failure cases. I'll work on that next.

[Ben Smith]

I have one outstanding CL to add WebAssembly equivalents of the SharedArrayBuffer postMessage tests: https://chromium-review.googlesource.com/c/chromium/src/+/1175490.

Most of these tests pass, aside from tests involving BroadcastChannel (which is a more involved change requiring mojo fixes: see https://bugs.chromium.org/p/chromium/issues/detail?id=874302), and the tests with nested workers, which AIUI are not currently enabled.

There are two tests from the checklist below that are not covered by the implementation, and do not currently have web platform tests. I believe the current implementation is more conservative than required for the spec, since each document is considered to be in its own agent cluster, so we can loosen this restriction in the future.

Are there concerns shipping with this status?

[Chris Harrelson]

LGTM1!

I think any remaining testing concerns can be resolved in parallel with shipping.

Another clarifying question: I see that since this intent started more than a year ago, all other browsers have shipped. Or at least that's what the chromestatus entry now says. Is that correct?

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/3928f877-9b4a-498c-ab1d-dd1e00b62524%40chromium.org.

[Chris Harrelson]

On Tue, Aug 21, 2018 at 3:06 PM Chris Harrelson <chri...@chromium.org> wrote:

LGTM1!

I think any remaining testing concerns can be resolved in parallel with shipping.

Another clarifying question: I see that since this intent started more than a year ago, all other browsers have shipped. Or at least that's what the chromestatus entry now says. Is that correct?

Sorry, that chromestatus entry was for WebAssembly in general. Just noticed that. Is there one for WebAssembly PostMessage yet? If not you should add a new one.

[Ben Smith]

On Tuesday, August 21, 2018 at 3:07:54 PM UTC-7, Chris Harrelson wrote:

On Tue, Aug 21, 2018 at 3:06 PM Chris Harrelson <chri...@chromium.org> wrote:

LGTM1!

I think any remaining testing concerns can be resolved in parallel with shipping.

Another clarifying question: I see that since this intent started more than a year ago, all other browsers have shipped. Or at least that's what the chromestatus entry now says. Is that correct?

Sorry, that chromestatus entry was for WebAssembly in general. Just noticed that. Is there one for WebAssembly PostMessage yet? If not you should add a new one.

You're right, this didn't have its own status because it was originally part of the initial launch. Added new status entry here: https://www.chromestatus.com/feature/6012085880225792.

 

 

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Yoav Weiss]

LGTM2

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/3928f877-9b4a-498c-ab1d-dd1e00b62524%40chromium.org.

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/f2aa5409-4d5a-4484-9671-314afc812186%40chromium.org.

[Rick Byers]

LGTM3

On Wed, Aug 29, 2018 at 4:17 PM Victor Valle <raid...@calebvalle.net.in> wrote:

Thank

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACj%3DBEhKHioyfOY%2BqOtJB0wHU7isQjBii1FB4NDZDnac%3DfKyxg%40mail.gmail.com.