Intent to extend experiment: WebAssembly Garbage Collection (WasmGC), plus stringref

[Emanuel Ziegler]

We are requesting to extend our Origin Trial on WasmGC by another three milestones, effectively ending with M120.

The reason is that the standardization process is taking a little longer than we had hoped for and will be delayed by about three months blocking the shipment of a stable version. Since Google Sheets is currently running user experiments using the trial with positive results (currently at 50% of Google corporate sheets showing ~40% calculation time improvement), we would like to continue this experiment for the time being to gather more data, especially extending it to non-Google users before shipping.

The current end milestone for the trial is M117 with the most optimistic shipment of the feature happening in M119, perhaps even later if last minute spec changes are requested before we can reach phase 4 of the proposal. We would therefore potentially end the trial before M120 if shipment were indeed to happen sooner or use the full three milestones if there are further delays.

Thank you!

Contact emails

ad...@chromium.org, jkum...@chromium.org

Explainer

https://github.com/WebAssembly/gc/blob/master/proposals/gc/Overview.md

https://github.com/WebAssembly/function-references/blob/main/proposals/function-references/Overview.md

Specification

https://github.com/WebAssembly/gc/tree/main/proposals/gc

Summary

The GC proposal adds efficient support for high-level managed languages to WebAssembly, via struct and array types that enable language compilers targeting Wasm to integrate with a garbage collector in the host VM. In Chrome, enabling this feature implies enabling Typed Function References, which allow function references to be stored in the aforementioned structs and arrays.

Blink component

Blink>JavaScript>WebAssembly

Search tags

wasm, webassembly, gc, managed objects, wasmgc

TAG review

https://github.com/w3ctag/design-reviews/issues/814

TAG review status

Issues addressed

Risks

Interoperability and Compatibility

Gecko: Positive

WebKit: No signal

Web developers: Positive Google Sheets, which is currently compiling Java to JavaScript, is experimenting with using WasmGC to speed up their calculation engine. JetBrains is working on a Kotlin -> WasmGC compiler. Dart is working on a Dart -> WasmGC compiler, in collaboration with Flutter.

Other signals:

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?

Goals for experimentation

Ongoing technical constraints

Debuggability

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

No

Is this feature fully tested by web-platform-tests?

No

Flag name

Requires code in //chrome?

False

Tracking bug

https://bugs.chromium.org/p/v8/issues/detail?id=7748

Launch bug

https://launch.corp.google.com/launch/4231622

Estimated milestones

OriginTrial desktop last	117
OriginTrial desktop first	112

OriginTrial Android last	117
OriginTrial Android first	112

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/6062715726462976

Links to previous Intent discussions

Intent to Experiment: https://groups.google.com/a/chromium.org/g/blink-dev/c/HDbvHCVFSW0

[Chris Harrelson]

Hi,

Could you share summary feedback and learnings from the OT so far? The Sheets performance results sound great, anything else to share?

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPAU7Ryv%3D-1UFkh%2BJ5GBxMFBQ88FMa9u%3Dvbboo0hCy9xcOGgAA%40mail.gmail.com.

[Emanuel Ziegler]

Hi Chris,

There is an extensive summary document from the Sheets colleagues (including a tl;dr section in the beginning): go/sheets-wasm-corp-50

We believe that error count and crash rate (mostly OOM) regressions are caused by user space code but investigations are ongoing and we expect improvements over the next weeks.

Jetbrains is also using the OT for Kotlin, but I'm not aware of detailed insights. They advertised it in a talk at WasmIO a few months back.

Best regards,

    Emanuel

[Chris Harrelson]

On Wed, Jul 26, 2023 at 9:43 AM Emanuel Ziegler <ecmzi...@chromium.org> wrote:

Hi Chris,

There is an extensive summary document from the Sheets colleagues (including a tl;dr section in the beginning): go/sheets-wasm-corp-50

We believe that error count and crash rate (mostly OOM) regressions are caused by user space code but investigations are ongoing and we expect improvements over the next weeks.

Thanks! This is a Google-internal document, can some of it be shared publicly?

 

Jetbrains is also using the OT for Kotlin, but I'm not aware of detailed insights. They advertised it in a talk at WasmIO a few months back.

Would you mind asking them for feedback? Not blocking extending the OT, but feedback from OT participants is important to gather.

 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPAU7RwLJsFGk6XBHrBqwoG5_MNCvPyF_dLb2R_Wsy6ALCAkYw%40mail.gmail.com.

[Emanuel Ziegler]

On Wed, Jul 26, 2023 at 7:22 PM Chris Harrelson <chri...@chromium.org> wrote:

On Wed, Jul 26, 2023 at 9:43 AM Emanuel Ziegler <ecmzi...@chromium.org> wrote:

Hi Chris,

There is an extensive summary document from the Sheets colleagues (including a tl;dr section in the beginning): go/sheets-wasm-corp-50

We believe that error count and crash rate (mostly OOM) regressions are caused by user space code but investigations are ongoing and we expect improvements over the next weeks.

Thanks! This is a Google-internal document, can some of it be shared publicly?

I can ask, but since these numbers are WIP on a Google-internal trial with a Google product, I would be cautious. After all, they might not tell that much about the state of WasmGC as a lot of this is still heavily influenced by user space code as mentioned above. The numbers are going to be more meaningful once we have reached a more stable state and are ready to roll it out to external users.

Jetbrains is also using the OT for Kotlin, but I'm not aware of detailed insights. They advertised it in a talk at WasmIO a few months back.

Would you mind asking them for feedback? Not blocking extending the OT, but feedback from OT participants is important to gather.

Sure. Jetbrains themselves surely have feedback on WasmGC in general but likely limited insights from the trial itself. They might have received feedback from their partners though that I can ask them about.

[Alex Russell]

I'd be much more likely to support the extension if there was a report-out on what we've learned this far, particularly given the increasing risks presented by big products using it (Sheets) given that we're proposing to stretch to 8 months with no breaking changes.

Best,

Alex

On Wednesday, July 26, 2023 at 11:16:42 AM UTC-7 Emanuel Ziegler wrote:

On Wed, Jul 26, 2023 at 7:22 PM Chris Harrelson <chri...@chromium.org> wrote:

On Wed, Jul 26, 2023 at 9:43 AM Emanuel Ziegler <ecmzi...@chromium.org> wrote:

Hi Chris,

There is an extensive summary document from the Sheets colleagues (including a tl;dr section in the beginning): go/sheets-wasm-corp-50

We believe that error count and crash rate (mostly OOM) regressions are caused by user space code but investigations are ongoing and we expect improvements over the next weeks.

Thanks! This is a Google-internal document, can some of it be shared publicly?

I can ask, but since these numbers are WIP on a Google-internal trial with a Google product, I would be cautious. Afterﾠall, they might not tell that much about the state of WasmGC as a lot of this is still heavily influenced by user space code as mentioned above. The numbers are going to be more meaningful once we have reached a more stable state and are ready to rollﾠit out to external users.

Jetbrains is also using the OT for Kotlin, but I'm not aware of detailed insights. They advertised it in a talk at WasmIOﾠa few months back.

Best regards,

    Emanuel

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPAU7Ryv%3D-1UFkh%2BJ5GBxMFBQ88FMa9u%3Dvbboo0hCy9xcOGgAA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Emanuel Ziegler]

Hi Alex,

Just to quickly reply on the breaking changes and what we've learnt: we have been in an active collaboration with Sheets on this for over 2.5 years now with a lot of development and measurements leading up to the origin trial. This allowed us to shape the proposal and the implementation before that. The main goal of the trial is to confirm that our expectations hold up in real-world scenarios and the performance numbers that we got (~40% calculation time improvement) are perfectly in line with our expectations. Other findings are mostly integration issues in user space but have fortunately not challenged the proposal nor our architecture. The extension is meant to allow us longer and broader testing on a larger variety of sheets.

We have made several changes during the trial and evaluated them together with J2CL and Sheets as well as other partners, but intentionally avoided breaking changes as these need to be propagated to Binaryen, J2Wasm and eventually Sheets which would require weeks of interruption every time. We know that there will be a breaking change coming once the final version of the proposal is merged in and that we will have to adapt our implementation before its stable release. This does not affect the functionality as the functionality is perfectly aligned, only the op codes are going to change.

I hope this helps alleviate your concerns.

Best,

    Emanuel

Best regards,

    Emanuel

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPAU7Ryv%3D-1UFkh%2BJ5GBxMFBQ88FMa9u%3Dvbboo0hCy9xcOGgAA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Alex Russell]

Hey Emanuel,

Sorry for the slow follow-up, and thanks for the background.

Avoiding breaking changes when we haven't shipped yet is against the spirit of OT. None of this meant to enable soft-launches of the sort you're describing.

This is our one and only opportunity to get the API shape right; we won't get another moment at which we can iterate and improve the shape of the API. The constraints of OTs are designed to price in inconvenience for a small number of partners to iterate with us, including breaking changes. We can also run OTs in parallel to accommodate multiple variants.

With that as preamble, I'm going to -1 an extension of this OT until we have a lot more clarity. Please consider this a chance to make updates you would have proposed as post-launch breaking changes, with an opening to launch a new OT or I2S for the updated feature.

Are you able to perhaps join us at next week's Blink API OWNERS meeting? Or we can discuss as a one-off? Want to make sure we get on the same page and unblock you.

Best,

Alex

Best regards,

    Emanuel

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPAU7Ryv%3D-1UFkh%2BJ5GBxMFBQ88FMa9u%3Dvbboo0hCy9xcOGgAA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Emanuel Ziegler]

Hi Alex,

Thanks for your reply. I feel there might be a misunderstanding on where we are with this proposal and the goals of the OT.

This is not an API that Google has come up with, but a W3C community group proposal that has seen several iterations over the last 3 years. At this point, we have agreement on the functionality and shape of the API across browsers, compiler toolchains and the community group. What we are trying to confirm with the trial is that our previous experiment findings also hold up in real-world scenarios (see also original I2E on the goals for experimentation). So far, everything looks promising and we just ask to gather more data to make a more informed decision when the proposal goes for its final vote. We certainly don't intend to soft launch the feature.

Adam and I would be happy to join the meeting next week to answer any questions that you might have. Just send us an invite.

Best regards,

    Emanuel

Best regards,

    Emanuel

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPAU7Ryv%3D-1UFkh%2BJ5GBxMFBQ88FMa9u%3Dvbboo0hCy9xcOGgAA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Adam Klein]

Alex & I chatted offline (to avoid blocking on the API Owners meeting), and I now understand the root of his concern, which had to do with pushing any breaking changes past the end of the OT and into the post-shipping period. Let me clarify the situation with respect to breaking changes, the Wasm Community Group's consensus process, and our tentative plans around shipping.

The upcoming breaking change that Emanuel referred to, reshuffling the opcodes in the binary format, is waiting on the Wasm CG to move the proposal to Phase 4, which will give us confidence that no more changes to the proposal are forthcoming. Phase 4 is also when our team traditionally sends an intent-to-ship, as it denotes strong cross-engine consensus (including at least two web VMs having implemented the feature). Thus the ordering of operations we intend is:

1. Extend the OT period to allow continued experimentation

2. Proposal moves to Phase 4 in Wasm CG (a meeting on this topic is scheduled for September 12)

3. Opcodes are renumbered in V8

4. Ship enabled-by-default in Chrome (with 3xLGTMs on an I2S thread)

To be perfectly clear, the opcode renumbering will happen before Wasm GC is enabled by default, and we will not be making breaking changes after that point. Users of the Origin Trial will have to adjust to the new opcodes when the final proposal ships in Chrome.

As Emanuel said in his last message, there is still experimental value to be had in extending the current OT, as our partners continue to gather performance data from the wild and we dig into any issues they encounter. And while we don't foresee the need to change the semantics of the proposal due to this feedback, we're open to doing so if appropriate (which would likely delay shipping, and would certainly cause us to wait for shipping until any changes are incorporated into V8's implementation).

I hope that clears things up!

Separately, we're working with the Sheets team (and other partners) to publicly share any OT feedback they have, as Chris requested.

Thanks,

Adam

[Alex Russell]

Thanks for all of this, Adam.

This all makes much more sense to me now. This ordering makes sense, and I'm sorry for the holdup. 

LGTM to extend this OT.

Looking forward to seeing the feedback data.

Best,

Alex

Best regards,

    Emanuel

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPAU7Ryv%3D-1UFkh%2BJ5GBxMFBQ88FMa9u%3Dvbboo0hCy9xcOGgAA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Zalim_ Bashorov]

Hi, folks!

Here is a bit of feedback from the Kotlin team (JetBrains).

We have been working on compiling kotlin to WebAssembly for a few years. The current implementation depends on Wasm GC and Exception Handling proposals. For now, it shows good performance and outperforms our JS implementation in many cases. However, there are other cases where Kotlin/JS is still faster or shows comparable performance. It seems to me there is a big room for further improvements. Users mostly confirm performance benefits too.

Having Origin Trail reduced the steps required for users to check out demos using new technologies and simplified trying new technologies by themselves and giving us feedback.

Thanks to OT Chrome users have the best experience on our kotlin playground while using wasm target.

We don't use Stringref on production builds, but our experiments show up to 60 times speedup on microbenchmarks and 3-5 times speedup in examples a bit closer to real-life, like DBMonster (no-stringref, with-stringref).

I think interactive demos (like jetsnack and creative) work much better than a gif or video, with OT our demos work out of the box for Chrome users. So it is easier to play with demos for users, and it's easier for us to convince users to give the new technology a try and give us feedback.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPAU7Ryv%3D-1UFkh%2BJ5GBxMFBQ88FMa9u%3Dvbboo0hCy9xcOGgAA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.