RFC: Promoting IREE web platform / Emscripten builds to "stable"

[Scott Todd]

We're making good progress adding support for WebGPU in IREE, on both the compiler and runtime sides. Development for the web platform has been mostly "experimental" so far, and I'd like to pause and collect some feedback before taking steps to remove that label. This will not mean that IREE+Wasm or IREE+WebGPU are "ready for production", per se - but it will mean that the associated code will be tested on presubmit and will move from the experimental/web/ folder.

What is in scope for supporting "the web platform"?

All things needed to use the IREE runtime in web browsers, including:

• Compiling IREE's C runtime to JavaScript/WebAssembly (using Emscripten, at least for now)
• Using IREE's llvm-cpu compiler target and WasmLinkerTool to generate WebAssembly CPU code
• Using IREE's webgpu compiler target to generate WGSL shader code for WebGPU
• Some platform/architecture-specific runtime code and build system flags for things like threading, event loops, and memory allocation
• A new runtime HAL driver for WebGPU, currently being developed on the benvanik-webgpu-skeleton branch
• Testing IREE's existing test suites via VMVX/Wasm/WebGPU in web browsers
• Benchmarking IREE's existing benchmark suites and being able to quantify/qualify performance characteristics
• Maintaining samples and/or language bindings (JavaScript/TypeScript. HTML/CSS)
• Documentation on https://iree-org.github.io/iree/

Note that our current focus for production web deployment is WebGPU (and possibly VMVX), not llvm-cpu generated WebAssembly. Infrastructure will be shared between each of these though.

What dependencies will this add?

• The baseline runtime build depends on Emscripten's emsdk, which provides a wrapper around CMake that we can build with. Installing and using this is comparable to installing the Android SDK/NDK to develop for Android. Our emscripten Dockerfile already has emsdk configured, and any relevant CMake files or compiler option tweaks will be guarded by if(EMSCRIPTEN).
• The llvm-cpu compiler target with the WasmLinkerTool uses wasm-ld/lld from LLVM and the WebAssembly LLVM target.
• The webgpu compiler target uses Tint to compile SPIR-V to WGSL. This is fetched on-demand (not included as a submodule), as it is a fairly large dependency to take for an optional component.
• The WebGPU runtime HAL may depend on Dawn and/or wgpu-native (both large and complex). We haven't yet decided if these are worth the effort to support alongside browser implementations - they may be useful for performance or correctness comparisons.
• Inclusion of the HAL will be guarded by the IREE_HAL_DRIVER_WEBGPU CMake option, which will default to off (unless running under Emscripten's emcmake, perhaps)
• Testing will likely involve installing Chromium or a similar browser along with some framework for scripting against it. I haven't started on that part yet, and I really hope it will be minimally invasive.
• Language bindings would involve other web ecosystem tools (webpack, nodejs, typescript, etc.). I haven't started on these either, but they may fit better in a separate repository. We do already have some deps and release machinery for Python and Java in the core repository though.

Can/should this be developed outside of the core IREE repository?

Possibly - the HAL driver could be external, and if language bindings get complex or introduce many dependencies then spinning out a new repository could be worthwhile.

Running builds/tests on presubmit and being able to make atomic changes to the runtime will help with development stability and velocity.

What's the expected work breakdown and timeline for this?

• I'm ready to add presubmit coverage for runtime Emscripten builds in PR#10319
• Presubmit coverage for runtime tests will require tweaking what's in experimental/web/testing/, probably finding a new place for scripts and HTML scaffolding in build_tools/
  
• experimental/web/sample_static/ and experimental/web/sample_dynamic/, along with an upcoming sample_webgpu could use new JS/TS code in https://github.com/iree-org/iree/tree/main/runtime/bindings . The application code (JS/HTML/CSS/scripts) could move to either https://github.com/iree-org/iree/tree/main/samples or https://github.com/iree-org/iree-samples
• The benvanik-webgpu-skeleton branch with the WebGPU HAL driver should be merged in < 2 months
  

Thoughts / concerns?

[Scott Todd]

Bumping this - does anyone have feedback on the proposed plan here? Any votes to put the web build/test/etc. code in a separate repository?

[Geoffrey Martin-Noble]

Sorry I totally forgot to respond to this! I'm +1 on graduating this and on the staged approach you're taking.

First a high-level point that is perhaps more of a question. Are we committing to putting a sustained effort into this as a target for IREE? We've had some cases in the past of prototypy things that were introduced but then bitrotted. As a litmus test, if we did some big migration, would this backend get migrated to work, or would it be disabled to be "fixed at some point"? My impression from the work you've been doing, we are indeed trying to make this first-class (subject to limitations around weird toolchain stuff), but I just want to double check that we are committed to that here.

I like keeping WASM, at least for testing that we can do on a CPU without a browser (which would be required for the other drivers IIUC). I'd also suggest we start with the emscripten build only on postsubmit to start. I know it's already had postsubmit CI for a bit now on Buildkite, but I'd like to make sure everything's stable on GitHub actions before we add it to presubmits. When adding tests, let's make sure to get unit tests running on emscripten before we add any e2e model tests or browser shenanigans.

Overall this LGTM. I think we should have the external drivers conversation eventually, and wouldn't object if we decided to have it with this in particular (especially if dependency stuff starts to get complicated), but I'm fine keeping it in-tree for now.

Regarding on-demand dependencies. Can we make these as not-magic as possible? Is this the sort of thing where it needs to be pinned to a really specific version or could it work with an existing install on a dev's device? Do we want to support a range of versions? In the latter cases I'd prefer we use an existing install, pin some specific version on the CI, and give devs a script they can run to install a known-good version. If we do fetch anything as part of the build system, please make sure that it is only fetched once, cleans up after itself, etc.

These latter things are all basically implementation details though. If we are indeed think the experiment has proved what it needed to, then I'm on board with graduating it in the measured fashion you describe.

Cheers,

Geoffrey

--
You received this message because you are subscribed to the Google Groups "iree-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to iree-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/iree-discuss/CAF-j_jd2Pa8r-cB5vhduUgGfuS2t7COj0yF6%2BcTKvgq_ZhoSvw%40mail.gmail.com.

[Scott Todd]

Are we committing to putting a sustained effort into this as a target for IREE?

The first goals are:

1. Provide feedback to the WebGPU developers/community about how suitable the API is for ML/compute and what features are missing
2. Prove that IREE can offer a competitive web backend

Both of these are longer term / strategic goals, vs more project/user-driven goals like "improve x86 performance for programs with convolutions". I don't think it is possible to meet those goals with a rough prototype / without aiming for a stable and fully supported configuration though. We could certainly find that WebGPU is too difficult to support without a few features, performance isn't up to par, or distributing regular releases isn't worth the effort proportional to the number of interested users - at which point we could drop support level down a tier and let some parts bitrot.

I'd also suggest we start with the emscripten build only on postsubmit to start. I know it's already had postsubmit CI for a bit now on Buildkite, but I'd like to make sure everything's stable on GitHub actions before we add it to presubmits.

That's fine with me, though I don't think that presubmit adds too much risk, so I'd propose starting with presubmit coverage and moving it to postsubmit if we encounter instability. To elaborate a bit, the Emscripten runtime build on https://github.com/iree-org/iree/pull/10319 takes 6 minutes, of which 3m30s is spent actually building (vs downloading/unpacking artifacts/source code).

When adding tests, let's make sure to get unit tests running on emscripten before we add any e2e model tests or browser shenanigans.

JavaScript-based tests can run under node.js or in browsers, but my experience with these more advanced APIs (Wasm / WebGL / WebGPU) is that node.js can be different enough from browsers that you end up paying an infrastructure cost supporting both, and coverage doesn't necessarily translate across them. FWIW, Dawn does have node.js bindings (for WebGPU) that are worth checking out: https://dawn.googlesource.com/dawn/+/refs/heads/main/src/dawn/node/.

Regarding on-demand dependencies. Can we make these as not-magic as possible? Is this the sort of thing where it needs to be pinned to a really specific version or could it work with an existing install on a dev's device?

I'm anticipating some pain here. WebGPU is still changing a bit, and features like WebAssembly threads (pthreads) are under development in Emscripten, so there have been cases like

• for WebGPU, the Emscripten SDK needs to match what the browser implements
• for WebAssembly, the version of LLVM that IREE uses for Wasm codegen needs to match what Emscripten uses

I haven't seen as much instability as in MLIR/LLVM though, so I think that just updating Emscripten's emsdk regularly [in our Docker image] will cover most cases.