Intent to Prototype: Web Neural Network API (WebNN)
Hu, Ningxin
Contact emails
ningx...@intel.com, wc...@microsoft.com, rafael....@microsoft.com
Explainer
https://github.com/webmachinelearning/webnn/blob/main/explainer.md
Specification
Design docs
https://docs.google.com/document/d/1KDVuz38fx3SpLVdE8FzCCqASjFfOBXcJWj124jP7ZZ4/edit?usp=sharing
Summary
Construct and execute computational graphs that are at the heart of neural networks, taking advantage of on-device hardware acceleration. Currently, JavaScript frameworks must repurpose web APIs like WebGL, WebGPU and WebAssembly for the individual compute operations when executing computational graphs. The WebNN API allows JavaScript frameworks to access the native operating system machine learning APIs and the hardware accelerators underneath for the best performance and reliability of results.
Blink component
Motivation
Deep Learning, a subfield of Machine Learning (ML), with its various neural network architectures enables new user experiences for web applications that range from improved video conferencing to accessibility-improving features, with potential for improved privacy over cloud-based solutions. Key neural network-based use cases that apply to a wide range of web applications include e.g., face detection, semantic segmentation, style transfer, super resolution, image captioning, machine translation, and noise suppression. While some of these use cases can be implemented in a constrained manner with existing Web APIs (e.g. WebGL, WebGPU and WebAssembly), the lack of access to platform capabilities beneficial for ML such as dedicated ML hardware accelerators constraints the scope of experiences and leads to inefficient implementations on modern hardware. This disadvantages the web platform in comparison to native platforms.
Initial public proposal
https://discourse.wicg.io/t/api-set-for-machine-learning-on-the-web/2491/10
TAG review
https://github.com/w3ctag/design-reviews/issues/570
TAG review status
Issues addressed
Risks
Interoperability and Compatibility
All major browser vendors participated in the Community Group [1] that successfully completed the incubation of this API and graduated it to the Working Group [2] for standardization.
A well-attended W3C workshop [3] that brought together browser vendors and machine learning practitioners reviewed the API and recommended it for standardization.
Public support from Edge and Microsoft [4] with key contributors involved in the work.
Significant interest and positive feedback from machine learning JS frameworks in using WebNN when it is available, see e.g. TensorFlow.js/TensorFlow-Lite Web [5] and OpenCV.js [6].
An interoperability risk is to define the core operators that can be implemented cross-platforms and support the breadth of ML JS frameworks, however the members of the working group have experience defining and testing such interoperable machine learning operator set.
[1]: https://www.w3.org/groups/cg/webmachinelearning
[2]: https://www.w3.org/groups/wg/webmachinelearning
[3]: https://www.w3.org/2020/06/machine-learning-workshop/
[4]: https://www.w3.org/blog/2021/04/w3c-launches-the-web-machine-learning-working-group/
[5]: https://github.com/tensorflow/sig-tfjs/pull/2
[6]: https://github.com/opencv/opencv/pull/20406
Gecko: No signal
WebKit: No signal
Web developers: Positive (https://github.com/tensorflow/sig-tfjs/pull/2) The RFC of WebNN Delegate for TensorFlow Lite Web got good feedback and approval.
Ergonomics
The WebNN graph building (MLGraphBuilder.build [1]) and execution (MLGraph.compute [2]) APIs are synchronous. This is required to implement a backend for Wasm-based ML frameworks, such as ONNX Runtime Execution Provider [3], TensorFlow Lite Delegate [4] and OpenCV.js DNN backend [5]. These frameworks are written in C++ and expect calling sync APIs in its backend implementation. To avoid blocking the main thread, the good practice is to use these async APIs in a worker context. WebML working group is collecting feedback about whether to restrict the sync APIs to only exist in workers [6] and support async APIs [7].
[1]: https://www.w3.org/TR/webnn/#dom-mlgraphbuilder-build
[2]: https://www.w3.org/TR/webnn/#dom-mlgraph-compute
[3]: https://onnxruntime.ai/docs/execution-providers/add-execution-provider.html
[4]: https://www.tensorflow.org/lite/performance/implementing_delegate
[5]: https://github.com/opencv/opencv/pull/20406
[6]: https://github.com/webmachinelearning/webnn/issues/229
[7]: https://github.com/webmachinelearning/webnn/issues/230
Activation
Developers could use webnn-polyfill [1] immediately. The machine learning JS frameworks with WebNN backend would make it easier to use.
[1]: https://github.com/webmachinelearning/webnn-polyfill
Ethical
The WebNN API spec incorporates ethical considerations. The group is working on a complementary document [1] that outlines ethical issues associated with using Machine Learning on the Web and discusses how implementers and web developers can mitigate associated risks.
[1]: https://webmachinelearning.github.io/ethical-webmachinelearning/
Debuggability
Warnings and errors are exposed via dev tools. Specialized tools for debugging are TBD.
Is this feature fully tested by web-platform-tests?
Not yet. Working in progress, e.g., WebNN IDL test merged in wpt [1]
[1]: https://github.com/web-platform-tests/wpt/pull/31201
Flag name
--enable-webnn
Requires code in //chrome?
False
Tracking bug
https://bugs.chromium.org/p/chromium/issues/detail?id=1273291
Estimated milestones
TBD
Link to entry on the Chrome Platform Status
https://www.chromestatus.com/feature/5738583487938560
This intent message was generated by Chrome Platform Status.
