Intent to Prototype: Web Neural Network API (WebNN)

519 views
Skip to first unread message

Hu, Ningxin

unread,
Nov 30, 2021, 9:11:17 PM11/30/21
to blin...@chromium.org, Kostiainen, Anssi, Corentin Wallez, Chai Chaoweeraprasit, Rafael Cintron, Daniel LaLiberte, pi...@google.com, Jonathan Bingham, Jeffrey Yasskin, Honglin Yu, Kai Ninomiya, Jon Napper, Zhang, Belem, Fu, Junwei, Dai, Feng

Contact emails

ningx...@intel.com, wc...@microsoft.com, rafael....@microsoft.com 

 

Explainer

https://github.com/webmachinelearning/webnn/blob/main/explainer.md

 

Specification

https://www.w3.org/TR/webnn/

 

Design docs

https://docs.google.com/document/d/1KDVuz38fx3SpLVdE8FzCCqASjFfOBXcJWj124jP7ZZ4/edit?usp=sharing

 

Summary

Construct and execute computational graphs that are at the heart of neural networks, taking advantage of on-device hardware acceleration. Currently, JavaScript frameworks must repurpose web APIs like WebGL, WebGPU and WebAssembly for the individual compute operations when executing computational graphs. The WebNN API allows JavaScript frameworks to access the native operating system machine learning APIs and the hardware accelerators underneath for the best performance and reliability of results.

 

Blink component

Blink>WebML

 

Motivation

Deep Learning, a subfield of Machine Learning (ML), with its various neural network architectures enables new user experiences for web applications that range from improved video conferencing to accessibility-improving features, with potential for improved privacy over cloud-based solutions. Key neural network-based use cases that apply to a wide range of web applications include e.g., face detection, semantic segmentation, style transfer, super resolution, image captioning, machine translation, and noise suppression. While some of these use cases can be implemented in a constrained manner with existing Web APIs (e.g. WebGL, WebGPU and WebAssembly), the lack of access to platform capabilities beneficial for ML such as dedicated ML hardware accelerators constraints the scope of experiences and leads to inefficient implementations on modern hardware. This disadvantages the web platform in comparison to native platforms.

 

Initial public proposal

https://discourse.wicg.io/t/api-set-for-machine-learning-on-the-web/2491/10

 

TAG review

https://github.com/w3ctag/design-reviews/issues/570

 

TAG review status

Issues addressed

 

Risks

 

Interoperability and Compatibility

All major browser vendors participated in the Community Group [1] that successfully completed the incubation of this API and graduated it to the Working Group [2] for standardization.

 

A well-attended W3C workshop [3] that brought together browser vendors and machine learning practitioners reviewed the API and recommended it for standardization.

 

Public support from Edge and Microsoft [4] with key contributors involved in the work.

 

Significant interest and positive feedback from machine learning JS frameworks in using WebNN when it is available, see e.g. TensorFlow.js/TensorFlow-Lite Web [5] and OpenCV.js [6].

 

An interoperability risk is to define the core operators that can be implemented cross-platforms and support the breadth of ML JS frameworks, however the members of the working group have experience defining and testing such interoperable machine learning operator set.

 

[1]: https://www.w3.org/groups/cg/webmachinelearning

[2]: https://www.w3.org/groups/wg/webmachinelearning

[3]: https://www.w3.org/2020/06/machine-learning-workshop/

[4]: https://www.w3.org/blog/2021/04/w3c-launches-the-web-machine-learning-working-group/

[5]: https://github.com/tensorflow/sig-tfjs/pull/2

[6]: https://github.com/opencv/opencv/pull/20406

 

Gecko: No signal

WebKit: No signal

Web developers: Positive (https://github.com/tensorflow/sig-tfjs/pull/2) The RFC of WebNN Delegate for TensorFlow Lite Web got good feedback and approval.

 

Ergonomics

The WebNN graph building (MLGraphBuilder.build [1]) and execution (MLGraph.compute [2]) APIs are synchronous. This is required to implement a backend for Wasm-based ML frameworks, such as ONNX Runtime Execution Provider [3], TensorFlow Lite Delegate [4] and OpenCV.js DNN backend [5]. These frameworks are written in C++ and expect calling sync APIs in its backend implementation. To avoid blocking the main thread, the good practice is to use these async APIs in a worker context. WebML working group is collecting feedback about whether to restrict the sync APIs to only exist in workers [6] and support async APIs [7].

 

[1]: https://www.w3.org/TR/webnn/#dom-mlgraphbuilder-build 

[2]: https://www.w3.org/TR/webnn/#dom-mlgraph-compute  

[3]: https://onnxruntime.ai/docs/execution-providers/add-execution-provider.html 

[4]: https://www.tensorflow.org/lite/performance/implementing_delegate  

[5]: https://github.com/opencv/opencv/pull/20406

[6]: https://github.com/webmachinelearning/webnn/issues/229

[7]: https://github.com/webmachinelearning/webnn/issues/230 

 

Activation

Developers could use webnn-polyfill [1] immediately. The machine learning JS frameworks with WebNN backend would make it easier to use.

 

[1]: https://github.com/webmachinelearning/webnn-polyfill

 

Ethical

The WebNN API spec incorporates ethical considerations. The group is working on a complementary document [1] that outlines ethical issues associated with using Machine Learning on the Web and discusses how implementers and web developers can mitigate associated risks.

 

[1]: https://webmachinelearning.github.io/ethical-webmachinelearning/ 

 

Debuggability

Warnings and errors are exposed via dev tools. Specialized tools for debugging are TBD.

 

Is this feature fully tested by web-platform-tests?

Not yet. Working in progress, e.g., WebNN IDL test merged in wpt [1]

 

[1]: https://github.com/web-platform-tests/wpt/pull/31201

 

Flag name

--enable-webnn

 

Requires code in //chrome?

False

 

Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1273291

 

Estimated milestones

TBD

 

Link to entry on the Chrome Platform Status

https://www.chromestatus.com/feature/5738583487938560

 

This intent message was generated by Chrome Platform Status.




Kaustubha Govind

unread,
Jan 11, 2022, 9:38:48 AMJan 11
to blink-dev, ningxin hu, anssi.ko...@intel.com, Corentin Wallez, Chai Chaoweeraprasit, Rafael Cintron, Daniel LaLiberte, pi...@google.com, Jonathan Bingham, Jeffrey Yasskin, Honglin Yu, Kai Ninomiya, Jon Napper, Zhang, Belem, Fu, Junwei, Dai, Feng
Hi Ningxin,

I am part of a group of Security & Privacy reviewers within the Chrome team who recently reviewed your explainer and design doc. We have some follow-up questions for you, which I will send to your list of email contacts off-thread. We will report back on any findings/recommendations on this thread after our review is complete.

Thank you,
Kaustubha Govind

ningxin hu

unread,
Jan 11, 2022, 8:28:37 PMJan 11
to blink-dev, kaust...@chromium.org, ningxin hu, Anssi Kostiainen, cwa...@chromium.org, Chai Chaoweeraprasit, Rafael Cintron, Daniel LaLiberte, pi...@google.com, Jonathan Bingham, jyas...@chromium.org, Honglin Yu, Kai Ninomiya, Jon Napper, Zhang, Belem, Fu, Junwei, Dai, Feng
Thanks Kaustubha and Chrome team for looking into it. I am excited to work with you on the security and privacy review. We'll discuss your follow-up questions within the W3C WebML WG and get back to you.

Regards,
-ningxin

Reply all
Reply to author
Forward
0 new messages