Intent to Prototype: Web Machine Learning: Model Loader API

246 views
Skip to first unread message

Jonathan Bingham

unread,
Dec 3, 2021, 2:51:00 PM12/3/21
to blin...@chromium.org

Contact emails

bing...@google.com, hong...@google.com, nap...@google.com


Explainer

https://github.com/webmachinelearning/model-loader/blob/master/explainer.md


Specification

https://webmachinelearning.github.io/model-loader/#api


Summary

Run machine learning models natively, taking advantage of hardware acceleration. Currently, javascript libraries must parse a model and call WebGL, WebGPU and WASM for the individual compute operations. Providing a model loader API gains performance by optimizing in the browser and potentially taking advantage of new ML processors like TPUs.



Blink component

Blink>WebML


Motivation

Model Loader is a proposed web API to load a custom, pre-trained machine learning model in a standard format, compile it for the available hardware, and apply it to example data in JavaScript in order to perform inference, like classification, regression, or ranking. The idea is to make it as easy as possible for web developers to use a custom, pre-built machine learning model in their web app, across devices and browsers.


Performing inference locally can:


- Preserve privacy, by not shipping user data across the network

- Improve performance, by eliminating network latency, running models natively and taking advantage of hardware acceleration, including specialized hardware not available with WebGL, WebGPU or WASM.

- Provide a fallback if network access is unavailable, possibly using a smaller and lower quality model


The graph-based Web NN API and the Model Loader API are complementary approaches. We'll need to do some benchmarking to understand if there are performance differences, and get feedback from developers to see if it's valuable to offer both types of API.


Initial public proposal

https://webmachinelearning.github.io/model-loader-intro/


TAG review



TAG review status

Pending


Risks


The Model Loader API is currently included as a Tentative Specification in the Web Machine Learning Working Group Charter.



Interoperability and Compatibility



Gecko: No signals


WebKit: No signals



Chrome/TensorFlow: Positive


Intel : Collaborating in Working Group


Microsoft/ONNX: Collaborating in Working Group


PyTorch: No signals


Salesforce: Collaborating in Working Group




Debuggability



Is this feature fully tested by web-platform-tests?

No


Flag name



Requires code in //chrome?

False


Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1263240


Estimated milestones

No milestones specified



Link to entry on the Chrome Platform Status

https://www.chromestatus.com/feature/6505078216196096


This intent message was generated by Chrome Platform Status.

Victor Miura

unread,
Dec 3, 2021, 5:24:17 PM12/3/21
to Jonathan Bingham, blin...@chromium.org
This specification references "tflite" model files.  Could you please add a reference to the specification of the model format that would be exposed?

Thanks,
Victor

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAEK6eFxSmK6ajiQ6_7rhPspiaDXO3xM3uNJnbSgCobQ_nk%2Bxpg%40mail.gmail.com.


--
I support flexible work schedules, and I’m sending this email now because it is within the hours I’m working today.  Please do not feel obliged to reply straight away - I understand that you will reply during the hours you work, which may not match mine. (credit: jparent@)

Jonathan Bingham

unread,
Dec 3, 2021, 9:17:15 PM12/3/21
to Victor Miura, Honglin Yu, blin...@chromium.org
Absolutely. We'll update it.

Thanks!
Jonathan
Reply all
Reply to author
Forward
0 new messages