Intent to Implement: Shape Detection API

[Miguel Casas-Sanchez]

Contact emails mca...@chromium.org,xianglu@chromium.org Spec https://wicg.github.io/shape-detection-api/ Summary Photos and images constitute the largest chunk of the Web, and many include recognisable features, such as human faces or QR codes. Hardware manufacturers have long been providing support for such operations given their (high) computational complexity. The Shape Detection API allows accessing these hardware-accelerated detectors where available. (Note that it does _not_ provide any software fallback)

Motivation

Detection of features in images is done in JS libraries which are inherently slow due to high computational complexity. This API surfaces hardware capabilities where available.

Interoperability risk Firefox: Public support Edge: No public signals Safari: No public signals Web developers: Positive

The API closely mimics the underlying platforms which are provided by stable, widespread mobile platforms, so not much change is expected in this front and, consequently, the (future potential) interoperability risk is low.

Websites use JS libraries which might provide slightly different recognition data events or characteristics, but the underlying functionality is clearly understood now.

Ongoing technical constraints

Only a few platforms implement this feature, e.g. Android and AVFoundation (both in iOS and Mac). Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)? Yes or no.

No.

Rationale: It will only be available where there is Hw/OS support for it, namely: certain Android, AVFoundation (Mac, iOS).

OWP launch tracking bug https://crbug.com/646035 Link to entry on the Chrome Platform Status https://www.chromestatus.com/features/4757990523535360 Requesting approval to ship?

No.

[Stephen Chenney]

This feature requires a design doc. It is not clear to me how you will implement it with decent performance, given many images and certainly video never leave the compositor.

Stephen.

[Alex Russell]

Many of the use-cases are scoped to video which comes from getUserMedia and which may have hardware support for the detection (without pulling the frames back into the document's main thread).

I'm hopeful that an explainer document will be added to the proposal repo to help clarify which problems are being solved and which are out of scope.

Regards

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Rick Byers]

Exposing the capabilities from Android and iOS here sounds great to me!

Have you started the process of figuring out at the various vendors or in the various related working groups might have the interest and expertise to collaborate on this?  Ultimately we'll probably want some support from at least one other browser before shipping, so trying to drum up some other browser interest once you have a good high-level explainer is probably a good idea.

Rick

[Miguel Casas-Sanchez]

DD document is being written in  http://tinyurl.com/shape-detection-in-chromium.