Intent to Ship: WebXR Raw Camera Access

[Piotr Bialecki]

Contact emails

bia...@chromium.org, alco...@chromium.org, kla...@chromium.org

Explainer

https://github.com/immersive-web/raw-camera-access/blob/main/explainer.md

Specification

https://immersive-web.github.io/raw-camera-access

Design docs

https://github.com/immersive-web/raw-camera-access/blob/main/explainer.md

https://immersive-web.github.io/raw-camera-access

Summary

Allows applications authored against the WebXR Device API to access pose-synchronized camera image textures in the contexts that also allow interacting with other AR features provided by WebXR.

Blink component

Blink>WebXR>AR

TAG review

https://github.com/w3ctag/design-reviews/issues/652

TAG review status

Issues addressed

Risks

Interoperability and Compatibility

There have been discussions happening in Immersive Web Community Group about providing camera access to the applications. One issue about incompatibility of the API shape with form factors other than smartphones has been identified, but the group's consensus was that this can be addressed in further updates to the API. There is interest (both internal and external) in this feature.

Gecko: No signal (https://github.com/mozilla/standards-positions/issues/667)

WebKit: No signal (https://github.com/WebKit/standards-positions/issues/37)

Web developers: No signals (https://github.com/immersive-web/raw-camera-access/issues/13) Some positive signals in internal communications.

Other signals:

Ergonomics

This feature is an extension of WebXR Device API. The feature will require applications to use WebGL to access the data returned by the API.

Activation

The feature is usable as-is, although it will be more accessible to developers if used through libraries like three.js, <model-viewer>, A-Frame, etc.

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?

N/A - WebXR is disabled for WebView in Chrome.

Debuggability

N/A

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

No

This will be supported on platforms where Chrome supports AR. Currently, this is only Android. There are no technical restrictions specific to this API preventing it from being implemented on other platforms.

The specification itself will be implemented in Blink for all platforms, but the spec text leaves room for the UAs to signal that a particular API depends on the device support.

Is this feature fully tested by web-platform-tests?

Yes

Flag name

WebXRIncubations

Requires code in //chrome?

False

Tracking bug

https://crbug.com/1090056

Launch bug

https://crbug.com/1264084

Non-OSS dependencies

Does the feature depend on any code or APIs outside the Chromium open source repository and its open-source dependencies to function?

In Chrome for Android, the feature is internally implemented by using ARCore SDK (https://github.com/google-ar/arcore-android-sdk), which in turn communicates with Google Play Services for AR app. More information can be found at https://developers.google.com/ar.

Sample links

https://storage.googleapis.com/chromium-webxr-test/latest.html?target=proposals/camera-access-barebones.html

Estimated milestones

DevTrial on Android 86

Anticipated spec changes

Open questions about a feature may be a source of future web compat or interop issues. Please list open issues (e.g. links to known github issues in the project for the feature specification) whose resolution may introduce web compat/interop risk (e.g., changing to naming or structure of the API in a non-backward-compatible way).

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5759984304390144

Links to previous Intent discussions

Intent to prototype: https://groups.google.com/a/chromium.org/g/blink-dev/c/4kMGCyCUIao/m/5vjPk4tuAQAJ

[slightlyoff via Chromestatus]

I've left some questions about the design in an issue here: https://github.com/immersive-web/raw-camera-access/issues/14

[Alex Cooper]

Thanks Alex,

I believe I've responded to all of your questions on the github thread, but I still had a few follow up clarifications. We'd been hoping to get this out in M106, but since that has already branched I definitely want to make sure we can get this out in M107.

Thanks,

Alex Cooper

On Fri, Aug 12, 2022 at 4:49 PM slightlyoff via Chromestatus <admin+sl...@cr-status.appspotmail.com> wrote:

I've left some questions about the design in an issue here: https://github.com/immersive-web/raw-camera-access/issues/14

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/00000000000031066705e613ef2f%40google.com.

[Alex Russell]

Hey Alex,

I'd like to approve this ASAP, but the explainer is still lacking critical detail about considered alternatives and some aspects of how the design is going to stay in sync with the options that are available for getUserMedia().

Best,

Alex

On Tuesday, August 23, 2022 at 1:11:35 PM UTC-7 Alexander Cooper wrote:

Thanks Alex,

I believe I've responded to all of your questions on the github thread, but I still had a few follow up clarifications. We'd been hoping to get this out in M106, but since that has already branched I definitely want to make sure we can get this out in M107.

Thanks,

Alex Cooper

On Fri, Aug 12, 2022 at 4:49 PM slightlyoff via Chromestatus <admin+slightlyoff@cr-status.appspotmail.com> wrote:

I've left some questions about the design in an issue here: https://github.com/immersive-web/raw-camera-access/issues/14

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Chris Harrelson]

LGTM1 to ship. In my view, the revised explainer is enough, and the feature received a thorough going-over with the TAG.

Side note: I greatly appreciate the team's responsiveness in discussions with the TAG and in this thread.

On Wed, Aug 24, 2022 at 8:55 AM Alex Russell <sligh...@chromium.org> wrote:

Hey Alex,

I'd like to approve this ASAP, but the explainer is still lacking critical detail about considered alternatives and some aspects of how the design is going to stay in sync with the options that are available for getUserMedia().

Best,

Alex

On Tuesday, August 23, 2022 at 1:11:35 PM UTC-7 Alexander Cooper wrote:

Thanks Alex,

I believe I've responded to all of your questions on the github thread, but I still had a few follow up clarifications. We'd been hoping to get this out in M106, but since that has already branched I definitely want to make sure we can get this out in M107.

Thanks,

Alex Cooper

On Fri, Aug 12, 2022 at 4:49 PM slightlyoff via Chromestatus <admin+sl...@cr-status.appspotmail.com> wrote:

I've left some questions about the design in an issue here: https://github.com/immersive-web/raw-camera-access/issues/14

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/00000000000031066705e613ef2f%40google.com.

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/6810925e-7d33-45f2-bc12-13563340101fn%40chromium.org.

[Mike Taylor]

LGTM2 - thanks for improving the explainer in https://github.com/immersive-web/raw-camera-access/pull/15/files.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOMQ%2Bw8dRgwg%3D%3D8mDkapu6AEyxyxTkK5ciDa4-94YvB3HBNaqA%40mail.gmail.com.

[Yoav Weiss]

LGTM3

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/7ae81c1d-db41-d3cc-09bf-6bc9bd4b1cdf%40chromium.org.

[Alex Cooper]

Thanks Everyone! I've gone ahead and merged PR15 and Piotr will work on the change to enable this!

[Joe Medley]

Is this shipping in 106 or 107?

Joe Medley | Technical Writer, Chrome DevRel | jme...@google.com | 816-678-7195

If an API's not documented it doesn't exist.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGOLbz1vpxiP%3Dj3ceSFNcq%3DEg6tAWYNsteue%3DQbHPvz%2BZHqMcw%40mail.gmail.com.

[Alex Cooper]

M107.