Intent to Ship: WebXR Depth API

[Piotr Bialecki]

Contact emails

bia...@chromium.org, mlam...@chromium.org

Explainer

https://github.com/bialpio/webxr-depth-api/blob/master/explainer.md

Specification

https://immersive-web.github.io/depth-sensing/

API spec

Yes

Design docs

https://github.com/bialpio/webxr-depth-api/blob/master/explainer.md

Summary

The Depth API is an extension to WebXR Device API that allows applications access to depth buffer information that conveys information about the user's environment, with primary focus on Augmented Reality scenarios.

Blink component

Blink

TAG review

https://github.com/w3ctag/design-reviews/issues/550

TAG review status

Pending

Link to origin trial feedback summary

https://docs.google.com/document/d/1x_gVQJuOROarn4bzXBLePozgJM19TaQ_nXJqPOZ8oZ8/

Risks

Interoperability and Compatibility

The feature is incubated in Immersive Web CG to kick off discussions about interoperability and the API design. The design itself took into account ARCore (Android) and ARKit (iOS, marked as Beta) APIs to present an API shape that could be implemented in both of those platforms, and tried to ensure that other platforms that offer higher-level APIs (world reconstruction) could implement it as well.

Gecko: No signal (https://github.com/mozilla/standards-positions/issues/487) WebXR Device API is considered as "worth prototyping": https://github.com/mozilla/standards-positions/issues/218

WebKit: No signal (https://lists.webkit.org/pipermail/webkit-dev/2021-February/031695.html)

Web developers: No signals

Ergonomics

This feature is an extension of WebXR Device API. The use of this particular feature does not depend on any other web features, although WebXR-enabled applications will most likely use WebGL.

Activation

The feature is usable as-is, although it will be more accessible to developers if used through libraries like three.js, <model-viewer>, A-Frame, etc.

Is this feature fully tested by web-platform-tests?

Yes

Tracking bug

https://crbug.com/1119487

Launch bug

https://crbug.com/1131625

Sample links

https://storage.googleapis.com/chromium-webxr-test/latest.html?target=proposals/phone-ar-depth.html
https://storage.googleapis.com/chromium-webxr-test/latest.html?target=proposals/phone-ar-depth-gpu.html

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5742647199137792

Links to previous Intent discussions

Intent to prototype: https://groups.google.com/a/chromium.org/d/msg/blink-dev/U5Iy4LrcwoI/dIlbC2OhAwAJ
Intent to Experiment: https://groups.google.com/a/chromium.org/d/msg/blink-dev/DjucMDeR-og/6yLpNN4wAAAJ

This intent message was generated by Chrome Platform Status.

[yo...@yoav.ws]

On Thursday, February 18, 2021 at 7:34:59 PM UTC+1 bia...@chromium.org wrote:

Contact emails

bia...@chromium.org, mlam...@chromium.org

Explainer

https://github.com/bialpio/webxr-depth-api/blob/master/explainer.md

Specification

https://immersive-web.github.io/depth-sensing/

API spec

Yes

Design docs

https://github.com/bialpio/webxr-depth-api/blob/master/explainer.md

Summary

The Depth API is an extension to WebXR Device API that allows applications access to depth buffer information that conveys information about the user's environment, with primary focus on Augmented Reality scenarios.

Blink component

Blink

TAG review

https://github.com/w3ctag/design-reviews/issues/550

TAG review status

Pending

Link to origin trial feedback summary

https://docs.google.com/document/d/1x_gVQJuOROarn4bzXBLePozgJM19TaQ_nXJqPOZ8oZ8/

Can the feedback be made public?

[Piotr Bialecki]

Publicly accessible link:

https://docs.google.com/document/d/17OmVddq9vWkdyglXWFVE7LcF87htGjrIVW1vbhAuYCs/

[yo...@yoav.ws]

Thanks for sharing the feedback.

It seems like this API hasn't gotten a lot of feedback (not at the OT, nor from the TAG, other vendors and the broader community).

Was this presented to the broader XR community? (e.g. during a WG F2F?)

Any interested partners waiting for this?

The use-case itself seems useful, but I'd love to see some more validation that the API is indeed solving the use-cases it's setting out to solve, and does so in a way that the broader community finds reasnoable.

That's extremely recent.

 

WebXR Device API is considered as "worth prototyping": https://github.com/mozilla/standards-positions/issues/218

WebKit: No signal (https://lists.webkit.org/pipermail/webkit-dev/2021-February/031695.html)

Web developers: No signals

Might be good to collect such signals.

[Piotr Bialecki]

> It seems like this API hasn't gotten a lot of feedback (not at the OT, nor from the TAG, other vendors and the broader community).

> Was this presented to the broader XR community? (e.g. during a WG F2F?)

> Any interested partners waiting for this?

We have discussed the API during our weekly WG/CG calls. The latest version was developed after addressing CG’s concerns about viability of implementation on other devices / frameworks (notably, potential Microsoft’s HoloLens implementation would not be performant when implemented as proposed by the earlier version of the API). The PR that introduced the changes asked for by the CG after going over the issues with the API during the call was reviewed by Facebook’s Oculus developer, and by a contributor to PlayCanvas WebGL game engine, and extended the API to allow for GPU-optimized usage scenarios. The additions to the API surface were non-trivial, but in my view they do not invalidate the feedback we’ve gathered for the CPU-focused scenario (I see them roughly as “this mode does the same thing but you don’t need to upload the data to the WebGL texture yourself”).

> That's extremely recent.

> Might be good to collect such signals.

WebKit’s official position regarding WebXR is “in development” and Apple is an active member of the group. However, Apple never gave an official position on any XR API when asked. We wouldn’t expect this to change. Mozilla lists WebXR as “worth prototyping”, though it’s worth noting that they have it implemented (and launched) in some products. Since then, they paused their work on WebXR so we wouldn’t expect feedback specific to this addition.

The developer response on Twitter was enthusiastic. There is a potential use case for ModelViewer to consume the API to bring WebXR’s backend to parity with SceneViewer.

[Yoav Weiss]

On Wed, Feb 24, 2021 at 8:23 PM Piotr Bialecki <bia...@chromium.org> wrote:

> It seems like this API hasn't gotten a lot of feedback (not at the OT, nor from the TAG, other vendors and the broader community).

> Was this presented to the broader XR community? (e.g. during a WG F2F?)

> Any interested partners waiting for this?

We have discussed the API during our weekly WG/CG calls. The latest version was developed after addressing CG’s concerns about viability of implementation on other devices / frameworks (notably, potential Microsoft’s HoloLens implementation would not be performant when implemented as proposed by the earlier version of the API). The PR that introduced the changes asked for by the CG after going over the issues with the API during the call was reviewed by Facebook’s Oculus developer, and by a contributor to PlayCanvas WebGL game engine, and extended the API to allow for GPU-optimized usage scenarios.

Good to hear that the CG's concerns were addressed and that this was reviewed by Oculus folks (and address Microsoft concerns).

 

The additions to the API surface were non-trivial, but in my view they do not invalidate the feedback we’ve gathered for the CPU-focused scenario (I see them roughly as “this mode does the same thing but you don’t need to upload the data to the WebGL texture yourself”).

> That's extremely recent.

> Might be good to collect such signals.

WebKit’s official position regarding WebXR is “in development” and Apple is an active member of the group. However, Apple never gave an official position on any XR API when asked. We wouldn’t expect this to change. Mozilla lists WebXR as “worth prototyping”, though it’s worth noting that they have it implemented (and launched) in some products. Since then, they paused their work on WebXR so we wouldn’t expect feedback specific to this addition.

The developer response on Twitter was enthusiastic. There is a potential use case for ModelViewer to consume the API to bring WebXR’s backend to parity with SceneViewer.

That's great! Can you add links to that to the developer signals bit in chromestatus and post them here as well?

 

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/071f9e0c-341e-401c-af46-bf232507eb05n%40chromium.org.

[Piotr Bialecki]

Quick Twitter search yielded the tweets (+ retweets / quote tweets):

https://twitter.com/mrmaxm/status/1329151944540254208

https://twitter.com/Tojiro/status/1329925041299984387
https://twitter.com/webmixedreality/status/1296551840834818050

https://twitter.com/DePanther/status/1318181914428268544

I've updated the chromestatus entry with the above and marked "Web developers" signals as "Positive".

[Mounir Lamouri]

Hi,

I think there is a misunderstanding about the outreach that was done for this API.

Comments were requested to the Immersive Web CG/WG participants and we received a lot of feedback that was acted on. The most obvious case is issue #2 where we asked for feedback to the main participants and received some from Facebook, Microsoft, Mozilla, 8th Wall, and PlayCanvas contributor. The API was also on the agenda of a call and discussed by mostly the same participants (Facebook, Mozilla, 8th Wall). Further discussions happened here, here, and here. Based on the feedback we received, we changed the API last month.

I believe we did the required outreach, received a substantial amount of feedback, and applied it.

-- Mounir

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/725dabb8-e3e1-4230-8f98-c68f36c1e66bn%40chromium.org.

[Chris Harrelson]

Thanks Piotr and Mounir for those data points.

With this additional indication of outreach and consensus-building, LGTM1

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2B-LeH-QDwwora3H9xnWc0k%3DQGVJCBEhVDp5SPY%2B4%3DNE6QZoJw%40mail.gmail.com.

[Yoav Weiss]

LGTM2

[TAMURA, Kent]

LGTM3

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACj%3DBEiGa4d3z-psqQoFUdxSxFEn5_2ur6-_acjZmB2W3%3D7Yow%40mail.gmail.com.

--

TAMURA Kent
Software Engineer, Google

[Joe Medley]

Are you shipping this in 90?

[Piotr Bialecki]

Yes, the plan is to ship in M90.