Intent to Ship: WebXR Hit-test

133 views
Skip to first unread message

Piotr Bialecki

unread,
Jan 22, 2020, 10:27:58 PM1/22/20
to blink-dev
mlam...@chromium.org,bia...@chromium.org https://github.com/immersive-web/hit-test/blob/master/hit-testing-explainer.md https://immersive-web.github.io/hit-test/ https://github.com/w3ctag/design-reviews/issues/463 The Hit-test API is an extension to the WebXR Device API that provides the capability of raycasting into the real world as understood by the underlying XR system. The goal is to provide a simple API that allows apps to place objects in the real world by casting rays that can be evaluated against the underlying platform's understanding of the real world geometry, regardless of how that is implemented (e.g., hit test against detected planes, point clouds, meshes, etc.). https://groups.google.com/a/chromium.org/d/msg/blink-dev/HYTqcRJu_aU/Dz0QTOGWCgAJ
This WebXR module is adding the ability to request hit tests against the world. It is a core feature for AR experiences and has been discussed in the Immersive Web Community Group and Working Group for a while. There is broad interest in this feature and we expect it to be implemented by other browsers in the group. One configuration option is still under discussion and is not going to launch (https://github.com/immersive-web/hit-test/issues/66). There was also a recently opened low risk issue about naming (https://github.com/immersive-web/hit-test/issues/75). Firefox: Public support (https://github.com/immersive-web/hit-test/issues/74#issuecomment-576937422) Response from one of main Mozilla contributors, not an official position from the company. Edge: Public support (https://github.com/immersive-web/hit-test/issues/74#issuecomment-577067376) Safari: No public signals Web developers: Positive Libraries such as Model Viewer (https://modelviewer.dev/examples/augmented-reality.html) and A-Frame (https://aframe.io/blog/webxr-ar-module/) have been implementing AR experiences behind experimental flags as it is a commonly requested feature. These experiences are directly (model-viewer) or indirectly (A-Frame) using Hit Tests. This feature is an extension of WebXR Device API. The use of this particular feature does not depend on any other web features, although WebXR-enabled applications will most likely use WebGL. The feature is usable as-is, although it will be more accessible to developers if used through libraries like three.js, <model-viewer>, A-Frame, etc. Some of these APIs already have experimental usage of the API.
No This will be supported on platforms where Chrome supports AR. Currently, this is only Android. WebView support is planned. There are no technical restrictions specific to this API preventing it from being implemented on other platforms. The specification itself will be implemented in Blink for all platforms, but the spec text leaves room for the UAs to signal that a particular API depends on the device support. Yes Results: https://wpt.fyi/results/webxr/hit-test?label=experimental&label=master&aligned Tests: https://github.com/web-platform-tests/wpt/tree/master/webxr/hit-test https://bugs.chromium.org/p/chromium/issues/detail?id=833633 https://storage.googleapis.com/chromium-webxr-test/latest.html?target=proposals/phone-ar-hit-test.html https://chromestatus.com/feature/4755348300759040
This intent message was generated by Chrome Platform Status.

Yoav Weiss

unread,
Jan 27, 2020, 11:13:01 AM1/27/20
to Piotr Bialecki, blink-dev
It's a shame that the TAG review wasn't filed earlier...
 
The Hit-test API is an extension to the WebXR Device API that provides the capability of raycasting into the real world as understood by the underlying XR system. The goal is to provide a simple API that allows apps to place objects in the real world by casting rays that can be evaluated against the underlying platform's understanding of the real world geometry, regardless of how that is implemented (e.g., hit test against detected planes, point clouds, meshes, etc.).

So, IIUC, this API allows the device to locate physical objects in the user's environment. Is that correct?
What's the user's opt-in model to that? The use-case seems critical for AR, but can also reveal information about the user

https://groups.google.com/a/chromium.org/d/msg/blink-dev/HYTqcRJu_aU/Dz0QTOGWCgAJ
This WebXR module is adding the ability to request hit tests against the world. It is a core feature for AR experiences and has been discussed in the Immersive Web Community Group and Working Group for a while. There is broad interest in this feature and we expect it to be implemented by other browsers in the group. One configuration option is still under discussion and is not going to launch (https://github.com/immersive-web/hit-test/issues/66). There was also a recently opened low risk issue about naming (https://github.com/immersive-web/hit-test/issues/75). Firefox: Public support (https://github.com/immersive-web/hit-test/issues/74#issuecomment-576937422) Response from one of main Mozilla contributors, not an official position from the company.

That's doesn't count as "public support" (but could probably pass as "positive signals")
Is there a Mozilla standards position issue open for this?
 
Edge: Public support (https://github.com/immersive-web/hit-test/issues/74#issuecomment-577067376) Safari: No public signals

Did we reach out? Are they involved in WebXR efforts?
 
Web developers: Positive Libraries such as Model Viewer (https://modelviewer.dev/examples/augmented-reality.html) and A-Frame (https://aframe.io/blog/webxr-ar-module/) have been implementing AR experiences behind experimental flags as it is a commonly requested feature. These experiences are directly (model-viewer) or indirectly (A-Frame) using Hit Tests. This feature is an extension of WebXR Device API. The use of this particular feature does not depend on any other web features, although WebXR-enabled applications will most likely use WebGL. The feature is usable as-is, although it will be more accessible to developers if used through libraries like three.js, <model-viewer>, A-Frame, etc. Some of these APIs already have experimental usage of the API.
No This will be supported on platforms where Chrome supports AR. Currently, this is only Android. WebView support is planned. There are no technical restrictions specific to this API preventing it from being implemented on other platforms. The specification itself will be implemented in Blink for all platforms, but the spec text leaves room for the UAs to signal that a particular API depends on the device support. Yes Results: https://wpt.fyi/results/webxr/hit-test?label=experimental&label=master&aligned Tests: https://github.com/web-platform-tests/wpt/tree/master/webxr/hit-test https://bugs.chromium.org/p/chromium/issues/detail?id=833633 https://storage.googleapis.com/chromium-webxr-test/latest.html?target=proposals/phone-ar-hit-test.html https://chromestatus.com/feature/4755348300759040
This intent message was generated by Chrome Platform Status.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/8bdc503b-02ef-40f9-af54-27ee65d6fe3c%40chromium.org.

Mounir Lamouri

unread,
Jan 27, 2020, 2:23:35 PM1/27/20
to Yoav Weiss, Piotr Bialecki, blink-dev
On Mon, 27 Jan 2020 at 08:13, Yoav Weiss <yo...@yoav.ws> wrote:
On Thu, Jan 23, 2020 at 4:28 AM Piotr Bialecki <bia...@chromium.org> wrote:

It's a shame that the TAG review wasn't filed earlier...

Yes. We waited to be sure to have an API shape that was reviewed by active members of the group as we didn't want to give the impression that we were going ahead with something that they didn't have the time to evaluate. The holiday break slowed things down. Given that this API is a module on top of the more complex WebXR API and that it was reviewed by experts of WebXR, we wouldn't expect to receive large substantive feedback but if it were to happen, we are, as usual, willing to change the API accordingly.
 
The Hit-test API is an extension to the WebXR Device API that provides the capability of raycasting into the real world as understood by the underlying XR system. The goal is to provide a simple API that allows apps to place objects in the real world by casting rays that can be evaluated against the underlying platform's understanding of the real world geometry, regardless of how that is implemented (e.g., hit test against detected planes, point clouds, meshes, etc.).

So, IIUC, this API allows the device to locate physical objects in the user's environment. Is that correct?
What's the user's opt-in model to that? The use-case seems critical for AR, but can also reveal information about the user

XR sessions have to be requested and UA can use a prompt. Chrome has its own prompt for AR that lets the user know that AR can be used to know about the geometry of its environment. However, hit test isn't the ideal way to do this. Using hit tests may help one figure out where walls are but it wouldn't be good enough to figure out details. Worth noting that the specification allows the UA to block requests if there are too many for regular use cases in order to mitigate the potential threat. Chrome doesn't do this at the moment but may if abuse is seen in the wild.

https://groups.google.com/a/chromium.org/d/msg/blink-dev/HYTqcRJu_aU/Dz0QTOGWCgAJ
This WebXR module is adding the ability to request hit tests against the world. It is a core feature for AR experiences and has been discussed in the Immersive Web Community Group and Working Group for a while. There is broad interest in this feature and we expect it to be implemented by other browsers in the group. One configuration option is still under discussion and is not going to launch (https://github.com/immersive-web/hit-test/issues/66). There was also a recently opened low risk issue about naming (https://github.com/immersive-web/hit-test/issues/75). Firefox: Public support (https://github.com/immersive-web/hit-test/issues/74#issuecomment-576937422) Response from one of main Mozilla contributors, not an official position from the company.

That's doesn't count as "public support" (but could probably pass as "positive signals")
Is there a Mozilla standards position issue open for this?
 
Even though Mozilla doesn't officially support the spec in their standards position tracker, they are implementing it as we speak.
 

Did we reach out? Are they involved in WebXR efforts?

Yes and yes. Apple is a member of the WG. Dean Jackson usually sits at face to face and meetings. We also asked him and other implementers for feedback on https://github.com/immersive-web/hit-test/issues/74. We did not hear back from Apple.
 
 
Web developers: Positive Libraries such as Model Viewer (https://modelviewer.dev/examples/augmented-reality.html) and A-Frame (https://aframe.io/blog/webxr-ar-module/) have been implementing AR experiences behind experimental flags as it is a commonly requested feature. These experiences are directly (model-viewer) or indirectly (A-Frame) using Hit Tests. This feature is an extension of WebXR Device API. The use of this particular feature does not depend on any other web features, although WebXR-enabled applications will most likely use WebGL. The feature is usable as-is, although it will be more accessible to developers if used through libraries like three.js, <model-viewer>, A-Frame, etc. Some of these APIs already have experimental usage of the API.
No This will be supported on platforms where Chrome supports AR. Currently, this is only Android. WebView support is planned. There are no technical restrictions specific to this API preventing it from being implemented on other platforms. The specification itself will be implemented in Blink for all platforms, but the spec text leaves room for the UAs to signal that a particular API depends on the device support. Yes Results: https://wpt.fyi/results/webxr/hit-test?label=experimental&label=master&aligned Tests: https://github.com/web-platform-tests/wpt/tree/master/webxr/hit-test https://bugs.chromium.org/p/chromium/issues/detail?id=833633 https://storage.googleapis.com/chromium-webxr-test/latest.html?target=proposals/phone-ar-hit-test.html https://chromestatus.com/feature/4755348300759040
This intent message was generated by Chrome Platform Status.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/8bdc503b-02ef-40f9-af54-27ee65d6fe3c%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

Yoav Weiss

unread,
Jan 28, 2020, 4:52:30 PM1/28/20
to Mounir Lamouri, Piotr Bialecki, blink-dev
On Mon, Jan 27, 2020 at 8:23 PM Mounir Lamouri <mlam...@google.com> wrote:
On Mon, 27 Jan 2020 at 08:13, Yoav Weiss <yo...@yoav.ws> wrote:

Looking at the spec, I see multiple sections that are marked as "unstable".
Is that description accurate? What's our level of confidence that the API is in its final shape?

Mounir Lamouri

unread,
Jan 28, 2020, 6:07:06 PM1/28/20
to Yoav Weiss, Piotr Bialecki, blink-dev
As Piotr mentioned in the original email, this is a configuration option that triggered a few discussions with very different opinions between Mozilla, Microsoft, and Google (https://github.com/immersive-web/hit-test/issues/66). Because this wasn't required for an initial launch, we decided to ship without it and mark it as unstable in the spec so we can discuss it without any time pressure. Because this option was trying to link with entity types that do not have their own specification yet (planes, meshes), we expect a design to crystallise as these entities get specified.

-- Mounir

Yoav Weiss

unread,
Jan 29, 2020, 4:04:14 AM1/29/20
to Mounir Lamouri, Piotr Bialecki, blink-dev
Thanks for clarifying! :)
LGTM1 assuming that we'd be willing to change API shape if the discussions on naming or other issues result in such conclusions.

TAMURA, Kent

unread,
Jan 29, 2020, 7:48:30 PM1/29/20
to Yoav Weiss, Mounir Lamouri, Piotr Bialecki, blink-dev
LGTM2


To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACj%3DBEjCmYz6WGfYzn%2BKqWWY3sW1b6GhCQz7hexOvktJV181SA%40mail.gmail.com.


--
TAMURA Kent
Software Engineer, Google


Chris Harrelson

unread,
Jan 29, 2020, 8:21:32 PM1/29/20
to TAMURA, Kent, Yoav Weiss, Mounir Lamouri, Piotr Bialecki, blink-dev
LGTM3

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGH7WqEE4-MBV-r_9PKL-xhO6jOYo9%3D%3DNiNNTMiZUakvjw6GrA%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages