Intent to Implement: WebXR Hit-test

[David Dorwin]

Contact emails

linco...@chromium.org, johnp...@chromium.org

Spec

Explainer: https://github.com/immersive-web/hit-test/blob/master/explainer.md

The incubations is still early, so we have not yet requested a TAG review.

Summary

The Hit-test API is an extension to the WebXR Device API that provides raycasting into the world understanding of the underlying XR device and system. The idea is to provide a simple API that allows applications to place objects in the real world by casting rays that can be evaluated against the underlying platform's understanding of real world geometry, regardless of how that is implemented (e.g., hit planes, point clouds).

Motivation

Native AR devices and capabilities are being released, especially on mobile where over 100 million devices are already capable of supporting AR applications. Currently, there is no way for web applications to access this technology or create AR experiences of comparable quality in arbitrary environments. Currently, creating AR applications on the web involves acquiring camera frames and the device orientation then applying custom algorithms to establish world understanding and 3D device pose in the real world. The lack of associated timestamps and limited frequency at which sensor data is exposed to the web limit the effectiveness of such approaches. The native capabilities, on the other hand, are more precise, performant, and provide a consistent experience that does not vary across applications. 

Some of the early use cases for AR applications are extensions of existing use cases, such as shopping, where the web is already strong and/or are a good fit for the web’s ephemeral nature.

Interoperability risk

Firefox: Public support

Edge: Public support

Safari: No public signals

Web developers: Positive

Firefox, Edge, and others are active participants in the CG. This API is a minimal extension to WebXR Device API. It could be easily removed without affecting VR applications, but doing so would break any AR applications unless a replacement is provided.

Compatibility risk

This is a new interface and does not affect existing content. There are currently no AR applications (based on standards-track APIs).

WebXR extensibility, especially related to detecting feature support and creating specific types of sessions is still an open issue. This will need to be resolved before exposing any AR functionality other than behind a flag.

Ongoing technical constraints

For Android, we will be adding ARCore integration, which is going through the Chrome process. Providing the camera frame of the real world to be composited with the application’s augmentation requires some plumbing but is fairly self-contained. This is also covered separately by the ARCore integration work.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux,

Chrome OS, Android, and Android WebView)?

No. At this time, we only plan to support AR capabilities on Android devices that support ARCore. (As with WebXR Device API, Android WebView support will likely come later for implementation reasons.) We will re-evaluate as hardware becomes available for other platforms (likely via the OpenVR or OpenXR SDKs). We will likely expose the IDL on all platforms but report that AR sessions are not supported via the appropriate WebXR Device API mechanism.

OWP launch tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=833633

Link to entry on the Chrome Platform Status

https://www.chromestatus.com/features/4755348300759040

Requesting approval to ship?

No

[andy...@chromium.org]

Hello,

I think the "Privacy and Security" section will need a lot of work. This API seems like it could be very intrusive and present lots of risks. In particular some risks are pointed out but I don't think they are really addressed.

Have you started considering how these issues will be addressed?

I understand that the API is in its early stages but the sooner the better when it comes to security and privacy.

Regards,

Andy Paicu

[David Dorwin]

Yes, we have been considering the risks from the beginning and have been working with the appropriate Chrome teams. Work in this area has picked up recently, and the CG has created a repo specifically for issues related to privacy and security in the immersive web. The work there will feed into the individual specs as appropriate. There are a few issues already filed, and we expect to have some initial explainer content soon.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/d3b99d75-0884-4d4c-85b2-a691b04681e1%40chromium.org.