Intent to Ship: WebXR dynamic viewport scaling

[Klaus Weidner]

Contact emails

kla...@chromium.org

Explainer

https://github.com/immersive-web/webxr/blob/master/explainer.md#dynamic-viewport-scaling

Specification

https://immersive-web.github.io/webxr/#dom-xrview-requestviewportscale

Design docs

https://github.com/immersive-web/webxr/blob/master/explainer.md#dynamic-viewport-scaling

Summary

Applications can optionally render to a subset of the WebXR viewport, using a scale factor that can be changed every animation frame. This is intended to be more efficient than resizing the full framebuffer which requires reallocation, and the UA can supply a recommended scale factor based on internal heuristics.

Blink component

Blink>WebXR

Search tags

WebXR

TAG review

None

TAG review status

Not applicable

Risks

Interoperability and Compatibility

This is a small addition to the WebXR API, there is consensus in the working group and agreement to go forward with it. The API is designed to be trivially easy to implement as a no-op. The UA can simply ignore the application-requested scale factor, and providing a suggested scale factor is also optional.

Gecko: No signal

WebKit: No signal

Web developers: No signals This feature was strongly requested by <model-viewer>, and initial feedback about the implementation was positive.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

No

The API is available on all WebXR-supported platforms, but this is currently a no-op on Windows VR platforms. It's supported for Android for AR and VR.

Is this feature fully tested by web-platform-tests?

Yes

Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1133381

Launch bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1135292

Sample links

https://vps3a.glitch.me/
https://modelviewer.dev/examples/webxr.html

Link to entry on the Chrome Platform Status

https://www.chromestatus.com/feature/5640976515203072

This intent message was generated by Chrome Platform Status.

[Alex Russell]

Hey Klaus,

Thanks for filing the intent. You list no TAG review, which is a bit of a red flag. Was this feature covered by a different review, perhaps? I know that the XR interfaces have been a topic of continual TAG attention since I served, which is now several years ago. When was this method added to the draft spec? A quick look suggests that this feature was in previous drafts, dropped, and is now being revived under this signature? Is that correct?

    https://github.com/immersive-web/webxr/issues?q=is%3Aissue+requestViewportScale

Regarding developer enthusiasm, would be great to either hear from developers here (as we're the first being asked to ship) or from the results of a completed Origin Trial. As I don't see an OT for this, can you perhaps outline why that wasn't pursued here?

Thanks

[Klaus Weidner]

[Updated Cc: for Chris and Emmett using their @google.com emails.]

On Fri, Nov 6, 2020 at 11:46 AM Alex Russell <sligh...@chromium.org> wrote:

Hey Klaus,

Thanks for filing the intent. You list no TAG review, which is a bit of a red flag. Was this feature covered by a different review, perhaps?

I was under the impression that this doesn't necessarily need a separate TAG review due to being a small modification of an existing API that has cross browser support. What's the recommended way to proceed here?

 

I know that the XR interfaces have been a topic of continual TAG attention since I served, which is now several years ago. When was this method added to the draft spec? A quick look suggests that this feature was in previous drafts, dropped, and is now being revived under this signature? Is that correct?

    https://github.com/immersive-web/webxr/issues?q=is%3Aissue+requestViewportScale

You're right about the history. Viewport scaling was originally intended to be part of the WebXR specification since the previous WebVR spec had a similar feature. The proposal to remove it was issue #617 from Apr 2019, and my impression was that this was largely due to it needing a more precise specification, concerns about implementation correctness, and missing a feedback mechanism, so the working group agreed to defer it until a later time to avoid delaying the overall WebXR spec. 

The proposal to bring it back was in issue #1091 from Jun 2020. We had discussions in that issue, and it was covered in detail in three of the group's virtual meetings to get details sorted out, including a final wrap-up after the change was merged into the specification to ensure all participants were aware and had a chance to provide feedback.

Overall I think we were able to reach a consensus that this addressed the previously open issues. The timing of operations is now clearly specified. UAs have to actively opt in to support this which reduces the risk of it not working right on some platforms, and there's now an optional feedback mechanism via recommendedViewportScale.

Regarding developer enthusiasm, would be great to either hear from developers here (as we're the first being asked to ship) or from the results of a completed Origin Trial. As I don't see an OT for this, can you perhaps outline why that wasn't pursued here?

This feature was strongly requested by <model-viewer>, and we were working closely with them for the API and implementation. <model-viewer> uses fairly complex shaders to get realistic model rendering, and frame times can vary dramatically depending on how much of the viewport is covered by the model. <model-viewer>'s non-AR canvas view uses dynamic scaling to address this, and the native scene viewer AR application also does so. Dynamic viewport scaling was considered to be an essential feature to ensure that WebXR rendering can get comparable results to a native AR application.

Thank you for the feedback!

[Emmett Lalish]

Yes, I've been pressing hard for this viewport scaling feature as <model-viewer> currently has significantly lagging performance in WebXR as compared to both its on-page rendering and as compared to the native Scene Viewer app. It has been fairly standard practice for some time to dynamically reduce resolution to maintain frame rate, as GPUs only spend time on pixel blocks that have things to render, which in AR is usually not most of them. Only when you have a huge object or get very close to something are you having to run your shaders on all the pixels. Decreased resolution (especially while moving) is much less noticeable than decreased frame rate, and dynamic scaling is the only reasonable way to solve this, since performance varies dramatically due to camera position. 

We have already implemented and tested support for this API by enabling it with the Chrome Canary flag. It works great and exactly as expected. Gives about a 4x increase in frame rate in the worst cases.

Thanks,

-Emmett

On Mon, Nov 9, 2020 at 2:55 PM Tifow Subane <tifo...@gmail.com> wrote:

RGFIIUU6R8U6UUU

[Chris Harrelson]

On Mon, Nov 9, 2020 at 3:28 PM 'Emmett Lalish' via blink-dev <blin...@chromium.org> wrote:

Yes, I've been pressing hard for this viewport scaling feature as <model-viewer> currently has significantly lagging performance in WebXR as compared to both its on-page rendering and as compared to the native Scene Viewer app. It has been fairly standard practice for some time to dynamically reduce resolution to maintain frame rate, as GPUs only spend time on pixel blocks that have things to render, which in AR is usually not most of them. Only when you have a huge object or get very close to something are you having to run your shaders on all the pixels. Decreased resolution (especially while moving) is much less noticeable than decreased frame rate, and dynamic scaling is the only reasonable way to solve this, since performance varies dramatically due to camera position. 

We have already implemented and tested support for this API by enabling it with the Chrome Canary flag. It works great and exactly as expected. Gives about a 4x increase in frame rate in the worst cases.

Thanks,

-Emmett

On Mon, Nov 9, 2020 at 2:55 PM Tifow Subane <tifo...@gmail.com> wrote:

RGFIIUU6R8U6UUU

On Monday, November 9, 2020 at 2:03:01 PM UTC-8 Klaus Weidner wrote:

[Updated Cc: for Chris and Emmett using their @google.com emails.]

On Fri, Nov 6, 2020 at 11:46 AM Alex Russell <sligh...@chromium.org> wrote:

Hey Klaus,

Thanks for filing the intent. You list no TAG review, which is a bit of a red flag. Was this feature covered by a different review, perhaps?

I was under the impression that this doesn't necessarily need a separate TAG review due to being a small modification of an existing API that has cross browser support. What's the recommended way to proceed here?

Even small changes might have significant problems that are not obvious without deeper review, so you should default to a TAG review in such cases.

If the new behavior is already specified *and* shipping in another browser, then skipping the TAG review is usually ok.

My suggestion is to file a combined TAG review for this intent as well as the two other WebXR ones.

Please file for official signals on these.

 

Web developers: No signals This feature was strongly requested by <model-viewer>, and initial feedback about the implementation was positive.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

No

The API is available on all WebXR-supported platforms, but this is currently a no-op on Windows VR platforms. It's supported for Android for AR and VR.

Is this feature fully tested by web-platform-tests?

Yes

Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1133381

Launch bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1135292

Sample links

https://vps3a.glitch.me/
https://modelviewer.dev/examples/webxr.html

Link to entry on the Chrome Platform Status

https://www.chromestatus.com/feature/5640976515203072

This intent message was generated by Chrome Platform Status.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAKTKFrauU5u8s%3Dbr6g7NKO2QAyG-%3DZe2giYZCrvyho8e7sNjnA%40mail.gmail.com.

[Alex Russell]

Thanks for the feedback, Emmet, it does help make the case that this is valuable. Do you know of other folks who have used this behind a flag or are anticipating a launch?

Klaus:

I agree with Chris's suggestion regarding a more unified request to the TAG, but again, if there's some history here that you can show of the feature being previously reviewed, or if +Alice Boxhall or +Kenneth Rohde Christiansen can comment on their perspective of this feature's longer arc, that might help reduce the need for a fresh review. But as Chris mentioned, our default position is that all features should have reviews.

On Wed, Nov 11, 2020 at 10:16 AM Chris Harrelson <chri...@chromium.org> wrote:

On Mon, Nov 9, 2020 at 3:28 PM 'Emmett Lalish' via blink-dev <blin...@chromium.org> wrote:

Yes, I've been pressing hard for this viewport scaling feature as <model-viewer> currently has significantly lagging performance in WebXR as compared to both its on-page rendering and as compared to the native Scene Viewer app. It has been fairly standard practice for some time to dynamically reduce resolution to maintain frame rate, as GPUs only spend time on pixel blocks that have things to render, which in AR is usually not most of them. Only when you have a huge object or get very close to something are you having to run your shaders on all the pixels. Decreased resolution (especially while moving) is much less noticeable than decreased frame rate, and dynamic scaling is the only reasonable way to solve this, since performance varies dramatically due to camera position. 

We have already implemented and tested support for this API by enabling it with the Chrome Canary flag. It works great and exactly as expected. Gives about a 4x increase in frame rate in the worst cases.

Thanks,

-Emmett

[Mounir Lamouri]

Regarding the TAG reviews, the Lighting Estimation API will get its own review and the interface we want to launch should be covered by the WebXR Layers review. I wouldn't bundle these together as they are fairly unrelated and aren't even part of the same document/specifications.

This change itself is an incremental change in the main WebXR spec, adding one method and one attribute. We could file a TAG review for it if it is deemed necessary but given the size of this change, I'm wondering if it may not just add up onto the TAG load without a clear expected benefit. In other words, should any web visible change require a TAG review even if the main spec was already reviewed?

Regarding the value, Emmett can provide specific feedback from developers using Model Viewer but we also received feedback from developers using native solutions that WebXR was slow and viewport scaling has been identified as the main reason why: native solutions are doing viewport scaling and it isn't possible in WebXR without this change.

-- Mounir

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CANr5HFXG%2B-9h3uySDS2puZsrHEx9%3D2jySbTtCL8tT0RQoaVOew%40mail.gmail.com.

[Alex Russell]

Hey all,

So, there isn't a way in our process today to skip filing for TAG review for new features, small or not, and in particular not when we're being asked to ship something first. We've been discussing for some time what such an exception might look like and we're actually working through a potential test case with a separate feature right now. I'm hopeful that we'll have some sort of text available in the next month that brings some clarity.

Until then, I suggest we get a review filed for this feature ASAP, and if M88 is somehow a hard requirement for folks, that we get an Origin Trial in place for the feature.

Regards

[Klaus Weidner]

I have requested a TAG review for dynamic viewport scaling at https://github.com/w3ctag/design-reviews/issues/588 .

We're now targeting M89 for the feature. I don't have specific information about developers other than <model-viewer> using the feature behind a flag at this time, I'll reach out again to see if I can get more feedback.

[Mounir Lamouri]

On Wed, 16 Dec 2020 at 15:40, Klaus Weidner <kla...@google.com> wrote:

We're now targeting M89 for the feature. I don't have specific information about developers other than <model-viewer> using the feature behind a flag at this time, I'll reach out again to see if I can get more feedback.

As mentioned before, because this API is meant to allow web developers to improve the performance of their application when using AR, we consider this something that everyone should be using. The Model Viewer project saw drastic improvements on low end devices where the default sized canvas setup by WebXR didn't allow the device to keep up at 30fps.

-- Mounir

[Klaus Weidner]

chri...@chromium.org wrote:

Please file for official signals on these.

Sorry, I had initially missed this in the response. I've filed requests for official signals:

 

  https://github.com/mozilla/standards-positions/issues/473

  https://lists.webkit.org/pipermail/webkit-dev/2021-January/031663.html

I haven't heard back from the TAG review so far. Would it be OK to proceed with enabling the feature for the 89 branch (see https://chromium-review.googlesource.com/c/chromium/src/+/2518764), and if needed revert that if the launch approval isn't ready in time for 89 stable?

[Chris Harrelson]

On Mon, Jan 11, 2021 at 6:37 PM 'Klaus Weidner' via blink-dev <blin...@chromium.org> wrote:

chri...@chromium.org wrote:

Please file for official signals on these.

Sorry, I had initially missed this in the response. I've filed requests for official signals:

 

  https://github.com/mozilla/standards-positions/issues/473

  https://lists.webkit.org/pipermail/webkit-dev/2021-January/031663.html

I haven't heard back from the TAG review so far. Would it be OK to proceed with enabling the feature for the 89 branch (see https://chromium-review.googlesource.com/c/chromium/src/+/2518764), and if needed revert that if the launch approval isn't ready in time for 89 stable?

Both the TAG review and signals are very new (especially due to the holidays). We'll discuss it at our meeting this week and get back to you.

Given these timelines, I'd prefer to not land the code and instead merge it after the branch if it gets approved.

 

On Wed, Dec 16, 2020 at 4:34 PM Mounir Lamouri <mlam...@google.com> wrote:

On Wed, 16 Dec 2020 at 15:40, Klaus Weidner <kla...@google.com> wrote:

We're now targeting M89 for the feature. I don't have specific information about developers other than <model-viewer> using the feature behind a flag at this time, I'll reach out again to see if I can get more feedback.

As mentioned before, because this API is meant to allow web developers to improve the performance of their application when using AR, we consider this something that everyone should be using. The Model Viewer project saw drastic improvements on low end devices where the default sized canvas setup by WebXR didn't allow the device to keep up at 30fps.

-- Mounir

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAFU2V80YYKU3XAHJR-TpK8NFb23_MDxjmztEqBtGOingHH9-yw%40mail.gmail.com.

[Mike West]

Hey folks!

We talked about this in tonight's meeting, and we still don't think we can approve the intent without signals from other vendors or the TAG. We're also not comfortable with enabling the feature by default in a way that requires backmerges to disable in the case that no feedback is forthcoming in the M89 release timeframe. Adding a `base::Feature` and wiring it up to Finch seems like a better option, as it would allow y'all to enable the feature post-approval, rather than disabling the feature at the last moment of whatever merging deadline the M89 release managers set.

-mike

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOMQ%2Bw8xjy0SRSGqENYbG6h7hqfpeBky%2B0%3DJGrXurSiYu3S28w%40mail.gmail.com.

[Mounir Lamouri]

Just to understand, the TAG review was sent a month ago, right? I assume there are no new expectations that the TAG actually give feedback on the review before we launch something, are they? A month of notice is more than most intent to ship usually have for the TAG review, not mentioning the small scope of the change.

Regarding the vendors feedback, Apple never gave feedback on WebXR APIs and we wouldn't imagine this to change for something of this scope. Mozilla used to co-edit the specification that this change was added to. That specification is in the REC track in a Working Group and is a few changes away to reach CR. Vendors had other venues to raise concerns a while ago.

We can turn this on after branch point but it's hard to understand why so much pushback for this in particular. It is a much needed change that allows better performance for an API that launched a while back and already got support from the Working Group this work is happening in.

-- Mounir

[Rik Cabanier]

Regarding other vendor's feedback, we at Oculus support this feature.

Even though we're not supporting it at the moment, we want to do so in the future because it is a natural way to fix performance of a slow scene.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAKXHy%3DcHUEUcbqjpvq9pc_joVGTRCrC1Pzde_5gHUiKNZZSxQA%40mail.gmail.com.

[Alex Russell]

Rik: Thanks for the note about Oculus; it helps.

Mounir: there has been predictable delay because the process was not followed and no other data was initially provided to help give us confidence that the design solved an important problem well (not even links to discussions or requests for input). We owe it to everyone who launches features to set fair expectations about what's required and when. We just (concurrently with this feature) pioneered an exception process for Unadjusted Pointer Lock, and we may be able to use that for features like this going forward, but in this instance we are not able to make an exception. In any case, filing for TAG review (even as an FYI) would have been required.

Alice: looking to understand if this was taken up at this week's F2F? The schedule suggested it would be, and if it hasn't been discussed, or if there's no resolution, I'm inclined to approve this now that we've followed the process and given some time for input from wide review.

Regards

[Alex Russell]

Another week has passed and I don't see new comments in the TAG review thread: https://github.com/w3ctag/design-reviews/issues/588

In the interest of not delaying this launch further, and given that we've now met our responsibilities to our process and requests for wide review, LGTM1

Regards

[Chris Harrelson]

LGTM2

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/bc79ad11-3f85-418d-a114-216274e5de13n%40chromium.org.

[Mike West]

LGTM3.

-mike