Intent to Experiment: lowLatency canvas contexts

[Miguel Casas-Sanchez]

Contact emails
mca...@chromium.org, fs...@chromium.org, tbuc...@chromium.org (use cases)

Spec
https://github.com/whatwg/html/issues/4087, see also [0]

Summary

This feature provides a low latency rendering mode for canvas 2D and WebGL contexts -- this is a "continuation" of the experiment called "Intent to Implement and Experiment: Single buffered canvas" sent on Oct 2017 by junov@ [1]. The said experiment lapsed without any follow up e-mails, but since it applied to OffscreenCanvas, we could conclude that the results were positive although no effort to ship to stable happened. 

Now for the real summary :-)

Just like the previous ITE, this is useful, for example, for stylus input. Low Latency mode is enabled with canvas.getContext('2d' -or- 'webgl', {lowLatency: true});

Motivation: NaCl/PPAPI provides native OpenGL rendering surfaces which are currently used to implement low latency interfaces; similarly, Android apps on ChromeOS have a low latency path. The deprecation of NaCl and the levelling of the playing field creates a need for this API. 

Goals for experimentation

Validate the impact of this feature on user experience with production web sites in order to justify moving forward with standardization. 

Experiment with different tradeoffs of tearing versus low latency, i.e. using a single blit per requestAnimationFrame() is good for tearing but bad for latency.

Experimental timeline

Start: M71

End: M73

Any risks when the experiment finishes?

If the experiment ends without the feature being shipped, web apps that were using it will not suffer any loss of functionality, they will just lose the advantage of single buffered mode.  Request for for the feature will be silently ignored (it is a parameter in a dictionary).

Ongoing technical constraints

To offer truly single buffered operation, hardware overlay buffers must be available. Otherwise the implementation will make a best effort to reduce latency (e.g. by skipping the "Renderer" compositor path), but rendering will not be truly single buffered due to the need to go through a compositing pass upstream of device scanout.

Debuggability

No DevTools support needed.

Will this feature be supported on all five Blink platforms supported by Origin Trials (Windows, Mac, Linux, Chrome OS, and Android)?

Yes, but with nuances: hardware overlays are only fully supported on Chrome OS and under certain circumstances (e.g. rotations etc, see [0] again). 

Other platforms would benefit from the low latency path, but might react badly to the app trying to render to the front buffer. (This experiment will identify those, e.g. Mac).

Link to entry on the feature dashboard

https://www.chromestatus.com/feature/6360971442388992

[0] tinyurl.com/lowlatency-canvas-on-chromeos

[1] https://groups.google.com/a/chromium.org/forum/?utm_medium=email&utm_source=footer#!msg/blink-dev/DCPyE6rPFpA/sqZwKAWXBAAJ

[Johnny Stenback]

Seems there's a risk of this causing increased main thread unresponsiveness if this ends up being over-used by sites. Any mitigations against that in the plans? Or is this intended to only ever work in workers?

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAKoAQ7nqzopNuQOny9cs0S5EXMG5Sge9%2BLyQFaiByBMch%2Bc4MA%40mail.gmail.com.

[Miguel Casas-Sanchez]

On Fri, 12 Oct 2018 at 14:39, Johnny Stenback <jste...@chromium.org> wrote:

Seems there's a risk of this causing increased main thread unresponsiveness if this ends up being over-used by sites. Any mitigations against that in the plans?

Processing of input events and rendering is still governed by the render message loop, i.e. 60 times a second, so in principle there should be no extra powers to override the main thread; the page could still be causing a lot of (2d/3d) drawing commands, but it would not be any worse that a non-lowLatency canvas. 

I think junov@ experimented with disconnecting the processing of the events and the message loop and did override the gpu process, so that wasn't a good idea, and was not pursued.

  

Or is this intended to only ever work in workers?

This particular ITE focuses on "onscreen" 2d/3d canvas contexts, not OffscreenCanvas ones.

[Philip Jägenstedt]

LGTM to experiment.

If it comes to standarding this, I suppose I'd wonder how web developers should reason about the trade-off, i.e. when they shouldn't use this feature. Or is it a possibility that we'll conclude that this should just be the one and only behavior, if it's better in every dimension that matters?

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPUS085XcxgOzAQaucZEAHbMSoNNpR3K33tTo6Y0pURM8SaFyg%40mail.gmail.com.

[Miguel Casas-Sanchez]

On Mon, 15 Oct 2018 at 14:58, Philip Jägenstedt <foo...@google.com> wrote:

LGTM to experiment.

If it comes to standarding this, I suppose I'd wonder how web developers should reason about the trade-off, i.e. when they shouldn't use this feature. Or is it a possibility that we'll conclude that this should just be the one and only behavior, if it's better in every dimension that matters?

It might be faster in general (or at least not slower), but it introduces tearing. The app needs to know these tradeoff, e.g. by drawing onto its own back buffer and then blitting onto the front buffer.

One thing that I brought but didn't get much traction is observability: how can a WebApp know that the lowLatency "hint" is actually supported by the platform? E.g. we (or at least I) don't know how to do proper front buffer rendering in Mac; should the WebApp have a way to know the the hint is not picked up (and maybe tell the user)?

[Ken Russell]

On Mon, Oct 15, 2018 at 3:27 PM Miguel Casas-Sanchez <mca...@chromium.org> wrote:

On Mon, 15 Oct 2018 at 14:58, Philip Jägenstedt <foo...@google.com> wrote:

LGTM to experiment.

If it comes to standarding this, I suppose I'd wonder how web developers should reason about the trade-off, i.e. when they shouldn't use this feature. Or is it a possibility that we'll conclude that this should just be the one and only behavior, if it's better in every dimension that matters?

It might be faster in general (or at least not slower), but it introduces tearing. The app needs to know these tradeoff, e.g. by drawing onto its own back buffer and then blitting onto the front buffer.

One thing that I brought but didn't get much traction is observability: how can a WebApp know that the lowLatency "hint" is actually supported by the platform? E.g. we (or at least I) don't know how to do proper front buffer rendering in Mac; should the WebApp have a way to know the the hint is not picked up (and maybe tell the user)?

The getContextAttributes() API in WebGL allows querying of the context to find out the "actual" context creation attributes that were used:

https://www.khronos.org/registry/webgl/specs/latest/1.0/#5.14.2

This can surface the "lowLatency" flag if it's supported. However, there's no way to know before attempting to create the context whether it would be supported by the browser.

We should generalize this in a backward-compatible way and support something similar in all canvas context APIs. (2d, webgl, and bitmaprenderer)

-Ken

 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPUS085zfLtPGwLN11a9B_Ewe48TNgMPhevOX%3DaqK_F%2BjaoC7w%40mail.gmail.com.