Intent to implement rederedPixelWidth/Height on HTMLCanvasElement

[Justin Novosad]

Contact emails

ju...@chromium.org

Spec

Feature proposal, not yet in spec: https://wiki.whatwg.org/wiki/CanvasRenderedPixelSize

Summary

rederedPixelWidth and rederedPixelHeight are read-only attributes that indicate the intrinsic size that must be set on a canvas for its pixels to map one-to-one to display device pixels. For applications that wish to render at the device's native resolution.

Motivation

There is high demand for an API that guarantees one-to-one pixel alignment between

a canvas element and the display device. Current hacks that sniff the device pixel ratio tend to be inexact.

Compatibility Risk

This feature does not interfere with any pre-existing functionality, and there is no immediate intent to ship.

Firefox has demonstrated public support for this feature (It's their proposal).

Ongoing technical constraints

There is currently some speculation about implementation issues related to interactions with layout (pixel coordinate snapping and the fact that the intrinsic size is in integers). An experimental implementation will allow us to explore these problems and possibly refine the feature proposal.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes

OWP launch tracking bug?

crbug.com/476067

Link to entry on the feature dashboard

https://www.chromestatus.com/features/5477772331843584

Requesting approval to ship?

No

[Brandon Jones]

Non-Owners LGTM! This is a frequent request from the WebGL community, and extremely important for image quality of rendered canvas content. It's way too easy to get into a situation where your canvas dimensions are 1-2 pixels off off the physical pixels (especially on mobile) and as a result get really nasty aliasing/resampling artifacts.

[Anne van Kesteren]

On Fri, Apr 10, 2015 at 10:21 PM, Justin Novosad <ju...@chromium.org> wrote:
> Feature proposal, not yet in spec:
> https://wiki.whatwg.org/wiki/CanvasRenderedPixelSize

As far as I can tell the first time this was mentioned on the list was
four hours ago? Was it discussed elsewhere perhaps? Also, where is
"renderedsizechange" defined?


--
https://annevankesteren.nl/

[Justin Novosad]

Yes, this is the first time this has been discussed on blink-dev AFAIK.

It was discussed on whatwg last june: https://lists.w3.org/Archives/Public/public-whatwg-archive/2014Jun/0043.html

I am not aware of a definition of renderedsizechange outside of the referenced proposal.

[Kenneth Russell]

Non-owner LGTM as well. Google's Maps team and others have requested
this for the reasons Brandon mentioned.

It'll be important to align the implementation with the pixel snapping
logic that goes on behind the scenes as mentioned in crbug.com/388532
.

[Alex Komoroske]

This intent was not automatically captured in bit.ly/blinkintents because the subject line did not have a colon after "Intent to implement". Just a reminder to folks to make sure your subject is formatted according to the blink intent template, it makes my life ever so slightly easier. :-)

[John Mellor]

Can you point to examples of why "Current hacks that sniff the device pixel ratio tend to be inexact"? It would helpful to understand how this differs.

[Justin Novosad]

On Tue, Apr 14, 2015 at 10:08 AM, John Mellor <joh...@chromium.org> wrote:

Can you point to examples of why "Current hacks that sniff the device pixel ratio tend to be inexact"? It would helpful to understand how this differs.

First of all, the result of layout calculation is theoretically tractable in javascript, but it can get very complicated to reproduce when using complex layout hierarchies. This feature spares that trouble.

More importantly, it is actually not possible compute the the rendered pixel size in a way that guarantees exactly correct results because the layout calculation are performed using fractional values (floating-point or fixed-point arithmetic), that are later rounded to integer pixel coordinates, which helps browsers produces visually sharp and symmetric results.  Two sub issues with this:

* The exact snapping logic is different from one browser to another, and even with the same browser, may be different depending on the OS and GPU

 configuration; and the algorithms may change over time as browsers evolve to add features, improve performance or improve rendering quality.

* Even if you re-wrote the exact same layout and coordinate snapping algorithm in JavaScript, it would not be guaranteed to provide an exact match due to arithmetic precision issues. For example, JS uses double-precision floats, which is probably not the case of the browser's layout engine, also if the arithmetic operations are not performed in exactly the same order as the code generated by the compiler, machine precision error propagation will cause differences.  These minute differences may cause the results to be off by an entire pixel when the values are rounded to integers.

[PhistucK]

Why not remove the requirement for a colon altogether? It is not intuitive (or even grammatically correct, I think)...

☆PhistucK

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Chris Harrelson]

I agree that this is an important use case, and am glad that you're pursuing it. Two pieces of feedback:

1. Going forward we should not add live JavaScript properties. Instead I think it should be two methods: getRenderedPixelWidth() and getRenderedPixelHeight()

2. What interpolation algorithm should the browser apply when the backing does not match pixels, but script has not had time to put up a new frame? For example, consider an animated 3D transform on the canvas. Should the image-rendering CSS property address this question already? (also related)

[Justin Novosad]

On Wed, Apr 15, 2015 at 1:12 AM, Chris Harrelson <chri...@chromium.org> wrote:

I agree that this is an important use case, and am glad that you're pursuing it. Two pieces of feedback:

1. Going forward we should not add live JavaScript properties. Instead I think it should be two methods: getRenderedPixelWidth() and getRenderedPixelHeight()

Rationale? Is it a policy that we prefer getters over read-only attributes?

2. What interpolation algorithm should the browser apply when the backing does not match pixels, but script has not had time to put up a new frame? For example, consider an animated 3D transform on the canvas. Should the image-rendering CSS property address this question already? (also related)

I see two issues here.

* When applying a 3D tranform, the browser makes a choice as to the rendered resolution of the transformed layer (the layer backing), so the intrinsic size suggested by renderedPixelWidth and renderedPixelHeight should allow the canvas to match the resolution of the layer backing. I think image-rendering is orthogonal.

* There is a synchronization issue here because a compositor commit may get scheduled between a layout change and the invocation of the renderedsizechanged event handler, so we may end-up rendering a frame in an inconsistent state where the content has not been adapted to the new layout. That is a problem we already have today with code that relies on mutation observers or window.onresize. It is not a problem that this feature aims to fix, but if anyone has any ideas, please share. I don't think it is a good idea to postpone commits when there is a pending renderedsizechanged event, because that could lock-up the main thread if the event handler touches layout.

[Nat Duca]

This seems like this would force a layout, right? In other threads, we have been talking about async versions of all layout-inducing apis, e.g. asyncGetBoundignClientRect so that you can get these things but eventually... that way you avoid layout thrashing.

Should we be contemplating this here?

[Justin Novosad]

On Wed, Apr 15, 2015 at 1:13 PM, Nat Duca <nd...@chromium.org> wrote:

This seems like this would force a layout, right? In other threads, we have been talking about async versions of all layout-inducing apis, e.g. asyncGetBoundignClientRect so that you can get these things but eventually... that way you avoid layout thrashing.

Should we be contemplating this here?

Absolutely! That would also solve an issue I raised in the proposal about the fact that the layout update may fire a renderedsizechange event while the rendered size is being queried, which is weird.

[Chris Harrelson]

On Wed, Apr 15, 2015 at 8:47 AM, Justin Novosad <ju...@chromium.org> wrote:

On Wed, Apr 15, 2015 at 1:12 AM, Chris Harrelson <chri...@chromium.org> wrote:

I agree that this is an important use case, and am glad that you're pursuing it. Two pieces of feedback:

1. Going forward we should not add live JavaScript properties. Instead I think it should be two methods: getRenderedPixelWidth() and getRenderedPixelHeight()

Rationale? Is it a policy that we prefer getters over read-only attributes?

I take back my objection to a read-only property instead of a get*() method. On reflection and doing some more research, they are equivalent.

However, this triggered me to think some more about how long it would take to compute rendered pixel width, and I realized that if layout is dirty then it could take an arbitrarily large amount of time. Reading the public mailing list, looks like you independently came to the same conclusion. The alternatives proposed on the public list sound promising. I think we definitely need to solve this issue correctly, it's at the core of the issues developers will face in practice IMO.

 

2. What interpolation algorithm should the browser apply when the backing does not match pixels, but script has not had time to put up a new frame? For example, consider an animated 3D transform on the canvas. Should the image-rendering CSS property address this question already? (also related)

I see two issues here.

* When applying a 3D tranform, the browser makes a choice as to the rendered resolution of the transformed layer (the layer backing), so the intrinsic size suggested by renderedPixelWidth and renderedPixelHeight should allow the canvas to match the resolution of the layer backing. I think image-rendering is orthogonal.

It doesn't necessarily get to decide that right? e.g. images have an intrinsic resolution that the developer chooses, and a CSS property that specifies what to do when scalinging it. Why not apply similar approaches to canvas?

 

* There is a synchronization issue here because a compositor commit may get scheduled between a layout change and the invocation of the renderedsizechanged event handler, so we may end-up rendering a frame in an inconsistent state where the content has not been adapted to the new layout. That is a problem we already have today with code that relies on mutation observers or window.onresize. It is not a problem that this feature aims to fix, but if anyone has any ideas, please share. I don't think it is a good idea to postpone commits when there is a pending renderedsizechanged event, because that could lock-up the main thread if the event handler touches layout.

Right. I think we need to fit the update into the right place in the frame rendering pipeline, wherever that might be.