Intent to Experiment: New Canvas 2D API

443 views
Skip to first unread message

Aaron Krajeski

unread,
Apr 22, 2021, 11:04:28 AMApr 22
to blink-dev, Fernando Serboncini
Contact emails

aar...@chromium.org, fs...@chromium.org


Explainer/Chrome Dev Summit Video

https://github.com/fserb/canvas2d

https://youtu.be/dfOKFSDG7IM


API spec

In progress

  WHATWG:

Text Modifiers

Filters

Perspective Transforms

reset()

CSSColorValue

willReadFrequently

RoundRect

ConicGradient

Context Loss

  MDN 

Conic gradient

Text Modifiers (via ChromeLabs/puppy-content)


Summary

Updated functionality for the Canvas2D API. Adds nine new features/functions to CanvasRenderingContext2D:

  - "ContextLost" and "ContextRestored" events

  - "willReadFrequently" option for canvases where lots of readback is expected

  - CSSColorValues as style inputs

  - More CSS text modifier support

  - A reset function

  - A roundRect draw primitive

  - Full 4x4 transformation matrices

  - Conic gradients

  - Better support for SVG filters


Blink component

Blink>Canvas


Timeline: M92-95


TAG review

None


TAG review status

Not applicable


Risks
Interoperability and Compatibility
Edge: Shipped (partially) (context loss email chain) Very interested in context loss. Helped with implementation.

Gecko: In development (https://github.com/whatwg/html/issues/5431) Already implemented conic gradient. Okay with willReadFrequently, transforms and reset. Push back on filters.


WebKit: Positive (https://github.com/whatwg/html/issues/5619) Some push back on perspective transforms. Positive signal on text modifiers, round rect and color input.


Web developers: Positive (https://www.youtube.com/watch?v=dfOKFSDG7IM) CDN talk in December was received very positively.


Goals for experimentation

Allow third party developers to use the API and provide feedback.


Reason this experiment is being extended

None


Ongoing technical constraints

None


Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes


Is this feature fully tested by web-platform-tests?

Yes (in progress)

RoundRect

reset()

Perspective Transforms

Context Loss (this is not testable on WPT)

Filters

CSSColorValue Input

WillReadFrequently (testable in WPT, work in progress to add tests there)

Conic Gradient

Text Modifiers


Flag name

NewCanvas2DAPI


Tracking bugs

Transforms: https://bugs.chromium.org/p/chromium/issues/detail?id=1140535

Color: https://bugs.chromium.org/p/chromium/issues/detail?id=1187575

Filters: https://bugs.chromium.org/p/chromium/issues/detail?id=1169216

RoundRect: crbug.com/1123971

Reset: crbug.com/1131137

Context Loss: crbug.com/1164059
Will read frequently: https://bugs.chromium.org/p/chromium/issues/detail?id=1090180

Conic gradient: crbug.com/1097034

Text Modifiers: https://bugs.chromium.org/p/chromium/issues/detail?id=1137055



Launch bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1201359


Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/6051647656558592


Aaron Krajeski

unread,
Apr 22, 2021, 11:24:22 AMApr 22
to blink-dev, Fernando Serboncini

Aaron Krajeski

unread,
Apr 22, 2021, 11:24:42 AMApr 22
to blink-dev, Fernando Serboncini
Looks like posting on blink-dev just merged the threads 🤦‍♀️. Double apologies for the noise. This is our real-bonafide intent to experiment thread. 

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/ec01b3df-dc82-4cbc-94c4-3efaf1186728n%40chromium.org.

Mike West

unread,
Apr 22, 2021, 3:01:42 PMApr 22
to Aaron Krajeski, blink-dev, Fernando Serboncini
Hey Aaron,

I have a few questions that I'll add here rather than inline, as the thread's formatting is already a little strange:

  1. Can you help me understand why no TAG review was requested? This is a somewhat substantial addition to an existing API, and their input seems like it might be valuable.

  2. Would you mind filing requests for signals with our friends at other vendors via the channels specified in https://bit.ly/blink-signals? It's certainly positive to see participation in GitHub threads, but they've asked us to use their official channels for these intents.

  3. You suggest that Edge is already shipping these APIs, but the link provided isn't public. Have they upstreamed an implementation to Chromium that's already enabled?

-mike


Chris Harrelson

unread,
Apr 22, 2021, 3:28:05 PMApr 22
to Aaron Krajeski, blink-dev, Fernando Serboncini
Hi, one comment.

This link looks broken..
 

Alex Russell

unread,
Apr 22, 2021, 3:43:33 PMApr 22
to blink-dev, mk...@chromium.org, blink-dev, fs...@chromium.org, Aaron Krajeski
Per today's API OWNERs meeting, very conditional LGTM to experiment with all the caveats of Mike's note. Would appreciate you reporting back about both what we learn from the trial from developers as well as any TAG feedback.

Aaron Krajeski

unread,
Apr 23, 2021, 12:32:29 PMApr 23
to Alex Russell, blink-dev, mk...@chromium.org, fs...@chromium.org
Thanks all for the feedback and thanks all for no one pointing out how gracelessly I started this thread!

@Chris Harrelson Sorry about that, here's a better link: https://github.com/whatwg/html/issues/4809


  1. Can you help me understand why no TAG review was requested? This is a somewhat substantial addition to an existing API, and their input seems like it might be valuable.
I was using Canvas color management I2E as a template. I'll ask for a TAG review and see their feedback.
  1. Would you mind filing requests for signals with our friends at other vendors via the channels specified in https://bit.ly/blink-signals? It's certainly positive to see participation in GitHub threads, but they've asked us to use their official channels for these intents.
Yes! will do.
  1. You suggest that Edge is already shipping these APIs, but the link provided isn't public. Have they upstreamed an implementation to Chromium that's already enabled?
Yes, they worked on the contextloss api, though ultimately our team ended up implementing it. Nothing is enabled yet, it's still all behind the `NewCanvas2DAPI` flag (which gets enabled with experimental web platform features.

I'll get back to you on 1, 2 (TAG and official signals, inlining comments messed up the list numbering) and respond to this thread.

  Cheers,
    Aaron

Aaron Krajeski

unread,
Apr 28, 2021, 11:24:20 AMApr 28
to Alex Russell, blink-dev, mk...@chromium.org, fs...@chromium.org

Yoav Weiss

unread,
Apr 30, 2021, 4:54:04 AMApr 30
to Aaron Krajeski, Alex Russell, blink-dev, mk...@chromium.org, fs...@chromium.org

Aaron Krajeski

unread,
May 1, 2021, 2:52:11 PMMay 1
to Yoav Weiss, Alex Russell, blink-dev, mk...@chromium.org, fs...@chromium.org
Thanks for the reminder Yoav!

Mozilla standards positions thread: https://github.com/mozilla/standards-positions/issues/519
I've sent an email describing the proposal to webki...@lists.webkit.org, I assume it will appear here soon https://lists.webkit.org/pipermail/webkit-dev/

Theodore Olsauskas-Warren

unread,
May 4, 2021, 10:21:59 AMMay 4
to blink-dev, Aaron Krajeski, Alex Russell, blink-dev, mk...@chromium.org, fs...@chromium.org, yoav...@chromium.org
Hi All,

Some quick feedback from the S&P review, we're have some concerns about the ContextLoss event being delivered to multiple tabs simultaneously, and thus allowing a join across 1P contexts.  Would it be feasible to restrict this to only the active tab? Perhaps delivering the event once the tab has regained focus if it occured whilst the tab didn't have focus? (Apologies if I've missed something that already mitigates this in the design).

PhistucK

unread,
May 4, 2021, 1:50:20 PMMay 4
to Theodore Olsauskas-Warren, blink-dev, Aaron Krajeski, Alex Russell, mk...@chromium.org, fs...@chromium.org, yoav...@chromium.org
> allowing a join across 1P contexts
Can you explain the concern given the existing storage event and the BroadcastChannel API (and others, I suppose)?

PhistucK


Theodore Olsauskas-Warren

unread,
May 5, 2021, 4:56:03 PMMay 5
to blink-dev, PhistucK, blink-dev, Aaron Krajeski, Alex Russell, mk...@chromium.org, fs...@chromium.org, yoav...@chromium.org, Theodore Olsauskas-Warren
I should clarify, we're concerned this allows a join across different 1P contexts. Both the examples you've given are scoped to the same origin. As an example of our concern, if the user has Site A open in one tab, and Site B open in another tab, and both Site A & Site B receive a context loss event at the same time, then they have a precise timestamp with which they can later use to join the user's separate identities on Site A & Site B. This is type of information is especially concerning as it is independent of other joinable information (such as an IP address, which may be shared or proxied).

Aaron Krajeski

unread,
May 6, 2021, 11:29:40 AMMay 6
to Theodore Olsauskas-Warren, blink-dev, PhistucK, Alex Russell, mk...@chromium.org, fs...@chromium.org, yoav...@chromium.org
sauski@: Talking around with the rest of the team and with the WebGL folks, there is a legitimate privacy concern with the contextloss event. The amount of data would be relatively small, as a tracker would have to be active on two domains simultaneously during a GPU or driver crash. Nevertheless, the data would be unique enough that it's worth designing around this vulnerability. WebGL and WebGPU will have the same issue.
  We're now actively looking into how to delay the event until focus.

Aaron Krajeski

unread,
Jun 30, 2021, 1:33:03 PMJun 30
to Theodore Olsauskas-Warren, blink-dev, PhistucK, Alex Russell, mk...@chromium.org, fs...@chromium.org, yoav...@chromium.org
Hey everyone! Sorry for the delay in response, blink-dev got removed from the "reply all" chain and I was shouting into the ether. We are code and test complete, just have a few more whatwg specs to merge.

@Theodore Olsauskas-Warren, re context loss: If we delayed context loss until focus events, would that be acceptable from an S&P perspective? 
  The remaining issue is for contexts within iframes. These will not receive a "focus" event unless the user focuses the iframe itself, but in the interest of getting this out we could live with that until we have a better idea.

Some updates from the TAG review (https://github.com/w3ctag/design-reviews/issues/627):
  1. For the CSSColorValue input we are going to rely on the fact that CSSColorValues can implicitly convert to string, and CanvasRenderingContext2D styles all accept strings as input. This makes for a much smaller spec change. We can still use the optimization of taking the object directly, as the functionality is the same.
  2. WebKit says Perspective Transforms are a no-go. With a heavy, affine heart we're putting this feature on the backburner for now.

The mozilla and webkit signals more or less rehash what's already debated on the TAG thread.

Thanks all for the feedback! Think we'll be ready to launch this origin trial soon?

Aaron

Jeremy Roman

unread,
Jun 30, 2021, 2:57:37 PMJun 30
to Aaron Krajeski, Theodore Olsauskas-Warren, blink-dev, PhistucK, Alex Russell, mk...@chromium.org, fs...@chromium.org, yoav...@chromium.org
On Wed, Jun 30, 2021 at 1:33 PM 'Aaron Krajeski' via blink-dev <blin...@chromium.org> wrote:
Hey everyone! Sorry for the delay in response, blink-dev got removed from the "reply all" chain and I was shouting into the ether. We are code and test complete, just have a few more whatwg specs to merge.

@Theodore Olsauskas-Warren, re context loss: If we delayed context loss until focus events, would that be acceptable from an S&P perspective? 
  The remaining issue is for contexts within iframes. These will not receive a "focus" event unless the user focuses the iframe itself, but in the interest of getting this out we could live with that until we have a better idea.

Some updates from the TAG review (https://github.com/w3ctag/design-reviews/issues/627):
  1. For the CSSColorValue input we are going to rely on the fact that CSSColorValues can implicitly convert to string, and CanvasRenderingContext2D styles all accept strings as input. This makes for a much smaller spec change. We can still use the optimization of taking the object directly, as the functionality is the same.

Make sure you check that toString hasn't been replaced on the prototype chain, and otherwise that this skipped step will actually be equivalent to converting to string and then to CSSColorValue. It's a common pitfall to assume this sort of optimization is valid but in general it isn't, at least without accounting for all the weird edges.
 

Theodore Olsauskas-Warren

unread,
Jul 2, 2021, 1:37:45 PMJul 2
to Jeremy Roman, Aaron Krajeski, blink-dev, PhistucK, Alex Russell, mk...@chromium.org, fs...@chromium.org, yoav...@chromium.org
@Theodore Olsauskas-Warren, re context loss: If we delayed context loss until focus events, would that be acceptable from an S&P perspective? 

Yep, that sounds good. When this came through the review process that was considered by the S&P reviewers as a reasonable mitigation.

Theo.

Sushanth Rajasankar

unread,
Jul 7, 2021, 1:03:47 PMJul 7
to blink-dev, Theodore Olsauskas-Warren, Aaron Krajeski, blink-dev, PhistucK, sligh...@google.com, mk...@chromium.org, fs...@chromium.org, yoav...@chromium.org, Jeremy Roman
Hello, 

Just catching up on this discussion. We are super interested in context lost and canvas 2d improvements.

Some questions :
Is it true that we are now limiting this API to fire on Iframes only when they gain focus ? 
This would be problematic for us as some of our properties that want to use this are served from within Iframes. 

- Even without Iframes, can we use page visibility as a trigger to fire these context lost rather than focus. 
This way if a tab is already visible it gets context lost right away. Iframes would use the main frame's page visibility and so
the user should not see broken content.

Thanks,
Sushanth

Aaron Krajeski

unread,
Sep 14, 2021, 9:54:09 AMSep 14
to blink-dev, Sush...@microsoft.com, Theodore Olsauskas-Warren, Aaron Krajeski, blink-dev, PhistucK, sligh...@google.com, Mike West, Fernando Serboncini, Yoav Weiss, Jeremy Roman
Is it true that we are now limiting this API to fire on Iframes only when they gain focus ? 

  No, we're just delaying contextloss events randomly now. All events are being delayed by a tunable amount of time.

Also, due to delays in getting this out we need to move the milestones from M92-95 to M95-98.
Reply all
Reply to author
Forward
0 new messages