Intent to Ship: Animation Worklet

[Majid Valipour]

Contact emails

maj...@chromium.org, fla...@chromium.org

Explainer

https://github.com/w3c/css-houdini-drafts/blob/master/css-animationworklet/README.md

Spec

https://www.w3.org/TR/css-animation-worklet-1/

TAG review

Summary

Animation Worklet is designed to provide an extensibility point for Web Animations and enables rich high-performance scripted animations which can potentially run off main rendering thread. 

Animation Worklet provides a powerful primitive (performance isolated scripted animations) which when combined with other upcoming features (e.g., Event for Worker/Worklets, ScrollTimeline, GroupEffect) allows many important interactive effects that are currently rAF-based and main-thread bound to move off the main thread which considerably benefits their smoothness. See Animation Worklet design principles and goals for a more extended discussion of this.

Link to “Intent to Implement” blink-dev discussion

https://groups.google.com/a/chromium.org/forum/#!searchin/blink-dev/animation$20worklet%7Csort:date/blink-dev/7CiC3CaPHek/Cy0CN0R6AwAJ

Link to Origin Trial feedback summary

Here is a summary of our Original Trial result which covers both ScrollTimeline and Animation Worklet.

Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.

Demo link

Blog Post

CDS Presentation

Demos

Debuggability

Devtools supports setting up breakpoint in worklets and this should work for Animation Worklet. Animation Worklet tasks are also exposed in performance timeline but this can be further improved by marking them as “Animation” and perhaps expose more details.

Risks

Interoperability and Compatibility

There are two main risks:

• Lack of adoption by other vendors would mean developers have to maintain a polyfill indefinitely which is not desirable and means that the API will not achieve one of its key goals which is to move some of the currently rAF-based animations on the web off main thread.

• API is not rich enough, forcing developers to continue using rAF. This is something that we are keeping a close eye on and will address over time based on the feedback and real world usage. For example on the animation input side we are working to allow ScrollTimeline and more broadly Input Events, and on the output side we are working on GroupEffect and better integration with Paint Worklet.

Edge: Positive. They have been actively contributing to the implementation of this feature in Blink. I have asked them to respond to this email as well to confirm this.

Firefox: No public signal. (standard position issue)

Safari: No public signal. (issue)

Web developers: Positive. This is the second most starred animation bug in Chromium.

The specification is being developed (since 2016) in CSSWG Houdini task force where all major browsers participate and have provided ongoing feedback. The API has been redesigned significantly addressing these feedback to ensure it is implementable by other engines and works well with existing APIs and rest of Houdini family (e.g., adopt Worklet, drop compositor thread/process requirements, support animating all properties, and integrate with WebAnimations). Working group is generally happy with the current API.

Ergonomics

Are there any other platform APIs this feature will frequently be used in tandem with?

We expect this API to be used in tandem with WebAnimations. Also as mentioned before, we plan to introduce additional APIs such as Events in Worklet/Worklets, Scroll Timeline, and GroupEffect that would work well with Animation Worklet.

Could the default usage of this API make it hard for Chrome to maintain good performance (i.e. synchronous return, must run on a certain thread, guaranteed return timing)?

No. Performance and smoothness are a key goal. In particular the feature is explicitly designed  to not require any specific thread.

Activation

The feature can be useful as is. We have a proof-of-concept polyfill which allows the API to be used in other browsers but without the performance benefits of being off-main thread.  (Polyfill is a bit behind the latest changes to the specified API but will be updated before we ship).

Is this feature fully tested by web-platform-tests? Link to test suite results from wpt.fyi.

Majority of our tests are in WPT with only a few blink-specific ones in Chromium.

Entry on the feature dashboard

https://www.chromestatus.com/features/5762982487261184

[Daniel Libby]

Edge is supportive of AnimationWorklets and, as Majid mentioned, has been contributing to the Blink implementation. We're excited to see this new animation primitive added to the Web Platform.

[Boris Zbarsky]

On 6/25/19 8:44 AM, Majid Valipour wrote:
> There are two main risks:

There's the additional risk that the spec in its current form can't
really be used as a basis for implementation because various parts of it
don't really make sense. I did a quick skim of the first few parts and
filed some issues [1], but someone really needs to go over this thing
carefully and make sure it actually says things that can be implemented.

Related, I took a quick look at
https://cs.chromium.org/chromium/src/third_party/blink/renderer/modules/animationworklet/animation_worklet_global_scope.cc?l=182&rcl=6b0fb62b1c4624c0c44edf48117afac21692ad08
(the implementation of registerAnimator) and it looks nothing like the
spec draft in terms of getting the "state" property from the passed-in
constructor, and whatnot. Again, this adds to interop risks, if what
Chrome is shipping is quite different from what the spec says.

I stopped reviewing the spec at that point, fwiw.

-Boris

P.S. It sure would be nice if Blink's idl parser caught
machine-checkable things like
https://github.com/w3c/css-houdini-drafts/issues/907 like Gecko's does. ;)

[1] https://github.com/w3c/css-houdini-drafts/issues/906
https://github.com/w3c/css-houdini-drafts/issues/907
https://github.com/w3c/css-houdini-drafts/issues/908

[Luke Bj]

On Wednesday, June 26, 2019 at 8:36:03 PM UTC-4, Boris Zbarsky wrote:

On 6/25/19 8:44 AM, Majid Valipour wrote:
> There are two main risks:

There's the additional risk that the spec in its current form can't
really be used as a basis for implementation because various parts of it
don't really make sense.  I did a quick skim of the first few parts and
filed some issues [1], but someone really needs to go over this thing
carefully and make sure it actually says things that can be implemented.

Related, I took a quick look at
https://cs.chromium.org/chromium/src/third_party/blink/renderer/modules/animationworklet/animation_worklet_global_scope.cc?l=182&rcl=6b0fb62b1c4624c0c44edf48117afac21692ad08
(the implementation of registerAnimator) and it looks nothing like the
spec draft in terms of getting the "state" property from the passed-in
constructor, and whatnot.  Again, this adds to interop risks, if what
Chrome is shipping is quite different from what the spec says.

I stopped reviewing the spec at that point, fwiw.

-Boris

P.S.  It sure would be nice if Blink's idl parser caught
machine-checkable things like
https://github.com/w3c/css-houdini-drafts/issues/907 like Gecko's does.  ;)

Agree; opened https://github.com/web-platform-tests/wpt/issues/17551 in WPT

[Majid Valipour]

Hi Boris,

On Wed, Jun 26, 2019 at 8:36 PM Boris Zbarsky <bzba...@mit.edu> wrote:

On 6/25/19 8:44 AM, Majid Valipour wrote:
> There are two main risks:

There's the additional risk that the spec in its current form can't
really be used as a basis for implementation because various parts of it
don't really make sense.  I did a quick skim of the first few parts and
filed some issues [1], but someone really needs to go over this thing
carefully and make sure it actually says things that can be implemented.

Thanks for filling the issues. I agree that the spec can be improved. In particular

improve current text and IDL for clarity and accuracy. This mostly my

shortcoming as a rookie spec editor and also the fact that as the implementor

I probably have implicit knowledge which may not have been captured properly in

the spec in enough details for good interoperable implementation.

I am going to reach out to some more experience spec writers in Blink team to 

do a thorough pass (and publicly file issues). I will make sure to address any issues

they may raise to ensure the spec is correct and detailed enough. Please fill free

to file bugs against spec as well. I appreciate the review you have done so far and

if you are willing would welcome a second pass once these issues are addressed.

 

Related, I took a quick look at
https://cs.chromium.org/chromium/src/third_party/blink/renderer/modules/animationworklet/animation_worklet_global_scope.cc?l=182&rcl=6b0fb62b1c4624c0c44edf48117afac21692ad08
(the implementation of registerAnimator) and it looks nothing like the
spec draft in terms of getting the "state" property from the passed-in
constructor, and whatnot.  Again, this adds to interop risks, if what
Chrome is shipping is quite different from what the spec says.

 

One difference in Blink implementation from current spec is related to how

we detect stateful and stateless animators. Previously the plan was to use

two superclasses which is what we had in the specification. In the most 

recent F2F (3 weeks ago) we resolved to drop that approach in favor of a 

simpler model. The spec does not yet reflect this change yet but Blink

implementation does. I will be making appropriate edits to the spec shortly

to ensure we are consistent.

I responded to #908 with more details. Although I may be misunderstanding

the concern because  registerAnimator should not deal with passed state value

but check existence of state function.

I have responded to  #906 and #907 as well and will be making edits to spec to

fix IDLs.

[Boris Zbarsky]

On 6/27/19 11:39 AM, Majid Valipour wrote:
> I am going to reach out to some more experience spec writers in Blink
> team to do a thorough pass (and publicly file issues).

Majid,

Thank you! That is very much appreciated.

I would be happy to take another look once this has happened.

> The spec does not yet reflect this change yet but Blink implementation does.

Ah, I see. OK.

> Although I may be misunderstanding
> the concern because  registerAnimator should not deal with passed state
> value but check existence of state function.

I think you understood the concern correctly. I'll just wait for the
spec updates. :)

Thank you for the quick response,
Boris

[Jochen Eisinger]

We've had information leaks in the past where some CSS filter was applied to cross-origin content, and based on how long the filter took to compute, one could derive individual pixel's color value.

I wonder whether an animation worklet might also run on cross origin pixels and could therefore be used to create similar side channels?

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAB8RdXuszFX5iW%3D_Ok3Anhg%3DRpOO26Kpetn6yMBVb9i-EDiVDg%40mail.gmail.com.

[Robert Flack]

AnimationWorklet is only ticked once per frame, and has no way of knowing how long drawing the current animation value took (unless it delays the start of the next frame). As a result, I don't think it would be possible to do an attack of this form in AnimationWorklet that wouldn't be easier and already possible on the main thread via requestAnimationFrame.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CALjhuifnpJLsPL%2B%2BGYeDVZhH5MLKi9LJUWwxY5OU6p7HYJua-A%40mail.gmail.com.

[Jochen Eisinger]

I think it's enough if the animation worklet has different runtimes depending on the pixel values, it doesn't need to observe the timing itself. For CSS animations, we fix those issues by ensuring that all executions take the same amount of time, which we can't guarantee here.

Would it be an option to have animation worklets not run on cross-origin pixels?

[Robert Flack]

I'm curious what you mean by we can't guarantee that here. AnimationWorklet can't make decisions based on actual pixel values as its only input is the animation time. Its output is a local time along an animation curve. This could all be done from the main thread setting style.filter for example and measuring if the next rAF comes on time or not.

[Robert Flack]

I realized when thinking about possible timing attacks though that you could probably detect sub-frame differences in overall commit time (style + layout + paint + commit), where paint was not usually accessible. See my demo here:

https://output.jsbin.com/davibez/

Note, that there's a fair bit of skew as to what the base time is detected as so while it doesn't work as an absolute commit time it does detect relative changes in commit time. I think that this ship may have already sailed with paint worklet though as you could get the time in your paint() function and delay by some main thread detectable duration in order to communicate it back to the main thread.

[Jochen Eisinger]

thanks for taking a look. I agree that paint worklet seems to have a superset of capabilities here.

[Chris Harrelson]

Quick update: API owners just met, and we'll wait for an update on the thread when the spec updates mentioned earlier have completed and it's ready for re-review.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CALjhuiczsV2mA5SQwyG_xqY6_YM7iRn0aaOL3-pT4b0yOx2TTg%40mail.gmail.com.