Synchronising script with animation events

[Brian Birtles]

[Reposting to dev-tech-layout since I didn't get any response on
dev-tech-dom]

Dear all,

Together with a few others, I'm currently working on specifying a means
for synchronising script and animation.

The problem is that it is not possible to create animations in response
to CSS animation events that remain in sync with other running animations.

Consider this use case:

* Animation A has a duration of 2s.
* Animation B has a duration of 4s.
* When animation A finishes, an 'animationend' event is dispatched and
the handler conditionally chooses animation C to run next. Animation C
has a duration of 2s.

The desired result is that animation C finishes at the same time as
animation B. This is currently not the case.

Test case here:
http://people.mozilla.org/~bbirtles/css-sync.html

In Gecko the visual result looks pretty good on fast machines with
simple animations (however, the debug output indicates the event handler
is not being run precisely at the end of the first animation).

In WebKit the lag is often evident even in simple animations. Adobe have
some more complex examples where the lack of synchronisation is really
obvious.

Zooming in on the script interaction at t=2s, the sequence of events is
something like the following:

1. Animation on 'a' finishes
2. Next refresh driver sample occurs
3. 'animationend' event is queued
4. 'animationend' event handler is run
5. New animation on 'c' begins

Ideally we want all these steps to happen in the same instant (from the
point of view of "animation time").

In Gecko I think we can have gaps--i.e. periods where the "animation
time" continues to advance--between 1 and 2, and between 3 and 4. In
WebKit I think the gaps are between 3 and 4, and between 4 and 5.

The approach that Flash takes is to pause the animation until the event
handlers have run. (Presumably, it is paused at the time associated with
the event.) After the event handlers finish, the animation does
a catch-up to the actual time it would be at if it wasn't paused.

This provides the following two advantages:

A) Event handlers can make assumptions about "what is on the
stage"--i.e. the state of the animation.

B) Animations created/modified by event handlers will be in sync with
other animations.

Obviously if event handlers do expensive operations you'll see stutter
so authors are warned not to do that.

I've been talking with a few others about how to provide this kind of
synchronisation for scripted interaction with animations on the Web
platform and the possibilities we've generated so far are:

i) Mimic Flash's pausing behaviour.

The main concern here is how well this approach scales to a Web context
where there could be many events in the queue. One suggestion to
ameliorate this is to introduce animation groups that limit the scope of
the pausing.

Also, with a wider range of implementations available than Flash simply
saying "don't do expensive things in your event handler" doesn't really
cut it since what's expensive can vary and an animation that doesn't
stutter on one implementation may look terrible on another.

At an implementation level, on each sample you'd check if there were
animation endpoints (with registered event handlers) between the last
sample and the "current time". Then you'd advance the animation time to
the earliest of those times and block all updates to the animation time
until the event handlers for that time have run. When the last of those
event handlers runs, you'd repeat the process until the animation time
matches the current time.

Depending on how event processing and the refresh driver interact, and
what mechanism is used to detect when all event handlers have run, you
could run into one of two problems:

* If the event handlers for all animation endpoints between the previous
sample and the current time are run before the next sample arrives, you
could potentially end up doing expensive operations like reflow much
more frequently than the refresh rate. But perhaps that is already the case?

* If you don't run all event handlers up to the current time (and
consequently don't update the animation time to the current time) then
you could have animations that lag badly or fall further and further behind.

ii) As with (i) but prioritise these animation events somehow (has there
been any talk of doing that before?) to try and minimise stutter.

I'm not sure how much this would actually help though.

iii) Don't "pause" the animation, just run the event handlers
synchronously like we do for MutationEvents (which, I know, everyone
hates). That is, eliminate the gap between 3 and 4 altogether.

However, you'd still want to step through the animation endpoints to
remove the potential time gap between 1 and 2 which means you'll run
into the problem mentioned above of potentially doing reflow etc. more
frequently than the refresh rate.

iv) Introduce the notion of a separate play and edit cursor. The play
cursor's time corresponds to "current time" and the edit cursor's time
corresponds to when the event was dispatched. Presentation uses the
animation time of the play cursor; when event handlers query values and
create animations, the animation time of the edit cursor is used.

A naive implementation would simply do a backwards seek from the play
cursor time to the edit cursor time every time an event handler was run.
Either the timing model needs to be defined such that a backwards seek
was cheap or the implementation needs another means of tracking the
different states.

Note that with this approach you could potentially see animations you're
not supposed to. For example, if an event handler cancels an animation
then you might see the animation running for a moment before the event
handler gets a chance to run.

v) Provide a means for getting the event time and producing animations
based on that.

For example, you could get the time of the event from the event itself,
and then use that to generate a new animation that starts at that time.

The problems with this are:
* It doesn't solve (A)--i.e. you can't make assumptions about what is on
the stage
* You have the problem of seeing animation you shouldn't
* It requires adding the concept of absolute times to CSS (which we
probably need anyway)


I'd really appreciate any input on this. My main concern is to solve
this at the spec level so authors have guaranteed synchronisation behaviour.

Are there any specs that already address this? Alternative proposals?
Implementation considerations?

Thanks,

Brian Birtles