UI Workers

[Robert O'Callahan]

Last week in Sydney I spent a lot of time talking to Chrome devs about
different approaches for 60fps effects in Web pages. There are three
different kinds of approaches being discussed (so far):
1) Apple's animation-timeline proposal, which lets CSS animations use
scroll position as an input instead of time.
2) UIWorker: some kind of JS worker that receives callbacks during
composition; each callback can take inputs such as time and scroll
position(s) as inputs and can update certain CSS properties (e.g.
transforms, opacity) on elements that the compositor then uses.
3) Provide a way for pages to turn off async scrolling and make everything
fast enough (and isolated enough) for pages to do 60fps updates from their
main thread.

All of these approaches have problems. Approach #1 is much more limited in
its expressiveness than the alternatives. Approach #3 is more fragile and
less composable than the alternatives --- sharing your main thread with any
JS you don't control could cause jank. I like #2. It's strictly more
powerful than #1 without the downsides of #3.

Obvious question: how do we stop UIWorkers janking the compositor? We could
give them a time budget (say 8ms). If a worker blows its budget, we notify
it by sending it an event, we give up on it and composite anyway, and we
run it separately from the compositor for a while. This requires an API
design that lets UIWorkers still work, with some lag, when the compositor
is not blocking on them, but that seems doable.

Should UIWorkers have access to the full Worker API? It seems like there's
no reason not to give them that.

How should we explain the CSS effects of UIWorkers? A promising idea is to
extend the Web Animations API to allow adding a new kind of animation
effect to DOM elements --- a UIWorker-controlled effect. Essentially the
UIWorker would then be responsible for computing the output of the timing
function in each frame. The UIWorker could then animate *any* CSS property,
though most property updates would require a round trip through main thread
layout/rendering before they get rendered.

One good thing about UIWorkers is extensibility. We can imagine providing
touch input coordinates to UIWorkers to enable 60fps object dragging (with
arbitrary effects like resistance, snapping, etc). UIWorkers could render
to canvases: this would let you render VR with minimum latency, and let you
render to canvases used by CSS masking for 60fps dissolves and clipping
effects. If you really want to, you could go all Flipboard and render your
entire UI to a canvas in the compositor --- if you keep hitting your
deadlines.

I like the idea of doing #2 before either #1 or #3.

Rob
--
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofooooolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

[James Long]

I'm not heavily involved in platform work, so take my input as a
somewhat naïve outsider. I've been tracking this kind of stuff very
closely, especially after I got to play with react native.

The place where this is most important is mobile, and I don't think #2
is going to cut it. It's not just scroll position that we're after. We
need to handle touch events and perform arbitrary animations based on
the gesture of the user. So if the finger moved X pixels in the x
direction, and Y in the y direction, native apps can very carefully
apply any changes to the layout or properties they want. If we only
allow scroll position to be taken into account, I can only imagine the
hacks that are going to take place: making random elements secretly
scroll in specific ways to apply animations (if that's even possible).

Personally I think what we *really* need to be working on is making
all of the DOM APIs asynchronous. That's what Servo needs anyway.
That's a step in the right direction for #3, and we can see much more
we can get out of the main thread after that.

> _______________________________________________
> dev-servo mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-servo

[James Long]

Sorry, I may have missed the last part of your email where you say
that you could provide touch positions as input too. This would be
neat research and I'd like to see more details.

Note however that people in the native app world believe that it's
very important that whatever applies the animations is run on the same
thread that handles input. Otherwise it's very easy for animations to
get out-of-sync. If you swipe a scroll view on iOS, and it has that
momentum scroll, and you place your finger down it immediately stops
where it is. I've talked to several engineers that try to do this kind
of stuff with CSS animations but because they are async, it's not
nearly as reliable as native. Native platforms fire the touch event,
and you apply the animation update immediately in the same thread.

Anyway, just trying to give feedback from a user. This is really
important to me!

[Robert O'Callahan]

On Fri, Feb 20, 2015 at 4:02 PM, James Long <long...@gmail.com> wrote:

Personally I think what we *really* need to be working on is making
all of the DOM APIs asynchronous. That's what Servo needs anyway.
That's a step in the right direction for #3, and we can see much more
we can get out of the main thread after that.

I don't see how making DOM APIs asynchronous really helps you achieve #3.
There are some specific cases where custom layouts need to repeatedly
measure content, that currently rely on APIs that trigger multiple
sequential synchronous reflows, which would be better served by less
synchronous APIs. We have nebulous plans for addressing that (it's not as
easy as "make everything a Promise"). But those aren't the main problems
mobile Web apps face right now AFAIK.

On Fri, Feb 20, 2015 at 4:11 PM, James Long <long...@gmail.com> wrote:

> Note however that people in the native app world believe that it's
> very important that whatever applies the animations is run on the same
> thread that handles input. Otherwise it's very easy for animations to
> get out-of-sync. If you swipe a scroll view on iOS, and it has that
> momentum scroll, and you place your finger down it immediately stops
> where it is. I've talked to several engineers that try to do this kind
> of stuff with CSS animations but because they are async, it's not
> nearly as reliable as native. Native platforms fire the touch event,
> and you apply the animation update immediately in the same thread.
>

Thanks, we'll take that into account.

> Anyway, just trying to give feedback from a user. This is really
> important to me!
>

Thanks!

[James Long]

On Thu, Feb 19, 2015 at 10:25 PM, Robert O'Callahan
<rob...@ocallahan.org> wrote:
> On Fri, Feb 20, 2015 at 4:02 PM, James Long <long...@gmail.com> wrote:
>
> Personally I think what we *really* need to be working on is making
> all of the DOM APIs asynchronous. That's what Servo needs anyway.
> That's a step in the right direction for #3, and we can see much more
> we can get out of the main thread after that.
>
> I don't see how making DOM APIs asynchronous really helps you achieve #3.
> There are some specific cases where custom layouts need to repeatedly
> measure content, that currently rely on APIs that trigger multiple
> sequential synchronous reflows, which would be better served by less
> synchronous APIs. We have nebulous plans for addressing that (it's not as
> easy as "make everything a Promise"). But those aren't the main problems
> mobile Web apps face right now AFAIK.

You may be right, but it just seems pretty awful that you might
accidentally stall the JS event loop because you touched the DOM and
you didn't expect it to take so long (say, 30ms, dropping a few
frames). The Servo team even said they want to look into making them
async: https://news.ycombinator.com/item?id=9011767 However, I have no
real evidence this will help anything. Eventually I'd love to see the
Servo model in real-world browsers where things are running on
separate threads. But that's, like, seriously long-term.

> On Fri, Feb 20, 2015 at 4:11 PM, James Long <long...@gmail.com> wrote:
>>
>> Note however that people in the native app world believe that it's
>> very important that whatever applies the animations is run on the same
>> thread that handles input. Otherwise it's very easy for animations to
>> get out-of-sync. If you swipe a scroll view on iOS, and it has that
>> momentum scroll, and you place your finger down it immediately stops
>> where it is. I've talked to several engineers that try to do this kind
>> of stuff with CSS animations but because they are async, it's not
>> nearly as reliable as native. Native platforms fire the touch event,
>> and you apply the animation update immediately in the same thread.
>
>
> Thanks, we'll take that into account.

If the standards committees are taking a serious look into this, I
would really recommend talking to one of the people at Facebook
working on React Native. They spent years trying to get this kind of
stuff working on the web and React Native is born out of problems they
faced with getting 60 FPS. I think they would have tons of good
feedback for you, for example like my paragraph above. I'm happy to
help someone from that team get involved here if you want.

(I'm sure you are well aware of the problems they would tell you, but
they have some really good reasons for their solution)

[Jonas Sicking]

On Thu, Feb 19, 2015 at 6:27 PM, Robert O'Callahan <rob...@ocallahan.org> wrote:
> Should UIWorkers have access to the full Worker API? It seems like there's
> no reason not to give them that.

There's two use-cases that I think argues against that.

First off I'd like to enable saving a webpage for later viewing. Right
now the best we can do is to save a pdf which means that all
scrollable areas are lost.

Saving a full page with all scripts is rarely going to work result in
the user seeing the page they did when saving the page.

What I'd like to be able to do is serialize the DOM, strip all script,
save all linked stylesheets, images and media and change the
references to them to use the correct relative URLs.

However this would require also saving the UIWorker and enabling it to
run in a reliable way. Otherwise the saved page won't have a working
layout.


Second, when printing, I'd like to enable the user to change page
layout and margins and have that reflow the page and generate a
correct layout. This means that we need to enable the UIWorker to work
on the cloned document in a reliable way.


I think both these usecases would work much better if we ensured that
the UIWorker couldn't "interact with the outside world".

/ Jonas

[Jonas Sicking]

On Thu, Feb 19, 2015 at 7:11 PM, James Long <long...@gmail.com> wrote:
> Note however that people in the native app world believe that it's
> very important that whatever applies the animations is run on the same
> thread that handles input. Otherwise it's very easy for animations to
> get out-of-sync.

Does that mean also doing the painting/compositing on the thread that
handles input?

It'd be good to understand which thread the UIWorker proposal intends
to run the worker. And if we expect any cross-thread communication to
risk causing lag.

/ Jonas

[Anne van Kesteren]

On Fri, Feb 20, 2015 at 3:27 AM, Robert O'Callahan <rob...@ocallahan.org> wrote:
> One good thing about UIWorkers is extensibility. We can imagine providing
> touch input coordinates to UIWorkers to enable 60fps object dragging (with
> arbitrary effects like resistance, snapping, etc). UIWorkers could render
> to canvases: this would let you render VR with minimum latency, and let you
> render to canvases used by CSS masking for 60fps dissolves and clipping
> effects. If you really want to, you could go all Flipboard and render your
> entire UI to a canvas in the compositor --- if you keep hitting your
> deadlines.

Perhaps we should discuss this separately, but if you have looked at
Flipboard and React et al it would be great to know if we see ways of
keeping content in the DOM while achieving the same performance. Going
Flipboard throws away a lot of functionality normally available to
users.


--
https://annevankesteren.nl/

[Christopher Lord]

It's great to see this being discussed! Personally, I'd like to see all
three of these, and I think they have quite particular use-cases. #2 is my
least favourite, however, and I'll discuss this below.

I'd like to see #1 implemented first for two reasons; 1- I know this is
easy to do given our platform, and I expect the same for other browser
vendors, and 2- behaviour here is 100% predictable. There is nothing
unexpected that can happen with this method and it's easy to employ. It
doesn't require writing any JS, and developers are already used to
optimising for the currently async-animatable CSS properties, so we can
expect people not to do anything too silly with it.

I'd like to see #3 implemented second, again for two reasons; 1- It's,
again, reasonably easy to do. You can bypass a lot of machinery already,
but making it explicit would make things easier and more obvious for
people. I expect that part of this would also involve handling scrolling
yourself too (we're on our way to removing the machinery to do synchronous
scrolling, and it'd never respond the same way as async, so I think it'd be
good to make it explicit). 2- There's a very clear use-case here. Not all
web-apps are based around vertically scrolling a complex text layout, which
is pretty much what the entire platform is built around. Games could get a
nice boost here, as could some more unusual applications (e.g. scientific
monitoring apps that mainly just want to draw charts very quickly (thinking
specifically of Vernier LabQuest-style applications)).

I'd like to see #2 implemented the least, mainly because I think it's the
most complex solution, from both the platform and the user side. It's
worrying that behaviour is unpredictable - if you have a time budget, the
same code on one device can deliver wildly different results on another,
and I think this would just introduce inconsistent and unexplainable
behaviour (from a user-perspective), and encourage developers to
special-case particular devices and do awful user-agent string matching.
This is a neat idea, but I just see it breaking more things than it solves,
vs. the other two ideas. I'd probably change my mind given a compelling
use-case, but I'm struggling to think of anything that couldn't be
implemented with a combination of #1 and #3. Also, I realise this is just a
throw-away detail, but an 8ms budget would be pretty huge. I doubt we can
get our compositing done on mobile if we halve our budget (we can barely
get it done as it is).

I do strongly think that #1 is the way to go immediately - people have been
asking for this for ages (myself included), and it covers just about 90% of
what you'd want to do, in a safe, easy-to-use and performant way.

Cheers,

--Chris

On Fri, Feb 20, 2015 at 2:27 AM, Robert O'Callahan <rob...@ocallahan.org>
wrote:

> Last week in Sydney I spent a lot of time talking to Chrome devs about
> different approaches for 60fps effects in Web pages. There are three
> different kinds of approaches being discussed (so far):
> 1) Apple's animation-timeline proposal, which lets CSS animations use
> scroll position as an input instead of time.
> 2) UIWorker: some kind of JS worker that receives callbacks during
> composition; each callback can take inputs such as time and scroll
> position(s) as inputs and can update certain CSS properties (e.g.
> transforms, opacity) on elements that the compositor then uses.
> 3) Provide a way for pages to turn off async scrolling and make everything
> fast enough (and isolated enough) for pages to do 60fps updates from their
> main thread.
>
> All of these approaches have problems. Approach #1 is much more limited in
> its expressiveness than the alternatives. Approach #3 is more fragile and
> less composable than the alternatives --- sharing your main thread with any
> JS you don't control could cause jank. I like #2. It's strictly more
> powerful than #1 without the downsides of #3.
>
> Obvious question: how do we stop UIWorkers janking the compositor? We could
> give them a time budget (say 8ms). If a worker blows its budget, we notify
> it by sending it an event, we give up on it and composite anyway, and we
> run it separately from the compositor for a while. This requires an API
> design that lets UIWorkers still work, with some lag, when the compositor
> is not blocking on them, but that seems doable.
>

> Should UIWorkers have access to the full Worker API? It seems like there's
> no reason not to give them that.
>

> How should we explain the CSS effects of UIWorkers? A promising idea is to
> extend the Web Animations API to allow adding a new kind of animation
> effect to DOM elements --- a UIWorker-controlled effect. Essentially the
> UIWorker would then be responsible for computing the output of the timing
> function in each frame. The UIWorker could then animate *any* CSS property,
> though most property updates would require a round trip through main thread
> layout/rendering before they get rendered.
>

> One good thing about UIWorkers is extensibility. We can imagine providing
> touch input coordinates to UIWorkers to enable 60fps object dragging (with
> arbitrary effects like resistance, snapping, etc). UIWorkers could render
> to canvases: this would let you render VR with minimum latency, and let you
> render to canvases used by CSS masking for 60fps dissolves and clipping
> effects. If you really want to, you could go all Flipboard and render your
> entire UI to a canvas in the compositor --- if you keep hitting your
> deadlines.
>

> I like the idea of doing #2 before either #1 or #3.
>

> Rob
> --
> oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
> owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
> osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
> owohooo
> osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
> oioso
> oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
> owohooo
> osoaoyoso,o o‘oYooouo ofooooolo!o’o owoiololo oboeo oiono odoaonogoeoro
> ooofo
> otohoeo ofoioroeo ooofo ohoeololo.

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>

[James Long]

On Fri, Feb 20, 2015 at 3:20 AM, Jonas Sicking <jo...@sicking.cc> wrote:
> On Thu, Feb 19, 2015 at 7:11 PM, James Long <long...@gmail.com> wrote:
>> Note however that people in the native app world believe that it's
>> very important that whatever applies the animations is run on the same
>> thread that handles input. Otherwise it's very easy for animations to
>> get out-of-sync.
>
> Does that mean also doing the painting/compositing on the thread that
> handles input?

Not necessarily, the important part is that when the user makes a
gesture, an animation is updated with the event as input, and the
layout or other properties are changed, all on the same thread. It can
be actually painted on a separate thread. The state of the render tree
just needs to be updated in sync with the user input.

As Anne mentioned, when React Native comes out next month I would
highly recommend looking deep into it. They have a lot of good reasons
for choosing that architecture, and is something we could think about
working towards (long-term). I know the team personally and I can get
someone here in touch with them.

[Julien Wajsberg]

Le 20/02/2015 04:25, Robert O'Callahan a écrit :
> On Fri, Feb 20, 2015 at 4:02 PM, James Long <long...@gmail.com> wrote:
>
> Personally I think what we *really* need to be working on is making
> all of the DOM APIs asynchronous. That's what Servo needs anyway.
> That's a step in the right direction for #3, and we can see much more
> we can get out of the main thread after that.
>
> I don't see how making DOM APIs asynchronous really helps you achieve #3.
> There are some specific cases where custom layouts need to repeatedly
> measure content, that currently rely on APIs that trigger multiple
> sequential synchronous reflows, which would be better served by less
> synchronous APIs. We have nebulous plans for addressing that (it's not as
> easy as "make everything a Promise"). But those aren't the main problems
> mobile Web apps face right now AFAIK.
>

In Firefox OS development, this is part of the biggest issues we have
for very specific tasks.
I say '"biggest" especially because it's really difficult to work
around. And even when we work around we have other issues (see bug 1126391).

I don't want to say this is more important than the initial issue you're
trying to solve. But yes it's important.

--
Julien

[Jonas Sicking]

On Fri, Feb 20, 2015 at 8:11 AM, Julien Wajsberg <jwaj...@mozilla.com> wrote:
> Le 20/02/2015 04:25, Robert O'Callahan a écrit :
>> On Fri, Feb 20, 2015 at 4:02 PM, James Long <long...@gmail.com> wrote:
>>
>> Personally I think what we *really* need to be working on is making
>> all of the DOM APIs asynchronous. That's what Servo needs anyway.
>> That's a step in the right direction for #3, and we can see much more
>> we can get out of the main thread after that.
>>
>> I don't see how making DOM APIs asynchronous really helps you achieve #3.
>> There are some specific cases where custom layouts need to repeatedly
>> measure content, that currently rely on APIs that trigger multiple
>> sequential synchronous reflows, which would be better served by less
>> synchronous APIs. We have nebulous plans for addressing that (it's not as
>> easy as "make everything a Promise"). But those aren't the main problems
>> mobile Web apps face right now AFAIK.
>
> In Firefox OS development, this is part of the biggest issues we have
> for very specific tasks.

To be clear, we're not talking about making the whole DOM async, right?

I.e. we're not talking about adding an async element.appendChild() or
an async element.innerHTML?

What we're specifically talking about are async functions for getting
layout information? So async element.getComputedStyle() and
element.getClientRects()?

/ Jonas

[Ms2ger]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

These are particular cases that have been suggested, yes. (I've heard
suggestions for async innerHTML too, but I don't think that was in the
context of Servo.)

Either way, while some people have been dreaming about such APIs,
we're not actually working on any, or planning to start working on them.

HTH
Ms2ger

-----BEGIN PGP SIGNATURE-----

iQEcBAEBAgAGBQJU57l9AAoJEOXgvIL+s8n2jQMH/1e+/4FTpq/r/7qTJ5MEQ0mL
T1MbR+oVBr6xDCX4hu72ZCarmEC992coPnuMC4Y8NI4gD91XpAZhToFraGifXC4Z
zEJaqYfUSH8YRMIbwqmH49j1aEQjLW/q0XwHfsE3M7V4+7P7BaccQhotSQQz5Ubo
+SGtPbqfTmQr4FeZhXGCOGJAU76tBAdoKr0NanDh6FvuiIz/vB+ZVUSStsK2gwYw
FbIdXBPiJLlbW+VQjSz4NDXLrLIfzZ7jb5t43Ad0ljMeKltNSh3X6ad8Wv6DTExM
RHx6xGojSYhEvwelfbMhsxI2zCmNrxHoJ2sBSmnG+2kr5BDaUP1c/jneD5zH/Vk=
=EYaK
-----END PGP SIGNATURE-----

[Robert O'Callahan]

On Fri, Feb 20, 2015 at 9:11 PM, Jonas Sicking <jo...@sicking.cc> wrote:

> On Thu, Feb 19, 2015 at 6:27 PM, Robert O'Callahan <rob...@ocallahan.org>
> wrote:
> > Should UIWorkers have access to the full Worker API? It seems like
> there's
> > no reason not to give them that.
>

> There's two use-cases that I think argues against that.
>
> First off I'd like to enable saving a webpage for later viewing. Right
> now the best we can do is to save a pdf which means that all
> scrollable areas are lost.
>
> Saving a full page with all scripts is rarely going to work result in
> the user seeing the page they did when saving the page.
>
> What I'd like to be able to do is serialize the DOM, strip all script,
> save all linked stylesheets, images and media and change the
> references to them to use the correct relative URLs.
>
> However this would require also saving the UIWorker and enabling it to
> run in a reliable way. Otherwise the saved page won't have a working
> layout.

Second, when printing, I'd like to enable the user to change page
> layout and margins and have that reflow the page and generate a
> correct layout. This means that we need to enable the UIWorker to work
> on the cloned document in a reliable way.
>
I think both these usecases would work much better if we ensured that
> the UIWorker couldn't "interact with the outside world".
>

Your use-cases already fail today because many Web pages use scroll event
handlers and JS custom layouts. UIWorkers won't make the problem any worse.

Furthermore, I think facilitating expressive performant mobile Web apps is
much more important than those use-cases, and preventing UIWorkers from
interacting with the outside world (including, I assume, the application
main thread) would severely limit their usefulness.

Personally I think it's futile to try to save a complex app "for later
viewing" and expect anything useful to happen unless the whole app is
captured and able to operate offline.

[Robert O'Callahan]

On Fri, Feb 20, 2015 at 10:50 PM, Christopher Lord <cl...@mozilla.com>
wrote:

> I'd like to see #1 implemented first for two reasons; 1- I know this is
> easy to do given our platform, and I expect the same for other browser
> vendors, and 2- behaviour here is 100% predictable. There is nothing
> unexpected that can happen with this method and it's easy to employ. It
> doesn't require writing any JS, and developers are already used to
> optimising for the currently async-animatable CSS properties, so we can
> expect people not to do anything too silly with it.
>
> I'd like to see #3 implemented second, again for two reasons; 1- It's,
> again, reasonably easy to do. You can bypass a lot of machinery already,
> but making it explicit would make things easier and more obvious for
> people. I expect that part of this would also involve handling scrolling
> yourself too (we're on our way to removing the machinery to do synchronous
> scrolling, and it'd never respond the same way as async, so I think it'd be
> good to make it explicit). 2- There's a very clear use-case here. Not all
> web-apps are based around vertically scrolling a complex text layout, which
> is pretty much what the entire platform is built around. Games could get a
> nice boost here, as could some more unusual applications (e.g. scientific
> monitoring apps that mainly just want to draw charts very quickly (thinking
> specifically of Vernier LabQuest-style applications)).
>

It's easy to disable async scrolling. It's not so easy to "make everything

fast enough (and isolated enough) for pages to do 60fps updates from their
main thread".

It sounds like you have a specific idea in mind for #3 that's more concrete
than what I wrote. I'd like to hear it :-).

I'd like to see #2 implemented the least, mainly because I think it's the
> most complex solution, from both the platform and the user side. It's
> worrying that behaviour is unpredictable - if you have a time budget, the
> same code on one device can deliver wildly different results on another,
> and I think this would just introduce inconsistent and unexplainable
> behaviour (from a user-perspective), and encourage developers to
> special-case particular devices and do awful user-agent string matching.
> This is a neat idea, but I just see it breaking more things than it solves,
> vs. the other two ideas. I'd probably change my mind given a compelling
> use-case, but I'm struggling to think of anything that couldn't be
> implemented with a combination of #1 and #3.
>

If we restrict all Web apps to effects that work on the very lowest-end
devices, people won't write Web apps. We need to facilitate graceful
degradation, but there must be degradation.

Also, I realise this is just a throw-away detail, but an 8ms budget would
> be pretty huge. I doubt we can get our compositing done on mobile if we
> halve our budget (we can barely get it done as it is).
>

When that's the case, we may need to add one frame of latency so we can
overlap compositing frame N with running the UIWorkers for frame N+1.

I do strongly think that #1 is the way to go immediately - people have been
> asking for this for ages (myself included), and it covers just about 90% of
> what you'd want to do, in a safe, easy-to-use and performant way.
>

Unfortunately I don't think it gets you near 90% of what people want to do.
For example, it doesn't let you express effects that depend on the
direction of scrolling, nor does it help you with any effects that aren't
related to scrolling.

[eti...@segonzac.info]

On Friday, February 20, 2015 at 10:50:12 AM UTC+1, Christopher Lord wrote:
> I'd like to see #1 implemented first for two reasons; 1- I know this is
> easy to do given our platform, and I expect the same for other browser
> vendors, and 2- behaviour here is 100% predictable. There is nothing
> unexpected that can happen with this method and it's easy to employ. It
> doesn't require writing any JS, and developers are already used to
> optimising for the currently async-animatable CSS properties, so we can
> expect people not to do anything too silly with it.
>

I agree that this could cover a lot for use-cases. You _can_ build the navigation and the core transitions of your app around scrolling. But it implies a fair amount of nested scrollable areas, user initiated scrolling + scrolling APIs calls, snapping etc...
So I don't think #1 alone takes us there, not without touch-action and snap points support.

> [...]

> I'd like to see #2 implemented the least, mainly because I think it's the
> most complex solution, from both the platform and the user side. It's
> worrying that behaviour is unpredictable - if you have a time budget, the
> same code on one device can deliver wildly different results on another,
> and I think this would just introduce inconsistent and unexplainable
> behaviour (from a user-perspective)

> [...]

Aren't we just going to skip frames while the worker "runs separately from the compositor for a while"?
Not sure I fully understand how (from a user-perspective) a UI worker jank would be worse than a main thread jank during an interaction.

> I'd probably change my mind given a compelling
> use-case, but I'm struggling to think of anything that couldn't be
> implemented with a combination of #1 and #3. Also, I realise this is just a
> throw-away detail, but an 8ms budget would be pretty huge. I doubt we can
> get our compositing done on mobile if we halve our budget (we can barely
> get it done as it is).

Funny, the first advanced use-case that came to my mind for UI workers was re-ordering icons on the gaia homescreen :)

There's a bit of everything: we need to drag one icon, transition a screenful of icons to different positions and scroll the container when we're at one of the page's extremity. I think it's a good benchmark.

If touch coordinates are provided to UI workers, I believe it could yield a cleaner and faster implementation where we pull all the live-reordering code in the worker and just mutate the DOM on touchend (or after a postMessage from the UI Worker?).

But you know this code better than I do, how much of #3 would be required to get it to 60fps?

[Jonas Sicking]

I agree that it's not making the problem worse. However #1 is making
the problem better, so unless we can address this, this becomes an
argument for #1 over #2.

> Furthermore, I think facilitating expressive performant mobile Web apps is
> much more important than those use-cases, and preventing UIWorkers from
> interacting with the outside world (including, I assume, the application
> main thread) would severely limit their usefulness.
>
> Personally I think it's futile to try to save a complex app "for later
> viewing" and expect anything useful to happen unless the whole app is
> captured and able to operate offline.

The lack of ability to save for later viewing is a big problem on mobile.

The fact that native is so much better at retaining content to make it
available later when the user is offline is one of the big reasons
that the web is having a hard time competing with native.

So I do think that this is an important problem to solve.

/ Jonas

[Gavin Sharp]

What does it mean to "save your <complex web app> for later viewing"?
I don't think there's a lot of overlap between sites that would use
the functionality roc is proposing, and sites that make sense to "save
for later viewing".

Gavin

[Gordon Brander]

A couple thoughts from the perspective of a web app dev who has struggled with this problem:

I get excited when I hear a proven solution with well-understood scope like “animations and user interactions on the same thread”. I get nervous about ambitious and unknown solutions (async DOM). The web needs a pragmatic fix for animation and interaction and it needs it yesterday.

On February 23, 2015 at 10:37:20 , Jonas Sicking (jo...@sicking.cc) wrote:

The fact that native is so much better at retaining content to make it 
available later when the user is offline is one of the big reasons 
that the web is having a hard time competing with native. 

ServiceWorkers give us a powerful way to serialize state for offline persistence on a per-app basis. This is also essentially the approach native apps take, right? Your app can persist to a local DB, or not. You can persist via iCloud, or not. It’s up to you to serialize. Magic persistence for complex apps sounds mythical. I want to be wrong about this.

---

Gordon Brander
Sr Design Strategist
Mozilla

[Jonas Sicking]

On Mon, Feb 23, 2015 at 10:56 AM, Gavin Sharp <ga...@gavinsharp.com> wrote:
> What does it mean to "save your <complex web app> for later viewing"?

In gmail it would mean saving the set of emails that you are currently
looking at.

For facebook it would mean the news-feed content that's currently on
the screen, or the event invitation details that you are currently
looking at.

That's what we would get if we serialized the current DOM+CSS+images
to disk without any additional smarts. And all of these seem useful
for users.

> I don't think there's a lot of overlap between sites that would use
> the functionality roc is proposing, and sites that make sense to "save
> for later viewing".

I don't think that's true. Websites like nytimes often want a more
beautiful layout than what CSS can offer today. And once you create a
library for handling popular layout systems I have no doubt that
various tumblr templates or blogs will adopt UIWorker.

When a better alternative to the DOM appeared in the form of jQuery,
they adopted that. So when a better alternative to CSS appears though
UIWorker, I would expect to see a lot of adoption of that as well.

And, in a sense, broad adoption is what we *should* hope for with most
platform features that we implement.

/ Jonas

>> The fact that native is so much better at retaining content to make it
>> available later when the user is offline is one of the big reasons
>> that the web is having a hard time competing with native.
>>

[Robert O'Callahan]

On Tue, Feb 24, 2015 at 8:56 AM, Jonas Sicking <jo...@sicking.cc> wrote:

> On Mon, Feb 23, 2015 at 10:56 AM, Gavin Sharp <ga...@gavinsharp.com>
> wrote:
> > What does it mean to "save your <complex web app> for later viewing"?
>
> In gmail it would mean saving the set of emails that you are currently
> looking at.
>
> For facebook it would mean the news-feed content that's currently on
> the screen, or the event invitation details that you are currently
> looking at.
>
> That's what we would get if we serialized the current DOM+CSS+images
> to disk without any additional smarts. And all of these seem useful
> for users.
>

If we serialize the current DOM + CSS + images and remove the scripts, we
will present something to the user which is little more than a screenshot.
Users will immediately try to interact with it and discover that's broken.
That's a bad experience.

To present these in a non-broken way we must run page scripts. We should
not try to restrict the functionality of Web APIs for the sake of avoiding
this.

[Gavin Sharp]

On Mon, Feb 23, 2015 at 11:56 AM, Jonas Sicking <jo...@sicking.cc> wrote:

> On Mon, Feb 23, 2015 at 10:56 AM, Gavin Sharp <ga...@gavinsharp.com>
> wrote:
> > What does it mean to "save your <complex web app> for later viewing"?
>
> In gmail it would mean saving the set of emails that you are currently
> looking at.
>
> For facebook it would mean the news-feed content that's currently on
> the screen, or the event invitation details that you are currently
> looking at.
>
> That's what we would get if we serialized the current DOM+CSS+images
> to disk without any additional smarts. And all of these seem useful
> for users.
>

Having the browser store a serialized DOM of the app you're using is not
really the best way to address those use cases, IMO. If you care about
saving the content of your FB newsfeed or of your Gmail inbox, there are
cleaner ways to do that (often explicitly supported by the apps themselves,
e.g. "Print this email (to PDF)" on Gmail or "Save this post for later" on
Facebook).

Gavin

[Robert O'Callahan]

On Tue, Feb 24, 2015 at 7:36 AM, Jonas Sicking <jo...@sicking.cc> wrote:

> On Sun, Feb 22, 2015 at 3:45 AM, Robert O'Callahan <rob...@ocallahan.org>
> wrote:
> > Your use-cases already fail today because many Web pages use scroll event
> > handlers and JS custom layouts. UIWorkers won't make the problem any
> worse.
>
> I agree that it's not making the problem worse. However #1 is making
> the problem better, so unless we can address this, this becomes an
> argument for #1 over #2.
>

We can still do #1 as well as #2. It's a matter of priorities, and I think
enabling richer performant mobile apps is much higher priority than
enabling more scroll effects in saved Web pages.

> Furthermore, I think facilitating expressive performant mobile Web apps is
> > much more important than those use-cases, and preventing UIWorkers from
> > interacting with the outside world (including, I assume, the application
> > main thread) would severely limit their usefulness.
> >
> > Personally I think it's futile to try to save a complex app "for later
> > viewing" and expect anything useful to happen unless the whole app is
> > captured and able to operate offline.
>
> The lack of ability to save for later viewing is a big problem on mobile.
>
> The fact that native is so much better at retaining content to make it
> available later when the user is offline is one of the big reasons
> that the web is having a hard time competing with native.
>
> So I do think that this is an important problem to solve.
>

Echoing Gordon and Gavin: saving the DOM+CSS is totally unlike the way
native apps work and not an adequate solution to this problem.

[Jonas Sicking]

On Mon, Feb 23, 2015 at 12:07 PM, Robert O'Callahan
<rob...@ocallahan.org> wrote:

> On Tue, Feb 24, 2015 at 8:56 AM, Jonas Sicking <jo...@sicking.cc> wrote:
>>
>> On Mon, Feb 23, 2015 at 10:56 AM, Gavin Sharp <ga...@gavinsharp.com>
>> wrote:
>> > What does it mean to "save your <complex web app> for later viewing"?
>>
>> In gmail it would mean saving the set of emails that you are currently
>> looking at.
>>
>> For facebook it would mean the news-feed content that's currently on
>> the screen, or the event invitation details that you are currently
>> looking at.
>>
>> That's what we would get if we serialized the current DOM+CSS+images
>> to disk without any additional smarts. And all of these seem useful
>> for users.
>
>

> If we serialize the current DOM + CSS + images and remove the scripts, we
> will present something to the user which is little more than a screenshot.
> Users will immediately try to interact with it and discover that's broken.
> That's a bad experience.
>
> To present these in a non-broken way we must run page scripts. We should not
> try to restrict the functionality of Web APIs for the sake of avoiding this.

I think this is letting perfect get in the way of good.

It's going to take many years before a majority of the websites that a
given user uses to have made themselves "offlineable". I think waiting
for that long is going to add a lot of risk to the web's chance of
surviving the switch to mobile.

/ Jonas

[Gordon Brander]

It’s funny: I have come to the opposite conclusion for the same reason.

The Good: getting 60fps interactions and animations in web apps using a proven approach (UI and interaction thread).
The Ideal: also automatically serializing those apps for offline use.

While I very much want the ideal to exist, I wouldn’t be willing to risk the “good” for it. Or perhaps it is possible to have it all. I don’t understand the tradeoffs deeply enough to know.

If I had to prioritize, I would say that interactions and animations are the basis of competition for UI and will be as long as we’re using touch. We lag way behind there. With Service Workers, we now have parity in offline capability with native apps (serialize it yourself), even if practical usage lags behind.

---
Gordon Brander
Sr Design Strategist
Mozilla

On February 23, 2015 at 13:21:55 , Jonas Sicking (jo...@sicking.cc(mailto:jo...@sicking.cc)) wrote:

> On Mon, Feb 23, 2015 at 12:07 PM, Robert O'Callahan

> wrote:

> > On Tue, Feb 24, 2015 at 8:56 AM, Jonas Sicking wrote:
> >>
> >> On Mon, Feb 23, 2015 at 10:56 AM, Gavin Sharp

> >> wrote:
> >> > What does it mean to "save your for later viewing"?

> >>
> >> In gmail it would mean saving the set of emails that you are currently
> >> looking at.
> >>
> >> For facebook it would mean the news-feed content that's currently on
> >> the screen, or the event invitation details that you are currently
> >> looking at.
> >>
> >> That's what we would get if we serialized the current DOM+CSS+images
> >> to disk without any additional smarts. And all of these seem useful
> >> for users.
> >
> >
> > If we serialize the current DOM + CSS + images and remove the scripts, we
> > will present something to the user which is little more than a screenshot.
> > Users will immediately try to interact with it and discover that's broken.
> > That's a bad experience.
> >
> > To present these in a non-broken way we must run page scripts. We should not
> > try to restrict the functionality of Web APIs for the sake of avoiding this.
>
> I think this is letting perfect get in the way of good.
>
> It's going to take many years before a majority of the websites that a
> given user uses to have made themselves "offlineable". I think waiting
> for that long is going to add a lot of risk to the web's chance of
> surviving the switch to mobile.
>
> / Jonas

[Robert O'Callahan]

On Tue, Feb 24, 2015 at 10:57 AM, Gordon Brander <gbra...@mozilla.com>
wrote:

> It’s funny: I have come to the opposite conclusion for the same reason.
>
> The Good: getting 60fps interactions and animations in web apps using a
> proven approach (UI and interaction thread).
> The Ideal: also automatically serializing those apps for offline use.
>

I agree with this.

Also, there is a way to get "save for later viewing" to work with complex
apps: serialize all the application state --- DOM, CSS, JS heap, workers,
etc --- and revive it later, possibly in a jail that blocks it from
accessing the network when revived, but possibly not. A lot of work to
implement, but not terribly difficult, and it would give you a great
experience, without restricting (explicitly or implicitly) what Web apps
can do.

[Jonas Sicking]

On Mon, Feb 23, 2015 at 3:02 PM, Robert O'Callahan <rob...@ocallahan.org> wrote:
> On Tue, Feb 24, 2015 at 10:57 AM, Gordon Brander <gbra...@mozilla.com>
> wrote:
>>
>> It's funny: I have come to the opposite conclusion for the same reason.
>>
>> The Good: getting 60fps interactions and animations in web apps using a
>> proven approach (UI and interaction thread).
>> The Ideal: also automatically serializing those apps for offline use.
>
>
> I agree with this.
>
> Also, there is a way to get "save for later viewing" to work with complex
> apps: serialize all the application state --- DOM, CSS, JS heap, workers,
> etc --- and revive it later, possibly in a jail that blocks it from
> accessing the network when revived, but possibly not.

I think this would fall over more often than not.

Most developers will not write their code to be resilient in the face
of being suspended for extended periods of time. Upon reopening they
would likely display error dialogs, or updated version of whatever was
saved.

In fact, I would argue that it's directly harmful to keep even some of
the app logic working. It could cause things like overwriting server
state with whatever state was serialized into the saved copy.

The usecase here isn't to magically make apps available offline.
That's unlikely to ever work well enough that it's worth doing without
author cooperation. The use case here is the ability to save some
*information* for later access.

If I'm about to leave house to visit a friend, i'm interested in
saving the directions for getting to that friend. I'm less interested
in having an updated facebook feed with me.

/ Jonas

[Karl Dubost]

Le 24 févr. 2015 à 08:02, Robert O'Callahan <rob...@ocallahan.org> a écrit :
> Also, there is a way to get "save for later viewing" to work with complex
> apps: serialize all the application state --- DOM, CSS, JS heap, workers,
> etc --- and revive it later, possibly in a jail that blocks it from
> accessing the network when revived, but possibly not.

Some use cases for this:

* The very useful scrapbook Add-on.
https://addons.mozilla.org/en-US/firefox/addon/scrapbook/

* The crash or quit your browser and restart (with 100 tabs) and out of the network.
We want to be able to see all the tabs content before the browser has been closed.

* A "browser time machine", aka the possibility to save dated copies of the same site
ala time machine backup, or web.archive.org or http://timetravel.mementoweb.org/about/
or google street view and it's timeline feature (top right these days).


--
Karl Dubost, Mozilla
http://www.la-grange.net/karl/moz

[noah...@gmail.com]

> I think this would fall over more often than not.
>
> Most developers will not write their code to be resilient in the face
> of being suspended for extended periods of time. Upon reopening they
> would likely display error dialogs, or updated version of whatever was
> saved.
>
> In fact, I would argue that it's directly harmful to keep even some of
> the app logic working. It could cause things like overwriting server
> state with whatever state was serialized into the saved copy.
>
> The usecase here isn't to magically make apps available offline.
> That's unlikely to ever work well enough that it's worth doing without
> author cooperation. The use case here is the ability to save some
> *information* for later access.
>
> If I'm about to leave house to visit a friend, i'm interested in
> saving the directions for getting to that friend. I'm less interested
> in having an updated facebook feed with me.
>
> / Jonas

The Service Worker spec addresses various offline use cases quite well. http://www.w3.org/TR/2015/WD-service-workers-20150205/

[Robert O'Callahan]

On Tue, Feb 24, 2015 at 12:09 PM, Jonas Sicking <jo...@sicking.cc> wrote:

> I think this would fall over more often than not.
>
> Most developers will not write their code to be resilient in the face
> of being suspended for extended periods of time. Upon reopening they
> would likely display error dialogs, or updated version of whatever was
> saved.
>
> In fact, I would argue that it's directly harmful to keep even some of
> the app logic working. It could cause things like overwriting server
> state with whatever state was serialized into the saved copy.
>

You missed the part about the jail.

[Jonas Sicking]

On Mon, Feb 23, 2015 at 4:01 PM, Robert O'Callahan <rob...@ocallahan.org> wrote:
> On Tue, Feb 24, 2015 at 12:09 PM, Jonas Sicking <jo...@sicking.cc> wrote:
>>
>> I think this would fall over more often than not.
>>
>> Most developers will not write their code to be resilient in the face
>> of being suspended for extended periods of time. Upon reopening they
>> would likely display error dialogs, or updated version of whatever was
>> saved.
>>
>> In fact, I would argue that it's directly harmful to keep even some of
>> the app logic working. It could cause things like overwriting server
>> state with whatever state was serialized into the saved copy.
>
> You missed the part about the jail.

Sure, the jail would prevent server overwrites.

But there's still problems like code behaving strange due to being
suspended so long, for example by removing content from the screen
that is "out of date".

Additionally the jail is more likely to cause error dialogs to appear
due to failed network requests.

I would also argue that keeping a lot of the page "working" by running
scripts, is going to be even more confusing when that page is then
unable to actually save anything due to the jail. Rather than nuking
all the scripts and ensuring that the only thing the user can do is
scroll around on the page.

/ Jonas

[Patrick Walton]

> 2) UIWorker: some kind of JS worker that receives callbacks during
composition; each callback can take inputs such as time and scroll
position(s) as inputs and can update certain CSS properties (e.g.
transforms, opacity) on elements that the compositor then uses.
>
>

How should we explain the CSS effects of UIWorkers? A promising idea is to
extend the Web Animations API to allow adding a new kind of animation
effect to DOM elements --- a UIWorker-controlled effect. Essentially the
UIWorker would then be responsible for computing the output of the timing
function in each frame. The UIWorker could then animate *any* CSS property,
though most property updates would require a round trip through main thread
layout/rendering before they get rendered.

This sounds great from Servo's point of view. I'd like to suggest that we
allow browsers to expand the set of properties that can be set off the main
thread beyond the set that are considered "compositor" or "accelerated"
properties today. (This was my main issue with Google's "AnimationProxy":
it exposed an API tailored only to those properties that could be handled
without a round trip through layout in current browser architectures, but
browser engines that have off-main-thread layout like Servo need not
restrict themselves to those.)

CSS transitions as implemented in my work-in-progress Servo branch go
through a "back door" into layout that allows them to directly poke
property values into the layout data structures without ever looking at the
DOM (i.e. without going through the "front door" of CSS style recalc),
which allows script to run concurrently with all CSS transitions. Since we
went through such pains to make this "back door" work in Servo, it seems
desirable to expose it directly to Web authors, and UIWorker sounds like as
reasonable an approach as any.

Patrick

On Thu, Feb 19, 2015 at 6:27 PM, Robert O'Callahan <rob...@ocallahan.org>
wrote:

> Last week in Sydney I spent a lot of time talking to Chrome devs about
> different approaches for 60fps effects in Web pages. There are three
> different kinds of approaches being discussed (so far):
> 1) Apple's animation-timeline proposal, which lets CSS animations use
> scroll position as an input instead of time.
> 2) UIWorker: some kind of JS worker that receives callbacks during
> composition; each callback can take inputs such as time and scroll
> position(s) as inputs and can update certain CSS properties (e.g.
> transforms, opacity) on elements that the compositor then uses.

> 3) Provide a way for pages to turn off async scrolling and make everything

> fast enough (and isolated enough) for pages to do 60fps updates from their

> main thread.
>
> All of these approaches have problems. Approach #1 is much more limited in
> its expressiveness than the alternatives. Approach #3 is more fragile and
> less composable than the alternatives --- sharing your main thread with any
> JS you don't control could cause jank. I like #2. It's strictly more
> powerful than #1 without the downsides of #3.
>
> Obvious question: how do we stop UIWorkers janking the compositor? We could
> give them a time budget (say 8ms). If a worker blows its budget, we notify
> it by sending it an event, we give up on it and composite anyway, and we
> run it separately from the compositor for a while. This requires an API
> design that lets UIWorkers still work, with some lag, when the compositor
> is not blocking on them, but that seems doable.
>

> Should UIWorkers have access to the full Worker API? It seems like there's
> no reason not to give them that.
>

> How should we explain the CSS effects of UIWorkers? A promising idea is to
> extend the Web Animations API to allow adding a new kind of animation
> effect to DOM elements --- a UIWorker-controlled effect. Essentially the
> UIWorker would then be responsible for computing the output of the timing
> function in each frame. The UIWorker could then animate *any* CSS property,
> though most property updates would require a round trip through main thread
> layout/rendering before they get rendered.
>
> One good thing about UIWorkers is extensibility. We can imagine providing
> touch input coordinates to UIWorkers to enable 60fps object dragging (with
> arbitrary effects like resistance, snapping, etc). UIWorkers could render
> to canvases: this would let you render VR with minimum latency, and let you
> render to canvases used by CSS masking for 60fps dissolves and clipping
> effects. If you really want to, you could go all Flipboard and render your
> entire UI to a canvas in the compositor --- if you keep hitting your
> deadlines.
>
> I like the idea of doing #2 before either #1 or #3.
>

> Rob
> --
> oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
> owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
> osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
> owohooo
> osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
> oioso
> oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
> owohooo
> osoaoyoso,o o‘oYooouo ofooooolo!o’o owoiololo oboeo oiono odoaonogoeoro
> ooofo
> otohoeo ofoioroeo ooofo ohoeololo.

[say...@gmail.com]

On Monday, 23 February 2015 10:37:09 UTC-8, Jonas Sicking wrote:
>
> The lack of ability to save for later viewing is a big problem on mobile.
>
> The fact that native is so much better at retaining content to make it
> available later when the user is offline is one of the big reasons
> that the web is having a hard time competing with native.
>
> So I do think that this is an important problem to solve.

ServiceWorker + IndexedDB should get you pretty far--most apps that users want to use offline are client-side rendered, and certainly all of the ones that get close to smooth on mobile are. This means all of the presentation data is encoded in cached JS+CSS already.

- Rob

[say...@gmail.com]

On Thursday, 19 February 2015 18:27:48 UTC-8, Robert O'Callahan wrote:
> Last week in Sydney I spent a lot of time talking to Chrome devs about
> different approaches for 60fps effects in Web pages.

One complaint I see that is pretty common: image decoding on the main thread in WebKit. I don't remember how this works in Gecko, but it's definitely enough to blow the 15ms budget while scrolling. It also bumps into various image-discarding strategies that are done in order to save memory.

- Rob

[Seth Fowler]

In Gecko, we will only decode images on the main thread:

(1) If they’re very small and this is the first time we’re decoding them and our heuristics suggest that decoding them quickly is important.
(2) If it’s absolutely required because of a synchronous API, e.g. canvas.drawImage.

We’ve been gradually tightening the rules that will trigger case 1, and in the long term I think it’s quite likely that we’ll get rid of it totally. We just need to make sure we don’t regress the user experience by doing so.

Case 2 cannot be totally avoided, though it should be possible to be clever about it and defer the work in some circumstances. That’s something I hope to investigate in the future, but we’re not there yet.

- Seth

[srdjan.p...@gmail.com]

On Friday, February 20, 2015 at 3:27:48 AM UTC+1, Robert O'Callahan wrote:
> Obvious question: how do we stop UIWorkers janking the compositor? We could
> give them a time budget (say 8ms).

Onboard USB hubs usually add 8ms latency, and John Carmack once noted that a repaint of an LCD monitor triggered by a game controller (with all the hardware latencies combined) can take longer than a transatlantic network roundtrip. Even if the introduced latency is consistent, it could make UI Workers less suitable for gaming.

Kind regards,
Srđan Prodanović
http://spamoji.com

[Jonas Sicking]

Having thought about this overnight...

Ultimately I don't feel like I'm convincing anyone on this thread. And
I definitely think this is the layout team's call and not mine.

I think we're making a different judgement call about how good your
average websites will be to use features like ServiceWorkers to cache
content for offline usage, and cache the content that users actually
want to be available. (I don't for example think that native apps are
particularly good at this).

And/or we're making different judgement calls about how important it
is for users to have content available offline.

I would be interested to hear what features that we are getting in
return though. I.e. what features we gain by enabling the UIWorker to
access the world outside of the DOM. Maybe those features are clearly
more valuable than browser-provided offline capabilities and printing.

Either way though, I am sad that in our pursuit of competing with
native, it feels like we're giving up more and more of the advantages
that we do have over native.

/ Jonas

[Robert O'Callahan]

I've posted a followup of sorts to public-houdini:
https://lists.w3.org/Archives/Public/public-houdini/2015Mar/0020.html
There have been no replies, so I suspect I've said something terribly
incomprehensible and/or wrong :-).