setImmediate?

[Tony Gentilcore]

I'd like to get a concise explanation (and consensus) for why we
should or should not implement setImmediate[1].

I'm personally not wild about it because I don't understand the need
for it (although Zakas makes a case[2]). The reason I'm bringing this
up is that there seems to be tremendous desire for this API in the web
dev community and I keep getting pinged about it.

Arguments I've heard against supporting it:
1. "We have to clamp setTimeout(0) because sites misuse it. So when
sites misuse setImmediate() we'll just have to clamp that." I think
this argument breaks down because sites are polyfilling it with
postMessage anyway so we'd have to clamp that if that comes up.
2. "We don't have to support it because there is a polyfill." The
problem with this is that the polyfills are 100-200 lines[3].
3. "We shouldn't encourage sites to yield so much that they need it."
This is where I'd like to understand what the use case really is?

There is a feature request[4] for it which we marked WontFix. I'm just
not seeing a satisfying justification one way or the other in that
thread.

-Tony

[1] https://dvcs.w3.org/hg/webperf/raw-file/tip/specs/setImmediate/Overview.html
[2] http://www.nczonline.net/blog/2013/07/09/the-case-for-setimmediate/
[3] https://github.com/NobleJS/setImmediate,
https://code.google.com/p/closure-library/source/browse/closure/goog/async/nexttick.js,
https://npmjs.org/package/set-immediate
[4] http://crbug.com/146172

[Adam Barth]

[+jamesr, who marked the bug WontFix]

It's possible the world has changed since we marked the bug WontFix.  In particular, it looks like the feature is now on the standards track.

James, do you still have concerns about the feature?

Adam

[PhistucK]

It was on the same standards track back then when the issue was filed and when the issue was WontFixed (which was, arguably, not too long ago, less than a year). The specification (an Editor's Draft from 2011, that is) is edited only by Microsoft and in the issue, James stated that they have been quiet about it on the mailing list.

However, for what it is worth, I support implementing this. setTimeout(callback, 0) always seemed like a hack.

☆PhistucK

[James Robinson]

Thanks for bringing this up, Tony.  I'm not sure a concise explanation is possible here, but I can try to explain my perspective.  Unfortunately much has been written about this topic based on incorrect information about how timers and/or setImmediate work and some of the issues are rather subtle.  We should definitely strive to explain this area more correctly and precisely.

Nearly everything I've read starts off with an incorrect idea about how timer clamping works or why it's in place.  The way timer clamping works [1] is every task has an associated timer nesting level.  If the task originates from a setTimeout() or setInterval() call, the nesting level is one greater than the nesting level of the task that invoked setTimeout() or the task of the most recent iteration of that setInterval(), otherwise it's zero.  The 4ms clamp only applies once the nesting level is 4 or higher.  Timers set within the context of an event handler, animation callback, or a timer that isn't deeply nested are not subject to the clamping.  This detail hasn't been reflected in the HTML spec until recently (and there's an off-by-one error in it right now), but has been true in WebKit/Blink since 2006 [2] and in Gecko since 2009 [3].  The practical effect of this is that setTimeout(..., x) means exactly what you think it would even for x in [0, 4) so long as the nesting level isn't too high.

Back before the initial public launch of Chrome, we did a number of experiments with this clamping including removing it completely.  We found that removing the clamp completely made us quickly aware that too many pages leak infinitely running zero-delay timers, most of which did nothing useful in all sorts of creative ways [4].  These sorts of errors are not immediately obvious to the page author since the page is generally visually and functionally fine but chews through the user's CPUUnclamped, these timers consumed 100% of the CPU but with even a fairly small clamp the CPU usages decreased significantly.  If each iteration of the timers takes time Xms then a clamp of Yms results in a CPU usage of X/Y, which is very small when X is very small.  Modern JS engines can give us very small values for X, so we dropped the clamp from 10ms down to 4ms but kept the nesting level behavior unchanged.

With a better understanding of timer clamping, let's consider the possible use cases for scheduling asynchronous work.  One is to time work relative to display updates.  Obviously requestAnimationFrame() is the right API to use for animations, but it's also the right choice for one-off uses.  To perform work immediately before the next display - for example to batch up graphical updates as data is streamed in off the network - just make a one-off requestAnimationFrame() call.  To perform work immediately -after- a display, just call setTimeout() from inside the requestAnimationFrame() handler.  The nesting level is zero within a rAF() callback so the timeout parameter will not be clamped to 4.

Another common use case is to run a portion of an event handler asynchronously.  If it's important that the asynchronous work happens after a display, for instance to make sure the user sees the visual effects from the synchronous part of the user handler, schedule the work using requestAnimationFrame().  Otherwise, setTimeout() works just fine even if the work happens in a few chunks.  The clamping only applies if the work executes in a large number of batches.

Cases where a large amount of work - more than can reasonably be performed in 2-3 batches - has to be executed without janking up the main thread.  Ideally, such work would be performed on a web worker where it can simply run to completion without interfering with the main thread, but that's not feasible for many use cases with the current web platform.  That leaves running the work on the main thread in small-ish chunks.  One way to structure this (as seen on several of the setImmediate demos from Microsoft such as http://ie.microsoft.com/testdrive/Performance/setImmediateSorting/Default.html) is this:

// Performs one atomic unit of work, returns true if there is more work to do.

function doAtom() { ... }

function tick() {

  if (doAtom())

    window.setImmediate(tick);  // or setTimeout

}

This looks valid at a glance but is actually a deeply unfortunate way to structure the code since it means jumping in and out of the JS VM for every atom of work.  Exiting and entering the VM is far more expensive than running small amounts of script in modern JS engines.  The Microsoft demo does a few bits of math and swaps two elements within an array in each callback.  Perversely, the faster JS implementations become more time is (proportionally) wasted on overhead since each iteration with its relatively fixed VM overhead takes less time to do the same amount of work. It's much better to batch the work up into reasonably-sized chunks before yielding:

var CHUNK_SIZE_MS = ...;

function tick() {

  var id = window.setTimeout(tick, CHUNK_SIZE_MS);

  var start = now();  // Remember to use a good time function!

  while (now() - start < CHUNK_SIZE_MS)) {

    if (!doAtom()) {

      // Ah, we're all done!  Clear the scheduled timer and return.

      window.clearTimeout(id);

      return;

    }

  }

  // We must have exceeded the timeout without finishing out work, so return control to the browser until we get called back.

}

This avoids jumping in and out of the VM on every iteration and actually gets more efficient as JS engines get faster since more work is executed per tick.  Additionally, so long as CHUNK_SIZE_MS is >= 4ms, timer clamping has no effect on this code.  By the time the tick() function exits, either the work is done or enough time has elapsed that the timer is ready to fire.

Of course, being ready to fire and firing are not the same thing.  This gets to the heart of what I suspect a lot of the discussion and misinformation is really trying to address which is the problem of queuing and scheduling on the main JS thread.  JavaScript is both single-threaded and run-to-completion and has to compete with plenty of other tasks that have to run on the main thread.  Since the browser can't interrupt a task to start running another without violating run-to-completion semantics, the only real choice the browser has when scheduling work is picking a task to run when the event loop is idle.  This is a very short part of the HTML spec [5] but one of the trickiest parts of a browser implementation.  We've spent a lot of time trying to improve this and have a lot of work left to do.

The problem statement from a browser's perspective is this: Given a set of runnable tasks, pick one and run it.  Not all pending tasks can be run at any time - pending tasks are ordered into task queues that must be drained in FIFO order.  If a page post two messages to a frame, the browser has to deliver them in the order posted although it can interleave other tasks between the two messages.  One a task is picked the thread is blocked until that task completes.  The goal is to get whatever the user wants done as quickly as possible, but that's not always easy to determine what the user wants and whether draining a particular task queue will get them closer to that goal.  We do what we can with the information we have.  We make sure that display-related tasks such as firing animation callbacks happens as regularly as we can and as close to in sync with the physical display as possible with throttling when we detect bottlenecks in the graphics pipeline.  We dispatch input-related tasks as quickly as we can to minimize latency, but also have some sophisticated throttling and batching logic to make sure high-frequency inputs like mouse moves don't end up flooding the system.  We're just starting to tweak how we deliver progress events for network loads when the user is interacting with the page to try to help some situations we've seen on mobile browsers.  For task sources where we don't have a strong signal or we just haven't paid much attention to yet, we default to a FIFO-ish ordering.

Timers are a bit of a black box in this system.  In contrast with input event handlers, which are tied to the input event and thus the user's perceived latency, or requestAnimationFrame callbacks, which are tied in to the display pipeline, timers do not have any semantic meaning other than some opaque work that the page wants to do as some point in time.  This means there isn't much a browser can do other than interleave timer tasks with other work as often as is possible.  The big disappointment of setImmediate() for me is that it provides no additional information that the browser can use to perform task servicing more efficiently.  If it produced tasks that contained some information about what they were trying to accomplish then we could try to service it at a better time.  As it is, it doesn't provide anything new except for avoiding timer clamping which only applies in cases where it's really useful.

For non-browser contexts using JS, such as NodeJS, a different set of constraints apply and I wouldn't at all be surprised if a different set of APIs made sense.  Node is a server technology so the UI responsiveness considerations do not apply.

I want to be really clear here that I don't think we are doing the best job we could in minimizing queuing delays.  In fact, we have a very long way to go to get there.  One thing we can do to reduce delays is to move as much work as possible off of the JS thread.  We've made a lot of progress here with our threaded HTML parsing and graphics stack, but have a lot more to do both in terms of implementation and in providing appropriately designed platform capabilities.  Another thing to do is to simply speed up JS and JS-related operations by adding more optimizations and primitives to the language and moving work like garbage collection out of the main thread.  Last, but certainly not least, we need to improve the way we choose tasks to run.  Today, it's not all that hard to change the performance characteristics of a page by sending work through different task sources depending on the vagaries of how these sources are scheduled in different browser implementations.  I don't think we batch up network and input tasks the way we should, perform style+layout work at the times we should, or service timers as the right frequency relative to other task sources.  These are all things we have been working on and will continue to work on.  What's important to get from the platform is enough information about what tasks are trying to accomplish to know when to run them.

The spec is as PhistucK notes in roughly the same shape today as it was a year ago.  The spec's quality is not very high - the non-normative text contradicts the normative text and the normative text is not very good.  It doesn't specify a task source at all for tasks it generates.  Given that the spec hasn't received any traction from browser vendors outside of IE, I expect that a year from today the spec will still be in the same state.

As for the polyfill, it puzzles me.  It's many hundreds of lines of code, yes, but most of this is to support older browsers like IE8 that will never have a newly proposed API.  Other than that, I don't really understand what the purpose of it is in the first place.  setTimeout(..., 0) will generally do what callers want unless they're already doing something inefficient, but that needs to be handled by looking at the caller.  I could believe that manipulating the task source changes the performance characteristics of a page, or especially a synthetic microbenchmark, as this is one of the most complex areas of browser implementations, but without a semantic underpinning any wins this produces will be fleeting.

- James

[1] http://www.whatwg.org/specs/web-apps/current-work/multipage/timers.html#timers

[2] http://trac.webkit.org/changeset/17156/trunk/WebCore/bindings/js/kjs_window.cpp

[3] http://hg.mozilla.org/mozilla-central/rev/350ffc1d793a

[4] One example was from a very popular newspaper website that included a fairly old version of a javascript library.  This library had a polyfill for a load event that used polling with a setInterval().  Unfortunately, the script stored the interval ID in a global and the embedding page included the script multiple times from different widgets, which clobbered the ID for all setInterval() calls but the last one.  The page then had multiple zero-delay setInterval()s running for the entire lifetime of the page, each doing nothing useful except for attempt to cancel themselves.

[5] http://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#processing-model-4

[Malte Ubl]

I'm the author of the hacky-fill in the closure library and Paul Irish asked me to chime in as to our use cases. Reading James' post I think a lot of the confusion comes from the impression that people who want a fast setImmediate-like scheduling mechanism are interested in drawing things. There might be drawing as the ultimate side effect, but initially the relevant code is just application logic.

1. Guarantee async execution – i.e. in promises implementations. i.e. to ensure that adding a callback to a promise never triggers executing the callback before the current JS execution ends. This avoids very hard to find bugs.

Example:
var foo = 1;
promiseThingie.addCallback(function() { console.log(foo) });
foo++;

In the above code we never want foo to be output as having the value 1 and the average JS programmer doesn't expect that. There is a significant large chunk of code in the real world written like that and it leads to deeply nested timers (think 5 or 10 levels deep). At 4ms clamping per timeout it is no longer possible to maintain 60fps given sufficiently deeply nested timers.

Essentially the real world code ends up being equivalent to

onAnimationFrame = function() {
  setTimeout(function() {
   setTimeout(function() {
    setTimeout(function() {
     setTimeout(function() {
      setTimeout(function() {
       setTimeout(function() {
        setTimeout(function() {
          // Draw!
        }, 0);
       }, 0);
      }, 0);
     }, 0);
    }, 0);
   }, 0);
  }, 0);
};

2. Allow pushing stuff to a queue during JS execution and then handling the pushed stuff after execution ends. I.e. to avoid sending more than one HTTP request to an endpoint that would support answering everything that is currently needed in a single response. Another classic example is implementing a dirty flag and then running code once to check for that flag.

We see much larger than expected performance wins of using our hacky-fill for setImmediate over setTimeout(…, 0). The reason is that we often use it to avoid making too many HTTP requests, but calling setTimeout will queue us too far in the future. In a scenario where you compete over resources with other (possibly non-cooperating) resources on a page, this can have a very large impact in the upper percentiles (as opposed to the 4ms that one would expect).

Arguably for these use cases it would be better to use a solution that does not require yielding to the event loop, but unfortunately that cannot be hacky-filled outside of Chrome without controlling all stack entry points.

I'm somewhat troubled by the argument that the existence of a polyfill would be relevant for not implementing – I thought we wanted to make the web platform good. Ironically the hacky-fill for old IEs is very simple. The worst currently is Firefox as it requires creating an iframe and thus carries significant memory overhead: Imagine a page with 2 different banner providers and 3 social buttons: That makes 5 additional iframes per window, just because there is no setImmediate implementation in the browser.

[Darin Fisher]

The above looks really bad.  I'm not sure why it starts with an animation frame as the first setTimeout defeats the purpose of using rAF.  This seems like anti-pattern to me.

setImmediate in place of setTimeout might often cause the Draw function to happen during the animation frame, but nothing ensures that it will.  setImmediate still just posts tasks to the main run loop, which could be behind a lot of other work in the run loop.

 

2. Allow pushing stuff to a queue during JS execution and then handling the pushed stuff after execution ends. I.e. to avoid sending more than one HTTP request to an endpoint that would support answering everything that is currently needed in a single response. Another classic example is implementing a dirty flag and then running code once to check for that flag.

It seems like if the workload involves swizzling visible DOM, then rAF is the correct way to defer work.  Otherwise, a timer seems fine.  But, in either case, it seems like nested calls to rAF or setTimeout is an anti-pattern.  It seems like one should only schedule a single rAF or setTimeout the first time a portion of code is made dirty.

For example:

var needsLayout = false;

function layout() {

  needsLayout = false;

  // Manipulate DOM, draw into canvases, etc.

}

function setNeedsLayout() {

  if (needsLayout)

    return;

  needsLayout = true;

  requestAnimationFrame(layout);

}

Now, all other events handlers just call setNeedsLayout() instead of directly manipulating their respective DOM.  They need to of course record the manipulations that they need to have done at layout time, and that may require some work to express, but it seems do-able.

It seems like you could build a similar system for non-UI cases, where a single setTimeout(0) is used to drive a work queue.  If you avoid nesting setTimeout, then you will avoid the clamping.

-Darin

[Rafael Weinstein]

I believe the curent idea is that DOM Promises will always be
scheduled on the current microtask work queue:

http://dom.spec.whatwg.org/#promises
https://www.w3.org/Bugs/Public/show_bug.cgi?id=22296

Malte,

Would this meet your requirements for the ideal solution? If not, why not?

On Mon, Jul 29, 2013 at 8:17 AM, Malte Ubl <malt...@google.com> wrote:

[Malte Ubl]

Just wanted to state that I wasn't talking about good code, I was talking about code that exists, that won't get rewritten in the next 3 years, and that is much faster with setImmediate in place

@Rafael: Yes, that is perfect. One issue as stated is that it can't be polyfilled without controlling all stack entry points. We do actually control all stack entry points in our code (Yes, by overriding global setTimeout, XHR, …. The reason is that as of now error reporting via window.onerror is broken), so it would be an option for us.

[Darin Fisher]

That sounds kind of scary.  As we use promises more and more, aren't we running the risk of starving out other workloads?  Are there more pointers to where this was discussed?

-Darin

On Mon, Jul 29, 2013 at 9:46 AM, Rafael Weinstein <raf...@google.com> wrote:

[Ben Vanik]

As someone who is using setImmediate in several places I want to emphasize what Maite said:

"I think a lot of the confusion comes from the impression that people who want a fast setImmediate-like scheduling mechanism are interested in drawing things."

Darin's points about layout/etc make sense, but I've rarely used it in that case. It's always for deferred execution from promises/event dispatches/etc. Nested timeouts like Maite mentions are an antipattern, sure, but the code is never written like that and Maite was just giving an example of what the expanded form looks like. When you're dealing with hundreds of thousands of lines of javascript from dozens to hundreds of different delay-loaded modules it may not be possible to 'just use a single setTImeout'.

Rafael:

That's awesome and will be great when it's implemented but not all code can use Promises in the near to medium term (legacy code/frameworks, need for compatibility with non-Promise-supporting browsers, etc). The reason people (such as myself) are asking for setImmediate is so that we can implement something like Promises ourselves in a way that is easy to patch into existing codebases.

On Mon, Jul 29, 2013 at 9:46 AM, Rafael Weinstein <raf...@google.com> wrote:

[Rafael Weinstein]

Igor started a thread about this here (although, I believe there has
been discussion elsewhere as well):

http://lists.w3.org/Archives/Public/www-dom/2013JulSep/0019.html

Keep in mind that the issue here is what happens when the promise gets
resolved -- not when it is created.

I imagine that most DOM APIs that are structured in terms of promises
will resolve as a result of external IO, meaning that the callback
*will* fire after some number of event loop turns. For these cases,
the difference between microtask and task will be undetectable. In
fact, for these cases, it's almost a bit silly for the resolving I/O
Task to simply schedule a later callback Task (having done no other
work).

The more interesting question is what happens to Promises that resolve
immediately (or within the same event/microtask where they were
created). I suspect there are two classes of these:

1) Something which *might* require external I/O, but in some cases may
not -- e.g. the previous result is still available from an in memory
cache. As Malte points out, in this case, it's tempting to fire the
callback on creation because the value is available, but this results
in non-deterministic results -- because it violates the assumption
that callback executes on a fresh callstack.

2) Something which *always* resolves immediately. These are probably
going to be more important for code readability (avoiding the n-level
nested callback), than for actually deferred resolution. In cases
where the code author knows that all the promises will resolve without
external I/O, it's important to allow all the promises to resolve
before returning to the event loop (e.g. paint, etc...) (this is
Igor's point).

You'll notice that both of the later two cases, the thing that's being
deferred is something that up till now would have been done
synchronously.

Obviously, there's nothing stopping bad code from triggering the
poorly behaved script dialog, but I feel like the question here is
simply one of code structuring.

[Rafael Weinstein]

On Mon, Jul 29, 2013 at 9:52 AM, Ben Vanik <benv...@google.com> wrote:
> As someone who is using setImmediate in several places I want to emphasize
> what Maite said:
> "I think a lot of the confusion comes from the impression that people who
> want a fast setImmediate-like scheduling mechanism are interested in drawing
> things."
>
> Darin's points about layout/etc make sense, but I've rarely used it in that
> case. It's always for deferred execution from promises/event dispatches/etc.
> Nested timeouts like Maite mentions are an antipattern, sure, but the code
> is never written like that and Maite was just giving an example of what the
> expanded form looks like. When you're dealing with hundreds of thousands of
> lines of javascript from dozens to hundreds of different delay-loaded
> modules it may not be possible to 'just use a single setTImeout'.
>
> Rafael:
> That's awesome and will be great when it's implemented but not all code can
> use Promises in the near to medium term (legacy code/frameworks, need for
> compatibility with non-Promise-supporting browsers, etc). The reason people
> (such as myself) are asking for setImmediate is so that we can implement
> something like Promises ourselves in a way that is easy to patch into
> existing codebases.

DOM Mutation Observers are already deployed widely (Blink/WebKit/Gecko
and IE11) and will give you higher-fidelity semantics for implementing
a microtask-resolved Promise polyfil than setImmediate and I'll be
*shocked* if it's any slower than using setImmediate (creating an
observing non-connected nodes is very cheap).

[Darin Fisher]

On Mon, Jul 29, 2013 at 12:56 PM, Rafael Weinstein <raf...@google.com> wrote:

Igor started a thread about this here (although, I believe there has
been discussion elsewhere as well):

http://lists.w3.org/Archives/Public/www-dom/2013JulSep/0019.html

Keep in mind that the issue here is what happens when the promise gets
resolved -- not when it is created.

Ah, that's the bit I was missing.  Thanks!  SGTM.

-Darin

[nich...@nczconsulting.com]

As the author of one of the posts Tony mentioned (http://www.nczonline.net/blog/2013/07/09/the-case-for-setimmediate/), I'm incredibly frustrated by the conversations around setImmediate(). I've outlined in that post at least two uses cases for it (async callbacks and chunked data processing without freezing the UI), yet for some reason the conversation always comes back to animation. Not every use of setTimeout is related to animations. The remaining cases are the ones that setImmediate solves. I don't want to recount everything I said in the post here, so I hope people will go read it. :)

Both of the use cases I've mentioned I've been using setTimeout(0) to achieve for a long time. I understand how timers work (and did a presentation on them at Velocity 2 years ago), and yet this still the way I'm forced to do things unless I want to use the hacked up postMessage shim.

Given that there are use cases for this behavior, a spec defining how it works, an implementation ins browser, an implementation in Node, and a demand for that implementation, I don't understand the resistance. By all accounts it doesn't seem like a tough implementation (at a basic level, you could include the postMessage polyfill natively).

What other data or insights are needed to make the case for setImmediate?

[Elliott Sprehn]

Note that Google Feedback's rendering engine does exactly as jamesr describes, I've even presented on it:

http://elliottsprehn.com/preso/fluentconf/#/slides/21

We define a work function as function -> function? and then a runloop that says:

while (not out of time):

  fn = fn()

if (fn):

  schedule async completion of loop. // setTimeout(0)

Google Feedback is very aggressively async using this for everything from looping through arrays, walking the DOM, cloning nodes, and all kinds of other things.

This has worked, but it does seem unfortunate that if we schedule 5 async completions that we're suddenly clamped to 4ms. I suppose we could work around it with a postMessage hack or with MutationObserver and then doing the setTimeout inside the microtask checkpoint so the nesting level was zero... but that doesn't seem right.

darin + jamesr: What are you expected to do for async work sharding once you hit 5 shards of work?

This appears to be the issue everyone is discussing here. You always end up with nested setTimeout calls since each async completion is a nested call.

[Elliott Sprehn]

On Mon, Jul 29, 2013 at 9:00 PM, Elliott Sprehn <esp...@chromium.org> wrote:

Note that Google Feedback's rendering engine does exactly as jamesr describes, I've even presented on it:

http://elliottsprehn.com/preso/fluentconf/#/slides/21

We define a work function as function -> function? and then a runloop that says:

while (not out of time):

  fn = fn()

if (fn):

  schedule async completion of loop. // setTimeout(0)

...

darin + jamesr: What are you expected to do for async work sharding once you hit 5 shards of work?

This appears to be the issue everyone is discussing here. You always end up with nested setTimeout calls since each async completion is a nested call.

Thinking about this more I realized what jamesr is saying! Schedule the async completion _first_ so that when time is exhausted your timer is ready to fire even though you got clamped to 4ms.

I think what's lacking here is a primitive that makes this easier. Google Feedback has AsyncExecutor that deals with functions that return more functions, perhaps we need something similar that does "execute as much work as possible and then yield" semantics and that handles the correct setTimeout()/clearTimeout() incantation.

- E

[Nat Duca]

If I might suggest something here in hope of breaking the deadlock a wee bit....

Lets suppose for just a moment that we want something better than setTimeout(, 0). BUT, for that conversation to move ahead, any proposed solution must meet the following criteria:

1. misuse of it cannot burn battery for background tabs

2. misuse of it wont add more jank to the main thread

As phrased, you can too easily make a setImmediate loop that fails on both 1 and 2. So, maybe rather than us spinning about programming idioms, we could figure out a primitive that at least meets those requirements. With such a thing in hand, I'm virtually certain the conversation would be more productive.

To whet the apetite:

Option A) setImmediate throttles to 4ms if the tab is backgrounded

Option B) setImmediate goes to a 4ms throtttle if you exceed a certain amount of unit work per frame (and we give you that unit work via, say, a new argument rAF)

Option C) instead of setImmediate, setIdle(callback) where it is defined to never tick in the background. Ever. If you want a setIdle to fire, listen to page visiblity going invisible and fire the queued idle callbacks yourself.

Any other options?

[Aaron Boodman]

How is setImmediate() different from a busy loop from a power perspective? Web developers already have myriad options to burn power, and it seems like the user agent should already feel free to lower the priority of background tabs if it so desires (I thought maybe we already did this in Chrome, but not sure).

- a

[Darin Fisher]

"lower the priority of background tabs" -> "throttle execution of background tabs"

Low priority tasks can still burn lots of CPU ;-)

The most compelling argument I have heard for setImmediate is in large codebase situations where a module of non-UI code wants to yield but doesn't necessarily know how it was reached or know about any top-level, application-defined run loop that it can integrate with.  (I haven't decided if this tips me over the edge.)

Also, the postMessage polyfill from NobleJS [*] seems pretty yucky.  Perhaps a polyfill built from a MutationObserver + setTimeout(0) would be slightly less so.

-Darin

[*] https://github.com/NobleJS/setImmediate/blob/master/setImmediate.js

[Nat Duca]

I think our solutions for misbehaving tabs these days are high-level: e.g. if you let your setTimeout-called function run for more than 30s, we'll show the hang dialog. On Android, we silently kill tabs when they go into the background and reload them automatically when restored. We do the same on CrOS under memory pressure. One could imagine doing the same for CPU hogs. These are all primitive solutions, admittedly, but thats what we do to deal with badly behaving tabs.

As pointed out by Darin, getting help from the kernel here is hard and imperfect. And there are at least 5 OS' in various versions to think through once we do latch on to any kernel-assisted technique.

The web of today is still, I think, cooperatively threaded. Even in a multi-process browser, we still rely on each tab behaving well for the responsiveness of the other apps.

A perfectly valid route forward is for us to say "Hey now, thats just wrong. WebApps should get preemptive scheduling so that no one page can trash the other." If thats where we should head, I'm fascinated! Maybe that's the right call... but surely, its gonna take us a while to get there.

While we're waiting for such a robust solution to come online on all 5 platforms, how do we handle setImmediate? Do we hope that it never misbehaves? Do we info-bar if it does? At least some contingency planning here is definitely worthwhile.

[nich...@nczconsulting.com]

Why is the possibility of misbehavior such a concern here? I see no greater risk with setImmediate() than I do with setTimeout() or requestAnimationFrame(). There's also nothing preventing postMessage()-backed pegging when used too frequently. From what I recall, the fact that requestAnimationFrame() must be nested was specifically to prevent the very situations that people are describing in this thread. If requestAnimationFrame() is now deemed "safe enough", it doesn't seem too far from what setImmediate() is doing already.

-N

[Adam Barth]

On Tue, Jul 30, 2013 at 9:49 AM, <nich...@nczconsulting.com> wrote:

Why is the possibility of misbehavior such a concern here?

The underlying reason is that users have a choice in browsers.  If we implement setImmediate and web sites often misuse the API to burn battery, users will choose other browsers that don't implement the API because they value their battery life.

I see no greater risk with setImmediate() than I do with setTimeout() or requestAnimationFrame().

That's true, but both setTimeout and requestAnimationFrame have mitigations that help web sites avoid burning battery.  If we implemented setImmediate and it becomes popular, we'd likely need to add a similar mitigation to battery drain, which means we won't have gained anything over setTimeout(.., 0).

 

There's also nothing preventing postMessage()-backed pegging when used too frequently.

That's true.  If that becomes a problem, we'll likely need to throttle postMessage as well.

 

From what I recall, the fact that requestAnimationFrame() must be nested was specifically to prevent the very situations that people are describing in this thread. If requestAnimationFrame() is now deemed "safe enough", it doesn't seem too far from what setImmediate() is doing already.

requestAnimationFrame is throttled to the frame rate already.

Adam

[Aaron Boodman]

On Mon, Jul 29, 2013 at 10:38 PM, Nat Duca <nd...@chromium.org> wrote:

As pointed out by Darin, getting help from the kernel here is hard and imperfect. And there are at least 5 OS' in various versions to think through once we do latch on to any kernel-assisted technique.

The web of today is still, I think, cooperatively threaded. Even in a multi-process browser, we still rely on each tab behaving well for the responsiveness of the other apps.

Can you expand on this?  On a multiprocess browser, I was under the impression that we had preemptive scheduling today between renderers. If I put one of my tabs on Chrome/Linux in a busy loop right now, I don't even notice it. Everything still works fine. Is this a problem only on some platforms? Or are you talking about starvation of some other resource besides CPU? Or are you talking about the (admittedly common) case where different sites/tabs cohabitate in a renderer? I think being specific about your concerns will help everyone understand the tradeoffs.

While we're waiting for such a robust solution to come online on all 5 platforms, how do we handle setImmediate? Do we hope that it never misbehaves? Do we info-bar if it does? At least some contingency planning here is definitely worthwhile.

We can't just hope things don't misbehave on the web. But I wonder why our existing mitigation options from similar features (slow script dialog, killing the tab, throttling execution where available) wouldn't work equally well here.

On Mon, Jul 29, 2013 at 10:35 PM, Darin Fisher <da...@chromium.org> wrote:

The most compelling argument I have heard for setImmediate is in large codebase situations where a module of non-UI code wants to yield but doesn't necessarily know how it was reached or know about any top-level, application-defined run loop that it can integrate with.  (I haven't decided if this tips me over the edge.)

Are you proposing that applications implement their own message loop and then post to that? How would that be different/better than posting to the native message loop?

On Tue, Jul 30, 2013 at 9:53 AM, Adam Barth <aba...@chromium.org> wrote:

That's true, but both setTimeout and requestAnimationFrame have mitigations that help web sites avoid burning battery.  If we implemented setImmediate and it becomes popular, we'd likely need to add a similar mitigation to battery drain, which means we won't have gained anything over setTimeout(.., 0).

I think that part of the issue with setTimeout(, 0) was not just the potential for abuse... it was that there was a lot of existing code that has assumptions that the delay to setTimeout was clamped. Suddenly honoring 0 would result in animations running too fast and lots of previously-OK pages consuming tons of CPU. A new API with different semantics would not have this problem.

- a

[Nicholas Zakas]

Given the approach of "if that becomes a problem, we'll mitigate it", isn't that an argument for implementing setImmediate() given that its characteristics are similar to postMessage() in this regard?

And perhaps I'm over-simplifying, but wouldn't you rather have setImmediate() as a signal of what the developer intends so that you can appropriately optimize the behavior and mitigate performance risk as opposed to people continuing to use postMessage()-based polyfills that blur the meaning of postMessage()? It would suck to throttle postMessage() because that's how it's being used.

-N

On Tue, Jul 30, 2013 at 9:53 AM, Adam Barth <aba...@chromium.org> wrote:

--

______________________________
Nicholas C. Zakas
@slicknet

Author, Professional JavaScript for Web Developers
Buy it at Amazon.com: http://www.amazon.com/Professional-JavaScript-Developers-Nicholas-Zakas/dp/1118026691/ref=sr_1_3

[Adam Barth]

On Tue, Jul 30, 2013 at 10:09 AM, Nicholas Zakas <nich...@nczconsulting.com> wrote:

Given the approach of "if that becomes a problem, we'll mitigate it", isn't that an argument for implementing setImmediate() given that its characteristics are similar to postMessage() in this regard?

It's important to weigh the benefits of introducing an API against the risks of abuse.  We talked about that when we added postMessage, but we decided that the benefits (non-hacky cross-origin communication) where worth the risks.  In the case of setImmediate, the benefit side of the equation is much weaker and the risk of abuse seem quite large.

I agree with Nat that we should think about other ways of addressing these use cases that don't have the same risk of abuse.  For example, one use case for setImmediate is to polyfill Promises.  At this point, it seems likely that we'll ship Promises before setImmediate.  If we're patient, perhaps that use case will be better addressed by Promises themselves.  If Promises are difficult to use in existing code bases, that's something we should improve about Promises.

And perhaps I'm over-simplifying, but wouldn't you rather have setImmediate() as a signal of what the developer intends so that you can appropriately optimize the behavior and mitigate performance risk as opposed to people continuing to use postMessage()-based polyfills that blur the meaning of postMessage()? It would suck to throttle postMessage() because that's how it's being used.

That's a false dichotomy.  I'd rather browsers supported APIs that made developers and users happy over the long term.  Rather than shipping setImmediate "because it's there," we should aim higher and design an API that works well for developers and mitigates the risk of burning battery.

Adam

[Nicholas Zakas]

Agreed on weighing risks, but given that this API has been out in the wild for over a year already (the spec is two years old!), is there not enough evidence to determine actual risk vs. hypothetical risk (which seems to be the only risk discussed in this thread)? People are already using a polyfill, so even for browsers that don't support setImmediate() natively, isn't there data there as well?

Saying that we could potentially one day maybe use promises for this same thing doesn't seem very compelling. In that case we'd once again be stuck with implementing setImmediate() as a polyfill on top of something else designed for a slightly different use. It's effectively no better than the postMessage() hack of today. 

[mic...@sencha.com]

Tony, Adam et al. FWIW, after talking internally, for Sencha, RAF covers all our user cases so we see no need for setImmediate(); and agree with James Robinson's line of argument pretty much 100%. 

 

[shog...@gmail.com]

That's a nice write-up James. There's one thing I'd like to understand better:

You say that using setImmediate() to schedule tasks - that is, atoms of JS work - is actually "a deeply unfortunate way to structure the code since it means jumping in and out of the JS VM for every atom of work.  Exiting and entering the VM is far more expensive than running small amounts of script in modern JS engines.  "

Does this jumping in and out the the JS VM also occur for DOM calls?

Would this jumping in and out of the JS VM occur for Promise scheduling of callbacks?
If not, then what would be different? And why couldn't setImmediate() (or an alternative) be done the same way?

regards,
Sean

[jer...@duckware.com]

James, thanks for that great explanation of how timers 'should' work in Chrome.  But Chrome has not worked the way you describe below for over five years!

Chrome since Dec 2008 has used a global nesting variable that is not reset properly, and is not 'worker' safe.  The end result is that a timer in Chrome will 'randomly' get clamped or not clamped.  Also, there is no setTimeout(...,0) as chrome will promote that to setTimeout(...,1) -- and then due to even more internal bugs, effectively become setTimeout(...,2)

The way timer clamping works [1] is every task has an associated timer nesting level.  If the task originates from a setTimeout() or setInterval() call, the nesting level is one greater than the nesting level of the task that invoked setTimeout() or the task of the most recent iteration of that setInterval(), otherwise it's zero.  The 4ms clamp only applies once the nesting level is 4 or higher.  Timers set within the context of an event handler, animation callback, or a timer that isn't deeply nested are not subject to the clamping.  ... The practical effect of this is that setTimeout(..., x) means exactly what you think it would even for x in [0, 4) so long as the nesting level isn't too high.

[a...@zenexity.com]

That seems about right; I've never seen a timer take less than 2ms, even in optimal conditions.

[Nat Duca]

+alex clarke who'se been poking at this.

[Alex Clarke]

There is a 1ms clamp in the DOMTimer constructor, and I've investigated removing that.  At best I have seen 0.1ms callbacks but some caveats apply:

1. The first time the callback fires is going to be slow (likely 2+ms) since v8 needs to compile the function

2. If the nesting level reaches 5 then the timer gets bumped up to 4ms

3. You're at the mercy of whatever else is in the chromium runloop

4. My timings where on a very high end linux pc. 

5. It did not help this benchmark:  http://jsperf.com/setimmediate-test/17  (I don't know why)

Unfortunately I think trying to make setTimeout 0 to really be 0 will cause a lot of problems with legacy code.  It would be safer (and likely more performant) to implement setImmediate but I understand that is somewhat controversial.

[Nat Duca]

Much of the concerns you list are well-known.

To me, the main new discovery and critical one, is that fixing crbug.com/402694 to make setTimeout(,0) actually be a 0-delay when it can is going to be hard due to compatibility, because it changes the ordering of tasks that are posted.

Because of these compatibility concerns, this makes me wonder if we should implement setImmediate but with throttling. Thoughts?

[Chris Harrelson]

On Tue, Jan 6, 2015 at 9:48 AM, Nat Duca <nd...@chromium.org> wrote:

Much of the concerns you list are well-known.

To me, the main new discovery and critical one, is that fixing crbug.com/402694 to make setTimeout(,0) actually be a 0-delay when it can is going to be hard due to compatibility, because it changes the ordering of tasks that are posted.

Because of these compatibility concerns, this makes me wonder if we should implement setImmediate but with throttling. Thoughts?

+1. I think this is the best approach.

 

On Tue, Jan 6, 2015 at 2:38 AM, Alex Clarke <alexc...@google.com> wrote:

There is a 1ms clamp in the DOMTimer constructor, and I've investigated removing that.  At best I have seen 0.1ms callbacks but some caveats apply:

1. The first time the callback fires is going to be slow (likely 2+ms) since v8 needs to compile the function

2. If the nesting level reaches 5 then the timer gets bumped up to 4ms

3. You're at the mercy of whatever else is in the chromium runloop

4. My timings where on a very high end linux pc. 

5. It did not help this benchmark:  http://jsperf.com/setimmediate-test/17  (I don't know why)

Unfortunately I think trying to make setTimeout 0 to really be 0 will cause a lot of problems with legacy code.  It would be safer (and likely more performant) to implement setImmediate but I understand that is somewhat controversial.

On Mon, Jan 5, 2015 at 6:41 PM, Nat Duca <nd...@chromium.org> wrote:

+alex clarke who'se been poking at this.

On Sun, Jan 4, 2015 at 3:05 PM, <a...@zenexity.com> wrote:

That seems about right; I've never seen a timer take less than 2ms, even in optimal conditions.

On Thursday, January 9, 2014 2:19:58 PM UTC+1, jer...@duckware.com wrote:

James, thanks for that great explanation of how timers 'should' work in Chrome.  But Chrome has not worked the way you describe below for over five years!

Chrome since Dec 2008 has used a global nesting variable that is not reset properly, and is not 'worker' safe.  The end result is that a timer in Chrome will 'randomly' get clamped or not clamped.  Also, there is no setTimeout(...,0) as chrome will promote that to setTimeout(...,1) -- and then due to even more internal bugs, effectively become setTimeout(...,2)

The way timer clamping works [1] is every task has an associated timer nesting level.  If the task originates from a setTimeout() or setInterval() call, the nesting level is one greater than the nesting level of the task that invoked setTimeout() or the task of the most recent iteration of that setInterval(), otherwise it's zero.  The 4ms clamp only applies once the nesting level is 4 or higher.  Timers set within the context of an event handler, animation callback, or a timer that isn't deeply nested are not subject to the clamping.  ... The practical effect of this is that setTimeout(..., x) means exactly what you think it would even for x in [0, 4) so long as the nesting level isn't too high.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Michael Davidson]

FWIW, that will lead to UA sniffing. IE has setImmediate, and doesn't appear to throttle. We use setImmediate there, but postMessage in Chrome, and will need to continue to do so.

Michael

[Chris Harrelson]

The throttling would be there for case when Chrome really needs to spend its time elsewhere, e.g. to avoid a tab getting out of control, or to maintain a responsive browser process.

I don't think that is incompatible with your goals.

Chris

[Michael Davidson]

Ah, sorry I missed that. +1 from Google Maps!

[Elliott Sprehn]

Do we have any other regressions except the layout tests and chrome://help? Any page that expected a style recalc after the setTImeout was already racing, and probably got way worse when we switched to only do them at vsync a few years ago, and even worse when we did things like not pump frames in background tabs.

I'd suggest just fixing setTimeout and seeing how bad it really is. There might be some broken pages, but they were already likely broken.

[Alex Clarke]

I think there's something to be said for fixing setTimeout (or implementing setImmediate) for use in event handlers, but for uses cases like maps I wonder if we'd get better results by trying something else.   The Blink Scheduler has the concept of idle time, which occurs after the compositor commit and ends right before input is delivered for the next frame. When an idle task is executed, it's given a deadline. If idle tasks where exposed in js it should be possible to implement a js task scheduler which hands execution back to the browser when it needs it.

[Elliott Sprehn]

On Tue, Jan 6, 2015 at 10:24 AM, 'Alex Clarke' via blink-dev <blin...@chromium.org> wrote:

I think there's something to be said for fixing setTimeout (or implementing setImmediate) for use in event handlers, but for uses cases like maps I wonder if we'd get better results by trying something else.   The Blink Scheduler has the concept of idle time, which occurs after the compositor commit and ends right before input is delivered for the next frame. When an idle task is executed, it's given a deadline. If idle tasks where exposed in js it should be possible to implement a js task scheduler which hands execution back to the browser when it needs it.

Google Feedback has a scheduler it uses to control when the work for the page copying and screenshot rendering is done. Neither of this is work you can do in a worker, so you have to break it up. It models having to build a really big DOM async.

http://www.elliottsprehn.com/preso/fluentconf/#/slides/20 (slides 20-26)

It worked pretty well, it uses a small time slice amount while animations run or the user is typing. Then when you click a button it increases the time slice amount to "finish fast". That does mean the app appears to lock up, but it also means you get the result a lot sooner.

It takes a little bit to get used to the cooperative multi-tasking, but overall it's not too bad if you use a library.

[jer...@duckware.com]

Timers in Chrome have some serious issues.  There are multiple threads setting a single global variable (issue 331912), which breaks detection of timer nesting.  You always get a delay that is 1ms more than requested (issue 328700).  All complicated by the fact that internal time computations in Chrome are not accurate (issue 331502).

[staffan...@gmail.com]

Understandably this is late, but knowing from James writeup that setTimeout typically runs with 0 ms timeout unless called recursively, wouldn't this make an easier polyfill for your case

(function() {

if (window.setImmediate) return;

var isRunningPending = false;

var pending = [];

window.setImmediate = function setImmediate(fn) {

pending.push(fn);

if (isRunningPending)  return; // TODO: Support clearing timeout

setTimeout(onTimeoutImmediate, 0);

}

function onTimeoutImmediate() {

isRunningPending = true;

try {

while(pending.length > 0) {

pending.splice(0, 1)();

}

}

finally { 

isRunningPending = false;

if (pending.length > 0) {

// On errors, we resume on normal setTimeout (possibly clamped)

setTimeout(onTimeoutImmediate, 0);

}

}

}

})();

On Monday, July 29, 2013 at 8:17:49 AM UTC-7, Malte Ubl wrote:

I'm the author of the hacky-fill in the closure library and Paul Irish asked me to chime in as to our use cases. Reading James' post I think a lot of the confusion comes from the impression that people who want a fast setImmediate-like scheduling mechanism are interested in drawing things. There might be drawing as the ultimate side effect, but initially the relevant code is just application logic.

1. Guarantee async execution – i.e. in promises implementations. i.e. to ensure that adding a callback to a promise never triggers executing the callback before the current JS execution ends. This avoids very hard to find bugs.

Example:
var foo = 1;
promiseThingie.addCallback(function() { console.log(foo) });
foo++;

In the above code we never want foo to be output as having the value 1 and the average JS programmer doesn't expect that. There is a significant large chunk of code in the real world written like that and it leads to deeply nested timers (think 5 or 10 levels deep). At 4ms clamping per timeout it is no longer possible to maintain 60fps given sufficiently deeply nested timers.

Essentially the real world code ends up being equivalent to

onAnimationFrame = function() {
  setTimeout(function() {
   setTimeout(function() {
    setTimeout(function() {
     setTimeout(function() {
      setTimeout(function() {
       setTimeout(function() {
        setTimeout(function() {
          // Draw!
        }, 0);
       }, 0);
      }, 0);
     }, 0);
    }, 0);
   }, 0);
  }, 0);
};

2. Allow pushing stuff to a queue during JS execution and then handling the pushed stuff after execution ends. I.e. to avoid sending more than one HTTP request to an endpoint that would support answering everything that is currently needed in a single response. Another classic example is implementing a dirty flag and then running code once to check for that flag.

We see much larger than expected performance wins of using our hacky-fill for setImmediate over setTimeout(…, 0). The reason is that we often use it to avoid making too many HTTP requests, but calling setTimeout will queue us too far in the future. In a scenario where you compete over resources with other (possibly non-cooperating) resources on a page, this can have a very large impact in the upper percentiles (as opposed to the 4ms that one would expect).

Arguably for these use cases it would be better to use a solution that does not require yielding to the event loop, but unfortunately that cannot be hacky-filled outside of Chrome without controlling all stack entry points.

I'm somewhat troubled by the argument that the existence of a polyfill would be relevant for not implementing – I thought we wanted to make the web platform good. Ironically the hacky-fill for old IEs is very simple. The worst currently is Firefox as it requires creating an iframe and thus carries significant memory overhead: Imagine a page with 2 different banner providers and 3 social buttons: That makes 5 additional iframes per window, just because there is no setImmediate implementation in the browser.

[Michael Davidson]

Your onTimeoutImmediate calls setTimeout, making this recursive. If we have work to do, we will end up calling it recursively, leading to clamping. (We went through this with chrishtr@, who had the same thought, and agreed with our conclusion.)

Michael

[Elliott Sprehn]

If the work you're doing is batched in a queue like that, and you schedule the work with setTimeout() at the start of task instead of the end as jamesr@ suggested, it doesn't seem like you should observe the clamping.

[Michael Davidson]

Again, we want to have work in <4ms increments so that we remain responsive. 

If we do work in 4ms chunks, we risk blowing frames, right?

Michael

[Elliott Sprehn]

On Tue, Jan 27, 2015 at 2:35 PM, Michael Davidson <m...@google.com> wrote:

Again, we want to have work in <4ms increments so that we remain responsive. 

If we do work in 4ms chunks, we risk blowing frames, right?

You have 16ms for a frame, if the browser isn't giving you 4ms (25%) to run script per frame then the browser is too slow and should be fixed.

Also using setImmediate() to remain responsive seems like a mistake. You want some system where the browser decides how and when to run your callbacks and for how long. setImmedate() seems to be more about avoiding clamping than about allowing the browser to schedule you at idle times inside frames: there's nothing immediate about that, we might push your work out 100ms or more, for example by not running callbacks at all while a user's finger is down on the screen.

I would support a proposal for a setIdle(fn) like api.

[Michael Davidson]

On Tue, Jan 27, 2015 at 3:13 PM, Elliott Sprehn <esp...@chromium.org> wrote:

On Tue, Jan 27, 2015 at 2:35 PM, Michael Davidson <m...@google.com> wrote:

Again, we want to have work in <4ms increments so that we remain responsive. 

If we do work in 4ms chunks, we risk blowing frames, right?

You have 16ms for a frame, if the browser isn't giving you 4ms (25%) to run script per frame then the browser is too slow and should be fixed.

Sorry, we want to do as much work as we can (to improve time to scene resolution), while remaining a responsive framerate. We want to do more than 4ms if there's time to do it! We want to fill all of the idle time that we can, and right now the way we do that is do 1ms chunks in postMessage callbacks.

 

I would support a proposal for a setIdle(fn) like api.

Me too. Then we would stop having this discussion every few months. ;) 

Michael

[Nat Duca]

Lets move this to a bug...

Sami, Ross, wanna file a bug to the effect of "setImmediate woes: good scheduling is crazy hard on the web" so peeps can star and follow along?

[Sami Kyostila]

Sure, let's continue this here: https://code.google.com/p/chromium/issues/detail?id=452834

- Sami