Understanding async generator performance

[Conrad Buck]

Hello,
I'm working a library for transforming async iterables, and I'm trying to understand where the perf costs of async generators and for await of loops really come from, especially for async iterators whose contents are mostly available synchronously.

For example I might read a chunk of 65,535 characters from storage. Those characters are available synchronously, and then an await is needed to fetch the 65536th character. My observation is that for await loops introduce significant additional costs to every character, making it unwise to ever represent input as an async iterator of characters. I'm trying to understand where the overhead cost for sync steps comes from and what options may exist to lower it. To demonstrate what I'm talking about I've made a repo containing a variety of relevant benchmarks. I know the language spec is part of the picture, as it dictates that synchronously resolved awaits must participate in the microtask queue. But is this really the source of the cost? If so, is it reasonable to expect that that cost might be optimized away in the future?

Thanks in advance,
Conrad

[Caitlin Potter]

Because microtasks are always scheduled concurrently, on the same thread, the VM needs to complete all the remaining work in the current microtask, as well as evaluate subsequent ones in order.

So,in the context of a for-await-of (or otherwise, any loop involving Await), at each step through the loop, the current task is suspended and the remaining microtasks are performed, before continuing the loop.

With Await, even if data is available synchronously, evaluation is always suspended and resumed in a later microtask.

With async genenerator yield statements, you defer evaluation for each value yielded to the caller (who themselves will likely await the result, deferring evaluation again), and also defer evaluation whenever the generator is resumed.

Basically, async functions and iterables introduce a lot of concurrent tasks which require waiting for the entire task queue to finish, and become more apparently slow if there are long-running microtasks.

On Apr 16, 2022, at 1:21 PM, Conrad Buck <conar...@gmail.com> wrote:

Hello,

--
--
v8-dev mailing list
v8-...@googlegroups.com
http://groups.google.com/group/v8-dev
---
You received this message because you are subscribed to the Google Groups "v8-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to v8-dev+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/v8-dev/af3b87d1-33d3-4f97-9b6f-452b5929caa2n%40googlegroups.com.

[Conrad Buck]

That makes sense to me I think.

In certain circumstances though might it not be possible to detect that that work would be unnecessary? You'd have to know that there was no other code which would need to be executed first, which as I understand would mean that the microtask queue would have to be empty and you'd have to be suspended (or about to suspend) at an await. Is there any other code which would need to run? If not, could such a condition be detected and evaluation allowed to proceed synchronously as an optimization?

I'm actually trying to make a case for an alteration to the language, `for await? (const chr of input)` and associated changes to allow generators to return some step values (from next()) that are not wrapped in promises (when no async work was required to compute them). I know this would solve the problem (and I have benchmarks to back it up), but I've gotten pushback from delegates that there's no need to remove the extra asynchronicity because engines will just make the problem go away without the language needing to do anything at all.