deciding when not to optimize

[seth.b...@microsoft.com]

Hi everyone,

On several occasions, website owners have asked me about seemingly unlimited growth over time in the "compiled code" section of a heap snapshot. They're concerned about what appears to be a memory leak in V8 which could affect their customers. They've also told me that writing automated tests to detect memory usage regressions is hard because the page never reaches a steady-state maximum memory usage when performing repeated actions. As far as I can tell, V8 is functioning as intended, and the memory growth is due to functions tiering up into more optimized states. After infinite time of interacting with the page, every used function would reach its maximally optimized state, and the growth would halt. However, my explanation that the memory usage is actually asymptotic toward a distant limit is not very reassuring, and I think that these website owners have a good point. To my knowledge, V8's tier-up heuristics are based on counting total usage per function, which is useful for quick startup and short-running benchmarks, but might not be optimal for long-lived web pages. If it took 20 minutes of occasional calls for a function to accumulate enough usage that it becomes optimized, then maybe that function should remain unoptimized to save memory.

Have any of you looked into this problem before? (And do you consider it a real problem?) I have some vague notions for how we might focus on recent usage rather than total usage when deciding whether to optimize a function, but I'd be keen to hear about any previous explorations or ideas.

Thanks,
Seth

[Jakob Kummerow]

Hi Seth,

I'll give this a shot since nobody else has replied so far...

We're aware that this is an aspect of V8's optimization behavior that isn't necessarily always ideal, depending on your perspective. One might argue that triggering optimization of a function after a certain amount of total usage of that function represents our guess that the function will likely see enough future usage that the CPU cost of optimizing it will likely be amortized eventually, and as such it's fair to optimize any function that accumulates enough total usage, no matter how slowly it reaches that point -- but there is also the memory angle, as you point out.

I think we would probably generally be fine with making changes that avoid (or further delay) optimization of functions that consume their budget extremely slowly, if you want to explore that problem space. I'm not aware of any concrete plans on our side to tackle this specific issue. That said, as you're probably aware we're in the process of rolling out Maglev (and improving it further) and refactoring Turbofan (mostly for better compilation speed), and both of those will likely shake up the existing rules and heuristics for optimization. In general terms, the existence of Maglev means that Turbofan optimization can happen later than it used to; OTOH Turbofan getting faster might counterbalance that to some extent. I personally don't know what the typical size ratio between Maglevved and Turbofanned code for the same function is; *if* there is a significant difference in either direction then that would obviously impact whether we want to make it common or uncommon that a Maglevved function is *not* Turbofanned later.

As a related data point, for Wasm functions unoptimized (Liftoff) code is typically (though not always) quite a bit bigger than optimized code, so the consideration that optimization has a memory cost does not apply to Wasm in the same way it applies to JavaScript (where bytecode is particularly space efficient).

Cheers,

Jakob

[seth.b...@microsoft.com]

Thanks for the thoughtful response, Jakob! You make a good point that the Maglev versus Turbofan code size is now a major factor. I think that the conversations I'm referring to occurred before Maglev was on by default, so the behavior may be notably different now. I'm not entirely sure when or if I'll get the time to dig into this topic further, but I just wanted to check that I wouldn't be working on an idea that somebody else had already tried and discarded.

Cheers,

Seth