Status of design document "Generalizing V8's heap interface"

[Erik Corry]

https://docs.google.com/document/d/1qVFbzenbzOIGsVC3PSP6fe_RQD0R4YwwQKcZYLyVtQ4/edit

This document from 2020 has no comments and no LGTMs, but it was referenced from commits. Were the proposals accepted in this approximate form, or is it a proposal that didn't get implemented in this form?

A different document, "Handles and Local Heaps" https://docs.google.com/document/d/17yKs-6apE2rGEag7tDsoyeRxg99c1dXyXQ2MfHe65tY/edit?tab=t.0 is from the same period and has more engagement.

I'm looking into making handles and scopes more efficient (by having linear backing areas with no limit checks), and the PersistentHandlesScope complicates this quite a lot, so I'm trying to understand the issues it solves.

-- 

Erik

[Leszek Swirski]

Hi Erik,

Both of those docs are a decent enough reflection of what we ended up doing. The purpose of PersistentHandles is to transfer ownership of a block of handles between threads -- you can think of it as a more efficient, single-owner block of global handles, which are expected to persist to the main thread (hence the name) after a background thread compilation completes. The PersistentHandlesScope is effectively a convenience scope to allow us to create Handles inside that scope using normal factory and Handle allocation methods, without needing to manually pass in which handle backing they need to go into.

I'd be interested to hear more about your plans to make HandleScopes more efficient though, they're still a performance pain point.

- Leszek

[Erik Corry]

I'm not sure I'll have time to work on faster handles, but essentially the idea is to use a large linear area as handle backing.  On 64 bit machines, virtual space is not very limited, so this makes more sense now than in the early days of V8. By using the right mmap flags you can make it auto-grow like a machine stack, so it only uses the high water memory.  There would be a guard page that segfaults for unbounded growth (see below).

This would simplify the code generated for both handle creation and HandleScope destruction by quite a bit.  As a thought experiment I have some handle-like code at https://godbolt.org/z/nWK7346TP  Three lines are commented out, so the initial state of the Godbolt example is what would happen if we had linear handle backing.  If you remove the comment punctuation on those three lines you get something more like what we have now, where a check of the limit is inlined, and there's an out-of-line call to extend the backing with a new block.  The generated code becomes much worse. (The example uses Local<long>, obviously in reality the handles would be Local<somepointertype>.)

The reason the linear code is so much nicer is that the compiler has full insight into what is going on. It can see that handle slots don't alias if they come from different HandleScope::CreateHandle calls. Creating a Handle becomes not much more than a pointer bump, and closing a scope is not much more than resetting a pointer.

There are some obvious issues with the idea and I have some ideas to fix them:

* I would make it so that the embedder can attach and detach handle scope backings.  This way you can only have one linear handle scope backing per thread or perhaps CPU.  Obviously this only makes sense back at the event loop where the area is empty anyway.  We can default to creating a linear area in the Locker if there is not already one. The Unlocker (which I think is not very heavily used) might need some modification too.
* Memory used is the high water mark of handle creation, which could be wasteful.  There are probably places (again, the event loop) where we can release some of the higher pages with an madvise call.
* Some patterns use arbitrary numbers of handles:
   * One of these is having a loop that doesn't have a handle scope at the top of it.  In this case if HandleScopes were much cheaper it would make sense to put a handle scope at the top of the loop. Hitting the guard page should show us where this needs fixing.
  * Another is collections that use handles as elements.  There are not too many cases of this in the code base.  They should be replaced with a single handle that holds a reference to a FixedArray.  If this is too slow (on-heap allocation, write barriers, etc.) then a new collection should be made where only the first N elements are put on-stack and use handle backing. Above a certain size they turn into a reference to an on-heap FixedArray.  This is a bit like https://chromium.googlesource.com/chromium/blink/+/refs/heads/main/Source/wtf/Vector.h which has a templated inline capacity and uses allocation for bigger arrays. You get the speed of on-stack operations for small vectors, and the flexibility of arbitrarily sized on-heap vectors when needed. The off-heap handles need to be reserved at collection creation time so we can push onto the collection inside inner HandleScopes.
* LocalHeap needs its own linear area.  This is not a big issue for us as we mainly run V8 in single-threaded mode. For Chrome it means a linear area per thread.
* PersistentHandlesScope would have to be done in a different way.  Currently several of the uses of PersistentHandlesScope are immediately followed by ReopenAndCanonicalizeHandlesInNewScope. It seems like this function could move the backings to a new area that was not in the linear handle backing.  This one is probably the hardest to solve unless the number of handles in a PersistentHandlesScope can be bounded.

Those are my ideas.  Like I said, I don't have much time for this at the moment, so it's on a back-burner. Happy to take a meeting to flesh out the idea though.  And perhaps there's a huge issue I have overlooked.

--
--
v8-dev mailing list
v8-...@googlegroups.com
http://groups.google.com/group/v8-dev
---
You received this message because you are subscribed to the Google Groups "v8-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to v8-dev+un...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/v8-dev/05027c6a-59ae-4755-8272-3517e1ba195dn%40googlegroups.com.

[Leszek Swirski]

Interesting, I like this idea in broad strokes. Unbounded growth is of course a problem as you say, especially for things less obvious than loops (like the compilers running optimizations to a fixed point). I wonder if we could use trap handlers to grow the backing store, maybe copying it and leaving the old store around only as a root set for the GC (which it could clean up after returning to the event loop). Probably the PersistentHandlesScope backing could use a similar trick, swapping itself in for the address and limit of the per-thread handle backing; ReopenAndCanonicalizeHandlesInNewScope just reads some values off existing handles, and creates new handles for them in the current block (whatever it is), so there's not really any difficult magic there beyond normal handle allocation.

All this said, we are also looking into conservative stack scanning (https://crbug.com/41480448) which, if done process-wide, could eliminate the need for HandleScopes entirely (you might have seen references to DirectHandle around). So, it could be that any effort put into HandleScope performance improvements is made obsolete by that project; right now there's still some questions about the performance and false-positive rate, but it's still an ongoing investigation and I'm hopeful.

- Leszek

[Erik Corry]

On Fri, Aug 1, 2025 at 3:11 PM Leszek Swirski <les...@chromium.org> wrote:

Interesting, I like this idea in broad strokes. Unbounded growth is of course a problem as you say, especially for things less obvious than loops (like the compilers running optimizations to a fixed point). I wonder if we could use trap handlers to grow the backing store, maybe copying it and leaving the old store around only as a root set for the GC (which it could clean up after returning to the event loop). Probably the PersistentHandlesScope backing could use a similar trick, swapping itself in for the address and limit of the per-thread handle backing; ReopenAndCanonicalizeHandlesInNewScope just reads some values off existing handles, and creates new handles for them in the current block (whatever it is), so there's not really any difficult magic there beyond normal handle allocation.

I don't think trap handlers are viable - too hard to fix up state at arbitrary places in Clang-generated code.  And anyway trap handlers only save virtual address space, and my proposition is that this is not a particularly scarce resource in 64 bit Chrome.

 

All this said, we are also looking into conservative stack scanning (https://crbug.com/41480448) which, if done process-wide, could eliminate the need for HandleScopes entirely (you might have seen references to DirectHandle around). So, it could be that any effort put into HandleScope performance improvements is made obsolete by that project; right now there's still some questions about the performance and false-positive rate, but it's still an ongoing investigation and I'm hopeful.

If something like this proposal makes conservative GC less attractive by reducing the performance benefit of direct handles I would personally see that as a plus.

A separate, but perhaps related proposal:  If HandleScopeImplementer::GetSpareOrNewBlock() did a little alignment work, then the isolate could be stored at the start of the handle block and this would mean the isolate could be quickly accessed with a rounding operation + a deref from any Handle, even a Handle containing a Smi.  This would eliminate most calls of Isolate::Current(), but I'm not sure what the performance of thread-locals is these days.  Could be that it's not a win.  This would also work with the linear handle backings, but it would not work with DirectHandle.

 

To view this discussion visit https://groups.google.com/d/msgid/v8-dev/05d4ae9b-c6ac-4fbd-8579-2bdef0c8853fn%40googlegroups.com.