GC tracing changes in V8 7.2?

[Kenton Varda]

Hi v8-users,

Upon upgrading to V8 7.2, we're seeing GC collecting objects more aggressively than before. It looks like we have another bug with our GC integration, as we're occasionally seeing objects collected that should be reachable using EmbedderHeapTracer.

I'm trying to isolate the issue (so far I haven't been able to reproduce outside of prod). But, meanwhile, are there any big changes to GC tracing in 7.2 that might hint at what I'm getting wrong? The release blog post didn't seem to mention anything: https://v8.dev/blog/v8-release-72

-Kenton

[Michael Lippautz]

V8 started to call to EmbedderHeapTracer during incremental marking steps that happen on allocation. 

This could mean that there is C++ stack sitting on top of entering V8. Any chance that there are C++ objects holding traced V8 references that cannot be found by the EmbedderHeapTracer? E.g., this could happen when a C++ object is only reachable from stack without a proper stack scanning mechanism.

The CL landed here.

-Michael

--
--
v8-users mailing list
v8-u...@googlegroups.com
http://groups.google.com/group/v8-users
---
You received this message because you are subscribed to the Google Groups "v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to v8-users+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kenton Varda]

Thanks, Michael, this gives me a better idea of what to look for.

I'm a little surprised because I had always assumed that tracing was triggered during allocation. When else does it run?

Also, what counts as allocation? Does allocating a new persistent handle in itself count, or is it strictly allocating objects?

I've designed my code such that when a reference to an object exists on the C++ stack, I call ClearWeak() on the persistent handle. Once all C++ stack references go away, I call SetWeak() again. Could this possibly confuse the tracer if a trace is in-progress? Should I, for example, be calling RegisterExternalReference() after SetWeak()?

-Kenton

[Michael Lippautz]

On Tue, Jan 15, 2019 at 7:12 PM 'Kenton Varda' via v8-users <v8-u...@googlegroups.com> wrote:

Thanks, Michael, this gives me a better idea of what to look for.

I'm a little surprised because I had always assumed that tracing was triggered during allocation. When else does it run?

We used to run the embedder tracing only as part of incremental tasks. For Blink this guarantees V8 that there's no relevant C++ stack when invoking the EmbedderHeapTracer.

Since 7.2 we are also able to run it as part of allocations.

 

Also, what counts as allocation? Does allocating a new persistent handle in itself count, or is it strictly allocating objects?

Allocating JS objects on the managed heap. This does not include persistent handles themselves

 

I've designed my code such that when a reference to an object exists on the C++ stack, I call ClearWeak() on the persistent handle. Once all C++ stack references go away, I call SetWeak() again. Could this possibly confuse the tracer if a trace is in-progress? Should I, for example, be calling RegisterExternalReference() after SetWeak()?

- The ClearWeak part should work if there's a final GC pause interrupting whatever is done with C++ stack.

- The SetWeak after being done may or may not require a manual registering call, depending on how EmbedderHeapTracer is implemented.

In Blink we implemented regular traced garbage collection for the EmbedderHeapTracer. This means that the object containing such an interesting references may have been processed by the EmbedderHeapTracer already. If the reference is then just marked as weak with SetWeak() the GC misses out on it as it never sees the containing object again. Blink emits a RegisterExternalReference() call for such objects. The general concept for solving mutation in the graph while a garbage collector is running is called (write barrier).

You can check whether this is the problem by disabling incremental marking for wrappers (--no_incremental_marking_wrappers).

 

-Michael

[Kenton Varda]

I'm still working on this off-and-on. The issue is not as urgent as it sounds because in the case that a wrapper object is collected prematurely, we simply remake it as needed. Still, this could cause issues if scripts add JS properties on native objects and expect them to stay there, but the issues seems to be affecting only a handful of specific scripts on our platform and none of them happen to do that... Still, we do need to fix the issue for future scripts.

On Wed, Jan 16, 2019 at 2:23 AM Michael Lippautz <mlip...@chromium.org> wrote:

- The ClearWeak part should work if there's a final GC pause interrupting whatever is done with C++ stack.

Sorry, I'm a GC noob. What happens in the "final GC pause", exactly? What does it mean for it to "interrupt whatever is done with the C++ stack"?

Does ClearWeak() implicitly mark the object, if tracing is already in-progress? Or is that what happens in the final GC pause? Does that object get traced?

Looking at the code, it seems like ClearWeak() does not mark the object. So I guess I should probably RegisterExternalReference() after ClearWeak()? Normally a strong reference would be a root and so would be marked at the beginning of the trace cycle, but if ClearWeak() happens mid-cycle it seems like there's an issue.

However, this doesn't seem to fit the pattern of the problems I'm seeing in production.

 

- The SetWeak after being done may or may not require a manual registering call, depending on how EmbedderHeapTracer is implemented. 

In Blink we implemented regular traced garbage collection for the EmbedderHeapTracer. This means that the object containing such an interesting references may have been processed by the EmbedderHeapTracer already. If the reference is then just marked as weak with SetWeak() the GC misses out on it as it never sees the containing object again. Blink emits a RegisterExternalReference() call for such objects. The general concept for solving mutation in the graph while a garbage collector is running is called (write barrier).

Not sure I follow. RegisterExternalReference() is only meaningful on objects that are currently white, right? But before SetWeak(), the handle was strong, making it a root. Roots should have been marked gray at the start of the tracer cycle?

In my case, I think the objects are frequently newly-allocated at the time SetWeak() is called. But a newly-allocated object should be marked black at allocation, right?

Thanks again for the help! Hopefully we'll be able to open source this glue library so others can reuse this work...

-Kenton

[Michael Lippautz]

On Mon, Jan 28, 2019 at 11:12 PM 'Kenton Varda' via v8-users <v8-u...@googlegroups.com> wrote:

I'm still working on this off-and-on. The issue is not as urgent as it sounds because in the case that a wrapper object is collected prematurely, we simply remake it as needed. Still, this could cause issues if scripts add JS properties on native objects and expect them to stay there, but the issues seems to be affecting only a handful of specific scripts on our platform and none of them happen to do that... Still, we do need to fix the issue for future scripts.

We are currently reworking APIs around the use of EmbedderHeapTracer and how it is used with V8 handles. In 7.4 we introduced a new type (TracedGlobal) that is tied to the use case of tracing through the embedder heap. This type is treated as root for scavenges unless explicitly opted in into treating it as non-root.

We advise against switching handles from strong to weak and vice versa as it is hard to follow what's going on exactly.

 

On Wed, Jan 16, 2019 at 2:23 AM Michael Lippautz <mlip...@chromium.org> wrote:

- The ClearWeak part should work if there's a final GC pause interrupting whatever is done with C++ stack.

Sorry, I'm a GC noob. What happens in the "final GC pause", exactly? What does it mean for it to "interrupt whatever is done with the C++ stack"?

Final pause is when the garbage collector finalizes the current cycle. The stack is re-scanned if needed. Handles are also re-scanned. So, if some handles is made strong by called ClearWeak() it will be considered as strong root during this phase.

"interrupt whatever is done in C++" was referring to the embedder situation where there's the V8 garbage collection is usually triggered when there's a native C++ stack. All of V8's objects are held through Local or some sort of Persistent handle when this happens.

 

Does ClearWeak() implicitly mark the object, if tracing is already in-progress? Or is that what happens in the final GC pause? Does that object get traced?

No to all of it. As mentioned above, it will be discovered as strong root during root scanning.

 

Looking at the code, it seems like ClearWeak() does not mark the object. So I guess I should probably RegisterExternalReference() after ClearWeak()? Normally a strong reference would be a root and so would be marked at the beginning of the trace cycle, but if ClearWeak() happens mid-cycle it seems like there's an issue.

ClearWeak makes the handle strong. It should be discovered during root scanning.

 

However, this doesn't seem to fit the pattern of the problems I'm seeing in production.

 

- The SetWeak after being done may or may not require a manual registering call, depending on how EmbedderHeapTracer is implemented. 

In Blink we implemented regular traced garbage collection for the EmbedderHeapTracer. This means that the object containing such an interesting references may have been processed by the EmbedderHeapTracer already. If the reference is then just marked as weak with SetWeak() the GC misses out on it as it never sees the containing object again. Blink emits a RegisterExternalReference() call for such objects. The general concept for solving mutation in the graph while a garbage collector is running is called (write barrier).

Not sure I follow. RegisterExternalReference() is only meaningful on objects that are currently white, right? But before SetWeak(), the handle was strong, making it a root. Roots should have been marked gray at the start of the tracer cycle?

This is an implementation detail, but yeah, V8 marks the full root set, including the handles, at the beginning of incremental marking. If the handle was strong at this point the object is transitioning through the marking phase. V8 also re-scans roots before finalizing the current cycle.

The embedder should not reason about object colors as they are an implementation detail.

 

In my case, I think the objects are frequently newly-allocated at the time SetWeak() is called. But a newly-allocated object should be marked black at allocation, right?

Depends and is an implementation detail. It is not safe to assume that new objects are allocated black during GC. (It does not hold for new space for various optimization reasons.)

So, if you want to use tracing and create a new reference of that sort using SetWeak() you also need to register the object using RegisterExternalReference. (In future, just use TracedGlobal and pass it to EmbedderHeapTracer::RegisterEmbedderReference.)

 

Thanks again for the help! Hopefully we'll be able to open source this glue library so others can reuse this work...

https://chromium-review.googlesource.com/c/v8/v8/+/1425523 could be interesting then. The idea is to separate out the tracing use case so that there's no mix up between regular strong and weak handles anymore.

-Michael

[Kenton Varda]

Hi Michael,

Thanks, I think I now see the problem: I had assumed that newly-allocated objects were always marked immediately on allocation, since they are clearly reachable at that time. But, now that I think about it, I suppose that would make it hard to quickly collect short-lived objects.

I think realistically the current embedding API requires an understanding of implementation details to use correctly. Since there is no written guide, the only way I've been able to understand how the pieces fit together is by understanding implementation details. In particular I think the concept of write barriers and when to use them is not very intuitive to those who haven't studied garbage collection. Once I get this sorted out I may try to write a beginner's guide from that perspective.

------

The TracedGlobal API looks like a good addition to make embedders' lives easier, but it might turn out it doesn't quite fit with the model we've settled on in our code. Specifically, we want to allocate JS wrapper objects for a C++ object lazily, on the first time the object is directly referenced from JS. This makes it tricky to use V8 handles to link C++ objects together, because there may not be a JS object yet for the handle to point to. Instead, we reference the C++ object directly, and the C++ object holds a handle to its own JS wrapper which is allocated lazily. If each reference from another C++ object required its own TracedHandle, then we'd have to keep a reverse mapping of all the references pointing to an object so that we could initialize all of the TracedHandles when the wrapper is lazily allocated.

A further problem is that the parent object may itself not have any wrapper allocated, but may have references from C++ which we considered to be strong references. Consider e.g. the case of a Request object containing a Headers object. Imagine the Headers object has a JS wrapper but the Request object does not, and is only referenced from C++. In this case, the reference from Request -> Headers will never be traced, and so we need to treat the Headers wrapper as having a strong reference. Later on, a wrapper may be allocated for the Request. Once that happens, then we only need a strong reference on the Request wrapper itself, and we can rely on tracing to find the Headers wrapper.

So, the problem here is that over the lifetime of a reference between two C++ objects, the reference's nature can change between being C++-only (no V8 handle), being a strong V8 handle, and being a traced V8 handle. So it doesn't seem like we can simply drop in a TracedGlobal here, unfortunately.

That said, if you would prefer that we move towards using weak handles strictly for registering the destructor callback, and strictly use TracedGlobal for traced handles, then I think we can work with that by having a scheme where each object potentially holds three different handles to its own wrapper:

- A weak independent handle, to get the weak callback.

- A strong handle, when any non-traced references exist from C++, to keep the wrapper alive.

- A TracedGlobal, when any traced references exist from C++, used to implement tracing.

Let me know if you think it would be a good idea for me to implement that instead of relying on one handle and doing SetWeak()/ClearWeak().

------

Also, another technical question clarifying something you said: If an object is first discovered and marked during "final pause" (e.g. because it has a strong persistent handle at that point which was never seen before), it still gets traced, right? I guess that means that the "final pause" could end up arbitrarily long, if some deep object tree managed to slip by the tracer until then?

Thanks,

-Kenton

--

--
v8-users mailing list
v8-u...@googlegroups.com
http://groups.google.com/group/v8-users
---

You received this message because you are subscribed to a topic in the Google Groups "v8-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/v8-users/EeIPnAmNa4g/unsubscribe.
To unsubscribe from this group and all its topics, send an email to v8-users+u...@googlegroups.com.

[Kenton Varda]

One more low-level question (where it seems like implementation details matter to embedders):

Based on what you've said, an object may or may not be "black" on allocation. It seems, though, that I need to do different things depending on the color. If it is allocated black, then I need to arrange to trace the object myself to register all its outgoing references. If it is white, then I need to call RegisterExternalReference(), and V8 will trace it later. If it's gray, I don't need to do anything.

But if I don't know the color, it seems I need to *both* call RegisterExternalReference() and initiate my own trace to cover all bases. But in the case that it's not already-black, this will result in a redundant trace later on.

Is it best to live with this potential redundant trace, or is there some way I can detect the color after allocation?

-Kenton

[Michael Lippautz]

On Wed, Jan 30, 2019 at 8:35 PM 'Kenton Varda' via v8-users <v8-u...@googlegroups.com> wrote:

Hi Michael,

Thanks, I think I now see the problem: I had assumed that newly-allocated objects were always marked immediately on allocation, since they are clearly reachable at that time. But, now that I think about it, I suppose that would make it hard to quickly collect short-lived objects.

That's the reason why we allocate white in the young generation, yes.

 

I think realistically the current embedding API requires an understanding of implementation details to use correctly. Since there is no written guide, the only way I've been able to understand how the pieces fit together is by understanding implementation details. In particular I think the concept of write barriers and when to use them is not very intuitive to those who haven't studied garbage collection. Once I get this sorted out I may try to write a beginner's guide from that perspective.

Well, barriers are only needed for incremental (and concurrent) collection. From an embedder perspective all references reported by V8 can be traced after EmbedderHeapTracer::EnterFinalPause() is called which would mean that there's no incremental collection and barriers are unnecessary.

 

------

The TracedGlobal API looks like a good addition to make embedders' lives easier, but it might turn out it doesn't quite fit with the model we've settled on in our code. Specifically, we want to allocate JS wrapper objects for a C++ object lazily, on the first time the object is directly referenced from JS. This makes it tricky to use V8 handles to link C++ objects together, because there may not be a JS object yet for the handle to point to. Instead, we reference the C++ object directly, and the C++ object holds a handle to its own JS wrapper which is allocated lazily. If each reference from another C++ object required its own TracedHandle, then we'd have to keep a reverse mapping of all the references pointing to an object so that we could initialize all of the TracedHandles when the wrapper is lazily allocated.

I don't think I can follow :)

It looks like you may be doing something similar as Blink where the JS object is lazily created. Once the JS object is there we have edges in both directions. See ScriptWrappable. At the point where we create the JS wrapper we know the C++ object and can create the links in both directions.

 

A further problem is that the parent object may itself not have any wrapper allocated, but may have references from C++ which we considered to be strong references. Consider e.g. the case of a Request object containing a Headers object. Imagine the Headers object has a JS wrapper but the Request object does not, and is only referenced from C++. In this case, the reference from Request -> Headers will never be traced, and so we need to treat the Headers wrapper as having a strong reference. Later on, a wrapper may be allocated for the Request. Once that happens, then we only need a strong reference on the Request wrapper itself, and we can rely on tracing to find the Headers wrapper.

We suggest using actually tracing for C++ to solve this problem. This way objects are traced through until they reach entry points into V8. 

It's more engineering effort but simple to reason about. 

 

So, the problem here is that over the lifetime of a reference between two C++ objects, the reference's nature can change between being C++-only (no V8 handle), being a strong V8 handle, and being a traced V8 handle. So it doesn't seem like we can simply drop in a TracedGlobal here, unfortunately.

That said, if you would prefer that we move towards using weak handles strictly for registering the destructor callback, and strictly use TracedGlobal for traced handles, then I think we can work with that by having a scheme where each object potentially holds three different handles to its own wrapper:

- A weak independent handle, to get the weak callback.

The weak callback is only for the destructor, right? How do you differentiate between a C++ object without JS wrapper (lazy creation) and a C++ object that should be destructed? 

 

- A strong handle, when any non-traced references exist from C++, to keep the wrapper alive.

These should usually be temporary (ideally Local<T>) and looks fine to me. 

 

- A TracedGlobal, when any traced references exist from C++, used to implement tracing.

Let me know if you think it would be a good idea for me to implement that instead of relying on one handle and doing SetWeak()/ClearWeak().

------

Also, another technical question clarifying something you said: If an object is first discovered and marked during "final pause" (e.g. because it has a strong persistent handle at that point which was never seen before), it still gets traced, right? I guess that means that the "final pause" could end up arbitrarily long, if some deep object tree managed to slip by the tracer until then?

Yes, correct. Worst case is a huge subgraph that is held alive by one object that is only discovered during the atomic pause. Fortunately, this does not happen too often :)

-Michael

[Michael Lippautz]

On Wed, Jan 30, 2019 at 9:15 PM 'Kenton Varda' via v8-users <v8-u...@googlegroups.com> wrote:

One more low-level question (where it seems like implementation details matter to embedders):

Based on what you've said, an object may or may not be "black" on allocation. It seems, though, that I need to do different things depending on the color. If it is allocated black, then I need to arrange to trace the object myself to register all its outgoing references. If it is white, then I need to call RegisterExternalReference(), and V8 will trace it later. If it's gray, I don't need to do anything.

But if I don't know the color, it seems I need to *both* call RegisterExternalReference() and initiate my own trace to cover all bases. But in the case that it's not already-black, this will result in a redundant trace later on.

The redundant trace can be avoided by having a color marker on the C++ object itself so that it only traces through it once.

Ultimately, solving all those issues leads to having a fully trace-based system with marking colors on the C++ side too. We went that way because we need a system to collect reference cycles and the easiest way to do so is by basing the whole system on tracing.

 

Is it best to live with this potential redundant trace, or is there some way I can detect the color after allocation?

There's no way to detect the color as we have changed the internal implementation in the past and may do so in future.

-Michael 

[Kenton Varda]

We've fixed this issue. I believe the problem was due to not correctly handling wrapper objects allocated white after their parent had already been traced. (In the past I had had a bug that occurred due to objects allocated black, and I had been left with the impression that objects were always allocated black... oops.)

On Wed, Jan 30, 2019 at 12:49 PM Michael Lippautz <mlip...@chromium.org> wrote:

Well, barriers are only needed for incremental (and concurrent) collection. From an embedder perspective all references reported by V8 can be traced after EmbedderHeapTracer::EnterFinalPause() is called which would mean that there's no incremental collection and barriers are unnecessary.

That would feel like a cop-out though. :) I'd like to get the full power of incremental tracing.

We suggest using actually tracing for C++ to solve this problem. This way objects are traced through until they reach entry points into V8. 

It's more engineering effort but simple to reason about. 

Yes, we do actually trace through the C++ objects (otherwise we couldn't collect cycles). But we don't track our own "roots", but instead try to keep track of which JS wrappers need to be considered "roots" at any particular time, which when combined with lazy wrapper allocation creates some complexity. I think I've figured it out at this point, though.

The weak callback is only for the destructor, right? How do you differentiate between a C++ object without JS wrapper (lazy creation) and a C++ object that should be destructed?

Our C++ objects are refcounted. We count references from other C++ objects, and then when we create a JS wrapper, we increment the refcount once for that as well. So the weak callback decrements the refcount.

This has the nice property that a bug in our GC integration has the worst-case effect that the wrapper gets dropped and recreated (losing any properties that were added to it from JS code), rather than a use-after-free.

- A strong handle, when any non-traced references exist from C++, to keep the wrapper alive.

These should usually be temporary (ideally Local<T>) and looks fine to me. 

These references aren't necessarily held on the stack, though.

On Wed, Jan 30, 2019 at 12:55 PM Michael Lippautz <mlip...@chromium.org> wrote:

The redundant trace can be avoided by having a color marker on the C++ object itself so that it only traces through it once.

Ultimately, solving all those issues leads to having a fully trace-based system with marking colors on the C++ side too. We went that way because we need a system to collect reference cycles and the easiest way to do so is by basing the whole system on tracing.

I ended up having each of my C++ objects remember the last trace cycle in which they we reached -- so if they're reached a second time in the same cycle, I can return early. I guess this is effectively a two-color system (lastSeen == currentTrace => black, lastSeen < currentTrace => white). I don't need a "gray" for my C++ objects because I do the tracing depth-first (which works fine because my C++ object trees are all very small).

-Kenton