P0194 & lambdas for garbage collection

[tim.s...@epicgames.com]

Will P0194 be extended to support lambdas, and specifically reflecting on the number and type of lambda captures?

Reflecting on lambda captures is critically important in the case of implementing a garbage collector on top of standard C++, without hardcoding knowledge of memory layout or other things.

Background

In standard C++, a general-purpose garbage collector can be implemented on top of smart pointers with reference counting.  Any allocation with a nonzero reference count is treated as a GC root.  To get from this starting point to real garbage collection, we can provide a mechanism for certain types (such as containers) which are themselves heap-allocated and reference counted, to release the reference counts of their contents once they're initialized.

This can be automated by replacing "new t(parms)" with "newref<t>(parms)", which allocations memory, calls a constructor, and ensures smart pointers release their reference counts immediately rather than in their destructor.  This approach breaks the reference-counting cycles for those heap-allocated types, while ensuring everything on the stack remains a GC root.

Using this approach, I have a neat concurrent, nonblocking garbage collector up and running on top of standard C++17.  Without static reflection, this requires manually implementing reference-count-releasing functions for essential types.

With C++2a, reflection could make it completely automatic.  So, instead of using raw pointers and new, you use a smart pointer and newref, and get free, safe GC within standard C++.  For this to work well, we'd need to reflect lambda captures.  If we can't reflect lambda captures, then we are nearly certain that they'll be held forever due to reference-counting cycles, because a lambda's purpose is often to manipulate an object it's stored in.  Thus the lambda pins the object, and the object contains the lambda, so it's never released.  Eager functional languages like ML require garbage collection solely because of these cyclic references between containers and lambdas within them.

Aside: Garbage Collection in Future C++

I believe C++ will fundamentally require concurrent GC in order to scale to many-threaded programs with complex data dependencies and asynchronous execution.  Objects, lambdas, and futures interact in so many subtle ways that manual memory management seems intractable. (Unreal Engine has relied on a hand-coded C++ garbage collector since 1998.)

I feel the N2670 garbage collection track is neither tenable nor desirable.  C++ isn't about heavyweight runtime plumbing; it's about giving the programmer control, and relying on standard and user libraries to solve common problems.  Give us a thorough version of P0194 and we'll have garbage collection soon enough.  And it won't be a conservative kluge that stops all threads and scans all stacks and memory looking for pointer-like things; it will be a standard C++ implementation which users can opt-in to, while remaining safe and composable with all other libraries, whether they use GC or not.

[Axel Naumann]

Hi Tim,

On 18.12.17 23:08, tim.s...@epicgames.com wrote:

Will P0194 be extended to support lambdas, and specifically reflecting on the number and type of lambda captures?

Reflecting on lambda captures is critically important in the case of implementing a garbage collector on top of standard C++, without hardcoding knowledge of memory layout or other things.

Wow, super-interesting. I'd love to see a CppCon talk on that, to better understand that idea.

And while P0194 excludes lambdas, its follow-up paper P0670 includes them and will appear in LEWG + EWG for the next C++ standards meeting, i.e. it's on track to catch up with P0194.

Could you check whether it does what you need?

Cheers, Axel.

--
You received this message because you are subscribed to the Google Groups "SG 7 - Reflection" group.
To unsubscribe from this group and stop receiving emails from it, send an email to reflection+...@isocpp.org.
To post to this group, send email to refle...@isocpp.org.
Visit this group at https://groups.google.com/a/isocpp.org/group/reflection/.
For more options, visit https://groups.google.com/a/isocpp.org/d/optout.

[tim.s...@epicgames.com]

Thanks for P0670! Its coverage of static reflection on functions and lambdas in particular looks like exactly what's needed for exposing lambda captures to a garbage collector.

As an aside, I agree with the importance of exposing function parameter names.  These will all be important in doc tools, RPC frameworks, and marshalling from scripting languages built on C++ reflection.

[Axel Naumann]

Excellent, thanks, Tim!

Now - did I overlook your promise to submit an abstract on your GC to CppCon? ;-)

Cheers, Axel.

On 20.12.17 20:56, tim.s...@epicgames.com wrote:

Thanks for P0670! Its coverage of static reflection on functions and lambdas in particular looks like exactly what's needed for exposing lambda captures to a garbage collector.

As an aside, I agree with the importance of exposing function parameter names.  These will all be important in doc tools, RPC frameworks, and marshalling from scripting languages built on C++ reflection.