t hepudds (Gerrit)

unread,

Aug 31, 2025, 1:39:19 PMAug 31

to goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com

t hepudds has uploaded the change for review

Commit message

design/74299-runtime-free.md: add runtime.free and use via compiler and stdlib to reduce GC work

Updates golang/go#74299

Change-Id: I56738000cc473fa95d848f2d078c3d160c6ed8cd

Change diff

diff --git a/design/74299-runtime-free.md b/design/74299-runtime-free.md
new file mode 100644
index 0000000..36ce073
--- /dev/null
+++ b/design/74299-runtime-free.md
@@ -0,0 +1,292 @@
+# Directly freeing user memory to reduce GC work
+
+* **Discussion**: https://go.dev/issue/74299
+* **CLs**: stack visible via https://go.dev/cl/673695, plus some not yet mailed.
+
+## DOCUMENT STATUS
+
+* This is currently a **draft** design document.
+* Almost all the content here currently was originally in the
+[CL 673695](https://go.dev/cl/673695) commit message, but was split out to this document
+in the proposal repo to make it easier to manage the design doc separately from the CLs.
+* Some of the details in the writeup here are slightly stale compared to latest implementation,
+  but the essence here is still correct.
+* The most notable deltas from what's written here vs. the current implementation:
+  * `runtime.freeSized` (the name used below) is currently planned to instead be named `runtime.free`.
+    The name has changed a few times in the implementation.
+  * There are likely 1-2 more `runtime` entry points not described below, included likely a `runtime.freeSlice`
+    (which is not described below; in the current implementation, it is primarily used as a test function,
+	but currently leaning towards using it in some spots in the stdlib as convenience for callers.)
+
+## Background
+
+For reducing allocations, there is sometimes a chicken and egg problem.
+
+For example, #72036 is a WIP approach to update escape analysis so
+that interface arguments do not always escape, with a primary target of
+w.Write(b) and friends, but many interesting byte slices are
+variable size known only at runtime. Currently, a make with an unknown
+capacity (beyond 32 bytes) causes a slice to be heap allocated, so
+solely improving escape analysis in this case has limited impact
+on actual applications -- if a slice is heap allocated for two reasons,
+eliminating one reason just means it is still heap allocated for the
+other reason, which lessens the value of doing the work to eliminate the
+first reason.
+
+## Overall approach
+
+In an attempt to help triangulate on what is possible for the runtime,
+compiler, and standard library to do together to reduce how much work
+the GC must do, this CL has a proof-of-concept set of runtime.free
+implementations, currently with these signatures:
+
+```
+ func free(ptr unsafe.Pointer)
+ func freesized(ptr unsafe.Pointer, uintptr size, noscan bool)
+ func freetracked(trackedObjs *[]trackedObj)
+```
+
+Currently, the first two APIs (`free` and `freesized`) can be called by the
+standard library, with initial targets in the map grow/split code,
+`strings.Builder`, possibly places like `slices.Collect` and
+`slices.AppendSeq`, and other locations where the code knows it owns a
+pointer and has knowledge of when it is logically dead. The third API
+(`freetracked`) is currently intended to be inserted by the compiler.
+`freetracked` is paired with a new slice allocation function that together
+allow for heap-allocated objects to be tracked and later automatically
+freed as a scope is exited:
+
+```
+ func makeslicetracked64(et *_type, len64 int64, cap64 int64, trackedObjs *[]trackedObj) unsafe.Pointer
+```
+
+More details on these four functions at the bottom.
+
+Additional CLs update the compiler to automatically insert free calls for slices allocated via
+make when the compiler can prove it is safe to do so, and separately updating maps to free map
+storage during growth and split as well as a small number of lower-level stdlib things like strings.Builder and
+(which are "low level" in the sense that they already encapsulate some low-level performance tricks, including
+use of unsafe, but also they let other higher-level pieces of the stdlib then safely benefit from their
+performance). It is not intended that higher-level portions of the stdlib (like net/http) would have manual
+calls to a runtime.free.
+
+## Example target
+
+Using strings.Builder as an example target -- this is already a somewhat
+low-level part of the stdlib that already helps encapsulate some
+performance-related trickery, including using unsafe and
+bytealg.MakeNoZero. The strings.Builder already owns its in-progress
+buffer that is being built up, until it returns a usually final string.
+The existing internal comment says:
+
+  // External users should never get direct access to this buffer, since
+  // the slice at some point will be converted to a string using unsafe,
+  // also data between len(buf) and cap(buf) might be uninitialized.
+  buf []byte
+
+If we update strings.Builder to explicitly call runtime.freesized on
+that buf after a resize operation (but without freeing the usually final
+incarnation of buf that will be returned to the user as a string), we
+can see some nice benchmark results on the existing strings benchmarks
+that call Builder.Write N times and then call Builder.String.  Here, the
+(uncommon) case of a single Builder.Write is not helped (given it never
+resizes after first alloc if there only one Write), but the impact grows
+such that it is up to ~2x faster as there are more resize operations due
+to more strings.Builder.Write calls:
+
+```
+                                               │     disabled.out │      new-free-20.txt                │
+                                               │      sec/op      │   sec/op     vs base                │
+BuildString_Builder/1Write_36Bytes_NoGrow-4           55.82n ± 2%   55.86n ± 2%        ~ (p=0.794 n=20)
+BuildString_Builder/2Write_36Bytes_NoGrow-4           125.2n ± 2%   115.4n ± 1%   -7.86% (p=0.000 n=20)
+BuildString_Builder/3Write_36Bytes_NoGrow-4           224.0n ± 1%   188.2n ± 2%  -16.00% (p=0.000 n=20)
+BuildString_Builder/5Write_36Bytes_NoGrow-4           239.1n ± 9%   205.1n ± 1%  -14.20% (p=0.000 n=20)
+BuildString_Builder/8Write_36Bytes_NoGrow-4           422.8n ± 3%   325.4n ± 1%  -23.04% (p=0.000 n=20)
+BuildString_Builder/10Write_36Bytes_NoGrow-4          436.9n ± 2%   342.3n ± 1%  -21.64% (p=0.000 n=20)
+BuildString_Builder/100Write_36Bytes_NoGrow-4         4.403µ ± 1%   2.381µ ± 2%  -45.91% (p=0.000 n=20)
+BuildString_Builder/1000Write_36Bytes_NoGrow-4        48.28µ ± 2%   21.38µ ± 2%  -55.71% (p=0.000 n=20)
+```
+
+An alternative to strings.Builder calling runtime.freesized itself would
+be to provide an alternate runtime entry point for appends where the
+backing array is known that it will be logically dead after a resize,
+and the runtime therefore knows it can do the free itself. Such a
+function could automatically be inserted by the compiler in many cases
+of appends in a loop, such as in the patterns shown by slices.Collect
+and slices.SeqAppend, though for more complex cases such as
+strings.Builder, it is probably unlikely that the compiler would be able
+to prove it is safe to automatically do so, including for
+strings.Builder, it must track whether Builder.String has already been
+called.  (I suspect it is not the common case to call Builder.String
+multiple times on a single instance, but the API allows it).
+
+Whether we ever use a runtime.free with strings.Builder is debatable
+(including there are alternatives that could be used for
+strings.Builder, such as a segmented buffer and/or sync.Pool
+such as used within cmd/compile). We are mainly using it here as a
+small-ish concrete example.
+
+There are other potential optimizations for strings.Builder,
+like allocating an initial buf (e.g., 64 bytes)
+on the first Write, faster growth than current 1.25x growth for larger
+slices, and "right sizing" the final string, where these could use
+runtime.freesized if those optimizations end up being worthwhile; an
+earlier cut did implement those optimizations, and the intent is to
+measure again with the more recent implementation.
+
+## Details
+
+The term "logically dead" here is distinct from the compiler's liveness
+analysis, and the runtime free and re-use implementations must be (and
+attempt to be) robust in sequences such as a first alloc of a given
+pointer, GC observes and queues the pointer, runtime.free, GC dequeues
+pointer for scanning, runtime re-uses the same pointer, all within a
+single GC phase. (TODO: more precise term than "logically dead"?)
+
+For the cases where a normal allocation is happening without any reuse,
+some initial micro-benchmarks suggest the impact of these changes could
+be small to neglible (at least with GOAMD64=v3):
+
+```
+goos: linux
+goarch: amd64
+pkg: runtime
+cpu: AMD EPYC 7B13
+                        │ base-512M-v3.bench │   ps16-512M-goamd64-v3.bench     │
+                        │        sec/op      │ sec/op     vs base               │
+Malloc8-16                      11.01n ± 1%   10.94n ± 1%  -0.68% (p=0.038 n=20)
+Malloc16-16                     17.15n ± 1%   17.05n ± 0%  -0.55% (p=0.007 n=20)
+Malloc32-16                     18.65n ± 1%   18.42n ± 0%  -1.26% (p=0.000 n=20)
+MallocTypeInfo8-16              18.63n ± 0%   18.36n ± 0%  -1.45% (p=0.000 n=20)
+MallocTypeInfo16-16             22.32n ± 0%   22.65n ± 0%  +1.50% (p=0.000 n=20)
+MallocTypeInfo32-16             23.37n ± 0%   23.89n ± 0%  +2.23% (p=0.000 n=20)
+geomean                         18.02n        18.01n       -0.05%
+```
+
+Slices seem to provide a good initial set of targets for a runtime free,
+though depending on the resulting performance costs and gains, it could
+be natural to extend to other allocations, perhaps with some size cut
+off.
+
+## API
+
+```go
+// freeSized records that a heap object is reusable and available for
+// immediate reuse in a subsequent mallocgc allocation, without
+// needing to wait for the GC cycle to progress.
+//
+// The information is recorded in a free list stored in the
+// current P's mcache. The caller must pass in the user size
+// and whether the object has pointers, which allows a faster free
+// operation.
+//
+// freeSized must be called by the effective owner of ptr who knows
+// the pointer is logically dead, with no possible aliases that might
+// be used past that moment. The intended callers are
+// a limited number of places in the standard library (e.g., strings.Builder)
+// or the runtime (e.g., maps grow/split code, append-handling
+// code). In the future, the compiler could also insert direct freeSized calls,
+// though currently a related API freeTracked serves that purpose.
+//
+// freeSized is a no-op if called on a stack pointer. It must not be called
+// on memory in the data or bss sections, which is partially enforced.
+//
+// In addition, the caller must know that ptr's object has no specials, such
+// as might have been created by a call to SetFinalizer or AddCleanup.
+// (Internally, the runtime deals appropriately with internally-created
+// specials, such as specials for memory profiling).
+//
+// If the size of ptr's object is less than or equal to 16 bytes or
+// greater than 32KiB bytes, freeSized is currently a no-op. It must only
+// be called in alloc-safe places.
+//
+// Note that freeSized accepts an unsafe.Pointer and hence keeps the pointer
+// alive. It therefore could be a pessimistation in some cases (such
+// as a long-lived function) if the caller does not call freeSized before
+// or roughly when the liveness analysis of the compiler
+// would otherwise have determined ptr is reclaimable by the GC.
+func freeSized(p unsafe.Pointer, size uintptr, noscan bool) bool
+
+// makeslicetracked64 is like the existing makeslice64, but the caller
+// also passes in a pointer to a trackedObj slice that allows the runtime
+// to track the heap-allocated backing array and possibly later
+// free the tracked objects via freetracked. The contents of
+// the trackedObj slice are opaque to the caller, but the caller
+// provides the space for the trackedObj slice.
+//
+// Currently, the only intended caller is code inserted by
+// the compiler, which which only does so when escape analysis can
+// prove the associated pointer have a known scope -- currently,
+// when it can prove the pointer does not outlive the current
+// function. This effectively creates a scope for the
+// associated heap pointers. The compiler also arranges for
+// freetracked to be called on function exit, so that
+// the heap objects are automatically freed when leaving the
+// scope. The compiler also arranges for the trackedOjb slice's
+// backing array to be placed on the user's normal goroutine
+// stack to provide storage for the tracking information,
+// with that storage matching the lifetime of the tracked
+// heap objects (that is, both match the lifetime of
+// the function call).
+func makeslicetracked64(et *_type, len64 int64, cap64 int64, trackedObjs *[]trackedObj) unsafe.Pointer
+
+// freetracked marks as reusable the objects allocated by
+// calls to makeslicetracked64 and tracked in the trackedObjs
+// slice, unless the GC has already possibly reclaimed
+// those objects.
+//
+// The trackedObj slice does not keep the associated pointers
+// alive, and freetracked becomes a no-op if the GC cycle
+// has progressed such that the GC might have already reclaimed
+// the tracked pointers before freetracked was called.
+// This might happen for example if a function call lasts
+// multiple GC cycles. This is currently decided per tracked
+// object.
+//
+// The compiler statically counts the maximum number of
+// possible makeslicetracked64 calls in a function
+// in order to statically know how much user
+// stack space is needed for the trackedObj slice.
+// Inside 'for' loops (or when a goto looks like it is
+// being used in an unstructured loop), the compiler
+// currently skips inserting the new makeslicetracked64 calls.
+// TODO: the plan is to have the compiler instead insert a separate
+// runtime entrypoint in loops to allow heap allocated slice memory
+// to be reclaimed on each loop iteration that returns
+// back to the same allocation spot when the compiler
+// can prove it is safe to do so, without requiring unbounded or
+// dynamic space in the trackedObj slice. The compiler can already
+// prove it is safe and the runtime entrypoint is prototyped.
+func freetracked(trackedObjs *[]trackedObj)
+
+// trackedObj is used to track a heap-allocated object that
+// has a proven scope and can potentially be freed by freetracked.
+//
+// trackedOjb does not keep the tracked object alive.
+//
+// trackedObj is opaque outside the allocation and free code.
+// The common case is the sweepgen matches the current sweepgen
+// but if there is a mismatch, the runtime does not attempt to
+// to reuse the tracked object.
+//
+// TODO: there is a plan to allow a trackedObj to have specials,
+// including user-created via SetFinalizer or AddCleanup, but
+// not yet implemented. (In short -- update runtime.addspecial
+// to update a bitmap of objIndexes that have specials so
+// that the fast path for allocation & free does not need to
+// walk an mspan's specials linked list, and then a trackedObj with
+// specials will not be reused. An alternative might be to
+// sweep the span and process the specials after reaching
+// the end of an mspan in the mcache if there are reusable
+// objects with specials, but that might be harder or perhaps
+// infeasible. More generally, sweeping sooner than the
+// normal GC but not in the allocation/free fast path (perhaps
+// in a goroutine managed by the runtime), might be worth
+// exploring for other reasons than just specials).
+type trackedObj struct {
+	ptr      uintptr   // tracked object
+	sweepgen uint32    // TODO: thought sweepgen was 64 bits.
+	spc      spanClass // 1 byte; not used yet
+	objIndex uint16    // 2 bytes; not used yet
+}
+```
\ No newline at end of file

Change information

Files:

A design/74299-runtime-free.md

Change size: L

Delta: 1 file changed, 292 insertions(+), 0 deletions(-)

Open in Gerrit

Related details

Attention set is empty

Submit Requirements:

Code-Review
No-Unresolved-Comments
Review-Enforcement

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

blocking_requirement

satisfied_requirement

unsatisfied_requirement

open

diffy

t hepudds (Gerrit)

unread,

Sep 2, 2025, 5:51:14 PMSep 2

to goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com

t hepudds uploaded new patchset

t hepudds uploaded patch set #2 to this change.

Open in Gerrit

Related details

Attention set is empty

Submit Requirements:

Code-Review
No-Unresolved-Comments
Review-Enforcement

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

blocking_requirement

satisfied_requirement

unsatisfied_requirement

open

diffy

t hepudds (Gerrit)

unread,

Sep 2, 2025, 8:20:32 PMSep 2

to goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com

t hepudds uploaded new patchset

t hepudds uploaded patch set #3 to this change.

Open in Gerrit

Related details

Attention set is empty

Submit Requirements:

Code-Review
No-Unresolved-Comments
Review-Enforcement

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

blocking_requirement

satisfied_requirement

unsatisfied_requirement

open

diffy

t hepudds (Gerrit)

unread,

Sep 2, 2025, 8:35:18 PMSep 2

to goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com

Attention needed from Michael Knyszek

t hepudds uploaded new patchset

t hepudds uploaded patch set #4 to this change.

Open in Gerrit

Related details

Attention is currently required from:

Michael Knyszek

Submit Requirements:

Code-Review
No-Unresolved-Comments
Review-Enforcement

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

blocking_requirement

satisfied_requirement

unsatisfied_requirement

open

diffy

t hepudds (Gerrit)

unread,

Sep 2, 2025, 10:49:48 PMSep 2

to goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com

Attention needed from Michael Knyszek

t hepudds uploaded new patchset

t hepudds uploaded patch set #5 to this change.

Open in Gerrit

Related details

Attention is currently required from:

Michael Knyszek

Submit Requirements:

Code-Review
No-Unresolved-Comments
Review-Enforcement

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

blocking_requirement

satisfied_requirement

unsatisfied_requirement

open

diffy

t hepudds (Gerrit)

unread,

Sep 2, 2025, 10:51:47 PMSep 2

to goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com

Attention needed from Michael Knyszek

t hepudds uploaded new patchset

t hepudds uploaded patch set #6 to this change.

Open in Gerrit

Related details

Attention is currently required from:

Michael Knyszek

Submit Requirements:

Code-Review
No-Unresolved-Comments
Review-Enforcement

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

blocking_requirement

satisfied_requirement

unsatisfied_requirement

open

diffy

t hepudds (Gerrit)

unread,

Sep 3, 2025, 8:08:58 AMSep 3

to goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com

Attention needed from Michael Knyszek

t hepudds uploaded new patchset

t hepudds uploaded patch set #7 to this change.

Open in Gerrit

Related details

Attention is currently required from:

Michael Knyszek

Submit Requirements:

Code-Review
No-Unresolved-Comments
Review-Enforcement

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

blocking_requirement

satisfied_requirement

unsatisfied_requirement

open

diffy

t hepudds (Gerrit)

unread,

Sep 3, 2025, 8:40:20 AMSep 3

to goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com

Attention needed from Michael Knyszek

t hepudds uploaded new patchset

t hepudds uploaded patch set #8 to this change.

Open in Gerrit

Related details

Attention is currently required from:

Michael Knyszek

Submit Requirements:

Code-Review
No-Unresolved-Comments
Review-Enforcement

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

blocking_requirement

satisfied_requirement

unsatisfied_requirement

open

diffy

t hepudds (Gerrit)

unread,

Sep 3, 2025, 11:17:23 AMSep 3

to goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com

Attention needed from Michael Knyszek

t hepudds uploaded new patchset

t hepudds uploaded patch set #9 to this change.

Open in Gerrit

Related details

Attention is currently required from:

Michael Knyszek

Submit Requirements:

Code-Review
No-Unresolved-Comments
Review-Enforcement

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

blocking_requirement

satisfied_requirement

unsatisfied_requirement

open

diffy

t hepudds (Gerrit)

unread,

Sep 3, 2025, 11:23:33 AMSep 3

to goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com

Attention needed from Michael Knyszek

t hepudds uploaded new patchset

t hepudds uploaded patch set #10 to this change.

Open in Gerrit

Related details

Attention is currently required from:

Michael Knyszek

Submit Requirements:

Code-Review
No-Unresolved-Comments
Review-Enforcement

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

blocking_requirement

satisfied_requirement

unsatisfied_requirement

open

diffy

t hepudds (Gerrit)

unread,

Sep 3, 2025, 11:46:45 AMSep 3

to goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com

Attention needed from Michael Knyszek

t hepudds uploaded new patchset

t hepudds uploaded patch set #11 to this change.

Open in Gerrit

Related details

Attention is currently required from:

Michael Knyszek

Submit Requirements:

Code-Review
No-Unresolved-Comments
Review-Enforcement

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

blocking_requirement

satisfied_requirement

unsatisfied_requirement

open

diffy

t hepudds (Gerrit)

unread,

Sep 3, 2025, 11:48:25 AMSep 3

to goph...@pubsubhelper.golang.org, Michael Knyszek, golang-co...@googlegroups.com

Attention needed from Michael Knyszek

t hepudds added 1 comment

Patchset-level comments

File-level comment, Patchset 11 (Latest):

t hepudds . unresolved

PTAL when convenient. (@mkny...@google.com, I think you've already read ~2/3 of this material when it was in the commit message for CL 673695, so feel free to skim 😄. The most "new to you" section might be the "Compiler changes overview" section).

Open in Gerrit

Related details

Attention is currently required from:

Michael Knyszek

Submit Requirements:

Code-Review

No-Unresolved-Comments

Review-Enforcement

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

blocking_requirement

unsatisfied_requirement

open

diffy

Carlo Alberto Ferraris (Gerrit)

unread,

Sep 17, 2025, 3:39:54 AMSep 17

to t hepudds, goph...@pubsubhelper.golang.org, Michael Knyszek, golang-co...@googlegroups.com

Attention needed from Michael Knyszek and t hepudds

Carlo Alberto Ferraris added 3 comments

File design/74299-runtime-free.md

Line 230, Patchset 11 (Latest): trackSlice(unsafe.Pointer(&s2), &freeables)

Carlo Alberto Ferraris . unresolved

how does `trackSlice` know how many slots in freeables are already populated?

Line 239, Patchset 11 (Latest):based on the count of `make` calls in a given function.

Carlo Alberto Ferraris . unresolved

Here is `make` a shorthand for all slice allocations, or will something like
`s1 := append([]int64{}, someOtherInt64Slice...)`
(and therefore also the allocate-slice-with-roundedup-capacity idiom) not be covered?

Line 243, Patchset 11 (Latest):currently skips automatically inserting these calls.

Carlo Alberto Ferraris . unresolved

This is a bit unfortunate. I think I can guess that handling the two cases (allocation does or does not outlive the current loop iteration) is more complex, but allocations in a loop is probably the most compelling use case of these explicit frees.

Open in Gerrit

Related details

Attention is currently required from:

Michael Knyszek
t hepudds

Submit Requirements:

Code-Review
No-Unresolved-Comments
Review-Enforcement

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

blocking_requirement

unsatisfied_requirement

open

diffy

t hepudds (Gerrit)

unread,

Sep 25, 2025, 12:36:08 AMSep 25

to goph...@pubsubhelper.golang.org, Carlo Alberto Ferraris, Michael Knyszek, golang-co...@googlegroups.com

Attention needed from Carlo Alberto Ferraris and Michael Knyszek

t hepudds added 4 comments

Patchset-level comments

File-level comment, Patchset 11 (Latest):

t hepudds . resolved

Hi Carlo, thanks for taking a look!

File design/74299-runtime-free.md

Line 230, Patchset 11 (Latest): trackSlice(unsafe.Pointer(&s2), &freeables)

Carlo Alberto Ferraris . resolved

how does `trackSlice` know how many slots in freeables are already populated?

t hepudds

`freeables` is an actual slice, so it moves the `len` field forward as each slot is populated. (It doesn't strictly need to be a slice, but it's convenient).

Line 239, Patchset 11 (Latest):based on the count of `make` calls in a given function.

Carlo Alberto Ferraris . resolved

Here is `make` a shorthand for all slice allocations, or will something like
`s1 := append([]int64{}, someOtherInt64Slice...)`
(and therefore also the allocate-slice-with-roundedup-capacity idiom) not be covered?

t hepudds

That should be handled as well (but is not implemented yet).

Line 243, Patchset 11 (Latest):currently skips automatically inserting these calls.

Carlo Alberto Ferraris . resolved

This is a bit unfortunate. I think I can guess that handling the two cases (allocation does or does not outlive the current loop iteration) is more complex, but allocations in a loop is probably the most compelling use case of these explicit frees.

t hepudds

One thing to bear in mind is that the work is not yet complete, and some of the text in this document is highlighting some limitations of what is currently implemented.

In this case, the limitation you are commenting on here would not be a limitation in the first version, assuming it all goes forward through review, etc.

Allocations in loops is definitely one of the key targets of this work.

(Also, there are different approaches, and not just `freeTracked`. For example, the next paragraph is not talking about using `freeTracked` -- there is a separate runtime entry point being described there, though I haven't picked a name for it yet).

Open in Gerrit

Related details

Attention is currently required from:

Carlo Alberto Ferraris
Michael Knyszek

Submit Requirements:

Code-Review
No-Unresolved-Comments
Review-Enforcement

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

blocking_requirement

unsatisfied_requirement

open

diffy

Carlo Alberto Ferraris (Gerrit)

unread,

Oct 8, 2025, 12:55:43 AMOct 8

to t hepudds, goph...@pubsubhelper.golang.org, Michael Knyszek, golang-co...@googlegroups.com

Attention needed from Michael Knyszek and t hepudds

Carlo Alberto Ferraris added 1 comment

File design/74299-runtime-free.md

Line 230, Patchset 11 (Latest): trackSlice(unsafe.Pointer(&s2), &freeables)

Carlo Alberto Ferraris . resolved

how does `trackSlice` know how many slots in freeables are already populated?

t hepudds

`freeables` is an actual slice, so it moves the `len` field forward as each slot is populated. (It doesn't strictly need to be a slice, but it's convenient).

Carlo Alberto Ferraris

ah I see, so on line 233 it should be `[:0]`? probably just nitpicking, but it's what led me astray

Open in Gerrit

Related details

Attention is currently required from:

Michael Knyszek
t hepudds

Submit Requirements:

Code-Review
No-Unresolved-Comments
Review-Enforcement

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

blocking_requirement

unsatisfied_requirement

open

diffy

Carlo Alberto Ferraris (Gerrit)

unread,

Oct 8, 2025, 12:56:17 AMOct 8

to t hepudds, goph...@pubsubhelper.golang.org, Michael Knyszek, golang-co...@googlegroups.com

Attention needed from Michael Knyszek and t hepudds

Carlo Alberto Ferraris added 1 comment

File design/74299-runtime-free.md

Line 230, Patchset 11 (Latest): trackSlice(unsafe.Pointer(&s2), &freeables)

Carlo Alberto Ferraris . resolved

how does `trackSlice` know how many slots in freeables are already populated?

t hepudds

`freeables` is an actual slice, so it moves the `len` field forward as each slot is populated. (It doesn't strictly need to be a slice, but it's convenient).

Carlo Alberto Ferraris

ah I see, so on line 233 it should be `[:0]`? probably just nitpicking, but it's what led me astray

Carlo Alberto Ferraris

*line 223, sorry

Open in Gerrit

Related details

Attention is currently required from:

Michael Knyszek
t hepudds

Submit Requirements:

Code-Review
No-Unresolved-Comments
Review-Enforcement

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

blocking_requirement

unsatisfied_requirement

open

diffy

t hepudds (Gerrit)

unread,

Oct 9, 2025, 1:48:06 PMOct 9

to goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com

Attention needed from Michael Knyszek and t hepudds

t hepudds uploaded new patchset

t hepudds uploaded patch set #12 to this change.

Open in Gerrit

Related details

Attention is currently required from:

Michael Knyszek
t hepudds

Submit Requirements:

Code-Review
No-Unresolved-Comments
Review-Enforcement

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

blocking_requirement

unsatisfied_requirement

open

diffy

t hepudds (Gerrit)

unread,

Oct 9, 2025, 1:48:57 PMOct 9

to goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com

Attention needed from Michael Knyszek and t hepudds

t hepudds uploaded new patchset

t hepudds uploaded patch set #13 to this change.

Open in Gerrit

Related details

Attention is currently required from:

Michael Knyszek
t hepudds

Submit Requirements:

Code-Review
No-Unresolved-Comments
Review-Enforcement

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

blocking_requirement

unsatisfied_requirement

open

diffy

t hepudds (Gerrit)

unread,

Oct 9, 2025, 1:53:51 PMOct 9

to goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com

Attention needed from Michael Knyszek and t hepudds

t hepudds uploaded new patchset

t hepudds uploaded patch set #14 to this change.

Open in Gerrit

Related details

Attention is currently required from:

Michael Knyszek
t hepudds

Submit Requirements:

Code-Review
No-Unresolved-Comments
Review-Enforcement

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

blocking_requirement

unsatisfied_requirement

open

diffy

t hepudds (Gerrit)

unread,

Oct 9, 2025, 1:59:07 PMOct 9

to goph...@pubsubhelper.golang.org, Carlo Alberto Ferraris, Michael Knyszek, golang-co...@googlegroups.com

Attention needed from Carlo Alberto Ferraris and Michael Knyszek

t hepudds added 1 comment

File design/74299-runtime-free.md

Line 230, Patchset 11: trackSlice(unsafe.Pointer(&s2), &freeables)

Carlo Alberto Ferraris . resolved

how does `trackSlice` know how many slots in freeables are already populated?

t hepudds

`freeables` is an actual slice, so it moves the `len` field forward as each slot is populated. (It doesn't strictly need to be a slice, but it's convenient).

Carlo Alberto Ferraris

ah I see, so on line 233 it should be `[:0]`? probably just nitpicking, but it's what led me astray

Carlo Alberto Ferraris

*line 223, sorry

t hepudds

Fixed, thanks!

Open in Gerrit

Related details

Attention is currently required from:

Carlo Alberto Ferraris
Michael Knyszek

Submit Requirements:

Code-Review
No-Unresolved-Comments
Review-Enforcement

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

blocking_requirement

unsatisfied_requirement

open

diffy

Michael Knyszek (Gerrit)

unread,

Oct 21, 2025, 6:03:05 PMOct 21

to t hepudds, goph...@pubsubhelper.golang.org, Carlo Alberto Ferraris, golang-co...@googlegroups.com

Attention needed from Carlo Alberto Ferraris and t hepudds

Michael Knyszek added 28 comments

File design/74299-runtime-free.md

Line 11, Patchset 14 (Latest):This is a design for implementing a runtime.free within the

Michael Knyszek . unresolved

as an aside, I wonder if we should name this `runtime.freegc` just to have a duality with `mallocgc`?

Line 11, Patchset 14 (Latest):

This is a design for implementing a runtime.free within the
runtime, and then using it via automatic calls inserted by the compiler
and via a limited set of explicit calls from the standard library.

Michael Knyszek . unresolved

maybe let's go a little more abstract. here's a suggestion (for this, and all my suggestions, feel free to take the text as written or change it as you see fit; I'm not _that_ picky :)):

"This is a design for a mechanism to free memory eagerly to the Go runtime to be reused within the same GC cycle. The mechanism is implemented in a new runtime function, `runtime.free`, to be used primarily by the compiler in cases where it can prove the memory is no longer used."

Line 15, Patchset 14 (Latest):

When triggered, the goals include faster reuse of user memory,
less overall CPU usage by the GC, longer times between GC cycles
(with less overall time with the write barrier enabled),
and more cache-friendly allocations for user code.

Michael Knyszek . unresolved

just some wording suggestions. maybe:

"When enabled, `runtime.free` allows more eager reuse of user memory, reducing the allocation rate from the GC's perspective, and thus the number of GC cycles. It also enables a more cache-friendly allocation pattern for user code."

Line 20, Patchset 14 (Latest):

This hopefully complements other on-going work like:
 * [memory regions](https://go.dev/issue/70257), which can dynamically cast a wider net via a new user-facing API.
 * [more stack allocations of slices](https://go.dev/cl/664299), which currently targets smaller slices such as <= 32 bytes.
 * [Green Tea](https://go.dev/issue/73581), which is a general performance improvement for the GC in many cases.

Michael Knyszek . unresolved

I think it might be worth just saying that this composes with stack allocation and memory regions. I think the other two things are somewhat self-evident.

"This design composes well with stack allocation by filling in gaps that stack allocation cannot currently handle. It also composes well with possible future memory management improvements like memory regions."

Line 28, Patchset 14 (Latest):does not escape, but in many cases today the allocation is

Michael Knyszek . unresolved

redundant "today"

Line 28, Patchset 14 (Latest):

does not escape, but in many cases today the allocation is
still placed on the heap (such as because of a variable size
that is not statically visible at compile time, or the size is
too large for the stack).

Michael Knyszek . unresolved

maybe "but may still choose to place it on the heap, such as ..."

Line 42, Patchset 14 (Latest):variable size known only at runtime. Currently, a make with an unknown

Michael Knyszek . unresolved

wrap this in backticks maybe?

Line 50, Patchset 14 (Latest):In other words, a runtime.free delivers performance benefits on its own,

Michael Knyszek . unresolved

wrap in backticks?

Line 68, Patchset 14 (Latest): * doing this safely (including in the face of the concurrent GC,

Michael Knyszek . unresolved

nit, maybe

"Safety, including ..."

Line 70, Patchset 14 (Latest): to when it is processed)

Michael Knyszek . unresolved

period or comma

Line 71, Patchset 14 (Latest): * doing this performantly (including without materially slowing down

Michael Knyszek . unresolved

same here, maybe "Performance, specifically ..."

Line 100, Patchset 14 (Latest):**Note**: It is not intended that higher-level portions of the stdlib (like net/http) would have manual

Michael Knyszek . unresolved

backticks

Line 101, Patchset 14 (Latest):calls to a runtime.free.

Michael Knyszek . unresolved

backticks

Line 122, Patchset 14 (Latest):6. Possibly extend to freeing more than just slices. (Slices are a natural

Michael Knyszek . unresolved

I think you can just drop the parentheses here

Line 135, Patchset 14 (Latest):Currently, the first API `freeSized` can be called by the


standard library, with initial targets in the map grow/split code,

Michael Knyszek . unresolved

based on recent discussions, it sounds like freeSized can also be called by the compiler. might be worth outlining exactly when that is.

Line 141, Patchset 14 (Latest):The second API `freeTracked` is automatically inserted by the compiler.

Michael Knyszek . unresolved

I think it would be helpful to explain how we got here, because I don't think it's quite as obvious as freeSized.

1. compiler liveness analysis is on the function level.
2. we may need to free multiple object in a given function.
3. multiple calls adds binary size and possibly icache pressure.

Line 141, Patchset 14 (Latest):

The second API `freeTracked` is automatically inserted by the compiler.
`freeTracked` is paired with a new slice allocation function. Together,

Michael Knyszek . unresolved

I think it would be helpful to explain exactly how you arrived at freeTracked:

1. the compiler's escape analysis tracks allocations at the granularity of function calls, so we want to

Line 248, Patchset 14 (Latest): defer freeTracked(&freeables) // only a single freeTracked call, regardless of how many tracked sites

Michael Knyszek . unresolved

this is fine from a semantics point of view, but I think we might want to find a way to avoid having this count toward the maximum defer count (or bump it by one), which could throw off people carefully optimizing their code.

Line 269, Patchset 14 (Latest):

However, a modest extension (and the current plan) is to have the compiler instead insert
a separate runtime entry point in loops to allow heap-allocated slice memory


to be reclaimed on each loop iteration that returns

back to the same allocation spot when the compiler

can prove it is safe to do so, without requiring unbounded or

dynamic space in the trackedObj slice. (The compiler can already

Michael Knyszek . unresolved

this sentence is a bit of a run-on. suggested alternative wording:

"However, a modest extension (and the current plan) is to have the compiler instead use a different runtime entry point for freeing in loops. The idea is to allow heap-allocated slice memory to be reclaimed on each loop iteration when it returns to the same allocation site, and when it's safe to do so. Because it's the same allocation site, this doesn't require unbounded or dynamic space in the trackedObj slice."

Line 279, Patchset 14 (Latest):Using strings.Builder as an example target -- this is already a somewhat

Michael Knyszek . unresolved

backticks

Line 281, Patchset 14 (Latest):performance-related trickery, including using unsafe and

Michael Knyszek . unresolved

backticks

Line 282, Patchset 14 (Latest):bytealg.MakeNoZero. The strings.Builder already owns its in-progress

Michael Knyszek . unresolved

backticks

Line 282, Patchset 14 (Latest):bytealg.MakeNoZero. The strings.Builder already owns its in-progress

Michael Knyszek . unresolved

backticks (and maybe "internal/bytealg"?)

Line 293, Patchset 14 (Latest):If we update strings.Builder to explicitly call runtime.freeSized on

Michael Knyszek . unresolved

backticks

Line 293, Patchset 14 (Latest):If we update strings.Builder to explicitly call runtime.freeSized on

Michael Knyszek . unresolved

backticks

Line 297, Patchset 14 (Latest):that call Builder.Write N times and then call Builder.String. Here, the

Michael Knyszek . unresolved

same here

Line 297, Patchset 14 (Latest):that call Builder.Write N times and then call Builder.String. Here, the

Michael Knyszek . unresolved

backticks (maybe also `strings.(*Builder).Write`)?

Line 475, Patchset 14 (Latest):type trackedObj struct {


	ptr      uintptr   // tracked object

	sweepgen uint32    // TODO: perhaps use more bits;  mheap_.sweepgen is 32 bits.
	spc      spanClass // 1 byte; potential optimization (not used yet)
}

Michael Knyszek . unresolved

I think you may want to copy this definition up early in the document, around where you first mention freeTracked, since this is its first reference.

I think the potential optimization here could be mentioned in a 'future work' section, since it's not used otherwise.

Open in Gerrit

Related details

Attention is currently required from:

Carlo Alberto Ferraris
t hepudds

Submit Requirements:

Code-Review
No-Unresolved-Comments

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

blocking_requirement

unsatisfied_requirement

open

diffy

Michael Knyszek (Gerrit)

unread,

Oct 21, 2025, 6:03:21 PMOct 21

to t hepudds, goph...@pubsubhelper.golang.org, Carlo Alberto Ferraris, golang-co...@googlegroups.com

Attention needed from Carlo Alberto Ferraris and t hepudds

Michael Knyszek voted Code-Review+2

Code-Review

+2

Open in Gerrit

Related details

Attention is currently required from:

Carlo Alberto Ferraris
t hepudds

Submit Requirements:

Code-Review

No-Unresolved-Comments

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

satisfied_requirement

unsatisfied_requirement

open

diffy

Carlo Alberto Ferraris (Gerrit)

unread,

Oct 21, 2025, 10:06:49 PMOct 21

to t hepudds, goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com

Attention needed from t hepudds

Carlo Alberto Ferraris added 1 comment

File design/74299-runtime-free.md

Line 482, Patchset 14 (Latest):## Other possible runtime APIs

Carlo Alberto Ferraris . unresolved

Will special objects (chan, map) require/deserve special handling? I would definitely argue that given the proliferation of throwaway channels (e.g. in context.Context) having the runtime be able to safely reuse a closed/dirty channel would be quite valuable, especially since channels are basically one of those few things that are basically impossible to reuse in other ways.

Open in Gerrit

Related details

Attention is currently required from:

t hepudds

Submit Requirements:

Code-Review
No-Unresolved-Comments

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

satisfied_requirement

unsatisfied_requirement

open

diffy

Keith Randall (Gerrit)

unread,

Oct 31, 2025, 6:34:41 PM (12 days ago) Oct 31

to t hepudds, goph...@pubsubhelper.golang.org, Keith Randall, Carlo Alberto Ferraris, golang-co...@googlegroups.com

Attention needed from t hepudds

Keith Randall added 4 comments

File design/74299-runtime-free.md

Line 198, Patchset 14 (Latest): sweepgen uint32

Keith Randall . unresolved

Do we need a separate sweepgen for every pointer? Or can we keep a single one somehow?
(but see below)

Line 201, Patchset 14 (Latest): defer freeTracked(&freeables)

Keith Randall . unresolved

I think using defer might be overkill here. I'm happy if we miss opportunities for `free` when panicking. So we could just add a call at every return point.

Line 253, Patchset 14 (Latest): s2 := make([]int64, size2)

Keith Randall . unresolved

It is unfortunate that this takes 2 runtime calls. Could we have a malloc variant that returns all the info we need (sweepgen,etc.)?

Line 474, Patchset 14 (Latest):// exploring for other reasons than just specials).

Keith Randall . unresolved

An idea to get rid of sweepgen: could we record, with some pcdata info, a region in the stack frame that the runtime would zero when sweepgen advances?
Then we can just throw raw pointers into that region, and know that if they are still non-nil, it is ok to free them.
(This is kind of a weak pointer concept.)

We could also zero the slice length of the trackedObj slice at the same time.

This way, we can use real (according to the GC) pointers, so things like stack copies just work.

Open in Gerrit

Related details

Attention is currently required from:

t hepudds

Submit Requirements:

Code-Review
No-Unresolved-Comments

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

satisfied_requirement

unsatisfied_requirement

open

diffy

t hepudds (Gerrit)

unread,

Nov 11, 2025, 4:24:41 PM (22 hours ago) Nov 11

to goph...@pubsubhelper.golang.org, Keith Randall, Carlo Alberto Ferraris, golang-co...@googlegroups.com

Attention needed from Keith Randall

t hepudds added 1 comment

File design/74299-runtime-free.md

Line 474, Patchset 14 (Latest):// exploring for other reasons than just specials).

Keith Randall . unresolved

An idea to get rid of sweepgen: could we record, with some pcdata info, a region in the stack frame that the runtime would zero when sweepgen advances?
Then we can just throw raw pointers into that region, and know that if they are still non-nil, it is ok to free them.
(This is kind of a weak pointer concept.)
We could also zero the slice length of the trackedObj slice at the same time.
This way, we can use real (according to the GC) pointers, so things like stack copies just work.

t hepudds

Hi Keith, this is an excellent idea. I will update the document to reflect this as the preferred approach.

I suspect it might touch more distinct parts of the code compared to the original proposed design, but it is conceptually cleaner and better overall.

so things like stack copies just work.

(I think I had it such that stack copies were not a concern with the original design / draft implementation of freeTracked and trackedObj, but hopefully that is a moot point.)

Open in Gerrit

Related details

Attention is currently required from:

Keith Randall

Submit Requirements:

Code-Review
No-Unresolved-Comments

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

satisfied_requirement

unsatisfied_requirement

open

diffy

t hepudds (Gerrit)

unread,

Nov 11, 2025, 4:32:35 PM (22 hours ago) Nov 11

to goph...@pubsubhelper.golang.org, Keith Randall, Carlo Alberto Ferraris, golang-co...@googlegroups.com

Attention needed from Keith Randall

t hepudds added 1 comment

File design/74299-runtime-free.md

Line 253, Patchset 14 (Latest): s2 := make([]int64, size2)

Keith Randall . resolved

It is unfortunate that this takes 2 runtime calls. Could we have a malloc variant that returns all the info we need (sweepgen,etc.)?

t hepudds

Hi Keith, just using one runtime call here was/is definitely the intent.

The document tried to explain above that `trackSlice` was a temporary shortcut:

 * the current intent is to use `makeslicetracked64` [...],
 but the compiler currently inserts calls to a new `runtime.trackSlice` instead,
 which initially seemed slightly easier to implement on the compiler side.
 The current plan is to have the compiler use `makeslicetracked64` instead.
 
(Basically, from initial poking around, I had thought the blast radius from changing to a new runtime call would be larger than it turned out to be, so it seemed simpler to start by introduce a new / second call, which is where things stood when I first posted this design document, though at this point I don't think it was actually very helpful as a shortcut.)

Open in Gerrit

Related details

Attention is currently required from:

Keith Randall

Submit Requirements:

Code-Review
No-Unresolved-Comments

Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings.

Gerrit

satisfied_requirement

unsatisfied_requirement

open

diffy

Reply all

Reply to author

Forward

[proposal] design/74299-runtime-free.md: add runtime.free and use via compiler and stdlib to reduce GC work

t hepudds (Gerrit)

t hepudds has uploaded the change for review

Commit message

Change diff

Change information

Related details

t hepudds (Gerrit)

t hepudds uploaded new patchset

Related details

t hepudds (Gerrit)

t hepudds uploaded new patchset

Related details

t hepudds (Gerrit)

t hepudds uploaded new patchset

Related details

t hepudds (Gerrit)

t hepudds uploaded new patchset

Related details

t hepudds (Gerrit)

t hepudds uploaded new patchset

Related details

t hepudds (Gerrit)

t hepudds uploaded new patchset

Related details

t hepudds (Gerrit)

t hepudds uploaded new patchset

Related details

t hepudds (Gerrit)

t hepudds uploaded new patchset

Related details

t hepudds (Gerrit)

t hepudds uploaded new patchset

Related details

t hepudds (Gerrit)

t hepudds uploaded new patchset

Related details

t hepudds (Gerrit)

t hepudds added 1 comment

Related details

Carlo Alberto Ferraris (Gerrit)

Carlo Alberto Ferraris added 3 comments

Related details

t hepudds (Gerrit)

t hepudds added 4 comments

Related details

Carlo Alberto Ferraris (Gerrit)

Carlo Alberto Ferraris added 1 comment

Related details

Carlo Alberto Ferraris (Gerrit)

Carlo Alberto Ferraris added 1 comment

Related details

t hepudds (Gerrit)

t hepudds uploaded new patchset

Related details

t hepudds (Gerrit)

t hepudds uploaded new patchset

Related details

t hepudds (Gerrit)

t hepudds uploaded new patchset

Related details

t hepudds (Gerrit)

t hepudds added 1 comment

Related details

Michael Knyszek (Gerrit)

Michael Knyszek added 28 comments

Related details

Michael Knyszek (Gerrit)

Michael Knyszek voted Code-Review+2

Related details

Carlo Alberto Ferraris (Gerrit)

Carlo Alberto Ferraris added 1 comment

Related details

Keith Randall (Gerrit)

Keith Randall added 4 comments

Related details

t hepudds (Gerrit)

t hepudds added 1 comment

Related details

t hepudds (Gerrit)