Go 1.17.1 runtime seems retain too much memory from OS

[Zhao Weng]

memstats show heapIdle - heapReleased > heapAlloc or heapInUse

I try to search the code and found the below comment:

// HeapIdle is bytes in idle (unused) spans.

//

// Idle spans have no objects in them. These spans could be

// (and may already have been) returned to the OS, or they can

// be reused for heap allocations, or they can be reused as

// stack memory.

//

// HeapIdle minus HeapReleased estimates the amount of memory

// that could be returned to the OS, but is being retained by

// the runtime so it can grow the heap without requesting more

// memory from the OS. If this difference is significantly

// larger than the heap size, it indicates there was a recent

// transient spike in live heap size.

HeapIdle uint64

and from mgcscavenge.go:

// That goal is defined as:

// (retainExtraPercent+100) / 100 * (heapGoal / lastHeapGoal) * last_heap_inuse

it seem to me heapIdle - heapReleased should be ~ heapInUse * 10%

can anybody help me understand this?

many thanks!!

[Amnon]

Have a look at https://tip.golang.org/doc/gc-guide

and 
https://opensourcelive.withgoogle.com/events/go-day-2022/watch?talk=talk4

[Zhao Weng]

sorry for being not specific.

my question is:

according to my understanding about memstat ( correct me if I'm wrong)

with GOGC set to 50, I can see from the graph above that GCGoal is ~ 26GB

and heapIdle - heapReleased is how much memory go runtime retain from OS for later usage, how can it (heapIdle - heapReleased) be 35GB which is 1.3x GCGoal ? 

it seem to me is something stop scavenger from releasing those memory back to OS.

and by reading the code:

// HeapIdle is bytes in idle (unused) spans.

//

// Idle spans have no objects in them. These spans could be

// (and may already have been) returned to the OS, or they can

// be reused for heap allocations, or they can be reused as

// stack memory.

//

// HeapIdle minus HeapReleased estimates the amount of memory

// that could be returned to the OS, but is being retained by

// the runtime so it can grow the heap without requesting more

// memory from the OS. If this difference is significantly

// larger than the heap size, it indicates there was a recent

// transient spike in live heap size.

HeapIdle uint64

and from mgcscavenge.go:

// That goal is defined as:

// (retainExtraPercent+100) / 100 * (heapGoal / lastHeapGoal) * last_heap_inuse

it seems (heapIdle - heapReleased) should be like 3GB (which is retainExtraPercent/100 * heapInUse ~ 10% * 26GB ~ 3GB)

and i'm really confused by this situation that (heapIdle - heapReleased) be 35GB