[go] runtime/debug: add SetMaxHeap API
Austin Clements (Gerrit)
Austin Clements would like Rick Hudson to review this change.
runtime/debug: add SetMaxHeap API
DO NOT SUBMIT. This is an experiment to get some experience with the
API and figure out if this is even a reasonable primitive. It needs
tests. The implementation is complete, but there are some questions
about the API in the code.
This adds an API to set a soft limit on the heap size. This augments
the existing GOGC-based GC policy by using the lower of the
GOGC-computed GC target and the heap limit.
When the garbage collector is bounded by the heap limit, it can no
longer amortize the cost of garbage collection against the cost of
growing the heap. Hence, callers of this API are required to register
for notifications of when the garbage collector is under pressure and
are strongly encouraged to use this signal to shed load.
Updates #16843.
Change-Id: I016de1d0ee0d7e34d2752c38cae64d0d0cfedb5f
---
M src/runtime/debug/garbage.go
M src/runtime/mgc.go
M src/runtime/mheap.go
M src/runtime/mstats.go
4 files changed, 198 insertions(+), 12 deletions(-)
diff --git a/src/runtime/debug/garbage.go b/src/runtime/debug/garbage.go
index 1c83656..fa3bc9e 100644
--- a/src/runtime/debug/garbage.go
+++ b/src/runtime/debug/garbage.go
@@ -167,3 +167,110 @@
// If SetTraceback is called with a level lower than that of the
// environment variable, the call is ignored.
func SetTraceback(level string)
+
+// TODO: Figure out how to describe this under the current name or
+// rename it to GCPolicy: "GCPolicy reports the garbage collector's
+// policy for controlling the heap size."
+type GCPressure struct {
+ // GCPercent is the current value of GOGC, as set by the GOGC
+ // environment variable or SetGCPercent.
+ //
+ // If GC is disabled, this will be -1.
+ GCPercent int
+
+ // MaxHeapBytes is the current soft heap limit set by
+ // SetMaxHeap, in bytes.
+ //
+ // If there is no heap limit set, this will be ^uintptr(0).
+ MaxHeapBytes uintptr
+
+ // EffectiveGCPercent is the heap growth allowed when
+ // accounting for both GCPercent and MaxHeapBytes.
+ //
+ // This value is percentage with the same interpretation as
+ // GCPercent.
+ //
+ // If this is equal to GCPercent, it means heap growth during
+ // the current garbage collection cycle will not exceed
+ // MaxHeapBytes. Otherwise, this will be less than GCPercent,
+ // indicating the garbage collector has to run more frequently
+ // to keep the heap under MaxHeapBytes.
+ //
+ // If this is less than GCPercent, the application should
+ // generally shed load and reduce its live heap size.
+ EffectiveGCPercent int
+}
+
+// SetMaxHeap sets a soft limit on the size of the Go heap and returns
+// the previous setting. By default, there is no limit.
+//
+// If a max heap is set, the garbage collector will endeavor to keep
+// the heap size under the specified size, even if this is lower than
+// would normally be determined by GOGC (see SetGCPercent).
+//
+// The garbage collector will send an up-to-date GCPressure value to
+// the notify channel whenever the value of any of the fields in
+// GCPressure changes.
+//
+// The application is strongly encouraged to use the GCPressure
+// notification to shed load and reduce its live heap size when
+// EffectiveGCPercent drops below GCPercent. Setting a maximum heap
+// size limits the garbage collector's ability to amortize the cost of
+// garbage collection when the heap reaches this limit. This is
+// particularly important in request-processing systems, where
+// increasing pressure on the garbage collector reduces CPU time
+// available to the application, making it less able to complete work,
+// leading to even more pressure on the garbage collector. The
+// application must shed load to avoid this.
+//
+// The limit set by SetMaxHeap is soft. If the garbage collector would
+// consume too much CPU to keep the heap under this limit (leading to
+// "thrashing"), it will allow the heap to grow larger than the
+// specified max heap.
+//
+// The heap size does not include everything in the process's memory
+// footprint. Notably, it does not include stacks, C-allocated memory,
+// or many runtime-internal structures.
+//
+// To disable the heap limit, pass ^uintptr(0) for the bytes argument.
+// In this case, notify can be nil.
+//
+// If bytes is 0, SetMaxHeap returns the current heap limit without
+// changing it and ignores notify.
+func SetMaxHeap(bytes uintptr, notify chan<- GCPressure) uintptr {
+ if bytes == 0 || bytes == ^uintptr(0) {
+ return gcSetMaxHeap(bytes, nil)
+ }
+ if notify == nil {
+ panic("SetMaxHeap requires a non-nil notify channel")
+ }
+ var prevGCP GCPressure
+ ReadGCPressure(&prevGCP)
+ return gcSetMaxHeap(bytes, func(gogc int, maxHeap uintptr, egogc int) {
+ // This function may run with runtime locks or during
+ // STW. Non-blocking channel sends are okay.
+ gcp := GCPressure{gogc, maxHeap, egogc}
+ // The runtime calls this function on every
+ // gcSetTriggerRatio, which may not actually change
+ // the GCPressure.
+ if gcp == prevGCP {
+ return
+ }
+ prevGCP = gcp
+ select {
+ case notify <- gcp:
+ default:
+ }
+ })
+}
+
+// gcSetMaxHeap is provided by package runtime.
+func gcSetMaxHeap(bytes uintptr, cb func(gogc int, maxHeap uintptr, egogc int)) uintptr
+
+// ReadGCPressure reads the current
+func ReadGCPressure(gcp *GCPressure) {
+ gcp.GCPercent, gcp.MaxHeapBytes, gcp.EffectiveGCPercent = gcReadPressure()
+}
+
+// gcReadPressure is provided by package runtime.
+func gcReadPressure() (gogc int, maxHeap uintptr, egogc int)
diff --git a/src/runtime/mgc.go b/src/runtime/mgc.go
index 57da402..2f99cda 100644
--- a/src/runtime/mgc.go
+++ b/src/runtime/mgc.go
@@ -189,6 +189,9 @@
// This will go into computing the initial GC goal.
memstats.heap_marked = uint64(float64(heapminimum) / (1 + memstats.triggerRatio))
+ // Disable heap limit initially.
+ mheap_.maxHeap = ^uintptr(0)
+
// Set gcpercent from the environment. This will also compute
// and set the GC trigger and goal.
_ = setGCPercent(readgogc())
@@ -233,6 +236,22 @@
return out
}
+//go:linkname gcSetMaxHeap runtime/debug.gcSetMaxHeap
+func gcSetMaxHeap(bytes uintptr, cb func(gogc int, maxHeap uintptr, egogc int)) uintptr {
+ lock(&mheap_.lock)
+ prev := mheap_.maxHeap
+ if bytes == 0 {
+ unlock(&mheap_.lock)
+ return prev
+ }
+ mheap_.maxHeap = bytes
+ mheap_.gcPressureChange = cb
+ // Updating pacing.
+ gcSetTriggerRatio(memstats.triggerRatio)
+ unlock(&mheap_.lock)
+ return prev
+}
+
// Garbage collector phase.
// Indicates to write barrier and synchronization task to perform.
var gcphase uint32
@@ -755,17 +774,40 @@
// This can be called any time. If GC is the in the middle of a
// concurrent phase, it will adjust the pacing of that phase.
//
-// This depends on gcpercent, memstats.heap_marked, and
-// memstats.heap_live. These must be up to date.
+// This depends on gcpercent, mheap_.maxHeap, memstats.heap_marked,
+// and memstats.heap_live. These must be up to date.
//
// mheap_.lock must be held or the world must be stopped.
func gcSetTriggerRatio(triggerRatio float64) {
// Compute the next GC goal, which is when the allocated heap
// has grown by GOGC/100 over the heap marked by the last
- // cycle.
+ // cycle, or maxHeap, whichever is lower.
goal := ^uint64(0)
if gcpercent >= 0 {
goal = memstats.heap_marked + memstats.heap_marked*uint64(gcpercent)/100
+ if goal > uint64(mheap_.maxHeap) {
+ // Use maxHeap-based goal.
+ goal = uint64(mheap_.maxHeap)
+
+ // Avoid thrashing by not letting the
+ // effective GOGC drop below 10.
+ //
+ // TODO(austin): This heuristic is pulled from
+ // thin air. It might be better to do
+ // something to more directly force
+ // amortization of GC costs, e.g., by limiting
+ // what fraction of the time GC can be active.
+ var minGOGC uint64 = 10
+ if uint64(gcpercent) < minGOGC {
+ // The user explicitly requested
+ // GOGC < minGOGC. Use that.
+ minGOGC = uint64(gcpercent)
+ }
+ lowerBound := memstats.heap_marked + memstats.heap_marked*minGOGC/100
+ if goal < lowerBound {
+ goal = lowerBound
+ }
+ }
}
// Set the trigger ratio, capped to reasonable bounds.
@@ -773,13 +815,9 @@
// This can happen if the mutator is allocating very
// quickly or the GC is scanning very slowly.
triggerRatio = 0
- } else if gcpercent >= 0 {
- // Ensure there's always a little margin so that the
- // mutator assist ratio isn't infinity.
- maxTriggerRatio := 0.95 * float64(gcpercent) / 100
- if triggerRatio > maxTriggerRatio {
- triggerRatio = maxTriggerRatio
- }
+ } else if gcpercent >= 0 && triggerRatio > float64(gcpercent)/100 {
+ // Cap trigger ratio at GOGC/100.
+ triggerRatio = float64(gcpercent) / 100
}
memstats.triggerRatio = triggerRatio
@@ -790,6 +828,12 @@
trigger := ^uint64(0)
if gcpercent >= 0 {
trigger = uint64(float64(memstats.heap_marked) * (1 + triggerRatio))
+ // Ensure there's always a little margin so that the
+ // mutator assist ratio isn't infinity.
+ if trigger > goal*95/100 {
+ trigger = goal * 95 / 100
+ }
+
// Don't trigger below the minimum heap size.
minTrigger := heapminimum
if !gosweepdone() {
@@ -862,6 +906,30 @@
atomic.Store64(&mheap_.pagesSweptBasis, pagesSwept)
}
}
+
+ // Notify the debug package of a GC pressure change.
+ if mheap_.gcPressureChange != nil {
+ mheap_.gcPressureChange(gcReadPressureLocked())
+ }
+}
+
+//go:linkname gcReadPressure runtime/debug.gcReadPressure
+func gcReadPressure() (gogc int, maxHeap uintptr, egogc int) {
+ lock(&mheap_.lock)
+ gogc, maxHeap, egogc = gcReadPressureLocked()
+ unlock(&mheap_.lock)
+ return
+}
+
+func gcReadPressureLocked() (gogc int, maxHeap uintptr, egogc int) {
+ goal := memstats.next_gc
+ if goal <= uint64(mheap_.maxHeap) {
+ egogc = int(gcpercent)
+ } else {
+ // Back out the effective GOGC from the goal.
+ egogc = int((goal - memstats.heap_marked) * 100 / memstats.heap_marked)
+ }
+ return int(gcpercent), mheap_.maxHeap, egogc
}
// gcGoalUtilization is the goal CPU utilization for background
diff --git a/src/runtime/mheap.go b/src/runtime/mheap.go
index 60676ab..1d1f805 100644
--- a/src/runtime/mheap.go
+++ b/src/runtime/mheap.go
@@ -154,6 +154,17 @@
specialfinalizeralloc fixalloc // allocator for specialfinalizer*
specialprofilealloc fixalloc // allocator for specialprofile*
speciallock mutex // lock for special record allocators.
+
+ // maxHeap is the GC heap limit.
+ //
+ // This is set by the user with debug.SetMaxHeap. GC will
+ // attempt to keep heap_live under maxHeap, even if it has to
+ // violate GOGC (up to a point).
+ maxHeap uintptr
+
+ // gcPressureChange is called after every gcSetTriggerRatio.
+ // It's provided by package debug. It may be nil.
+ gcPressureChange func(gogc int, maxHeap uintptr, egogc int)
}
var mheap_ mheap
diff --git a/src/runtime/mstats.go b/src/runtime/mstats.go
index 1cb44a1..2c7c185 100644
--- a/src/runtime/mstats.go
+++ b/src/runtime/mstats.go
@@ -344,8 +344,8 @@
//
// The garbage collector's goal is to keep HeapAlloc ≤ NextGC.
// At the end of each GC cycle, the target for the next cycle
- // is computed based on the amount of reachable data and the
- // value of GOGC.
+ // is computed based on the amount of reachable data, the
+ // value of GOGC, and the max heap size (if set).
NextGC uint64
// LastGC is the time the last garbage collection finished, as
To view, visit change 46751. To unsubscribe, visit settings.
Gobot Gobot (Gerrit)
Gobot Gobot posted comments on this change.
Patch set 1:
TryBots beginning. Status page: https://farmer.golang.org/try?commit=773a302d
Gobot Gobot (Gerrit)
Gobot Gobot posted comments on this change.
Patch set 1:TryBot-Result +1
TryBots are happy.
Ian Lance Taylor (Gerrit)
Ian Lance Taylor posted comments on this change.
Patch set 1:
(3 comments)
File src/runtime/debug/garbage.go:
Patch Set #1, Line 190:
// This value is percentage with the same interpretation ass/percentage/a percentage/
heap growth during
// the current garbage collection cycle will not exceed
// MaxHeapBytes
Does it mean that _heap growth_ will not exceed MaxHeapBytes, or does it mean that _the heap size_ will not exceed MaxHeapBytes.
will send an up-to-date GCPressure value to
// the notify channel
Is there any requirement that this be a buffered channel? It's not obvious to me what a synchronous channel means when the sender is the runtime. The closest comparison I can think of today is os/signal.Notify, but there the sender on the channel is not actually the runtime, and a notification is explicitly dropped if there is no room.
In general, what are the consequences of missing a notification? We could do a different approach, such as a goroutine that sleeps until something changes, then writes to a channel to say that something has changed, then goes back to sleep. The user code would just receive a notification that something has changed, and would call a different runtime function to get the current values. That way the user code is always working with the up to date status.
I bring this up because this API is most useful when the program is under heavy load, which is exactly when the program might be slow to read from the channel informing it that it is under heavy load. We should do our best to make sure that when the program is shedding load it is doing so with the most timely information we can provide.
To view, visit change 46751. To unsubscribe, visit settings.
Austin Clements (Gerrit)
3 comments:
File src/runtime/debug/garbage.go:
Patch Set #1, Line 190:
// This value is percentage with the same interpretation ass/percentage/a percentage/
Done
heap growth during
// the current garbage collection cycle will not exceed
// MaxHeapBytes
Does it mean that _heap growth_ will not exceed MaxHeapBytes, or does it mean that _the heap size_ w […]
Actually I'm not sure what I was trying to say here. I've rewritten the comment.
will send an up-to-date GCPressure value to
// the notify channelIs there any requirement that this be a buffered channel? It's not obvious to me what a synchronous […]
The send is non-blocking, just like os/signal.Notify, and gets dropped on the floor if there's no room.
But you make an interesting point that, under load, it's more stable to send a content-less "there's been a change" to the channel, since those would be safe to drop from a buffered channel.
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Austin Clements (Gerrit)
Austin Clements uploaded patch set #2 to this change.
runtime/debug: add SetMaxHeap API
DO NOT SUBMIT. This is an experiment to get some experience with the
API and figure out if this is even a reasonable primitive. It needs
tests. The implementation is complete, but there are some questions
about the API in the code.
TODO: Tests.
This adds an API to set a soft limit on the heap size. This augments
the existing GOGC-based GC policy by using the lower of the
GOGC-computed GC target and the heap limit.
When the garbage collector is bounded by the heap limit, it can no
longer amortize the cost of garbage collection against the cost of
growing the heap. Hence, callers of this API are required to register
for notifications of when the garbage collector is under pressure and
are strongly encouraged to use this signal to shed load.
Updates #16843.
Change-Id: I016de1d0ee0d7e34d2752c38cae64d0d0cfedb5f
---
M src/runtime/debug/garbage.go
M src/runtime/mgc.go
M src/runtime/mheap.go
M src/runtime/mstats.go
4 files changed, 209 insertions(+), 12 deletions(-)
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
TryBots beginning. Status page: https://farmer.golang.org/try?commit=8f3966e9
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
TryBots are happy.
Patch set 2:TryBot-Result +1
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Rick Hudson (Gerrit)
4 comments:
Patch Set #1, Line 22:
ing the heap. Hencestrongly encouraged, if not expected, to use
// If a max heap is s
The send is non-blocking, just like os/signal. […]
I tend to agree. Another advantage is that a "there's been a change" API is future proof.
Patch Set #1, Line 213:
the heap size under"Whenever the GCPressure changes in response to memory pressure, a notification is sent on channel c. "
We don't want to send notifications as a result of calls to SetGCPercent or SetMaxHeap.
File src/runtime/debug/garbage.go:
Patch Set #2, Line 237:
// specified max heap.MaxHeapBytes will not be changed but the EffectiveGCPercent will go negative. I noticed you used type int for this so perhaps you already considered this.
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Austin Clements (Gerrit)
I've found some bugs that I need to work out related to the minimum trigger. Currently, the trigger isn't allowed to go below 4MB*GOGC/100 or below heap_live+1MB*GOGC/100 (if we're still sweeping). Unfortunately, this means if you set GOGC to something huge expecting to depend only on the max heap setting, the trigger may wind up set the something huge anyway.
Patch set 2:Code-Review -2
3 comments:
Patch Set #1, Line 22:
ing the heap. Hencestrongly encouraged, if not expected, to use
Done
File src/runtime/debug/garbage.go:
Patch Set #1, Line 213:
the heap size under"Whenever the GCPressure changes in response to memory pressure, a notification is sent on channel c […]
I think we *do* want to send a notification if a call to SetGCPercent or SetMaxHeap changes the pressure. Otherwise any practical use is just going to have to manually call its "respond to pressure change" function as soon as it does this. Besides, with the "there's been a change" API, "spurious" notifications shouldn't really be problematic.
File src/runtime/debug/garbage.go:
Patch Set #2, Line 237:
// specified max heap.
MaxHeapBytes will not be changed but the EffectiveGCPercent will go negative. […]
EffectiveGCPercent can't go negative precisely because of this thrashing avoidance. I've rewritten the comment on EffectiveGCPercent to hopefully be more clear.
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Austin Clements (Gerrit)
Austin Clements uploaded patch set #3 to this change.
runtime/debug: add SetMaxHeap API
DO NOT SUBMIT. This is an experiment to get some experience with the
API and figure out if this is even a reasonable primitive. It needs
tests. The implementation is complete, but there are some questions
about the API in the code.
This adds an API to set a soft limit on the heap size. This augments
the existing GOGC-based GC policy by using the lower of the
GOGC-computed GC target and the heap limit.
When the garbage collector is bounded by the heap limit, it can no
longer amortize the cost of garbage collection against the cost of
growing the heap. Hence, callers of this API are required to register
for notifications of when the garbage collector is under pressure and
are strongly encouraged/expected to use this signal to shed load.
Updates #16843.
Change-Id: I016de1d0ee0d7e34d2752c38cae64d0d0cfedb5f
---
M src/runtime/debug/garbage.go
M src/runtime/debug/garbage_test.go
M src/runtime/mgc.go
M src/runtime/mheap.go
M src/runtime/mstats.go
5 files changed, 299 insertions(+), 16 deletions(-)
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
TryBots beginning. Status page: https://farmer.golang.org/try?commit=927b4c66
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
Build is still in progress...
This change failed on windows-386-2008:
See https://storage.googleapis.com/go-build-log/927b4c66/windows-386-2008_7304cf3a.log
Consult https://build.golang.org/ to see whether it's a new failure. Other builds still in progress; subsequent failure notices suppressed until final report.
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
3 of 21 TryBots failed:
Failed on windows-386-2008: https://storage.googleapis.com/go-build-log/927b4c66/windows-386-2008_7304cf3a.log
Failed on linux-amd64-race: https://storage.googleapis.com/go-build-log/927b4c66/linux-amd64-race_7d03fdd2.log
Failed on linux-386: https://storage.googleapis.com/go-build-log/927b4c66/linux-386_16049408.log
Consult https://build.golang.org/ to see whether they are new failures.
Patch set 3:TryBot-Result -1
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
TryBots beginning. Status page: https://farmer.golang.org/try?commit=cca040af
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
Build is still in progress...
This change failed on linux-amd64-race:
See https://storage.googleapis.com/go-build-log/cca040af/linux-amd64-race_33d2dba9.log
Consult https://build.golang.org/ to see whether it's a new failure. Other builds still in progress; subsequent failure notices suppressed until final report.
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
3 of 21 TryBots failed:
Failed on linux-amd64-race: https://storage.googleapis.com/go-build-log/cca040af/linux-amd64-race_33d2dba9.log
Failed on linux-386: https://storage.googleapis.com/go-build-log/cca040af/linux-386_580ecb1c.log
Failed on windows-386-2008: https://storage.googleapis.com/go-build-log/cca040af/windows-386-2008_1958ba4a.log
Consult https://build.golang.org/ to see whether they are new failures.
Patch set 4:TryBot-Result -1
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Austin Clements (Gerrit)
Austin Clements uploaded patch set #5 to this change.
runtime/debug: add SetMaxHeap API
DO NOT SUBMIT. This is an experiment to get some experience with the
API and figure out if this is even a reasonable primitive. It needs
tests. The implementation is complete, but there are some questions
about the API in the code.
This adds an API to set a soft limit on the heap size. This augments
the existing GOGC-based GC policy by using the lower of the
GOGC-computed GC target and the heap limit.
When the garbage collector is bounded by the heap limit, it can no
longer amortize the cost of garbage collection against the cost of
growing the heap. Hence, callers of this API are required to register
for notifications of when the garbage collector is under pressure and
are strongly encouraged/expected to use this signal to shed load.
Updates #16843.
Change-Id: I016de1d0ee0d7e34d2752c38cae64d0d0cfedb5f
---
M src/runtime/debug/garbage.go
M src/runtime/debug/garbage_test.go
M src/runtime/mgc.go
M src/runtime/mheap.go
M src/runtime/mstats.go
5 files changed, 299 insertions(+), 16 deletions(-)
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
TryBots beginning. Status page: https://farmer.golang.org/try?commit=265f490c
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
Build is still in progress...
This change failed on linux-amd64-race:
See https://storage.googleapis.com/go-build-log/265f490c/linux-amd64-race_597cc7c3.log
Consult https://build.golang.org/ to see whether it's a new failure. Other builds still in progress; subsequent failure notices suppressed until final report.
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
1 of 21 TryBots failed:
Failed on linux-amd64-race: https://storage.googleapis.com/go-build-log/265f490c/linux-amd64-race_597cc7c3.log
Consult https://build.golang.org/ to see whether they are new failures.
Patch set 5:TryBot-Result -1
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Austin Clements (Gerrit)
Patch Set 5: TryBot-Result-1
1 of 21 TryBots failed:
Failed on linux-amd64-race: https://storage.googleapis.com/go-build-log/265f490c/linux-amd64-race_597cc7c3.log
Real. (Or real-ish. It is protected by runtime locking, but the race detector can't see that.)
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Austin Clements (Gerrit)
Austin Clements uploaded patch set #6 to this change.
runtime/debug: add SetMaxHeap API
DO NOT SUBMIT. This is an experiment to get some experience with the
API and figure out if this is even a reasonable primitive. It needs
tests. The implementation is complete, but there are some questions
about the API in the code.
This adds an API to set a soft limit on the heap size. This augments
the existing GOGC-based GC policy by using the lower of the
GOGC-computed GC target and the heap limit.
When the garbage collector is bounded by the heap limit, it can no
longer amortize the cost of garbage collection against the cost of
growing the heap. Hence, callers of this API are required to register
for notifications of when the garbage collector is under pressure and
are strongly encouraged/expected to use this signal to shed load.
Updates #16843.
Change-Id: I016de1d0ee0d7e34d2752c38cae64d0d0cfedb5f
---
M src/runtime/debug/garbage.go
M src/runtime/debug/garbage_test.go
M src/runtime/mgc.go
M src/runtime/mheap.go
M src/runtime/mstats.go
5 files changed, 307 insertions(+), 16 deletions(-)
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
TryBots beginning. Status page: https://farmer.golang.org/try?commit=da956b2e
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
TryBots are happy.
Patch set 6:TryBot-Result +1
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Austin Clements (Gerrit)
Adding Damien for API review.
Patch set 6:-Code-Review
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Austin Clements (Gerrit)
Austin Clements uploaded patch set #7 to this change.
runtime/debug: add SetMaxHeap API
DO NOT SUBMIT. This is an experiment to get some experience with the
API and figure out if this is even a reasonable primitive.
This adds an API to set a soft limit on the heap size. This augments
the existing GOGC-based GC policy by using the lower of the
GOGC-computed GC target and the heap limit.
When the garbage collector is bounded by the heap limit, it can no
longer amortize the cost of garbage collection against the cost of
growing the heap. Hence, callers of this API are required to register
for notifications of when the garbage collector is under pressure and
are strongly encouraged/expected to use this signal to shed load.
Updates #16843.
Change-Id: I016de1d0ee0d7e34d2752c38cae64d0d0cfedb5f
---
M src/runtime/debug/garbage.go
M src/runtime/debug/garbage_test.go
M src/runtime/mgc.go
M src/runtime/mheap.go
M src/runtime/mstats.go
5 files changed, 307 insertions(+), 16 deletions(-)
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Rick Hudson (Gerrit)
5 comments:
File src/runtime/debug/garbage.go:
Patch Set #7, Line 204:
If AvailGCPercent >= GCPercent,I don't appreciated the situation where AvailGCPercent is > GCPercent.
Patch Set #7, Line 237:
reachesapproaches??
// If bytes is 0, SetMaxHeap returns the current heap limit without
// changing it and ignores notify.Can't you do this with ReadGCPolicy, if not let's beef up ReadGCPolicy. One way to do one thing is better.
File src/runtime/debug/garbage_test.go:
Patch Set #7, Line 202:
SetMaxHeap(limit, notify)should return value be noted and dealt with. See the reason not to return a value in previous comment.
This heuristic is pulled from
// thin air. It might be better to do
// something to more directly force
// amortization of GC costs, e.g., by limiting
// what fraction of the time GC can be active.I agree with this. It will give us something to do in the next CL.
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Damien Neil (Gerrit)
API LGTM other than comment on SetMaxHeap.
Patch set 7:Code-Review +1
1 comment:
// If bytes is 0, SetMaxHeap returns the current heap limit without
// changing it and ignores notify.
Can't you do this with ReadGCPolicy, if not let's beef up ReadGCPolicy. […]
+1
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Austin Clements (Gerrit)
Austin Clements uploaded patch set #8 to this change.
runtime/debug: add SetMaxHeap API
DO NOT SUBMIT. This is an experiment to get some experience with the
API and figure out if this is even a reasonable primitive.
This adds an API to set a soft limit on the heap size. This augments
the existing GOGC-based GC policy by using the lower of the
GOGC-computed GC target and the heap limit.
When the garbage collector is bounded by the heap limit, it can no
longer amortize the cost of garbage collection against the cost of
growing the heap. Hence, callers of this API are required to register
for notifications of when the garbage collector is under pressure and
are strongly encouraged/expected to use this signal to shed load.
Updates #16843.
Change-Id: I016de1d0ee0d7e34d2752c38cae64d0d0cfedb5f
---
M src/runtime/debug/garbage.go
M src/runtime/debug/garbage_test.go
M src/runtime/mgc.go
M src/runtime/mheap.go
M src/runtime/mstats.go
5 files changed, 309 insertions(+), 16 deletions(-)
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
TryBots beginning. Status page: https://farmer.golang.org/try?commit=f7979471
Gobot Gobot (Gerrit)
TryBots are happy.
Patch set 8:TryBot-Result +1
Austin Clements (Gerrit)
5 comments:
File src/runtime/debug/garbage.go:
Patch Set #7, Line 204:
If AvailGCPercent >= GCPercent,
I don't appreciated the situation where AvailGCPercent is > GCPercent.
Partly I'm future-proofing this. Right now it can't be, but you could imagine that, for example, if we're operating on a very small heap, AvailGCPercent could be higher than GCPercent to indicate that there's additional head-room created by this lower bound.
Patch Set #7, Line 237:
reachesapproaches??
I really do mean "reaches". If the heap is stays under the heap size limit (even if it's approaching it), we can still use the GOGC-based goal and amortize the cost.
Technically, what I mean here is "when the GOGC-based goal exceeds the heap size limit", but I haven't used the term "goal" anywhere else. I'm open to better wording, but I don't think "approaches" is right.
nc SetMaxHeap(bytes uintptr, notify chan<- struct{}) uintptr {
if bytes == ^uintptr(0) {
return gcSetMaxHeap(bytes, nil)+1
Ah, good call. I think this is left over from an earlier API design. Removed this special case. I still think it should return the previous setting in general for consistency with several other APIs in this package.
File src/runtime/debug/garbage_test.go:
Patch Set #7, Line 202:
prev := SetMaxHeap(limit,should return value be noted and dealt with. […]
Modified the test to check the return value.
of the time GC can be active.
var minGOGC uint64 = 10
if gcpercent >= 0 && uint64(gcpercent) < minGOGC {
// The user explicitly requested
// GOGC < minGOGC. Use that.
I agree with this. It will give us something to do in the next CL.
Got to keep ourselves busy somehow!
One question is how these sort of utilization limits would be reflected in GCPolicy, if at all. They probably can't meaningfully affect AvailGCPercent. But should there be a way to read them? To set them?
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Rick Hudson (Gerrit)
Patch set 8:Code-Review +2
4 comments:
Patch Set #7, Line 204:
If AvailGCPercent >= GCPercent,
Partly I'm future-proofing this. […]
Ack
Patch Set #7, Line 237:
reaches
I really do mean "reaches". […]
Ack
nc SetMaxHeap(bytes uintptr, notify chan<- struct{}) uintptr {
if bytes == ^uintptr(0) {
return gcSetMaxHeap(bytes, nil)
Ah, good call. I think this is left over from an earlier API design. Removed this special case. […]
Ack
File src/runtime/debug/garbage_test.go:
Patch Set #7, Line 202:
prev := SetMaxHeap(limit,
Modified the test to check the return value.
Ack
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Ian Lance Taylor (Gerrit)
2 comments:
File src/runtime/debug/garbage.go:
Patch Set #8, Line 189:
// AvailGCPercent is the heap space available for allocationHow confident are we that AvailGCPercent will remain meaningful if we significantly change the GC algorithm?
Patch Set #8, Line 227:
// Whenever the garbage collector's scheduling policy changes as aIt appears that there is only notify channel: if SetMaxHeap is called again, only that notify channel will be used. That is fine but needs to be documented.
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Austin Clements (Gerrit)
2 comments:
Patch Set #8, Line 189:
// AvailGCPercent is the heap space available for allocation
How confident are we that AvailGCPercent will remain meaningful if we significantly change the GC al […]
That's a good question. It's obviously hard to say, but AvailGCPercent is well-defined as long as there's a "last heap live" and a "goal heap" size. These are meaningful in a generational GC (though may not measure as well how hard the garbage collector is working), since we update the live heap size on each full GC. Similarly, I'm pretty sure this is meaningful if we start eagerly freeing heap objects. I think you could even say they're meaningful with manual memory management, though there your live heap would theoretically update on every malloc/free, and the goal would probably only be finite if a max heap size is set.
Patch Set #8, Line 227:
// Whenever the garbage collector's scheduling policy changes as a
It appears that there is only notify channel: if SetMaxHeap is called again, only that notify channe […]
Yes, thanks for catching that.
Though I'm still not sure that's actually the right API choice.
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Austin Clements (Gerrit)
Austin Clements uploaded patch set #9 to this change.
runtime/debug: add SetMaxHeap API
DO NOT SUBMIT. This is an experiment to get some experience with the
API and figure out if this is even a reasonable primitive.
This adds an API to set a soft limit on the heap size. This augments
the existing GOGC-based GC policy by using the lower of the
GOGC-computed GC target and the heap limit.
When the garbage collector is bounded by the heap limit, it can no
longer amortize the cost of garbage collection against the cost of
growing the heap. Hence, callers of this API are required to register
for notifications of when the garbage collector is under pressure and
are strongly encouraged/expected to use this signal to shed load.
Updates #16843.
Change-Id: I016de1d0ee0d7e34d2752c38cae64d0d0cfedb5f
---
M src/runtime/debug/garbage.go
M src/runtime/debug/garbage_test.go
M src/runtime/mgc.go
M src/runtime/mheap.go
M src/runtime/mstats.go
5 files changed, 311 insertions(+), 16 deletions(-)
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
TryBots beginning. Status page: https://farmer.golang.org/try?commit=0021eed3
Gobot Gobot (Gerrit)
TryBots are happy.
Patch set 9:TryBot-Result +1
Austin Clements (Gerrit)
Austin Clements uploaded patch set #10 to this change.
runtime/debug: add SetMaxHeap API
DO NOT SUBMIT. This is an experiment to get some experience with the
API and figure out if this is even a reasonable primitive.
This adds an API to set a soft limit on the heap size. This augments
the existing GOGC-based GC policy by using the lower of the
GOGC-computed GC target and the heap limit.
When the garbage collector is bounded by the heap limit, it can no
longer amortize the cost of garbage collection against the cost of
growing the heap. Hence, callers of this API are required to register
for notifications of when the garbage collector is under pressure and
are strongly encouraged/expected to use this signal to shed load.
Updates #16843.
Change-Id: I016de1d0ee0d7e34d2752c38cae64d0d0cfedb5f
---
M src/runtime/debug/garbage.go
M src/runtime/debug/garbage_test.go
M src/runtime/mgc.go
M src/runtime/mheap.go
M src/runtime/mstats.go
5 files changed, 311 insertions(+), 16 deletions(-)
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
TryBots beginning. Status page: https://farmer.golang.org/try?commit=a3660517
Gobot Gobot (Gerrit)
TryBots are happy.
Patch set 10:TryBot-Result +1
Austin Clements (Gerrit)
Austin Clements uploaded patch set #11 to this change.
runtime/debug: add SetMaxHeap API
DO NOT SUBMIT. This is an experiment to get some experience with the
API and figure out if this is even a reasonable primitive.
This adds an API to set a soft limit on the heap size. This augments
the existing GOGC-based GC policy by using the lower of the
GOGC-computed GC target and the heap limit.
When the garbage collector is bounded by the heap limit, it can no
longer amortize the cost of garbage collection against the cost of
growing the heap. Hence, callers of this API are required to register
for notifications of when the garbage collector is under pressure and
are strongly encouraged/expected to use this signal to shed load.
Updates #16843.
Change-Id: I016de1d0ee0d7e34d2752c38cae64d0d0cfedb5f
---
M src/runtime/debug/garbage.go
M src/runtime/debug/garbage_test.go
M src/runtime/mgc.go
M src/runtime/mheap.go
M src/runtime/mstats.go
5 files changed, 356 insertions(+), 16 deletions(-)
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
TryBots beginning. Status page: https://farmer.golang.org/try?commit=8b016466
Gobot Gobot (Gerrit)
Build is still in progress...
This change failed on linux-386:
See https://storage.googleapis.com/go-build-log/8b016466/linux-386_3007e844.log
Consult https://build.golang.org/ to see whether it's a new failure. Other builds still in progress; subsequent failure notices suppressed until final report.
Gobot Gobot (Gerrit)
4 of 17 TryBots failed:
Failed on linux-386: https://storage.googleapis.com/go-build-log/8b016466/linux-386_3007e844.log
Failed on nacl-amd64p32: https://storage.googleapis.com/go-build-log/8b016466/nacl-amd64p32_8bc59802.log
Failed on nacl-386: https://storage.googleapis.com/go-build-log/8b016466/nacl-386_342e144d.log
Failed on windows-386-2008: https://storage.googleapis.com/go-build-log/8b016466/windows-386-2008_8a88716f.log
Consult https://build.golang.org/ to see whether they are new failures.
Patch set 11:TryBot-Result -1
Austin Clements (Gerrit)
Austin Clements uploaded patch set #12 to this change.
runtime/debug: add SetMaxHeap API
DO NOT SUBMIT. This is an experiment to get some experience with the
API and figure out if this is even a reasonable primitive.
This adds an API to set a soft limit on the heap size. This augments
the existing GOGC-based GC policy by using the lower of the
GOGC-computed GC target and the heap limit.
When the garbage collector is bounded by the heap limit, it can no
longer amortize the cost of garbage collection against the cost of
growing the heap. Hence, callers of this API are required to register
for notifications of when the garbage collector is under pressure and
are strongly encouraged/expected to use this signal to shed load.
Updates #16843.
Change-Id: I016de1d0ee0d7e34d2752c38cae64d0d0cfedb5f
---
M src/runtime/debug/garbage.go
M src/runtime/debug/garbage_test.go
M src/runtime/mgc.go
M src/runtime/mheap.go
M src/runtime/mstats.go
5 files changed, 357 insertions(+), 16 deletions(-)
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
TryBots beginning. Status page: https://farmer.golang.org/try?commit=37fc66e6
Gobot Gobot (Gerrit)
TryBots are happy.
Patch set 12:TryBot-Result +1
Joe Tsai (Gerrit)
Will this CL be submitted for Go1.11?
Gobot Gobot (Gerrit)
TryBots beginning. Status page: https://farmer.golang.org/try?commit=d2cd4303
Gobot Gobot (Gerrit)
TryBots are happy.
Patch set 13:TryBot-Result +1
Austin Clements (Gerrit)
Austin Clements uploaded patch set #14 to this change.
runtime/debug: add SetMaxHeap API
DO NOT SUBMIT. This is an experiment to get some experience with the
API and figure out if this is even a reasonable primitive.
This adds an API to set a soft limit on the heap size. This augments
the existing GOGC-based GC policy by using the lower of the
GOGC-computed GC target and the heap limit.
When the garbage collector is bounded by the heap limit, it can no
longer amortize the cost of garbage collection against the cost of
growing the heap. Hence, callers of this API are required to register
for notifications of when the garbage collector is under pressure and
are strongly encouraged/expected to use this signal to shed load.
This CL incorporates fixes from CL 151540 and CL 156917 by
mkny...@google.com.
Updates #16843.
Change-Id: I016de1d0ee0d7e34d2752c38cae64d0d0cfedb5f
---
M src/runtime/debug/garbage.go
M src/runtime/debug/garbage_test.go
M src/runtime/mgc.go
M src/runtime/mstats.go
4 files changed, 404 insertions(+), 19 deletions(-)
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
TryBots beginning. Status page: https://farmer.golang.org/try?commit=315fdaf0
Gobot Gobot (Gerrit)
TryBots are happy.
Patch set 14:TryBot-Result +1
Gobot Gobot (Gerrit)
TryBots beginning. Status page: https://farmer.golang.org/try?commit=9ae75d77
Gobot Gobot (Gerrit)
TryBots are happy.
Patch set 15:TryBot-Result +1
Austin Clements (Gerrit)
Austin Clements uploaded patch set #16 to this change.
runtime/debug: add SetMaxHeap API
DO NOT SUBMIT. This is an experiment to get some experience with the
API and figure out if this is even a reasonable primitive.
This adds an API to set a soft limit on the heap size. This augments
the existing GOGC-based GC policy by using the lower of the
GOGC-computed GC target and the heap limit.
When the garbage collector is bounded by the heap limit, it can no
longer amortize the cost of garbage collection against the cost of
growing the heap. Hence, callers of this API are required to register
for notifications of when the garbage collector is under pressure and
are strongly encouraged/expected to use this signal to shed load.
This CL incorporates fixes from CL 151540 and CL 156917 by
mkny...@google.com.
Updates #16843.
Change-Id: I016de1d0ee0d7e34d2752c38cae64d0d0cfedb5f
---
M src/runtime/debug/garbage.go
M src/runtime/debug/garbage_test.go
M src/runtime/mgc.go
M src/runtime/mstats.go
4 files changed, 405 insertions(+), 19 deletions(-)
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Austin Clements (Gerrit)
PS16 rebases to current master (approaching Go 1.13 rc1)
Gobot Gobot (Gerrit)
TryBots beginning. Status page: https://farmer.golang.org/try?commit=544f5c5c
Gobot Gobot (Gerrit)
TryBots are happy.
Patch set 16:TryBot-Result +1
Austin Clements (Gerrit)
Hmm. I tried merging in CL 183317, but all.bash is failing, so I have to debug that.
Austin Clements (Gerrit)
Austin Clements uploaded patch set #17 to this change.
runtime/debug: add SetMaxHeap API
DO NOT SUBMIT. This is an experiment to get some experience with the
API and figure out if this is even a reasonable primitive.
This adds an API to set a soft limit on the heap size. This augments
the existing GOGC-based GC policy by using the lower of the
GOGC-computed GC target and the heap limit.
When the garbage collector is bounded by the heap limit, it can no
longer amortize the cost of garbage collection against the cost of
growing the heap. Hence, callers of this API are required to register
for notifications of when the garbage collector is under pressure and
are strongly encouraged/expected to use this signal to shed load.
This CL incorporates fixes from CL 151540, CL 156917, and CL 183317 by
mkny...@google.com.
Updates #16843.
Change-Id: I016de1d0ee0d7e34d2752c38cae64d0d0cfedb5f
---
M src/runtime/debug/garbage.go
M src/runtime/debug/garbage_test.go
M src/runtime/mgc.go
M src/runtime/mstats.go
4 files changed, 420 insertions(+), 19 deletions(-)
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
TryBots beginning. Status page: https://farmer.golang.org/try?commit=0d7c0325
Austin Clements (Gerrit)
PS17 incorporates Michael's fixes from CL 183317 (plus a minor fix to his fixes)
Gobot Gobot (Gerrit)
1 of 21 TryBots failed:
Failed on windows-amd64-2016: https://storage.googleapis.com/go-build-log/0d7c0325/windows-amd64-2016_ba2c4b74.log
Consult https://build.golang.org/ to see whether they are new failures. Keep in mind that TryBots currently test *exactly* your git commit, without rebasing. If your commit's git parent is old, the failure might've already been fixed.
Patch set 17:TryBot-Result -1
Austin Clements (Gerrit)
Patch Set 17: TryBot-Result-1
1 of 21 TryBots failed:
Failed on windows-amd64-2016: https://storage.googleapis.com/go-build-log/0d7c0325/windows-amd64-2016_ba2c4b74.log
I can't reproduce this timeout locally on linux/amd64. I'm going to try re-running the trybots.
Austin Clements (Gerrit)
Austin Clements removed a vote from this change.
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
TryBots are happy.
Patch set 17:TryBot-Result +1
Caleb Spare (Gerrit)
FWIW I reviewed the API+documentation and it makes sense to me as a user.
Patch set 17:Code-Review +1
2 comments:
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Austin Clements (Gerrit)
Austin Clements removed a vote from this change.
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
Build is still in progress...
This change failed on openbsd-amd64-64:
See https://storage.googleapis.com/go-build-log/0d7c0325/openbsd-amd64-64_eb9a9dd3.log
Other builds still in progress; subsequent failure notices suppressed until final report. Consult https://build.golang.org/ to see whether they are new failures. Keep in mind that TryBots currently test *exactly* your git commit, without rebasing. If your commit's git parent is old, the failure might've already been fixed.
Gobot Gobot (Gerrit)
1 of 21 TryBots failed:
Failed on openbsd-amd64-64: https://storage.googleapis.com/go-build-log/0d7c0325/openbsd-amd64-64_eb9a9dd3.log
Consult https://build.golang.org/ to see whether they are new failures. Keep in mind that TryBots currently test *exactly* your git commit, without rebasing. If your commit's git parent is old, the failure might've already been fixed.
Patch set 17:TryBot-Result -1
Austin Clements (Gerrit)
Patch Set 17: TryBot-Result-1
1 of 21 TryBots failed:
Failed on openbsd-amd64-64: https://storage.googleapis.com/go-build-log/0d7c0325/openbsd-amd64-64_eb9a9dd3.log
I wrote a super silly dynamic lock cycle detector for runtime locks, pointed it at the TestAvailGCPercent, and found that we do both:
- Acquire mheap_.lock in setGCPercent (also gcSetMaxHeap) and then gcPressure.lock in gcSetTriggerRatio.
- Acquire gcPressure.lock on the G stack and then call functions that may grow the stack, which may acquire mheap_.lock.
If both happened at the same time, it could deadlock. I suspect that it's not showing up on linux/amd64 because we happen to not grow the stack while holding gcPressure.lock. In fact, in my local testing, this is exactly what I see. But the stack is a slightly different size at this point on some trybots, so they exhibit the problem.
Austin Clements (Gerrit)
Austin Clements uploaded patch set #18 to this change.
runtime/debug: add SetMaxHeap API
DO NOT SUBMIT. This is an experiment to get some experience with the
API and figure out if this is even a reasonable primitive.
This adds an API to set a soft limit on the heap size. This augments
the existing GOGC-based GC policy by using the lower of the
GOGC-computed GC target and the heap limit.
When the garbage collector is bounded by the heap limit, it can no
longer amortize the cost of garbage collection against the cost of
growing the heap. Hence, callers of this API are required to register
for notifications of when the garbage collector is under pressure and
are strongly encouraged/expected to use this signal to shed load.
This CL incorporates fixes from CL 151540, CL 156917, and CL 183317 by
mkny...@google.com.
Updates #16843.
Change-Id: I016de1d0ee0d7e34d2752c38cae64d0d0cfedb5f
---
M src/runtime/debug/garbage.go
M src/runtime/debug/garbage_test.go
M src/runtime/mgc.go
M src/runtime/mstats.go
4 files changed, 438 insertions(+), 19 deletions(-)
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
TryBots beginning. Status page: https://farmer.golang.org/try?commit=24c576be
Gobot Gobot (Gerrit)
Build is still in progress...
This change failed on linux-amd64-race:
See https://storage.googleapis.com/go-build-log/24c576be/linux-amd64-race_2a8e896f.log
Other builds still in progress; subsequent failure notices suppressed until final report. Consult https://build.golang.org/ to see whether they are new failures. Keep in mind that TryBots currently test *exactly* your git commit, without rebasing. If your commit's git parent is old, the failure might've already been fixed.
Gobot Gobot (Gerrit)
1 of 21 TryBots failed:
Failed on linux-amd64-race: https://storage.googleapis.com/go-build-log/24c576be/linux-amd64-race_2a8e896f.log
Consult https://build.golang.org/ to see whether they are new failures. Keep in mind that TryBots currently test *exactly* your git commit, without rebasing. If your commit's git parent is old, the failure might've already been fixed.
Patch set 18:TryBot-Result -1
Austin Clements (Gerrit)
PS18 should fix the locking discipline around mheap_.lock and gcPressure.lock to prevent deadlocks.
Austin Clements (Gerrit)
Patch Set 18: TryBot-Result-1
1 of 21 TryBots failed:
Failed on linux-amd64-race: https://storage.googleapis.com/go-build-log/24c576be/linux-amd64-race_2a8e896f.log
Sigh. I guess you can't raceacquire on the system stack, which is unfortunate because we can't lock gcPressure.lock *not* on the system stack.
Austin Clements (Gerrit)
Austin Clements uploaded patch set #19 to this change.
runtime/debug: add SetMaxHeap API
DO NOT SUBMIT. This is an experiment to get some experience with the
API and figure out if this is even a reasonable primitive.
This adds an API to set a soft limit on the heap size. This augments
the existing GOGC-based GC policy by using the lower of the
GOGC-computed GC target and the heap limit.
When the garbage collector is bounded by the heap limit, it can no
longer amortize the cost of garbage collection against the cost of
growing the heap. Hence, callers of this API are required to register
for notifications of when the garbage collector is under pressure and
are strongly encouraged/expected to use this signal to shed load.
This CL incorporates fixes from CL 151540, CL 156917, and CL 183317 by
mkny...@google.com.
Updates #16843.
Change-Id: I016de1d0ee0d7e34d2752c38cae64d0d0cfedb5f
---
M src/runtime/debug/garbage.go
M src/runtime/debug/garbage_test.go
M src/runtime/mgc.go
M src/runtime/mstats.go
4 files changed, 439 insertions(+), 19 deletions(-)
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
TryBots beginning. Status page: https://farmer.golang.org/try?commit=47a79ba3
Gobot Gobot (Gerrit)
TryBots are happy.
Patch set 19:TryBot-Result +1
Austin Clements (Gerrit)
That's promising. Going to do one more run of the trybots.
Austin Clements (Gerrit)
Austin Clements removed a vote from this change.
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
wangde...@gmail.com
Austin Clements (Gerrit)
Austin Clements removed a vote from this change.
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Austin Clements (Gerrit)
Austin Clements uploaded patch set #20 to this change.
runtime/debug: add SetMaxHeap API
DO NOT SUBMIT. This is an experiment to get some experience with the
API and figure out if this is even a reasonable primitive.
This adds an API to set a soft limit on the heap size. This augments
the existing GOGC-based GC policy by using the lower of the
GOGC-computed GC target and the heap limit.
When the garbage collector is bounded by the heap limit, it can no
longer amortize the cost of garbage collection against the cost of
growing the heap. Hence, callers of this API are required to register
for notifications of when the garbage collector is under pressure and
are strongly encouraged/expected to use this signal to shed load.
This CL incorporates fixes from CL 151540, CL 156917, and CL 183317 by
mkny...@google.com.
Updates #16843.
Change-Id: I016de1d0ee0d7e34d2752c38cae64d0d0cfedb5f
---
M src/runtime/debug/garbage.go
M src/runtime/debug/garbage_test.go
M src/runtime/mgc.go
M src/runtime/mstats.go
4 files changed, 453 insertions(+), 38 deletions(-)
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
TryBots beginning. Status page: https://farmer.golang.org/try?commit=e26186e1
Gobot Gobot (Gerrit)
Build is still in progress...
This change failed on (x/tools) linux-amd64:
See https://storage.googleapis.com/go-build-log/e26186e1/linux-amd64_6adebbdf.log
Other builds still in progress; subsequent failure notices suppressed until final report. Consult https://build.golang.org/ to see whether they are new failures. Keep in mind that TryBots currently test *exactly* your git commit, without rebasing. If your commit's git parent is old, the failure might've already been fixed.
Austin Clements (Gerrit)
PS20 rebases on top of current master (mid-Go 1.15 dev). There was a conflict with CL 225637, which introduced the lower bound on the trigger ratio. It's not clear how to do the exact same logic, since that bound depends on gcpercent. This resolves it by applying an equivalent lower bound directly to the computed trigger (in bytes).
Gobot Gobot (Gerrit)
20 of 20 TryBots failed:
Failed on (x/tools) linux-amd64: https://storage.googleapis.com/go-build-log/e26186e1/linux-amd64_6adebbdf.log
Failed on linux-amd64: https://storage.googleapis.com/go-build-log/e26186e1/linux-amd64_60978dc7.log
Failed on js-wasm: https://storage.googleapis.com/go-build-log/e26186e1/js-wasm_35ebdd2e.log
Failed on freebsd-amd64-12_0: https://storage.googleapis.com/go-build-log/e26186e1/freebsd-amd64-12_0_ad8f3361.log
Failed on misc-compile-linuxarm: https://storage.googleapis.com/go-build-log/e26186e1/misc-compile-linuxarm_106579c4.log
Failed on misc-compile-other: https://storage.googleapis.com/go-build-log/e26186e1/misc-compile-other_b4a2b478.log
Failed on misc-compile-netbsd: https://storage.googleapis.com/go-build-log/e26186e1/misc-compile-netbsd_59fb9524.log
Failed on misc-compile-solaris: https://storage.googleapis.com/go-build-log/e26186e1/misc-compile-solaris_ad96e09a.log
Failed on misc-compile-ppc: https://storage.googleapis.com/go-build-log/e26186e1/misc-compile-ppc_00cdd4ce.log
Failed on misc-compile-mips: https://storage.googleapis.com/go-build-log/e26186e1/misc-compile-mips_650a109e.log
Failed on misc-compile-plan9: https://storage.googleapis.com/go-build-log/e26186e1/misc-compile-plan9_9b3b9698.log
Failed on misc-compile-openbsd: https://storage.googleapis.com/go-build-log/e26186e1/misc-compile-openbsd_c51f6d71.log
Failed on openbsd-amd64-64: https://storage.googleapis.com/go-build-log/e26186e1/openbsd-amd64-64_891d5baa.log
Failed on windows-amd64-2016: https://storage.googleapis.com/go-build-log/e26186e1/windows-amd64-2016_7ba2fd1b.log
Failed on misc-compile-freebsd: https://storage.googleapis.com/go-build-log/e26186e1/misc-compile-freebsd_3cbd3cfa.log
Failed on misc-compile-darwin: https://storage.googleapis.com/go-build-log/e26186e1/misc-compile-darwin_5b8d5107.log
Failed on linux-amd64-race: https://storage.googleapis.com/go-build-log/e26186e1/linux-amd64-race_947a9e44.log
Failed on linux-386: https://storage.googleapis.com/go-build-log/e26186e1/linux-386_226e5823.log
Failed on android-amd64-emu: https://storage.googleapis.com/go-build-log/e26186e1/android-amd64-emu_87844744.log
Failed on windows-386-2008: https://storage.googleapis.com/go-build-log/e26186e1/windows-386-2008_03a571e1.log
Consult https://build.golang.org/ to see whether they are new failures. Keep in mind that TryBots currently test *exactly* your git commit, without rebasing. If your commit's git parent is old, the failure might've already been fixed.
Patch set 20:TryBot-Result -1
Austin Clements (Gerrit)
Sorry, rebase fail.
Austin Clements (Gerrit)
Austin Clements uploaded patch set #21 to this change.
runtime/debug: add SetMaxHeap API
DO NOT SUBMIT. This is an experiment to get some experience with the
API and figure out if this is even a reasonable primitive.
This adds an API to set a soft limit on the heap size. This augments
the existing GOGC-based GC policy by using the lower of the
GOGC-computed GC target and the heap limit.
When the garbage collector is bounded by the heap limit, it can no
longer amortize the cost of garbage collection against the cost of
growing the heap. Hence, callers of this API are required to register
for notifications of when the garbage collector is under pressure and
are strongly encouraged/expected to use this signal to shed load.
This CL incorporates fixes from CL 151540, CL 156917, and CL 183317 by
mkny...@google.com.
Updates #16843.
Change-Id: I016de1d0ee0d7e34d2752c38cae64d0d0cfedb5f
---
M src/runtime/debug/garbage.go
M src/runtime/debug/garbage_test.go
M src/runtime/mgc.go
M src/runtime/mstats.go
4 files changed, 453 insertions(+), 38 deletions(-)
To view, visit change 46751. To unsubscribe, or for help writing mail filters, visit settings.
Gobot Gobot (Gerrit)
TryBots beginning. Status page: https://farmer.golang.org/try?commit=83b1ff0e
Gobot Gobot (Gerrit)
Build is still in progress...
This change failed on freebsd-amd64-12_0:
See https://storage.googleapis.com/go-build-log/83b1ff0e/freebsd-amd64-12_0_cc301642.log
Other builds still in progress; subsequent failure notices suppressed until final report. Consult https://build.golang.org/ to see whether they are new failures. Keep in mind that TryBots currently test *exactly* your git commit, without rebasing. If your commit's git parent is old, the failure might've already been fixed.
Gobot Gobot (Gerrit)
8 of 20 TryBots failed:
Failed on freebsd-amd64-12_0: https://storage.googleapis.com/go-build-log/83b1ff0e/freebsd-amd64-12_0_cc301642.log
Failed on linux-amd64: https://storage.googleapis.com/go-build-log/83b1ff0e/linux-amd64_8e902f9e.log
Failed on windows-amd64-2016: https://storage.googleapis.com/go-build-log/83b1ff0e/windows-amd64-2016_1b25ab1a.log
Failed on linux-386: https://storage.googleapis.com/go-build-log/83b1ff0e/linux-386_f9b21da8.log
Failed on android-amd64-emu: https://storage.googleapis.com/go-build-log/83b1ff0e/android-amd64-emu_5efe3d85.log
Failed on openbsd-amd64-64: https://storage.googleapis.com/go-build-log/83b1ff0e/openbsd-amd64-64_d3fa41f5.log
Failed on windows-386-2008: https://storage.googleapis.com/go-build-log/83b1ff0e/windows-386-2008_8e2f0b9e.log
Failed on linux-amd64-race: https://storage.googleapis.com/go-build-log/83b1ff0e/linux-amd64-race_bfd75385.log
Consult https://build.golang.org/ to see whether they are new failures. Keep in mind that TryBots currently test *exactly* your git commit, without rebasing. If your commit's git parent is old, the failure might've already been fixed.
Patch set 21:TryBot-Result -1
Austin Clements (Gerrit)
Patch Set 21: TryBot-Result-1
8 of 20 TryBots failed:
I don't think this is related to the new min trigger ratio code, which would have been the obvious culprit since that's where the merge conflict was.
If I revert to PS19, "go test -short -run SetMaxHeap runtime/debug" reliably passes on my linux/amd64 laptop.
If I cherry-pick PS19 and just delete the min trigger ratio code (resolving the conflict to the code that was in PS19), "go test -short -run SetMaxHeap runtime/debug" reliably fails in the exact ways the trybots are failing.
David Chase (Gerrit)
This doesn't seem to include Michael's changes to guard against weird trigger ratios.
https://github.com/golang/go/commit/74af7fc603
Is that intended?