[go] cmd/internal/pgo: accept CPU profiles from external tools like Linux perf
Gerrit Bot (Gerrit)
Gerrit Bot has uploaded the change for review![]( "Open in Gerrit")
Commit message
cmd/internal/pgo: accept CPU profiles from external tools like Linux perf
PGO consumes a CPU pprof profile, but FromPProf only accepted the value types emitted by Go's own runtime/pprof ("samples"/"count" and "cpu"/"nanoseconds") and rejected everything else with a hard error. This blocked CPU profiles produced outside the Go runtime, most notably Linux perf profiles converted with perf_data_converter (perf_to_profile), which name their sample values after the profiled event (e.g. "cycles_event"/ "count") rather than using Go's names.
Such profiles are still CPU profiles: PGO only reads the selected value as an edge weight and thresholds it as a fraction of the total, so the value's name and unit do not affect the result beyond selecting the column. When no Go-native CPU value type is present, select the column via the profile's default sample type (falling back to the last value type when unset, per the pprof format), while still rejecting value types from known non-CPU profiles (heap, mutex, block) so that passing e.g. a heap profile to -pgo still fails clearly.
The other blocker noted on the issue, missing Function.start_line in symbolized perf profiles, was fixed separately in google/pprof and is already vendored into the toolchain, so no change is needed here for it.
Fixes #64489
Change diff
diff --git a/src/cmd/internal/pgo/pprof.go b/src/cmd/internal/pgo/pprof.go
index 5bcd8c5..1a51ffd 100644
--- a/src/cmd/internal/pgo/pprof.go
+++ b/src/cmd/internal/pgo/pprof.go
@@ -12,9 +12,19 @@
"fmt"
"internal/profile"
"io"
+ "slices"
"sort"
)
+// nonCPUSampleTypes are the sample value types from Go's non-CPU profiles
+// (heap, mutex, block). A profile whose selected value type is one of these is
+// not a CPU profile and is rejected.
+var nonCPUSampleTypes = []string{
+ "alloc_objects", "alloc_space",
+ "inuse_objects", "inuse_space",
+ "contentions", "delay",
+}
+
// FromPProf parses Profile from a pprof profile.
func FromPProf(r io.Reader) (*Profile, error) {
p, err := profile.Parse(r)
@@ -31,19 +41,9 @@
return emptyProfile(), nil
}
- valueIndex := -1
- for i, s := range p.SampleType {
- // Samples count is the raw data collected, and CPU nanoseconds is just
- // a scaled version of it, so either one we can find is fine.
- if (s.Type == "samples" && s.Unit == "count") ||
- (s.Type == "cpu" && s.Unit == "nanoseconds") {
- valueIndex = i
- break
- }
- }
-
- if valueIndex == -1 {
- return nil, fmt.Errorf(`profile does not contain a sample index with value/type "samples/count" or cpu/nanoseconds"`)
+ valueIndex, err := sampleValueIndex(p)
+ if err != nil {
+ return nil, err
}
g := profile.NewGraph(p, &profile.Options{
@@ -71,6 +71,34 @@
}, nil
}
+// sampleValueIndex returns the index of the sample value to use as the PGO
+// weight, or an error if the profile is not a CPU profile.
+func sampleValueIndex(p *profile.Profile) (int, error) {
+ for i, s := range p.SampleType {
+ if (s.Type == "samples" && s.Unit == "count") ||
+ (s.Type == "cpu" && s.Unit == "nanoseconds") {
+ return i, nil
+ }
+ }
+
+ // Not a Go CPU profile. Use the default sample type, or the last value
+ // type if there is no default.
+ index := len(p.SampleType) - 1
+ if d := p.DefaultSampleType; d != "" {
+ for i, s := range p.SampleType {
+ if s.Type == d {
+ index = i
+ break
+ }
+ }
+ }
+
+ if index < 0 || slices.Contains(nonCPUSampleTypes, p.SampleType[index].Type) {
+ return 0, fmt.Errorf(`profile does not contain a sample index with value/type "samples/count" or "cpu/nanoseconds", and does not default to a CPU sample type`)
+ }
+ return index, nil
+}
+
// createNamedEdgeMap builds a map of callsite-callee edge weights from the
// profile-graph.
//
diff --git a/src/cmd/internal/pgo/pprof_test.go b/src/cmd/internal/pgo/pprof_test.go
new file mode 100644
index 0000000..a77d970
--- /dev/null
+++ b/src/cmd/internal/pgo/pprof_test.go
@@ -0,0 +1,107 @@
+// Copyright 2026 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package pgo
+
+import (
+ "bytes"
+ "internal/profile"
+ "testing"
+)
+
+func testProfile(sampleTypes []*profile.ValueType, defaultSampleType string, values []int64) *profile.Profile {
+ main := &profile.Function{ID: 1, Name: "main.main", StartLine: 10}
+ foo := &profile.Function{ID: 2, Name: "main.foo", StartLine: 20}
+ mainLoc := &profile.Location{ID: 1, Line: []profile.Line{{Function: main, Line: 12}}}
+ fooLoc := &profile.Location{ID: 2, Line: []profile.Line{{Function: foo, Line: 22}}}
+ return &profile.Profile{
+ SampleType: sampleTypes,
+ DefaultSampleType: defaultSampleType,
+ Function: []*profile.Function{main, foo},
+ Location: []*profile.Location{mainLoc, fooLoc},
+ Sample: []*profile.Sample{
+ // Stacks are leaf-first: foo called from main.
+ {Location: []*profile.Location{fooLoc, mainLoc}, Value: values},
+ },
+ }
+}
+
+func serializeProfile(t *testing.T, p *profile.Profile) *bytes.Reader {
+ t.Helper()
+ var buf bytes.Buffer
+ if err := p.Write(&buf); err != nil {
+ t.Fatalf("writing profile: %v", err)
+ }
+ return bytes.NewReader(buf.Bytes())
+}
+
+func TestFromPProfSampleType(t *testing.T) {
+ perf := []*profile.ValueType{
+ {Type: "cycles_sample", Unit: "count"},
+ {Type: "cycles_event", Unit: "count"},
+ }
+
+ tests := []struct {
+ name string
+ sampleTypes []*profile.ValueType
+ defaultType string
+ values []int64
+ wantErr bool
+ wantTotalWeight int64
+ }{
+ {
+ name: "native samples/count",
+ sampleTypes: []*profile.ValueType{{Type: "samples", Unit: "count"}, {Type: "cpu", Unit: "nanoseconds"}},
+ values: []int64{5, 500},
+ wantTotalWeight: 5,
+ },
+ {
+ name: "native cpu/nanoseconds",
+ sampleTypes: []*profile.ValueType{{Type: "cpu", Unit: "nanoseconds"}},
+ values: []int64{500},
+ wantTotalWeight: 500,
+ },
+ {
+ name: "perf via default sample type",
+ sampleTypes: perf,
+ defaultType: "cycles_event",
+ values: []int64{5, 500},
+ wantTotalWeight: 500,
+ },
+ {
+ name: "perf without default uses last value type",
+ sampleTypes: perf,
+ values: []int64{5, 500},
+ wantTotalWeight: 500,
+ },
+ {
+ name: "heap profile rejected",
+ sampleTypes: []*profile.ValueType{{Type: "alloc_objects", Unit: "count"}, {Type: "inuse_space", Unit: "bytes"}},
+ defaultType: "inuse_space",
+ values: []int64{1, 2},
+ wantErr: true,
+ },
+ }
+
+ for _, tc := range tests {
+ t.Run(tc.name, func(t *testing.T) {
+ got, err := FromPProf(serializeProfile(t, testProfile(tc.sampleTypes, tc.defaultType, tc.values)))
+ if tc.wantErr {
+ if err == nil {
+ t.Fatal("FromPProf succeeded, want error")
+ }
+ return
+ }
+ if err != nil {
+ t.Fatalf("FromPProf: %v", err)
+ }
+ if len(got.NamedEdgeMap.ByWeight) == 0 {
+ t.Error("empty edge map, want a call edge")
+ }
+ if got.TotalWeight != tc.wantTotalWeight {
+ t.Errorf("TotalWeight = %d, want %d", got.TotalWeight, tc.wantTotalWeight)
+ }
+ })
+ }
+}
Change information
- M src/cmd/internal/pgo/pprof.go
- A src/cmd/internal/pgo/pprof_test.go
Related details
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Gopher Robot (Gerrit)
Gopher Robot added 1 comment![]( "Open in Gerrit")
I spotted some possible problems with your PR:
1. You have a long 539 character line in the commit message body. Please add line breaks to long lines that should be wrapped. Lines in the commit message body should be wrapped at ~76 characters unless needed for things like URLs or tables. (Note: GitHub might render long lines as soft-wrapped, so double-check in the Gerrit commit message shown above.)
Please address any problems by updating the GitHub PR.
When complete, mark this comment as 'Done' and click the [blue 'Reply' button](https://go.dev/wiki/GerritBot#i-left-a-reply-to-a-comment-in-gerrit-but-no-one-but-me-can-see-it) above. These findings are based on heuristics; if a finding does not apply, briefly reply here saying so.
To update the commit title or commit message body shown here in Gerrit, you must edit the GitHub PR title and PR description (the first comment) in the GitHub web interface using the 'Edit' button or 'Edit' menu entry there. Note: pushing a new commit to the PR will not automatically update the commit message used by Gerrit.
For more details, see:
- [how to update commit messages](https://go.dev/wiki/GerritBot/#how-does-gerritbot-determine-the-final-commit-message) for PRs imported into Gerrit.
- the Go project's [conventions for commit messages](https://go.dev/doc/contribute#commit_messages) that you should follow.
(In general for Gerrit code reviews, the change author is expected to [log in to Gerrit](https://go-review.googlesource.com/login/) with a Gmail or other Google account and then close out each piece of feedback by marking it as 'Done' if implemented as suggested or otherwise reply to each review comment. See the [Review](https://go.dev/doc/contribute#review) section of the Contributing Guide for details.)
Related details
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Michael Pratt (Gerrit)
Michael Pratt voted Commit-Queue+1![]( "Open in Gerrit")
| Commit-Queue | +1 |
Related details
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Paweł Królikowski (Gerrit)
Paweł Królikowski added 1 comment![]( "Open in Gerrit")
if index < 0 || slices.Contains(nonCPUSampleTypes, p.SampleType[index].Type) {Another idea would be to have a known list of non-go sample types and allow them. I don't know enough to say if this is safe (how many sample tyles are there?)
What sample types do you need right now? Is this a small/large list?
Related details
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Krishna Swaroop (Gerrit)
Krishna Swaroop added 1 comment![]( "Open in Gerrit")
if index < 0 || slices.Contains(nonCPUSampleTypes, p.SampleType[index].Type) {Another idea would be to have a known list of non-go sample types and allow them. I don't know enough to say if this is safe (how many sample tyles are there?)
What sample types do you need right now? Is this a small/large list?
The number of perf profile types are open ended. `perf_to_profile` names it's value type after the sampled event (could be `cycles`, `instructions`, `cpu-clock` etc). So the sample types I need are infinite. The sample types that aren't compatible are pretty small and fixed (heap & mutex/block profiles), hence the disallow list.
Related details
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Gerrit Bot (Gerrit)
Gerrit Bot uploaded new patchset![]( "Open in Gerrit")
Related details
- Michael Pratt
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Michael Pratt (Gerrit)
Michael Pratt added 3 comments![]( "Open in Gerrit")
for i, s := range p.SampleType {```suggestion
// Prefer runtime/pprof's CPU profile sample types.
for i, s := range p.SampleType {
```
if index < 0 || slices.Contains(nonCPUSampleTypes, p.SampleType[index].Type) {Krishna SwaroopAnother idea would be to have a known list of non-go sample types and allow them. I don't know enough to say if this is safe (how many sample tyles are there?)
What sample types do you need right now? Is this a small/large list?
The number of perf profile types are open ended. `perf_to_profile` names it's value type after the sampled event (could be `cycles`, `instructions`, `cpu-clock` etc). So the sample types I need are infinite. The sample types that aren't compatible are pretty small and fixed (heap & mutex/block profiles), hence the disallow list.
I'm also not a huge fan of just allowing anything (a perf LLC cache miss profile is not a CPU profile either), but given perf sample types are open ended we don't have a lot of options.
Let's reframe the comments and error message here to be about catching common mistakes. i.e., we'll take whatever default is in your profile, but will error if it looks like a mistake (you passed a heap profile).
I think you can remove the slice in favor of an inline switch with specialized error messages. Something like:
```
switch p.SampleType[index].Type {
case "alloc_objects", "alloc_space", "inuse_objects", "inuse_space":
return 0, fmt.Errorf(`profile must be a CPU profile (default sample type %q indicates that this is a Go heap profile)", p.SampleType[index].Type)
// More cases
}
```
if index < 0 || slices.Contains(nonCPUSampleTypes, p.SampleType[index].Type) {This is only possible if SampleType is empty. Let's make that a separate error case. (It's probably not likely except in a profile with no samples, which is handled in FromPProf, but we should still check to be safe).
Related details
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Gerrit Bot (Gerrit)
Gerrit Bot uploaded new patchset![]( "Open in Gerrit")
- TryBots-Pass: LUCI-TryBot-Result+1 by golang...@luci-project-accounts.iam.gserviceaccount.com
Related details
- Michael Pratt
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Krishna Swaroop (Gerrit)
Krishna Swaroop added 4 comments![]( "Open in Gerrit")
I spotted some possible problems with your PR:
1. You have a long 539 character line in the commit message body. Please add line breaks to long lines that should be wrapped. Lines in the commit message body should be wrapped at ~76 characters unless needed for things like URLs or tables. (Note: GitHub might render long lines as soft-wrapped, so double-check in the Gerrit commit message shown above.)Please address any problems by updating the GitHub PR.
When complete, mark this comment as 'Done' and click the [blue 'Reply' button](https://go.dev/wiki/GerritBot#i-left-a-reply-to-a-comment-in-gerrit-but-no-one-but-me-can-see-it) above. These findings are based on heuristics; if a finding does not apply, briefly reply here saying so.
To update the commit title or commit message body shown here in Gerrit, you must edit the GitHub PR title and PR description (the first comment) in the GitHub web interface using the 'Edit' button or 'Edit' menu entry there. Note: pushing a new commit to the PR will not automatically update the commit message used by Gerrit.
For more details, see:
- [how to update commit messages](https://go.dev/wiki/GerritBot/#how-does-gerritbot-determine-the-final-commit-message) for PRs imported into Gerrit.
- the Go project's [conventions for commit messages](https://go.dev/doc/contribute#commit_messages) that you should follow.
(In general for Gerrit code reviews, the change author is expected to [log in to Gerrit](https://go-review.googlesource.com/login/) with a Gmail or other Google account and then close out each piece of feedback by marking it as 'Done' if implemented as suggested or otherwise reply to each review comment. See the [Review](https://go.dev/doc/contribute#review) section of the Contributing Guide for details.)
Done
```suggestion
// Prefer runtime/pprof's CPU profile sample types.
for i, s := range p.SampleType {
```
Done
if index < 0 || slices.Contains(nonCPUSampleTypes, p.SampleType[index].Type) {Krishna SwaroopAnother idea would be to have a known list of non-go sample types and allow them. I don't know enough to say if this is safe (how many sample tyles are there?)
What sample types do you need right now? Is this a small/large list?
The number of perf profile types are open ended. `perf_to_profile` names it's value type after the sampled event (could be `cycles`, `instructions`, `cpu-clock` etc). So the sample types I need are infinite. The sample types that aren't compatible are pretty small and fixed (heap & mutex/block profiles), hence the disallow list.
Done
if index < 0 || slices.Contains(nonCPUSampleTypes, p.SampleType[index].Type) {This is only possible if SampleType is empty. Let's make that a separate error case. (It's probably not likely except in a profile with no samples, which is handled in FromPProf, but we should still check to be safe).
- Michael Pratt
- Paweł Królikowski
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Michael Pratt (Gerrit)
Michael Pratt voted![]( "Open in Gerrit")
- Paweł Królikowski
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Mark Freeman (Gerrit)
Mark Freeman voted Code-Review+1![]( "Open in Gerrit")
| Code-Review | +1 |
Related details
- Paweł Królikowski
Code-Review
No-Unresolved-Comments
Review-Enforcement
TryBots-Pass
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Gopher Robot (Gerrit)
Gopher Robot submitted the change![]( "Open in Gerrit")
Change information
cmd/internal/pgo: accept CPU profiles from external tools like Linux perf
PGO consumes a CPU pprof profile, but FromPProf only accepted the value
types emitted by Go's own runtime/pprof ("samples"/"count" and
"cpu"/"nanoseconds") and rejected everything else with a hard error. This
blocked CPU profiles produced outside the Go runtime, most notably Linux
perf profiles converted with perf_data_converter (perf_to_profile), which
name their sample values after the profiled event (e.g. "cycles_event"/
"count") rather than using Go's names.
Such profiles are still CPU profiles: PGO only reads the selected value as
an edge weight and thresholds it as a fraction of the total, so the value's
name and unit do not affect the result beyond selecting the column. When no
Go-native CPU value type is present, select the column via the profile's
default sample type (falling back to the last value type when unset, per the
pprof format), while still rejecting value types from known non-CPU profiles
(heap, mutex, block) so that passing e.g. a heap profile to -pgo still fails
clearly.
The other blocker noted on the issue, missing Function.start_line in
symbolized perf profiles, was fixed separately in google/pprof and is
already vendored into the toolchain, so no change is needed here for it.
Fixes #64489
- M src/cmd/internal/pgo/pprof.go
- A src/cmd/internal/pgo/pprof_test.go
Code-Review: +2 by Michael Pratt, +1 by Mark Freeman
Review-Enforcement: Code-Review+2 by Michael Pratt, Code-Review+1 by Mark Freeman
TryBots-Pass: LUCI-TryBot-Result+1 by golang...@luci-project-accounts.iam.gserviceaccount.com
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |