go:inline directive

[Andrey Bokhanko]

Hi Fellow Gophers,

Sorry for rehashing a topic extensively discussed in the past already, but I do believe there are a couple of new arguments warranting further discussion / clarification.

Most active discussion happened here: https://github.com/golang/go/issues/21536 (I added major participants to CC list), but this issue is now closed; moreover, @ianlancetaylor suggested (https://github.com/golang/go/issues/21536#issuecomment-387823313) to use goland-dev as a more appropriate place for discussion.

Key conclusive messages from issue #21536 are this one (https://github.com/golang/go/issues/21536#issuecomment-481800866):

===

Re #17566, package-runtime-limited annotations are fine to experiment with, without a proposal. We have plenty of those already.

But for the broader space of all Go packages, micromanaging optimizations at the source code level like //go:inline or __attribute__((hot)) etc cuts against the grain of Go's approach to software development. This is why we have only one GC tuning knob for example (just GOGC). Make the tools smart enough to make a good decision instead of turning all the programmers into tuning AIs.

Closing again.

===

and this one (https://github.com/golang/go/issues/21536#issuecomment-482202368):

===

A few people pointed out that I didn't respond directly to the various examples of automatic inlining not being optimal for various performance results. To address that explicitly, the answer needs to be to improve the inlining rules. Look at what the compiler does, understand why it's wrong, and fix it.

An inline keyword is like C's old register keyword. It's telling the compiler something it should be able to do on its own. If our compiler is not doing a good enough job, the bugs to file are specific instances where it should be doing better, so we can fix those.

===

(both from @rsc)

The key points, as I read it, are:

1) Go avoids proliferation of tuning knobs, as they entail learning, maintanance and semantic costs

2) If the compiler is not doing a good enough job of inlining, this should be fixed in the compiler itself

Let me add my modest +1 to the point #1. I fully agree that modern C/C++ compilers have *way* too much options / pragmas / directives / attributes / whatever for their own good. Unfortunately, some of them, while can be ignored by 99% of users, are unavoidable for some categories of users, like ones who write performance-critical code -- and I honestly believe go:inline is one of them.

Why?

> 2) If the compiler is not doing a good enough job of inlining, this should be fixed in the compiler itself

Unfortunately, modern compiler science failed to invent a "perfect" inlining heuristic. That's the reason "always_inline" pragma appeared in GCC/Clang. Note that it doesn't *guarantee* inlining; the documentation (https://clang.llvm.org/docs/AttributeReference.html#always-inline-force-inline) clearly says so: "Inlining heuristics are disabled and inlining is always attempted regardless of optimization level ... Does not guarantee that inline substitution actually occurs."

There are at least two fundamental reasons why such heuristic can't be invented:

1) Multitude of hardware targets, their variety and complexity require precise modeling of final machine code latencies and hazards to make an optimal decision at all times. C++ compilers do their best, but still far away of this; as I understand, there is no machine code modeling in Go compiler at all. Simply counting # of statements is nearly not enough.

2) Inlining entails heavy compile time and code size costs, so a "generic" heuristic tries to find a reasonable balance. It might do a good work in general, but invariably fail in some specific cases -- when, for example, a big function is on a critical path and should be inlined even if it's way beyound heuristic limitations. At the same time, other similarly sized, but less performance-critical functions in the same application shouldn't be inlined at all.

As for "turning all the programmers into tuning AIs" comment, it's the way performance engineering works:

1) You profile your application to find hot spots

2) You collect more precise performance data (like counters from Linux perf tool) for the hot spots you found

3) You do your best -- optimizing alogorithm, data structures allocation and -- yes -- using compiler knobs, like go:inline -- to speed-up hot spots based on the what you learned from precise performance data

It's just the way it is, and I don't think Go can invent a silver bullet here.

As for costs of adding go:inline:

> 1) Go avoids proliferation of tuning knobs, as they entail learning, maintanance and semantic costs

they seem to be pretty insignificant. 99% of Go users can simply forget that such a knob exists at all; also, as I understand a similar go:noinline directive accepted many years ago (https://github.com/golang/go/issues/12312) on much lesser "compiler testing" grounds (just my opinion only!) and well proved that the costs are miniscule.

I believe potential benefits (our experiments suggest +2-3% of performance gain, both on our internal code and go1 benchmark) far outweight the costs.

Yours,

Andrey

===

Advanced Software Technology Lab

Huawei

[Andrey Bokhanko]

> I added major participants to CC list

Sorry, it turned out I can't get specific e-mail addresses from
golang-dev mailing list, so I actually don't have any CC list at all

> --
> You received this message because you are subscribed to a topic in the Google Groups "golang-dev" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-dev/H7TTw9mXZSg/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to golang-dev+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/f0b7346d-ee17-480b-9977-83f2aa64c244n%40googlegroups.com.

[Andrey Bokhanko]

Gently pinging @rsc and other guys involved in compiler development --
would be great to know your perspective on the points I raised.

Yours,
Andrey