The right way for golang binary to reduce iTLB cache miss

[rmfr]

One of our Golang applications has a very huge binary size and the size of the .text segment itself in the elf is approximately 34MB. The iTLB load miss reaches about 87% of all iTLB cache hits.

Is there any advice for big Golang applications to reduce the iTLB cache miss? Two solutions come to me, PGO and using hugepages to load the .text segment. But they are both seem very difficult to implement in Golang.

[Robert Engels]

This is the icache or code - not the data the code is accessing. 

On Sep 17, 2021, at 8:12 AM, peterGo <go.pe...@gmail.com> wrote:



rmfr,

What is your data, what is its structure, and what are the access paths and frequencies?

Your data is unlikely to be a randomly accessed blob of bits. Is there a set of Pareto access paths that you can use to order your data by access frequency? You want the most frequently accessed data to be adjacent.

Peter

On Friday, September 17, 2021 at 5:09:32 AM UTC-4 rmfr wrote:

One of our Golang applications has a very huge binary size and the size of the .text segment itself in the elf is approximately 34MB. The iTLB load miss reaches about 87% of all iTLB cache hits.

Is there any advice for big Golang applications to reduce the iTLB cache miss? Two solutions come to me, PGO and using hugepages to load the .text segment. But they are both seem very difficult to implement in Golang.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/1778bb58-e2e1-4bba-a869-1effb1e3952an%40googlegroups.com.

[rmfr]

Thanks a lot :-D

> This is the icache or code - not the data the code is accessing. 

Yes, it is the cache miss of iTLB, i.e. Instruction Translation Lookaside Buffer.

I have tried the https://github.com/facebookincubator/BOLT, but the linker Golang used seems does not allow `--emit-relocs` option. I also tried libhugetlbfs, but it seems could not be applied to Golang.

[Jesper Louis Andersen]

On Fri, Sep 17, 2021 at 11:09 AM rmfr <remus.cl...@gmail.com> wrote:

One of our Golang applications has a very huge binary size and the size of the .text segment itself in the elf is approximately 34MB. The iTLB load miss reaches about 87% of all iTLB cache hits.

Is there any advice for big Golang applications to reduce the iTLB cache miss? Two solutions come to me, PGO and using hugepages to load the .text segment. But they are both seem very difficult to implement in Golang.

Shot in the dark: is the application running next to some other application on the hardware which wipes the TLB?

[robert engels]

I will add that over aggressive inlining can cause this - it explodes the binary size.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAGrdgiWDY%2Bjh2%3DPzAatJU5Ckit0_maex%3DgGC3E4M-t8VB94Qig%40mail.gmail.com.

[rmfr]

One of our Golang applications has a very huge binary size and the size of the .text segment itself in the elf is approximately 34MB. The iTLB load miss reaches about 87% of all iTLB cache hits.

>> Is there any advice for big Golang applications to reduce the iTLB cache miss? Two solutions come to me, PGO and using hugepages to load the .text segment. But they are both seem very difficult to implement in Golang.

> Shot in the dark: is the application running next to some other application on the hardware which wipes the TLB?

Nope, it occupies a standalone x86 physical server.

[rmfr]

The problem is this application is a quite big Golang project. It has about 500, 000 lines of Golang codes and didn't even count these required 3-rd party libraries.

[rmfr]

I have tried to disable the inline during compiling, but it results in no big difference.

[robert engels]

I can’t imagine the app has 34 mb of code in the hot path where this would matter - so I assume the application is partitioned with various Go routines doing a “type” of work.

Which encounters a problem with the Go design. You can’t partition Go routines, and you can’t set cpu/core affinity for a Go routine.

So it can be very difficult to manage the cache (code & data) in a large multipurpose app, or an app where there data is naturally partitioned (e.g. a financial exchange where different stocks and asset classes are independent).

I didn’t look but I believe there is a FR/issue to address the above deficiencies.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/7de12443-2916-4190-a08d-0825605f6ca5n%40googlegroups.com.

[rmfr]

I have tried to disable the inline during compiling, but it results in no big difference. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/7de12443-2916-4190-a08d-0825605f6ca5n%40googlegroups.com.

— View in Memo

[Robert Engels]

I can’t imagine the app has 34 mb of code in the hot path where this would matter - so I assume the application is partitioned with various Go routines doing a “type” of work. Which encounters a problem with the Go design. You can’t partition Go routines, and you can’t set cpu/core affinity for a Go routine. So it can be very difficult to manage the cache (code & data) in a large multipurpose app, or an app where there data is naturally partitioned (e.g. a financial exchange where different stocks and asset classes are independent). I didn’t look but I believe there is a FR/issue to address the above deficiencies. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/03F9275B-85F4-4E25-992A-108BFE01FA83%40ix.netcom.com.

— View in Memo

[rmfr]

The problem is this application is a quite big Golang project. It has about 500, 000 lines of Golang codes and didn't even count these required 3-rd party libraries. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/12a18e2b-e860-4ee5-8c64-8b5801dce3f9n%40googlegroups.com.

— View in Memo

[rmfr]

One of our Golang applications has a very huge binary size and the size of the .text segment itself in the elf is approximately *34MB*. The iTLB load miss reaches about *87%* of all iTLB cache hits. >> Is there any advice for big Golang applications to reduce the iTLB cache miss? Two solutions come to me, PGO <https://en.wikipedia.org/wiki/Profile-guided_optimization> and using hugepages to load the .text segment. But they are both seem very difficult to implement in Golang. > Shot in the dark: is the application running next to some other application on the hardware which wipes the TLB? Nope, it occupies a standalone x86 physical server. > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/60aa2e83-6639-4c91-be34-37351ebf70b7n%40googlegroups.com.

— View in Memo