Ballooning Executable Sizes

[stephen.t....@gmail.com]

Hello,

I have a reasonably sized project that produces executables that have ballooned in size between two relatively simple commits.

I've tested with three compiler versions and in each case the executable size is significantly larger in one case than the other.

The changes between the two versions amount to 172 insertions and 210 deletions. No module has changed version. No binary data is embedded in either version. The only additional import is to a file within the project and which is imported from many other places.

I can live with the new file size if that's the way it has to be, but it's one heck of leap in size for a seemingly simple change.

I've isolated the change and uploaded to a new repository for easy comparison. I would be very appreciative if someone could help me understand what is happening.

https://github.com/JetSetIlly/Gopher2600-filesize-test

Regards

Stephen Illingworth

[Rob Pike]

Filed https://github.com/golang/go/issues/42729

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/9006c660-bd92-4293-87fc-fab8af1aca26n%40googlegroups.com.

[Ian Lance Taylor]

On Thu, Nov 19, 2020 at 11:34 AM stephen.t....@gmail.com
<stephen.t....@gmail.com> wrote:
>
> I have a reasonably sized project that produces executables that have ballooned in size between two relatively simple commits.
>
> I've tested with three compiler versions and in each case the executable size is significantly larger in one case than the other.
>
>

> The changes between the two versions amount to 172 insertions and 210 deletions. No module has changed version. No binary data is embedded in either version. The only additional import is to a file within the project and which is imported from many other places.
>
> I can live with the new file size if that's the way it has to be, but it's one heck of leap in size for a seemingly simple change.
>
> I've isolated the change and uploaded to a new repository for easy comparison. I would be very appreciative if someone could help me understand what is happening.
>
> https://github.com/JetSetIlly/Gopher2600-filesize-test

What version of Go are you using?

Ian

[Robert Engels]

Based on the OP it is occurring with 1.13,14,15

> On Nov 19, 2020, at 2:19 PM, Ian Lance Taylor <ia...@golang.org> wrote:
>
> On Thu, Nov 19, 2020 at 11:34 AM stephen.t....@gmail.com

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAOyqgcXkgaP5xDR_2vbVTQ1YHbcjhQdYH0KvM9adwacQJwhz8g%40mail.gmail.com.

[Keith Randall]

This was traced down to passing very large (multi-MB) types by value. TL;DR, don't do that. See the issue for more details.

[stephen.t....@gmail.com]

Yes. Thanks for your help. I've been reflecting on this and I have a couple of observations:

1) In some ways, the suddenly bloated executable is useful because, as in this case, it points to a design flaw in the program.

2) However, it was some weeks before I noticed that a problem had arisen because I very rarely build an executable. Using "go run . " is sufficient during the development cycle. This suggests that building and inspecting the binary should be done periodically or possibly, for "go run" to optionally output the resulting binary size and other statistics.

3) I've solved the problem by replacing the fixed sized array with a made slice. However, I can also solve the problem by making sure I use pointer receivers in all instances, not just those that alter the state of the type. Unless I've missed it I don't think there is a linter for that kind of thing. It would be very useful. I might have a crack at it myself.

What all of this shows is that I don't have a good mental model of how the compiler works. Naively, I thought it was "okay" but I've been surprised by a couple of the things that I've learned. For instance, why any of this affects the binary size. I understand why it would affect runtime memory usage (I would have got around to optimising that once I was confident in this part of the program's functionality) but why it affects the binary size escapes me.

So, I have a learning opportunity here: can anyone suggest good literature on modern compiler design? I'd like to extend my knowledge of this subject.

Thanks again to the Go team for helping.