huge binaries

[notnot]

Why is a.out of "hello world" 1.2 MB big? Is there anything one can do
to minimize the size of the executables?

[notnot]

a.out... what were my fingers thinking... hello.6 i mean

[Rob 'Commander' Pike]

On Oct 28, 2010, at 3:59 PM, notnot wrote:

> Why is a.out of "hello world" 1.2 MB big? Is there anything one can do
> to minimize the size of the executables?
>

tubenose=% 6g helloworld.go
tubenose=% ls -l helloworld.6
-rw-r--r-- 1 r eng 2140 Oct 28 16:13 helloworld.6
tubenose=% 6l helloworld.6
tubenose=% ls -l 6.out
-rwxr-xr-x 1 r eng 190458 Oct 28 16:14 6.out
tubenose=%

Don't know what system you're on but your number is excessive.

-rob

[notnot]

This is what I did on OS X :

protel:hello jpad$ 6g hello.go
protel:hello jpad$ 6l hello.6
protel:hello jpad$ ls -l
total 2408
-rwxr-xr-x 1 jpad jpad 1218238 Oct 29 01:19 6.out
-rw-r--r-- 1 jpad jpad 5569 Oct 29 01:19 hello.6
-rw-r--r-- 1 jpad jpad 69 Oct 29 01:19 hello.go


p.s. sorry for the a.out / hello.6 confusion, it is the size of 6.out
that surpises me..

[Rob 'Commander' Pike]

Your hello.go probably has more dependencies than mine. It's time to repeat my regular request to include all relevant information about the problem you're reporting, including a complete example.

Go programs are bigger than, say, C programs because they contain a run-time system and full type info. If you use fmt, for example, that pulls in a lot of data. Modern computing is like that. There's also been some recent changes to support GDB that add yet more data to the binary. Modern computing is like that.

-rob

[Andrew Gerrand]

My guess is that you're importing "fmt", while Rob's program just uses
the built-in println function.

The reason the binary is so big is that fmt does a lot of clever
stuff, and depends on the io, os, reflect, strconv, strings, unicode,
and utf8 packages. Each of them may have their own dependencies.

fmt.Println does some runtime inspection of its arguments to determine
how they should be formatted. The removal of the unused code paths is
hard because, as far as the compiler can tell through static analysis,
all the dependences are necessary.

That's not to say that it can't get better. Given enough time and
effort, it's certainly possible to bring the binary size down. (And,
indeed, some recent changes to the linker brought the binary size down
by about 30%.) It's a matter of priorities, and this matter is fairly
low on the list.

Andrew

[notnot]

On Oct 29, 1:26 am, "Rob 'Commander' Pike" <r...@google.com> wrote:
> Your hello.go probably has more dependencies than mine.  It's time to repeat my regular request to include all relevant information about the problem you're reporting, including a complete example.
>

Ok, here's my first program, almost the simplest of simplest :

package main

import "fmt"

func main() {
fmt.Printf("hello")

}


> Go programs are bigger than, say, C programs because they contain a run-time system and full type info.  If you use fmt, for example, that pulls in a lot of data.   Modern computing is like that.  There's also been some recent changes to support GDB that add yet more data to the binary.  Modern computing is like that.

I see, the strings inside 6.out are many, and an interesting read for
a go newbie ;) If I understand it correctly, this 6.out does not use
any dynamically loaded libraries, instead, it has all the
functionality built in statically?

protel:hello jpad$ strings 6.out > hello.txt

protel:hello jpad$ ls -l

total 2640

-rwxr-xr-x 1 jpad jpad 1218238 Oct 29 01:19 6.out
-rw-r--r-- 1 jpad jpad 5569 Oct 29 01:19 hello.6
-rw-r--r-- 1 jpad jpad 69 Oct 29 01:19 hello.go

-rw-r--r-- 1 jpad jpad 117089 Oct 29 01:34 hello.txt


But still, it seems overly excessive to have a 1.2 MB "hello world"
executable. Your 190458 bytes seem modest in comparison!

I downloaded and installed the go tools today, this is my first ever
experience with go, i'm interested :)

protel:hello jpad$ 6g -V
6g version 6645

Playing on a Mac Pro quad core, OS X 10.5.8

[Andrew Gerrand]

On 29 October 2010 10:45, notnot <snes...@gmail.com> wrote:
> I see, the strings inside 6.out are many, and an interesting read for
> a go newbie ;) If I understand it correctly, this 6.out does not use
> any dynamically loaded libraries, instead, it has all the
> functionality built in statically?

That's correct. With the exception of C libraries that can be linked
dynamically using a tool called cgo, all Go code is statically linked.

> But still, it seems overly excessive to have a 1.2 MB "hello world"
> executable. Your 190458 bytes seem modest in comparison!

It does seem excessive compared to a minimal C program that does the
same thing, but once you see what the Go runtime and fmt can do for
you it's not quite so shocking.

> I downloaded and installed the go tools today, this is my first ever
> experience with go, i'm interested :)

Welcome! :-)

If you have more questions you can either ask this mailing list (as
you have done), or try the IRC channel #go-nuts on irc.freenode.net.

Andrew

[Russ Cox]

There are various pieces in the hello world binary
that are not strictly necessary, but we haven't worked
on the tools enough to help them figure that out.
There's no fundamental reason why we couldn't make
the binary smaller, but there is more pressing work.
If you want to help out, feel free.

When I build a fmt.Println hello world on Linux,
readelf -S shows the following sizes.

228867 .text
717164 .rodata
9560 .data
388132 .gosymtab
19712 .gopclntab

Most of the bulk is the read-only data, which includes
large tables and reflection data.

For example, package fmt uses package strings,
which uses the Unicode tables for strings.ToLower,
but fmt doesn't call strings.ToLower, so those tables
should be omitted. But because the master map for
the tables gets generated by code rather than laid
out as compile-time data, the implicit init function for
the strings package causes that data to be kept.
If we provided the linker with a list of the initializations
and the order to run them in, then it could discard the
unused data a little better. About 180 kB is due to
the Unicode tables. Another 740 kB is due to the
reflection tables that define the types in the runtime
itself (you can reflect on reflection!). These need to
be made smaller; the next time we iterate on the
reflection interface we will do that.

Another example is that if you refer to a type using
an interface or reflection, then you're referring to its
type descriptor, which has a list of its methods, for
use in dynamic interface checks and reflection.
That can drag in code that might otherwise go unused.
For example, fmt.Println calls fmt.Fprintln passing
os.Stdout as an io.Writer (interface) value. That means
the type information for *os.File is being passed, which
means that all the methods on *os.File are referred to
and thus linked into the binary. So (*os.File).Close is
in the binary even though it is never called, because
the compiler and linker cannot see that fmt doesn't,
say, do a dynamic check to see whether the argument
to fmt.Fprintln is an io.Closer.

Russ

[peterGo]

Rob,

> Don't know what system you're on but your number is excessive.

I get similar numbers for $GOROOT/doc/progs/helloworld.go on Ubuntu
10.04.

$ 6g helloworld.go
$ 6l helloworld.6
$ ./6.out
Hello, world; or Καλημέρα κόσμε; or こんにちは 世界
$ ls -l
-rwxr-xr-x 1 peter peter 1211527 2010-10-28 20:49 6.out
--rw-r--r-- 1 peter peter 5925 2010-10-28 20:49 helloworld.6
-rw-r--r-- 1 peter peter 337 2010-10-28 20:40 helloworld.go


Peter

[peterGo]

Russ,

Here's an analysis that I made for an earlier release.

Big size of HelloWord executable - golang-nuts
http://groups.google.com/group/golang-nuts/msg/79672b46e6cae148

Peter

[notnot]

Thanks Rob, Andrew and Russ for your detailed information. There
appears to be a lot going on under the hood
of a program that looks deceptively simple! Of course, I can live with
such binary sizes on my box, but shaving
off the unnecessary has its charms. I understand that this is not the
most important task to spend your time on
right now, i can be patient :) And if I ever feel like I could
contribute something, I will.

[Russ B.]

Try telling the linker to strip the extra tables:

$ 6g hello.go
$ 6l -s -o hello hello.6

Shaves off about 900kb on my 64-bit Ubuntu system.

[peterGo]

Russ B,

> $ 6l -s -o hello hello.6
> Shaves off about 900kb on my 64-bit Ubuntu system.

For me, the result was a decline from 1,211,527 to 943,448 bytes, a
decline of 268,079 bytes.

Peter

[Russ Cox]

On Thu, Oct 28, 2010 at 20:33, Russ B. <russell....@gmail.com> wrote:
> Try telling the linker to strip the extra tables:
>
> $ 6g hello.go
> $ 6l -s -o hello hello.6
>
> Shaves off about 900kb on my 64-bit Ubuntu system.

I doubt it. Unless you also changed fmt.Println to println.

Russ

[Florian Weimer]

* Andrew Gerrand:

> That's correct. With the exception of C libraries that can be linked
> dynamically using a tool called cgo, all Go code is statically linked.

By the way, is there some documentation about how cgo works? It seems
to me that the binaries created by the Go compilers do not actually
follow the libc ABI, so it is a bit curious that you can link in DSOs
which depend more or less intimately on GNU libc.

[Gustavo Niemeyer]

> By the way, is there some documentation about how cgo works?  It seems
> to me that the binaries created by the Go compilers do not actually
> follow the libc ABI, so it is a bit curious that you can link in DSOs
> which depend more or less intimately on GNU libc.

Yes, and actually libc is linked in as well, otherwise it'd be pretty
hard to make C programs work. The cross-calling between the two
conventions is done in assembly.

--
Gustavo Niemeyer
http://niemeyer.net
http://niemeyer.net/blog
http://niemeyer.net/twitter

[Russ B.]

You're right. I think I made the mistake of comparing an unstripped
executable that was built with an earlier release of go and I built
the stripped version using the trunk from yesterday. Oops.

Still, saving ~200kb is at least a small step towards reducing the
file size.

On Oct 29, 7:27 am, Russ Cox <r...@golang.org> wrote: