Is Go not really statically-compiled? Also, does "debugging code" bloat binaries unnecessarily?

[Shawn Milochik]

I've been using Docker a lot lately for fun & profit. I got interested in the topic of minimal Docker containers (smallest possible Linux images). When using super-minimal base images (scratch and busybox are two good examples), a Go binary does not run. The error is "no such file or directory."

I learned two very interesting things from this blog post[1]:

1. If I set the environment variable CGO_ENABLED=0 and -a to recompile everything, my Go binary *will* run in a minimal Docker container. Running "ldd myapp" before and after show it had dependencies before being compiled that way, and afterwards is "not a dynamic executable."

2. If I pass "-ldflags '-s'" to "go build" the binary size is cut in half and still works.

So my questions: 

A. My understanding of a "static binary" was that it runs without dependencies is compiled for the proper architecture, which I assumed was primarily the kernel. That is clearly not true. Is there an explanation for this that is comprehensible to someone who doesn't have a deep understanding of things like compilers?

B. It seems that it's common in the Go community to excuse the binary size by saying it is what allows it to work without dependencies. In today's reality they're really not big, but if they can easily be half the size and still work, why isn't that the default?

[1] http://blog.xebia.com/2014/07/04/create-the-smallest-possible-docker-container/

-------------------------------------------------------------------------------------------------------------------------

Extra stuff for those interested in playing with it (requires Docker):

Create a file named Dockerfile with these contents:

FROM scratch

COPY ./listen /listen

CMD ["/listen"]

Run a Go program of your choosing, or try this one I wrote and have been using:

http://play.golang.org/p/xsWfITuMp2

Build & run:

CGO_ENABLED=0 go build -a -o listen listen.go \

&& docker build -t minimal . \

&& docker run -it --rm -p 5000:5000 minimal

Then use wget or curl to hit localhost as a test. If you do the same thing without the CGO_ENABLED it won't work. Note: It seems to work without the extra stuff for a "hello world" program that imports nothing but fmt.

[Harmen B]

On Fri, Oct 24, 2014 at 6:06 PM, Shawn Milochik <shawn...@gmail.com> wrote:

I've been using Docker a lot lately for fun & profit. I got interested in the topic of minimal Docker containers (smallest possible Linux images). When using super-minimal base images (scratch and busybox are two good examples), a Go binary does not run. The error is "no such file or directory."

I learned two very interesting things from this blog post[1]:

1. If I set the environment variable CGO_ENABLED=0 and -a to recompile everything, my Go binary *will* run in a minimal Docker container. Running "ldd myapp" before and after show it had dependencies before being compiled that way, and afterwards is "not a dynamic executable."

2. If I pass "-ldflags '-s'" to "go build" the binary size is cut in half and still works.

So my questions: 

A. My understanding of a "static binary" was that it runs without dependencies is compiled for the proper architecture, which I assumed was primarily the kernel. That is clearly not true. Is there an explanation for this that is comprehensible to someone who doesn't have a deep understanding of things like compilers?

This old post explains the non-static-ness:

"By default, even programs that are pure Go do link aginst libc if they 

do any DNS lookups or user name lookups.  This is because those 
operations are highly system-dependent to deal with things like 
firewalls and LDAP. "

https://groups.google.com/d/topic/golang-nuts/4Q4HLHRcHQY/discussion

[Ken Allen]

1. By default go relies on a few things from libc related to networking. By passing CGO_ENABLED it falls back to a more basic implementation that doesn't rely on libc.

2. Debugging symbols are like log levels, you're better off having debugging stuff there all the time because then it's there when things explode.

It's cool that this works though, when I first heard about go I envisioned tiny servers that are basically kernel + go app + maybe busybox.

[Shawn Milochik]

On Fri, Oct 24, 2014 at 12:13 PM, Harmen B <har...@typetypetype.net> wrote:

This old post explains the non-static-ness:

"By default, even programs that are pure Go do link aginst libc if they 

do any DNS lookups or user name lookups.  This is because those 
operations are highly system-dependent to deal with things like 
firewalls and LDAP. "

https://groups.google.com/d/topic/golang-nuts/4Q4HLHRcHQY/discussion

Thanks, that's interesting. So it's only stuff in the "net" package. At least according to that thread.

Based on the thread you sent, I see that this works also:

go build -tags netgo -a -o listen listen.go 

(Use -tags -netgo instead of the CGO environment variable). Either case still requires -a.

So if it works just fine without a dependency, and it's available to override and fall back on, why isn't that the default?

[Konstantin Khomoutov]

On Fri, 24 Oct 2014 12:06:45 -0400
Shawn Milochik <shawn...@gmail.com> wrote:

[...]

> 1. If I set the environment variable CGO_ENABLED=0 and -a to recompile
> everything, my Go binary *will* run in a minimal Docker container.
> Running "ldd myapp" before and after show it had dependencies before
> being compiled that way, and afterwards is "not a dynamic executable."
>
> 2. If I pass "-ldflags '-s'" to "go build" the binary size is cut in
> half and still works.
>
> So my questions:
>
> A. My understanding of a "static binary" was that it runs without
> dependencies is compiled for the proper architecture, which I assumed
> was primarily the kernel. That is clearly not true. Is there an
> explanation for this that is comprehensible to someone who doesn't
> have a deep understanding of things like compilers?

Yes. By default Go built for Linux still links against libc to get
hold onto the system's DNS resolver -- basically, getaddrinfo() and
gai_strerror(), AFAIK.

Disabling this feature will enable some built in Go code which will do
DNS resolving by itself which is said to be less versatile (don't know
in which way exactly).

IOW, libc's getaddrinfo() seems to provide certain functionality not
available when using kernel syscalls directly.

> B. It seems that it's common in the Go community to excuse the binary
> size by saying it is what allows it to work without dependencies. In
> today's reality they're really not big, but if they can easily be
> half the size and still work, why isn't that the default?

After you've stripped the debug symbols, the nearest unhandled panic()
in your program would not print a useful stack trace. I think it might
even crash mysteriously when trying to gracefully handle that panic().
I would just try and see. I'm also positive that certain kinds of
runtime introspection will cease to work.

[Konstantin Khomoutov]

On Fri, 24 Oct 2014 12:20:34 -0400
Shawn Milochik <sh...@milochik.com> wrote:

[...]

> Thanks, that's interesting. So it's only stuff in the "net" package.
> At least according to that thread.
>
> Based on the thread you sent, I see that this works also:
>
> go build -tags netgo -a -o listen listen.go
>
>
> (Use -tags -netgo instead of the CGO environment variable). Either
> case still requires -a.
>
> So if it works just fine without a dependency, and it's available to
> override and fall back on, why isn't that the default?

My take is that most sensible Linux systems have libc anyway, so the
default works for the vast majority of users. Those with special needs
(you, for example) are able to do the necessary tweaks to cater for
their setups.

[Ian Taylor]

On Fri, Oct 24, 2014 at 9:20 AM, Shawn Milochik <sh...@milochik.com> wrote:
> On Fri, Oct 24, 2014 at 12:13 PM, Harmen B <har...@typetypetype.net> wrote:
>>
>>
>> This old post explains the non-static-ness:
>>
>> "By default, even programs that are pure Go do link aginst libc if they
>> do any DNS lookups or user name lookups. This is because those
>> operations are highly system-dependent to deal with things like
>> firewalls and LDAP. "
>>
>> https://groups.google.com/d/topic/golang-nuts/4Q4HLHRcHQY/discussion
>>
>
> Thanks, that's interesting. So it's only stuff in the "net" package. At
> least according to that thread.

Also the os/user package.

> Based on the thread you sent, I see that this works also:
>
> go build -tags netgo -a -o listen listen.go
>
>
> (Use -tags -netgo instead of the CGO environment variable). Either case
> still requires -a.
>
> So if it works just fine without a dependency, and it's available to
> override and fall back on, why isn't that the default?

Because it doesn't work just fine. On Darwin systems it fails to work
through the default firewall. On GNU/Linux systems it ignores
/etc/nsswitch.conf. On all systems it ignores some of the more
esoteric functionality available in /etc/resolv.conf. Implementing
all the system-specific stuff that the libc resolver uses is a red
queen's race, and may not be possible at all on Darwin.

The thinking is that for the ordinary user, it's better if Go programs
behave like other programs on the system with regard to the complex
details of DNS lookup and username lookup. As you've discovered,
other options are available for the advanced user.

Ian

[Ian Taylor]

On Fri, Oct 24, 2014 at 12:16 PM, Ian Taylor <ia...@golang.org> wrote:
> On Fri, Oct 24, 2014 at 9:20 AM, Shawn Milochik <sh...@milochik.com> wrote:
>> On Fri, Oct 24, 2014 at 12:13 PM, Harmen B <har...@typetypetype.net> wrote:
>>>
>>>
>>> This old post explains the non-static-ness:
>>>
>>> "By default, even programs that are pure Go do link aginst libc if they
>>> do any DNS lookups or user name lookups. This is because those
>>> operations are highly system-dependent to deal with things like
>>> firewalls and LDAP. "
>>>
>>> https://groups.google.com/d/topic/golang-nuts/4Q4HLHRcHQY/discussion
>>>
>>
>> Thanks, that's interesting. So it's only stuff in the "net" package. At
>> least according to that thread.
>
>
> Also the os/user package.

And also the crypto/x509 package on Darwin, I always forget about that
one. On Darwin we need to call into the C library to get the trusted
root certificates.

Ian

[Andreas Klauer]

Am Freitag, 24. Oktober 2014 18:07:19 UTC+2 schrieb Shawn Milochik:

A. My understanding of a "static binary" was that it runs without dependencies

Even a binary that says "ldd: not a dynamic executable" may have some library dependencies.

glibc wants to load some for DNS lookups (libnss, libresolv, whatever), so even if you bake the glibc into your binary, making it "static", DNS still does not work.