Security downsides of static linking

[Damian Gryski]

In discussions with a sysadmin at $WORK, he mentioned that Go's static-linking is a deal-breaker for him.  His example is if a security problem with a shared library (say, openssl) is discovered, only a single package (the vulnerable ssl lib)  needs to be upgraded.  If a problem with Go's SSL implementation is discovered, every Go application that might use that library needs to be rebuilt, and for packages without source code you'd never know which ones include the vulnerable code.  He does, however, agree that the 'single binary' deployment is an improvement over fighting with multitudes of Perl or Python modules.

I am aware of the "dynamic linking considered harmful" page, and I've read the FAQ and know that static linking was a design decision.  Has anyone else encountered this problem before? How did you solve it?  (Note that "problem" in this sense is the security aspect of having to rebuild/redeploy everything instead of just the single shared library.  I'm not interested in stories about how you convinced your co-wokers to switch to Go :)

Damian

[Jan Mercl]

On Wed, Jan 30, 2013 at 11:39 AM, Damian Gryski <dgr...@gmail.com> wrote:
> In discussions with a sysadmin at $WORK, he mentioned that Go's
> static-linking is a deal-breaker for him. His example is if a security
> problem with a shared library (say, openssl) is discovered, only a single
> package (the vulnerable ssl lib) needs to be upgraded. If a problem with
> Go's SSL implementation is discovered, every Go application that might use
> that library needs to be rebuilt, and for packages without source code you'd
> never know which ones include the vulnerable code.

That argument seems pretty weak to me, b/c that's a double-edged sword
issue. With dynamic linking, a new version of library makes all it's
client vulnerable to any exploit which may happen to be introduced by
it. But static-linked Go apps are safe from introducing _new_ exploits
into them via the shared lib vector.

-j

[Damian Gryski]

   We're not, however, upgrading libraries regularly -- *only* for security updates.  So the 'introduction of new exploits' vector is not a concern.

   Damian

[Jan Mercl]

On Wed, Jan 30, 2013 at 11:55 AM, Damian Gryski <dgr...@gmail.com> wrote:
> We're not, however, upgrading libraries regularly -- *only* for security
> updates. So the 'introduction of new exploits' vector is not a concern.

Did you just say you don't update libraries except for when you update
libraries? ;-)

-j

[André Moraes]

> package (the vulnerable ssl lib) needs to be upgraded. If a problem with
> Go's SSL implementation is discovered, every Go application that might use
> that library needs to be rebuilt, and for packages without source code you'd
> never know which ones include the vulnerable code. He does, however, agree
> that the 'single binary' deployment is an improvement over fighting with
> multitudes of Perl or Python modules.

Go make's that better, since you have the source code and they are
static-linked, you can update the code, recompile everything and
redeploy without much trouble, one single makefile could do that.

And, given how Go works, you could replace the wrong library with your
fixed one. Now you can have a fix at source level and fix the library
yourself if the main author aren't helpful.

> and for packages without source code

Even if you are using dynamic-linked, using packages without the
source isn't "safe". You will need to trust the author's of the
package.


--
André Moraes
http://amoraes.info

[Damian Gryski]

Le mercredi 30 janvier 2013 11:39:37 UTC+1, Damian Gryski a écrit :

 His example is if a security problem with a shared library (say, openssl) is discovered, only a single package (the vulnerable ssl lib)  needs to be upgraded.  If a problem with Go's SSL implementation is discovered, every Go application that might use that library needs to be rebuilt, and for packages without source code you'd never know which ones include the vulnerable code.  He does, however, agree that the 'single binary' deployment is an improvement over fighting with multitudes of Perl or Python modules.

   I want to clarify the problem here.  People are talking about "recompling the application and redeploy".  However, the objection is at the _system_ level.  So, on one of our servers there are 81 binaries in /usr/bin linked against libssl.so .  All 81 applications will be updated against a security vulnerability by updating _one_ shared library.  The objection (from the sysadmin team) is that with static linking you now need to recompile and deploy 81 packages (or however many rpms they actually come from).

   Damian

[Dave Cheney]

Do you sysadmin team deal with Java applications? If so, how do they deal with this class of problems with Java applications or application servers.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[André Moraes]

Do they have compiled any large Go programs?

Compiling 81 binaries in Go should take less time than compiling the
libssl.so alone.

[Jesse McNelis]

On Wed, Jan 30, 2013 at 10:53 PM, Damian Gryski <dgr...@gmail.com> wrote:

   I want to clarify the problem here.  People are talking about "recompling the application and redeploy".  However, the objection is at the _system_ level.  So, on one of our servers there are 81 binaries in /usr/bin linked against libssl.so .  All 81 applications will be updated against a security vulnerability by updating _one_ shared library.  The objection (from the sysadmin team) is that with static linking you now need to recompile and deploy 81 packages (or however many rpms they actually come from).

First, how come you have the source for the library, so you can patch it and recompile, but don't have the source for the applications that use the library? How does dynamic linking help you if you have a vulnerable library that you can't recompile?

In terms of having applications that you don't have source for, these generally come from some vendor that is going to statically link them(or provide their own copy of the library) anyway because they don't want to deal with the problems of support software using dependencies that they can't test with. The vendor is then responsible for patching their program.

In the case that you have the source,

Sure 81 binaries need to be reinstalled, but your package management software takes care of that. It knows which binaries need updates and updates them, it also knows which binaries use libssl but don't use the part of it that was patched and thus don't need to be updated.  From the perspective of systems management static linking is more reliable and any pain is hidden by your package management system.

--
=====================
http://jessta.id.au

[Nate Finch]

You mean 81 applications will break when the behavior of the DLL changes?  Or, probably more likely, 3 will break in immediately obvious ways, and 20 will break in subtle ways that aren't readily apparent, and will bite you in the butt just when it is most painful.

I understand the solution of "rebuilding" does not always apply - if there's a bug in 3rd party software, you can't rebuild their software.  But by the same token, replacing code their software depends on is just asking for trouble. You can't know if the new DLL will break their code.  It's like replacing the basement of a house you don't live in and expecting the occupants not to notice their washing machine got moved.

[Jim Robinson]

On Wednesday, January 30, 2013 3:53:21 AM UTC-8, Damian Gryski wrote:

   I want to clarify the problem here.  People are talking about "recompling the application and redeploy".  However, the objection is at the _system_ level.  So, on one of our servers there are 81 binaries in /usr/bin linked against libssl.so .  All 81 applications will be updated against a security vulnerability by updating _one_ shared library.  The objection (from the sysadmin team) is that with static linking you now need to recompile and deploy 81 packages (or however many rpms they actually come from).

The deployment aspect of the problem is why systems like cfengine are a good idea for any large organization.

 

[Jan Mercl]

On Wed, Jan 30, 2013 at 1:49 PM, <den...@kaarsemaker.net> wrote:
> I'm the sysadmin

> I cannot see which libraries are
> linked into a specific go binary.

ldd /path-to-specific-go/binary ?

-j

[Jan Mercl]

On Wed, Jan 30, 2013 at 2:33 PM, Jan Mercl <0xj...@gmail.com> wrote:

I wrote BS. Sorry.

-j

[Sebastien Binet]

that doesn't work, indeed, but something based on go/types.GcImporter
could fit the bill:

http://code.google.com/p/go/source/browse/src/pkg/go/types/gcimporter.go#109

(I used something like that for go-eval and igo -- which need to be
updated for the new go/types-go1.1...)

-s

[Nate Finch]

You should know you're vulnerable because the authors of the binary you're running should tell you what libraries they're using. The same as any other binary authors telling you they use libfoo. You have to trust that when they say they're using libfoo, that they actually are, and not something else.  Regardless of whether they link to it or not, they either might not be actually using the linked dll... or they might have renamed/rebuilt it and linked against a private copy... I've seen both of these situations in production systems.

This does mean that you need to rely on the software vendor to update their software when a vulnerability occurs, instead of installing a new basement under their house for them. This is probably a good idea. And if they're not reliable enough to get this done ASAP, maybe consider a different vendor, or consider open source, where you can see the code and fix it yourself.

I know this is not the black and white case you feel like you have with replacing DLLs, but that world is a fallacy anyway.

On Wednesday, January 30, 2013 7:49:32 AM UTC-5, den...@kaarsemaker.net wrote:

I'm the sysadmin Damian refers to and would like to clarify my reasons a bit, as they seem to be misinterpreted.

Currently, on our linux systems, when there is a vulnerability in libfoo (for any random shared library foo, ssl is merely my favourite here), I update that library and all os, homegrown and third party applications that use it are no longer vulnerable. Go (and java, as has been pointed out) make my life a lot more difficult, especially in the face of large third party commercial applications, or even distribution-provided binary packages (deb, rpm). These don't (afaik) yet exist in the go universe, but in the java universe this is a royal pain in the ass, similar to static linking. Instead of being able to fix problems ourselves, we have to completely rely on an outside party to update their binaries. And java jar bundles are at least simple zip files and can be inspected, I cannot see which libraries are linked into a specific go binary. So if libfoo has a security vulnerability, I won't even know I'm vulnerable unless I happen to know that one or more of the apps I use, use this library.

--
Dennis K.

[Anthony Martin]

Sebastien Binet <seb....@gmail.com> once said:
> Jan Mercl <0xj...@gmail.com> writes:
> > On Wed, Jan 30, 2013 at 1:49 PM, <den...@kaarsemaker.net> wrote:
> >> I'm the sysadmin
> >
> >> I cannot see which libraries are
> >> linked into a specific go binary.
> >
> > ldd /path-to-specific-go/binary ?
>
> that doesn't work, indeed, but something based on go/types.GcImporter
> could fit the bill:

You don't even need that. Just objdump -t $binary | grep $importpath.

Anthony

[Kyle Lemons]

On Wed, Jan 30, 2013 at 2:39 AM, Damian Gryski <dgr...@gmail.com> wrote:

In discussions with a sysadmin at $WORK, he mentioned that Go's static-linking is a deal-breaker for him.  His example is if a security problem with a shared library (say, openssl) is discovered, only a single package (the vulnerable ssl lib)  needs to be upgraded.  If a problem with Go's SSL implementation is discovered, every Go application that might use that library needs to be rebuilt, and for packages without source code you'd never know which ones include the vulnerable code.

A few things here.  Vulnerabilities in Go will tend to be on a much different axis than vulnerabilities in C/C++.  I know some security guys who have spent quite a bit of time hammering at Go and they haven't (without using unsafe) been able to break through the runtime to do anything beyond crash the app in anything resembling normal code, and even that required specifically crafted, racy Go.  Problems in Go libraries will tend to be bugs, and will tend to be surfaced in different ways for different apps.  A bug in the way that SSL is used in HTTP could do a number of problematic things and would affect a reasonable number of apps, but they're more likely to leak too much information than they are to allow arbitrary code execution.  Deploying a shared library requires knowing where all such shared libraries are and notifying all engineering teams, which seems like about as much work as knowing who's importing the affected package and telling them to update their binaries (though obviously it's a bit slower).

 

He does, however, agree that the 'single binary' deployment is an improvement over fighting with multitudes of Perl or Python modules.

I am aware of the "dynamic linking considered harmful" page, and I've read the FAQ and know that static linking was a design decision.  Has anyone else encountered this problem before? How did you solve it?  (Note that "problem" in this sense is the security aspect of having to rebuild/redeploy everything instead of just the single shared library.  I'm not interested in stories about how you convinced your co-wokers to switch to Go :)

Damian

--