Secure Go binaries

[Alex]

I've spent some time over the past month or so trying to make a secure Go binary and exploiting a Go binary. I posted a writeup about it and feel like others could be interested.

http://www.andrew.cmu.edu/user/areece/securego/

tl;dr: The deterministic, executable heap leads to an interesting exploit.

~Alex Reece

[David Leimbach]

I was just playing with pointers yesterday, trying to understand a memory issue I thought I was having (me not remembering the memory model mostly).  

I came to the conclusion that pointers were showing up at very much the same memory addresses each time, and that perhaps randomizing this would be a good idea.

Dave

 

~Alex Reece

[Russ Cox]

You linked buggy C code into a Go program and then exploited that
buggy C code. In short, C has buffer overflows and you can take
advantage of them to run other code. This is not surprising.

You claim that "the behavior I performed in cgo could have trivially
been perfomed in pure Go", but I strongly disagree. There is no way
in pure Go to write code with the bug in this C code (from your
c-log.c):

void Log(char* prefix, char* message) {
char buf[128];
sprintf(buf, "%s%s", prefix, message);
printf("%s\n", buf);
}

You are coming at this from a C mindset, you used C, and you have
found C results. In fact many Go programs do not pass arbitrary data
to C code or even link external C libraries at all. It would be much
more relevant to Go if you focused on concerns specific to the Go
language. There are possible avenues in a badly written Go program,
but they are significantly rarer than buffer overflows are in C.

Your page does not explain what your attack model is. You call the
binary webapp, yet it clearly isn't a web app. The attacker seems to
be allowed to run the binary, provide arbitrary input, and observe
standard error. In a real web app, standard error (where the stack
traces go) is not connected to the attacker, so printing a stack trace
on panic is not a big deal. On the other hand, if the attacker is on
your local machine, you have other problems to worry about. But this
is all secondary to the fact that Go is not C, and if you stick to
writing Go code instead of linking in C code, there is no stack
smashing.

Russ

[Russ Cox]

On Mon, Apr 30, 2012 at 21:51, David Leimbach <lei...@gmail.com> wrote:
> I was just playing with pointers yesterday, trying to understand a memory
> issue I thought I was having (me not remembering the memory model mostly).
>
> I came to the conclusion that pointers were showing up at very much the same
> memory addresses each time, and that perhaps randomizing this would be a
> good idea.

Address space randomization is an OS-level workaround for a
language-level problem, namely that simple C programs tend to be full
of exploitable buffer overflows. Go fixes this at the language level,
with bounds-checked arrays and slices and no dangling pointers, which
makes the OS-level workaround much less important. In return, we
receive the incredible debuggability of deterministic address space
layout. I would not give that up lightly.

Russ

[Alex Reece]

I don't think you quite understood the point I was trying to make. 

I explicitly, artificially, created a vulnerable Go binary by linking against a completely artificial vulnerable C library. This is something that simply isn't done in practice. However, what I wanted to focus on was the process of turning a vulnerability in a Go binary into a working exploit. I was *not* talking about creating a vulnerability in the Go application (in fact, I explicitly assume that there is a vulnerability somewhere without really justifying where it came from and I had to jump through some significant hoops to even get an exploitable vulnerability). 

I'm imagining the scenario (perhaps somewhat artificial) where someone can read a binary and interact with a service running as a different user (maybe a Go daemon?). Supposing there were a vulnerability in Go, such as one introduced by a C library or via a bug in the standard library, someone could take steps similar to the ones I took in exploiting this binary.

Please don't get me wrong - I'm not claiming Go is vulnerable. I'm saying that many of the design decisions make it more easily exploitable. I find it much more prudent to adopt some minimal protections rather than to adopt that mindset that proper Go binaries never have vulnerabilities, although I recognize that this attitude is not shared by everyone in the Go community.

~Alex 

[Peter Weinberger]

As I understand it, your thought is that there are a bunch of things that are commonly done to make it harder to exploit executables, for instance ASLR, stack sentries, encrypted pointers, or whatever, and Go doesn't do these but should.  I think there are two responses, one possibly unconvincing and pragmatic, and the other principled but debatable.

The first is that until there's an issue, why bother?  Security is relative to the threats, not some absolute property, so one could wait until there are Go exploits to see what to do.  This might seem like it is ignoring a decade of experience, but most of the security mechanisms being recommended are responses to buffer overflows.  Who knows what sort of weakness some clever person will find in Go, and that's the time to think of countermeasures.

The second is that each security mechanism comes with a cost.  ASLR makes attacks harder, but not impossible.  [On 32-bit systems does it add as much as 20 bits of entropy?  So attacking 100 million machines means only 100 get infected?  Is that good or bad?]  And ASLR makes debugging harder.  Stack goo comes with some execution cost, some space cost, and the (presumably minimal) cost of getting it right.  etc.  I'm sure the Go folks understand all of this, and made their tradeoffs deliberately.

[Aram Hăvărneanu]

Peter Weinberger wrote:
> On 32-bit systems does it add as much
> as 20 bits of entropy?

In the Windows XP implementation, it only added 4 bits of entropy, it
was laughable. It got better, I can't remember if now it's 8 or 16
bits, but it's certainly not a large number.

Of course ASLR is useless on Windows since the equivalent of a libc is
a non-relocatable shared library, but I'm getting way offtopic.

Deterministic address space has aided me greatly in debugging Go
programs. It's a little gem.

--
Aram Hăvărneanu

[Alex]

I must admit, I didn't think that people would find a deterministic address space that useful (the only times I have cared are when I was writing exploits). Perhaps this is another one of those areas where being to compile to a debug mode vs a hardened, production mode would be nice? Although it is important to note that Go actually has partial ASLR and a non-executable stack on my system (the stack is randomized, at least https://gist.github.com/2568738). Why protect the stack if we never have vulnerabilities? ;-)

But more seriously, I don't understand why one would refuse to take advantage of the years of security research that have gone into securing the heap. Vulnerabilities only happen when some developer does something really stupid - they read into a fixed sized buffer, they accidentally write too much memory, they allow the user to control the format string, etc. One can *always* make the claim that properly written code won't be exploitable. People protect their binaries because they expect that someone else will screw up and want to guard against that. 

If Go offers cgo as a way to link (potentially vulnerable) C libraries against a Go binary, I think it is irresponsible for Go to ignore the (heap) protections that are so important for C programs. If the claim is the cgo isn't used, perhaps it should be removed to prevent people from inadvertently introducing vulnerabilities. Not to mention, the claim that the Go standard library has no bugs and never will is a *really* strong claim. Personally, I'd rather add some protections than claim that the Go runtime will never have a bug. Policing only the front door (with bounds checking, secure format strings, etc) only works if there are no back doors (and cgo provides a pretty big back door). 

~Alex

[Russ Cox]

On Tue, May 1, 2012 at 00:07, Alex Reece <awr...@gmail.com> wrote:
> I explicitly, artificially, created a vulnerable Go binary by linking
> against a completely artificial vulnerable C library. This is something that
> simply isn't done in practice. However, what I wanted to focus on was the
> process of turning a vulnerability in a Go binary into a working exploit.

My primary objection is exactly this. You are not turning a
vulnerability in a Go binary into a working exploit. You are turning
a vulnerability in a C binary into a working exploit, and that is
well-trodden ground. Your argument boils down to the assumption that
there will be enough C in a typical Go program to provide a useful
attack surface, so that that adding all these C-centric security
mechanisms provide enough benefit to offset their very real cost in
complexity of implementation and hindrance of debugging. In the Go
programs I have worked on, this is not true. There is very little C
code, and even less of it is processing untrusted user input.
Furthermore, the C code is running on C-created stacks (which do get
randomized) and in typical use is processing C-allocated data blocks
(C.CString and C.malloc call into the standard C allocator).

> Why protect the stack if we never have vulnerabilities? ;-)

The kernel does this behind Go's back. Go did not ask for it. Also,
by looking at /proc/pid/maps you are only seeing where the kernel
placed the process's stack segment. In reality the Go code runs on a
stack allocated from Go's heap, so it will have a more deterministic
address and also happens to be executable. The C code does run on an
OS-allocated stack, so it has a less deterministic address and is not
executable. Your exploit, it should be noted, jumps to the Go copy of
the input, not the C buffer. In a real web app (as opposed to a
command-line program) you'd be lucky indeed to guess the address used
for a particular copy of the input.

> But more seriously, I don't understand why one would refuse to take
> advantage of the years of security research that have gone into securing the
> heap.

There is a cost to all defenses. It only makes sense to employ them
if the cost is outweighed by a benefit. I argue that in this case the
benefit is very slight. If the techniques you mentioned were a 100%
cure, then maybe it would make sense to adopt them. But in reality
they are just one more hurdle to pass, and there is a definite point
of diminishing returns.

> Vulnerabilities only happen when some developer does something really
> stupid - they read into a fixed sized buffer, they accidentally write too
> much memory, they allow the user to control the format string, etc. One can
> *always* make the claim that properly written code won't be exploitable.

That is not the claim I am making. Obviously code will have bugs, and
I would not argue that any of these are "really stupid". We all have
bad days and make mistakes. My claim is that Go is a different
language than C. When you make one of these mistakes - read into a
fixed-size buffer, write too much memory, use the wrong format string
- in Go, that mistake does not lead to arbitrary memory corruption,
and thus it cannot be leveraged into an exploitable security hole.

> If the claim is the cgo
> isn't used, perhaps it should be removed to prevent people
> from inadvertently introducing vulnerabilities.

That is not the claim, and to be very clear, most days I would be more
than happy to remove cgo, as I think it is overused and people expect
too much from it. However, this is a perfect demonstration of my
point. If you consider the removal of cgo for security reasons, the
benefit of removing the possibility of exploitable C code is, it seems
to me, dramatically outweighed by the cost of not being able to link
against important existing C libraries. People do use and benefit
from cgo.

> Not to mention, the claim
> that the Go standard library has no bugs and never will is a *really* strong
> claim.

Again, this is not the claim I am making. See above.

> Personally, I'd rather add some protections than claim that the Go
> runtime will never have a bug. Policing only the front door (with bounds
> checking, secure format strings, etc) only works if there are no back doors
> (and cgo provides a pretty big back door).

Again, it comes down to costs vs benefits. Here's an easy way to
eliminate a large class of remote exploit vulnerabilities: unplug your
network cable. This is a general, effective solution, but it comes at
a significant cost. For some people, that cost is outweighed by the
potential cost of being exploited, and so they adopt this approach.
Other people weigh the tradeoff differently and decide to be
connected.

As Peter said, we understand the points you are making, but we made
our tradeoffs deliberately. It's okay that we see them differently
than you do, just as it's okay that some people choose to use an air
gap and others do not.

Russ

[Ian Lance Taylor]

Alex <awr...@gmail.com> writes:

> Vulnerabilities only happen when some developer does something
> really stupid - they read into a fixed sized buffer,
> they accidentally write too much memory, they allow the user to control the
> format string, etc.

I think it's worth stressing that in Go code none of those programmer
errors will lead to memory corruption or introduce a new security hole.

For your broader argument you need to do a cost/benefit analysis. I
think you are slightly overstating the benefits, and you are ignoring
the costs.

Ian

[Alex Reece]

You make some fair points, although I would like to provide a minor clarification: The program I used to determine that stacks were randomized and non-executable was a pure Go program (and main.main uses this stack https://gist.github.com/2569437).

~Alex

[Russ Cox]

On Tue, May 1, 2012 at 12:37 PM, Alex Reece <awr...@gmail.com> wrote:
> You make some fair points, although I would like to provide a minor
> clarification: The program I used to determine that stacks were randomized
> and non-executable was a pure Go program (and main.main uses this
> stack https://gist.github.com/2569437).

All I can say in my defense is this: I wrote the code, there is no
attempt at randomness, and the allocation asks for execute permission.
However, small stacks come from a different area of memory separate
from the garbage collected heap, and the allocations for the stacks
come from a mmap call that lets the kernel choose the address. If the
kernel is configured to hand back memory at somewhat randomized
locations, then the small stacks will end up somewhat randomized. The
allocation asks for PROT_READ|PROT_WRITE|PROT_EXEC. You should be
able to verify both of these claims using strace -f.

Russ

[Alex]

No need for a defense, I was never on the offense! :)

As it turns out, I overlooked that 0xb7f8ffbc (a stack address used by main.main) isn't actually the stack assigned to the go binary by the OS (0xbffffff). It is reserved (as you said) via a call to a mmap(NULL, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS) (which will probably return a randomized region of memory). https://gist.github.com/2578123

Still, the C stacks are randomized and non-executable and the C libraries are randomized.

[Thomas Bushnell, BSG]

Right. This is protection against a class of programming bugs that C is prone to. It is not needed in go, because for some of those bugs, go is not prone to them at all, and for others, go is not able to suffer memory corruption from them.

Thomas

[Alex]

Well, assuming that there are no memory corruption bugs in the Go runtime itself (and the assumption you were already making that Go binaries are never linked against C libraries).

I would like to point out that the vulnerable program (webapp) is exploitable on a 64 bit machine almost exclusively because the go runtime eschews the protections that are considered crucial for C programs. The vulnerability introduced by the C code isn't exploitable without the weak go runtime (ALSR, NX, and libc randomization pretty much eliminate any surface for exploiting the C parts of the program).

~Alex

[Thomas Bushnell, BSG]

On Wed, May 2, 2012 at 10:19 AM, Alex <awr...@gmail.com> wrote:

Well, assuming that there are no memory corruption bugs in the Go runtime itself (and the assumption you were already making that Go binaries are never linked against C libraries).

I would like to point out that the vulnerable program (webapp) is exploitable on a 64 bit machine almost exclusively because the go runtime eschews the protections that are considered crucial for C programs. The vulnerability introduced by the C code isn't exploitable without the weak go runtime (ALSR, NX, and libc randomization pretty much eliminate any surface for exploiting the C parts of the program).

ALSR, NX, and libc do not eliminate surface for exploiting the C parts of the program.

You're engaging in a common mistake: knowing that it is important to have seat belts on cars, because of the risks that cars are prone to, you're now arguing for seat belts in restaurants (in case a car should crash into the restaurant), seat belts in trains (even though the risk/reward ratio is entirely different), seat belts on bicycles, and so forth.

Go will surely have its own security concerns, but they are not likely to be the same ones as C.

Thomas

[Alex]

Thomas,

I respectfully disagree? I'm not sure there is an exploit for that binary that doesn't take advantage of the Go runtime or assumes weak ASLR, NX, or libc randomization. I would be interested in seeing that exploit, at the very least :)

~Alex

[Paul Borman]

It seems the attack surface you are trying to get to is one in which

a)  a buggy C function is linked into a go program

b)  that go alloc'ed memory vs C malloced memory will be used

So you say you want to go through some extra effort to secure a Go program.  This would mean that you  have already identified the Go program in question.  You have already taken care of stupid design decisions (like assuming you can just call sh on a string from a web page because you gave them that string in the first place :-)   So now you are talking about the "buffer overrun" class of bugs (that appears to be what you are talking about).

If you are already going through the efforts to make sure the design of the program is not at fault it isn't too much more to check:

a) does it import cgo/swigged C code?

b) do any packages use "unsafe"

If the answer to both is "no" then trying to thwart a buffer overrun is not very productive.  If the answer to a) is "yes" then you should make sure the swig/cgo interface uses C.malloc for data and never pass in a go allocated buffer that can be written into.  Not adding this protection invalidates Go's memory guarantees. If the answer to b) is "yes" then you must do as the documentation says and vet the code yourself to preserve the guarantee.

I think (hope) with time there will be less and less need to cgo or swig.

    -Paul

[Alex]

Paul, Russ, Thomas, Ian, etc,

I think we've been on different wavelengths about something for a while. My post was not on the ease of finding vulnerabilities in Go - it was on the ease of turning them into working exploits. My claim is that this process is easier in Go because it does not have many of the protections that come standard in C binaries.

I acknowledge that it is hard (perhaps impossible) to find vulnerabilities in pure Go binaries - one would probably have to find a bug in the Go runtime itself. But that's just it - by eschewing common protections, Go is widening the class of bugs (in its runtime) that could turn into viable exploits.

Regardless, the Go community's response is quite clear: since Go does not have vulnerabilities, protections are unnecessary. I respectfully disagree (vulnerabilities could come from cgo or bugs in the go runtime/compiler), but don't want to frustrate anyone by pushing it further. :) Thanks for taking the time to read my post and think about what I had to say.

Best,

~Alex

[Jan Mercl]

Quoting your blog post:

"Lastly, the behavior I performed in cgo could have trivially been perfomed in pure Go."

Quoting your last mail list post:

"I acknowledge that it is hard (perhaps impossible) to find vulnerabilities in pure Go binaries -one would probably have to find a bug in the Go runtime itself. But that's just it -by eschewing common protections, Go is widening the class of bugs (in its runtime) that could turn into viable exploits."

Please let me know which of them should be considered your opinion. I think they pretty much contradict each other.

[Devon H. O'Dell]

Op 2 mei 2012 15:24 heeft Jan Mercl <0xj...@gmail.com> het volgende geschreven:
> Quoting your blog post:
>
>  "Lastly, the behavior I performed in cgo could have trivially been perfomed in pure Go."

I guess if you use unsafe, it's possible. But I guess that's also the
point of unsafe.

--dho

[Alex]

Jan:

>  "Lastly, the behavior I performed in cgo could have trivially been perfomed in pure Go."

Uh, its pretty easy to print to standard out in Go? That's really all the levellog package does.

~Alex 

[Devon H. O'Dell]

Op 2 mei 2012 15:05 heeft Alex <awr...@gmail.com> het volgende geschreven:
> Regardless, the Go community's response is quite clear: since Go does not
> have vulnerabilities, protections are unnecessary. I respectfully disagree
> (vulnerabilities could come from cgo or bugs in the go runtime/compiler),
> but don't want to frustrate anyone by pushing it further. :) Thanks for
> taking the time to read my post and think about what I had to say.

This is an unfair characterization of the "community's" response.
Nobody claimed that there are no vulnerabilities. Indeed, the last
time you injected this interpretation, Russ specifically responded,

"Again, this is not the claim I am making."

1) You had to go through hoops in passing compiler flags to disable
safety in the C code you used with cgo. Maybe that's not true for all
systems, but it seems like a weak argument against cgo.

2) Bugs in the runtime / compiler are potentially problematic. Nobody
has claimed them to be bug-free. The only suggestion has been that the
benefits to the lack of these protections outweigh the costs of
implementing them. Otherwise the assumption is indeed that there
aren't any bugs. But no claim is made. This was also acknowledged and
clarified earlier in this thread.

You may disagree with the severity and justification of both points,
but that does not justify such heinous misrepresentation of the
"community" response. This is even ignoring the fact that the response
you have mischaracterized comes from the input of select individuals
and is absolutely not representative of the entire group's view. This
is actually rather offensive.

> Best,
> ~Alex

Please retract and / or re-word your summation of this thread for truthfulness.

--dho

[Kyle Lemons]

It's not the standard output that is being exploited, it's the use of a format string.  I encourage you to try to exploit a format string in pure Go.  I think you will find that, unless you write your own using unsafe, it will be difficult if not impossible to achieve.

[Kyle Lemons]

Okay, sorry, it's a buffer overflow.  Either way, I think you'll find it hard to exploit in pure Go without unsafe.

[Ian Lance Taylor]

Alex <awr...@gmail.com> writes:

> I think we've been on different wavelengths about something for a while. My
> post was not on the ease of finding vulnerabilities in Go - it was on the
> ease of turning them into working exploits. My claim is that this process
> is easier in Go because it does not have many of the protections that come
> standard in C binaries.

Yes, I understood that to be your claim. And, in fact, I agree with
you: given a vulnerability, it is easier to turn it into a working
exploit in Go than in C.

> I acknowledge that it is hard (perhaps impossible) to
> find vulnerabilities in pure Go binaries - one would probably have to find
> a bug in the Go runtime itself. But that's just it - by eschewing common
> protections, Go is widening the class of bugs (in its runtime) that could
> turn into viable exploits.

Correct.

> Regardless, the Go community's response is quite clear: since Go does not
> have vulnerabilities, protections are unnecessary.

Incorrect. I have not seen anybody make that statement.

The statement I have seen is a different one. Every security tradeoff
has costs and benefits that must be weighed. The costs and benefits are
different for C programs and Go programs. Any decision about changes to
improve security most consider both the costs and the benefits. In your
messages, you have only considered the benefits, not the costs.

Ian

[Alex]

Ok, I have been somewhat unfair and I'm sorry.

There are definitely people in the Go community who recognize the benefit of adding protections to Go. 

It appears, however, that most people feel that vulnerabilities are too unlikely to be worth the effort/complexity. This is still a statement I disagree with, but I'm not up to pushing the Go community about it.

Best,

~Alex

[Jesse McNelis]

On Thu, May 3, 2012 at 1:36 PM, Alex <awr...@gmail.com> wrote:
> It appears, however, that most people feel that vulnerabilities are too
> unlikely to be worth the effort/complexity. This is still a statement I
> disagree with, but I'm not up to pushing the Go community about it.

There are vulnerabilities in Go code. It's pretty much guaranteed in
any large amount of code that there will be security issues. But they
are very unlikely to be buffer overflows, format string attacks etc.
They are far more likely to be DOS attacks, SQL injections, unescaped
shell commands, open proxies, weak crypto, insecure file permissions,
vulnerable race conditions or any of the other millions of high level
attacks.

Buffer overflows were such a great problem in C because every person
writing C could introduce them, millions upon millions of lines of C
written by people at all different skill levels leads to a real
problem. The surface area for buffer overflows in most Go programs is
just the runtime, 20kloc of well written C written by experts.

If there are buffer overflows in the Go runtime it's cheaper and
easier to fix them when they come up than to attempt to prevent them
now since, from a security perspective, that time would be better
spent auditing Go code for the endless list of high level security
issues.


--
=====================
http://jessta.id.au

[Paulo Pinto]

Yeah, C and C++ have brought upon us a legacy of exploit
possibilities.

Only when they get replaced by something better, we can rest assured
the lower
layers of our OS are in a better state.

Surely you can still try to modify the generated code, but that is
beyond the control
of the compiler.

--
Paulo

On May 3, 8:37 am, Jesse McNelis <jes...@jessta.id.au> wrote:

[Mario Vilas]

I'm a bit confused.

On May 2, 9:55 pm, "Devon H. O'Dell" <devon.od...@gmail.com> wrote:
> 2) Bugs in the runtime / compiler are potentially problematic. Nobody
> has claimed them to be bug-free. The only suggestion has been that the
> benefits to the lack of these protections outweigh the costs of
> implementing them. Otherwise the assumption is indeed that there
> aren't any bugs. But no claim is made. This was also acknowledged and
> clarified earlier in this thread.

On May 2, 7:00 pm, "Thomas Bushnell, BSG" <tbushn...@google.com>
wrote:

[David Symonds]

On Wed, May 9, 2012 at 5:56 PM, Mario Vilas <mvi...@gmail.com> wrote:

> I'm a bit confused.

The two quotes you posted are consistent.

The first is making a statement about the particular implementations
of the Go tools. They were written by people (well, as people-like as
Ken and Russ are), and those people can make mistakes. Those mistakes
can lead to programs behaving in a way that the language spec does not
permit.

The second is making a statement about Go (the language), and that, by
design, typical buffer overflows are impossible because the runtime
does bounds checking, as compared to C where, by design, there is no
bounds checking done. Thus a programmer error in C can lead to a
serious security problem (namely, arbitrary memory access), whereas in
Go it will only lead to the program panicking. (bounds checking is an
example)


Dave.

[Kyle Lemons]

To expand a bit: Go is not necessarily a more secure language than any other; just because you can't overflow a buffer doesn't mean you can't induce a race condition to get around some security check or DoS a server or inject SQL or perform XSS or any number of other attacks.  It strikes me as a more fruitful venture to work on developing easy ways to mitigate these hard security problems than to work on hardening the Go runtime against, for instance, exploits leveraging C code included with cgo or mistakes made with "unsafe".

[toxicnaan]

Okay, Interesting posts, but perhaps if no one can agree what the correct security stance of the language should be or even if a runtime can ever be secure, etc etc etc etc??????

All these C botches, ASR, stack protectors , and the other nonsense, it kinda works, but it's all patches on patches because you can't change direction… it C…love it or hate it!, you stuck with all it's short comings (assembly for newbies as we call it).

golang is/was a chance to rethink a system language , and architecturally 'code out' these kind of security problems, rather than a band aid there, and a patch there.

perhaps a simple compiler warning when you link to a potentially security prone language, thus if f you do link to memory exploitable language , golang can always give A WARNING!!!!….

if things do blow up in your face….

'well we did WARN you' ;-) then we can blame the explosion on you. neat huh!

ho ho ho.

I do believe that there's a place for a golang 'secure coding manual', within go's documentation, there are ways to code securely and it's disappointing that most coders learn secure coding last….. when it should be touched on from day one….. , okay the runtime may have solved a number of issue where C can be dangerous, but there's more to secure coding than just relying on what the runtime enforces. I've seen supposedly 'secure' code in many 'safe' languages that would be exploited just as east as input driven, data dependent, memory execution exploits :-) 

There's an awful lot research been done on why programs fail, and fail in such a way that can be exploited in some way. It would be great if golang could be as pragmatic about security than it is about language design, and try and enforce some this research either through changes in the language, code quality, or policy enforcement… some times it's just a set of 'best practices' for coders to follow can suffice!! :-) 

Secure code, is usually better running code, better optimised code, and better understood code and lot less Klocs of crap…. and that's why coders who can do it are very expensive … :-)

at the end of the day, your still running golang on top of millions of lines of C anyway (the kernel + OS + libs) …… I look forward to the golang kernel and OS soon :-)

Right golang dragons, gear up your smoking nostrils, and prepare to douse me buckets of flames!!…. arrrrggggghhh been nice knowing you.

Lee

[Kyle Lemons]

On Wed, Jan 15, 2014 at 7:31 AM, toxicnaan <toxi...@gmail.com> wrote:

On Wednesday, 9 May 2012 16:52:38 UTC+1, Kyle Lemons wrote:

On Wed, May 9, 2012 at 2:20 AM, David Symonds <dsym...@golang.org> wrote:

On Wed, May 9, 2012 at 5:56 PM, Mario Vilas <mvi...@gmail.com> wrote:

> I'm a bit confused.

The two quotes you posted are consistent.

The first is making a statement about the particular implementations
of the Go tools. They were written by people (well, as people-like as
Ken and Russ are), and those people can make mistakes. Those mistakes
can lead to programs behaving in a way that the language spec does not
permit.

The second is making a statement about Go (the language), and that, by
design, typical buffer overflows are impossible because the runtime
does bounds checking, as compared to C where, by design, there is no
bounds checking done. Thus a programmer error in C can lead to a
serious security problem (namely, arbitrary memory access), whereas in
Go it will only lead to the program panicking. (bounds checking is an
example)

To expand a bit: Go is not necessarily a more secure language than any other; just because you can't overflow a buffer doesn't mean you can't induce a race condition to get around some security check or DoS a server or inject SQL or perform XSS or any number of other attacks.  It strikes me as a more fruitful venture to work on developing easy ways to mitigate these hard security problems than to work on hardening the Go runtime against, for instance, exploits leveraging C code included with cgo or mistakes made with "unsafe".

Okay, Interesting posts, but perhaps if no one can agree what the correct security stance of the language should be or even if a runtime can ever be secure, etc etc etc etc??????

I don't understand your point.  Go is designed to be a memory safe language, not a "secure" language.  Security is another dimension entirely.  There's also the trust problem, and the fact that there will always be bugs in the compilers and runtime.

 

All these C botches, ASR, stack protectors , and the other nonsense, it kinda works, but it's all patches on patches because you can't change direction… it C…love it or hate it!, you stuck with all it's short comings (assembly for newbies as we call it).

golang is/was a chance to rethink a system language , and architecturally 'code out' these kind of security problems, rather than a band aid there, and a patch there.

Go does a lot to help you.  The standard templating package for HTML includes contextual autoescaping, for instance, to help guard against XSS.  The standard HTTP package has some knobs you can tune to defend against certain classes of attacks.  There's nothing the language can do to make your binary secure; it can provide tools and libraries, but you still have to decide how much time and effort you can afford to devote to securing your application.  The calculus is different for every application out there.

 

perhaps a simple compiler warning when you link to a potentially security prone language, thus if f you do link to memory exploitable language , golang can always give A WARNING!!!!….

The go compiler doesn't do warnings.  It's true that some libraries might use cgo and not tell you about it, but you can always set CGO_ENABLED=0 if you want to stick with pure-go everything.

 

if things do blow up in your face….

'well we did WARN you' ;-) then we can blame the explosion on you. neat huh!

ho ho ho.

I do believe that there's a place for a golang 'secure coding manual', within go's documentation, there are ways to code securely and it's disappointing that most coders learn secure coding last…..

Security is an application-specific thing, by nature.  Should we really have a document explaining all of the nasty ways someone can exploit an integer overflow?  I don't think it's even possible.  We document what the semantics of an int will be so you can know how an overflow will behave, and let you decide if you care and how to address it.  The same goes for pretty much everything else.

when it should be touched on from day one….. , okay the runtime may have solved a number of issue where C can be dangerous, but there's more to secure coding than just relying on what the runtime enforces. I've seen supposedly 'secure' code in many 'safe' languages that would be exploited just as east as input driven, data dependent, memory execution exploits :-) 

There's an awful lot research been done on why programs fail, and fail in such a way that can be exploited in some way. It would be great if golang could be as pragmatic about security than it is about language design, and try and enforce some this research either through changes in the language, code quality, or policy enforcement… some times it's just a set of 'best practices' for coders to follow can suffice!! :-) 

 

Secure code, is usually better running code, better optimised code, and better understood code and lot less Klocs of crap…. and that's why coders who can do it are very expensive … :-)

 

at the end of the day, your still running golang on top of millions of lines of C anyway (the kernel + OS + libs) …… I look forward to the golang kernel and OS soon :-)

The old adage can probably be just as accurately rephrased as "the only completely secure program is a program that was never written".

 

Right golang dragons, gear up your smoking nostrils, and prepare to douse me buckets of flames!!…. arrrrggggghhh been nice knowing you.

Lee

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[minux]

On Wed, Jan 15, 2014 at 10:31 AM, toxicnaan <toxi...@gmail.com> wrote:

Secure code, is usually better running code, better optimised code, and better understood code and lot less Klocs of crap…. and that's why coders who can do it are very expensive … :-)

This statement is not true. Most security measurements decrease code performance rather than

increase it. Also, they also tend to increase the code and make the code less understandable.

(compare bytes.Equal(a, b) and a constant time implementation of bytes.Equal(a, b))

One simple example can illustrate the fact, side channel attacks are more and more common

nowadays, but they are very difficult to defend against (we don't even know all possible side

channels yet!).

One way to mitigate timing side channels is to make every possible operation take the same

time, so they're indistinguishable from attackers, and that, by definition, will make code run at

the worst possible speed.

I think you're asking too much from the language you use. I don't think the any practical language

can provide you the security you want (if there is one that is also practical, then we probably

have already switched to that language).

[Ian Lance Taylor]

On Thu, Jan 16, 2014 at 7:36 AM, toxicnaan <toxi...@gmail.com> wrote:
>
> So how can a system language not have security at least in mind? I wonder
> what the core team think of this? Does golang have a security officer? What
> steps are being made to ensure the runtime is bug free? Can't we help
> developer by at least warning them if they are going to do things are could
> be unsafe.

I'm not sure what to make of your note, which does not seem to be a
response to what minux actually said. To be clear, Go does have
security in mind, to the extent that that is possible for a language
rather than an actual program. If you do not import the unsafe
package and you avoid race conditions, Go does not permit buffer
overflows, invalid pointers, writable executable memory, etc.

Ian

[minux]

On Thu, Jan 16, 2014 at 10:36 AM, toxicnaan <toxi...@gmail.com> wrote:

It's heating up in here,  flame proof suit zipped up… From both post I see , security is hard, lets give up.

The last post of mine just pointed out your statement that secure code is more optimized,

more easily understandable is simply not true, how am I implying that we shouldn't do

security because it's hard (it's indeed very hard, and you can never be sure it's secure

just like that you can never be sure your code is bug free). 

You'll never be secure …  oh dear.. 

In some sense, yes. 

Lets not think about security..is that even ethical? should you be allowed to write production code for say gmail, if your not at least 'thinking' about how to make

your code more secure?

Using Go is at least less susceptible to common security problems like buffer overflows

in C, and to a less extend, C++. There are a few truly security related issues for Go in

the past, and I think the Go Team did a very good job of fixing them promptly. 

If you writing a few games at home, them perhaps security is not a big thing for you, but go is a systems language, I've seen my fair few of 'systems', they are big, sprawling, contain very important and valuable data. 

So how can a system language not have security at least in mind? I wonder what the core team think of this? Does golang have a security officer? What steps are being made to ensure the runtime is bug free? Can't we help developer by at least warning them if they are going to do things are could be unsafe. 

That's why the package unsafe is called unsafe, and we have world-class race detector in Go.

Also, security auditing is not as easy as you might imagine. I don't think the compiler can

(at least in current state of the art) warn you whenever you have done anything insecure.

I'm not sure if you two are part of the core development go team, but i certainly hope that the core developers think about this stuff. 

What stuff? 

kyle/minux was are you use cases for deploying go into production? Would you be interested in writing secure code in the future? Have you written secure code in the past? Security is really about context , i'm trying to gauge where you fit in to this. 

I firmly believe if the (standalone) program binary into the wrong hands, all bets are off.

So networking services, security is entirely different than client side security. 

Why did we goto the moon?

Not because it easy , because it was hard. (or did they fake the moon landings, tinfoil hat current in place).

What's you point here?

[Niklas Schnelle]

Since this discussion has been a bit heated and lacking in respect at times I'd like to add a bit of positive thinking here. 

First I'd like to say that as someone following most of this list quite closely I have seen plenty of outstanding engineering

especially when it comes to security. I really have a hard time even beginning to see where the Go team should be thinking too

little about security but let me give a few examples where Go and it's standard library provide world-class engineering for security related issues:

-  The Go language specification incorporates a lot of security and quality assurance measures

   - Mandatory bounds checking

   - An extremely strict type system that prevents a lot of programming errors

   - Strong restrictions on pointers barring the clearly named unsafe package

   - Go's clear coding guidelines, more or less mandatory formatting standard and simplicity make 

     it one of the best languages to read other peoples code and thus increases reviewability

- The standard Go implementation does much higher than industry standard QA

   - It's developed with extensive review efforts and we routinely see renowned experts putting a lot of serious effort  into this

   - There recently was quite an effort to do static analysis on the C parts of the runtime on top of the fact that every change is reviewed and it does little interaction with

     user supplied input

   - The implementation of maps has seen a lot of effort to thwart exploitable key collisions that would make web applications

     easily susceptible to DDoS attacks and such attacks have been problematic for several other languages in the past

   - I have already seen efforts to make less of the memory exectuable e.g. when the need to generate code for interfaces at runtime was removed

- The standard library especially tries to implement stuff that is hard to extremely hard to get right and a lot of that is security related
  - Go has by far the best crypto library in any standard distribution of a programming language and agl it's primary developer

    is a well known expert in the field. To the point where he was just recently mentioned by the security researchers of the well known BEAST attack as

    the single person in the crypto community who managed to come up with a proper fix for that attack as well as independently discovering it. Actually that's

    a bit of a funny story because one of the proofs of his suspicion for a timing leak was actually a comment in the source code of Go's SSL implementation.

  - Go unlike many other systems comes with a non exponential runtime regex library. Regex exponential runtime DDoS attacks have hit several major web applications in the past

  - The standard library provides security critical parsing functionality for commonly problematic input like HTTP, escaping in HTML, SMTP and many more
  - The crypto library does not only provide several implementations of secure ciphers but also implements well designed primitives to implement crypto algorithms 

    with constant time operations

  - The standard library is reviewed just as rigorously as the runtime

Maybe it makes sense to look at whether security for C code running in cgo can be improved but stating that Go isn't developed with security in mind

is just plain wrong and doesn't do it's developers justice. On the contrary I think the developers have done a great job so far and if they keep this up

I have no doubt in my mind that Go is and will be one of the most secure foundations to implement secure software systems. 

[Dan Kortschak]

On Thu, 2014-01-16 at 13:57 -0800, Niklas Schnelle wrote:
> - Go has by far the best crypto library in any standard distribution of a
> programming language and agl it's primary developer
> is a well known expert in the field. To the point where he was just
> recently mentioned by the security researchers of the well known BEAST
> attack as
> the single person in the crypto community who managed to come up with a
> proper fix for that attack as well as independently discovering it.
> Actually that's
> a bit of a funny story because one of the proofs of his suspicion for a
> timing leak was actually a comment in the source code of Go's SSL
> implementation.

Yeah, it's pretty clear that he had a good idea of what was going on
well before the publication of the attack. When I saw his comment in the
source I was amazed.

https://groups.google.com/d/topic/golang-nuts/HF5O5vAKRcQ/discussion

[Anthony Martin]

Niklas Schnelle <niklas....@gmail.com> once said:
> Go has by far the best crypto library in any standard distribution of
> a programming language and agl it's primary developer is a well known
> expert in the field. To the point where he was just recently mentioned
> by the security researchers of the well known BEAST attack as the single
> person in the crypto community who managed to come up with a proper fix
> for that attack as well as independently discovering it.

This was the Lucky 13 attack by AlFardan and Paterson, not BEAST.
Mitigations for the latter are vastly simpler (I added the fix to
crypto/tls in ~10 lines).

Adam's fix for Lucky 13 was for OpenSSL and never made it's way
into the Go standard library.

> Actually that's a bit of a funny story because one of the proofs of his
> suspicion for a timing leak was actually a comment in the source code
> of Go's SSL implementation.

Indeed, that was pretty amazing.

Anthony

[toxicnaan]

Thanks for this, that's exactly the sort information i was looking for :-) :-) :-) . For the record i never said that the go team didn't have security in mind, it was just a question to see what their security stance is , i think your post pretty well sums up the effort they are making. A very Positive and concise summary of go security movements at last! :-)

Although the go runtime does protect you from nasty C like memory exploits, I still believe that a secure coding 'cook book' would be indeed a great addition to the standard go documentation. I have seen many programmers learn languages, then learn secure coding practises as after thought (if at all). It would be a great addition,  if your going to learn go, then learn secure coding at practises at the same time along with everything else :-).

Hopefully go programmers won't make all the same mistakes all over again, something akin to the film 'ground hog day'.  

Hopefully go will turn from 'let's have another go at language design' but to 'go do things the go way!'  :-)

go go go!

and don't worry go people, everyone makes mistakes, that's why projects like metaspolit exist, just to remind us to. 'code safe' ;-)

Cheers,

flame suit off!

[andrey mirtchovski]

> I still believe that a secure coding 'cook book' would be indeed a great
> addition to the standard go documentation.

how does "do not import package unsafe; use the race condition
instrumentation on your code frequently" sound?

[andrey mirtchovski]

> how does "do not import package unsafe; use the race condition
> instrumentation on your code frequently" sound?

and "do not ignore errors", of course :)

[Aaron Wood]

Hi Russ,

This is a rather old comment of yours but I was curious about a few things you said. I agree that Go does take care of a lot of these problems (and most other high-level languages try to too) so I'm curious as to why Go has exposed a build option for generating position independent executables (buildmode=pie). This is something I use and think it is a great feature to have exposed but if the goal was to address most of this stuff at the language level then what was the reason for exposing this to users of Go? I don't think most other high-level languages let you control this setting...although I guess most of them don't really compile down to a native binary in the end.

-Aaron

On Monday, April 30, 2012 at 10:21:06 PM UTC-4, Russ Cox wrote:

On Mon, Apr 30, 2012 at 21:51, David  Leimbach <lei...@gmail.com> wrote:
> I was just playing with pointers yesterday, trying to understand a memory
> issue I thought I was having (me not remembering the memory model mostly).
>
> I came to the conclusion that pointers were showing up at very much the same
> memory addresses each time, and that perhaps randomizing this would be a
> good idea.

Address space randomization is an OS-level workaround for a
language-level problem, namely that simple C programs tend to be full
of exploitable buffer overflows.  Go fixes this at the language level,
with bounds-checked arrays and slices and no dangling pointers, which
makes the OS-level workaround much less important.  In return, we
receive the incredible debuggability of deterministic address space
layout.  I would not give that up lightly.

Russ

[Ian Lance Taylor]

On Thu, Dec 22, 2016 at 2:09 PM, Aaron Wood <aaron...@gmail.com> wrote:
>
> This is a rather old comment of yours but I was curious about a few things
> you said. I agree that Go does take care of a lot of these problems (and
> most other high-level languages try to too) so I'm curious as to why Go has
> exposed a build option for generating position independent executables
> (buildmode=pie). This is something I use and think it is a great feature to
> have exposed but if the goal was to address most of this stuff at the
> language level then what was the reason for exposing this to users of Go? I
> don't think most other high-level languages let you control this
> setting...although I guess most of them don't really compile down to a
> native binary in the end.

Go exposes supports generating PIE because an increasing number of
operating system distros require it, or at least use it by default
(for example, I believe that non-Java Android apps must be PIE). The
distros do this because the distro maintainers feel that it addresses
problems with programs written in C/C++. Since from the operating
system perspective Go programs are indistinguishable from C/C++
programs, Go needs to be able to generate PIE just as C/C++ do.

As you say, discussing PIE is only relevant for a language that
compiles into a native binary.

Ian

> On Monday, April 30, 2012 at 10:21:06 PM UTC-4, Russ Cox wrote:
>>
>> On Mon, Apr 30, 2012 at 21:51, David Leimbach <lei...@gmail.com> wrote:
>> > I was just playing with pointers yesterday, trying to understand a
>> > memory
>> > issue I thought I was having (me not remembering the memory model
>> > mostly).
>> >
>> > I came to the conclusion that pointers were showing up at very much the
>> > same
>> > memory addresses each time, and that perhaps randomizing this would be a
>> > good idea.
>>
>> Address space randomization is an OS-level workaround for a
>> language-level problem, namely that simple C programs tend to be full
>> of exploitable buffer overflows. Go fixes this at the language level,
>> with bounds-checked arrays and slices and no dangling pointers, which
>> makes the OS-level workaround much less important. In return, we
>> receive the incredible debuggability of deterministic address space
>> layout. I would not give that up lightly.
>>
>> Russ
>

> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts...@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

[Aaron Wood]

Interesting, thanks for the info Ian. Other than the OS and distro requirements, is there any real benefit from generating PIE binaries with Go? There must be some vector that it protects again, even though most things are already bounds-checked. The only situation I can think of is if you're calling C from Go. There must be other rare, obscure ways to exploit overflows in code generated from Go, no?

[Ian Lance Taylor]

On Thu, Dec 22, 2016 at 2:39 PM, Aaron Wood <aaron...@gmail.com> wrote:
>
> There must be other rare, obscure ways to exploit
> overflows in code generated from Go, no?

I sure hope not. If there are any, they are bugs that should be fixed.

Ian