Fuzz-a-thon!

[Damian Gryski]

By now hopefully everybody has heard about Dmitry Vyukov's go-fuzz: https://github.com/dvyukov/go-fuzz

This randomized testing tool has found a huge number of bugs in the standard library.  But it has been used in relatively few third-party packages.

In the week or so leading up to the release of 1.5, we as a community should turn out attention to the rest of the ecosystem.  Go is already known for reliability due to the ingrained habit of error checking.  Gracefully handling unexpected inputs is another area in which we ought to excel.

Getting started is straight forward -- the README file lays out the steps: https://github.com/dvyukov/go-fuzz/blob/master/README.md

You can start with your own packages, a dependency you use, or just pick a random one from https://github.com/avelino/awesome-go .  Good candidates are file formats, protocols, and parsers.  Basically anything that's dealing with user input in some form.

For the bugs you find, file a bug to the repository, preferably with a fix. And don't forget to file one to update go-fuzz's trophy case.

It doens't take a lot of time, either.  Dmitry has obviously expended lots of CPU time across all the packages he tested, but even 30 minutes of fuzzing on a laptop is enough to generally shake out a few crashers.

Tweet your successes tagging it with #golangfuzz

Damian

[Kamil Kisiel]

I think the Gorilla toolkit packages are good candidates for fuzz testing. Particularly sessions, securecookie, and schema. I started with fuzz testing on securecookie a while back but unfortunately I'm pretty busy with other work at the moment. If someone is interested in working on this I'm willing to lend as much help as I can manage, and write patches for any bugs that turn up. 

[Kiki Sugiaman]

I'm playing with securecookie as I'm typing this. Still trying to make
sense of the output....

[Damian Gryski]

The first thing (more or less) that securecookie does in Decode() is to check the signature. You will need to dodge this check in order to fuzz, since a random input will almost certainly fail the cryptographic MAC.

Damian

[Kiki Sugiaman]

I did 3 runs with this code:
http://play.golang.org/p/pqtS1V53PL

The first 2 went out of memory and crashed my comp. I have ~5GB free mem
with browsers/editor/mail-client/terminals open.

The 3rd:
2015/07/28 04:59:03 slaves: 8, corpus: 628 (0s ago), crashers: 1,
restarts: 1/9703, execs: 37899999 (33869/sec), cover: 1852, uptime: 18m39s

I'll upload the dump somewhere in a minute.

[Kiki Sugiaman]

I still can't find what ate the ~5GB of ram and caused the crash.

The dump:
https://goo.gl/BdP6ZA

$ go version
go version go1.5beta2 linux/amd64

$ free -m ## (not running fuzz)
total used free shared buffers cached
Mem: 7661 5343 2317 938 115 3164
-/+ buffers/cache: 2063 5598
Swap: 0 0 0

[Kamil Kisiel]

Weird, I ran a bunch of the crashers from your dump and they all seem to be fine...

In the output file it shows them as either hanging or running out of memory, but I have no such problem running them here.

[Kiki Sugiaman]

When the first two crashes happened, I was away from the comp so not
sure what happened. I did another run with nothing else open for about 2
minutes and stared at the output from top. At 1 point, the memory
consumption went over 80% (of 8GB). So it was probably just memory
consumption rather than a bug.

[Damian Gryski]

On Monday, July 27, 2015 at 3:58:04 PM UTC+2, Damian Gryski wrote:

By now hopefully everybody has heard about Dmitry Vyukov's go-fuzz: https://github.com/dvyukov/go-fuzz

Some successes:

CloudFlare just had a great blog post on their work fuzzing https://github.com/miekg/dns : https://blog.cloudflare.com/dns-parser-meet-go-fuzzer/

In addition, bugs have been fuzzed in InfluxDB, CockroachDB, Caddy, Apcera's gnatsd, and Google's Open Location Code.  The full list of trophies in https://github.com/dvyukov/go-fuzz/blob/master/README.md keeps growing.

If you need some help getting started, please ask either here, IRC, Gopher Slack,  or Twitter under #golangfuzz , #golang, or even to me directly at @dgryski.

Damian

[Dmitry Vyukov]

Hi!

Great effort!

I am also ready to help. In particular if you have any issues with go-fuzz itself, or with corpus collection or writing of the Fuzz function.

 

[bep]

I have revistied some Fuzzers I wrote for Hugo a while ago. A couple of points of notice:

1) All the crashes from the Go template related parts are gone thanks to bugfixes in the stdlib in Go 1.5.
2) go-fuzz now found some issues in some other parts of the program, which must mean that the fuzzer itself has gotten better (will report the issues to the README when I'm done)

One question, though:

For crashes that happens in third-party libraries, I create a bug report and or a pull request if I can fix it myself. But this can take time.

I have one such nasty issue which creates a lot of crashes on the form of a `log.Fatalf(...)` which does a `os.Exit(1)`. I understand how to exclude panics, but this failure kind of blocks further fuzzing in this area of code. Any tip on how to put a temporary exclude on such an issue?

bep

[bep]

Thinking about the last point: I'll just fix the os.Exit issue and create a PR -- then handle the panic.

bep

[Dmitry Vyukov]

On Sat, Aug 8, 2015 at 4:08 PM, bep <bjorn.eri...@gmail.com> wrote:
> I have revistied some Fuzzers I wrote for Hugo a while ago. A couple of
> points of notice:
>
> 1) All the crashes from the Go template related parts are gone thanks to
> bugfixes in the stdlib in Go 1.5.
> 2) go-fuzz now found some issues in some other parts of the program, which
> must mean that the fuzzer itself has gotten better (will report the issues
> to the README when I'm done)

cool

> One question, though:
>
> For crashes that happens in third-party libraries, I create a bug report and
> or a pull request if I can fix it myself. But this can take time.
>
> I have one such nasty issue which creates a lot of crashes on the form of a
> `log.Fatalf(...)` which does a `os.Exit(1)`. I understand how to exclude
> panics, but this failure kind of blocks further fuzzing in this area of
> code. Any tip on how to put a temporary exclude on such an issue?

If you fix the bug in the code that leads to log.Fatal call, then it
should resolve the problem :)
Or you can temporary replace log.Fatal with a panic.

[Damian Gryski]

On Saturday, August 8, 2015 at 7:37:50 PM UTC+2, Dmitry Vyukov wrote:

If you fix the bug in the code that leads to log.Fatal call, then it
should resolve the problem :)
Or you can temporary replace log.Fatal with a panic.

gofmt -w -r 'log.Fatalf -> log.Panicf'  *.go

:)

Damian

 

[bep]

Obvious answers followed by smiley faces aren't very helpful :-)

I have opened a PR for the library in question replacing the Fatalf with a Panic, so that solves my question.

bep

[Damian Gryski]

Here's a quick tutorial on using go-fuzz to find two crashes in github.com/arolek/ase

https://medium.com/@dgryski/go-fuzz-github-com-arolek-ase-3c74d5a3150c

[Dmitry Vyukov]

Hi Damian,

Great tutorial!

I've added it and the dns blog to go-fuzz readme (as you suggested on
twitter some time ago):
https://github.com/dvyukov/go-fuzz/blob/master/README.md#external-articles

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/4PmyYvcnpIs/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Paul Graydon]

I cut an issue a week or so ago, but would it be possible for you to pick an open source licence for the software? Without having one you're likely to stop many people from using your software. I know I certainly can't at my employer, and I wouldn't be surprised if the Go team at Google had similar restrictions.

[Dmitry Vyukov]

Done. Added Apache 2 license.

On Wed, Aug 19, 2015 at 3:41 AM, Paul Graydon <paulg...@gmail.com> wrote:
> I cut an issue a week or so ago, but would it be possible for you to pick an open source licence for the software? Without having one you're likely to stop many people from using your software. I know I certainly can't at my employer, and I wouldn't be surprised if the Go team at Google had similar restrictions.
>

[Иван Перевезенцев]

Huh, that was quickly :)

пятница, 21 августа 2015 г., 20:52:43 UTC+3 пользователь Dmitry Vyukov написал:

[Jérôme LAFORGE]

Hello all,
Since one week, I did lot of fuzzing on our internal network stack and that allows me to find couple af nasty bugs (infinite loop with memory leak, index out of range error and some corrupt data with bad use of slice).
So first of all, I want to thank you for this great golang variant of AFL.

I put more and more stuff to test into func Fuzz(data []byte) for increasing my fuzing scope on this network stack.
My fuzz test does lot modifications or append on the byte array that is passed with func Fuzz(data []byte)

I notice that :
- If I don't copy the data []byte, I have high rate (~1/500) but without crasher
- If I copy* the data []byte, I have expected rate (~1/10000)

* : the copy is done like this :

func Fuzz(roData [] data) int {
rwData := make([]data, len(roData))
copy(rwData, roData)
...

}

Should I consider this original data as read only data or this behavior could hidden another nasty bug?

Thx in adv
Regard
Jérôme

[Dmitry Vyukov]

On Fri, Oct 30, 2015 at 8:05 PM, Jérôme LAFORGE
<jerome....@gmail.com> wrote:
> Hello all,
> Since one week, I did lot of fuzzing on our internal network stack and that
> allows me to find couple af nasty bugs (infinite loop with memory leak,
> index out of range error and some corrupt data with bad use of slice).
> So first of all, I want to thank you for this great golang variant of AFL.

Sounds great.
Thanks!
Note that go-fuzz is better than AFL in some aspects.

> I put more and more stuff to test into func Fuzz(data []byte) for increasing
> my fuzing scope on this network stack.
> My fuzz test does lot modifications or append on the byte array that is
> passed with func Fuzz(data []byte)
>
> I notice that :
> - If I don't copy the data []byte, I have high rate (~1/500) but without
> crasher
> - If I copy* the data []byte, I have expected rate (~1/10000)
>
> * : the copy is done like this :
> func Fuzz(roData [] data) int {
> rwData := make([]data, len(roData))
> copy(rwData, roData)
> ...
>
> }
>
> Should I consider this original data as read only data or this behavior
> could hidden another nasty bug?

I would expect that modifying the data should be fine. Of course, if
you don't do it after Fuzz function returns.
Please file a bug with reproducer at:
https://github.com/dvyukov/go-fuzz/issues/new

[Jérôme LAFORGE]

I would expect that modifying the data should be fine. Of course, if
you don't do it after Fuzz function returns.
Please file a bug with reproducer at:
https://github.com/dvyukov/go-fuzz/issues/new

Ok, I try to make reproducer, and fill a bug report.
Many thx for this great tool.