Looking for some nicely broken image files (gif, png, and jpeg)

[Dan Sugalski]

Specifically ones that are compatible with the Go license. The more broken they are the better. Bonus points for ones that trigger degenerate behaviour in the decoding libraries.

The tldr here is that I'm making a bunch of changes to the basic image libraries to add in metadata support. Since I'm in there anyway I figure I may as well add in some safety measures against files that consume an unreasonable amount of time and/or space to decode. (Or encode if anyone's got an image.Image that triggers bad behaviour) Having a good suite of known-bad files to check against will be handy to make sure the code actually works which is, y'know, kinda nice.

(I am aware of some bad files kicking around the web, and I've grabbed some for testing, but none I've run across so far have licenses on them that'd allow me to toss them up on github as part of the tests for this stuff. Hence the asking around)

[Jake Montgomery]

This may be slightly tangential, but this seems like the kind of code that would benefit greatly from fuzz testing, like  go-fuzz. For this kind of code, go-fuzz can really help harden it against bad, malformed and malicious input. I have used it to really good effect. It requires thoughtful choices for the initial corpus, and a machine where you can let it run for a long time, preferable just leave it running for weeks.

[Dan Sugalski]

Fuzz testing is absolutely reasonable and something I want to set up once I've got the first pass of code done. 

[Dan Sugalski]

Also it's probably easiest if folks have these handy to just propose an addition to the test repository at https://github.com/drswork/image/tree/master/testdata since sending busted images via email is fraught with all sorts of exciting peril.

[t hepudds]

Hello Dan,

I would definitely echo the fuzzing suggestion from Jake.

In addition, in your quest to find broken images, I would suggest grabbing the images from the dvyukov/go-fuzz corpus, which has a bunch of images that are already broken in "creative" ways for you based on the coverage-guided fuzzing that has happened these in the past. For example, here are a bunch of images from there:

  https://github.com/dvyukov/go-fuzz-corpus/tree/master/jpeg/corpus

  https://github.com/dvyukov/go-fuzz-corpus/tree/master/png/corpus

  https://github.com/dvyukov/go-fuzz-corpus/tree/master/gif/corpus

You could just use those as inputs to your tests now, even if you don't want to fuzz anything yet.  There are also other public fuzzing corpuses for other languages or image libraries that you can find to run your tests against.

That go-fuzz corpus repository also contains sample fuzzing functions for jpeg, png, and gif so that you don't have to start from scratch to fuzz your changes.

Also, some of those fuzzing functions from the go-fuzz corpus have also somewhat recently been moved into the Go standard library. For example, acln0 moved the png fuzz function from the go-fuzz corpus into the Go standard library. 

I mention that in part because as part of your effort to make sure you are not introducing bugs, you could consider bringing over the jpeg and gif fuzzing functions from the go-fuzz corpus into the Go standard library, which would help exercise your code (including it would then be a ~2 line PR to google/oss-fuzz to pick up continuous fuzzing funded by Google for those new functions, which would give you on-going coverage of your changes).

Finally, one way to fuzz using the fuzzing functions in the standard library (using image/png as the example here) is as follows:

  $ go get -u github.com/dvyukov/go-fuzz/...

  $ go get -u github.com/thepudds/fzgo

  $ fzgo test -fuzz=. image/png

fzgo is a wrapper on top of go-fuzz that provides some additional conveniences, but perhaps more interesting is that fzgo is a simple prototype of how cmd/go could behave under the proposal in #19109:

    #19109 proposal: make fuzzing a first class citizen, like tests or benchmarks 

    https://github.com/golang/go/issues/19109

Sorry for the long post, but I believe you'll get pretty quick payback on the small amount of time it would take to start fuzzing your changes via some of the starting points listed above.

Regards,

thepudds

[t hepudds]

Hello Dan,

Sorry, one quick clarification -- I had misremembered which image packages are already running in google/oss-fuzz. 

Of the three image packages you mentioned, I think it is just gif that is missing currently (and not both gif and jpeg missing):

https://github.com/google/oss-fuzz/blob/master/projects/golang/build.sh#L32

thepudds

[Dan Sugalski]

Ah, awesome, I hadn't done any real looking for fuzzing solutions yet. I'll grab that one and at least use it to start setting up some local testing while I'm hunting around for test data I can include. (I don't think I can grab apache 2 licensed data and toss it into a package with the standard go license)

Separately I'll look at the integration with the standard library. No promises there, but the less I have to maintain myself the better.