Issue 376 in webp: oss-fuzz

[pdk… via monorail]

Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 376 by pdk...@gmail.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376

I'm interested if you're interested in having libwebp integrated into oss-fuzz.

https://github.com/google/oss-fuzz

You don't have to do anything other than give an email address to be notified when bugs are found. (Only project owners can be notified for obvious reasons.)

I've been sporadically running three libwebp fuzz targets for a while now, and recently tried them with the docker-based oss-fuzz system locally, which works well.

--
You received this message because:
1. The project was configured to send all issue notifications to this address

You may adjust your notification preferences at:
https://bugs.chromium.org/hosting/settings

[johannko… via monorail]

Comment #1 on issue 376 by johannko...@google.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c1

There is no distinct template for libwebm but you can use:
https://bugs.chromium.org/p/webm/issues/entry?template=Security

This would be less brittle than relying on specific users.

[johannko… via monorail]

Comment #2 on issue 376 by johannko...@google.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c2

Sorry I misread libwebp as libwebm. I believe the Security template is still the preferred path though.

[pdk… via monorail]

Comment #3 on issue 376 by pdk...@gmail.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c3

The oss-fuzz bot automatically files the bugs on its own bug tracker.

https://bugs.chromium.org/p/oss-fuzz/issues/list

The mail address (which can be multiple) is just to CC project owners to bugs. In particular to security bugs (which are otherwise hidden obviously.)

[jz… via monorail]

Comment #4 on issue 376 by jz...@google.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c4

Thanks for suggesting this. We have code elsewhere running on libfuzzer and coverage in chrome, but additional coverage would be great, with the one caveat that we may need to tune the build to avoid a lot of OOM noise as these environments are often resource constrained. If you want to add me as an owner and if I can make changes then I can add others.

[pdk… via monorail]

Comment #5 on issue 376 by pdk...@gmail.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c5

I noticed that the particular code path used by Chrome (or skia rather) is already covered by many fuzzers, but simple API and animation API perhaps not as much, or maybe not at all. And even in the advanced API there are different paths, like WebPIAppend or WebPIUpdate, decoding in a single or multiple chunks, using internal and external buffer, passing different WebPDecoderOptions and so forth.

I've not noticed any OOM by limiting the image (or canvas) size to 1024**2, and limiting to not too many frames.

Now the question is how do you want to proceed? The fuzz targets, which are just single files, can be either hosted by oss-fuzz, or directly in the libwebp repository. oss-fuzz prefers the later.

[pdk… via monorail]

Comment #6 on issue 376 by pdk...@gmail.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c6

(A fuzz target is just the code that gets fuzzed.) I'm attaching a simple target I've used for the simple API as an example. The function LLVMFuzzerTestOneInput is called repeatedly by the fuzzer.

Attachments:
simple_api.cc 1.2 KB

[jz… via monorail]

Updates:
Labels: -Type-Defect Type-Enhancement
Status: Accepted

Comment #7 on issue 376 by jz...@google.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c7

> I noticed that the particular code path used by Chrome (or skia rather) is
> already covered by many fuzzers, but simple API and animation API perhaps not
> as much, or maybe not at all.

We do have animation API imagio coverage in a non-user facing setup.

> And even in the advanced API there are
> different paths, like WebPIAppend or WebPIUpdate, decoding in a single or
> multiple chunks, using internal and external buffer, passing different
> WebPDecoderOptions and so forth.
>

True, more combinations would be good to have.

> Now the question is how do you want to proceed? The fuzz targets, which are
> just single files, can be either hosted by oss-fuzz, or directly in the
> libwebp repository. oss-fuzz prefers the later.

We could put them in a subfolder in libwebp-test-data to keep the size of the
library repo down. The source file could live in libwebp, then we could hook up
libfuzzer.

[pascal.m… via monorail]

Comment #8 on issue 376 by pascal.m...@gmail.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c8

> We could put them in a subfolder in libwebp-test-data to keep the size of the
> library repo down. The source file could live in libwebp, then we could hook up
> libfuzzer.

Good idea. fuzz targets and extra code seems a good fit for a special subdir in libwebp-test-data.

[pdk… via monorail]

Comment #9 on issue 376 by pdk...@gmail.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c9

By fuzz targets I actually meant the source files. The seed corpus, just a zip for each fuzz target, could go into libwebp-test-data. Or should the fuzz targets also go there? I had planned to make a sub-dir perhaps name fuzzer in libwebp, which could hold the few fuzz targets, and a README to get people set-up who want to try to fuzz the targets themselves. With the latest stable clang released last week, running libFuzzer is easy. (oss-fuzz also uses afl and honggfuzz, which are more difficult to set-up).

Also I notice now that the sample simple API fuzz target I attached was not quite as simple as I thought, using advanced API features.

[pdk… via monorail]

Comment #10 on issue 376 by pdk...@gmail.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c10

Just want to post an update. Rather than going for full integration (like integrating it into the repo with multiple fuzz targets), it's probably better to go step-wise. So I'll submit a first target hosted by oss-fuzz soon, and additional steps can then be made afterwards.

[jz… via monorail]

Comment #11 on issue 376 by jz...@google.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c11

That makes sense, thanks for doing this and keeping us up to date.

[pdk… via monorail]

Comment #12 on issue 376 by pdk...@gmail.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c12

Alright, the first fuzz target is ready to go. I'll wait for an assessment of issue 383 before I submit it.

[pdk… via monorail]

Comment #13 on issue 376 by pdk...@gmail.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c13

The fuzzer should be running now. You can check its status at https://oss-fuzz.com. I have two more fuzz targets queued, plus better integration into libwebp.

To add people to be CC'd to bugs you have to submit a PR against this file. (I can do it too if you list the address here.)

https://github.com/google/oss-fuzz/blob/master/projects/libwebp/project.yaml

With the following addition/modification.

auto_ccs:
- ma...@address.here

[pdk… via monorail]

Comment #14 on issue 376 by pdk...@gmail.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c14

This is how I'd like to proceed. Please let me know if you agree.

I'll add two more fuzz targets. After they've run for 1-2 days without problems, you copy most of the files in the oss-fuzz libwebp directory into a directory somewhere in the libwebp repo. That has the benefit that you don't need to make PRs on GitHub to modify the fuzz targets (or add new). Then I'll make the changes on the oss-fuzz side to reflect it, and that's it.

[jz… via monorail]

Updates:
Cc: pascal.m...@gmail.com jz...@google.com

Comment #15 on issue 376 by jz...@google.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c15

From a maintenance perspective that makes sense to me.

[pdk… via monorail]

Comment #16 on issue 376 by pdk...@gmail.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c16

Please also review issue 385 before I submit additional targets. Thanks.

[jz… via monorail]

Updates:
Blockedon: 386 385 387 388

Comment #17 on issue 376 by jz...@google.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c17

I think everything has been cleared now, it should be all right to transition the bug reporting to oss-fuzz.

[pdk… via monorail]

Comment #18 on issue 376 by pdk...@gmail.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c18

As you've noticed, I submitted the additional targets. That should be all. You should get a few reports for fuzz_demux_api right away. I still had 2-3 bugs pending which I didn't report.

[jz… via monorail]

Comment #19 on issue 376 by jz...@google.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c19

Thanks for coming up with the additional targets, the coverage looks pretty good.

[pdk… via monorail]

Comment #20 on issue 376 by pdk...@gmail.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c20

I notice from the recent commits that a few bugs have rolled in. Unless there are any bugs for the fuzz targets (not libwebp), I think you can copy fuzz* now.

[pdk… via monorail]

Comment #21 on issue 376 by pdk...@gmail.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c21

Do you still plan to copy the targets over to the libwebp repo?

If so, it might be a good idea to also copy the files from the intitial corpus I made (in Dockerfile) to libwebp-test-data. Then the corpus could be made automatically with files from libwebp-test-data. While oss-fuzz expands its corpus automatically, it re-reads (and re-fuzzes) the initial corpus every day.

[jz… via monorail]

Comment #22 on issue 376 by jz...@google.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c22

The response time has been pretty good on pull requests, but it still might be useful to pull them over. It's not a high priority right now since things are working currently.

[pdk… via monorail]

Comment #23 on issue 376 by pdk...@gmail.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c23

If you don't mind I'll go ahead and already change the initial corpus to be made dynamically from libwebp-test-data. Any files added there then get additional regression testing.

I didn't do this initially because there are multi-MB files there which would significantly (catastrophically really) reduce the efficiency the fuzzer. So I'll just limit it to files of a reasonable size like 32KB or so.

Then you can still copy the files I made later. Those are about a dozen <1KB fuzzer-friendly files which cover basically all features.

http://cdn.pwmon.org/oss-fuzz/libwebp/fuzz_seed_corpus.zip

[jz… via monorail]

Comment #24 on issue 376 by jz...@google.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c24

That sounds good to me, thanks for following up. The large file was only meant for performance testing so I think it makes sense to avoid it.

[pdk… via monorail]

Comment #25 on issue 376 by pdk...@gmail.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c25

@yguyon: As you added a target recently, and seem to be in charge of this at Google now, I think you can also replace the MAINTAINER in Dockerfile now with your next PR. It doesn't really matter (as it doesn't give access to anything), but still.

[jz… via monorail]

Updates:
Cc: ygu...@google.com
Status: Fixed

Comment #26 on issue 376 by jz...@google.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c26

Thanks, that makes sense. The state of the code and tests seems stable. We've talked about moving the code around to make it more generally available, but it's a low priority now. We can probably close this bug given the main task has been completed and open news ones to track any further changes. Thanks for proposing the integration and for all the work in implementing the fuzzers!

For the record, the last item addressed in this bug was moving the corpus over to our google cloud storage [1].

[1] https://github.com/google/oss-fuzz/pull/1688

[pdk… via monorail]

Comment #27 on issue 376 by pdk...@gmail.com: oss-fuzz
https://bugs.chromium.org/p/webp/issues/detail?id=376#c27

By the way, I don't know if you noticed the new and improved (public) coverage reports. Coverage is really good.

https://storage.googleapis.com/oss-fuzz-coverage/libwebp/reports/20180925/linux/report.html