Announcing FuzzTests: fuzzing and property tests in your test suites

[Adrian Taylor]

Hi all,

TL;DR: you can now write this in your existing Chromium unit test files:

static void CanParseAnythingWithoutCrashing(const std::string& input) {   JSONReader::Read(input, JSON_PARSE_CHROMIUM_EXTENSIONS); } FUZZ_TEST(JSONReaderTest, CanParseAnythingWithoutCrashing);

Our testing and fuzzing infrastructure will attempt to disprove that statement. More detailed how-to here, or see below.

More details:

Fuzztest is a property testing framework we’ve now integrated into Chromium. You can add FUZZ_TESTs alongside your existing unit tests. You can use it in lots of ways - as above, to check that functions never crash for any input - or as below, to check that an assertion is true for a certain domain of input:

void MyApiAlwaysSucceedsOnPositiveIntegers(int i) {   bool success = MyApi(i);   EXPECT_TRUE(success); } FUZZ_TEST(MyApiTest, MyApiAlwaysSucceedsOnPositiveIntegers)     .WithDomains(/*i:*/fuzztest::Positive<int>());

In Chromium, there’s one additional line required in your gn file.

The clever bit is this: the same FUZZ_TEST will run briefly in unit test mode, and will be scooped up to run on ClusterFuzz as a full-fledged fuzzer on our 25,000 cores. Any bugs found will be reported just like any other fuzzer discoveries.

We encourage you to add FUZZ_TESTs to your code wherever possible, and especially if you have code handling input from the web. Finding bugs early through fuzzing is cheaper and less stressful than finding them later when they annoy users. This is a great way to improve Chromium development velocity, reduce your team stress, and make users more secure all at the same time. Please go ahead and FUZZ_TEST!

If you run into any problems please raise a bug here or get in touch with chrome-fuzzin...@google.com.

Thanks everyone,

Ade on behalf of the Chrome fuzzing team