Code-Review | +1 |
LGTM. It would have been nice to have a way to register initialization functions in a generic way for test_suite.cc to call later, but this works!
// We need argv_as_pointers_.data() to have type char**, so we can't use
```suggestion
// We need fuzztest_argv_raw_.data() to have type char**, so we can't use
```
extern void (*initialization_function)(int* argc, char*** argv);
Should we put this in a namespace? It seems ripe for collisions.
Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
// We need argv_as_pointers_.data() to have type char**, so we can't use
```suggestion
// We need fuzztest_argv_raw_.data() to have type char**, so we can't use
```
Done
extern void (*initialization_function)(int* argc, char*** argv);
Should we put this in a namespace? It seems ripe for collisions.
Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Code-Review | +1 |
Still LGTM, with one more optional comment.
namespace maybe_fuzztest {
I think I would put everything in the `fuzztest` namespace, except maybe this initialization function in `fuzztest_internal`, to make it clear that users of this header should not mess with this. This works, though.
Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
namespace maybe_fuzztest {
I think I would put everything in the `fuzztest` namespace, except maybe this initialization function in `fuzztest_internal`, to make it clear that users of this header should not mess with this. This works, though.
Thanks, but I'm going to decline to do that because the `fuzztest` namespace is owned by the fuzztest team not us, and the `fuzztest_internal` namespace feels like it would be even more so (though it doesn't exist right now). This is kind of meta-fuzztest code owned by Chromium. So this seemed like the least bad choice.
unit test targets (though this is still not encouaged by Chromium's
encouraged
This extra call in base::TestSuite sets up FuzzTest ready.
This sentence doesn't quite parse; wonder if this is missing a word?
// changed it.
So random Chromium code is changing the original argc/argv??
That's... fun.
"really_init_fuzztest.cc",
I hate to suggest this, because renaming files will remove CR+1s but I don't love the current names.
namespace maybe_fuzztest {
Adrian TaylorI think I would put everything in the `fuzztest` namespace, except maybe this initialization function in `fuzztest_internal`, to make it clear that users of this header should not mess with this. This works, though.
Thanks, but I'm going to decline to do that because the `fuzztest` namespace is owned by the fuzztest team not us, and the `fuzztest_internal` namespace feels like it would be even more so (though it doesn't exist right now). This is kind of meta-fuzztest code owned by Chromium. So this seemed like the least bad choice.
How about something like `fuzztest_support`?
FuzztestInitializer _static_initializer;
Is this a special name that fuzztest requires?
Otherwise, we should probably avoid names starting with a leading `_`; I believe these are reserved for the implementation.
Code-Review | +1 |
unit test targets (though this is still not encouaged by Chromium's
Adrian Taylorencouraged
Done
This extra call in base::TestSuite sets up FuzzTest ready.
This sentence doesn't quite parse; wonder if this is missing a word?
Done
I hate to suggest this, because renaming files will remove CR+1s but I don't love the current names.
- init_helper.{cc,h} for the "maybe" stuff
- "something else" for really_init_fuzztest (though I have no idea what)
Done. I went with `confirm_init.cc` for `really_init_fuzztest.cc`
namespace maybe_fuzztest {
Adrian TaylorI think I would put everything in the `fuzztest` namespace, except maybe this initialization function in `fuzztest_internal`, to make it clear that users of this header should not mess with this. This works, though.
Daniel ChengThanks, but I'm going to decline to do that because the `fuzztest` namespace is owned by the fuzztest team not us, and the `fuzztest_internal` namespace feels like it would be even more so (though it doesn't exist right now). This is kind of meta-fuzztest code owned by Chromium. So this seemed like the least bad choice.
How about something like `fuzztest_support`?
I chose `fuzztest_init_helper`
Is this a special name that fuzztest requires?
Otherwise, we should probably avoid names starting with a leading `_`; I believe these are reserved for the implementation.
Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Initialize FuzzTest for all Chromium tests.
FUZZ_TEST is a new type of test which is half way between a unit test
and a fuzzer (the same code can run in two modes).
Eventually, we hope to allow their free inclusion in existing Chromium
unit test targets (though this is still not encouraged by Chromium's
fuzzing documentation as there are a number of things we need to resolve
first).
This extra call in base::TestSuite sets up FuzzTest ready for this.
base/test:test_support is included in all sorts of test targets, not
just test suites. And, of the test suites where it ends up being
included, some of them will have fuzztests and some of them won't. This
CL is heavily structured such that the new code impacts only test suites
which have fuzztests.
Specifically,
* base/test:test_support ends up being included in various pre-existing
fuzzers. This is a problem because those fuzzers might
have implementations of core fuzzing functions such as
LLVMFuzzerCustomMutator, which would conflict with the fuzztest
implementation. We therefore take pains not to include
src/fuzztest/internal/compatibility_mode.cc except if the test suite
really does have fuzztests.
* base/test:test_support is sometimes linked from targets that include
protobuf_full (notably some GPU test targets). fuzztest depends on
protobuf_lite. On component builds this causes a conflict - a build
failure on Windows and ODR violations on other platforms. We therefore
need to avoid including fuzztests' dependencies into
base/test:test_support unless there really are fuzztests in the test
suite.
Fortunately, testing/test.gni already knows whether there are fuzztests
in a given test suite, as it needs to generate wrapper executables for
each fuzztest.
We therefore use dependency injection, where base/test:test_support calls
a MaybeInitFuzztest function. In test suites containing real fuzztests,
a function pointer will have been populated with a real function that
can initialize fuzztests.
A couple of other details:
* Fuzztest takes the liberty of looking at its command line arguments
"later". Chromium turns out to modify its command line, and thus the
fuzztest command line would become invalid. We store a copy to avoid
this problem.
* When test suites are running in fuzzing mode, we avoid going into
multiple process test mode.
Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |