afl as a self-fuzzing fuzzer [0]

[Peter Gutmann]

It seems that afl will fuzz files even when it's not trying to do so (I was
wondering why I was getting crash reports for non-crashing files). See if you
can spot the problem with the following:

afl-as.c: inf = fopen(input_file, "r");
afl-as.c: outf = fdopen(outfd, "w");
afl-fuzz.c: FILE* f = fopen("/proc/stat", "r");
afl-fuzz.c: f = fopen(fn, "r");
afl-fuzz.c: f = fopen("/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor", "r");
afl-fuzz.c: f = fopen("/sys/devices/system/cpu/cpu0/cpufreq/scaling_min_freq", "r");
afl-fuzz.c: f = fopen("/sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq", "r");
afl-fuzz.c: FILE* f = fopen("/proc/stat", "r");
afl-fuzz.c: f = fdopen(fd, "w");
afl-fuzz.c: f = fdopen(fd, "w");
afl-fuzz.c: plot_file = fdopen(fd, "w");
afl-showmap.c: f = fdopen(fd, "w");

Peter.

[0] It's 4am, apologies for the lack of a more witty subject line.

[Michal Zalewski]

> It seems that afl will fuzz files even when it's not trying to do so (I was
> wondering why I was getting crash reports for non-crashing files). See if you
> can spot the problem with the following:

Can you be a bit less cryptic?:-) Looks like you grepped for fopen and
fdopen across all files. Did you run into a specific bug? What are the
symptoms? Do you believe it's related to one of these calls?

/mz

[Peter Gutmann]

Michal Zalewski <lca...@gmail.com> writes:

>Can you be a bit less cryptic?:-) Looks like you grepped for fopen and fdopen
>across all files. Did you run into a specific bug?

Sorry, I thought the pattern would be visible (did I mentioned that 4am
thing?). You're opening the files in text, not binary mode, which corrupts
them in ways other than intended by the fuzzing. For example if you have a
file:

xxxx 0D yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy

then as long as the program under test detects the 0D -> 0A corruption and
rejects the input as invalid, none of the fuzzing you do in the yyyyyy section
will have any effect.

I noticed this while fuzzing SSH, "hmac-sha2-256" is 0D bytes long but afl
makes it appear 0A bytes long. Since "hmac-sha2-" isn't a valid algorithm and
gets immediately rejected, no fuzzing beyond that point ever affects the SSH
parser.

Peter.

[Michal Zalewski]

> Sorry, I thought the pattern would be visible (did I mentioned that 4am
> thing?). You're opening the files in text, not binary mode, which corrupts
> them in ways other than intended by the fuzzing. For example if you have a
> file:

As far as I can tell, all these fopen / fdopen calls are for reading
and writing to various auxiliary text files. None of them ever touch
the data passed to the fuzzed program.

/mz

[Ben Nagy]

On Sun, Feb 22, 2015 at 10:19 AM, Peter Gutmann
<pgu...@cs.auckland.ac.nz> wrote:
> Michal Zalewski <lca...@gmail.com> writes:
>
>>Can you be a bit less cryptic?:-) Looks like you grepped for fopen and fdopen
>>across all files. Did you run into a specific bug?
>

> You're opening the files in text, not binary mode, which corrupts
> them in ways other than intended by the fuzzing.

Seriously?? What OS? I thought that particular bug was just for ruby
programmers :)

(also, Fuzzing Fail #4!)

[Michal Zalewski]

> As far as I can tell, all these fopen / fdopen calls are for reading
> and writing to various auxiliary text files. None of them ever touch
> the data passed to the fuzzed program.

And just to be clear, I definitely can't reproduce any newline
translation taking place on my system. It would have immediately
caused pronounced problems with almost any binary protocol. I suspect
you're seeing something else?

/mz

[Peter Gutmann]

Michal Zalewski <lca...@gmail.com> writes:

>As far as I can tell, all these fopen / fdopen calls are for reading and
>writing to various auxiliary text files. None of them ever touch the data
>passed to the fuzzed program.

Yeah, I've now gone through it in more detail and there's a mix of open(),
fopen(), and open()+fdopen(), but it looks like nothing will result in a text-
translated read or write. Going back to the original input files (not the
fuzzed ones), the corruption is already present in those, so something between
the originating and fuzzing systems seems to have decided that they're text
files, probably some scp implementation that was too smart for its own good
(because of the SSH messages' heavy text content, it's quite possible that
some heuristics would identify them as text rather than binary).

Apologies for the false alarm...

Peter.

[Peter Gutmann]

Michal Zalewski <lca...@gmail.com> writes:

>It would have immediately caused pronounced problems with almost any binary
>protocol.

Not necessarily, because all that fuzzing is checking for is "does it crash",
not "does it declare supposedly-valid input to be invalid". Consider a
hypothetical fuzzer nullfuzz that reads an input file, zeroes the in-memory
contents, and writes it to the output as its "fuzzed" file. Hopefully pretty
much any app will see that it's not a valid file and exit without crashing.
Now that'll never find any crashes at all, but let's modify nullfuzz to
halfnullfuzz which only zeroes the second half of the file while correctly
fuzzing the first. This will appear to work, it just won't find any crashes
caused by problems in the second half of the file. The 0A -> 0D corruption
will have a similar effect, you'd still find some crashes, just not ones that
are caused by changes after the corruption.

In any case though it's not an afl issue, I'll wait to hear back from the
source of the data to see what could have happened.

Peter.

[Sami Liedes]

On Sat, Feb 21, 2015 at 05:46:08PM -0800, Michal Zalewski wrote:
> And just to be clear, I definitely can't reproduce any newline
> translation taking place on my system. It would have immediately
> caused pronounced problems with almost any binary protocol. I suspect
> you're seeing something else?

AFAIK binary mode is a no-op on Linux and most Unices, where the text
file's representation in files and in C are the same. On Windows I
believe "r" will convert \r\n to \n and "w" will convert \n to \r\n.

Sami