afl-clang-fast creating false positives with memory sanitizer

[Hanno Böck]

Hi,

I'm trying to track down some strange behavior I get when using
afl-clang-fast with memory sanitizer.
I'm not sure how widely this is used and if it's considered supported,
but something weird is going on here.

Take this example code (let's name it test.c):

#include <stdio.h>
#include <string.h>
int main()
{
char text[100];
sprintf(text, "hello");
printf("%lu\n", strlen(text));
}

This code should be completely valid and not access any uninitialized
memory. A char array with 100 bytes is initialized on the stack,
sprintf fills that with the 0-terminated string "hello" and strlen is
called on that string.

If I do
AFL_USE_MSAN=1 afl-clang-fast test.c
./a.out

I get an error from msan:
Uninitialized bytes in __interceptor_strlen at offset 0 inside [0x7ffc50164110, 6)
==6454==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0x4972c0 in main /tmp/test.c:7:2
#1 0x7f690d9b878f in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.23-r2/work/glibc-2.23/csu/../csu/libc-start.c:289
#2 0x419868 in _start (/tmp/a.out+0x419868)

SUMMARY: MemorySanitizer: use-of-uninitialized-value /tmp/test.c:7:2 in main
Exiting

However if I do the same with plain clang + msan, it works fine:
clang -fsanitize=memory test.c
./a.out

This seems to depend on a number of conditions to show the error. If I
comment out the #include lines (which gives some warnings), the error
will disappear. Also just trying to print the "text" variable (and not
calling strlen on it) doesn't give an error.

Has anyone any ideas what's wrong here and if this is a bug in
afl-clang-fast or in msan (or if I'm missing something here)?

--
Hanno Böck
https://hboeck.de/

mail/jabber: ha...@hboeck.de
GPG: BBB51E42

[Michal Zalewski]

> However if I do the same with plain clang + msan, it works fine:
> clang -fsanitize=memory test.c
> ./a.out

I'm assuming it also works OK if you add -O3 -funroll-loops?

/mz

[Hanno Böck]

On Thu, 23 Jun 2016 06:52:50 -0700
Michal Zalewski <lca...@gmail.com> wrote:

> I'm assuming it also works OK if you add -O3 -funroll-loops?

Yes.

I am unable to create any condition where plain clang produces this
bug, tried several things.

[Michal Zalewski]

> I am unable to create any condition where plain clang produces this
> bug, tried several things.

Weird, especially since afl-clang-fast doesn't really muck with the
assembly or do anything else that could obviously interfere with
MSAN... but maybe something else is amiss... can other folks repro?

Looking at the disassembly may be instructive, too...

/mz

[Michal Zalewski]

> Weird, especially since afl-clang-fast doesn't really muck with the
> assembly or do anything else that could obviously interfere with
> MSAN... but maybe something else is amiss... can other folks repro?
>
> Looking at the disassembly may be instructive, too...

Btw, if I didn't make it clear: I wasn't able to repro, but I'll grab
some additional VM images and try with other distros / versions of
clang.

As to your original question - llvm_mode should be a first-class
citizen and I want it to work well. The only reason it's not default
is that it has dependencies that aren't met on many systems (no clang,
no LLVM dev headers, no support for modules).

/mz

[Hanno Böck]

On Thu, 23 Jun 2016 13:34:53 -0700
Michal Zalewski <lca...@gmail.com> wrote:

> Btw, if I didn't make it clear: I wasn't able to repro, but I'll grab
> some additional VM images and try with other distros / versions of
> clang.

Okay, no, that wasn't clear :-)

clang is 3.8.0, system is Gentoo, afl is 2.12b.

[Michal Zalewski]

> clang is 3.8.0, system is Gentoo, afl is 2.12b.

Hmm, OK. Tested Ubuntu with clang 3.4 (stock package) & CentOS with
clang 3.8.0 (prebuilt binaries from llvm.org). Both seem to be just
fine (latest AFL).

/mz

[Konstantin Serebryany]

Same for me. 

Ubuntu 14.04, fresh clang from trunk, AFL 2.21b -- everything works. 

--kcc 

--
You received this message because you are subscribed to the Google Groups "afl-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to afl-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Konstantin Serebryany]

Hanno and I figured out that FORTIFY is guilty (Hanno had AFL_HARDEN=1).

Msan is not compatible with FORTIFY and we should warn users about it. 

Filed a bug https://sourceware.org/bugzilla/show_bug.cgi?id=20422 to implement the warning on the glibc side. 

--kcc 

[Dominique Pellé]

Hi

-D_FORTIFY_SOURCE=2 is known to break some valid programs, even
without msan/asan.

-D_FORTIFY_SOURCE=1 should always be valid I think.

=== BEGIN QUOTE FROM
https://gcc.gnu.org/ml/gcc-patches/2004-09/msg02055.html ===
The intended use in glibc is that by default no protection is
done, when the above GCC 4.0+ and -D_FORTIFY_SOURCE=1 is used
at optimization level 1 and above, security measures that
shouldn't change behaviour of conforming programs are taken.
With -D_FORTIFY_SOURCE=2 some more checking is added, but
some conforming programs might fail.
=== END QUOTE ===

Are afl-fuzz/msan/asan incompatible with -D_FORTIFY_SOURCE=2 only,
or are they also incompatible with -D_FORTIFY_SOURCE=1?

Regards
Dominique


On Thu, Jul 28, 2016 at 4:41 PM, Konstantin Serebryany

[Konstantin Serebryany]

On Thu, Jul 28, 2016 at 8:07 AM, Dominique Pellé <dominiq...@gmail.com> wrote:

Hi

-D_FORTIFY_SOURCE=2 is known to break some valid programs, even
without msan/asan.

-D_FORTIFY_SOURCE=1 should always be valid I think.

=== BEGIN QUOTE FROM
https://gcc.gnu.org/ml/gcc-patches/2004-09/msg02055.html ===
The intended use in glibc is that by default no protection is
done, when the above GCC 4.0+ and -D_FORTIFY_SOURCE=1 is used
at optimization level 1 and above, security measures that
shouldn't change behaviour of conforming programs are taken.
With -D_FORTIFY_SOURCE=2 some more checking is added, but
some conforming programs might fail.
=== END QUOTE ===

Are afl-fuzz/msan/asan incompatible with -D_FORTIFY_SOURCE=2 only,
or are they also incompatible with -D_FORTIFY_SOURCE=1?

sanitizers are incompatible with both -D_FORTIFY_SOURCE=1 and -D_FORTIFY_SOURCE=2. 

they will work, but will miss some of the bugs because fortify will stand in their way 

--kcc

 

>>> email to afl-users+unsubscribe@googlegroups.com.

>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "afl-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to afl-users+unsubscribe@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "afl-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to afl-users+unsubscribe@googlegroups.com.

[Michal Zalewski]

Phew, OK, sorry about the delay. I'm testing both approaches right
now, will keep you posted :-)

>> >>> email to afl-users+...@googlegroups.com.

>> >>> For more options, visit https://groups.google.com/d/optout.
>> >>
>> >>
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "afl-users" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an

>> > email to afl-users+...@googlegroups.com.

>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "afl-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an

>> email to afl-users+...@googlegroups.com.

>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "afl-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to afl-users+...@googlegroups.com.

[Michal Zalewski]

> Phew, OK, sorry about the delay. I'm testing both approaches right
> now, will keep you posted :-)

Wait, replied in the wrong thread. Meant the one about breaking down
comparisons :-)

/mz