Does AFL detects any warning message as crash or hang?

[Nishh]

Hello,

I have a llvm sanitizer (say, address sanitizer) instrumented binary and it will generate warnings when you run with the right input. It can also be configured to generate a log file.

But does AFL considers these as crashes or hangs? Or is it recorded anywhere else?

I tried to fuzz my toy application with sanitizer instrumented and the seed input as the one which will generate the warning message, but I didn't see any warning messages/explicit log files with warning message. AFL kept running as well without any crash/hang notifications.

When I run the same toy program with the same input explicitly, it throws the sanitizer warning!

[Michal Zalewski]

> I have a llvm sanitizer (say, address sanitizer) instrumented binary and it
> will generate warnings when you run with the right input.

The only thing that AFL interprets as a crash is a crash (i.e.,
termination of a process with a signal).

AFL has some special code for ASAN (-fsanitize=address) and MSAN
(-fsanitize=memory), setting environmental variables so that they
"crash" (call abort()) when they detect a violation. It also disables
the detection of some less helpful violations, such as malloc()
returning NULL or memory not being freed - although you can override
this.

For other types of sanitizers or other types of issues, this may be
something that needs to be set manually.

> But does AFL considers these as crashes or hangs? Or is it recorded anywhere
> else?

See above. AFL doesn't keep stdout / stderr / any other logs for
individual executions.

You generally don't want to set any "expensive" options, such as debug
logging, crash symbolization, etc, since anything that makes the
program run slower (or crash slower) can cause issues.

/mz

[Nishh]

On Sunday, December 3, 2017 at 6:19:09 PM UTC+1, Michal Zalewski wrote:

> I have a llvm sanitizer (say, address sanitizer) instrumented binary and it
> will generate warnings when you run with the right input.

The only thing that AFL interprets as a crash is a crash (i.e.,
termination of a process with a signal).

AFL has some special code for ASAN (-fsanitize=address) and MSAN
(-fsanitize=memory), setting environmental variables so that they
"crash" (call abort()) when they detect a violation. It also disables
the detection of some less helpful violations, such as malloc()
returning NULL or memory not being freed - although you can override
this.

May I know what I have to do to invoke the "crash" when using address/memory sanitizer? Or will it be called by default?

 

For other types of sanitizers or other types of issues, this may be
something that needs to be set manually. 

Did you mean to call an abort from the sanitizer's code in the llvm source code?

 

> But does AFL considers these as crashes or hangs? Or is it recorded anywhere
> else?

See above. AFL doesn't keep stdout / stderr / any other logs for
individual executions.

Is this including the warning logs generated by sanitizers? I mean to ask if AFL avoids the log file creation by sanitizers, even after passing the flag for it explicitly to generate logs to a specific folder?

[Michal Zalewski]

> May I know what I have to do to invoke the "crash" when using address/memory
> sanitizer? Or will it be called by default?

Sorry, not sure I follow; what are you trying to accomplish? As
mentioned, when using ASAN or MSAN with AFL, they will automatically
"crash" when encountering a violation such as access to uninitialized
memory. If you're using ASAN or MSAN, you don't need to do anything
else.

> I mean to ask if
> AFL avoids the log file creation by sanitizers, even after passing the flag
> for it explicitly to generate logs to a specific folder?

It doesn't disable that, but I don't recommend turning it on. It's not
going to produce the results you expect and it will slow things down
and risk missing some of the crashes.

/mz

[Nishh]

On Sunday, December 3, 2017 at 11:13:09 PM UTC+1, Michal Zalewski wrote:

> May I know what I have to do to invoke the "crash" when using address/memory
> sanitizer? Or will it be called by default?

Sorry, not sure I follow; what are you trying to accomplish? As
mentioned, when using ASAN or MSAN with AFL, they will automatically
"crash" when encountering a violation such as access to uninitialized
memory. If you're using ASAN or MSAN, you don't need to do anything
else.

Thanks for this clarification.

 

> I mean to ask if
> AFL avoids the log file creation by sanitizers, even after passing the flag
> for it explicitly to generate logs to a specific folder?

It doesn't disable that, but I don't recommend turning it on. It's not
going to produce the results you expect and it will slow things down
and risk missing some of the crashes.

I tried running a toy program with a seed input of which will throw a TSAN warning, but with afl-fuzz I didn't see it (no logs at all), even after fuzzing for long time! So was wondering what afl-fuzz does actually.

 

/mz

[Michal Zalewski]

> I tried running a toy program with a seed input of which will throw a TSAN
> warning, but with afl-fuzz I didn't see it (no logs at all), even after
> fuzzing for long time! So was wondering what afl-fuzz does actually.

TSAN is not supported by AFL out of the box and will probably need
some work to get results. Some folks tried their luck with TSAN, they
might be able to chime in.

/mz

[Jakub Wilk]

* Nishh <nischai...@gmail.com>, 2017-12-03, 16:26:

>I tried running a toy program with a seed input of which will throw a
>TSAN warning, but with afl-fuzz I didn't see it

AFL+TSAN is pretty much an unexplored territory:
https://groups.google.com/d/topic/afl-users/dFiuYyBisPI

--
Jakub Wilk

[Nishh]

Yes Jakub, I am facing exactly the same issue!! 

Even though the TSAN_OPTIONS are set, including the log_path, no warnings were generated when ran with afl-fuzz. That's strange. I thought in the afl-fuzz code somewhere theses warnings were being suppressed. But couldn't find anything interesting in the code. :(

[Nishh]

I also tried passing exitcode=137 explicitly and using abort_on_error=1, which gives an exit code of 134 otherwise. Still AFL-Fuzz is not catching these!

If I understand it correctly, AFL should consider these as crashes, right?

[Filip Zarzyński]

Have you tried exit code 86 (masan's legacy)?

It's just a wild guess and I don't know how it supposed to work but some time ago I was fighting with tmin on asan crashes an Jakub Wilk suggested to use:

afl-tmin ... -- sh -c 'frobnicate @@ 2>&1 | grep -w bad-free && kill $$'

but on my non-x86 setup it was not detected as crash instead I ended with

afl-tmin ... sh -c 'app @@ 2>&1 | grep -w bad-free && exit 86'

which miraculously worked

(134 127 anything else I've tried was not working)

FZ

via Android

--
You received this message because you are subscribed to the Google Groups "afl-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to afl-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Nishh]

Hi,

Thank you all for all the suggestions and apologies.

Actually it was a mistake from my end! I didn't add '@@' while fuzzing and hence the inputs from the mentioned input folder was never executed. AFL does recognises TSAN_OPTIONS="abort_on_error=1" which sends an exitcode of 134. When I tried to fuzz (ofcorse with '@@' this time) with seed inputs which throws TSAN warnings, afl-fuzz crashed with the message (as expected). There was a TSAN warning log file also created at the specified path.

I am trying to fuzz the application without any crashing seed inputs now to see if AFL can trigger any more TSAN warnings.

To unsubscribe from this group and stop receiving emails from it, send an email to afl-users+...@googlegroups.com.

[mark....@gmail.com]

Hi Nishh,

u mean the 2 @@ signs at the end of afl-fuzz and did u use the following commands for TSAN in linux.

afl-clang-fast -fsanitize=thread -fno-stack-protector  thread2.c

 

TSAN_OPTIONS=abort_on_error=1:symbolize=0 afl-fuzz -m none -i ./test_thread/ -o ./results/ ./a.out

thanks 

[Nishh]

On Tuesday, December 12, 2017 at 8:47:47 AM UTC+1, mark....@gmail.com wrote:

Hi Nishh,

u mean the 2 @@ signs at the end of afl-fuzz and did u use the following commands for TSAN in linux.

Yes Mark, I missed the 2 @@. so I guess afl-fuzz considers command line input instead of inputs from the mentioned input folder, right? 

 

afl-clang-fast -fsanitize=thread -fno-stack-protector  thread2.c

I didn't use -fno-stack-protector and was using afl-clang/afl-gcc

 

TSAN_OPTIONS=abort_on_error=1:symbolize=0 afl-fuzz -m none -i ./test_thread/ -o ./results/ ./a.out

Again, here also, I didn't use symbolize=0, but I tried with abort_on_error=1 and exitcode=137

 

thanks 

[Nishh]

In fact, I am facing an another issue now!

The abort_on_error=1 and exitcode=<something> doesn't work properly when afl-fuzz the application. So I changed the afl-fuzz.c to return a FAULT_CRASH when it gets an exitcode of 66 (which is the default exitcode of TSAN Warnings)

This works fine. But the problem is sometimes afl-fuzz doesn't throw TSAN warnings (especially datarace ones) for even the obvious inputs! It did work well when the input threw deadlock warnings!

I couldn't reason this behaviour of afl. Any suggestions?