'Driller' tool uses AFL

[Heiko Eißfeldt]

Hello,

I found this article on 'Driller' (which combines AFL fuzzing with symbolic execution) quite fascinating:

https://www.internetsociety.org/sites/default/files/blogs-media/driller-augmenting-fuzzing-through-selective-symbolic-execution.pdf

Enjoy,

Heiko

[Michal Zalewski]

> https://www.internetsociety.org/sites/default/files/blogs-media/driller-augmenting-fuzzing-through-selective-symbolic-execution.pdf

Yup, saw that. Would love to see the code. In general, I think it's a
viable approach, and I have seen several public and non-public
experiments that validate it.

The use of QEMU and DARPA decoy targets wouldn't be my first choice,
but it's probably a legit benchmark. QEMU testing is slow and
definitely not the fuzzer's primary use case or its strength. I
haven't look at the DARPA samples closely, but as always, finding new
bugs in real-world, well-fuzzed code is a stronger validation for any
tool :-)

The paper at some point defines a "stuck" fuzzer as "unable to
identify any new interesting [and resorting] to random input
mutation", which seems like an odd way to phrase it (resorting to
random mutations is not how I'd define a fuzzer being stuck), but not
sure if this is just semantics, or some fundamental issue. Probably
the former?

The "sync" feature allows AFL to run in parallel with any other
"corpus extension" tools, without having to stop and restart the
fuzzing process (which can be very costly, depending on how it's
done). I should probably document it a bit better.

/mz

[Heiko Eißfeldt]

Am Donnerstag, 17. März 2016 17:19:03 UTC+1 schrieb Michal Zalewski:

The paper at some point defines a "stuck" fuzzer as "unable to
identify any new interesting [and resorting] to random input
mutation", which seems like an odd way to phrase it (resorting to
random mutations is not how I'd define a fuzzer being stuck), but not
sure if this is just semantics, or some fundamental issue. Probably
the former? 

I think, they meant 'stuck' whenever new paths are 'hidden' behind a
condition that is difficult to trigger (like in comparisons for one value
out of 2³²). So not stuck in an absolute sense, but needing a lot of
trials to overcome the barrier which might be easy for a constraint solver.

The "sync" feature allows AFL to run in parallel with any other
"corpus extension" tools, without having to stop and restart the
fuzzing process (which can be very costly, depending on how it's
done). I should probably document it a bit better.

Agreed.

How about a filter function hook that could enable the pruning of 'unwanted' stimuli
mutations in order to guide AFL?

bool prune(unsigned char *stimulus);

For example I want my stimuli to obey some rules, lets say they
need to match a regular expression. With a user defined prune
function I could let AFL focus on it.

Would that be useful?

Heiko

[Michal Zalewski]

> How about a filter function hook that could enable the pruning of 'unwanted'
> stimuli mutations in order to guide AFL?
>
> bool prune(unsigned char *stimulus);
>
> For example I want my stimuli to obey some rules, lets say they
> need to match a regular expression. With a user defined prune
> function I could let AFL focus on it.
>
> Would that be useful?

This is long implemented =) See experimental/post_library/

/mz

[Mate Soos]

It's nice to see things published that I guess most of us thought
of/heard about. As always, the actual implementation and its usability
matters a lot. I maintain/write some SMT/SAT tools and even I can't use
KLEE on them in plain mode (only with e.g. S2E), it's just way too
painful. One day we'll get there and it'll be fun, though. Until then,
AFL rocks :)

Mate

> --
> You received this message because you are subscribed to the Google
> Groups "afl-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to afl-users+...@googlegroups.com
> <mailto:afl-users+...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

[Peter Gutmann]

Heiko Eißfeldt <he...@hexco.de> writes:

Looks nice, particularly the way it gets around a known issue in AFL where you
need to brute-force things like checks for magic numbers, but it suffers from
the same problem that any number of similar "here is a cool new tool" papers
suffer from: It's actually a "here is a cool new tool that we're not going to
give you" paper. Of the quite significant number of papers that have been
published in this area, the only one that immediately springs to mind that
made the tool available for use was STACK, the integer-overflow checker.

Peter.

[Jacopo Corbetta]

Hey guys,

Thanks for reading the paper! We want to clarify a couple of things:

• "Stuck" was probably an unhappy choice of words: what we mean is "appears stuck in coverage" -- as in, AFL has not found new state transitions in quite a bit.
• We do indeed sync state with AFL and keep it running. However you distribute your computing resources between fuzzing and static analysis, the gist is:
• have AFL sort out the majority of the program “compartments” and gather knowledge (inputs and coverage),
• call in the static analyzer only when necessary, and
• "limit" it with that knowledge.
• As trivial an idea as it may be, we _do_ think this is “the” practical approach with today’s tools.
• Don't be fooled by the simplified ABI: many of DARPA’s CGC binaries are decently complicated.
• Most importantly, it’s much easier to model compared to the full Linux one. Opinions may differ, obviously, but I believe their system _does_ capture most of the complexity of analyzing binaries (traditional shared memory being the only big omission), and simply does away with the grueling task of "knowing" and modeling hundreds of syscalls and their nuances.
• I may be biased on this, but I don't think other systems faithfully implement the entire Windows or Linux ABIs either (even qemu-user has differences!).
• But since we're all in the Shellphish team it’s true that the possibility of winning two million dollars this August may have biased us a bit :)
• angr *is* open source!
• mkvirtualenv angr && pip install angr right now! :D

Unfortunately all angr core authors are super-busy right now, and I can’t speak much for the project internals. But this I can tell you:

• The driller script "essence" will need to be "disentangled" from our game system, so it may be some time before it ends up in the open-source repo.
• Nonetheless, you can reproduce driller with the current open-source angr, and have it handle Linux binaries if you want.
• w.r.t static analysis tools having poor usability: this _has_ been a motivating factor behind the angr project. While still very far from AFL in its ease-of-use, angr's (i)python interface helps quite a bit in getting symbolic execution to do what you need.

As a final note, thanks a lot for AFL! It rocks indeed :D

And wish us good luck for the finals at DEFCON :)

Jacopo

--
You received this message because you are subscribed to the Google Groups "afl-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to afl-users+...@googlegroups.com.

[hawk...@gmail.com]

On Thursday, March 17, 2016 at 12:19:03 PM UTC-4, Michal Zalewski wrote:

The "sync" feature allows AFL to run in parallel with any other
"corpus extension" tools, without having to stop and restart the
fuzzing process (which can be very costly, depending on how it's
done). I should probably document it a bit better.

/mz

Sorry for dredging up an old thread, but I was wondering if you'd be willing to point me at the place in the code you are referring to with this paragraph? I'm willing to read the source to figure out how to use this feature, but I just wanted to make sure that I understood exactly what you are referencing.

Thanks for being so active with your users, Michal (and, of course, for writing an awesome tool!)

Will

[Michal Zalewski]

> Sorry for dredging up an old thread, but I was wondering if you'd be willing
> to point me at the place in the code you are referring to with this
> paragraph? I'm willing to read the source to figure out how to use this
> feature, but I just wanted to make sure that I understood exactly what you
> are referencing.

Check out docs/parallel_fuzzing.txt, and in particular, the first list
item in section #5 :-) The operating principle is very simple - run
afl-fuzz in -M or -S mode, collect findings from the queue, dump new
inputs you want to add in a designated location using sequential
filenames, and you should be all set.

/mz

[Will Hawkins]

Michal,

Thank you for your response! I will try what you said and let you know
how I progress. Thanks again for taking the time to stay up-to-date
with your users and for all the work that you do on AFL!

Will

> --
> You received this message because you are subscribed to a topic in the Google Groups "afl-users" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/afl-users/YZxrPZP2e84/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to afl-users+...@googlegroups.com.

[Gustavo Grieco]

Driller seems to be open-source now:

https://github.com/shellphish/driller
https://github.com/shellphish/driller-afl

> You received this message because you are subscribed to the Google Groups "afl-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to afl-users+...@googlegroups.com.

[Heiko Eißfeldt]

Thanks!

driller-afl is based on AFL version 1.83b.

Heiko

Am Donnerstag, 25. August 2016 13:01:08 UTC+2 schrieb Gustavo Grieco:

[Edward Hackbarth]

I would like to create a VM with Driller and AFL installed. I have tried several approaches and install paths with no success.

First, I am not sure the best high level starting point of: 

 

https://github.com/shellphish/driller 

https://github.com/shellphish/fuzzer, or

https://blog.grimm-co.com/post/guided-fuzzing-with-driller/

I have tried all with varying levels of success. I have honed in on shellphish/fuzzer primarily because it seems to have the best instructions, and is comprehensive (apparently). Still can't get it to completely install & its dependents w/o issue. I did get it to install completely on a bare metal Ubuntu 16.04 system, and it runs, but can't repeat for a VM successfully.

I am focusing on using Ubuntu 16.04. 18 cannot install QEMU and I had other issues with Debian. 

Anyone have a good clean install experience they can share, instructions, or perhaps any other guidance from the experienced?

Thanks!

-- Edward Hackbarth

[Stefan Nagy]

Oh boy... I feel your pain. I remember this being near-impossible - their documentation and repo organization are total disasters. :(

I believe my starting point was https://github.com/shellphish/fuzzer. I believe I needed to use a Python VE.

They provide a Docker image which supposedly has everything installed: https://hub.docker.com/r/shellphish/mechaphish/

You might be able to ask zardus directly on their angr slack channel here: angr.slack.com

Good luck!

-Stefan

[fish...@gmail.com]

Hi,

angr developer here. I would strongly recommend the mechaphish Docker image (https://hub.docker.com/r/shellphish/mechaphish/) as a starting point. We make sure that every new commit pushed to master branches of angr and its submodules does not break components of MechaPhish that rely on them. If you have trouble running the mechaphish docker image, feel free to reach us on angr Slack. You can get an invite to the Slack workspace at https://angr.io/invite.

Best,
Fish

[Edward Hackbarth]

Thanks Stefan and Fish!

Having the container should be awesome, will check it out asap. Thank you!

In the meantime I attempted another VM build on Ubuntu following the instructions from "Guided Fuzzing with Driller" article. Almost there. 

Am getting what appears to be one specific error, multiple times. Happens twice in the tracer pip install, and again if I go on with the driller pip install. Driller does not get installed. 

Here is an excerpt from the console showing the error messages, same each time it happens.

...

copying psutil/tests/test_memory_leaks.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_bsd.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_linux.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_contracts.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_sunos.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_aix.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/__init__.py -> build/lib.linux-x86_64-3.5/psutil/tests
copying psutil/tests/test_process.py -> build/lib.linux-x86_64-3.5/psutil/tests
running build_ext
building 'psutil._psutil_linux' extension
creating build/temp.linux-x86_64-3.5
creating build/temp.linux-x86_64-3.5/psutil
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=562 -DPSUTIL_LINUX=1 -I/usr/include/python3.5m -I/home/driller/driller/venv/include/python3.5m -c psutil/_psutil_common.c -o build/temp.linux-x86_64-3.5/psutil/_psutil_common.o
psutil/_psutil_common.c:9:20: fatal error: Python.h: No such file or directory
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1  

  

----------------------------------------
Failed building wheel for psutil  

Seems like it should be easy to resolve (missing Python.h). Anyone have ideas on a resolution?

Thanks again!

-- Edward

[☣Adam]

I believe the answer is:

sudo apt install python virtualenv git python-dev

The important one being: python-dev

Glad you find the "Guided Fuzzing with Driller" article helpful. I plan on doing more like that in the future.

Cheers,

Adam

[Edward Hackbarth]

Thanks Adam. I think your write up is terrific.

I had run the apt install you suggest. python-dev is already the newest version (2.7.12-1~16.04). Still a no go.

Question. If I manage to get driller installed, how do I install the shellphuzz script on top of everything? Is it as simple as a download, or do I need to do the full pip install? Any pointers would be helpful.

Also, one issue with your install instructions. I found I needed to create the virtual environment with Python3, like so:

$ virtualenv -p python3 venv

-- Edward

[Fish Wang]

Hi Edward,

 

angr and Driller have been migrated to Python 3 since 2017. You’ll need `python3-dev` instead of `python-dev` I believe.

 

Best,

Fish

[Edward Hackbarth]

Fish, thanks! That did the trick. Now have driller pip install complete with no complaints, and it is there in the venv. Very nice.

Does anyone, esp. perhaps Adam, have an answer to my question above. After following Adam's instructions in his fine article, how do I install the shellphuzz script into my venv, and not duplicate install a bunch of stuff?

Thanks again all for the assistance.

-- Edward

[☣Adam]

I'll get the blog post updated to work with the latest versions of Driller/angr/etc. That should be quick and easy.

At the time the article was written shellphuzz did not extend the length of input, so we rolled our own scripts. I think we also submitted a patch so shellphuzz would be able to do that in the future, but I haven't been tracking it to see if it's been accepted (or if it fell through the cracks and we forgot to submit it, which could have happened). If it has this feature now, we'll link to it. This way readers will still be able to understand what's going on, but can use the maintained repo instead of hand rolled scripts.

As for using mechaphish, we haven't tried to set up the while thing. There seems to be a lot of documentation on it, but "Keep in mind that this was never designed to be turn-key, might not install without extreme effort, and might not work without a lot of tweaking." Source: 

https://github.com/mechaphish/mecha-docs/blob/master/README.md

If you get it working, I'm pretty confident patches you write to the documentation will be accepted, as will scripts which automate the install/setup.

I suspect working through that process and improving the documentation is the same priority for Shellphish as it is for me: in the "someday maybe" pile. If I can make a case for putting together a week longadvanced Vulnerability Research training course, it'll shoot way up in my list of priorities! :-)

Best of luck,

Adam

[☣Adam]

Blog post has been updated to use python3 and Ubuntu 18.04. We go over installing qemu on 18.04 (there's a required patch), so you should be able to switch away from 16.04 if that was the only thing holding you back.

Same URL:

https://blog.grimm-co.com/post/guided-fuzzing-with-driller/

If this doesn't work contact me off list and I'll see if I can help you out.

Cheers,

Adam