Fuzzing Vim with AFL-Fuzz

[Greg McSwagger]

I'm trying to fuzz afl-fuzz. I compiled vim with afl-gcc and afl-gcc worked fine. I ran the commands
$ export AFL_USE_ASAN=1
and then
 $ afl-fuzz -M fuzzer01 -m 99999999 -t 4000 -i in/ -o out/ \
      ./vim -u NONE -X -Z -e -s \
      -c 'call search(getline("."))'' -c ':qa!' @@
but it gave me the error
bash: ! ': event not found.

https://groups.google.com/forum/#!topic/vim_dev/YqIhn_PKuXE

[Jacek Wielemborek]

W dniu 17.08.2016 o 14:53, Greg McSwagger pisze:

> I'm trying to fuzz afl-fuzz. I compiled vim with afl-gcc and afl-gcc worked
> fine. I ran the commands
> $ export AFL_USE_ASAN=1
> and then
> $ afl-fuzz -M fuzzer01 -m 99999999 -t 4000 -i in/ -o out/ \
> ./vim -u NONE -X -Z -e -s \
> -c 'call search(getline("."))'' -c ':qa!' @@
> but it gave me the error
> bash: ! ': event not found.

Looks more like a bash problem to me. Are you sure you meant double ''
before the second -c?

[Dominique Pellé]

Oops, that's my fault. I wrote the post you linked to, and
there is a typo. It's also preferable to add the -N Vim option.
So it should be:

$ afl-fuzz -M fuzzer01 -m 99999999 -t 4000 -i in/ -o out/ \

./vim -u NONE -N -X -Z -e -s \

-c 'call search(getline("."))' -c ':qa!' @@

I'll reply soon to that post, in case the correction is useful
to someone else. Nice hearing someone else fuzzing vim!

Regards
Dominique

[Michal Zalewski]

IIRC, there is also an explicit call to sleep() when vim is starting
up. You want to comment that out, rather than setting a long timeout.

/mz

> --
> You received this message because you are subscribed to the Google Groups "afl-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to afl-users+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Dominique Pellé]

Michal Zalewski <lca...@gmail.com> wrote:

> IIRC, there is also an explicit call to sleep() when vim is starting
> up. You want to comment that out, rather than setting a long timeout.
>
> /mz

Hi Michal

I see that there is a sleep of 100ms when exiting Vim here:

Breakpoint 1, mch_delay (msec=100, ignoreinput=0) at os_unix.c:660
(gdb) bt
#0 mch_delay (msec=100, ignoreinput=0) at os_unix.c:659
#1 0x00000000005d196a in stoptermcap () at term.c:3181
#2 0x0000000000535b17 in mch_exit (r=0) at os_unix.c:3299
#3 0x0000000000622250 in getout (exitval=0) at main.c:1495
#4 0x0000000000482cc0 in ex_quit (eap=0x7fffffffd630) at ex_docmd.c:7195
#5 0x000000000047ac29 in do_one_cmd (cmdlinep=0x7fffffffd750,
sourcing=1, cstack=0x7fffffffd840, fgetline=0x0, cookie=0x0) at
ex_docmd.c:2925
#6 0x0000000000477915 in do_cmdline (cmdline=0x7fffffffe282 "q",
fgetline=0x0, cookie=0x0, flags=11) at ex_docmd.c:1110
#7 0x0000000000476f51 in do_cmdline_cmd (cmd=0x7fffffffe282 "q") at
ex_docmd.c:715
#8 0x00000000006243b4 in exe_commands (parmp=0x8cbdc0 <params>) at main.c:2896
#9 0x00000000006214e1 in vim_main2 () at main.c:781
#10 0x0000000000620de2 in main (argc=7, argv=0x7fffffffde98) at main.c:415

However, that sleep does not apply when using
Vim options -e -s. Example:

$ time vim -X -u NONE -N -c q

real 0m0.116s
user 0m0.012s
sys 0m0.004s

$ time vim -e -s -X -u NONE -N -c q

real 0m0.010s
user 0m0.003s
sys 0m0.009s

I just commented out the content of mch_delay(...) in Vim
and I get the same fuzzing speed of ~ 65 exec/sec
fuzzing a simple VimL script, which also indicates that
there was no sleep of 100 ms, otherwise I would get less
than 10 exec/sec.

That said, if someone has useful tips to fuzz vim,
they are more than welcome! I had not fuzzed in
weeks, and trying today, I found 2 bugs in Vim:

https://groups.google.com/forum/#!topic/vim_dev/YK8ijMJq1IA
https://groups.google.com/forum/#!topic/vim_dev/0s3HcVyepUM

Regards
Dominique

[Greg McSwagger]

Can I use a txt file as my input (test case). I'm getting 56 exec per second. Is there anyway I can speed that up?