afl-fuzz quemu_mode OOM conditions / forkserver
ยอดดู 1,113 ครั้ง
ข้ามไปที่ข้อความที่ยังไม่อ่านรายการแรก
Fabian Freyer
25 ต.ค. 2560 15:26:2625/10/60
ถึง afl-...@googlegroups.com
(Repost of this mail to the group, the original was just To: lcamtuf)
Hi!
I’m using afl-fuzz to fuzz a closed-source binary. First of all, thanks for the awesome work!
First of all, I’m using qemu_mode with an i386 binary, so I have built afl-qemu-trace with CPU_TARGET=i386. Fuzzing works (albeit very slowly) with AFL_NO_FORKSRV=1:
env AFL_NO_FORKSRV=1 afl-fuzz -Q -m 4096 -i in/ -o out/ ./fuzzed_binary @@
However, it fails when this is not set:
[*] Spinning up the fork server...
[+] All right - fork server is up.
[-] PROGRAM ABORT : Unable to communicate with fork server (OOM?)
Location : run_target(), afl-fuzz.c:2404
Running afl-qemu-trace and afl-showmap by themselves works well:
$ afl-showmap -Q -m none -o /dev/null ./fuzzed_binary in/testcase
afl-showmap 2.51b by <lca...@google.com>
[*] Executing './fuzz_static'...
-- Program output begins --
[output snipped]
-- Program output ends --
[+] Captured 55 tuples in '/dev/null'.
$ afl-qemu-trace ./fuzzed_binary in/testcase
[output snipped]
I’m running this on a 64GB machine, and it doesn’t really look like I’m having any OOM conditions. My ulimit is unlimited:
$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 257668
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) unlimited
cpu time (seconds, -t) unlimited
max user processes (-u) 257668
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
I’ve been running strace -f on afl-fuzz, and it seems there are some segfaults involved:
[ stuff snipped ]
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x2b9942de6310) = 22967
strace: Process 22967 attached
[pid 22966] close(9) = 0
[pid 22966] close(8) = 0
[pid 22966] setitimer(ITIMER_REAL, {it_interval={0, 0}, it_value={10, 0}}, <unfinished ...>
[pid 22967] getrlimit(RLIMIT_NOFILE, <unfinished ...>
[pid 22966] <... setitimer resumed> NULL) = 0
[pid 22967] <... getrlimit resumed> {rlim_cur=1024, rlim_max=1024*1024}) = 0
[pid 22966] read(7, <unfinished ...>
[pid 22967] setrlimit(RLIMIT_AS, {rlim_cur=4194304*1024, rlim_max=4194304*1024}) = 0
[pid 22967] setrlimit(RLIMIT_CORE, {rlim_cur=0, rlim_max=0}) = 0
[pid 22967] setsid() = 22967
[pid 22967] dup2(4, 1) = 1
[pid 22967] dup2(4, 2) = 2
[pid 22967] dup2(4, 0) = 0
[pid 22967] dup2(9, 198) = 198
[pid 22967] dup2(8, 199) = 199
[pid 22967] close(9) = 0
[pid 22967] close(10) = 0
[pid 22967] close(7) = 0
[pid 22967] close(8) = 0
[pid 22967] close(3) = 0
[pid 22967] close(4) = 0
[pid 22967] close(5) = 0
[pid 22967] close(6) = 0
[pid 22967] execve("/usr/local/bin/afl-qemu-trace", ["/usr/local/bin/afl-qemu-trace", "--", “./fuzzed_binary”, “/path/to/out//.cur_input"], [/* 25 vars */]) = 0
[stuff snipped]
[pid 22969] write(197, "\250\307X\366\0\0\0\0\263\0@\0\0\0\0\0", 16) = 16
[pid 22967] <... read resumed> "\250\307X\366\0\0\0\0\263\0@\0\0\0\0\0", 16) = 16
[pid 22967] --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_ACCERR, si_addr=0x2b99ee8a77a8} ---
[pid 22969] write(197, "$\311X\366\0\0\0\0\263\0@\0\0\0\0\0", 16 <unfinished ...>
[pid 22967] rt_sigprocmask(SIG_SETMASK, [], <unfinished ...>
[pid 22969] <... write resumed> ) = 16
[pid 22967] <... rt_sigprocmask resumed> NULL, 8) = 0
[pid 22967] getrlimit(RLIMIT_CORE, <unfinished ...>
[pid 22969] write(197, "\261\307X\366\0\0\0\0\263\0@\0\0\0\0\0", 16) = 16
[pid 22967] <... getrlimit resumed> {rlim_cur=0, rlim_max=0}) = 0
[pid 22967] getrlimit(RLIMIT_CORE, <unfinished ...>
[pid 22969] write(197, "9\312X\366\0\0\0\0\263\0@\0\0\0\0\0", 16) = 16
[pid 22967] <... getrlimit resumed> {rlim_cur=0, rlim_max=0}) = 0
[pid 22967] setrlimit(RLIMIT_CORE, {rlim_cur=0, rlim_max=0} <unfinished ...>
[pid 22969] write(197, "\226\312X\366\0\0\0\0\263\0@\0\0\0\0\0", 16 <unfinished ...>
[pid 22967] <... setrlimit resumed> ) = 0
[pid 22969] <... write resumed> ) = 16
[pid 22967] futex(0x2b98f4ea6880, FUTEX_WAKE_PRIVATE, 2147483647 <unfinished ...>
[pid 22969] write(197, "B\312X\366\0\0\0\0\263\0@\0\0\0\0\0", 16) = 16
[pid 22967] <... futex resumed> ) = 0
[pid 22967] write(2, "qemu: uncaught target signal 11 "..., 67 <unfinished ...>
[pid 22969] write(197, "R\312X\366\0\0\0\0\263\0@\0\0\0\0\0", 16 <unfinished ...>
[pid 22967] <... write resumed> ) = 67
[pid 22969] <... write resumed> ) = 16
[pid 22967] rt_sigaction(SIGSEGV, {SIG_DFL, ~[RTMIN RT_1], SA_RESTORER, 0x2b98f48d4390}, NULL, 8) = 0
[pid 22969] write(197, "\324\307X\366\0\0\0\0\263\0@\0\0\0\0\0", 16 <unfinished ...>
[pid 22967] kill(22967, SIGSEGV <unfinished ...>
[pid 22969] <... write resumed> ) = 16
[pid 22967] <... kill resumed> ) = 0
[pid 22967] --- SIGSEGV {si_signo=SIGSEGV, si_code=SI_USER, si_pid=22967, si_uid=0} ---
[pid 22968] +++ killed by SIGSEGV +++
[pid 22966] <... read resumed> "", 4) = 0
[pid 22969] write(197, "h\312X\366\0\0\0\0\263\0@\0\0\0\0\0", 16 <unfinished ...>
[pid 22967] +++ killed by SIGSEGV +++
[pid 22966] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_KILLED, si_pid=22967, si_uid=0, si_status=SIGSEGV, si_utime=5, si_stime=3} ---
[pid 22966] write(1, "\17\33)B\33[?25h\33[0m\33[1;91m\n", 22 <unfinished ...>
[pid 22969] <... write resumed> ) = -1 EPIPE (Broken pipe)
[pid 22966] <... write resumed> ) = 22
[pid 22966] write(1, "[-] PROGRAM ABORT : \33[1;97mUnabl"..., 80 <unfinished ...>
Any further ideas how to debug this?
Fabian
Hi!
I’m using afl-fuzz to fuzz a closed-source binary. First of all, thanks for the awesome work!
First of all, I’m using qemu_mode with an i386 binary, so I have built afl-qemu-trace with CPU_TARGET=i386. Fuzzing works (albeit very slowly) with AFL_NO_FORKSRV=1:
env AFL_NO_FORKSRV=1 afl-fuzz -Q -m 4096 -i in/ -o out/ ./fuzzed_binary @@
However, it fails when this is not set:
[*] Spinning up the fork server...
[+] All right - fork server is up.
[-] PROGRAM ABORT : Unable to communicate with fork server (OOM?)
Location : run_target(), afl-fuzz.c:2404
Running afl-qemu-trace and afl-showmap by themselves works well:
$ afl-showmap -Q -m none -o /dev/null ./fuzzed_binary in/testcase
afl-showmap 2.51b by <lca...@google.com>
[*] Executing './fuzz_static'...
-- Program output begins --
[output snipped]
-- Program output ends --
[+] Captured 55 tuples in '/dev/null'.
$ afl-qemu-trace ./fuzzed_binary in/testcase
[output snipped]
I’m running this on a 64GB machine, and it doesn’t really look like I’m having any OOM conditions. My ulimit is unlimited:
$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 257668
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) unlimited
cpu time (seconds, -t) unlimited
max user processes (-u) 257668
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
I’ve been running strace -f on afl-fuzz, and it seems there are some segfaults involved:
[ stuff snipped ]
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x2b9942de6310) = 22967
strace: Process 22967 attached
[pid 22966] close(9) = 0
[pid 22966] close(8) = 0
[pid 22966] setitimer(ITIMER_REAL, {it_interval={0, 0}, it_value={10, 0}}, <unfinished ...>
[pid 22967] getrlimit(RLIMIT_NOFILE, <unfinished ...>
[pid 22966] <... setitimer resumed> NULL) = 0
[pid 22967] <... getrlimit resumed> {rlim_cur=1024, rlim_max=1024*1024}) = 0
[pid 22966] read(7, <unfinished ...>
[pid 22967] setrlimit(RLIMIT_AS, {rlim_cur=4194304*1024, rlim_max=4194304*1024}) = 0
[pid 22967] setrlimit(RLIMIT_CORE, {rlim_cur=0, rlim_max=0}) = 0
[pid 22967] setsid() = 22967
[pid 22967] dup2(4, 1) = 1
[pid 22967] dup2(4, 2) = 2
[pid 22967] dup2(4, 0) = 0
[pid 22967] dup2(9, 198) = 198
[pid 22967] dup2(8, 199) = 199
[pid 22967] close(9) = 0
[pid 22967] close(10) = 0
[pid 22967] close(7) = 0
[pid 22967] close(8) = 0
[pid 22967] close(3) = 0
[pid 22967] close(4) = 0
[pid 22967] close(5) = 0
[pid 22967] close(6) = 0
[pid 22967] execve("/usr/local/bin/afl-qemu-trace", ["/usr/local/bin/afl-qemu-trace", "--", “./fuzzed_binary”, “/path/to/out//.cur_input"], [/* 25 vars */]) = 0
[stuff snipped]
[pid 22969] write(197, "\250\307X\366\0\0\0\0\263\0@\0\0\0\0\0", 16) = 16
[pid 22967] <... read resumed> "\250\307X\366\0\0\0\0\263\0@\0\0\0\0\0", 16) = 16
[pid 22967] --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_ACCERR, si_addr=0x2b99ee8a77a8} ---
[pid 22969] write(197, "$\311X\366\0\0\0\0\263\0@\0\0\0\0\0", 16 <unfinished ...>
[pid 22967] rt_sigprocmask(SIG_SETMASK, [], <unfinished ...>
[pid 22969] <... write resumed> ) = 16
[pid 22967] <... rt_sigprocmask resumed> NULL, 8) = 0
[pid 22967] getrlimit(RLIMIT_CORE, <unfinished ...>
[pid 22969] write(197, "\261\307X\366\0\0\0\0\263\0@\0\0\0\0\0", 16) = 16
[pid 22967] <... getrlimit resumed> {rlim_cur=0, rlim_max=0}) = 0
[pid 22967] getrlimit(RLIMIT_CORE, <unfinished ...>
[pid 22969] write(197, "9\312X\366\0\0\0\0\263\0@\0\0\0\0\0", 16) = 16
[pid 22967] <... getrlimit resumed> {rlim_cur=0, rlim_max=0}) = 0
[pid 22967] setrlimit(RLIMIT_CORE, {rlim_cur=0, rlim_max=0} <unfinished ...>
[pid 22969] write(197, "\226\312X\366\0\0\0\0\263\0@\0\0\0\0\0", 16 <unfinished ...>
[pid 22967] <... setrlimit resumed> ) = 0
[pid 22969] <... write resumed> ) = 16
[pid 22967] futex(0x2b98f4ea6880, FUTEX_WAKE_PRIVATE, 2147483647 <unfinished ...>
[pid 22969] write(197, "B\312X\366\0\0\0\0\263\0@\0\0\0\0\0", 16) = 16
[pid 22967] <... futex resumed> ) = 0
[pid 22967] write(2, "qemu: uncaught target signal 11 "..., 67 <unfinished ...>
[pid 22969] write(197, "R\312X\366\0\0\0\0\263\0@\0\0\0\0\0", 16 <unfinished ...>
[pid 22967] <... write resumed> ) = 67
[pid 22969] <... write resumed> ) = 16
[pid 22967] rt_sigaction(SIGSEGV, {SIG_DFL, ~[RTMIN RT_1], SA_RESTORER, 0x2b98f48d4390}, NULL, 8) = 0
[pid 22969] write(197, "\324\307X\366\0\0\0\0\263\0@\0\0\0\0\0", 16 <unfinished ...>
[pid 22967] kill(22967, SIGSEGV <unfinished ...>
[pid 22969] <... write resumed> ) = 16
[pid 22967] <... kill resumed> ) = 0
[pid 22967] --- SIGSEGV {si_signo=SIGSEGV, si_code=SI_USER, si_pid=22967, si_uid=0} ---
[pid 22968] +++ killed by SIGSEGV +++
[pid 22966] <... read resumed> "", 4) = 0
[pid 22969] write(197, "h\312X\366\0\0\0\0\263\0@\0\0\0\0\0", 16 <unfinished ...>
[pid 22967] +++ killed by SIGSEGV +++
[pid 22966] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_KILLED, si_pid=22967, si_uid=0, si_status=SIGSEGV, si_utime=5, si_stime=3} ---
[pid 22966] write(1, "\17\33)B\33[?25h\33[0m\33[1;91m\n", 22 <unfinished ...>
[pid 22969] <... write resumed> ) = -1 EPIPE (Broken pipe)
[pid 22966] <... write resumed> ) = 22
[pid 22966] write(1, "[-] PROGRAM ABORT : \33[1;97mUnabl"..., 80 <unfinished ...>
Any further ideas how to debug this?
Fabian
☣Adam
25 ต.ค. 2560 15:36:1625/10/60
ถึง afl-...@googlegroups.com
Try adding: -m none
I'm pretty sure afl-fuzz puts restrictions on the target process in addition to the ulimit of your shell.
--
You received this message because you are subscribed to the Google Groups "afl-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to afl-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
☣Adam
25 ต.ค. 2560 15:39:0425/10/60
ถึง afl-...@googlegroups.com
Also, fork() can cause problems and it looks like there's a clone() in your strace, so that appears to be relevant. If the child process crashes, it won't be detected (as I was reminded of yesterday when I ran into something similar).
Fabian Freyer
25 ต.ค. 2560 15:48:5325/10/60
ถึง afl-...@googlegroups.com
Hi,
On 25 Oct 2017, at 21:39, ☣Adam wrote:
> Also, fork() can cause problems and it looks like there's a clone() in your
> strace, so that appears to be relevant. If the child process crashes, it
> won't be detected (as I was reminded of yesterday when I ran into something
> similar).
The clone() is the clone from AFL’s fork server. There is no fork() in the binary I’m fuzzing.
I’ve tried this, it doesn’t help.
>>> <(214)%20748-3647> <unfinished ...>
>>> email to afl-users+...@googlegroups.com.
On 25 Oct 2017, at 21:39, ☣Adam wrote:
> Also, fork() can cause problems and it looks like there's a clone() in your
> strace, so that appears to be relevant. If the child process crashes, it
> won't be detected (as I was reminded of yesterday when I ran into something
> similar).
I’ve tried this, it doesn’t help.
> --
> You received this message because you are subscribed to the Google Groups "afl-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to afl-users+...@googlegroups.com.
> You received this message because you are subscribed to the Google Groups "afl-users" group.
Michal Zalewski
25 ต.ค. 2560 16:10:2625/10/60
ถึง afl-users
> I’ve tried this, it doesn’t help.
The only thing that's evident from the information you included is
that the target binary crashes and that it's probably related to /
triggered by the forkserver, but I think it's basically impossible to
diagnose it any further without either knowing what you're fuzzing, or
diving in with a debugger. It could be a bug in AFL, it could be a bug
or incompatibility in QEMU, it could be some obfuscation /
anti-debugging feature in your target program, etc.
Cheers,
/mz
Fabian Freyer
25 ต.ค. 2560 17:11:4925/10/60
ถึง afl-users
Hi,
I’ve grabbed a backtrace of the segmentation fault (followed the child in
the first fork in init_forkserver at afl-fuzz.c:1986, then following the
parent in the second in afl_forkserver at
qemu_mode/patches/afl-qemu-cpu-inl.h:187):
#0 disas_insn (s=s@entry=0x7fffffffd9a0, pc_start=pc_start@entry=0xf658c7a8, env=0x555557960540) at /var/afl-2.51b/qemu_mode/qemu-2.3.0/target-i386/translate.c:4432
#1 0x00005555555f797c in gen_intermediate_code_internal (search_pc=0x0, tb=0x2aaaabf98970, cpu=<optimized out>) at /var/afl-2.51b/qemu_mode/qemu-2.3.0/target-i386/translate.c:8028
#2 gen_intermediate_code (env=env@entry=0x555557960540, tb=tb@entry=0x2aaaabf98970) at /var/afl-2.51b/qemu_mode/qemu-2.3.0/target-i386/translate.c:8109
#3 0x0000555555587090 in cpu_x86_gen_code (gen_code_size_ptr=<synthetic pointer>, tb=0x2aaaabf98970, env=0x555557960540) at /var/afl-2.51b/qemu_mode/qemu-2.3.0/translate-all.c:160
#4 tb_gen_code (cpu=<optimized out>, pc=pc@entry=0xf658c7a8, cs_base=cs_base@entry=0x0, flags=0x4000b3, cflags=cflags@entry=0x0) at /var/afl-2.51b/qemu_mode/qemu-2.3.0/translate-all.c:1069
#5 0x0000555555588742 in tb_find_slow (env=env@entry=0x555557960540, pc=0xf658c7a8, cs_base=0x0, flags=0x4000b3) at /var/afl-2.51b/qemu_mode/qemu-2.3.0/cpu-exec.c:302
#6 0x00005555555891d5 in afl_wait_tsl (fd=0x4, env=0x555557960540) at /var/afl-2.51b/qemu_mode/qemu-2.3.0/../patches/afl-qemu-cpu-inl.h:287
#7 afl_forkserver (env=0x555557960540) at /var/afl-2.51b/qemu_mode/qemu-2.3.0/../patches/afl-qemu-cpu-inl.h:210
#8 cpu_x86_exec (env=0x555557960540) at /var/afl-2.51b/qemu_mode/qemu-2.3.0/cpu-exec.c:501
#9 0x00005555555b4a78 in cpu_loop (env=0x555557960540) at /var/afl-2.51b/qemu_mode/qemu-2.3.0/linux-user/main.c:287
#10 0x00005555555859d7 in main (argc=argc@entry=0x4, argv=argv@entry=0x7fffffffe448, envp=<optimized out>) at /var/afl-2.51b/qemu_mode/qemu-2.3.0/linux-user/main.c:4412
#11 0x00002aaaab732830 in __libc_start_main (main=0x5555555850f0 <main>, argc=0x4, argv=0x7fffffffe448, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>,
stack_end=0x7fffffffe438) at ../csu/libc-start.c:291
#12 0x0000555555585e09 in _start ()
It’s pretty clear what’s segfaulting here:
[----------------------------------registers-----------------------------------]
RAX: 0xf658c7a8
RBX: 0x7fffffffd9a0 --> 0xffffffff
RCX: 0xf658c7a9
RDX: 0xf658c7a8
RSI: 0x2aaaaef4d000 --> 0x0
RDI: 0x7fffffffd9a0 --> 0xffffffff
RBP: 0xf658c7a8
RSP: 0x7fffffffd8b0 --> 0x5555558e1bc0 --> 0x555557961018 --> 0x300000002
RIP: 0x5555555e3115 (<disas_insn+101>: movzx r12d,BYTE PTR [rax+rsi*1])
R8 : 0x555555922350 --> 0x0
R9 : 0x0
R10: 0x4000b3
R11: 0x0
R12: 0x555557960540 --> 0xf677ad0000000000
R13: 0x0
R14: 0x1
R15: 0x2aaaabf98970 --> 0xf658c7a8
EFLAGS: 0x10246 (carry PARITY adjust ZERO sign trap INTERRUPT direction overflow)
[-------------------------------------code-------------------------------------]
0x5555555e3109 <disas_insn+89>: mov DWORD PTR [rbx+0x28],0x0
0x5555555e3110 <disas_insn+96>: mov eax,edx
0x5555555e3112 <disas_insn+98>: lea ecx,[rdx+0x1]
=> 0x5555555e3115 <disas_insn+101>: movzx r12d,BYTE PTR [rax+rsi*1]
0x5555555e311a <disas_insn+106>: mov DWORD PTR [rbx+0x10],ecx
0x5555555e311d <disas_insn+109>: cmp r12b,0x66
0x5555555e3121 <disas_insn+113>: je 0x5555555e3298 <disas_insn+488>
0x5555555e3127 <disas_insn+119>: ja 0x5555555e3160 <disas_insn+176>
But I’m not sure whether this is a bug in qemu, where I should look next or how to
proceed with debugging. I also don’t know why this is working without the
Forkserver, what does it do differently than when running with AFL_NO_FORKSRV=1?
Fabian
the first fork in init_forkserver at afl-fuzz.c:1986, then following the
parent in the second in afl_forkserver at
qemu_mode/patches/afl-qemu-cpu-inl.h:187):
#0 disas_insn (s=s@entry=0x7fffffffd9a0, pc_start=pc_start@entry=0xf658c7a8, env=0x555557960540) at /var/afl-2.51b/qemu_mode/qemu-2.3.0/target-i386/translate.c:4432
#1 0x00005555555f797c in gen_intermediate_code_internal (search_pc=0x0, tb=0x2aaaabf98970, cpu=<optimized out>) at /var/afl-2.51b/qemu_mode/qemu-2.3.0/target-i386/translate.c:8028
#2 gen_intermediate_code (env=env@entry=0x555557960540, tb=tb@entry=0x2aaaabf98970) at /var/afl-2.51b/qemu_mode/qemu-2.3.0/target-i386/translate.c:8109
#3 0x0000555555587090 in cpu_x86_gen_code (gen_code_size_ptr=<synthetic pointer>, tb=0x2aaaabf98970, env=0x555557960540) at /var/afl-2.51b/qemu_mode/qemu-2.3.0/translate-all.c:160
#4 tb_gen_code (cpu=<optimized out>, pc=pc@entry=0xf658c7a8, cs_base=cs_base@entry=0x0, flags=0x4000b3, cflags=cflags@entry=0x0) at /var/afl-2.51b/qemu_mode/qemu-2.3.0/translate-all.c:1069
#5 0x0000555555588742 in tb_find_slow (env=env@entry=0x555557960540, pc=0xf658c7a8, cs_base=0x0, flags=0x4000b3) at /var/afl-2.51b/qemu_mode/qemu-2.3.0/cpu-exec.c:302
#6 0x00005555555891d5 in afl_wait_tsl (fd=0x4, env=0x555557960540) at /var/afl-2.51b/qemu_mode/qemu-2.3.0/../patches/afl-qemu-cpu-inl.h:287
#7 afl_forkserver (env=0x555557960540) at /var/afl-2.51b/qemu_mode/qemu-2.3.0/../patches/afl-qemu-cpu-inl.h:210
#8 cpu_x86_exec (env=0x555557960540) at /var/afl-2.51b/qemu_mode/qemu-2.3.0/cpu-exec.c:501
#9 0x00005555555b4a78 in cpu_loop (env=0x555557960540) at /var/afl-2.51b/qemu_mode/qemu-2.3.0/linux-user/main.c:287
#10 0x00005555555859d7 in main (argc=argc@entry=0x4, argv=argv@entry=0x7fffffffe448, envp=<optimized out>) at /var/afl-2.51b/qemu_mode/qemu-2.3.0/linux-user/main.c:4412
#11 0x00002aaaab732830 in __libc_start_main (main=0x5555555850f0 <main>, argc=0x4, argv=0x7fffffffe448, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>,
stack_end=0x7fffffffe438) at ../csu/libc-start.c:291
#12 0x0000555555585e09 in _start ()
It’s pretty clear what’s segfaulting here:
[----------------------------------registers-----------------------------------]
RAX: 0xf658c7a8
RBX: 0x7fffffffd9a0 --> 0xffffffff
RCX: 0xf658c7a9
RDX: 0xf658c7a8
RSI: 0x2aaaaef4d000 --> 0x0
RDI: 0x7fffffffd9a0 --> 0xffffffff
RBP: 0xf658c7a8
RSP: 0x7fffffffd8b0 --> 0x5555558e1bc0 --> 0x555557961018 --> 0x300000002
RIP: 0x5555555e3115 (<disas_insn+101>: movzx r12d,BYTE PTR [rax+rsi*1])
R8 : 0x555555922350 --> 0x0
R9 : 0x0
R10: 0x4000b3
R11: 0x0
R12: 0x555557960540 --> 0xf677ad0000000000
R13: 0x0
R14: 0x1
R15: 0x2aaaabf98970 --> 0xf658c7a8
EFLAGS: 0x10246 (carry PARITY adjust ZERO sign trap INTERRUPT direction overflow)
[-------------------------------------code-------------------------------------]
0x5555555e3109 <disas_insn+89>: mov DWORD PTR [rbx+0x28],0x0
0x5555555e3110 <disas_insn+96>: mov eax,edx
0x5555555e3112 <disas_insn+98>: lea ecx,[rdx+0x1]
=> 0x5555555e3115 <disas_insn+101>: movzx r12d,BYTE PTR [rax+rsi*1]
0x5555555e311a <disas_insn+106>: mov DWORD PTR [rbx+0x10],ecx
0x5555555e311d <disas_insn+109>: cmp r12b,0x66
0x5555555e3121 <disas_insn+113>: je 0x5555555e3298 <disas_insn+488>
0x5555555e3127 <disas_insn+119>: ja 0x5555555e3160 <disas_insn+176>
But I’m not sure whether this is a bug in qemu, where I should look next or how to
proceed with debugging. I also don’t know why this is working without the
Forkserver, what does it do differently than when running with AFL_NO_FORKSRV=1?
Fabian
Fabian Freyer
26 ต.ค. 2560 02:26:3126/10/60
ถึง afl-users
Hi again,
On 25 Oct 2017, at 23:11, Fabian Freyer wrote:
> On 25 Oct 2017, at 22:10, Michal Zalewski wrote:
>
>>> I’ve tried this, it doesn’t help.
>>
>> The only thing that's evident from the information you included is
>> that the target binary crashes and that it's probably related to /
>> triggered by the forkserver, but I think it's basically impossible to
>> diagnose it any further without either knowing what you're fuzzing, or
>> diving in with a debugger. It could be a bug in AFL, it could be a bug
>> or incompatibility in QEMU, it could be some obfuscation /
>> anti-debugging feature in your target program, etc.
So, another follow-up:
I’ve tracked the bug down to shared library loading. I’m using a dynamic
executable with a loader that loads shared libraries. In the child thread, the
loader is executed and the shared libraries are mapped into memory, but
on the parent thread that doesn’t happen. Subsequently, while s->pc in
disas_insn points to valid code in the child process, it points into a
PROT_NONE mapping in the parent, and tb_find_slow segfaults in the parent
thread.
I’m guessing there could be the following approaches to fixing this
a) communicate the memory of the basic block on the t_fd to be read from the
child to the parent
b) share memory between the child and the parent.
For a), I think the main problem would be the overhead of sending the complete
basic block via the pipe.
Option b) would have a persistent memory space between separate fork()s, but
I think this would really only become a problem if the binary rewrites it’s
own .text section. Also I’m guessing it would require rewriting any mmap calls
to shmat or similar.
For now I’m looking at how to do a), to communicate memory reads, see attached
patch. What’s working so far is that now the tb_gen_code calls in the parent
and the child seem to be emitting mostly the same code, except for jumps, which
point to the wrong address. The first child is no longer segfaulting, now the
Second one is, probably due to the incorrect jump addresses. I’m having some
troubles debugging this however, as I’m running into the afl timeouts. Is there
an easy way to turn those off?
Am I on the right track here, or am I going completely into the wrong direction?
Fabian
On 25 Oct 2017, at 23:11, Fabian Freyer wrote:
> On 25 Oct 2017, at 22:10, Michal Zalewski wrote:
>
>>> I’ve tried this, it doesn’t help.
>>
>> The only thing that's evident from the information you included is
>> that the target binary crashes and that it's probably related to /
>> triggered by the forkserver, but I think it's basically impossible to
>> diagnose it any further without either knowing what you're fuzzing, or
>> diving in with a debugger. It could be a bug in AFL, it could be a bug
>> or incompatibility in QEMU, it could be some obfuscation /
>> anti-debugging feature in your target program, etc.
> But I’m not sure whether this is a bug in qemu, where I should look next or how to
> proceed with debugging. I also don’t know why this is working without the
> forkserver, what does it do differently than when running with AFL_NO_FORKSRV=1?
> proceed with debugging. I also don’t know why this is working without the
So, another follow-up:
I’ve tracked the bug down to shared library loading. I’m using a dynamic
executable with a loader that loads shared libraries. In the child thread, the
loader is executed and the shared libraries are mapped into memory, but
on the parent thread that doesn’t happen. Subsequently, while s->pc in
disas_insn points to valid code in the child process, it points into a
PROT_NONE mapping in the parent, and tb_find_slow segfaults in the parent
thread.
I’m guessing there could be the following approaches to fixing this
a) communicate the memory of the basic block on the t_fd to be read from the
child to the parent
b) share memory between the child and the parent.
For a), I think the main problem would be the overhead of sending the complete
basic block via the pipe.
Option b) would have a persistent memory space between separate fork()s, but
I think this would really only become a problem if the binary rewrites it’s
own .text section. Also I’m guessing it would require rewriting any mmap calls
to shmat or similar.
For now I’m looking at how to do a), to communicate memory reads, see attached
patch. What’s working so far is that now the tb_gen_code calls in the parent
and the child seem to be emitting mostly the same code, except for jumps, which
point to the wrong address. The first child is no longer segfaulting, now the
Second one is, probably due to the incorrect jump addresses. I’m having some
troubles debugging this however, as I’m running into the afl timeouts. Is there
an easy way to turn those off?
Am I on the right track here, or am I going completely into the wrong direction?
Fabian
Fabian Freyer
26 ต.ค. 2560 02:53:1826/10/60
ถึง afl-users
On 26 Oct 2017, at 8:26, Fabian Freyer wrote:
> For now I’m looking at how to do a), to communicate memory reads, see attached
> patch. What’s working so far is that now the tb_gen_code calls in the parent
> and the child seem to be emitting mostly the same code, except for jumps, which
> point to the wrong address. The first child is no longer segfaulting, now the
> Second one is, probably due to the incorrect jump addresses. I’m having some
> troubles debugging this however, as I’m running into the afl timeouts. Is there
> an easy way to turn those off?
>
> Am I on the right track here, or am I going completely into the wrong direction?
Andrew Griffiths
26 ต.ค. 2560 08:23:5826/10/60
ถึง afl-users
Hello,
Does the loader respect environment variables along the lines of LD_PRELOAD? You might be able to use that to get the library loaded in the parent address space.
Thanks,
Andrew Griffiths
--
You received this message because you are subscribed to the Google Groups "afl-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to afl-users+unsubscribe@googlegroups.com.
Fabian Freyer
26 ต.ค. 2560 09:02:5826/10/60
ถึง Andrew Griffiths
On 26 Oct 2017, at 14:23, 'Andrew Griffiths' via afl-users wrote:
> Hello,
>
> Does the loader respect environment variables along the lines of
> LD_PRELOAD? You might be able to use that to get the library loaded in the
> parent address space.
It seems like neither LD_PRELOAD, AFL_PRELOAD or patching get_qemu_argv to add
> Hello,
>
> Does the loader respect environment variables along the lines of
> LD_PRELOAD? You might be able to use that to get the library loaded in the
> parent address space.
-E LD_PRELOAD=… helps in any way.
Thanks,
Fabian
Michal Zalewski
4 พ.ย. 2560 21:38:324/11/60
ถึง afl-users
> It seems like neither LD_PRELOAD, AFL_PRELOAD or patching get_qemu_argv to add
> -E LD_PRELOAD=… helps in any way.
You should be able to do this with
> -E LD_PRELOAD=… helps in any way.
QEMU_SET_ENV=LD_PRELOAD=/full/path/to/library...
But then, it will probably only work if the binary itself is
dynamically linked. Unfortunately, the issue with dynamically loaded
code is a known issue for the QEMU mode. There are some possible
fixes, but they are not free, and this is a relatively niche use
case... maybe we could put it behind a flag or somesuch...
/mz
ตอบทุกคน
ตอบกลับผู้สร้าง
ส่งต่อ
ข้อความใหม่ 0 รายการ