uvm_fault: sogetopt

[syzbot]

Hello,

syzbot found the following crash on:

HEAD commit: 3f7c3e6a6fe6 regen
git tree: https://github.com/openbsd/src.git master
console output: https://syzkaller.appspot.com/x/log.txt?x=15f23625400000
dashboard link: https://syzkaller.appspot.com/bug?extid=2cd350dfe5c96f6469f2
compiler:

Unfortunately, I don't have any reproducer for this crash yet.

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+2cd350...@syzkaller.appspotmail.com

uvm_fault(0xffffff007f12be70, 0x48, 0, 1) -> e
kernel: page fault trap, code=0
Stopped at sogetopt+0x3ae: testb $0x1,0x48(%r15)
ddb>
ddb> set $lines = 0
ddb> show panic
kernel page fault
uvm_fault(0xffffff007f12be70, 0x48, 0, 1) -> e
sogetopt(ffffff00680c6d30,ffff8000211744d0,ffffff006e709d88,ffff80002118af28)
at
sogetopt+0x3ae
end trace frame: 0xffff80002118aed0, count: 0
ddb> trace
sogetopt(ffffff00680c6d30,ffff8000211744d0,ffffff006e709d88,ffff80002118af28)
at
sogetopt+0x3ae
sys_getsockopt(ffff80002118afb0,ffff8000211744d0,ffff8000210a5cb0) at
sys_getsockopt+0x13c
syscall(0) at syscall+0x3e4
Xsyscall(6,0,ffffffffffffff67,0,5,8ba960f0010) at Xsyscall+0x128
end of kernel
end trace frame: 0x8bd65a20240, count: -4
ddb> show registers
rdi 0xffffffff81e1ac30 netlock
rsi 0xffffffff814d5dbc soassertlocked+0x7c
rbp 0xffff80002118ae70
rbx 0xffffff006ee5ee00
rdx 0xffff800002acc000
rcx 0x57
rax 0x1
r8 0xffffff006ee5ee00
r9 0
r10 0xf3f2555ade15410d
r11 0xffffffff8186f430 pool_lock_mtx_leave
r12 0x1022 __ALIGN_SIZE+0x22
r13 0xffff __ALIGN_SIZE+0xefff
r14 0xffffff006ee5ee00
r15 0
rip 0xffffffff81a25ffe sogetopt+0x3ae
cs 0x8
rflags 0x10246 __ALIGN_SIZE+0xf246
rsp 0xffff80002118ae50
ss 0x10
sogetopt+0x3ae: testb $0x1,0x48(%r15)
ddb> show proc
PROC (syz-executor0) pid=258679 stat=onproc
flags process=0 proc=4000000<THREAD>
pri=81, usrpri=81, nice=20
forw=0xffffffffffffffff, list=0xffff800021174e30,0xffff8000211757a0
process=0xffff8000210a5cb0 user=0xffff800021186000,
vmspace=0xffffff007f12be70
estcpu=36, cpticks=0, pctcpu=0.0
user=0, sys=0, intr=0
ddb> ps
PID TID PPID UID S FLAGS WAIT COMMAND
99091 336913 37742 0 2 0 syz-executor0
*99091 258679 37742 0 7 0x4000000 syz-executor0
51596 429723 37212 0 2 0 syz-executor1
51596 50717 37212 0 3 0x4000080 ttyout syz-executor1
51596 254613 37212 0 3 0x4000080 fsleep syz-executor1
24658 51249 1 0 3 0x100083 ttyin getty
22309 435656 0 0 3 0x14200 bored sosplice
37742 423484 52127 0 3 0x82 nanosleep syz-executor0
37212 261438 52127 0 3 0x82 nanosleep syz-executor1
52127 21371 42699 0 3 0x82 thrsleep syz-fuzzer
52127 235343 42699 0 3 0x4000082 thrsleep syz-fuzzer
52127 449205 42699 0 3 0x4000082 thrsleep syz-fuzzer
52127 415449 42699 0 3 0x4000082 thrsleep syz-fuzzer
52127 311254 42699 0 3 0x4000082 kqread syz-fuzzer
52127 419696 42699 0 3 0x4000082 thrsleep syz-fuzzer
52127 143592 42699 0 3 0x4000082 thrsleep syz-fuzzer
42699 965 7973 0 3 0x10008a pause ksh
7973 84823 70497 0 3 0x92 select sshd
70497 202428 1 0 3 0x80 select sshd
46672 442864 29617 73 2 0x100090 syslogd
29617 377127 1 0 3 0x100082 netio syslogd
56534 256164 1 77 3 0x100090 poll dhclient
9472 248763 1 0 3 0x80 poll dhclient
77922 282913 0 0 2 0x14200 zerothread
42280 45784 0 0 3 0x14200 aiodoned aiodoned
54328 391827 0 0 3 0x14200 syncer update
23127 38482 0 0 3 0x14200 cleaner cleaner
47850 437729 0 0 3 0x14200 reaper reaper
87426 305917 0 0 3 0x14200 pgdaemon pagedaemon
54904 270963 0 0 3 0x14200 bored crynlk
78040 105621 0 0 3 0x14200 bored crypto
3666 285207 0 0 3 0x40014200 acpi0 acpi0
13911 208216 0 0 3 0x14200 bored softnet
77087 163346 0 0 3 0x14200 bored systqmp
23515 424232 0 0 3 0x14200 bored systq
59845 210649 0 0 3 0x40014200 bored softclock
42314 381120 0 0 3 0x40014200 idle0
1 122950 0 0 3 0x82 wait init
0 0 -1 0 3 0x10200 scheduler swapper


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#bug-status-tracking for how to communicate with
syzbot.

[syzbot]

syzbot has found a reproducer for the following crash on:

HEAD commit: 3f7c3e6a6fe6 regen
git tree: https://github.com/openbsd/src.git master

console output: https://syzkaller.appspot.com/x/log.txt?x=15c711a3400000

dashboard link: https://syzkaller.appspot.com/bug?extid=2cd350dfe5c96f6469f2
compiler:

syz repro: https://syzkaller.appspot.com/x/repro.syz?x=17fb74db400000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=17cc816d400000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+2cd350...@syzkaller.appspotmail.com

uvm_fault(0xffffff007f12b948, 0x48, 0, 1) -> e

kernel: page fault trap, code=0
Stopped at sogetopt+0x3ae: testb $0x1,0x48(%r15)
ddb>
ddb> set $lines = 0
ddb> show panic
kernel page fault

uvm_fault(0xffffff007f12b948, 0x48, 0, 1) -> e
sogetopt(ffffff006e490170,ffff8000210c2e20,ffffff006e705788,ffff8000210fa328)
at
sogetopt+0x3ae
end trace frame: 0xffff8000210fa2d0, count: 0
ddb> trace
sogetopt(ffffff006e490170,ffff8000210c2e20,ffffff006e705788,ffff8000210fa328)
at
sogetopt+0x3ae
sys_getsockopt(ffff8000210fa3b0,ffff8000210c2e20,ffff8000210a5010) at
sys_getsockopt+0x13c
syscall(0) at syscall+0x3e4
Xsyscall(6,0,0,0,1,7f7ffffbebc8) at Xsyscall+0x128
end of kernel
end trace frame: 0x7f7ffffbeb80, count: -4

ddb> show registers
rdi 0xffffffff81e1ac30 netlock

rsi 0xffff __ALIGN_SIZE+0xefff
rbp 0xffff8000210fa270
rbx 0xffffff006d91ab00
rdx 0x1022 __ALIGN_SIZE+0x22
rcx 0x1
rax 0x1
r8 0xffffff006d91ab00
r9 0
r10 0x8b6ea16accbec4a8

r11 0xffffffff8186f430 pool_lock_mtx_leave
r12 0x1022 __ALIGN_SIZE+0x22
r13 0xffff __ALIGN_SIZE+0xefff

r14 0xffffff006d91ab00

r15 0
rip 0xffffffff81a25ffe sogetopt+0x3ae
cs 0x8
rflags 0x10246 __ALIGN_SIZE+0xf246

rsp 0xffff8000210fa250

ss 0x10
sogetopt+0x3ae: testb $0x1,0x48(%r15)
ddb> show proc

PROC (syz-executor9364) pid=384203 stat=onproc
flags process=2<EXEC> proc=0
pri=51, usrpri=51, nice=20
forw=0xffffffffffffffff, list=0xffff8000210c3078,0xffffffff81e956a0
process=0xffff8000210a5010 user=0xffff8000210f5000,
vmspace=0xffffff007f12b948
estcpu=1, cpticks=1, pctcpu=0.0
user=0, sys=1, intr=0

ddb> ps
PID TID PPID UID S FLAGS WAIT COMMAND

*56471 384203 74318 0 7 0x2 syz-executor9364
74318 452817 71580 0 3 0x10008a pause ksh
71580 260478 44287 0 3 0x92 select sshd
3879 246509 1 0 3 0x100083 ttyin getty
44287 510322 1 0 3 0x80 select sshd
21790 384146 60926 73 2 0x100090 syslogd
60926 492157 1 0 3 0x100082 netio syslogd
32316 148290 1 77 3 0x100090 poll dhclient
21967 349865 1 0 3 0x80 poll dhclient
21334 313344 0 0 2 0x14200 zerothread
55762 353713 0 0 3 0x14200 aiodoned aiodoned
35012 278323 0 0 3 0x14200 syncer update
10409 32443 0 0 3 0x14200 cleaner cleaner
58435 46163 0 0 3 0x14200 reaper reaper
70989 150172 0 0 3 0x14200 pgdaemon pagedaemon
25990 343055 0 0 3 0x14200 bored crynlk
33981 120958 0 0 3 0x14200 bored crypto
36834 151323 0 0 3 0x40014200 acpi0 acpi0
3014 500201 0 0 3 0x14200 bored softnet
565 511338 0 0 3 0x14200 bored systqmp
95081 211626 0 0 3 0x14200 bored systq
32606 177822 0 0 3 0x40014200 bored softclock
71050 236547 0 0 3 0x40014200 idle0
1 274328 0 0 3 0x82 wait init

0 0 -1 0 3 0x10200 scheduler swapper

ddb>

[Greg Steuck]

This is the offending line:

https://github.com/openbsd/src/blob/7c13478cbf7a624ad524dc377f8c2a7e497c0f3b/sys/kern/uipc_socket.c#L1909

case SO_PEERCRED:

if (so->so_proto->pr_protocol == AF_UNIX) {

struct unpcb *unp = sotounpcb(so);

if (unp->unp_flags & UNP_FEIDS) {

I want to automate this whole objdump -dlr business, too much manual work.

[Greg Steuck]

Even though I have no idea what I'm doing, the patch below is enough to thwart the reproducer. There are multiple places where the result of sotounpcb is used without checking the result, but I don't know which invariants are established non-locally. 

Please do me a favor when committing this or a proper fix and heed syzkaller's request:

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+2cd350...@syzkaller.appspotmail.com

--- a/sys/kern/uipc_socket.c

+++ b/sys/kern/uipc_socket.c

@@ -1905,6 +1905,8 @@ sogetopt(struct socket *so, int level, int optname, struct mbuf *m)

                case SO_PEERCRED:

                        if (so->so_proto->pr_protocol == AF_UNIX) {

                                struct unpcb *unp = sotounpcb(so);

+                               if (unp == NULL)

+                                       return (EINVAL);

 

                                if (unp->unp_flags & UNP_FEIDS) {

                                        m->m_len = sizeof(unp->unp_connid);

--

nest.cx is Gmail hosted, use PGP for anything private. Key: http://goo.gl/6dMsr
Fingerprint: 5E2B 2D0E 1E03 2046 BEC3  4D50 0B15 42BD 8DF5 A1B0

[Anton Lindqvist]

On Sat, Dec 01, 2018 at 03:13:34PM -0800, Greg Steuck wrote:
> This is the offending line:
> https://github.com/openbsd/src/blob/7c13478cbf7a624ad524dc377f8c2a7e497c0f3b/sys/kern/uipc_socket.c#L1909
> case SO_PEERCRED:
> if (so->so_proto->pr_protocol == AF_UNIX) {
> struct unpcb *unp = sotounpcb(so);
>

> * if (unp->unp_flags & UNP_FEIDS) {*

>
> I want to automate this whole objdump -dlr business, too much manual work.

I have the following utility. Could probably used as a starting point,
the backtrace just needs to be parsed.

$ echo db_enter+0xa | bt2line -e /sys/arch/amd64/compile/SYZKALLER.MP/obj/bsd.gdb
db_enter
/usr/src/sys/arch/amd64/amd64/db_interface.c:399
$ cat `which bt2line`
#!/bin/sh

set -e

usage() {
echo "usage: bt2line [-e executable]" 1>&2
exit 1
}

faddr() {
readelf -s "$1" | grep -Fw -m 1 "$2" | awk '{print $2}'
}

hexify() {
echo "$1" | sed -e 's/^0x//' | tr '[:lower:]' '[:upper:]'
}

hexsum() {
bc <<-EOF
obase = 16
ibase = 16
${1} + ${2}
EOF
}

kernel() {
local _a=$(uname -m) \
_mp=$([ $(sysctl -n hw.ncpu) -gt 1 ] && echo '.MP')

printf '/sys/arch/%s/compile/GENERIC%s/obj/bsd.gdb' "$_a" "$_mp"
}

KERNEL=$(kernel)

while getopts "e:" opt; do
case "$opt" in
e) KERNEL=$OPTARG;;
*) usage;;
esac
done
shift $((OPTIND - 1))

[ $# -ne 0 ] && usage

while read line; do
prefix="${line%%+*}"
fn=$(faddr "$KERNEL" "$prefix")
if [ -z "$fn" ]; then
echo "bt2line: ${prefix}: symbol not found" 1>&2
exit 1
fi

suffix=${line#*+}
hexsum "$(hexify "$fn")" "$(hexify "$suffix")"
done | addr2line -f -e "$KERNEL"

[Dmitry Vyukov]

We have addr2line and nm support already:
https://github.com/google/syzkaller/blob/master/pkg/symbolizer/symbolizer.go
But need to add frame extraction for openbsd similar to:
https://github.com/google/syzkaller/blob/master/pkg/report/linux.go#L291-L384

[Greg Steuck]

Nice script Anton!

Dmitry, I can't shake off my parsing-shyness. I understand why ddb and such are useful for debugging in the wild. We on the other hand are debugging in the lab. We have complete access to all sources, generated files and tools. Should we simply dump the stacks in hex (plus whatever runtime randomization) and feed that into gdb or some tool based on libunwind running on syz-manager side? How much am underestimating the effort of implementing this? If anything, we should be able to not worry about ddb not implementing argument parsing correctly, e.g. https://marc.info/?l=openbsd-tech&m=154101449732660&w=2

[Dmitry Vyukov]

The existing addr2line code should work for openbsd too. Why do you
want to use gdb/libunwind?

[Anton Lindqvist]

On Sun, Dec 02, 2018 at 07:25:43PM +0000, Dmitry Vyukov wrote:
> The existing addr2line code should work for openbsd too. Why do you
> want to use gdb/libunwind?

I can look into adding support for frame extraction to OpenBSD.