WARNING: refcount bug in p9_req_put

[syzbot]

Hello,

syzbot found the following crash on:

HEAD commit: ccda4af0f4b9 Linux 4.20-rc2
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=166b66a3400000
kernel config: https://syzkaller.appspot.com/x/.config?x=4a0a89f12ca9b0f5
dashboard link: https://syzkaller.appspot.com/bug?extid=edec7868af5997928fe9
compiler: gcc (GCC) 8.0.1 20180413 (experimental)

Unfortunately, I don't have any reproducer for this crash yet.

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+edec78...@syzkaller.appspotmail.com

audit: type=1400 audit(1542156774.207:23570): avc: denied { map } for
pid=3935 comm="blkid" path="/sbin/blkid" dev="sda1" ino=16128
scontext=system_u:system_r:kernel_t:s0 tcontext=system_u:object_r:file_t:s0
tclass=file permissive=1
XFS (loop1): device supports 512 byte sectors (not 0)
------------[ cut here ]------------
refcount_t: underflow; use-after-free.
WARNING: CPU: 0 PID: 3966 at lib/refcount.c:187
refcount_sub_and_test_checked+0x2c9/0x310 lib/refcount.c:187
Kernel panic - not syncing: panic_on_warn set ...
CPU: 0 PID: 3966 Comm: syz-executor0 Not tainted 4.20.0-rc2+ #112
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x244/0x39d lib/dump_stack.c:113
panic+0x2ad/0x55c kernel/panic.c:188
__warn.cold.8+0x20/0x45 kernel/panic.c:540
report_bug+0x254/0x2d0 lib/bug.c:186
fixup_bug arch/x86/kernel/traps.c:178 [inline]
do_error_trap+0x11b/0x200 arch/x86/kernel/traps.c:271
do_invalid_op+0x36/0x40 arch/x86/kernel/traps.c:290
invalid_op+0x14/0x20 arch/x86/entry/entry_64.S:969
RIP: 0010:refcount_sub_and_test_checked+0x2c9/0x310 lib/refcount.c:187
Code: 89 de e8 ea 1a ed fd 84 db 74 07 31 db e9 4d ff ff ff e8 0a 1a ed fd
48 c7 c7 20 ae 60 88 c6 05 7b fd 7e 06 01 e8 67 7d b6 fd <0f> 0b 31 db e9
2c ff ff ff 48 89 cf e8 a6 67 30 fe e9 41 fe ff ff
RSP: 0018:ffff88817e87f330 EFLAGS: 00010282
RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffc90005e51000
RDX: 00000000000222c2 RSI: ffffffff8165e7e5 RDI: 0000000000000005
RBP: ffff88817e87f418 R08: ffff8881866ba640 R09: ffffed103b5c5020
R10: ffffed103b5c5020 R11: ffff8881dae28107 R12: ffff88817c7a7008
R13: 00000000ffffffff R14: ffff88817e87f3f0 R15: ffff8881c1dc9d68
refcount_dec_and_test_checked+0x1a/0x20 lib/refcount.c:212
kref_put include/linux/kref.h:69 [inline]
p9_req_put+0x20/0x60 net/9p/client.c:395
p9_conn_destroy net/9p/trans_fd.c:880 [inline]
p9_fd_close+0x39f/0x6b0 net/9p/trans_fd.c:913
p9_client_create+0xbd0/0x1674 net/9p/client.c:1062
v9fs_session_init+0x217/0x1bb0 fs/9p/v9fs.c:421
v9fs_mount+0x7c/0x8f0 fs/9p/vfs_super.c:135
mount_fs+0xae/0x31d fs/super.c:1261
vfs_kern_mount.part.35+0xdc/0x4f0 fs/namespace.c:961
vfs_kern_mount fs/namespace.c:951 [inline]
do_new_mount fs/namespace.c:2469 [inline]
do_mount+0x581/0x31f0 fs/namespace.c:2801
ksys_mount+0x12d/0x140 fs/namespace.c:3017
__do_sys_mount fs/namespace.c:3031 [inline]
__se_sys_mount fs/namespace.c:3028 [inline]
__x64_sys_mount+0xbe/0x150 fs/namespace.c:3028
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x457569
Code: fd b3 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7
48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff
ff 0f 83 cb b3 fb ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007f1d0faf5c78 EFLAGS: 00000246 ORIG_RAX: 00000000000000a5
RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 0000000000457569
RDX: 0000000020000600 RSI: 00000000200005c0 RDI: 0000000000000000
RBP: 000000000072bf00 R08: 0000000020000240 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007f1d0faf66d4
R13: 00000000004c2b12 R14: 00000000004d4278 R15: 00000000ffffffff
Kernel Offset: disabled
Rebooting in 86400 seconds..


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#bug-status-tracking for how to communicate with
syzbot.

[Dominique Martinet]

syzbot wrote on Thu, Nov 15, 2018:
> RIP: 0010:refcount_sub_and_test_checked+0x2c9/0x310 lib/refcount.c:187
> Code: 89 de e8 ea 1a ed fd 84 db 74 07 31 db e9 4d ff ff ff e8 0a 1a
> ed fd 48 c7 c7 20 ae 60 88 c6 05 7b fd 7e 06 01 e8 67 7d b6 fd <0f>
> 0b 31 db e9 2c ff ff ff 48 89 cf e8 a6 67 30 fe e9 41 fe ff ff
> RSP: 0018:ffff88817e87f330 EFLAGS: 00010282
> RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffc90005e51000
> RDX: 00000000000222c2 RSI: ffffffff8165e7e5 RDI: 0000000000000005
> RBP: ffff88817e87f418 R08: ffff8881866ba640 R09: ffffed103b5c5020
> R10: ffffed103b5c5020 R11: ffff8881dae28107 R12: ffff88817c7a7008
> R13: 00000000ffffffff R14: ffff88817e87f3f0 R15: ffff8881c1dc9d68
> refcount_dec_and_test_checked+0x1a/0x20 lib/refcount.c:212
> kref_put include/linux/kref.h:69 [inline]
> p9_req_put+0x20/0x60 net/9p/client.c:395
> p9_conn_destroy net/9p/trans_fd.c:880 [inline]
> p9_fd_close+0x39f/0x6b0 net/9p/trans_fd.c:913
> p9_client_create+0xbd0/0x1674 net/9p/client.c:1062
> v9fs_session_init+0x217/0x1bb0 fs/9p/v9fs.c:421

So the latest ref put I added on destroy for m->rreq looks like it's not
always a good idea... The worker thread is supposed to be stoped at this
point so I'm not sure what went wrong, but while looking at this I found
a race with the read work function and the cancelled callback --
although that would cause a list corruption so it's not what happened
here.

Will think on it a bit more, and try to reproduce by adding some random
delays to make some races more likely.
I'll fix the race in trans_fd's cancelled after/together with the
async flush patches, I've got something working except for cancelled on
this end so will probably submit something around next week after I've
had time to test a bit more extensively

--
Dominique

[syzbot]

syzbot has found a reproducer for the following crash on:

HEAD commit: 459e3a21 gcc-9: properly declare the {pv,hv}clock_page sto..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=136c9284a00000
kernel config: https://syzkaller.appspot.com/x/.config?x=ef1b87b455c397cf
dashboard link: https://syzkaller.appspot.com/bug?extid=edec7868af5997928fe9
compiler: gcc (GCC) 9.0.0 20181231 (experimental)
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1642ee48a00000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+edec78...@syzkaller.appspotmail.com

WARNING: CPU: 1 PID: 7959 at lib/refcount.c:190
refcount_sub_and_test_checked lib/refcount.c:190 [inline]
WARNING: CPU: 1 PID: 7959 at lib/refcount.c:190
refcount_sub_and_test_checked+0x1d0/0x200 lib/refcount.c:180

Kernel panic - not syncing: panic_on_warn set ...

CPU: 1 PID: 7959 Comm: syz-executor.1 Not tainted 5.1.0-rc7+ #96

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]

dump_stack+0x172/0x1f0 lib/dump_stack.c:113
panic+0x2cb/0x65c kernel/panic.c:214
__warn.cold+0x20/0x45 kernel/panic.c:571
report_bug+0x263/0x2b0 lib/bug.c:186
fixup_bug arch/x86/kernel/traps.c:179 [inline]
fixup_bug arch/x86/kernel/traps.c:174 [inline]
do_error_trap+0x11b/0x200 arch/x86/kernel/traps.c:272
do_invalid_op+0x37/0x50 arch/x86/kernel/traps.c:291
invalid_op+0x14/0x20 arch/x86/entry/entry_64.S:973
RIP: 0010:refcount_sub_and_test_checked lib/refcount.c:190 [inline]
RIP: 0010:refcount_sub_and_test_checked+0x1d0/0x200 lib/refcount.c:180
Code: 1d c8 30 2a 06 31 ff 89 de e8 0c 32 40 fe 84 db 75 94 e8 c3 30 40 fe
48 c7 c7 60 79 a1 87 c6 05 a8 30 2a 06 01 e8 ae de 12 fe <0f> 0b e9 75 ff
ff ff e8 a4 30 40 fe e9 6e ff ff ff 48 89 df e8 37
RSP: 0018:ffff888089ce7860 EFLAGS: 00010282
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
RDX: 0000000000000000 RSI: ffffffff815afcb6 RDI: ffffed101139cefe
RBP: ffff888089ce78f8 R08: ffff8880a4c584c0 R09: ffffed1015d25011
R10: ffffed1015d25010 R11: ffff8880ae928087 R12: 00000000ffffffff
R13: 0000000000000001 R14: ffff888089ce78d0 R15: 0000000000000000
refcount_dec_and_test_checked+0x1b/0x20 lib/refcount.c:220
kref_put include/linux/kref.h:66 [inline]
p9_req_put+0x20/0x60 net/9p/client.c:401
p9_conn_destroy net/9p/trans_fd.c:880 [inline]
p9_fd_close+0x2ee/0x570 net/9p/trans_fd.c:913
p9_client_create+0x998/0x1400 net/9p/client.c:1083
v9fs_session_init+0x1e7/0x1960 fs/9p/v9fs.c:421
v9fs_mount+0x7d/0x920 fs/9p/vfs_super.c:135
legacy_get_tree+0xf2/0x200 fs/fs_context.c:584
vfs_get_tree+0x123/0x450 fs/super.c:1481
do_new_mount fs/namespace.c:2622 [inline]
do_mount+0x1436/0x2c40 fs/namespace.c:2942
ksys_mount+0xdb/0x150 fs/namespace.c:3151
__do_sys_mount fs/namespace.c:3165 [inline]
__se_sys_mount fs/namespace.c:3162 [inline]
__x64_sys_mount+0xbe/0x150 fs/namespace.c:3162
do_syscall_64+0x103/0x610 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x458da9
Code: ad b8 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7

48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff

ff 0f 83 7b b8 fb ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007f2e68c68c78 EFLAGS: 00000246 ORIG_RAX: 00000000000000a5
RAX: ffffffffffffffda RBX: 00007f2e68c68c90 RCX: 0000000000458da9
RDX: 0000000020000100 RSI: 00000000200000c0 RDI: 0000000000000000
RBP: 000000000073bf00 R08: 00000000200013c0 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007f2e68c696d4
R13: 00000000004c4da7 R14: 00000000004d8a20 R15: 0000000000000005

[syzbot]

syzbot suspects this issue was fixed by commit:

commit a39c46067c845a8a2d7144836e9468b7f072343e
Author: Christoph Hellwig <h...@lst.de>
Date: Fri Jul 10 08:57:22 2020 +0000

net/9p: validate fds in p9_fd_open

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=1015f012900000
start commit: 459e3a21 gcc-9: properly declare the {pv,hv}clock_page sto..
git tree: upstream

kernel config: https://syzkaller.appspot.com/x/.config?x=ef1b87b455c397cf
dashboard link: https://syzkaller.appspot.com/bug?extid=edec7868af5997928fe9

syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1642ee48a00000

If the result looks correct, please mark the issue as fixed by replying with:

#syz fix: net/9p: validate fds in p9_fd_open

For information about bisection process see: https://goo.gl/tpsmEJ#bisection

[Dmitry Vyukov]