possible deadlock in io_submit_one (3)

10 views
Skip to first unread message

syzbot

unread,
Apr 3, 2020, 2:25:16ā€ÆAM4/3/20
to bc...@kvack.org, linu...@kvack.org, linux-...@vger.kernel.org, linux-...@vger.kernel.org, syzkall...@googlegroups.com, vi...@zeniv.linux.org.uk
Hello,

syzbot found the following crash on:

HEAD commit: 7be97138 Merge tag 'xfs-5.7-merge-8' of git://git.kernel.o..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=13d37663e00000
kernel config: https://syzkaller.appspot.com/x/.config?x=ec722f9d4eb221d2
dashboard link: https://syzkaller.appspot.com/bug?extid=343f75cdeea091340956
compiler: clang version 10.0.0 (https://github.com/llvm/llvm-project/ c2443155a0fb245c8f17f2c1c72b6ea391e86e81)

Unfortunately, I don't have any reproducer for this crash yet.

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+343f75...@syzkaller.appspotmail.com

=====================================================
WARNING: SOFTIRQ-safe -> SOFTIRQ-unsafe lock order detected
5.6.0-syzkaller #0 Not tainted
-----------------------------------------------------
syz-executor.2/11911 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire:
ffff888088003548 (&pid->wait_pidfd){+.+.}-{2:2}, at: spin_lock include/linux/spinlock.h:353 [inline]
ffff888088003548 (&pid->wait_pidfd){+.+.}-{2:2}, at: aio_poll fs/aio.c:1767 [inline]
ffff888088003548 (&pid->wait_pidfd){+.+.}-{2:2}, at: __io_submit_one fs/aio.c:1841 [inline]
ffff888088003548 (&pid->wait_pidfd){+.+.}-{2:2}, at: io_submit_one+0x10f5/0x1a80 fs/aio.c:1878

and this task is already holding:
ffff888045d05c98 (&ctx->ctx_lock){..-.}-{2:2}, at: spin_lock_irq include/linux/spinlock.h:378 [inline]
ffff888045d05c98 (&ctx->ctx_lock){..-.}-{2:2}, at: aio_poll fs/aio.c:1765 [inline]
ffff888045d05c98 (&ctx->ctx_lock){..-.}-{2:2}, at: __io_submit_one fs/aio.c:1841 [inline]
ffff888045d05c98 (&ctx->ctx_lock){..-.}-{2:2}, at: io_submit_one+0x10cb/0x1a80 fs/aio.c:1878
which would create a new lock dependency:
(&ctx->ctx_lock){..-.}-{2:2} -> (&pid->wait_pidfd){+.+.}-{2:2}

but this new dependency connects a SOFTIRQ-irq-safe lock:
(&ctx->ctx_lock){..-.}-{2:2}

... which became SOFTIRQ-irq-safe at:
lock_acquire+0x169/0x480 kernel/locking/lockdep.c:4923
__raw_spin_lock_irq include/linux/spinlock_api_smp.h:128 [inline]
_raw_spin_lock_irq+0x67/0x80 kernel/locking/spinlock.c:167
spin_lock_irq include/linux/spinlock.h:378 [inline]
free_ioctx_users+0x30/0x1c0 fs/aio.c:618
percpu_ref_put_many include/linux/percpu-refcount.h:309 [inline]
percpu_ref_put+0x18d/0x1a0 include/linux/percpu-refcount.h:325
rcu_do_batch kernel/rcu/tree.c:2206 [inline]
rcu_core+0x816/0x1120 kernel/rcu/tree.c:2433
__do_softirq+0x268/0x80c kernel/softirq.c:292
do_softirq_own_stack+0x2a/0x40 arch/x86/entry/entry_64.S:1082
do_softirq+0xf9/0x190 kernel/softirq.c:337
__local_bh_enable_ip+0x18b/0x230 kernel/softirq.c:189
spin_unlock_bh include/linux/spinlock.h:398 [inline]
netif_addr_unlock_bh include/linux/netdevice.h:4182 [inline]
dev_uc_add+0x374/0x440 net/core/dev_addr_lists.c:593
macsec_dev_open+0x8b/0x670 drivers/net/macsec.c:3487
__dev_open+0x27c/0x410 net/core/dev.c:1436
__dev_change_flags+0x198/0x650 net/core/dev.c:8143
dev_change_flags+0x85/0x190 net/core/dev.c:8214
do_setlink+0xb17/0x3900 net/core/rtnetlink.c:2598
__rtnl_newlink net/core/rtnetlink.c:3266 [inline]
rtnl_newlink+0x1509/0x1c00 net/core/rtnetlink.c:3391
rtnetlink_rcv_msg+0x889/0xd40 net/core/rtnetlink.c:5454
netlink_rcv_skb+0x190/0x3a0 net/netlink/af_netlink.c:2469
netlink_unicast_kernel net/netlink/af_netlink.c:1303 [inline]
netlink_unicast+0x786/0x940 net/netlink/af_netlink.c:1329
netlink_sendmsg+0xa57/0xd70 net/netlink/af_netlink.c:1918
sock_sendmsg_nosec net/socket.c:652 [inline]
sock_sendmsg net/socket.c:672 [inline]
__sys_sendto+0x3f3/0x590 net/socket.c:2000
__do_sys_sendto net/socket.c:2012 [inline]
__se_sys_sendto net/socket.c:2008 [inline]
__x64_sys_sendto+0xda/0xf0 net/socket.c:2008
do_syscall_64+0xf3/0x1b0 arch/x86/entry/common.c:295
entry_SYSCALL_64_after_hwframe+0x49/0xb3

to a SOFTIRQ-irq-unsafe lock:
(&pid->wait_pidfd){+.+.}-{2:2}

... which became SOFTIRQ-irq-unsafe at:
...
lock_acquire+0x169/0x480 kernel/locking/lockdep.c:4923
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:151
spin_lock include/linux/spinlock.h:353 [inline]
proc_pid_make_inode+0x187/0x2d0 fs/proc/base.c:1880
proc_pid_instantiate+0x4b/0x1a0 fs/proc/base.c:3285
proc_pid_lookup+0x218/0x2f0 fs/proc/base.c:3320
proc_root_lookup+0x1b/0x50 fs/proc/root.c:243
__lookup_slow+0x240/0x370 fs/namei.c:1530
lookup_slow fs/namei.c:1547 [inline]
walk_component+0x442/0x680 fs/namei.c:1846
link_path_walk+0x66d/0xba0 fs/namei.c:2165
path_openat+0x21d/0x38b0 fs/namei.c:3342
do_filp_open+0x2b4/0x3a0 fs/namei.c:3375
do_sys_openat2+0x463/0x6f0 fs/open.c:1148
do_sys_open fs/open.c:1164 [inline]
ksys_open include/linux/syscalls.h:1386 [inline]
__do_sys_open fs/open.c:1170 [inline]
__se_sys_open fs/open.c:1168 [inline]
__x64_sys_open+0x1af/0x1e0 fs/open.c:1168
do_syscall_64+0xf3/0x1b0 arch/x86/entry/common.c:295
entry_SYSCALL_64_after_hwframe+0x49/0xb3

other info that might help us debug this:

Possible interrupt unsafe locking scenario:

CPU0 CPU1
---- ----
lock(&pid->wait_pidfd);
local_irq_disable();
lock(&ctx->ctx_lock);
lock(&pid->wait_pidfd);
<Interrupt>
lock(&ctx->ctx_lock);

*** DEADLOCK ***

1 lock held by syz-executor.2/11911:
#0: ffff888045d05c98 (&ctx->ctx_lock){..-.}-{2:2}, at: spin_lock_irq include/linux/spinlock.h:378 [inline]
#0: ffff888045d05c98 (&ctx->ctx_lock){..-.}-{2:2}, at: aio_poll fs/aio.c:1765 [inline]
#0: ffff888045d05c98 (&ctx->ctx_lock){..-.}-{2:2}, at: __io_submit_one fs/aio.c:1841 [inline]
#0: ffff888045d05c98 (&ctx->ctx_lock){..-.}-{2:2}, at: io_submit_one+0x10cb/0x1a80 fs/aio.c:1878

the dependencies between SOFTIRQ-irq-safe lock and the holding lock:
-> (&ctx->ctx_lock){..-.}-{2:2} {
IN-SOFTIRQ-W at:
lock_acquire+0x169/0x480 kernel/locking/lockdep.c:4923
__raw_spin_lock_irq include/linux/spinlock_api_smp.h:128 [inline]
_raw_spin_lock_irq+0x67/0x80 kernel/locking/spinlock.c:167
spin_lock_irq include/linux/spinlock.h:378 [inline]
free_ioctx_users+0x30/0x1c0 fs/aio.c:618
percpu_ref_put_many include/linux/percpu-refcount.h:309 [inline]
percpu_ref_put+0x18d/0x1a0 include/linux/percpu-refcount.h:325
rcu_do_batch kernel/rcu/tree.c:2206 [inline]
rcu_core+0x816/0x1120 kernel/rcu/tree.c:2433
__do_softirq+0x268/0x80c kernel/softirq.c:292
do_softirq_own_stack+0x2a/0x40 arch/x86/entry/entry_64.S:1082
do_softirq+0xf9/0x190 kernel/softirq.c:337
__local_bh_enable_ip+0x18b/0x230 kernel/softirq.c:189
spin_unlock_bh include/linux/spinlock.h:398 [inline]
netif_addr_unlock_bh include/linux/netdevice.h:4182 [inline]
dev_uc_add+0x374/0x440 net/core/dev_addr_lists.c:593
macsec_dev_open+0x8b/0x670 drivers/net/macsec.c:3487
__dev_open+0x27c/0x410 net/core/dev.c:1436
__dev_change_flags+0x198/0x650 net/core/dev.c:8143
dev_change_flags+0x85/0x190 net/core/dev.c:8214
do_setlink+0xb17/0x3900 net/core/rtnetlink.c:2598
__rtnl_newlink net/core/rtnetlink.c:3266 [inline]
rtnl_newlink+0x1509/0x1c00 net/core/rtnetlink.c:3391
rtnetlink_rcv_msg+0x889/0xd40 net/core/rtnetlink.c:5454
netlink_rcv_skb+0x190/0x3a0 net/netlink/af_netlink.c:2469
netlink_unicast_kernel net/netlink/af_netlink.c:1303 [inline]
netlink_unicast+0x786/0x940 net/netlink/af_netlink.c:1329
netlink_sendmsg+0xa57/0xd70 net/netlink/af_netlink.c:1918
sock_sendmsg_nosec net/socket.c:652 [inline]
sock_sendmsg net/socket.c:672 [inline]
__sys_sendto+0x3f3/0x590 net/socket.c:2000
__do_sys_sendto net/socket.c:2012 [inline]
__se_sys_sendto net/socket.c:2008 [inline]
__x64_sys_sendto+0xda/0xf0 net/socket.c:2008
do_syscall_64+0xf3/0x1b0 arch/x86/entry/common.c:295
entry_SYSCALL_64_after_hwframe+0x49/0xb3
INITIAL USE at:
lock_acquire+0x169/0x480 kernel/locking/lockdep.c:4923
__raw_spin_lock_irq include/linux/spinlock_api_smp.h:128 [inline]
_raw_spin_lock_irq+0x67/0x80 kernel/locking/spinlock.c:167
spin_lock_irq include/linux/spinlock.h:378 [inline]
free_ioctx_users+0x30/0x1c0 fs/aio.c:618
percpu_ref_put_many include/linux/percpu-refcount.h:309 [inline]
percpu_ref_put+0x18d/0x1a0 include/linux/percpu-refcount.h:325
rcu_do_batch kernel/rcu/tree.c:2206 [inline]
rcu_core+0x816/0x1120 kernel/rcu/tree.c:2433
__do_softirq+0x268/0x80c kernel/softirq.c:292
do_softirq_own_stack+0x2a/0x40 arch/x86/entry/entry_64.S:1082
do_softirq+0xf9/0x190 kernel/softirq.c:337
__local_bh_enable_ip+0x18b/0x230 kernel/softirq.c:189
spin_unlock_bh include/linux/spinlock.h:398 [inline]
netif_addr_unlock_bh include/linux/netdevice.h:4182 [inline]
dev_uc_add+0x374/0x440 net/core/dev_addr_lists.c:593
macsec_dev_open+0x8b/0x670 drivers/net/macsec.c:3487
__dev_open+0x27c/0x410 net/core/dev.c:1436
__dev_change_flags+0x198/0x650 net/core/dev.c:8143
dev_change_flags+0x85/0x190 net/core/dev.c:8214
do_setlink+0xb17/0x3900 net/core/rtnetlink.c:2598
__rtnl_newlink net/core/rtnetlink.c:3266 [inline]
rtnl_newlink+0x1509/0x1c00 net/core/rtnetlink.c:3391
rtnetlink_rcv_msg+0x889/0xd40 net/core/rtnetlink.c:5454
netlink_rcv_skb+0x190/0x3a0 net/netlink/af_netlink.c:2469
netlink_unicast_kernel net/netlink/af_netlink.c:1303 [inline]
netlink_unicast+0x786/0x940 net/netlink/af_netlink.c:1329
netlink_sendmsg+0xa57/0xd70 net/netlink/af_netlink.c:1918
sock_sendmsg_nosec net/socket.c:652 [inline]
sock_sendmsg net/socket.c:672 [inline]
__sys_sendto+0x3f3/0x590 net/socket.c:2000
__do_sys_sendto net/socket.c:2012 [inline]
__se_sys_sendto net/socket.c:2008 [inline]
__x64_sys_sendto+0xda/0xf0 net/socket.c:2008
do_syscall_64+0xf3/0x1b0 arch/x86/entry/common.c:295
entry_SYSCALL_64_after_hwframe+0x49/0xb3
}
... key at: [<ffffffff8b596090>] ioctx_alloc.__key+0x0/0x10
... acquired at:
lock_acquire+0x169/0x480 kernel/locking/lockdep.c:4923
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:151
spin_lock include/linux/spinlock.h:353 [inline]
aio_poll fs/aio.c:1767 [inline]
__io_submit_one fs/aio.c:1841 [inline]
io_submit_one+0x10f5/0x1a80 fs/aio.c:1878
__do_sys_io_submit fs/aio.c:1937 [inline]
__se_sys_io_submit+0x117/0x220 fs/aio.c:1907
do_syscall_64+0xf3/0x1b0 arch/x86/entry/common.c:295
entry_SYSCALL_64_after_hwframe+0x49/0xb3


the dependencies between the lock to be acquired
and SOFTIRQ-irq-unsafe lock:
-> (&pid->wait_pidfd){+.+.}-{2:2} {
HARDIRQ-ON-W at:
lock_acquire+0x169/0x480 kernel/locking/lockdep.c:4923
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:151
spin_lock include/linux/spinlock.h:353 [inline]
proc_pid_make_inode+0x187/0x2d0 fs/proc/base.c:1880
proc_pid_instantiate+0x4b/0x1a0 fs/proc/base.c:3285
proc_pid_lookup+0x218/0x2f0 fs/proc/base.c:3320
proc_root_lookup+0x1b/0x50 fs/proc/root.c:243
__lookup_slow+0x240/0x370 fs/namei.c:1530
lookup_slow fs/namei.c:1547 [inline]
walk_component+0x442/0x680 fs/namei.c:1846
link_path_walk+0x66d/0xba0 fs/namei.c:2165
path_openat+0x21d/0x38b0 fs/namei.c:3342
do_filp_open+0x2b4/0x3a0 fs/namei.c:3375
do_sys_openat2+0x463/0x6f0 fs/open.c:1148
do_sys_open fs/open.c:1164 [inline]
ksys_open include/linux/syscalls.h:1386 [inline]
__do_sys_open fs/open.c:1170 [inline]
__se_sys_open fs/open.c:1168 [inline]
__x64_sys_open+0x1af/0x1e0 fs/open.c:1168
do_syscall_64+0xf3/0x1b0 arch/x86/entry/common.c:295
entry_SYSCALL_64_after_hwframe+0x49/0xb3
SOFTIRQ-ON-W at:
lock_acquire+0x169/0x480 kernel/locking/lockdep.c:4923
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:151
spin_lock include/linux/spinlock.h:353 [inline]
proc_pid_make_inode+0x187/0x2d0 fs/proc/base.c:1880
proc_pid_instantiate+0x4b/0x1a0 fs/proc/base.c:3285
proc_pid_lookup+0x218/0x2f0 fs/proc/base.c:3320
proc_root_lookup+0x1b/0x50 fs/proc/root.c:243
__lookup_slow+0x240/0x370 fs/namei.c:1530
lookup_slow fs/namei.c:1547 [inline]
walk_component+0x442/0x680 fs/namei.c:1846
link_path_walk+0x66d/0xba0 fs/namei.c:2165
path_openat+0x21d/0x38b0 fs/namei.c:3342
do_filp_open+0x2b4/0x3a0 fs/namei.c:3375
do_sys_openat2+0x463/0x6f0 fs/open.c:1148
do_sys_open fs/open.c:1164 [inline]
ksys_open include/linux/syscalls.h:1386 [inline]
__do_sys_open fs/open.c:1170 [inline]
__se_sys_open fs/open.c:1168 [inline]
__x64_sys_open+0x1af/0x1e0 fs/open.c:1168
do_syscall_64+0xf3/0x1b0 arch/x86/entry/common.c:295
entry_SYSCALL_64_after_hwframe+0x49/0xb3
INITIAL USE at:
lock_acquire+0x169/0x480 kernel/locking/lockdep.c:4923
__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0x9e/0xc0 kernel/locking/spinlock.c:159
__wake_up_common_lock kernel/sched/wait.c:122 [inline]
__wake_up+0xb8/0x150 kernel/sched/wait.c:142
do_notify_pidfd kernel/signal.c:1900 [inline]
do_notify_parent+0x167/0xce0 kernel/signal.c:1927
exit_notify kernel/exit.c:660 [inline]
do_exit+0x12c5/0x1f80 kernel/exit.c:816
call_usermodehelper_exec_async+0x47c/0x480 kernel/umh.c:125
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352
}
... key at: [<ffffffff8aae6790>] alloc_pid.__key+0x0/0x10
... acquired at:
lock_acquire+0x169/0x480 kernel/locking/lockdep.c:4923
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:151
spin_lock include/linux/spinlock.h:353 [inline]
aio_poll fs/aio.c:1767 [inline]
__io_submit_one fs/aio.c:1841 [inline]
io_submit_one+0x10f5/0x1a80 fs/aio.c:1878
__do_sys_io_submit fs/aio.c:1937 [inline]
__se_sys_io_submit+0x117/0x220 fs/aio.c:1907
do_syscall_64+0xf3/0x1b0 arch/x86/entry/common.c:295
entry_SYSCALL_64_after_hwframe+0x49/0xb3


stack backtrace:
CPU: 0 PID: 11911 Comm: syz-executor.2 Not tainted 5.6.0-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x1e9/0x30e lib/dump_stack.c:118
print_bad_irq_dependency kernel/locking/lockdep.c:2132 [inline]
check_irq_usage kernel/locking/lockdep.c:2330 [inline]
check_prev_add kernel/locking/lockdep.c:2519 [inline]
check_prevs_add kernel/locking/lockdep.c:2620 [inline]
validate_chain+0x8479/0x8920 kernel/locking/lockdep.c:3237
__lock_acquire+0x116c/0x2b90 kernel/locking/lockdep.c:4344
lock_acquire+0x169/0x480 kernel/locking/lockdep.c:4923
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:151
spin_lock include/linux/spinlock.h:353 [inline]
aio_poll fs/aio.c:1767 [inline]
__io_submit_one fs/aio.c:1841 [inline]
io_submit_one+0x10f5/0x1a80 fs/aio.c:1878
__do_sys_io_submit fs/aio.c:1937 [inline]
__se_sys_io_submit+0x117/0x220 fs/aio.c:1907
do_syscall_64+0xf3/0x1b0 arch/x86/entry/common.c:295
entry_SYSCALL_64_after_hwframe+0x49/0xb3
RIP: 0033:0x45c849
Code: ad b6 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 7b b6 fb ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007f0fdf51ec78 EFLAGS: 00000246 ORIG_RAX: 00000000000000d1
RAX: ffffffffffffffda RBX: 00007f0fdf51f6d4 RCX: 000000000045c849
RDX: 0000000020000040 RSI: 0000000020000103 RDI: 00007f0fdf4fe000
RBP: 000000000076bf00 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff
R13: 00000000000001f9 R14: 00000000004c422b R15: 000000000076bf0c


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

syzbot

unread,
Apr 6, 2020, 9:36:14ā€ÆPM4/6/20
to bc...@kvack.org, linu...@kvack.org, linux-...@vger.kernel.org, linux-...@vger.kernel.org, syzkall...@googlegroups.com, vi...@zeniv.linux.org.uk
syzbot has found a reproducer for the following crash on:

HEAD commit: 7e634208 Merge tag 'acpi-5.7-rc1-2' of git://git.kernel.or..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=139b71c7e00000
kernel config: https://syzkaller.appspot.com/x/.config?x=12205d036cec317f
dashboard link: https://syzkaller.appspot.com/bug?extid=343f75cdeea091340956
compiler: gcc (GCC) 9.0.0 20181231 (experimental)
userspace arch: i386
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=105d592be00000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+343f75...@syzkaller.appspotmail.com

=====================================================
WARNING: SOFTIRQ-safe -> SOFTIRQ-unsafe lock order detected
5.6.0-syzkaller #0 Not tainted
-----------------------------------------------------
syz-executor.0/9005 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire:
ffff88808b9c1b48 (&pid->wait_pidfd){+.+.}-{2:2}, at: spin_lock include/linux/spinlock.h:353 [inline]
ffff88808b9c1b48 (&pid->wait_pidfd){+.+.}-{2:2}, at: aio_poll fs/aio.c:1767 [inline]
ffff88808b9c1b48 (&pid->wait_pidfd){+.+.}-{2:2}, at: __io_submit_one fs/aio.c:1841 [inline]
ffff88808b9c1b48 (&pid->wait_pidfd){+.+.}-{2:2}, at: io_submit_one+0xc1b/0x2ec0 fs/aio.c:1878

and this task is already holding:
ffff8880a81b1cd8 (&ctx->ctx_lock){..-.}-{2:2}, at: spin_lock_irq include/linux/spinlock.h:378 [inline]
ffff8880a81b1cd8 (&ctx->ctx_lock){..-.}-{2:2}, at: aio_poll fs/aio.c:1765 [inline]
ffff8880a81b1cd8 (&ctx->ctx_lock){..-.}-{2:2}, at: __io_submit_one fs/aio.c:1841 [inline]
ffff8880a81b1cd8 (&ctx->ctx_lock){..-.}-{2:2}, at: io_submit_one+0xbd6/0x2ec0 fs/aio.c:1878
which would create a new lock dependency:
(&ctx->ctx_lock){..-.}-{2:2} -> (&pid->wait_pidfd){+.+.}-{2:2}

but this new dependency connects a SOFTIRQ-irq-safe lock:
(&ctx->ctx_lock){..-.}-{2:2}

... which became SOFTIRQ-irq-safe at:
lock_acquire+0x1f2/0x8f0 kernel/locking/lockdep.c:4923
__raw_spin_lock_irq include/linux/spinlock_api_smp.h:128 [inline]
_raw_spin_lock_irq+0x5b/0x80 kernel/locking/spinlock.c:167
spin_lock_irq include/linux/spinlock.h:378 [inline]
free_ioctx_users+0x2b/0x450 fs/aio.c:618
percpu_ref_put_many include/linux/percpu-refcount.h:309 [inline]
percpu_ref_put include/linux/percpu-refcount.h:325 [inline]
percpu_ref_call_confirm_rcu lib/percpu-refcount.c:131 [inline]
percpu_ref_switch_to_atomic_rcu+0x494/0x540 lib/percpu-refcount.c:166
rcu_do_batch kernel/rcu/tree.c:2206 [inline]
rcu_core+0x59f/0x1370 kernel/rcu/tree.c:2433
__do_softirq+0x26c/0x9f7 kernel/softirq.c:292
invoke_softirq kernel/softirq.c:373 [inline]
irq_exit+0x192/0x1d0 kernel/softirq.c:413
exiting_irq arch/x86/include/asm/apic.h:546 [inline]
smp_apic_timer_interrupt+0x19e/0x600 arch/x86/kernel/apic/apic.c:1140
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:829
preempt_count_add+0x0/0x140 kernel/sched/core.c:6758
__raw_spin_lock include/linux/spinlock_api_smp.h:141 [inline]
_raw_spin_lock+0xe/0x40 kernel/locking/spinlock.c:151
spin_lock include/linux/spinlock.h:353 [inline]
lockref_put_or_lock+0x14/0x80 lib/lockref.c:174
fast_dput fs/dcache.c:728 [inline]
dput+0x4a3/0xdf0 fs/dcache.c:846
path_put+0x2d/0x60 fs/namei.c:482
vfs_statx+0x14d/0x1e0 fs/stat.c:202
vfs_stat include/linux/fs.h:3279 [inline]
__do_sys_newstat+0x96/0x120 fs/stat.c:351
do_syscall_64+0xf6/0x7d0 arch/x86/entry/common.c:295
entry_SYSCALL_64_after_hwframe+0x49/0xb3

to a SOFTIRQ-irq-unsafe lock:
(&pid->wait_pidfd){+.+.}-{2:2}

... which became SOFTIRQ-irq-unsafe at:
...
lock_acquire+0x1f2/0x8f0 kernel/locking/lockdep.c:4923
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:151
spin_lock include/linux/spinlock.h:353 [inline]
proc_pid_make_inode+0x1f9/0x3c0 fs/proc/base.c:1880
proc_pid_instantiate+0x51/0x150 fs/proc/base.c:3285
proc_pid_lookup+0x1da/0x340 fs/proc/base.c:3320
proc_root_lookup+0x20/0x60 fs/proc/root.c:243
__lookup_slow+0x256/0x490 fs/namei.c:1530
lookup_slow fs/namei.c:1547 [inline]
walk_component+0x418/0x6a0 fs/namei.c:1846
link_path_walk.part.0+0x4f1/0xb50 fs/namei.c:2166
link_path_walk fs/namei.c:2098 [inline]
path_openat+0x25a/0x27d0 fs/namei.c:3342
do_filp_open+0x192/0x260 fs/namei.c:3373
do_sys_openat2+0x585/0x7d0 fs/open.c:1148
do_sys_open+0xc3/0x140 fs/open.c:1164
do_syscall_64+0xf6/0x7d0 arch/x86/entry/common.c:295
entry_SYSCALL_64_after_hwframe+0x49/0xb3

other info that might help us debug this:

Possible interrupt unsafe locking scenario:

CPU0 CPU1
---- ----
lock(&pid->wait_pidfd);
local_irq_disable();
lock(&ctx->ctx_lock);
lock(&pid->wait_pidfd);
<Interrupt>
lock(&ctx->ctx_lock);

*** DEADLOCK ***

1 lock held by syz-executor.0/9005:
#0: ffff8880a81b1cd8 (&ctx->ctx_lock){..-.}-{2:2}, at: spin_lock_irq include/linux/spinlock.h:378 [inline]
#0: ffff8880a81b1cd8 (&ctx->ctx_lock){..-.}-{2:2}, at: aio_poll fs/aio.c:1765 [inline]
#0: ffff8880a81b1cd8 (&ctx->ctx_lock){..-.}-{2:2}, at: __io_submit_one fs/aio.c:1841 [inline]
#0: ffff8880a81b1cd8 (&ctx->ctx_lock){..-.}-{2:2}, at: io_submit_one+0xbd6/0x2ec0 fs/aio.c:1878

the dependencies between SOFTIRQ-irq-safe lock and the holding lock:
-> (&ctx->ctx_lock){..-.}-{2:2} {
IN-SOFTIRQ-W at:
lock_acquire+0x1f2/0x8f0 kernel/locking/lockdep.c:4923
__raw_spin_lock_irq include/linux/spinlock_api_smp.h:128 [inline]
_raw_spin_lock_irq+0x5b/0x80 kernel/locking/spinlock.c:167
spin_lock_irq include/linux/spinlock.h:378 [inline]
free_ioctx_users+0x2b/0x450 fs/aio.c:618
percpu_ref_put_many include/linux/percpu-refcount.h:309 [inline]
percpu_ref_put include/linux/percpu-refcount.h:325 [inline]
percpu_ref_call_confirm_rcu lib/percpu-refcount.c:131 [inline]
percpu_ref_switch_to_atomic_rcu+0x494/0x540 lib/percpu-refcount.c:166
rcu_do_batch kernel/rcu/tree.c:2206 [inline]
rcu_core+0x59f/0x1370 kernel/rcu/tree.c:2433
__do_softirq+0x26c/0x9f7 kernel/softirq.c:292
invoke_softirq kernel/softirq.c:373 [inline]
irq_exit+0x192/0x1d0 kernel/softirq.c:413
exiting_irq arch/x86/include/asm/apic.h:546 [inline]
smp_apic_timer_interrupt+0x19e/0x600 arch/x86/kernel/apic/apic.c:1140
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:829
preempt_count_add+0x0/0x140 kernel/sched/core.c:6758
__raw_spin_lock include/linux/spinlock_api_smp.h:141 [inline]
_raw_spin_lock+0xe/0x40 kernel/locking/spinlock.c:151
spin_lock include/linux/spinlock.h:353 [inline]
lockref_put_or_lock+0x14/0x80 lib/lockref.c:174
fast_dput fs/dcache.c:728 [inline]
dput+0x4a3/0xdf0 fs/dcache.c:846
path_put+0x2d/0x60 fs/namei.c:482
vfs_statx+0x14d/0x1e0 fs/stat.c:202
vfs_stat include/linux/fs.h:3279 [inline]
__do_sys_newstat+0x96/0x120 fs/stat.c:351
do_syscall_64+0xf6/0x7d0 arch/x86/entry/common.c:295
entry_SYSCALL_64_after_hwframe+0x49/0xb3
INITIAL USE at:
lock_acquire+0x1f2/0x8f0 kernel/locking/lockdep.c:4923
__raw_spin_lock_irq include/linux/spinlock_api_smp.h:128 [inline]
_raw_spin_lock_irq+0x5b/0x80 kernel/locking/spinlock.c:167
spin_lock_irq include/linux/spinlock.h:378 [inline]
free_ioctx_users+0x2b/0x450 fs/aio.c:618
percpu_ref_put_many include/linux/percpu-refcount.h:309 [inline]
percpu_ref_put include/linux/percpu-refcount.h:325 [inline]
percpu_ref_call_confirm_rcu lib/percpu-refcount.c:131 [inline]
percpu_ref_switch_to_atomic_rcu+0x494/0x540 lib/percpu-refcount.c:166
rcu_do_batch kernel/rcu/tree.c:2206 [inline]
rcu_core+0x59f/0x1370 kernel/rcu/tree.c:2433
__do_softirq+0x26c/0x9f7 kernel/softirq.c:292
invoke_softirq kernel/softirq.c:373 [inline]
irq_exit+0x192/0x1d0 kernel/softirq.c:413
exiting_irq arch/x86/include/asm/apic.h:546 [inline]
smp_apic_timer_interrupt+0x19e/0x600 arch/x86/kernel/apic/apic.c:1140
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:829
preempt_count_add+0x0/0x140 kernel/sched/core.c:6758
__raw_spin_lock include/linux/spinlock_api_smp.h:141 [inline]
_raw_spin_lock+0xe/0x40 kernel/locking/spinlock.c:151
spin_lock include/linux/spinlock.h:353 [inline]
lockref_put_or_lock+0x14/0x80 lib/lockref.c:174
fast_dput fs/dcache.c:728 [inline]
dput+0x4a3/0xdf0 fs/dcache.c:846
path_put+0x2d/0x60 fs/namei.c:482
vfs_statx+0x14d/0x1e0 fs/stat.c:202
vfs_stat include/linux/fs.h:3279 [inline]
__do_sys_newstat+0x96/0x120 fs/stat.c:351
do_syscall_64+0xf6/0x7d0 arch/x86/entry/common.c:295
entry_SYSCALL_64_after_hwframe+0x49/0xb3
}
... key at: [<ffffffff8c67af20>] __key.55262+0x0/0x40
... acquired at:
lock_acquire+0x1f2/0x8f0 kernel/locking/lockdep.c:4923
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:151
spin_lock include/linux/spinlock.h:353 [inline]
aio_poll fs/aio.c:1767 [inline]
__io_submit_one fs/aio.c:1841 [inline]
io_submit_one+0xc1b/0x2ec0 fs/aio.c:1878
__do_compat_sys_io_submit fs/aio.c:1979 [inline]
__se_compat_sys_io_submit fs/aio.c:1949 [inline]
__ia32_compat_sys_io_submit+0x1bf/0x530 fs/aio.c:1949
do_syscall_32_irqs_on arch/x86/entry/common.c:337 [inline]
do_fast_syscall_32+0x270/0xe90 arch/x86/entry/common.c:396
entry_SYSENTER_compat+0x70/0x7f arch/x86/entry/entry_64_compat.S:139


the dependencies between the lock to be acquired
and SOFTIRQ-irq-unsafe lock:
-> (&pid->wait_pidfd){+.+.}-{2:2} {
HARDIRQ-ON-W at:
lock_acquire+0x1f2/0x8f0 kernel/locking/lockdep.c:4923
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:151
spin_lock include/linux/spinlock.h:353 [inline]
proc_pid_make_inode+0x1f9/0x3c0 fs/proc/base.c:1880
proc_pid_instantiate+0x51/0x150 fs/proc/base.c:3285
proc_pid_lookup+0x1da/0x340 fs/proc/base.c:3320
proc_root_lookup+0x20/0x60 fs/proc/root.c:243
__lookup_slow+0x256/0x490 fs/namei.c:1530
lookup_slow fs/namei.c:1547 [inline]
walk_component+0x418/0x6a0 fs/namei.c:1846
link_path_walk.part.0+0x4f1/0xb50 fs/namei.c:2166
link_path_walk fs/namei.c:2098 [inline]
path_openat+0x25a/0x27d0 fs/namei.c:3342
do_filp_open+0x192/0x260 fs/namei.c:3373
do_sys_openat2+0x585/0x7d0 fs/open.c:1148
do_sys_open+0xc3/0x140 fs/open.c:1164
do_syscall_64+0xf6/0x7d0 arch/x86/entry/common.c:295
entry_SYSCALL_64_after_hwframe+0x49/0xb3
SOFTIRQ-ON-W at:
lock_acquire+0x1f2/0x8f0 kernel/locking/lockdep.c:4923
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:151
spin_lock include/linux/spinlock.h:353 [inline]
proc_pid_make_inode+0x1f9/0x3c0 fs/proc/base.c:1880
proc_pid_instantiate+0x51/0x150 fs/proc/base.c:3285
proc_pid_lookup+0x1da/0x340 fs/proc/base.c:3320
proc_root_lookup+0x20/0x60 fs/proc/root.c:243
__lookup_slow+0x256/0x490 fs/namei.c:1530
lookup_slow fs/namei.c:1547 [inline]
walk_component+0x418/0x6a0 fs/namei.c:1846
link_path_walk.part.0+0x4f1/0xb50 fs/namei.c:2166
link_path_walk fs/namei.c:2098 [inline]
path_openat+0x25a/0x27d0 fs/namei.c:3342
do_filp_open+0x192/0x260 fs/namei.c:3373
do_sys_openat2+0x585/0x7d0 fs/open.c:1148
do_sys_open+0xc3/0x140 fs/open.c:1164
do_syscall_64+0xf6/0x7d0 arch/x86/entry/common.c:295
entry_SYSCALL_64_after_hwframe+0x49/0xb3
INITIAL USE at:
lock_acquire+0x1f2/0x8f0 kernel/locking/lockdep.c:4923
__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0x8c/0xbf kernel/locking/spinlock.c:159
__wake_up_common_lock+0xb4/0x130 kernel/sched/wait.c:122
do_notify_pidfd kernel/signal.c:1900 [inline]
do_notify_parent+0x19e/0xe60 kernel/signal.c:1927
exit_notify kernel/exit.c:660 [inline]
do_exit+0x238f/0x2dd0 kernel/exit.c:816
call_usermodehelper_exec_async+0x507/0x710 kernel/umh.c:125
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352
}
... key at: [<ffffffff8bbbe680>] __key.53786+0x0/0x40
... acquired at:
lock_acquire+0x1f2/0x8f0 kernel/locking/lockdep.c:4923
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:151
spin_lock include/linux/spinlock.h:353 [inline]
aio_poll fs/aio.c:1767 [inline]
__io_submit_one fs/aio.c:1841 [inline]
io_submit_one+0xc1b/0x2ec0 fs/aio.c:1878
__do_compat_sys_io_submit fs/aio.c:1979 [inline]
__se_compat_sys_io_submit fs/aio.c:1949 [inline]
__ia32_compat_sys_io_submit+0x1bf/0x530 fs/aio.c:1949
do_syscall_32_irqs_on arch/x86/entry/common.c:337 [inline]
do_fast_syscall_32+0x270/0xe90 arch/x86/entry/common.c:396
entry_SYSENTER_compat+0x70/0x7f arch/x86/entry/entry_64_compat.S:139


stack backtrace:
CPU: 0 PID: 9005 Comm: syz-executor.0 Not tainted 5.6.0-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x188/0x20d lib/dump_stack.c:118
print_bad_irq_dependency kernel/locking/lockdep.c:2132 [inline]
check_irq_usage.cold+0x566/0x6de kernel/locking/lockdep.c:2330
check_prev_add kernel/locking/lockdep.c:2519 [inline]
check_prevs_add kernel/locking/lockdep.c:2620 [inline]
validate_chain kernel/locking/lockdep.c:3237 [inline]
__lock_acquire+0x2c39/0x4e00 kernel/locking/lockdep.c:4344
lock_acquire+0x1f2/0x8f0 kernel/locking/lockdep.c:4923
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:151
spin_lock include/linux/spinlock.h:353 [inline]
aio_poll fs/aio.c:1767 [inline]
__io_submit_one fs/aio.c:1841 [inline]
io_submit_one+0xc1b/0x2ec0 fs/aio.c:1878
__do_compat_sys_io_submit fs/aio.c:1979 [inline]
__se_compat_sys_io_submit fs/aio.c:1949 [inline]
__ia32_compat_sys_io_submit+0x1bf/0x530 fs/aio.c:1949
do_syscall_32_irqs_on arch/x86/entry/common.c:337 [inline]
do_fast_syscall_32+0x270/0xe90 arch/x86/entry/common.c:396
entry_SYSENTER_compat+0x70/0x7f arch/x86/entry/entry_64_compat.S:139

syzbot

unread,
Apr 7, 2020, 5:31:03ā€ÆAM4/7/20
to adob...@gmail.com, ak...@linux-foundation.org, all...@lohutok.net, are...@redhat.com, aubr...@linux.intel.com, ava...@gmail.com, bc...@kvack.org, chri...@brauner.io, cyp...@cyphar.com, ebie...@xmission.com, gre...@linuxfoundation.org, gu...@fb.com, jo...@joelfernandes.org, kees...@chromium.org, linm...@huawei.com, linu...@kvack.org, linux-...@vger.kernel.org, linux-...@vger.kernel.org, mho...@suse.com, mi...@kernel.org, ol...@redhat.com, pet...@infradead.org, sar...@sargun.me, syzkall...@googlegroups.com, tg...@linutronix.de, vi...@zeniv.linux.org.uk
syzbot has bisected this bug to:

commit 7bc3e6e55acf065500a24621f3b313e7e5998acf
Author: Eric W. Biederman <ebie...@xmission.com>
Date: Thu Feb 20 00:22:26 2020 +0000

proc: Use a list of inodes to flush from proc

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=12b3c9c7e00000
start commit: 7e634208 Merge tag 'acpi-5.7-rc1-2' of git://git.kernel.or..
git tree: upstream
final crash: https://syzkaller.appspot.com/x/report.txt?x=11b3c9c7e00000
console output: https://syzkaller.appspot.com/x/log.txt?x=16b3c9c7e00000
Reported-by: syzbot+343f75...@syzkaller.appspotmail.com
Fixes: 7bc3e6e55acf ("proc: Use a list of inodes to flush from proc")

For information about bisection process see: https://goo.gl/tpsmEJ#bisection
Reply all
Reply to author
Forward
0 new messages