possible deadlock in aio_poll

23 views
Skip to first unread message

syzbot

unread,
Sep 10, 2018, 3:41:05 AM9/10/18
to bc...@kvack.org, linu...@kvack.org, linux-...@vger.kernel.org, linux-...@vger.kernel.org, syzkall...@googlegroups.com, vi...@zeniv.linux.org.uk
Hello,

syzbot found the following crash on:

HEAD commit: f8f65382c98a Merge tag 'for-linus' of git://git.kernel.org..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=1587e266400000
kernel config: https://syzkaller.appspot.com/x/.config?x=8f59875069d721b6
dashboard link: https://syzkaller.appspot.com/bug?extid=5b1df0420c523b45a953
compiler: gcc (GCC) 8.0.1 20180413 (experimental)
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1753bdca400000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+5b1df0...@syzkaller.appspotmail.com

8021q: adding VLAN 0 to HW filter on device team0

=====================================================
WARNING: SOFTIRQ-safe -> SOFTIRQ-unsafe lock order detected
4.19.0-rc2+ #229 Not tainted
-----------------------------------------------------
syz-executor2/9399 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire:
00000000126506e0 (&ctx->fd_wqh){+.+.}, at: spin_lock
include/linux/spinlock.h:329 [inline]
00000000126506e0 (&ctx->fd_wqh){+.+.}, at: aio_poll+0x760/0x1420
fs/aio.c:1747

and this task is already holding:
000000002bed6bf6 (&(&ctx->ctx_lock)->rlock){..-.}, at: spin_lock_irq
include/linux/spinlock.h:354 [inline]
000000002bed6bf6 (&(&ctx->ctx_lock)->rlock){..-.}, at:
aio_poll+0x738/0x1420 fs/aio.c:1746
which would create a new lock dependency:
(&(&ctx->ctx_lock)->rlock){..-.} -> (&ctx->fd_wqh){+.+.}

but this new dependency connects a SOFTIRQ-irq-safe lock:
(&(&ctx->ctx_lock)->rlock){..-.}

... which became SOFTIRQ-irq-safe at:
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3901
__raw_spin_lock_irq include/linux/spinlock_api_smp.h:128 [inline]
_raw_spin_lock_irq+0x61/0x80 kernel/locking/spinlock.c:160
spin_lock_irq include/linux/spinlock.h:354 [inline]
free_ioctx_users+0xbc/0x710 fs/aio.c:603
percpu_ref_put_many include/linux/percpu-refcount.h:284 [inline]
percpu_ref_put include/linux/percpu-refcount.h:300 [inline]
percpu_ref_call_confirm_rcu lib/percpu-refcount.c:123 [inline]
percpu_ref_switch_to_atomic_rcu+0x62c/0x820 lib/percpu-refcount.c:158
__rcu_reclaim kernel/rcu/rcu.h:236 [inline]
rcu_do_batch kernel/rcu/tree.c:2576 [inline]
invoke_rcu_callbacks kernel/rcu/tree.c:2880 [inline]
__rcu_process_callbacks kernel/rcu/tree.c:2847 [inline]
rcu_process_callbacks+0xf23/0x2670 kernel/rcu/tree.c:2864
__do_softirq+0x30b/0xad8 kernel/softirq.c:292
invoke_softirq kernel/softirq.c:372 [inline]
irq_exit+0x17f/0x1c0 kernel/softirq.c:412
exiting_irq arch/x86/include/asm/apic.h:536 [inline]
smp_apic_timer_interrupt+0x1cb/0x760 arch/x86/kernel/apic/apic.c:1056
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:864
native_safe_halt+0x6/0x10 arch/x86/include/asm/irqflags.h:57
arch_safe_halt arch/x86/include/asm/paravirt.h:94 [inline]
default_idle+0xbf/0x490 arch/x86/kernel/process.c:498
arch_cpu_idle+0x10/0x20 arch/x86/kernel/process.c:489
default_idle_call+0x6d/0x90 kernel/sched/idle.c:93
cpuidle_idle_call kernel/sched/idle.c:153 [inline]
do_idle+0x3db/0x5b0 kernel/sched/idle.c:262
cpu_startup_entry+0x10c/0x120 kernel/sched/idle.c:368
start_secondary+0x523/0x750 arch/x86/kernel/smpboot.c:271
secondary_startup_64+0xa4/0xb0 arch/x86/kernel/head_64.S:242

to a SOFTIRQ-irq-unsafe lock:
(&ctx->fd_wqh){+.+.}

... which became SOFTIRQ-irq-unsafe at:
...
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3901
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2d/0x40 kernel/locking/spinlock.c:144
spin_lock include/linux/spinlock.h:329 [inline]
userfaultfd_ctx_read+0x2e4/0x2180 fs/userfaultfd.c:1029
userfaultfd_read+0x1e2/0x2c0 fs/userfaultfd.c:1191
do_loop_readv_writev fs/read_write.c:700 [inline]
do_iter_read+0x4a3/0x650 fs/read_write.c:924
vfs_readv+0x175/0x1c0 fs/read_write.c:986
do_readv+0x11a/0x310 fs/read_write.c:1019
__do_sys_readv fs/read_write.c:1106 [inline]
__se_sys_readv fs/read_write.c:1103 [inline]
__x64_sys_readv+0x75/0xb0 fs/read_write.c:1103
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe

other info that might help us debug this:

Possible interrupt unsafe locking scenario:

CPU0 CPU1
---- ----
lock(&ctx->fd_wqh);
local_irq_disable();
lock(&(&ctx->ctx_lock)->rlock);
lock(&ctx->fd_wqh);
<Interrupt>
lock(&(&ctx->ctx_lock)->rlock);

*** DEADLOCK ***

1 lock held by syz-executor2/9399:
#0: 000000002bed6bf6 (&(&ctx->ctx_lock)->rlock){..-.}, at: spin_lock_irq
include/linux/spinlock.h:354 [inline]
#0: 000000002bed6bf6 (&(&ctx->ctx_lock)->rlock){..-.}, at:
aio_poll+0x738/0x1420 fs/aio.c:1746

the dependencies between SOFTIRQ-irq-safe lock and the holding lock:
-> (&(&ctx->ctx_lock)->rlock){..-.} ops: 387 {
IN-SOFTIRQ-W at:
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3901
__raw_spin_lock_irq
include/linux/spinlock_api_smp.h:128 [inline]
_raw_spin_lock_irq+0x61/0x80
kernel/locking/spinlock.c:160
spin_lock_irq include/linux/spinlock.h:354 [inline]
free_ioctx_users+0xbc/0x710 fs/aio.c:603
percpu_ref_put_many include/linux/percpu-refcount.h:284
[inline]
percpu_ref_put include/linux/percpu-refcount.h:300
[inline]
percpu_ref_call_confirm_rcu lib/percpu-refcount.c:123
[inline]
percpu_ref_switch_to_atomic_rcu+0x62c/0x820
lib/percpu-refcount.c:158
__rcu_reclaim kernel/rcu/rcu.h:236 [inline]
rcu_do_batch kernel/rcu/tree.c:2576 [inline]
invoke_rcu_callbacks kernel/rcu/tree.c:2880 [inline]
__rcu_process_callbacks kernel/rcu/tree.c:2847 [inline]
rcu_process_callbacks+0xf23/0x2670
kernel/rcu/tree.c:2864
__do_softirq+0x30b/0xad8 kernel/softirq.c:292
invoke_softirq kernel/softirq.c:372 [inline]
irq_exit+0x17f/0x1c0 kernel/softirq.c:412
exiting_irq arch/x86/include/asm/apic.h:536 [inline]
smp_apic_timer_interrupt+0x1cb/0x760
arch/x86/kernel/apic/apic.c:1056
apic_timer_interrupt+0xf/0x20
arch/x86/entry/entry_64.S:864
native_safe_halt+0x6/0x10
arch/x86/include/asm/irqflags.h:57
arch_safe_halt arch/x86/include/asm/paravirt.h:94
[inline]
default_idle+0xbf/0x490 arch/x86/kernel/process.c:498
arch_cpu_idle+0x10/0x20 arch/x86/kernel/process.c:489
default_idle_call+0x6d/0x90 kernel/sched/idle.c:93
cpuidle_idle_call kernel/sched/idle.c:153 [inline]
do_idle+0x3db/0x5b0 kernel/sched/idle.c:262
cpu_startup_entry+0x10c/0x120 kernel/sched/idle.c:368
start_secondary+0x523/0x750
arch/x86/kernel/smpboot.c:271
secondary_startup_64+0xa4/0xb0
arch/x86/kernel/head_64.S:242
INITIAL USE at:
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3901
__raw_spin_lock_irq include/linux/spinlock_api_smp.h:128
[inline]
_raw_spin_lock_irq+0x61/0x80
kernel/locking/spinlock.c:160
spin_lock_irq include/linux/spinlock.h:354 [inline]
free_ioctx_users+0xbc/0x710 fs/aio.c:603
percpu_ref_put_many include/linux/percpu-refcount.h:284
[inline]
percpu_ref_put include/linux/percpu-refcount.h:300
[inline]
percpu_ref_call_confirm_rcu lib/percpu-refcount.c:123
[inline]
percpu_ref_switch_to_atomic_rcu+0x62c/0x820
lib/percpu-refcount.c:158
__rcu_reclaim kernel/rcu/rcu.h:236 [inline]
rcu_do_batch kernel/rcu/tree.c:2576 [inline]
invoke_rcu_callbacks kernel/rcu/tree.c:2880 [inline]
__rcu_process_callbacks kernel/rcu/tree.c:2847 [inline]
rcu_process_callbacks+0xf23/0x2670 kernel/rcu/tree.c:2864
__do_softirq+0x30b/0xad8 kernel/softirq.c:292
invoke_softirq kernel/softirq.c:372 [inline]
irq_exit+0x17f/0x1c0 kernel/softirq.c:412
exiting_irq arch/x86/include/asm/apic.h:536 [inline]
smp_apic_timer_interrupt+0x1cb/0x760
arch/x86/kernel/apic/apic.c:1056
apic_timer_interrupt+0xf/0x20
arch/x86/entry/entry_64.S:864
native_safe_halt+0x6/0x10
arch/x86/include/asm/irqflags.h:57
arch_safe_halt arch/x86/include/asm/paravirt.h:94
[inline]
default_idle+0xbf/0x490 arch/x86/kernel/process.c:498
arch_cpu_idle+0x10/0x20 arch/x86/kernel/process.c:489
default_idle_call+0x6d/0x90 kernel/sched/idle.c:93
cpuidle_idle_call kernel/sched/idle.c:153 [inline]
do_idle+0x3db/0x5b0 kernel/sched/idle.c:262
cpu_startup_entry+0x10c/0x120 kernel/sched/idle.c:368
start_secondary+0x523/0x750 arch/x86/kernel/smpboot.c:271
secondary_startup_64+0xa4/0xb0
arch/x86/kernel/head_64.S:242
}
... key at: [<ffffffff8b3dc960>] __key.50120+0x0/0x40
... acquired at:
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3901
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2d/0x40 kernel/locking/spinlock.c:144
spin_lock include/linux/spinlock.h:329 [inline]
aio_poll+0x760/0x1420 fs/aio.c:1747
io_submit_one+0xab8/0x1090 fs/aio.c:1850
__do_sys_io_submit fs/aio.c:1916 [inline]
__se_sys_io_submit fs/aio.c:1887 [inline]
__x64_sys_io_submit+0x1b9/0x5d0 fs/aio.c:1887
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe


the dependencies between the lock to be acquired
and SOFTIRQ-irq-unsafe lock:
-> (&ctx->fd_wqh){+.+.} ops: 2209 {
HARDIRQ-ON-W at:
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3901
__raw_spin_lock include/linux/spinlock_api_smp.h:142
[inline]
_raw_spin_lock+0x2d/0x40 kernel/locking/spinlock.c:144
spin_lock include/linux/spinlock.h:329 [inline]
userfaultfd_ctx_read+0x2e4/0x2180 fs/userfaultfd.c:1029
userfaultfd_read+0x1e2/0x2c0 fs/userfaultfd.c:1191
do_loop_readv_writev fs/read_write.c:700 [inline]
do_iter_read+0x4a3/0x650 fs/read_write.c:924
vfs_readv+0x175/0x1c0 fs/read_write.c:986
do_readv+0x11a/0x310 fs/read_write.c:1019
__do_sys_readv fs/read_write.c:1106 [inline]
__se_sys_readv fs/read_write.c:1103 [inline]
__x64_sys_readv+0x75/0xb0 fs/read_write.c:1103
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
SOFTIRQ-ON-W at:
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3901
__raw_spin_lock include/linux/spinlock_api_smp.h:142
[inline]
_raw_spin_lock+0x2d/0x40 kernel/locking/spinlock.c:144
spin_lock include/linux/spinlock.h:329 [inline]
userfaultfd_ctx_read+0x2e4/0x2180 fs/userfaultfd.c:1029
userfaultfd_read+0x1e2/0x2c0 fs/userfaultfd.c:1191
do_loop_readv_writev fs/read_write.c:700 [inline]
do_iter_read+0x4a3/0x650 fs/read_write.c:924
vfs_readv+0x175/0x1c0 fs/read_write.c:986
do_readv+0x11a/0x310 fs/read_write.c:1019
__do_sys_readv fs/read_write.c:1106 [inline]
__se_sys_readv fs/read_write.c:1103 [inline]
__x64_sys_readv+0x75/0xb0 fs/read_write.c:1103
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
INITIAL USE at:
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3901
__raw_spin_lock include/linux/spinlock_api_smp.h:142
[inline]
_raw_spin_lock+0x2d/0x40 kernel/locking/spinlock.c:144
spin_lock include/linux/spinlock.h:329 [inline]
userfaultfd_ctx_read+0x2e4/0x2180 fs/userfaultfd.c:1029
userfaultfd_read+0x1e2/0x2c0 fs/userfaultfd.c:1191
do_loop_readv_writev fs/read_write.c:700 [inline]
do_iter_read+0x4a3/0x650 fs/read_write.c:924
vfs_readv+0x175/0x1c0 fs/read_write.c:986
do_readv+0x11a/0x310 fs/read_write.c:1019
__do_sys_readv fs/read_write.c:1106 [inline]
__se_sys_readv fs/read_write.c:1103 [inline]
__x64_sys_readv+0x75/0xb0 fs/read_write.c:1103
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
}
... key at: [<ffffffff8b3dc6e0>] __key.43670+0x0/0x40
... acquired at:
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3901
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2d/0x40 kernel/locking/spinlock.c:144
spin_lock include/linux/spinlock.h:329 [inline]
aio_poll+0x760/0x1420 fs/aio.c:1747
io_submit_one+0xab8/0x1090 fs/aio.c:1850
__do_sys_io_submit fs/aio.c:1916 [inline]
__se_sys_io_submit fs/aio.c:1887 [inline]
__x64_sys_io_submit+0x1b9/0x5d0 fs/aio.c:1887
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe


stack backtrace:
CPU: 0 PID: 9399 Comm: syz-executor2 Not tainted 4.19.0-rc2+ #229
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x1c4/0x2b4 lib/dump_stack.c:113
print_bad_irq_dependency kernel/locking/lockdep.c:1569 [inline]
check_usage.cold.58+0x6d5/0xad1 kernel/locking/lockdep.c:1601
check_irq_usage kernel/locking/lockdep.c:1657 [inline]
check_prev_add_irq kernel/locking/lockdep_states.h:8 [inline]
check_prev_add kernel/locking/lockdep.c:1867 [inline]
check_prevs_add kernel/locking/lockdep.c:1975 [inline]
validate_chain kernel/locking/lockdep.c:2416 [inline]
__lock_acquire+0x2400/0x4ec0 kernel/locking/lockdep.c:3412
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3901
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2d/0x40 kernel/locking/spinlock.c:144
spin_lock include/linux/spinlock.h:329 [inline]
aio_poll+0x760/0x1420 fs/aio.c:1747
io_submit_one+0xab8/0x1090 fs/aio.c:1850
__do_sys_io_submit fs/aio.c:1916 [inline]
__se_sys_io_submit fs/aio.c:1887 [inline]
__x64_sys_io_submit+0x1b9/0x5d0 fs/aio.c:1887
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x457099
Code: fd b4 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7
48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff
ff 0f 83 cb b4 fb ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007fa4bd11bc78 EFLAGS: 00000246 ORIG_RAX: 00000000000000d1
RAX: ffffffffffffffda RBX: 00007fa4bd11c6d4 RCX: 0000000000457099
RDX: 0000000020000b00 RSI: 0000000000000001 RDI: 00007fa4bd13e000
RBP: 00000000009301e0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff
R13: 00000000004cd990 R14: 00000000004c40a7 R15: 0000000000000002
kobject: 'loop5' (0000000044e01f3d): kobject_uevent_env
kobject: 'loop5' (0000000044e01f3d): fill_kobj_path: path
= '/devices/virtual/block/loop5'
kobject: 'loop6' (00000000790743e9): kobject_uevent_env
kobject: 'loop6' (00000000790743e9): fill_kobj_path: path
= '/devices/virtual/block/loop6'
kobject: 'loop0' (000000005501af28): kobject_uevent_env
kobject: 'loop0' (000000005501af28): fill_kobj_path: path
= '/devices/virtual/block/loop0'
kobject: 'loop3' (000000001d0a0601): kobject_uevent_env
kobject: 'loop3' (000000001d0a0601): fill_kobj_path: path
= '/devices/virtual/block/loop3'
kobject: 'loop1' (00000000fd2f09a6): kobject_uevent_env
kobject: 'loop1' (00000000fd2f09a6): fill_kobj_path: path
= '/devices/virtual/block/loop1'
kobject: 'loop4' (000000003f6c580a): kobject_uevent_env
kobject: 'loop4' (000000003f6c580a): fill_kobj_path: path
= '/devices/virtual/block/loop4'
kobject: 'loop7' (00000000b52cfd8a): kobject_uevent_env
kobject: 'loop7' (00000000b52cfd8a): fill_kobj_path: path
= '/devices/virtual/block/loop7'
kobject: 'loop2' (000000002d051810): kobject_uevent_env
kobject: 'loop2' (000000002d051810): fill_kobj_path: path
= '/devices/virtual/block/loop2'
kobject: 'loop6' (00000000790743e9): kobject_uevent_env
kobject: 'loop6' (00000000790743e9): fill_kobj_path: path
= '/devices/virtual/block/loop6'
kobject: 'loop5' (0000000044e01f3d): kobject_uevent_env
kobject: 'loop5' (0000000044e01f3d): fill_kobj_path: path
= '/devices/virtual/block/loop5'
kobject: 'loop7' (00000000b52cfd8a): kobject_uevent_env
kobject: 'loop7' (00000000b52cfd8a): fill_kobj_path: path
= '/devices/virtual/block/loop7'
kobject: 'loop3' (000000001d0a0601): kobject_uevent_env
kobject: 'loop3' (000000001d0a0601): fill_kobj_path: path
= '/devices/virtual/block/loop3'
kobject: 'loop0' (000000005501af28): kobject_uevent_env
kobject: 'loop0' (000000005501af28): fill_kobj_path: path
= '/devices/virtual/block/loop0'
kobject: 'loop1' (00000000fd2f09a6): kobject_uevent_env
kobject: 'loop1' (00000000fd2f09a6): fill_kobj_path: path
= '/devices/virtual/block/loop1'
kobject: 'loop4' (000000003f6c580a): kobject_uevent_env
kobject: 'loop4' (000000003f6c580a): fill_kobj_path: path
= '/devices/virtual/block/loop4'
kobject: 'loop2' (000000002d051810): kobject_uevent_env
kobject: 'loop2' (000000002d051810): fill_kobj_path: path
= '/devices/virtual/block/loop2'
kobject: 'loop6' (00000000790743e9): kobject_uevent_env
kobject: 'loop6' (00000000790743e9): fill_kobj_path: path
= '/devices/virtual/block/loop6'
kobject: 'loop5' (0000000044e01f3d): kobject_uevent_env
kobject: 'loop5' (0000000044e01f3d): fill_kobj_path: path
= '/devices/virtual/block/loop5'
kobject: 'loop1' (00000000fd2f09a6): kobject_uevent_env
kobject: 'loop1' (00000000fd2f09a6): fill_kobj_path: path
= '/devices/virtual/block/loop1'
kobject: 'loop3' (000000001d0a0601): kobject_uevent_env
kobject: 'loop3' (000000001d0a0601): fill_kobj_path: path
= '/devices/virtual/block/loop3'
kobject: 'loop2' (000000002d051810): kobject_uevent_env
kobject: 'loop2' (000000002d051810): fill_kobj_path: path
= '/devices/virtual/block/loop2'
kobject: 'loop7' (00000000b52cfd8a): kobject_uevent_env
kobject: 'loop7' (00000000b52cfd8a): fill_kobj_path: path
= '/devices/virtual/block/loop7'
kobject: 'loop0' (000000005501af28): kobject_uevent_env
kobject: 'loop0' (000000005501af28): fill_kobj_path: path
= '/devices/virtual/block/loop0'
kobject: 'loop4' (000000003f6c580a): kobject_uevent_env
kobject: 'loop4' (000000003f6c580a): fill_kobj_path: path
= '/devices/virtual/block/loop4'
kobject: 'loop1' (00000000fd2f09a6): kobject_uevent_env
kobject: 'loop1' (00000000fd2f09a6): fill_kobj_path: path
= '/devices/virtual/block/loop1'
kobject: 'loop6' (00000000790743e9): kobject_uevent_env
kobject: 'loop6' (00000000790743e9): fill_kobj_path: path
= '/devices/virtual/block/loop6'
kobject: 'loop0' (000000005501af28): kobject_uevent_env
kobject: 'loop0' (000000005501af28): fill_kobj_path: path
= '/devices/virtual/block/loop0'
kobject: 'loop3' (000000001d0a0601): kobject_uevent_env
kobject: 'loop3' (000000001d0a0601): fill_kobj_path: path
= '/devices/virtual/block/loop3'
kobject: 'loop2' (000000002d051810): kobject_uevent_env
kobject: 'loop2' (000000002d051810): fill_kobj_path: path
= '/devices/virtual/block/loop2'
kobject: 'loop7' (00000000b52cfd8a): kobject_uevent_env
kobject: 'loop7' (00000000b52cfd8a): fill_kobj_path: path
= '/devices/virtual/block/loop7'
kobject: 'loop5' (0000000044e01f3d): kobject_uevent_env
kobject: 'loop5' (0000000044e01f3d): fill_kobj_path: path
= '/devices/virtual/block/loop5'
kobject: 'loop4' (000000003f6c580a): kobject_uevent_env
kobject: 'loop4' (000000003f6c580a): fill_kobj_path: path
= '/devices/virtual/block/loop4'
kobject: 'loop1' (00000000fd2f09a6): kobject_uevent_env
kobject: 'loop1' (00000000fd2f09a6): fill_kobj_path: path
= '/devices/virtual/block/loop1'
kobject: 'loop6' (00000000790743e9): kobject_uevent_env
kobject: 'loop6' (00000000790743e9): fill_kobj_path: path
= '/devices/virtual/block/loop6'
kobject: 'loop0' (000000005501af28): kobject_uevent_env
kobject: 'loop0' (000000005501af28): fill_kobj_path: path
= '/devices/virtual/block/loop0'
kobject: 'loop7' (00000000b52cfd8a): kobject_uevent_env
kobject: 'loop7' (00000000b52cfd8a): fill_kobj_path: path
= '/devices/virtual/block/loop7'
kobject: 'loop2' (000000002d051810): kobject_uevent_env
kobject: 'loop2' (000000002d051810): fill_kobj_path: path
= '/devices/virtual/block/loop2'
kobject: 'loop3' (000000001d0a0601): kobject_uevent_env
kobject: 'loop3' (000000001d0a0601): fill_kobj_path: path
= '/devices/virtual/block/loop3'
kobject: 'loop5' (0000000044e01f3d): kobject_uevent_env
kobject: 'loop5' (0000000044e01f3d): fill_kobj_path: path
= '/devices/virtual/block/loop5'
kobject: 'loop4' (000000003f6c580a): kobject_uevent_env
kobject: 'loop4' (000000003f6c580a): fill_kobj_path: path
= '/devices/virtual/block/loop4'
kobject: 'loop1' (00000000fd2f09a6): kobject_uevent_env
kobject: 'loop1' (00000000fd2f09a6): fill_kobj_path: path
= '/devices/virtual/block/loop1'
kobject: 'loop6' (00000000790743e9): kobject_uevent_env
kobject: 'loop6' (00000000790743e9): fill_kobj_path: path
= '/devices/virtual/block/loop6'
kobject: 'loop3' (000000001d0a0601): kobject_uevent_env
kobject: 'loop3' (000000001d0a0601): fill_kobj_path: path
= '/devices/virtual/block/loop3'
kobject: 'loop5' (0000000044e01f3d): kobject_uevent_env
kobject: 'loop5' (0000000044e01f3d): fill_kobj_path: path
= '/devices/virtual/block/loop5'
kobject: 'loop0' (000000005501af28): kobject_uevent_env
kobject: 'loop0' (000000005501af28): fill_kobj_path: path
= '/devices/virtual/block/loop0'
kobject: 'loop2' (000000002d051810): kobject_uevent_env
kobject: 'loop2' (000000002d051810): fill_kobj_path: path
= '/devices/virtual/block/loop2'
kobject: 'loop4' (000000003f6c580a): kobject_uevent_env
kobject: 'loop4' (000000003f6c580a): fill_kobj_path: path
= '/devices/virtual/block/loop4'
kobject: 'loop7' (00000000b52cfd8a): kobject_uevent_env
kobject: 'loop7' (00000000b52cfd8a): fill_kobj_path: path
= '/devices/virtual/block/loop7'
kobject: 'loop1' (00000000fd2f09a6): kobject_uevent_env
kobject: 'loop1' (00000000fd2f09a6): fill_kobj_path: path
= '/devices/virtual/block/loop1'
kobject: 'loop6' (00000000790743e9): kobject_uevent_env
kobject: 'loop6' (00000000790743e9): fill_kobj_path: path
= '/devices/virtual/block/loop6'
kobject: 'loop3' (000000001d0a0601): kobject_uevent_env
kobject: 'loop3' (000000001d0a0601): fill_kobj_path: path
= '/devices/virtual/block/loop3'
kobject: 'loop5' (0000000044e01f3d): kobject_uevent_env
kobject: 'loop5' (0000000044e01f3d): fill_kobj_path: path
= '/devices/virtual/block/loop5'
kobject: 'loop4' (000000003f6c580a): kobject_uevent_env
kobject: 'loop4' (000000003f6c580a): fill_kobj_path: path
= '/devices/virtual/block/loop4'
kobject: 'loop7' (00000000b52cfd8a): kobject_uevent_env
kobject: 'loop7' (00000000b52cfd8a): fill_kobj_path: path
= '/devices/virtual/block/loop7'
kobject: 'loop2' (000000002d051810): kobject_uevent_env
kobject: 'loop2' (000000002d051810): fill_kobj_path: path
= '/devices/virtual/block/loop2'
kobject: 'loop0' (000000005501af28): kobject_uevent_env
kobject: 'loop0' (000000005501af28): fill_kobj_path: path
= '/devices/virtual/block/loop0'
kobject: 'loop1' (00000000fd2f09a6): kobject_uevent_env
kobject: 'loop1' (00000000fd2f09a6): fill_kobj_path: path
= '/devices/virtual/block/loop1'
kobject: 'loop6' (00000000790743e9): kobject_uevent_env
kobject: 'loop6' (00000000790743e9): fill_kobj_path: path
= '/devices/virtual/block/loop6'
kobject: 'loop2' (000000002d051810): kobject_uevent_env
kobject: 'loop2' (000000002d051810): fill_kobj_path: path
= '/devices/virtual/block/loop2'
kobject: 'loop7' (00000000b52cfd8a): kobject_uevent_env
kobject: 'loop7' (00000000b52cfd8a): fill_kobj_path: path
= '/devices/virtual/block/loop7'
kobject: 'loop3' (000000001d0a0601): kobject_uevent_env
kobject: 'loop3' (000000001d0a0601): fill_kobj_path: path
= '/devices/virtual/block/loop3'
kobject: 'loop5' (0000000044e01f3d): kobject_uevent_env
kobject: 'loop5' (0000000044e01f3d): fill_kobj_path: path
= '/devices/virtual/block/loop5'
kobject: 'loop4' (000000003f6c580a): kobject_uevent_env
kobject: 'loop4' (000000003f6c580a): fill_kobj_path: path
= '/devices/virtual/block/loop4'
kobject: 'loop0' (000000005501af28): kobject_uevent_env
kobject: 'loop0' (000000005501af28): fill_kobj_path: path
= '/devices/virtual/block/loop0'
kobject: 'loop1' (00000000fd2f09a6): kobject_uevent_env
kobject: 'loop1' (00000000fd2f09a6): fill_kobj_path: path
= '/devices/virtual/block/loop1'
kobject: 'loop6' (00000000790743e9): kobject_uevent_env
kobject: 'loop6' (00000000790743e9): fill_kobj_path: path
= '/devices/virtual/block/loop6'
kobject: 'loop7' (00000000b52cfd8a): kobject_uevent_env
kobject: 'loop7' (00000000b52cfd8a): fill_kobj_path: path
= '/devices/virtual/block/loop7'
kobject: 'loop2' (000000002d051810): kobject_uevent_env
kobject: 'loop2' (000000002d051810): fill_kobj_path: path
= '/devices/virtual/block/loop2'
kobject: 'loop3' (000000001d0a0601): kobject_uevent_env
kobject: 'loop3' (000000001d0a0601): fill_kobj_path: path
= '/devices/virtual/block/loop3'
kobject: 'loop5' (0000000044e01f3d): kobject_uevent_env
kobject: 'loop5' (0000000044e01f3d): fill_kobj_path: path
= '/devices/virtual/block/loop5'
kobject: 'loop4' (000000003f6c580a): kobject_uevent_env
kobject: 'loop4' (000000003f6c580a): fill_kobj_path: path
= '/devices/virtual/block/loop4'
kobject: 'loop0' (000000005501af28): kobject_uevent_env
kobject: 'loop0' (000000005501af28): fill_kobj_path: path
= '/devices/virtual/block/loop0'
kobject: 'loop1' (00000000fd2f09a6): kobject_uevent_env
kobject: 'loop1' (00000000fd2f09a6): fill_kobj_path: path
= '/devices/virtual/block/loop1'
kobject: 'loop6' (00000000790743e9): kobject_uevent_env
kobject: 'loop6' (00000000790743e9): fill_kobj_path: path
= '/devices/virtual/block/loop6'
kobject: 'loop7' (00000000b52cfd8a): kobject_uevent_env
kobject: 'loop7' (00000000b52cfd8a): fill_kobj_path: path
= '/devices/virtual/block/loop7'
kobject: 'loop2' (000000002d051810): kobject_uevent_env
kobject: 'loop2' (000000002d051810): fill_kobj_path: path
= '/devices/virtual/block/loop2'
kobject: 'loop5' (0000000044e01f3d): kobject_uevent_env
kobject: 'loop5' (0000000044e01f3d): fill_kobj_path: path
= '/devices/virtual/block/loop5'
kobject: 'loop4' (000000003f6c580a): kobject_uevent_env
kobject: 'loop4' (000000003f6c580a): fill_kobj_path: path
= '/devices/virtual/block/loop4'
kobject: 'loop3' (000000001d0a0601): kobject_uevent_env
kobject: 'loop3' (000000001d0a0601): fill_kobj_path: path
= '/devices/virtual/block/loop3'
kobject: 'loop0' (000000005501af28): kobject_uevent_env
kobject: 'loop0' (000000005501af28): fill_kobj_path: path
= '/devices/virtual/block/loop0'


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#bug-status-tracking for how to communicate with
syzbot.
syzbot can test patches for this bug, for details see:
https://goo.gl/tpsmEJ#testing-patches

Christoph Hellwig

unread,
Sep 10, 2018, 12:53:19 PM9/10/18
to syzbot, bc...@kvack.org, linu...@kvack.org, linux-...@vger.kernel.org, linux-...@vger.kernel.org, syzkall...@googlegroups.com, vi...@zeniv.linux.org.uk, Andrea Arcangeli, ak...@linux-foundation.org
On Mon, Sep 10, 2018 at 12:41:05AM -0700, syzbot wrote:
> =====================================================
> WARNING: SOFTIRQ-safe -> SOFTIRQ-unsafe lock order detected
> 4.19.0-rc2+ #229 Not tainted
> -----------------------------------------------------
> syz-executor2/9399 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire:
> 00000000126506e0 (&ctx->fd_wqh){+.+.}, at: spin_lock
> include/linux/spinlock.h:329 [inline]
> 00000000126506e0 (&ctx->fd_wqh){+.+.}, at: aio_poll+0x760/0x1420
> fs/aio.c:1747
>
> and this task is already holding:
> 000000002bed6bf6 (&(&ctx->ctx_lock)->rlock){..-.}, at: spin_lock_irq
> include/linux/spinlock.h:354 [inline]
> 000000002bed6bf6 (&(&ctx->ctx_lock)->rlock){..-.}, at: aio_poll+0x738/0x1420
> fs/aio.c:1746
> which would create a new lock dependency:
> (&(&ctx->ctx_lock)->rlock){..-.} -> (&ctx->fd_wqh){+.+.}

ctx->fd_wqh seems to only exist in userfaultfd, which indeed seems
to do strange open coded waitqueue locking, and seems to fail to disable
irqs. Something like this should fix it:

diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
index bfa0ec69f924..356d2b8568c1 100644
--- a/fs/userfaultfd.c
+++ b/fs/userfaultfd.c
@@ -1026,7 +1026,7 @@ static ssize_t userfaultfd_ctx_read(struct userfaultfd_ctx *ctx, int no_wait,
struct userfaultfd_ctx *fork_nctx = NULL;

/* always take the fd_wqh lock before the fault_pending_wqh lock */
- spin_lock(&ctx->fd_wqh.lock);
+ spin_lock_irq(&ctx->fd_wqh.lock);
__add_wait_queue(&ctx->fd_wqh, &wait);
for (;;) {
set_current_state(TASK_INTERRUPTIBLE);
@@ -1112,13 +1112,13 @@ static ssize_t userfaultfd_ctx_read(struct userfaultfd_ctx *ctx, int no_wait,
ret = -EAGAIN;
break;
}
- spin_unlock(&ctx->fd_wqh.lock);
+ spin_unlock_irq(&ctx->fd_wqh.lock);
schedule();
- spin_lock(&ctx->fd_wqh.lock);
+ spin_lock_irq(&ctx->fd_wqh.lock);
}
__remove_wait_queue(&ctx->fd_wqh, &wait);
__set_current_state(TASK_RUNNING);
- spin_unlock(&ctx->fd_wqh.lock);
+ spin_unlock_irq(&ctx->fd_wqh.lock);

if (!ret && msg->event == UFFD_EVENT_FORK) {
ret = resolve_userfault_fork(ctx, fork_nctx, msg);

Miklos Szeredi

unread,
Sep 10, 2018, 2:14:22 PM9/10/18
to Christoph Hellwig, syzbot, bcrl, linux-aio, linux-...@vger.kernel.org, linux-...@vger.kernel.org, syzkaller-bugs, Al Viro, Andrea Arcangeli, Andrew Morton
On Mon, Sep 10, 2018 at 6:53 PM, Christoph Hellwig <h...@infradead.org> wrote:
> On Mon, Sep 10, 2018 at 12:41:05AM -0700, syzbot wrote:
>> =====================================================
>> WARNING: SOFTIRQ-safe -> SOFTIRQ-unsafe lock order detected
>> 4.19.0-rc2+ #229 Not tainted
>> -----------------------------------------------------
>> syz-executor2/9399 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire:
>> 00000000126506e0 (&ctx->fd_wqh){+.+.}, at: spin_lock
>> include/linux/spinlock.h:329 [inline]
>> 00000000126506e0 (&ctx->fd_wqh){+.+.}, at: aio_poll+0x760/0x1420
>> fs/aio.c:1747
>>
>> and this task is already holding:
>> 000000002bed6bf6 (&(&ctx->ctx_lock)->rlock){..-.}, at: spin_lock_irq
>> include/linux/spinlock.h:354 [inline]
>> 000000002bed6bf6 (&(&ctx->ctx_lock)->rlock){..-.}, at: aio_poll+0x738/0x1420
>> fs/aio.c:1746
>> which would create a new lock dependency:
>> (&(&ctx->ctx_lock)->rlock){..-.} -> (&ctx->fd_wqh){+.+.}
>
> ctx->fd_wqh seems to only exist in userfaultfd, which indeed seems
> to do strange open coded waitqueue locking, and seems to fail to disable
> irqs. Something like this should fix it:

Why do pollable waitqueues need to disable interrupts generally?

I don't see anything fundamental in the poll interface to force this
requirement on users of that interface.

Thanks,
Miklos

Christoph Hellwig

unread,
Sep 11, 2018, 2:33:39 AM9/11/18
to Miklos Szeredi, Christoph Hellwig, syzbot, bcrl, linux-aio, linux-...@vger.kernel.org, linux-...@vger.kernel.org, syzkaller-bugs, Al Viro, Andrea Arcangeli, Andrew Morton
On Mon, Sep 10, 2018 at 08:14:20PM +0200, Miklos Szeredi wrote:
> Why do pollable waitqueues need to disable interrupts generally?

Any waitqueue needs to disable interrupts for consistency. We
always use spin_lock_irqsave in __wake_up_common_lock() for example.

Miklos Szeredi

unread,
Sep 11, 2018, 3:20:42 AM9/11/18
to Christoph Hellwig, syzbot, bcrl, linux-aio, linux-...@vger.kernel.org, linux-...@vger.kernel.org, syzkaller-bugs, Al Viro, Andrea Arcangeli, Andrew Morton
There are the _locked (non _irq) variants that do not.

And poll/select/etc don't impose non-interuptibility on wakeups
either. So it looks like it's just aio that has weird spin lock
dependencies that forces this requirement on a waitq used in ->poll().

Thanks,
Miklos

Andrea Arcangeli

unread,
Oct 17, 2018, 7:55:35 PM10/17/18
to Christoph Hellwig, syzbot, bc...@kvack.org, linu...@kvack.org, linux-...@vger.kernel.org, linux-...@vger.kernel.org, syzkall...@googlegroups.com, vi...@zeniv.linux.org.uk, ak...@linux-foundation.org
Reviewed-by: Andrea Arcangeli <aarc...@redhat.com>

This is lock inversion with userfaultfd_poll that takes the fq_wqh
after the irqsafe aio lock. And the aio lock can be taken from softirq
(so potentially for interrupts) leading to a potential lock inversion
deadlock.

I suggest to add a comment about the above in the code before the
first spin_lock_irq to explain why it needs to be _irq or it's not
obvious.

c430d1e848ff1240d126e79780f3c26208b8aed9 was just a false positive
instead.

I didn't comment on c430d1e848ff1240d126e79780f3c26208b8aed9 because I
was too busy with the speculative execution issues at the time and it
was just fine to drop the microoptimization, but while at it can we
look in how to add a spin_acquire or find a way to teach lockdep in
another way, so it's happy even if we restore the microoptimization?

If we do that, in addition we should also initialize the
ctx->fault_wqh spinlock to locked in the same patch (a spin_lock
during uffd ctx creation will do) to be sure nobody takes it as
further robustness feature against future modification (it gets more
self documenting too that it's not supposed to be taken and the
fault_pending_wq.lock has to be taken instead).

Thanks,
Andrea

syzbot

unread,
Oct 27, 2018, 2:16:04 AM10/27/18
to aarc...@redhat.com, ak...@linux-foundation.org, bc...@kvack.org, h...@infradead.org, linu...@kvack.org, linux-...@vger.kernel.org, linux-...@vger.kernel.org, mik...@szeredi.hu, syzkall...@googlegroups.com, vi...@zeniv.linux.org.uk
syzbot has found a reproducer for the following crash on:

HEAD commit: 18d0eae30e6a Merge tag 'char-misc-4.20-rc1' of git://git.k..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=14728be5400000
kernel config: https://syzkaller.appspot.com/x/.config?x=342f43de913c81b9
dashboard link: https://syzkaller.appspot.com/bug?extid=5b1df0420c523b45a953
compiler: gcc (GCC) 8.0.1 20180413 (experimental)
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=161d6999400000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=110f4cf5400000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+5b1df0...@syzkaller.appspotmail.com


=====================================================
WARNING: SOFTIRQ-safe -> SOFTIRQ-unsafe lock order detected
4.19.0+ #84 Not tainted
-----------------------------------------------------
syz-executor781/7254 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire:
00000000e70e96f7 (&ctx->fd_wqh){+.+.}, at: spin_lock
include/linux/spinlock.h:329 [inline]
00000000e70e96f7 (&ctx->fd_wqh){+.+.}, at: aio_poll+0x760/0x1420
fs/aio.c:1747

and this task is already holding:
000000009957d7d7 (&(&ctx->ctx_lock)->rlock){..-.}, at: spin_lock_irq
include/linux/spinlock.h:354 [inline]
000000009957d7d7 (&(&ctx->ctx_lock)->rlock){..-.}, at:
aio_poll+0x738/0x1420 fs/aio.c:1746
which would create a new lock dependency:
(&(&ctx->ctx_lock)->rlock){..-.} -> (&ctx->fd_wqh){+.+.}

but this new dependency connects a SOFTIRQ-irq-safe lock:
(&(&ctx->ctx_lock)->rlock){..-.}

... which became SOFTIRQ-irq-safe at:
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3844
__raw_spin_lock_irq include/linux/spinlock_api_smp.h:128 [inline]
_raw_spin_lock_irq+0x61/0x80 kernel/locking/spinlock.c:160
spin_lock_irq include/linux/spinlock.h:354 [inline]
free_ioctx_users+0xbc/0x710 fs/aio.c:603
percpu_ref_put_many include/linux/percpu-refcount.h:285 [inline]
percpu_ref_put include/linux/percpu-refcount.h:301 [inline]
percpu_ref_call_confirm_rcu lib/percpu-refcount.c:123 [inline]
percpu_ref_switch_to_atomic_rcu+0x563/0x730 lib/percpu-refcount.c:158
__rcu_reclaim kernel/rcu/rcu.h:240 [inline]
rcu_do_batch kernel/rcu/tree.c:2437 [inline]
invoke_rcu_callbacks kernel/rcu/tree.c:2716 [inline]
rcu_process_callbacks+0x100a/0x1ac0 kernel/rcu/tree.c:2697
__do_softirq+0x308/0xb7e kernel/softirq.c:292
invoke_softirq kernel/softirq.c:373 [inline]
irq_exit+0x17f/0x1c0 kernel/softirq.c:413
exiting_irq arch/x86/include/asm/apic.h:536 [inline]
smp_apic_timer_interrupt+0x1cb/0x760 arch/x86/kernel/apic/apic.c:1061
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:801
native_safe_halt+0x6/0x10 arch/x86/include/asm/irqflags.h:57
arch_safe_halt arch/x86/include/asm/paravirt.h:151 [inline]
default_idle+0xbf/0x490 arch/x86/kernel/process.c:498
arch_cpu_idle+0x10/0x20 arch/x86/kernel/process.c:489
default_idle_call+0x6d/0x90 kernel/sched/idle.c:93
cpuidle_idle_call kernel/sched/idle.c:153 [inline]
do_idle+0x49b/0x5c0 kernel/sched/idle.c:262
cpu_startup_entry+0x18/0x20 kernel/sched/idle.c:353
start_secondary+0x487/0x5f0 arch/x86/kernel/smpboot.c:271
secondary_startup_64+0xa4/0xb0 arch/x86/kernel/head_64.S:243

to a SOFTIRQ-irq-unsafe lock:
(&ctx->fd_wqh){+.+.}

... which became SOFTIRQ-irq-unsafe at:
...
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3844
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2d/0x40 kernel/locking/spinlock.c:144
spin_lock include/linux/spinlock.h:329 [inline]
userfaultfd_ctx_read+0x2e4/0x2180 fs/userfaultfd.c:1029
userfaultfd_read+0x1e2/0x2c0 fs/userfaultfd.c:1191
__vfs_read+0x117/0x9b0 fs/read_write.c:416
vfs_read+0x17f/0x3c0 fs/read_write.c:452
ksys_read+0x101/0x260 fs/read_write.c:578
__do_sys_read fs/read_write.c:588 [inline]
__se_sys_read fs/read_write.c:586 [inline]
__x64_sys_read+0x73/0xb0 fs/read_write.c:586
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe

other info that might help us debug this:

Possible interrupt unsafe locking scenario:

CPU0 CPU1
---- ----
lock(&ctx->fd_wqh);
local_irq_disable();
lock(&(&ctx->ctx_lock)->rlock);
lock(&ctx->fd_wqh);
<Interrupt>
lock(&(&ctx->ctx_lock)->rlock);

*** DEADLOCK ***

1 lock held by syz-executor781/7254:
#0: 000000009957d7d7 (&(&ctx->ctx_lock)->rlock){..-.}, at: spin_lock_irq
include/linux/spinlock.h:354 [inline]
#0: 000000009957d7d7 (&(&ctx->ctx_lock)->rlock){..-.}, at:
aio_poll+0x738/0x1420 fs/aio.c:1746

the dependencies between SOFTIRQ-irq-safe lock and the holding lock:
-> (&(&ctx->ctx_lock)->rlock){..-.} {
IN-SOFTIRQ-W at:
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3844
__raw_spin_lock_irq
include/linux/spinlock_api_smp.h:128 [inline]
_raw_spin_lock_irq+0x61/0x80
kernel/locking/spinlock.c:160
spin_lock_irq include/linux/spinlock.h:354 [inline]
free_ioctx_users+0xbc/0x710 fs/aio.c:603
percpu_ref_put_many include/linux/percpu-refcount.h:285
[inline]
percpu_ref_put include/linux/percpu-refcount.h:301
[inline]
percpu_ref_call_confirm_rcu lib/percpu-refcount.c:123
[inline]
percpu_ref_switch_to_atomic_rcu+0x563/0x730
lib/percpu-refcount.c:158
__rcu_reclaim kernel/rcu/rcu.h:240 [inline]
rcu_do_batch kernel/rcu/tree.c:2437 [inline]
invoke_rcu_callbacks kernel/rcu/tree.c:2716 [inline]
rcu_process_callbacks+0x100a/0x1ac0
kernel/rcu/tree.c:2697
__do_softirq+0x308/0xb7e kernel/softirq.c:292
invoke_softirq kernel/softirq.c:373 [inline]
irq_exit+0x17f/0x1c0 kernel/softirq.c:413
exiting_irq arch/x86/include/asm/apic.h:536 [inline]
smp_apic_timer_interrupt+0x1cb/0x760
arch/x86/kernel/apic/apic.c:1061
apic_timer_interrupt+0xf/0x20
arch/x86/entry/entry_64.S:801
native_safe_halt+0x6/0x10
arch/x86/include/asm/irqflags.h:57
arch_safe_halt arch/x86/include/asm/paravirt.h:151
[inline]
default_idle+0xbf/0x490 arch/x86/kernel/process.c:498
arch_cpu_idle+0x10/0x20 arch/x86/kernel/process.c:489
default_idle_call+0x6d/0x90 kernel/sched/idle.c:93
cpuidle_idle_call kernel/sched/idle.c:153 [inline]
do_idle+0x49b/0x5c0 kernel/sched/idle.c:262
cpu_startup_entry+0x18/0x20 kernel/sched/idle.c:353
start_secondary+0x487/0x5f0
arch/x86/kernel/smpboot.c:271
secondary_startup_64+0xa4/0xb0
arch/x86/kernel/head_64.S:243
INITIAL USE at:
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3844
__raw_spin_lock_irq include/linux/spinlock_api_smp.h:128
[inline]
_raw_spin_lock_irq+0x61/0x80
kernel/locking/spinlock.c:160
spin_lock_irq include/linux/spinlock.h:354 [inline]
free_ioctx_users+0xbc/0x710 fs/aio.c:603
percpu_ref_put_many include/linux/percpu-refcount.h:285
[inline]
percpu_ref_put include/linux/percpu-refcount.h:301
[inline]
percpu_ref_call_confirm_rcu lib/percpu-refcount.c:123
[inline]
percpu_ref_switch_to_atomic_rcu+0x563/0x730
lib/percpu-refcount.c:158
__rcu_reclaim kernel/rcu/rcu.h:240 [inline]
rcu_do_batch kernel/rcu/tree.c:2437 [inline]
invoke_rcu_callbacks kernel/rcu/tree.c:2716 [inline]
rcu_process_callbacks+0x100a/0x1ac0
kernel/rcu/tree.c:2697
__do_softirq+0x308/0xb7e kernel/softirq.c:292
invoke_softirq kernel/softirq.c:373 [inline]
irq_exit+0x17f/0x1c0 kernel/softirq.c:413
exiting_irq arch/x86/include/asm/apic.h:536 [inline]
smp_apic_timer_interrupt+0x1cb/0x760
arch/x86/kernel/apic/apic.c:1061
apic_timer_interrupt+0xf/0x20
arch/x86/entry/entry_64.S:801
native_safe_halt+0x6/0x10
arch/x86/include/asm/irqflags.h:57
arch_safe_halt arch/x86/include/asm/paravirt.h:151
[inline]
default_idle+0xbf/0x490 arch/x86/kernel/process.c:498
arch_cpu_idle+0x10/0x20 arch/x86/kernel/process.c:489
default_idle_call+0x6d/0x90 kernel/sched/idle.c:93
cpuidle_idle_call kernel/sched/idle.c:153 [inline]
do_idle+0x49b/0x5c0 kernel/sched/idle.c:262
cpu_startup_entry+0x18/0x20 kernel/sched/idle.c:353
start_secondary+0x487/0x5f0 arch/x86/kernel/smpboot.c:271
secondary_startup_64+0xa4/0xb0
arch/x86/kernel/head_64.S:243
}
... key at: [<ffffffff8aed9b20>] __key.50623+0x0/0x40
... acquired at:
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3844
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2d/0x40 kernel/locking/spinlock.c:144
spin_lock include/linux/spinlock.h:329 [inline]
aio_poll+0x760/0x1420 fs/aio.c:1747
io_submit_one+0xa49/0xf80 fs/aio.c:1850
__do_sys_io_submit fs/aio.c:1916 [inline]
__se_sys_io_submit fs/aio.c:1887 [inline]
__x64_sys_io_submit+0x1b7/0x580 fs/aio.c:1887
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe


the dependencies between the lock to be acquired
and SOFTIRQ-irq-unsafe lock:
-> (&ctx->fd_wqh){+.+.} {
HARDIRQ-ON-W at:
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3844
__raw_spin_lock include/linux/spinlock_api_smp.h:142
[inline]
_raw_spin_lock+0x2d/0x40 kernel/locking/spinlock.c:144
spin_lock include/linux/spinlock.h:329 [inline]
userfaultfd_ctx_read+0x2e4/0x2180 fs/userfaultfd.c:1029
userfaultfd_read+0x1e2/0x2c0 fs/userfaultfd.c:1191
__vfs_read+0x117/0x9b0 fs/read_write.c:416
vfs_read+0x17f/0x3c0 fs/read_write.c:452
ksys_read+0x101/0x260 fs/read_write.c:578
__do_sys_read fs/read_write.c:588 [inline]
__se_sys_read fs/read_write.c:586 [inline]
__x64_sys_read+0x73/0xb0 fs/read_write.c:586
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
SOFTIRQ-ON-W at:
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3844
__raw_spin_lock include/linux/spinlock_api_smp.h:142
[inline]
_raw_spin_lock+0x2d/0x40 kernel/locking/spinlock.c:144
spin_lock include/linux/spinlock.h:329 [inline]
userfaultfd_ctx_read+0x2e4/0x2180 fs/userfaultfd.c:1029
userfaultfd_read+0x1e2/0x2c0 fs/userfaultfd.c:1191
__vfs_read+0x117/0x9b0 fs/read_write.c:416
vfs_read+0x17f/0x3c0 fs/read_write.c:452
ksys_read+0x101/0x260 fs/read_write.c:578
__do_sys_read fs/read_write.c:588 [inline]
__se_sys_read fs/read_write.c:586 [inline]
__x64_sys_read+0x73/0xb0 fs/read_write.c:586
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
INITIAL USE at:
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3844
__raw_spin_lock include/linux/spinlock_api_smp.h:142
[inline]
_raw_spin_lock+0x2d/0x40 kernel/locking/spinlock.c:144
spin_lock include/linux/spinlock.h:329 [inline]
userfaultfd_ctx_read+0x2e4/0x2180 fs/userfaultfd.c:1029
userfaultfd_read+0x1e2/0x2c0 fs/userfaultfd.c:1191
__vfs_read+0x117/0x9b0 fs/read_write.c:416
vfs_read+0x17f/0x3c0 fs/read_write.c:452
ksys_read+0x101/0x260 fs/read_write.c:578
__do_sys_read fs/read_write.c:588 [inline]
__se_sys_read fs/read_write.c:586 [inline]
__x64_sys_read+0x73/0xb0 fs/read_write.c:586
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
}
... key at: [<ffffffff8aed98a0>] __key.44253+0x0/0x40
... acquired at:
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3844
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2d/0x40 kernel/locking/spinlock.c:144
spin_lock include/linux/spinlock.h:329 [inline]
aio_poll+0x760/0x1420 fs/aio.c:1747
io_submit_one+0xa49/0xf80 fs/aio.c:1850
__do_sys_io_submit fs/aio.c:1916 [inline]
__se_sys_io_submit fs/aio.c:1887 [inline]
__x64_sys_io_submit+0x1b7/0x580 fs/aio.c:1887
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe


stack backtrace:
CPU: 0 PID: 7254 Comm: syz-executor781 Not tainted 4.19.0+ #84
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x244/0x39d lib/dump_stack.c:113
print_bad_irq_dependency kernel/locking/lockdep.c:1570 [inline]
check_usage.cold.58+0x6d5/0xad1 kernel/locking/lockdep.c:1602
check_irq_usage kernel/locking/lockdep.c:1658 [inline]
check_prev_add_irq kernel/locking/lockdep_states.h:8 [inline]
check_prev_add kernel/locking/lockdep.c:1868 [inline]
check_prevs_add kernel/locking/lockdep.c:1976 [inline]
validate_chain kernel/locking/lockdep.c:2347 [inline]
__lock_acquire+0x238a/0x4c20 kernel/locking/lockdep.c:3341
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3844
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2d/0x40 kernel/locking/spinlock.c:144
spin_lock include/linux/spinlock.h:329 [inline]
aio_poll+0x760/0x1420 fs/aio.c:1747
io_submit_one+0xa49/0xf80 fs/aio.c:1850
__do_sys_io_submit fs/aio.c:1916 [inline]
__se_sys_io_submit fs/aio.c:1887 [inline]
__x64_sys_io_submit+0x1b7/0x580 fs/aio.c:1887
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x447dc9
Code: e8 9c ba 02 00 48 83 c4 18 c3 0f 1f 80 00 00 00 00 48 89 f8 48 89 f7
48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff
ff 0f 83 5b 07 fc ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007fc840e69da8 EFLAGS: 00000293 ORIG_RAX: 00000000000000d1
RAX: ffffffffffffffda RBX: 00000000006e39e8 RCX: 0000000000447dc9
RDX: 0000000020000b00 RSI: 0000000000000001 RDI: 00007fc840e39000
RBP: 00000000006e39e0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000293 R12: 00000000006e39ec
R13: 702f74656e2f666c R14: 65732f636f72702f R15: 0000000000000000

Eric Biggers

unread,
Jun 27, 2019, 2:31:04 AM6/27/19
to syzbot, syzkall...@googlegroups.com
On Mon, Sep 10, 2018 at 12:41:05AM -0700, syzbot wrote:
#syz fix: userfaultfd: disable irqs when taking the waitqueue lock
Reply all
Reply to author
Forward
0 new messages