WARNING in io_disable_sqo_submit

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: 7c53f6b6 Linux 5.11-rc3
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=12a76f70d00000
kernel config: https://syzkaller.appspot.com/x/.config?x=c60c9ff9cc916cbc
dashboard link: https://syzkaller.appspot.com/bug?extid=2f5d1785dc624932da78
compiler: gcc (GCC) 10.1.0-syz 20200507

Unfortunately, I don't have any reproducer for this issue yet.

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+2f5d17...@syzkaller.appspotmail.com

------------[ cut here ]------------
WARNING: CPU: 1 PID: 9094 at fs/io_uring.c:8884 io_disable_sqo_submit+0x106/0x130 fs/io_uring.c:8884
Modules linked in:
CPU: 1 PID: 9094 Comm: syz-executor.5 Not tainted 5.11.0-rc3-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
RIP: 0010:io_disable_sqo_submit+0x106/0x130 fs/io_uring.c:8884
Code: b6 14 02 48 89 f8 83 e0 07 83 c0 03 38 d0 7c 04 84 d2 75 1d 83 8b 14 01 00 00 01 48 89 ef 5b 5d e9 ef bc 23 07 e8 5a e5 9a ff <0f> 0b e9 35 ff ff ff e8 3e a1 dd ff eb dc e8 67 a1 dd ff e9 65 ff
RSP: 0018:ffffc9000188fea0 EFLAGS: 00010212
RAX: 0000000000000044 RBX: ffff888079dbe000 RCX: ffffc90013b54000
RDX: 0000000000040000 RSI: ffffffff81d7e466 RDI: ffff888079dbe0d0
RBP: ffff8880201c0c80 R08: 0000000000000000 R09: 00000000278d0001
R10: ffffffff81d7e705 R11: 0000000000000001 R12: ffff888079dbe000
R13: ffff8880278d0001 R14: ffff888079dbe040 R15: ffff888079dbe0d0
FS: 00007fe461a71700(0000) GS:ffff8880b9f00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000080 CR3: 0000000011fd1000 CR4: 0000000000350ee0
Call Trace:
io_uring_flush+0x28b/0x3a0 fs/io_uring.c:9099
filp_close+0xb4/0x170 fs/open.c:1280
close_fd+0x5c/0x80 fs/file.c:626
__do_sys_close fs/open.c:1299 [inline]
__se_sys_close fs/open.c:1297 [inline]
__x64_sys_close+0x2f/0xa0 fs/open.c:1297
do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46
entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x45e219
Code: 0d b4 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 db b3 fb ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007fe461a70c68 EFLAGS: 00000246 ORIG_RAX: 0000000000000003
RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 000000000045e219
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000007
RBP: 000000000119bfb0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 000000000119bf8c
R13: 00007ffc626b58ff R14: 00007fe461a719c0 R15: 000000000119bf8c


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

[Pavel Begunkov]

On 15/01/2021 23:08, syzbot wrote:
> Hello,
>
> syzbot found the following issue on:
>
> HEAD commit: 7c53f6b6 Linux 5.11-rc3
> git tree: upstream
> console output: https://syzkaller.appspot.com/x/log.txt?x=12a76f70d00000
> kernel config: https://syzkaller.appspot.com/x/.config?x=c60c9ff9cc916cbc
> dashboard link: https://syzkaller.appspot.com/bug?extid=2f5d1785dc624932da78
> compiler: gcc (GCC) 10.1.0-syz 20200507
>
> Unfortunately, I don't have any reproducer for this issue yet.
>
> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> Reported-by: syzbot+2f5d17...@syzkaller.appspotmail.com
>
> ------------[ cut here ]------------
> WARNING: CPU: 1 PID: 9094 at fs/io_uring.c:8884 io_disable_sqo_submit+0x106/0x130 fs/io_uring.c:8884

This one is a false positive warn_once, I'll fix it up

Pavel Begunkov

[Hillf Danton]

On Fri, 15 Jan 2021 23:18:16 +0000 Pavel Begunkov wrote:
> On 15/01/2021 23:08, syzbot wrote:
> > Hello,
> >
> > syzbot found the following issue on:
> >
> > HEAD commit: 7c53f6b6 Linux 5.11-rc3
> > git tree: upstream
> > console output: https://syzkaller.appspot.com/x/log.txt?x=12a76f70d00000
> > kernel config: https://syzkaller.appspot.com/x/.config?x=c60c9ff9cc916cbc
> > dashboard link: https://syzkaller.appspot.com/bug?extid=2f5d1785dc624932da78
> > compiler: gcc (GCC) 10.1.0-syz 20200507
> >
> > Unfortunately, I don't have any reproducer for this issue yet.
> >
> > IMPORTANT: if you fix the issue, please add the following tag to the commit:
> > Reported-by: syzbot+2f5d17...@syzkaller.appspotmail.com
> >
> > ------------[ cut here ]------------
> > WARNING: CPU: 1 PID: 9094 at fs/io_uring.c:8884 io_disable_sqo_submit+0x106/0x130 fs/io_uring.c:8884
>
> This one is a false positive warn_once, I'll fix it up

Thanks for sharing your plan to fix it; that would estimatedly save me
a couple of hours at least toe curling this weekend.

Feel free to Cc Hillf if it is not too heavy a difficult to add him on
your Cc list because he would like to read the diffs you are going to
post with nothing set up in the spam filter.

Hillf

[syzbot]

syzbot has found a reproducer for the following issue on:

HEAD commit: a1339d63 Merge tag 'powerpc-5.11-4' of git://git.kernel.or..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=17532a58d00000

kernel config: https://syzkaller.appspot.com/x/.config?x=c60c9ff9cc916cbc
dashboard link: https://syzkaller.appspot.com/bug?extid=2f5d1785dc624932da78
compiler: gcc (GCC) 10.1.0-syz 20200507

syz repro: https://syzkaller.appspot.com/x/repro.syz?x=10f207c7500000

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+2f5d17...@syzkaller.appspotmail.com

------------[ cut here ]------------

WARNING: CPU: 0 PID: 9113 at fs/io_uring.c:8917 io_disable_sqo_submit+0x13d/0x180 fs/io_uring.c:8917
Modules linked in:
CPU: 1 PID: 9113 Comm: syz-executor.0 Not tainted 5.11.0-rc3-syzkaller #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011

RIP: 0010:io_disable_sqo_submit+0x13d/0x180 fs/io_uring.c:8917
Code: e0 07 83 c0 03 38 d0 7c 04 84 d2 75 2e 83 8b 14 01 00 00 01 4c 89 e7 e8 31 0a 24 07 5b 5d 41 5c e9 98 e1 9a ff e8 93 e1 9a ff <0f> 0b e9 00 ff ff ff e8 a7 a1 dd ff e9 37 ff ff ff e8 6d a1 dd ff
RSP: 0018:ffffc9000311fe98 EFLAGS: 00010293
RAX: 0000000000000000 RBX: ffff888024b43000 RCX: 0000000000000000
RDX: ffff888147071bc0 RSI: ffffffff81d7e82d RDI: ffff888024b430d0
RBP: ffff8880115d1900 R08: 0000000000000000 R09: 0000000014555c01
R10: ffffffff81d7eae5 R11: 0000000000000001 R12: ffff888024b43000
R13: ffff888014555c01 R14: ffff888024b43040 R15: ffff888024b430d0
FS: 00007f85abf55700(0000) GS:ffff8880b9e00000(0000) knlGS:0000000000000000

CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 00007fd3adeb5000 CR3: 00000000115d2000 CR4: 0000000000350ef0
Call Trace:
io_uring_flush+0x28b/0x3a0 fs/io_uring.c:9134

filp_close+0xb4/0x170 fs/open.c:1280
close_fd+0x5c/0x80 fs/file.c:626
__do_sys_close fs/open.c:1299 [inline]
__se_sys_close fs/open.c:1297 [inline]
__x64_sys_close+0x2f/0xa0 fs/open.c:1297
do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46
entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x45e219
Code: 0d b4 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 db b3 fb ff c3 66 2e 0f 1f 84 00 00 00 00

RSP: 002b:00007f85abf54c68 EFLAGS: 00000246 ORIG_RAX: 0000000000000003

RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 000000000045e219

RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000004

RBP: 000000000119bfb0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 000000000119bf8c

R13: 00007ffe5217973f R14: 00007f85abf559c0 R15: 000000000119bf8c

[syzbot]

syzbot has bisected this issue to:

commit dcd479e10a0510522a5d88b29b8f79ea3467d501
Author: Johannes Berg <johann...@intel.com>
Date: Fri Oct 9 12:17:11 2020 +0000

mac80211: always wind down STA state

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=13b8b83b500000
start commit: a1339d63 Merge tag 'powerpc-5.11-4' of git://git.kernel.or..
git tree: upstream
final oops: https://syzkaller.appspot.com/x/report.txt?x=1078b83b500000
console output: https://syzkaller.appspot.com/x/log.txt?x=17b8b83b500000

kernel config: https://syzkaller.appspot.com/x/.config?x=c60c9ff9cc916cbc
dashboard link: https://syzkaller.appspot.com/bug?extid=2f5d1785dc624932da78

syz repro: https://syzkaller.appspot.com/x/repro.syz?x=10f207c7500000

Reported-by: syzbot+2f5d17...@syzkaller.appspotmail.com
Fixes: dcd479e10a05 ("mac80211: always wind down STA state")

For information about bisection process see: https://goo.gl/tpsmEJ#bisection

[Pavel Begunkov]

On 18/01/2021 04:27, syzbot wrote:
> syzbot has found a reproducer for the following issue on:
>
> HEAD commit: a1339d63 Merge tag 'powerpc-5.11-4' of git://git.kernel.or..
> git tree: upstream
> console output: https://syzkaller.appspot.com/x/log.txt?x=17532a58d00000
> kernel config: https://syzkaller.appspot.com/x/.config?x=c60c9ff9cc916cbc
> dashboard link: https://syzkaller.appspot.com/bug?extid=2f5d1785dc624932da78
> compiler: gcc (GCC) 10.1.0-syz 20200507
> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=10f207c7500000
>

#syz test: git://git.kernel.dk/linux-block io_uring-5.11

--
Pavel Begunkov

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
INFO: task hung in io_sq_thread_stop

INFO: task kworker/u4:0:8 blocked for more than 143 seconds.
Not tainted 5.11.0-rc1-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:kworker/u4:0 state:D stack:24056 pid: 8 ppid: 2 flags:0x00004000
Workqueue: events_unbound io_ring_exit_work
Call Trace:
context_switch kernel/sched/core.c:4313 [inline]
__schedule+0x90c/0x21a0 kernel/sched/core.c:5064
schedule+0xcf/0x270 kernel/sched/core.c:5143
schedule_timeout+0x1d8/0x250 kernel/time/timer.c:1854
do_wait_for_common kernel/sched/completion.c:85 [inline]
__wait_for_common kernel/sched/completion.c:106 [inline]
wait_for_common kernel/sched/completion.c:117 [inline]
wait_for_completion+0x163/0x260 kernel/sched/completion.c:138
kthread_park+0x122/0x1b0 kernel/kthread.c:557
io_sq_thread_park fs/io_uring.c:7445 [inline]
io_sq_thread_park fs/io_uring.c:7439 [inline]
io_sq_thread_stop+0xfe/0x570 fs/io_uring.c:7463
io_finish_async fs/io_uring.c:7481 [inline]
io_ring_ctx_free fs/io_uring.c:8646 [inline]
io_ring_exit_work+0x62/0x6d0 fs/io_uring.c:8739
process_one_work+0x98d/0x15f0 kernel/workqueue.c:2275
worker_thread+0x64c/0x1120 kernel/workqueue.c:2421
kthread+0x3b1/0x4a0 kernel/kthread.c:292
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:296

Showing all locks held in the system:
3 locks held by kworker/u4:0/8:
#0: ffff888010069138 ((wq_completion)events_unbound){+.+.}-{0:0}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
#0: ffff888010069138 ((wq_completion)events_unbound){+.+.}-{0:0}, at: atomic64_set include/asm-generic/atomic-instrumented.h:856 [inline]
#0: ffff888010069138 ((wq_completion)events_unbound){+.+.}-{0:0}, at: atomic_long_set include/asm-generic/atomic-long.h:41 [inline]
#0: ffff888010069138 ((wq_completion)events_unbound){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:616 [inline]
#0: ffff888010069138 ((wq_completion)events_unbound){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:643 [inline]
#0: ffff888010069138 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x871/0x15f0 kernel/workqueue.c:2246
#1: ffffc90000cd7da8 ((work_completion)(&ctx->exit_work)){+.+.}-{0:0}, at: process_one_work+0x8a5/0x15f0 kernel/workqueue.c:2250
#2: ffff88801bfd4870 (&sqd->lock){+.+.}-{3:3}, at: io_sq_thread_park fs/io_uring.c:7444 [inline]
#2: ffff88801bfd4870 (&sqd->lock){+.+.}-{3:3}, at: io_sq_thread_park fs/io_uring.c:7439 [inline]
#2: ffff88801bfd4870 (&sqd->lock){+.+.}-{3:3}, at: io_sq_thread_stop+0xd6/0x570 fs/io_uring.c:7463
1 lock held by khungtaskd/1647:
#0: ffffffff8b373aa0 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x53/0x260 kernel/locking/lockdep.c:6254
1 lock held by in:imklog/8164:
#0: ffff8880151b8870 (&f->f_pos_lock){+.+.}-{3:3}, at: __fdget_pos+0xe9/0x100 fs/file.c:947
2 locks held by kworker/u4:6/8415:
2 locks held by kworker/0:4/8690:
#0: ffff88801007c538 ((wq_completion)rcu_gp){+.+.}-{0:0}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
#0: ffff88801007c538 ((wq_completion)rcu_gp){+.+.}-{0:0}, at: atomic64_set include/asm-generic/atomic-instrumented.h:856 [inline]
#0: ffff88801007c538 ((wq_completion)rcu_gp){+.+.}-{0:0}, at: atomic_long_set include/asm-generic/atomic-long.h:41 [inline]
#0: ffff88801007c538 ((wq_completion)rcu_gp){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:616 [inline]
#0: ffff88801007c538 ((wq_completion)rcu_gp){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:643 [inline]
#0: ffff88801007c538 ((wq_completion)rcu_gp){+.+.}-{0:0}, at: process_one_work+0x871/0x15f0 kernel/workqueue.c:2246
#1: ffffc9000288fda8 ((work_completion)(&rew.rew_work)){+.+.}-{0:0}, at: process_one_work+0x8a5/0x15f0 kernel/workqueue.c:2250
1 lock held by syz-executor.3/8865:
#0: ffff888146ddcd88 (&xt[i].mutex){+.+.}-{3:3}, at: xt_find_table_lock+0x41/0x540 net/netfilter/x_tables.c:1206
1 lock held by syz-executor.2/8867:
#0: ffff888146ddcd88 (&xt[i].mutex){+.+.}-{3:3}, at: xt_find_table_lock+0x41/0x540 net/netfilter/x_tables.c:1206
2 locks held by syz-executor.5/8869:
#0: ffff888146ddcd88 (&xt[i].mutex){+.+.}-{3:3}, at: xt_find_table_lock+0x41/0x540 net/netfilter/x_tables.c:1206
#1: ffffffff8b37c368 (rcu_state.exp_mutex){+.+.}-{3:3}, at: exp_funnel_lock kernel/rcu/tree_exp.h:290 [inline]
#1: ffffffff8b37c368 (rcu_state.exp_mutex){+.+.}-{3:3}, at: synchronize_rcu_expedited+0x4f2/0x610 kernel/rcu/tree_exp.h:836
1 lock held by syz-executor.4/8870:
#0: ffff888146ddcd88 (&xt[i].mutex){+.+.}-{3:3}, at: xt_find_table_lock+0x41/0x540 net/netfilter/x_tables.c:1206
1 lock held by syz-executor.0/8872:
#0: ffff888146ddcd88 (&xt[i].mutex){+.+.}-{3:3}, at: xt_find_table_lock+0x41/0x540 net/netfilter/x_tables.c:1206
1 lock held by syz-executor.1/8873:
#0: ffff888146ddcd88 (&xt[i].mutex){+.+.}-{3:3}, at: xt_find_table_lock+0x41/0x540 net/netfilter/x_tables.c:1206

=============================================

NMI backtrace for cpu 1
CPU: 1 PID: 1647 Comm: khungtaskd Not tainted 5.11.0-rc1-syzkaller #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011

Call Trace:
__dump_stack lib/dump_stack.c:79 [inline]
dump_stack+0x107/0x163 lib/dump_stack.c:120
nmi_cpu_backtrace.cold+0x44/0xd7 lib/nmi_backtrace.c:105
nmi_trigger_cpumask_backtrace+0x1b3/0x230 lib/nmi_backtrace.c:62
trigger_all_cpu_backtrace include/linux/nmi.h:146 [inline]
check_hung_uninterruptible_tasks kernel/hung_task.c:209 [inline]
watchdog+0xd43/0xfa0 kernel/hung_task.c:294
kthread+0x3b1/0x4a0 kernel/kthread.c:292
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:296
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 8415 Comm: kworker/u4:6 Not tainted 5.11.0-rc1-syzkaller #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011

Workqueue: bat_events batadv_nc_worker
RIP: 0010:__this_cpu_preempt_check+0xd/0x20 lib/smp_processor_id.c:70
Code: 00 00 48 c7 c6 00 d9 9e 89 48 c7 c7 40 d9 9e 89 e9 98 fe ff ff 0f 1f 84 00 00 00 00 00 55 48 89 fd 0f 1f 44 00 00 48 89 ee 5d <48> c7 c7 80 d9 9e 89 e9 77 fe ff ff cc cc cc cc cc cc cc 0f 1f 44
RSP: 0018:ffffc9000c507af0 EFLAGS: 00000046
RAX: 0000000000000001 RBX: 0000000000000000 RCX: 1ffffffff1a077ab
RDX: 0000000000000000 RSI: ffffffff894bac40 RDI: ffffffff894bac40
RBP: ffffffff8b3739e0 R08: 0000000000000000 R09: ffffffff8d038b8f
R10: fffffbfff1a07171 R11: 0000000000000000 R12: 0000000000000001
R13: ffff88802f858bc0 R14: 00000000ffffffff R15: ffffffff889a5430
FS: 0000000000000000(0000) GS:ffff8880b9e00000(0000) knlGS:0000000000000000

CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 00007fbcc03ca000 CR3: 0000000011523000 CR4: 0000000000350ef0
Call Trace:
lockdep_recursion_inc kernel/locking/lockdep.c:432 [inline]
lock_is_held_type+0x34/0x100 kernel/locking/lockdep.c:5475
lock_is_held include/linux/lockdep.h:271 [inline]
rcu_read_lock_sched_held+0x3a/0x70 kernel/rcu/update.c:123
trace_lock_release include/trace/events/lock.h:58 [inline]
lock_release+0x5b7/0x710 kernel/locking/lockdep.c:5448
__raw_spin_unlock_bh include/linux/spinlock_api_smp.h:174 [inline]
_raw_spin_unlock_bh+0x12/0x30 kernel/locking/spinlock.c:207
spin_unlock_bh include/linux/spinlock.h:399 [inline]
batadv_nc_purge_paths+0x2a5/0x3a0 net/batman-adv/network-coding.c:467
batadv_nc_worker+0x831/0xe50 net/batman-adv/network-coding.c:716
process_one_work+0x98d/0x15f0 kernel/workqueue.c:2275
worker_thread+0x64c/0x1120 kernel/workqueue.c:2421
kthread+0x3b1/0x4a0 kernel/kthread.c:292
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:296


Tested on:

commit: a1235e44 io_uring: cancel all requests on task exit
git tree: git://git.kernel.dk/linux-block io_uring-5.11
console output: https://syzkaller.appspot.com/x/log.txt?x=10c53584d00000
kernel config: https://syzkaller.appspot.com/x/.config?x=c6b6b5cccb0f38f2

[Hillf Danton]

Mon, 18 Jan 2021 04:46:10 -0800

>
> syzbot has tested the proposed patch but the reproducer is still triggering an issue:
> INFO: task hung in io_sq_thread_stop
>
> INFO: task kworker/u4:0:8 blocked for more than 143 seconds.
> Not tainted 5.11.0-rc1-syzkaller #0
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> task:kworker/u4:0 state:D stack:24056 pid: 8 ppid: 2 flags:0x00004000
> Workqueue: events_unbound io_ring_exit_work
> Call Trace:
> context_switch kernel/sched/core.c:4313 [inline]
> __schedule+0x90c/0x21a0 kernel/sched/core.c:5064
> schedule+0xcf/0x270 kernel/sched/core.c:5143
> schedule_timeout+0x1d8/0x250 kernel/time/timer.c:1854
> do_wait_for_common kernel/sched/completion.c:85 [inline]
> __wait_for_common kernel/sched/completion.c:106 [inline]
> wait_for_common kernel/sched/completion.c:117 [inline]
> wait_for_completion+0x163/0x260 kernel/sched/completion.c:138

Wakeup did not come within 120 seconds.

Always do wakeup before park.

--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -7460,6 +7460,14 @@ static void io_sq_thread_stop(struct io_
wake_up_process(sqd->thread);
wait_for_completion(&ctx->sq_thread_comp);

+ /*
+ * We may arrive here from io_ring_exit_work() and it
+ * is too late for a wakeup as per IORING_SQ_NEED_WAKEUP
+ * to come, do it before park to cut the risk of hang -
+ * we will wait for the wakeup from parkme while the
+ * kthread itself is sleeping.
+ */
+ wake_up_process(sqd->thread);
io_sq_thread_park(sqd);
}

[syzbot]

syzbot has found a reproducer for the following issue on:

HEAD commit: 9f29bd8b Merge tag 'fs_for_v5.11-rc5' of git://git.kernel...
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=169f4e9f500000
kernel config: https://syzkaller.appspot.com/x/.config?x=39701af622f054a9

dashboard link: https://syzkaller.appspot.com/bug?extid=2f5d1785dc624932da78
compiler: gcc (GCC) 10.1.0-syz 20200507

syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1156bd20d00000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=15ce819f500000

The issue was bisected to:

commit dcd479e10a0510522a5d88b29b8f79ea3467d501
Author: Johannes Berg <johann...@intel.com>
Date: Fri Oct 9 12:17:11 2020 +0000

mac80211: always wind down STA state

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=13b8b83b500000

final oops: https://syzkaller.appspot.com/x/report.txt?x=1078b83b500000
console output: https://syzkaller.appspot.com/x/log.txt?x=17b8b83b500000

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+2f5d17...@syzkaller.appspotmail.com

Fixes: dcd479e10a05 ("mac80211: always wind down STA state")

------------[ cut here ]------------
WARNING: CPU: 0 PID: 8572 at fs/io_uring.c:8917 io_disable_sqo_submit+0x13d/0x180 fs/io_uring.c:8917
Modules linked in:
CPU: 1 PID: 8572 Comm: syz-executor518 Not tainted 5.11.0-rc4-syzkaller #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011

RIP: 0010:io_disable_sqo_submit+0x13d/0x180 fs/io_uring.c:8917
Code: e0 07 83 c0 03 38 d0 7c 04 84 d2 75 2e 83 8b 14 01 00 00 01 4c 89 e7 e8 d1 6d 25 07 5b 5d 41 5c e9 48 22 9b ff e8 43 22 9b ff <0f> 0b e9 00 ff ff ff e8 87 a1 dd ff e9 37 ff ff ff e8 4d a1 dd ff
RSP: 0018:ffffc90001c17df0 EFLAGS: 00010293
RAX: 0000000000000000 RBX: ffff88801c409000 RCX: 0000000000000000
RDX: ffff8880287e8040 RSI: ffffffff81d7aa8d RDI: ffff88801c4090d0
RBP: ffff8880198a1780 R08: 0000000000000000 R09: 0000000012c8a801
R10: ffffffff81d7ad45 R11: 0000000000000001 R12: ffff88801c409000
R13: ffff888012c8a801 R14: ffff88801c409040 R15: ffff88801c4090d0
FS: 00007f60e950b700(0000) GS:ffff8880b9f00000(0000) knlGS:0000000000000000

CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 00007f60e950adb8 CR3: 0000000015b41000 CR4: 00000000001506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400

Call Trace:
io_uring_flush+0x28b/0x3a0 fs/io_uring.c:9134
filp_close+0xb4/0x170 fs/open.c:1280

do_dup2+0x294/0x520 fs/file.c:1024
ksys_dup3+0x22f/0x360 fs/file.c:1136
__do_sys_dup2 fs/file.c:1162 [inline]
__se_sys_dup2 fs/file.c:1150 [inline]
__x64_sys_dup2+0x71/0x3a0 fs/file.c:1150
do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46
entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x447019
Code: e8 0c e8 ff ff 48 83 c4 18 c3 0f 1f 80 00 00 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 db 06 fc ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007f60e950ace8 EFLAGS: 00000246 ORIG_RAX: 0000000000000021
RAX: ffffffffffffffda RBX: 00000000006dbc38 RCX: 0000000000447019
RDX: 0000000000447019 RSI: 0000000000000003 RDI: 0000000000000005
RBP: 00000000006dbc30 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000006dbc3c
R13: 00007ffc5b18d21f R14: 00007f60e950b9c0 R15: 00000000006dbc30

[Pavel Begunkov]

On 18/01/2021 12:46, syzbot wrote:
> Hello,
>
> syzbot has tested the proposed patch but the reproducer is still triggering an issue:
> INFO: task hung in io_sq_thread_stop

#syz test: git://git.kernel.dk/linux-block for-5.12/io_uring

--
Pavel Begunkov

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:

WARNING in io_uring_cancel_task_requests

------------[ cut here ]------------
WARNING: CPU: 1 PID: 10843 at fs/io_uring.c:9039 io_uring_cancel_task_requests+0xe55/0x10c0 fs/io_uring.c:9039
Modules linked in:
CPU: 1 PID: 10843 Comm: syz-executor.3 Not tainted 5.11.0-rc5-syzkaller #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011

RIP: 0010:io_uring_cancel_task_requests+0xe55/0x10c0 fs/io_uring.c:9039
Code: 00 00 e9 1c fe ff ff 48 8b 7c 24 18 e8 14 21 db ff e9 f2 fc ff ff 48 8b 7c 24 18 e8 05 21 db ff e9 64 f2 ff ff e8 9b a0 98 ff <0f> 0b e9 ed f2 ff ff e8 ff 20 db ff e9 c8 f5 ff ff 4c 89 ef e8 72
RSP: 0018:ffffc9000cc37950 EFLAGS: 00010293
RAX: 0000000000000000 RBX: ffff888027fcc000 RCX: 0000000000000000
RDX: ffff888045a1a040 RSI: ffffffff81da2255 RDI: ffff888027fcc0d0
RBP: ffff888027fcc0e8 R08: 0000000000000000 R09: ffff888045a1a047
R10: ffffffff81da14cf R11: 0000000000000000 R12: ffff888027fcc000
R13: ffff888045a1a040 R14: ffff88802e748000 R15: ffff88803ca86018

FS: 0000000000000000(0000) GS:ffff8880b9e00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 00007f09d5e60d40 CR3: 0000000028319000 CR4: 00000000001506f0

DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:

io_uring_flush+0x47b/0x6e0 fs/io_uring.c:9224
filp_close+0xb4/0x170 fs/open.c:1286
close_files fs/file.c:403 [inline]
put_files_struct fs/file.c:418 [inline]
put_files_struct+0x1cc/0x350 fs/file.c:415
exit_files+0x7e/0xa0 fs/file.c:435
do_exit+0xc22/0x2ae0 kernel/exit.c:820
do_group_exit+0x125/0x310 kernel/exit.c:922
get_signal+0x427/0x20f0 kernel/signal.c:2773
arch_do_signal_or_restart+0x2a8/0x1eb0 arch/x86/kernel/signal.c:811
handle_signal_work kernel/entry/common.c:147 [inline]
exit_to_user_mode_loop kernel/entry/common.c:171 [inline]
exit_to_user_mode_prepare+0x148/0x250 kernel/entry/common.c:201
__syscall_exit_to_user_mode_work kernel/entry/common.c:291 [inline]
syscall_exit_to_user_mode+0x19/0x50 kernel/entry/common.c:302
entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x465b09
Code: Unable to access opcode bytes at RIP 0x465adf.
RSP: 002b:00007f21a56f2108 EFLAGS: 00000202 ORIG_RAX: 00000000000001a9
RAX: 0000000000000004 RBX: 000000000056c0b0 RCX: 0000000000465b09
RDX: 00000000206d4000 RSI: 00000000200002c0 RDI: 0000000000000187
RBP: 00000000200002c0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000000
R13: 00000000206d4000 R14: 0000000000000000 R15: 0000000020ee7000


Tested on:

commit: 1d538571 io_uring: check kthread parked flag before sqthre..
git tree: git://git.kernel.dk/linux-block for-5.12/io_uring
console output: https://syzkaller.appspot.com/x/log.txt?x=14532690d00000
kernel config: https://syzkaller.appspot.com/x/.config?x=fe3e1032f57d6d25

[Pavel Begunkov]

On 01/02/2021 15:30, syzbot wrote:
> Hello,
>
> syzbot has tested the proposed patch but the reproducer is still triggering an issue:
> WARNING in io_uring_cancel_task_requests

#syz fix: io_uring: fix sqo ownership false positive warning

--
Pavel Begunkov

[Hillf Danton]

On 1 Feb 2021 23:36 Pavel Begunkov wrote:

>On 01/02/2021 15:30, syzbot wrote:

>> Hello,

>>

>> syzbot has tested the proposed patch but the reproducer is still triggering an issue:

>> WARNING in io_uring_cancel_task_requests

> 

>#syz fix: io_uring: fix sqo ownership false positive warning

 

It is more helpful if you can add a link to the fix in any form

because it will save two minutes at least for those who would

like to take a look at it.