[syzbot] INFO: task hung in nbd_start_device_ioctl

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: a5541c0811a0 Merge branch 'for-next/core' into for-kernelci
git tree: git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci
console output: https://syzkaller.appspot.com/x/log.txt?x=134d1af3880000
kernel config: https://syzkaller.appspot.com/x/.config?x=cbd4e584773e9397
dashboard link: https://syzkaller.appspot.com/bug?extid=9df558a2e82f6f511bfb
compiler: Debian clang version 13.0.1-++20220126092033+75e33f71c2da-1~exp1~20220126212112.63, GNU ld (GNU Binutils for Debian) 2.35.2
userspace arch: arm64
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=16b387fb880000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=153b534d880000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/4b7702208fb9/disk-a5541c08.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/9ec0153ec051/vmlinux-a5541c08.xz
kernel image: https://storage.googleapis.com/syzbot-assets/6f8725ad290a/Image-a5541c08.gz.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+9df558...@syzkaller.appspotmail.com

INFO: task syz-executor270:3108 blocked for more than 143 seconds.
Not tainted 6.1.0-rc8-syzkaller-33330-ga5541c0811a0 #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor270 state:D stack:0 pid:3108 ppid:3107 flags:0x00000009
Call trace:
__switch_to+0x180/0x298 arch/arm64/kernel/process.c:555
context_switch kernel/sched/core.c:5209 [inline]
__schedule+0x408/0x594 kernel/sched/core.c:6521
schedule+0x64/0xa4 kernel/sched/core.c:6597
schedule_timeout+0x64/0x1b4 kernel/time/timer.c:1911
do_wait_for_common+0xf4/0x184 kernel/sched/completion.c:85
__wait_for_common kernel/sched/completion.c:106 [inline]
wait_for_common kernel/sched/completion.c:117 [inline]
wait_for_completion+0x48/0x60 kernel/sched/completion.c:138
__flush_workqueue+0x2b0/0x6dc kernel/workqueue.c:2861
nbd_start_device_ioctl+0x1e4/0x2b8 drivers/block/nbd.c:1419
__nbd_ioctl+0x16c/0x334 drivers/block/nbd.c:1481
nbd_ioctl+0xec/0x13c drivers/block/nbd.c:1521
blkdev_ioctl+0x24c/0x6d8 block/ioctl.c:614
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:870 [inline]
__se_sys_ioctl fs/ioctl.c:856 [inline]
__arm64_sys_ioctl+0xd0/0x140 fs/ioctl.c:856
__invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
invoke_syscall arch/arm64/kernel/syscall.c:52 [inline]
el0_svc_common+0x138/0x220 arch/arm64/kernel/syscall.c:142
do_el0_svc+0x48/0x140 arch/arm64/kernel/syscall.c:197
el0_svc+0x58/0x150 arch/arm64/kernel/entry-common.c:637
el0t_64_sync_handler+0x84/0xf0 arch/arm64/kernel/entry-common.c:655
el0t_64_sync+0x190/0x194 arch/arm64/kernel/entry.S:584

Showing all locks held in the system:
1 lock held by rcu_tasks_kthre/11:
#0: ffff80000d4a4768 (rcu_tasks.tasks_gp_mutex){+.+.}-{3:3}, at: rcu_tasks_one_gp+0x3c/0x450 kernel/rcu/tasks.h:507
1 lock held by rcu_tasks_trace/12:
#0: ffff80000d4a4db8 (rcu_tasks_trace.tasks_gp_mutex){+.+.}-{3:3}, at: rcu_tasks_one_gp+0x3c/0x450 kernel/rcu/tasks.h:507
1 lock held by khungtaskd/27:
#0: ffff80000d4a4640 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire+0x4/0x48 include/linux/rcupdate.h:303
2 locks held by kworker/u5:0/46:
#0: ffff0000c6236d38 ((wq_completion)nbd0-recv){+.+.}-{0:0}, at: process_one_work+0x270/0x504 kernel/workqueue.c:2262
#1: ffff80000f703d80 ((work_completion)(&args->work)){+.+.}-{0:0}, at: process_one_work+0x29c/0x504 kernel/workqueue.c:2264
2 locks held by getty/2759:
#0: ffff0000c8153898 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x28/0x58 drivers/tty/tty_ldisc.c:244
#1: ffff80000f6be2f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0x19c/0x89c drivers/tty/n_tty.c:2177

=============================================



---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
syzbot can test patches for this issue, for details see:
https://goo.gl/tpsmEJ#testing-patches

[Hillf Danton]

On 12 Dec 2022 13:09:38 -0800

> syzbot found the following issue on:
>
> HEAD commit: a5541c0811a0 Merge branch 'for-next/core' into for-kernelci
> git tree: git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci

> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=153b534d880000

Reproduce to print lock chain like

mutex_lock(&nbd->config_lock);
__nbd_ioctl();
mutex_unlock(&nbd->config_lock);

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
INFO: task hung in nbd_start_device_ioctl

INFO: task syz-executor.0:5958 blocked for more than 143 seconds.
Not tainted 6.2.0-rc2-syzkaller-00127-g69b41ac87e4a #0

"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

task:syz-executor.0 state:D stack:0 pid:5958 ppid:4867 flags:0x00000009

Call trace:
__switch_to+0x180/0x298 arch/arm64/kernel/process.c:555

context_switch kernel/sched/core.c:5244 [inline]
__schedule+0x438/0x5a0 kernel/sched/core.c:6555
schedule+0x64/0xa4 kernel/sched/core.c:6631
schedule_preempt_disabled+0x18/0x2c kernel/sched/core.c:6690
__mutex_lock_common+0x788/0xca8 kernel/locking/mutex.c:679
__mutex_lock kernel/locking/mutex.c:747 [inline]
mutex_lock_nested+0x38/0x44 kernel/locking/mutex.c:799
nbd_start_device_ioctl+0x1f0/0x2b8 drivers/block/nbd.c:1421
__nbd_ioctl+0x16c/0x334 drivers/block/nbd.c:1482
nbd_ioctl+0xec/0x13c drivers/block/nbd.c:1522
blkdev_ioctl+0x24c/0x6d8 block/ioctl.c:616

vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:870 [inline]
__se_sys_ioctl fs/ioctl.c:856 [inline]
__arm64_sys_ioctl+0xd0/0x140 fs/ioctl.c:856
__invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
invoke_syscall arch/arm64/kernel/syscall.c:52 [inline]
el0_svc_common+0x138/0x220 arch/arm64/kernel/syscall.c:142
do_el0_svc+0x48/0x140 arch/arm64/kernel/syscall.c:197
el0_svc+0x58/0x150 arch/arm64/kernel/entry-common.c:637
el0t_64_sync_handler+0x84/0xf0 arch/arm64/kernel/entry-common.c:655
el0t_64_sync+0x190/0x194 arch/arm64/kernel/entry.S:584

INFO: task syz-executor.0:5959 blocked for more than 143 seconds.
Not tainted 6.2.0-rc2-syzkaller-00127-g69b41ac87e4a #0

"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

task:syz-executor.0 state:D stack:0 pid:5959 ppid:4867 flags:0x00000001

Call trace:
__switch_to+0x180/0x298 arch/arm64/kernel/process.c:555

context_switch kernel/sched/core.c:5244 [inline]
__schedule+0x438/0x5a0 kernel/sched/core.c:6555
schedule+0x64/0xa4 kernel/sched/core.c:6631
blk_mq_freeze_queue_wait+0x94/0xf4 block/blk-mq.c:180
blk_freeze_queue+0x94/0xa8 block/blk-mq.c:207
blk_mq_freeze_queue+0x20/0x30 block/blk-mq.c:216
nbd_add_socket+0x78/0x27c drivers/block/nbd.c:1122
__nbd_ioctl+0x80/0x334 drivers/block/nbd.c:1465
nbd_ioctl+0xec/0x13c drivers/block/nbd.c:1522
blkdev_ioctl+0x24c/0x6d8 block/ioctl.c:616

vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:870 [inline]
__se_sys_ioctl fs/ioctl.c:856 [inline]
__arm64_sys_ioctl+0xd0/0x140 fs/ioctl.c:856
__invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
invoke_syscall arch/arm64/kernel/syscall.c:52 [inline]
el0_svc_common+0x138/0x220 arch/arm64/kernel/syscall.c:142
do_el0_svc+0x48/0x140 arch/arm64/kernel/syscall.c:197
el0_svc+0x58/0x150 arch/arm64/kernel/entry-common.c:637
el0t_64_sync_handler+0x84/0xf0 arch/arm64/kernel/entry-common.c:655
el0t_64_sync+0x190/0x194 arch/arm64/kernel/entry.S:584

Showing all locks held in the system:
1 lock held by rcu_tasks_kthre/11:

#0: ffff80000d635648 (rcu_tasks.tasks_gp_mutex){+.+.}-{3:3}, at: rcu_tasks_one_gp+0x3c/0x450 kernel/rcu/tasks.h:507

1 lock held by rcu_tasks_trace/12:

#0: ffff80000d635c98 (rcu_tasks_trace.tasks_gp_mutex){+.+.}-{3:3}, at: rcu_tasks_one_gp+0x3c/0x450 kernel/rcu/tasks.h:507

1 lock held by khungtaskd/27:

#0: ffff80000d635520 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire+0x4/0x48 include/linux/rcupdate.h:324
2 locks held by getty/4074:
#0: ffff0000c546c898 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x28/0x58 drivers/tty/tty_ldisc.c:244
#1: ffff80000f8462f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0x19c/0x89c drivers/tty/n_tty.c:2177
1 lock held by udevd/4866:
#0: ffff0000c424ccc8 (&disk->open_mutex){+.+.}-{3:3}, at: blkdev_get_by_dev+0xfc/0x2fc block/bdev.c:813
1 lock held by syz-executor.0/5958:
#0: ffff0000c5eec5a0 (&nbd->config_lock){+.+.}-{3:3}, at: nbd_start_device_ioctl+0x1f0/0x2b8 drivers/block/nbd.c:1421
1 lock held by syz-executor.0/5959:
#0: ffff0000c5eec5a0 (&nbd->config_lock){+.+.}-{3:3}, at: nbd_ioctl+0x70/0x13c drivers/block/nbd.c:1515
1 lock held by syz-executor293/5981:
#0: ffff0000c424ccc8 (&disk->open_mutex){+.+.}-{3:3}, at: blkdev_get_by_dev+0xfc/0x2fc block/bdev.c:813
1 lock held by syz-executor293/5982:
#0: ffff0000c424ccc8 (&disk->open_mutex){+.+.}-{3:3}, at: blkdev_get_by_dev+0xfc/0x2fc block/bdev.c:813

=============================================



Tested on:

commit: 69b41ac8 Merge tag 'for-6.2-rc2-tag' of git://git.kern..
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master
console output: https://syzkaller.appspot.com/x/log.txt?x=102810aa480000
kernel config: https://syzkaller.appspot.com/x/.config?x=970a539d6092d0

dashboard link: https://syzkaller.appspot.com/bug?extid=9df558a2e82f6f511bfb
compiler: Debian clang version 13.0.1-++20220126092033+75e33f71c2da-1~exp1~20220126212112.63, GNU ld (GNU Binutils for Debian) 2.35.2
userspace arch: arm64

Note: no patches were applied.