[syzbot] [block?] INFO: task hung in bdev_release

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: 8c9660f65153 Add linux-next specific files for 20231124
git tree: linux-next
console+strace: https://syzkaller.appspot.com/x/log.txt?x=14c8a334e80000
kernel config: https://syzkaller.appspot.com/x/.config?x=ca1e8655505e280
dashboard link: https://syzkaller.appspot.com/bug?extid=4da851837827326a7cd4
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=119809d0e80000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=13930542e80000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/345ed4af3a0d/disk-8c9660f6.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/191053c69d57/vmlinux-8c9660f6.xz
kernel image: https://storage.googleapis.com/syzbot-assets/aac7ee5e55e0/bzImage-8c9660f6.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+4da851...@syzkaller.appspotmail.com

INFO: task syz-executor136:5067 blocked for more than 143 seconds.
Not tainted 6.7.0-rc2-next-20231124-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor136 state:D stack:26736 pid:5067 tgid:5066 ppid:5064 flags:0x00004006
Call Trace:
<TASK>
context_switch kernel/sched/core.c:5399 [inline]
__schedule+0xf15/0x5c00 kernel/sched/core.c:6726
__schedule_loop kernel/sched/core.c:6801 [inline]
schedule+0xe7/0x270 kernel/sched/core.c:6816
schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:6873
__mutex_lock_common kernel/locking/mutex.c:679 [inline]
__mutex_lock+0x5b4/0x9c0 kernel/locking/mutex.c:747
bdev_release+0xcd/0xa90 block/bdev.c:967
blkdev_release+0x37/0x50 block/fops.c:616
__fput+0x270/0xbb0 fs/file_table.c:394
task_work_run+0x14c/0x240 kernel/task_work.c:180
ptrace_notify+0x10a/0x130 kernel/signal.c:2390
ptrace_report_syscall include/linux/ptrace.h:411 [inline]
ptrace_report_syscall_exit include/linux/ptrace.h:473 [inline]
syscall_exit_work kernel/entry/common.c:251 [inline]
syscall_exit_to_user_mode_prepare+0x122/0x230 kernel/entry/common.c:278
__syscall_exit_to_user_mode_work kernel/entry/common.c:283 [inline]
syscall_exit_to_user_mode+0xe/0x60 kernel/entry/common.c:296
do_syscall_64+0x4d/0x110 arch/x86/entry/common.c:88
entry_SYSCALL_64_after_hwframe+0x62/0x6a
RIP: 0033:0x7f7015ea8479
RSP: 002b:00007f7015e66218 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: 0000000000000000 RBX: 00007f7015f2f328 RCX: 00007f7015ea8479
RDX: 0000000000000000 RSI: 000000000000ab03 RDI: 0000000000000005
RBP: 00007f7015f2f320 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007f7015f2f32c
R13: 00007f7015efc18c R14: 64626e2f7665642f R15: 00000000ffffff43
</TASK>

Showing all locks held in the system:
1 lock held by khungtaskd/29:
#0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:301 [inline]
#0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:747 [inline]
#0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x75/0x340 kernel/locking/lockdep.c:6613
2 locks held by getty/4817:
#0: ffff88802ae300a0 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x24/0x80 drivers/tty/tty_ldisc.c:243
#1: ffffc90002f062f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0xfc4/0x1490 drivers/tty/n_tty.c:2201
1 lock held by udevd/5057:
#0: ffff888143bbf4c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_open_by_dev+0x27c/0xed0 block/bdev.c:857
1 lock held by syz-executor136/5067:
#0: ffff888143bbf4c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_release+0xcd/0xa90 block/bdev.c:967

=============================================

NMI backtrace for cpu 1
CPU: 1 PID: 29 Comm: khungtaskd Not tainted 6.7.0-rc2-next-20231124-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/10/2023
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xd9/0x1b0 lib/dump_stack.c:106
nmi_cpu_backtrace+0x277/0x390 lib/nmi_backtrace.c:113
nmi_trigger_cpumask_backtrace+0x299/0x300 lib/nmi_backtrace.c:62
trigger_all_cpu_backtrace include/linux/nmi.h:160 [inline]
check_hung_uninterruptible_tasks kernel/hung_task.c:222 [inline]
watchdog+0xf86/0x1210 kernel/hung_task.c:379
kthread+0x2c1/0x3a0 kernel/kthread.c:389
ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242
</TASK>
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 59 Comm: kworker/u4:4 Not tainted 6.7.0-rc2-next-20231124-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/10/2023
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:arch_static_branch arch/x86/include/asm/jump_label.h:27 [inline]
RIP: 0010:static_key_false include/linux/jump_label.h:207 [inline]
RIP: 0010:native_write_msr arch/x86/include/asm/msr.h:147 [inline]
RIP: 0010:wrmsrl arch/x86/include/asm/msr.h:262 [inline]
RIP: 0010:native_x2apic_icr_write arch/x86/include/asm/apic.h:216 [inline]
RIP: 0010:__x2apic_send_IPI_dest arch/x86/kernel/apic/x2apic_phys.c:113 [inline]
RIP: 0010:x2apic_send_IPI+0x96/0xe0 arch/x86/kernel/apic/x2apic_phys.c:50
Code: 8b 13 0f ae f0 0f ae e8 b9 00 04 00 00 41 83 fc 02 44 89 e0 48 0f 44 c1 48 c1 e2 20 b9 30 08 00 00 48 09 d0 48 c1 ea 20 0f 30 <66> 90 5b 5d 41 5c c3 5b 31 d2 48 89 c6 bf 30 08 00 00 5d 41 5c e9
RSP: 0018:ffffc900015a7900 EFLAGS: 00000202
RAX: 00000001000000fb RBX: ffff8880b9921a2c RCX: 0000000000000830
RDX: 0000000000000001 RSI: 00000000000000fb RDI: ffffffff8ca75a68
RBP: 0000000000000001 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000006 R12: 00000000000000fb
R13: 000000000003bccc R14: 0000000000000001 R15: ffff8880b983d8c0
FS: 0000000000000000(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055750a5bb680 CR3: 000000000cd78000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<NMI>
</NMI>
<TASK>
arch_send_call_function_single_ipi arch/x86/include/asm/smp.h:101 [inline]
send_call_function_single_ipi kernel/smp.c:117 [inline]
smp_call_function_many_cond+0x12ef/0x1570 kernel/smp.c:837
on_each_cpu_cond_mask+0x40/0x90 kernel/smp.c:1023
on_each_cpu include/linux/smp.h:71 [inline]
text_poke_sync arch/x86/kernel/alternative.c:2008 [inline]
text_poke_bp_batch+0x655/0x750 arch/x86/kernel/alternative.c:2218
text_poke_flush arch/x86/kernel/alternative.c:2409 [inline]
text_poke_flush arch/x86/kernel/alternative.c:2406 [inline]
text_poke_finish+0x30/0x40 arch/x86/kernel/alternative.c:2416
arch_jump_label_transform_apply+0x1c/0x30 arch/x86/kernel/jump_label.c:146
jump_label_update+0x1d7/0x400 kernel/jump_label.c:829
static_key_enable_cpuslocked+0x1b7/0x270 kernel/jump_label.c:205
static_key_enable+0x1a/0x20 kernel/jump_label.c:218
toggle_allocation_gate mm/kfence/core.c:830 [inline]
toggle_allocation_gate+0xf4/0x250 mm/kfence/core.c:822
process_one_work+0x8a4/0x15f0 kernel/workqueue.c:2633
process_scheduled_works kernel/workqueue.c:2706 [inline]
worker_thread+0x8b6/0x1290 kernel/workqueue.c:2787
kthread+0x2c1/0x3a0 kernel/kthread.c:389
ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242
</TASK>
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.905 msecs


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup

[Edward Adam Davis]

please test task hung in bdev_release

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git 8c9660f65153

diff --git a/block/bdev.c b/block/bdev.c
index 6f73b02d549c..17ead61b00e2 100644
--- a/block/bdev.c
+++ b/block/bdev.c
@@ -854,6 +854,7 @@ struct bdev_handle *bdev_open_by_dev(dev_t dev, blk_mode_t mode, void *holder,

disk_block_events(disk);

+ printk("next om, b: %p, disk: %p, %s\n", bdev, disk, __func__);
mutex_lock(&disk->open_mutex);
ret = -ENXIO;
if (!disk_live(disk))
@@ -887,6 +888,7 @@ struct bdev_handle *bdev_open_by_dev(dev_t dev, blk_mode_t mode, void *holder,
}
}
mutex_unlock(&disk->open_mutex);
+ printk("out om, b: %p, disk: %p, %s\n", bdev, disk, __func__);

if (unblock_events)
disk_unblock_events(disk);
@@ -900,6 +902,7 @@ struct bdev_handle *bdev_open_by_dev(dev_t dev, blk_mode_t mode, void *holder,
if (holder)
bd_abort_claiming(bdev, holder);
mutex_unlock(&disk->open_mutex);
+ printk("out om, b: %p, %s\n", bdev, __func__);
disk_unblock_events(disk);
put_blkdev:
blkdev_put_no_open(bdev);
@@ -964,6 +967,7 @@ void bdev_release(struct bdev_handle *handle)
if (atomic_read(&bdev->bd_openers) == 1)
sync_blockdev(bdev);

+ printk("nxt om, b: %p, dk: %p, %s\n", bdev, disk, __func__);
mutex_lock(&disk->open_mutex);
bdev_yield_write_access(bdev, handle->mode);

@@ -982,6 +986,7 @@ void bdev_release(struct bdev_handle *handle)
else
blkdev_put_whole(bdev);
mutex_unlock(&disk->open_mutex);
+ printk("out om, b: %p, dk: %p, %s\n", bdev, disk, __func__);

module_put(disk->fops->owner);
blkdev_put_no_open(bdev);
diff --git a/block/ioctl.c b/block/ioctl.c
index 9c73a763ef88..67825e6bec13 100644
--- a/block/ioctl.c
+++ b/block/ioctl.c
@@ -483,6 +483,7 @@ static int blkdev_bszset(struct block_device *bdev, blk_mode_t mode,
if (mode & BLK_OPEN_EXCL)
return set_blocksize(bdev, n);

+ printk("s: %d, b: %p, bd: %p, %s\n", n, bdev, bdev->bd_dev, __func__);
handle = bdev_open_by_dev(bdev->bd_dev, mode, &bdev, NULL);
if (IS_ERR(handle))
return -EBUSY;
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index b6414e1e645b..3cc7993b0b67 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -1137,6 +1137,7 @@ static int nbd_add_socket(struct nbd_device *nbd, unsigned long arg,
int err;

/* Arg will be cast to int, check it to avoid overflow */
+ printk("arg: %d, nbd: %p, %s\n", arg, nbd, __func__);
if (arg > INT_MAX)
return -EINVAL;
sock = nbd_get_socket(nbd, arg, &err);
@@ -1188,10 +1189,12 @@ static int nbd_add_socket(struct nbd_device *nbd, unsigned long arg,
socks[config->num_connections++] = nsock;
atomic_inc(&config->live_connections);
blk_mq_unfreeze_queue(nbd->disk->queue);
+ printk("arg: %d, nbd: %p, nd: %p, nc: %d, %s\n", arg, nbd, nbd->disk, config->num_connections, __func__);

return 0;

put_socket:
+ printk("nbd: %p, %s\n", nbd, __func__);
blk_mq_unfreeze_queue(nbd->disk->queue);
sockfd_put(sock);
return err;
@@ -1372,6 +1375,7 @@ static int nbd_start_device(struct nbd_device *nbd)
int num_connections = config->num_connections;
int error = 0, i;

+ printk("dev: %p, nc: %d, pid: %d, socks: %p, %s\n", nbd, num_connections, nbd->pid, config->socks, __func__);
if (nbd->pid)
return -EBUSY;
if (!config->socks)
@@ -1425,6 +1429,7 @@ static int nbd_start_device(struct nbd_device *nbd)
args->index = i;
queue_work(nbd->recv_workq, &args->work);
}
+ printk("bs: %lld, blks: %lld, %s\n", config->bytesize, nbd_blksize(config), __func__);
return nbd_set_size(nbd, config->bytesize, nbd_blksize(config));
}

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
INFO: task hung in bdev_release

INFO: task syz-executor.0:5479 blocked for more than 143 seconds.
Not tainted 6.7.0-rc2-next-20231124-syzkaller-dirty #0

"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

task:syz-executor.0 state:D stack:27744 pid:5479 tgid:5478 ppid:5422 flags:0x00004006

Call Trace:
<TASK>
context_switch kernel/sched/core.c:5399 [inline]
__schedule+0xf15/0x5c00 kernel/sched/core.c:6726
__schedule_loop kernel/sched/core.c:6801 [inline]
schedule+0xe7/0x270 kernel/sched/core.c:6816
schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:6873
__mutex_lock_common kernel/locking/mutex.c:679 [inline]
__mutex_lock+0x5b4/0x9c0 kernel/locking/mutex.c:747

bdev_release+0xe6/0xac0 block/bdev.c:971

blkdev_release+0x37/0x50 block/fops.c:616
__fput+0x270/0xbb0 fs/file_table.c:394
task_work_run+0x14c/0x240 kernel/task_work.c:180

get_signal+0x105a/0x2770 kernel/signal.c:2669
arch_do_signal_or_restart+0x90/0x7e0 arch/x86/kernel/signal.c:309
exit_to_user_mode_loop kernel/entry/common.c:168 [inline]
exit_to_user_mode_prepare+0x11e/0x240 kernel/entry/common.c:204
__syscall_exit_to_user_mode_work kernel/entry/common.c:285 [inline]
syscall_exit_to_user_mode+0x1e/0x60 kernel/entry/common.c:296
do_syscall_64+0x4d/0x110 arch/x86/entry/common.c:88
entry_SYSCALL_64_after_hwframe+0x62/0x6a
RIP: 0033:0x7f32c047cae9
RSP: 002b:00007f32c11ec0c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: 0000000000000000 RBX: 00007f32c059bf80 RCX: 00007f32c047cae9

RDX: 0000000000000000 RSI: 000000000000ab03 RDI: 0000000000000005

RBP: 00007f32c04c847a R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 000000000000000b R14: 00007f32c059bf80 R15: 00007fff376969d8

</TASK>

Showing all locks held in the system:

2 locks held by kworker/u4:0/11:
#0: ffff8880b993c718 (&rq->__lock){-.-.}-{2:2}, at: raw_spin_rq_lock_nested+0x29/0x130 kernel/sched/core.c:558
#1: ffff8880b9928888 (&per_cpu_ptr(group->pcpu, cpu)->seq){-.-.}-{0:0}, at: psi_task_switch+0x2d9/0x900 kernel/sched/psi.c:988

1 lock held by khungtaskd/29:
#0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:301 [inline]
#0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:747 [inline]
#0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x75/0x340 kernel/locking/lockdep.c:6613

2 locks held by getty/4814:
#0: ffff88802b1f20a0 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x24/0x80 drivers/tty/tty_ldisc.c:243
#1: ffffc900031332f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0xfc4/0x1490 drivers/tty/n_tty.c:2201
1 lock held by udevd/5436:
#0: ffff888143f344c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_open_by_dev+0x2ae/0xf50 block/bdev.c:858
1 lock held by syz-executor.0/5479:
#0: ffff888143f344c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_release+0xe6/0xac0 block/bdev.c:971
1 lock held by syz-executor.0/5796:
#0: ffff888143f344c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_open_by_dev+0x2ae/0xf50 block/bdev.c:858
1 lock held by syz-executor.0/5812:
#0: ffff888143f344c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_open_by_dev+0x2ae/0xf50 block/bdev.c:858

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 29 Comm: khungtaskd Not tainted 6.7.0-rc2-next-20231124-syzkaller-dirty #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023

Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xd9/0x1b0 lib/dump_stack.c:106
nmi_cpu_backtrace+0x277/0x390 lib/nmi_backtrace.c:113
nmi_trigger_cpumask_backtrace+0x299/0x300 lib/nmi_backtrace.c:62
trigger_all_cpu_backtrace include/linux/nmi.h:160 [inline]
check_hung_uninterruptible_tasks kernel/hung_task.c:222 [inline]
watchdog+0xf86/0x1210 kernel/hung_task.c:379
kthread+0x2c1/0x3a0 kernel/kthread.c:389
ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242
</TASK>

Sending NMI from CPU 0 to CPUs 1:

NMI backtrace for cpu 1

CPU: 1 PID: 48 Comm: kworker/u4:3 Not tainted 6.7.0-rc2-next-20231124-syzkaller-dirty #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:smp_call_function_many_cond+0x1213/0x1570 kernel/smp.c:845
Code: e8 b2 b5 0b 00 84 db 0f 84 cc fa ff ff e8 75 ba 0b 00 e8 50 29 84 ff e9 c9 fa ff ff e8 66 ba 0b 00 90 0f 0b 90 e9 85 fc ff ff <e8> 58 ba 0b 00 e8 03 73 12 00 4c 8b 74 24 30 31 f6 4c 89 f7 e8 b4
RSP: 0018:ffffc90000b8f920 EFLAGS: 00000006
RAX: 0000000000000000 RBX: 0000000000000001 RCX: ffffffff817c4cc2
RDX: ffff88801a6c3b80 RSI: 0000000000000000 RDI: 0000000000000007
RBP: 0000000000000200 R08: 0000000000000007 R09: 0000000000000000
R10: 0000000000000200 R11: 0000000000000006 R12: 0000000000000001
R13: 000000000003bccc R14: 0000000000000000 R15: ffff8880b993d8c0
FS: 0000000000000000(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000

CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 000055dc28b15680 CR3: 000000000cd78000 CR4: 00000000003506f0

DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<NMI>
</NMI>
<TASK>

on_each_cpu_cond_mask+0x40/0x90 kernel/smp.c:1023
on_each_cpu include/linux/smp.h:71 [inline]
text_poke_sync arch/x86/kernel/alternative.c:2008 [inline]

text_poke_bp_batch+0x561/0x750 arch/x86/kernel/alternative.c:2301

text_poke_flush arch/x86/kernel/alternative.c:2409 [inline]
text_poke_flush arch/x86/kernel/alternative.c:2406 [inline]
text_poke_finish+0x30/0x40 arch/x86/kernel/alternative.c:2416
arch_jump_label_transform_apply+0x1c/0x30 arch/x86/kernel/jump_label.c:146
jump_label_update+0x1d7/0x400 kernel/jump_label.c:829
static_key_enable_cpuslocked+0x1b7/0x270 kernel/jump_label.c:205
static_key_enable+0x1a/0x20 kernel/jump_label.c:218
toggle_allocation_gate mm/kfence/core.c:830 [inline]
toggle_allocation_gate+0xf4/0x250 mm/kfence/core.c:822
process_one_work+0x8a4/0x15f0 kernel/workqueue.c:2633
process_scheduled_works kernel/workqueue.c:2706 [inline]
worker_thread+0x8b6/0x1290 kernel/workqueue.c:2787
kthread+0x2c1/0x3a0 kernel/kthread.c:389
ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242
</TASK>

Tested on:

commit: 8c9660f6 Add linux-next specific files for 20231124
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
console output: https://syzkaller.appspot.com/x/log.txt?x=14080556e80000

kernel config: https://syzkaller.appspot.com/x/.config?x=ca1e8655505e280
dashboard link: https://syzkaller.appspot.com/bug?extid=4da851837827326a7cd4
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=176a65e6e80000

[Hillf Danton]

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:

INFO: task hung in blkdev_put

INFO: task syz-executor.0:5497 blocked for more than 143 seconds.
Not tainted 6.7.0-rc5-syzkaller-00214-gc8e97fc6b4c0 #0

"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

task:syz-executor.0 state:D stack:27872 pid:5497 tgid:5496 ppid:5434 flags:0x00004006
Call Trace:
<TASK>
context_switch kernel/sched/core.c:5376 [inline]
__schedule+0xedb/0x5af0 kernel/sched/core.c:6688
__schedule_loop kernel/sched/core.c:6763 [inline]
schedule+0xe9/0x270 kernel/sched/core.c:6778
schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:6835
__mutex_lock_common kernel/locking/mutex.c:679 [inline]
__mutex_lock+0x5b9/0x9d0 kernel/locking/mutex.c:747
blkdev_put+0xb0/0x8e0 block/bdev.c:930
bdev_release+0x4f/0x80 block/bdev.c:954
blkdev_release+0x37/0x50 block/fops.c:616
__fput+0x270/0xb70 fs/file_table.c:394
task_work_run+0x14d/0x240 kernel/task_work.c:180
get_signal+0x106f/0x2790 kernel/signal.c:2680
arch_do_signal_or_restart+0x90/0x7f0 arch/x86/kernel/signal.c:309
exit_to_user_mode_loop kernel/entry/common.c:168 [inline]
exit_to_user_mode_prepare+0x121/0x240 kernel/entry/common.c:204

__syscall_exit_to_user_mode_work kernel/entry/common.c:285 [inline]
syscall_exit_to_user_mode+0x1e/0x60 kernel/entry/common.c:296

do_syscall_64+0x4d/0x110 arch/x86/entry/common.c:89
entry_SYSCALL_64_after_hwframe+0x63/0x6b
RIP: 0033:0x7f766767cae9
RSP: 002b:00007f766845b0c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: 0000000000000000 RBX: 00007f766779bf80 RCX: 00007f766767cae9

RDX: 0000000000000000 RSI: 000000000000ab03 RDI: 0000000000000005

RBP: 00007f76676c847a R08: 0000000000000000 R09: 0000000000000000

R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000

R13: 000000000000000b R14: 00007f766779bf80 R15: 00007ffc44ea6c78

</TASK>

Showing all locks held in the system:

1 lock held by khungtaskd/29:

#0: ffffffff8cfab760 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:301 [inline]
#0: ffffffff8cfab760 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:747 [inline]
#0: ffffffff8cfab760 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x75/0x340 kernel/locking/lockdep.c:6614
2 locks held by getty/4818:
#0: ffff88814b8ea0a0 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x24/0x80 drivers/tty/tty_ldisc.c:243
#1: ffffc90002f062f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0xfc6/0x1490 drivers/tty/n_tty.c:2201
1 lock held by udevd/5426:
#0: ffff888140b5d4c8 (&disk->open_mutex){+.+.}-{3:3}, at: blkdev_get_by_dev.part.0+0x4ea/0xb10 block/bdev.c:788
1 lock held by syz-executor.0/5497:
#0: ffff888140b5d4c8 (&disk->open_mutex){+.+.}-{3:3}, at: blkdev_put+0xb0/0x8e0 block/bdev.c:930
1 lock held by syz-executor.0/5806:
#0: ffff888140b5d4c8 (&disk->open_mutex){+.+.}-{3:3}, at: blkdev_get_by_dev.part.0+0x4ea/0xb10 block/bdev.c:788
1 lock held by syz-executor.0/5826:
#0: ffff888140b5d4c8 (&disk->open_mutex){+.+.}-{3:3}, at: blkdev_get_by_dev.part.0+0x4ea/0xb10 block/bdev.c:788

=============================================

NMI backtrace for cpu 1
CPU: 1 PID: 29 Comm: khungtaskd Not tainted 6.7.0-rc5-syzkaller-00214-gc8e97fc6b4c0 #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xd9/0x1b0 lib/dump_stack.c:106
nmi_cpu_backtrace+0x277/0x390 lib/nmi_backtrace.c:113
nmi_trigger_cpumask_backtrace+0x299/0x300 lib/nmi_backtrace.c:62
trigger_all_cpu_backtrace include/linux/nmi.h:160 [inline]
check_hung_uninterruptible_tasks kernel/hung_task.c:222 [inline]

watchdog+0xf87/0x1210 kernel/hung_task.c:379
kthread+0x2c6/0x3a0 kernel/kthread.c:388

ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242
</TASK>

Sending NMI from CPU 1 to CPUs 0:

NMI backtrace for cpu 0

CPU: 0 PID: 2415 Comm: kworker/u4:9 Not tainted 6.7.0-rc5-syzkaller-00214-gc8e97fc6b4c0 #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023

Workqueue: events_unbound cfg80211_wiphy_work
RIP: 0010:__sanitizer_cov_trace_pc+0x59/0x60 kernel/kcov.c:225
Code: 82 d8 15 00 00 83 f8 02 75 20 48 8b 8a e0 15 00 00 8b 92 dc 15 00 00 48 8b 01 48 83 c0 01 48 39 d0 73 07 48 89 01 48 89 34 c1 <c3> 66 0f 1f 44 00 00 f3 0f 1e fa 41 57 41 56 49 89 d6 41 55 41 54
RSP: 0018:ffffc9000a8b79d0 EFLAGS: 00000293
RAX: 0000000000000000 RBX: ffff88801a3ba000 RCX: 1ffffffff23e7cce
RDX: ffff8880259f8000 RSI: ffffffff813b3858 RDI: ffff88801a3ba000
RBP: 0000000080000000 R08: 0000000000000001 R09: fffffbfff23e25dd
R10: ffffffff91f12eef R11: 0000000000000003 R12: 0000000000000000
R13: 0000000000000000 R14: ffff88806fc7cc80 R15: ffff88806af7d550
FS: 0000000000000000(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000

CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 0000562c2ba01600 CR3: 000000000cd77000 CR4: 00000000003506f0

DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<NMI>
</NMI>
<TASK>

__phys_addr+0x18/0x140 arch/x86/mm/physaddr.c:17
virt_to_folio include/linux/mm.h:1281 [inline]
kfree+0x45/0x150 mm/slab_common.c:1048
ieee80211_rx_mgmt_probe_beacon net/mac80211/ibss.c:1578 [inline]
ieee80211_ibss_rx_queued_mgmt+0x1b00/0x3120 net/mac80211/ibss.c:1604
ieee80211_iface_process_skb net/mac80211/iface.c:1589 [inline]
ieee80211_iface_work+0xa67/0xda0 net/mac80211/iface.c:1643
cfg80211_wiphy_work+0x24e/0x330 net/wireless/core.c:437
process_one_work+0x886/0x15d0 kernel/workqueue.c:2627
process_scheduled_works kernel/workqueue.c:2700 [inline]
worker_thread+0x8b9/0x1290 kernel/workqueue.c:2781
kthread+0x2c6/0x3a0 kernel/kthread.c:388

ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242
</TASK>


Tested on:

commit: c8e97fc6 Merge tag 'arm64-fixes' of git://git.kernel.o..
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master
console output: https://syzkaller.appspot.com/x/log.txt?x=132d7556e80000
kernel config: https://syzkaller.appspot.com/x/.config?x=48e9d2b9b4b93f29

dashboard link: https://syzkaller.appspot.com/bug?extid=4da851837827326a7cd4
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40

Note: no patches were applied.

[Edward Adam Davis]

please test task hung in bdev_release

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git 8c9660f65153

diff --git a/block/bdev.c b/block/bdev.c
index 6f73b02d549c..59a3a23ed281 100644
--- a/block/bdev.c
+++ b/block/bdev.c
@@ -645,6 +645,7 @@ static int blkdev_get_whole(struct block_device *bdev, blk_mode_t mode)
int ret;

if (disk->fops->open) {
+ printk(" b: %p, disk: %p, %s\n", bdev, disk, __func__);
ret = disk->fops->open(disk, mode);
if (ret) {
/* avoid ghost partitions on a removed medium */
@@ -660,6 +661,7 @@ static int blkdev_get_whole(struct block_device *bdev, blk_mode_t mode)
if (test_bit(GD_NEED_PART_SCAN, &disk->state))
bdev_disk_changed(disk, false);
atomic_inc(&bdev->bd_openers);
+ printk("out, b: %p, disk: %p, %s\n", bdev, disk, __func__);
return 0;
}

@@ -854,24 +856,31 @@ struct bdev_handle *bdev_open_by_dev(dev_t dev, blk_mode_t mode, void *holder,

disk_block_events(disk);

+ printk("next om, b: %p, disk: %p, %s\n", bdev, disk, __func__);
mutex_lock(&disk->open_mutex);
ret = -ENXIO;
if (!disk_live(disk))

goto abort_claiming;
+ printk("in1, b: %p, disk: %p, %s\n", bdev, disk, __func__);
if (!try_module_get(disk->fops->owner))
goto abort_claiming;
+ printk("in2, b: %p, disk: %p, %s\n", bdev, disk, __func__);
ret = -EBUSY;
if (!bdev_may_open(bdev, mode))
goto abort_claiming;
+ printk("in3, b: %p, disk: %p, %s\n", bdev, disk, __func__);
if (bdev_is_partition(bdev))
ret = blkdev_get_part(bdev, mode);
else
ret = blkdev_get_whole(bdev, mode);
+ printk("in4, b: %p, disk: %p, %s\n", bdev, disk, __func__);
if (ret)
goto put_module;
bdev_claim_write_access(bdev, mode);
+ printk("in5, b: %p, disk: %p, %s\n", bdev, disk, __func__);
if (holder) {
bd_finish_claiming(bdev, holder, hops);
+ printk("in6, b: %p, disk: %p, %s\n", bdev, disk, __func__);

/*
* Block event polling for write claims if requested. Any write
@@ -887,6 +896,7 @@ struct bdev_handle *bdev_open_by_dev(dev_t dev, blk_mode_t mode, void *holder,

}
}
mutex_unlock(&disk->open_mutex);
+ printk("out om, b: %p, disk: %p, %s\n", bdev, disk, __func__);

if (unblock_events)
disk_unblock_events(disk);

@@ -900,6 +910,7 @@ struct bdev_handle *bdev_open_by_dev(dev_t dev, blk_mode_t mode, void *holder,

if (holder)
bd_abort_claiming(bdev, holder);
mutex_unlock(&disk->open_mutex);
+ printk("out om, b: %p, %s\n", bdev, __func__);
disk_unblock_events(disk);
put_blkdev:
blkdev_put_no_open(bdev);

@@ -964,6 +975,7 @@ void bdev_release(struct bdev_handle *handle)

if (atomic_read(&bdev->bd_openers) == 1)
sync_blockdev(bdev);

+ printk("nxt om, b: %p, dk: %p, %s\n", bdev, disk, __func__);
mutex_lock(&disk->open_mutex);
bdev_yield_write_access(bdev, handle->mode);

@@ -982,6 +994,7 @@ void bdev_release(struct bdev_handle *handle)

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:

INFO: task hung in bdev_release

INFO: task syz-executor.0:5863 blocked for more than 143 seconds.
Not tainted 6.7.0-rc2-next-20231124-syzkaller-dirty #0

"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

task:syz-executor.0 state:D stack:28560 pid:5863 tgid:5860 ppid:5422 flags:0x00004006
Call Trace:
<TASK>

context_switch kernel/sched/core.c:5399 [inline]
__schedule+0xf15/0x5c00 kernel/sched/core.c:6726
__schedule_loop kernel/sched/core.c:6801 [inline]
schedule+0xe7/0x270 kernel/sched/core.c:6816
schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:6873
__mutex_lock_common kernel/locking/mutex.c:679 [inline]
__mutex_lock+0x5b4/0x9c0 kernel/locking/mutex.c:747

bdev_release+0xe6/0xac0 block/bdev.c:979

blkdev_release+0x37/0x50 block/fops.c:616
__fput+0x270/0xbb0 fs/file_table.c:394
task_work_run+0x14c/0x240 kernel/task_work.c:180
get_signal+0x105a/0x2770 kernel/signal.c:2669

arch_do_signal_or_restart+0x90/0x7e0 arch/x86/kernel/signal.c:309
exit_to_user_mode_loop kernel/entry/common.c:168 [inline]
exit_to_user_mode_prepare+0x11e/0x240 kernel/entry/common.c:204

__syscall_exit_to_user_mode_work kernel/entry/common.c:285 [inline]
syscall_exit_to_user_mode+0x1e/0x60 kernel/entry/common.c:296

do_syscall_64+0x4d/0x110 arch/x86/entry/common.c:88
entry_SYSCALL_64_after_hwframe+0x62/0x6a
RIP: 0033:0x7fa439e7cae9
RSP: 002b:00007fa43ab210c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: 0000000000000000 RBX: 00007fa439f9c120 RCX: 00007fa439e7cae9

RDX: 0000000000000000 RSI: 000000000000ab03 RDI: 0000000000000005

RBP: 00007fa439ec847a R08: 0000000000000000 R09: 0000000000000000

R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000

R13: 000000000000006e R14: 00007fa439f9c120 R15: 00007ffcbfcf7228

</TASK>

Showing all locks held in the system:
1 lock held by khungtaskd/29:

#0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:301 [inline]
#0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:747 [inline]
#0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x75/0x340 kernel/locking/lockdep.c:6613
1 lock held by klogd/4501:
#0: ffff8880b983c718 (&rq->__lock){-.-.}-{2:2}, at: raw_spin_rq_lock_nested+0x29/0x130 kernel/sched/core.c:558

2 locks held by getty/4814:

#0: ffff88802b0ea0a0 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x24/0x80 drivers/tty/tty_ldisc.c:243
#1: ffffc90002f062f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0xfc4/0x1490 drivers/tty/n_tty.c:2201
1 lock held by udevd/5406:
#0: ffff888143f624c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_open_by_dev+0x2a4/0xf80 block/bdev.c:860
1 lock held by syz-executor.0/5863:
#0: ffff888143f624c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_release+0xe6/0xac0 block/bdev.c:979
1 lock held by syz-executor.0/5879:
#0: ffff888143f624c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_open_by_dev+0x2a4/0xf80 block/bdev.c:860
1 lock held by syz-executor.0/5901:
#0: ffff888143f624c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_open_by_dev+0x2a4/0xf80 block/bdev.c:860

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 29 Comm: khungtaskd Not tainted 6.7.0-rc2-next-20231124-syzkaller-dirty #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xd9/0x1b0 lib/dump_stack.c:106
nmi_cpu_backtrace+0x277/0x390 lib/nmi_backtrace.c:113
nmi_trigger_cpumask_backtrace+0x299/0x300 lib/nmi_backtrace.c:62
trigger_all_cpu_backtrace include/linux/nmi.h:160 [inline]
check_hung_uninterruptible_tasks kernel/hung_task.c:222 [inline]

watchdog+0xf86/0x1210 kernel/hung_task.c:379
kthread+0x2c1/0x3a0 kernel/kthread.c:389

ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242
</TASK>

Sending NMI from CPU 0 to CPUs 1:

NMI backtrace for cpu 1

CPU: 1 PID: 2803 Comm: kworker/u4:8 Not tainted 6.7.0-rc2-next-20231124-syzkaller-dirty #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023
Workqueue: events_unbound cfg80211_wiphy_work

RIP: 0010:__sanitizer_cov_trace_pc+0x18/0x60 kernel/kcov.c:203
Code: ff ff 31 c0 c3 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 f3 0f 1e fa 65 48 8b 15 a4 aa 7b 7e 65 8b 05 a5 aa 7b 7e a9 00 01 ff 00 <48> 8b 34 24 74 0f f6 c4 01 74 35 8b 82 fc 15 00 00 85 c0 74 2b 8b
RSP: 0018:ffffc9000aedf6b8 EFLAGS: 00000246
RAX: 0000000080000000 RBX: 0000000000000006 RCX: ffffffff8a06b051
RDX: ffff888027ccd940 RSI: 00000000000000f4 RDI: 0000000000000001
RBP: 0000000000000000 R08: 0000000000000001 R09: 00000000000000f4
R10: 0000000000000000 R11: 0000000000000003 R12: ffff88804fa52d74
R13: ffff88804fa52d75 R14: 0000000000000000 R15: dffffc0000000000
FS: 0000000000000000(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000

CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 0000555556ea5938 CR3: 000000001c3ba000 CR4: 00000000003506f0

DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<NMI>
</NMI>
<TASK>

_ieee802_11_parse_elems_full+0x77d/0x3b70 net/mac80211/util.c:1094
ieee802_11_parse_elems_full+0x7f5/0x13a0 net/mac80211/util.c:1647
ieee802_11_parse_elems_crc net/mac80211/ieee80211_i.h:2288 [inline]
ieee802_11_parse_elems net/mac80211/ieee80211_i.h:2295 [inline]
ieee80211_rx_mgmt_probe_beacon net/mac80211/ibss.c:1573 [inline]
ieee80211_ibss_rx_queued_mgmt+0xd41/0x3140 net/mac80211/ibss.c:1604

ieee80211_iface_process_skb net/mac80211/iface.c:1589 [inline]
ieee80211_iface_work+0xa67/0xda0 net/mac80211/iface.c:1643

cfg80211_wiphy_work+0x24e/0x330 net/wireless/core.c:435

process_one_work+0x8a4/0x15f0 kernel/workqueue.c:2633
process_scheduled_works kernel/workqueue.c:2706 [inline]
worker_thread+0x8b6/0x1290 kernel/workqueue.c:2787
kthread+0x2c1/0x3a0 kernel/kthread.c:389

ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242
</TASK>


Tested on:

commit: 8c9660f6 Add linux-next specific files for 20231124
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git

console output: https://syzkaller.appspot.com/x/log.txt?x=151fd5b2e80000
kernel config: https://syzkaller.appspot.com/x/.config?x=ca1e8655505e280

dashboard link: https://syzkaller.appspot.com/bug?extid=4da851837827326a7cd4
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=14a70b56e80000

[Edward Adam Davis]

please test task hung in bdev_release

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git 8c9660f65153

diff --git a/block/bdev.c b/block/bdev.c
index 6f73b02d549c..06ce1a73b4d0 100644
--- a/block/bdev.c
+++ b/block/bdev.c
@@ -646,11 +646,13 @@ static int blkdev_get_whole(struct block_device *bdev, blk_mode_t mode)

if (disk->fops->open) {

ret = disk->fops->open(disk, mode);

+ printk("r: %d, b: %p, disk: %p, %s\n", ret, bdev, disk, __func__);

if (ret) {
/* avoid ghost partitions on a removed medium */

if (ret == -ENOMEDIUM &&
test_bit(GD_NEED_PART_SCAN, &disk->state))
bdev_disk_changed(disk, true);
+ printk("r: %d, b: %p, disk: %p, %s\n", ret, bdev, disk, __func__);
return ret;
}
}
@@ -660,6 +662,7 @@ static int blkdev_get_whole(struct block_device *bdev, blk_mode_t mode)

if (test_bit(GD_NEED_PART_SCAN, &disk->state))
bdev_disk_changed(disk, false);
atomic_inc(&bdev->bd_openers);
+ printk("out, b: %p, disk: %p, %s\n", bdev, disk, __func__);
return 0;
}

@@ -863,10 +866,12 @@ struct bdev_handle *bdev_open_by_dev(dev_t dev, blk_mode_t mode, void *holder,

ret = -EBUSY;
if (!bdev_may_open(bdev, mode))
goto abort_claiming;

+ printk("in1, b: %p, disk: %p, h: %p, %s\n", bdev, disk, holder, __func__);

if (bdev_is_partition(bdev))
ret = blkdev_get_part(bdev, mode);
else
ret = blkdev_get_whole(bdev, mode);

+ printk("in2, b: %p, disk: %p, %s\n", bdev, disk, __func__);

if (ret)
goto put_module;
bdev_claim_write_access(bdev, mode);

@@ -887,6 +892,7 @@ struct bdev_handle *bdev_open_by_dev(dev_t dev, blk_mode_t mode, void *holder,

}
}
mutex_unlock(&disk->open_mutex);
+ printk("out om, b: %p, disk: %p, %s\n", bdev, disk, __func__);

if (unblock_events)
disk_unblock_events(disk);

@@ -900,6 +906,7 @@ struct bdev_handle *bdev_open_by_dev(dev_t dev, blk_mode_t mode, void *holder,

if (holder)
bd_abort_claiming(bdev, holder);
mutex_unlock(&disk->open_mutex);
+ printk("out om, b: %p, %s\n", bdev, __func__);
disk_unblock_events(disk);
put_blkdev:
blkdev_put_no_open(bdev);

@@ -964,6 +971,7 @@ void bdev_release(struct bdev_handle *handle)

if (atomic_read(&bdev->bd_openers) == 1)
sync_blockdev(bdev);

+ printk("nxt om, b: %p, dk: %p, %s\n", bdev, disk, __func__);
mutex_lock(&disk->open_mutex);
bdev_yield_write_access(bdev, handle->mode);

@@ -982,6 +990,7 @@ void bdev_release(struct bdev_handle *handle)

else
blkdev_put_whole(bdev);
mutex_unlock(&disk->open_mutex);
+ printk("out om, b: %p, dk: %p, %s\n", bdev, disk, __func__);

module_put(disk->fops->owner);
blkdev_put_no_open(bdev);

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index b6414e1e645b..090cdef5899d 100644

@@ -1596,6 +1601,7 @@ static int nbd_open(struct gendisk *disk, blk_mode_t mode)
struct nbd_config *config;
int ret = 0;

+ printk("d: %p, %s\n", disk, __func__);
mutex_lock(&nbd_index_mutex);
nbd = disk->private_data;
if (!nbd) {
@@ -1629,6 +1635,7 @@ static int nbd_open(struct gendisk *disk, blk_mode_t mode)
set_bit(GD_NEED_PART_SCAN, &disk->state);
}
out:
+ printk("ret: %d, out, d: %p, %s\n", ret, disk, __func__);
mutex_unlock(&nbd_index_mutex);
return ret;
}

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
INFO: task hung in bdev_release

INFO: task syz-executor.0:5590 blocked for more than 143 seconds.

Not tainted 6.7.0-rc2-next-20231124-syzkaller-dirty #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

task:syz-executor.0 state:D stack:27232 pid:5590 tgid:5590 ppid:5422 flags:0x00004006

Call Trace:
<TASK>
context_switch kernel/sched/core.c:5399 [inline]
__schedule+0xf15/0x5c00 kernel/sched/core.c:6726
__schedule_loop kernel/sched/core.c:6801 [inline]
schedule+0xe7/0x270 kernel/sched/core.c:6816
schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:6873
__mutex_lock_common kernel/locking/mutex.c:679 [inline]
__mutex_lock+0x5b4/0x9c0 kernel/locking/mutex.c:747

bdev_release+0xe6/0xac0 block/bdev.c:975

blkdev_release+0x37/0x50 block/fops.c:616
__fput+0x270/0xbb0 fs/file_table.c:394

__fput_sync+0x47/0x50 fs/file_table.c:475
__do_sys_close fs/open.c:1591 [inline]
__se_sys_close fs/open.c:1576 [inline]
__x64_sys_close+0x86/0xf0 fs/open.c:1576
do_syscall_x64 arch/x86/entry/common.c:51 [inline]
do_syscall_64+0x40/0x110 arch/x86/entry/common.c:82
entry_SYSCALL_64_after_hwframe+0x62/0x6a
RIP: 0033:0x7f59a8c7b9da
RSP: 002b:00007ffc5134e390 EFLAGS: 00000293 ORIG_RAX: 0000000000000003
RAX: ffffffffffffffda RBX: 0000000000000006 RCX: 00007f59a8c7b9da
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000005
RBP: 00007f59a8d9d980 R08: 0000001b2e960000 R09: 0000000000000001
R10: 00007ffc513ba080 R11: 0000000000000293 R12: 000000000001db88
R13: ffffffffffffffff R14: 00007f59a8800000 R15: 000000000001d847

</TASK>

Showing all locks held in the system:

4 locks held by kworker/1:1/27:

#0: ffff8880b993c718 (&rq->__lock){-.-.}-{2:2}, at: raw_spin_rq_lock_nested+0x29/0x130 kernel/sched/core.c:558

#1: ffff8880b9928888 (&per_cpu_ptr(group->pcpu, cpu)->seq){-.-.}-{0:0}, at: psi_task_switch+0x2d9/0x900 kernel/sched/psi.c:988
#2: ffff8880b992a898 (&base->lock){-.-.}-{2:2}, at: lock_timer_base+0x5d/0x200 kernel/time/timer.c:999
#3: ffffffff929aeca0 (&obj_hash[i].lock){-.-.}-{2:2}, at: debug_object_activate+0x195/0x540 lib/debugobjects.c:708

1 lock held by khungtaskd/29:
#0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:301 [inline]
#0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:747 [inline]
#0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x75/0x340 kernel/locking/lockdep.c:6613

2 locks held by getty/4819:
#0: ffff8880285880a0 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x24/0x80 drivers/tty/tty_ldisc.c:243
#1: ffffc900015b72f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0xfc4/0x1490 drivers/tty/n_tty.c:2201
1 lock held by udevd/5416:
#0: ffff888141f694c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_open_by_dev+0x278/0xfa0 block/bdev.c:860
1 lock held by syz-executor.0/5590:
#0: ffff888141f694c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_release+0xe6/0xac0 block/bdev.c:975
1 lock held by syz-executor.0/5824:
#0: ffff888141f694c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_open_by_dev+0x278/0xfa0 block/bdev.c:860
1 lock held by syz-executor.0/5841:
#0: ffff888141f694c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_open_by_dev+0x278/0xfa0 block/bdev.c:860

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 29 Comm: khungtaskd Not tainted 6.7.0-rc2-next-20231124-syzkaller-dirty #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xd9/0x1b0 lib/dump_stack.c:106
nmi_cpu_backtrace+0x277/0x390 lib/nmi_backtrace.c:113
nmi_trigger_cpumask_backtrace+0x299/0x300 lib/nmi_backtrace.c:62
trigger_all_cpu_backtrace include/linux/nmi.h:160 [inline]
check_hung_uninterruptible_tasks kernel/hung_task.c:222 [inline]
watchdog+0xf86/0x1210 kernel/hung_task.c:379
kthread+0x2c1/0x3a0 kernel/kthread.c:389
ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242
</TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1

CPU: 1 PID: 11 Comm: kworker/u4:0 Not tainted 6.7.0-rc2-next-20231124-syzkaller-dirty #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023

Workqueue: bat_events batadv_nc_worker
RIP: 0010:separate_irq_context kernel/locking/lockdep.c:4627 [inline]
RIP: 0010:__lock_acquire+0xa6e/0x3b10 kernel/locking/lockdep.c:5120
Code: 04 02 84 c0 74 08 3c 03 0f 8e 71 2e 00 00 48 8b 04 24 8b 80 b8 0a 00 00 85 c0 74 75 48 8b 74 24 28 48 8d 04 80 48 8d 6c c6 d8 <48> b8 00 00 00 00 00 fc ff df 48 8d 7d 21 48 89 fa 48 c1 ea 03 0f
RSP: 0018:ffffc90000107968 EFLAGS: 00000002
RAX: 000000000000000a RBX: 19143be48a2d0d7b RCX: ffffffff81683d4c
RDX: 1ffff110029588c7 RSI: ffff888014ac4640 RDI: ffffffff91f24f40
RBP: ffff888014ac4668 R08: 0000000000000000 R09: fffffbfff23e49e8
R10: ffffffff91f24f47 R11: 0000000000000002 R12: ffffed10029588c7
R13: ffff888014ac4640 R14: ffff888014ac4690 R15: 0000000000000004

FS: 0000000000000000(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 0000564996f05680 CR3: 00000000263fa000 CR4: 00000000003506f0

DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<NMI>
</NMI>
<TASK>

lock_acquire kernel/locking/lockdep.c:5753 [inline]
lock_acquire+0x1b1/0x530 kernel/locking/lockdep.c:5718
rcu_lock_acquire include/linux/rcupdate.h:301 [inline]
rcu_read_lock include/linux/rcupdate.h:747 [inline]
batadv_nc_purge_orig_hash net/batman-adv/network-coding.c:408 [inline]
batadv_nc_worker+0x16e/0x10e0 net/batman-adv/network-coding.c:719

process_one_work+0x8a4/0x15f0 kernel/workqueue.c:2633
process_scheduled_works kernel/workqueue.c:2706 [inline]
worker_thread+0x8b6/0x1290 kernel/workqueue.c:2787
kthread+0x2c1/0x3a0 kernel/kthread.c:389
ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242
</TASK>


Tested on:

commit: 8c9660f6 Add linux-next specific files for 20231124
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git

console output: https://syzkaller.appspot.com/x/log.txt?x=14446c06e80000

kernel config: https://syzkaller.appspot.com/x/.config?x=ca1e8655505e280
dashboard link: https://syzkaller.appspot.com/bug?extid=4da851837827326a7cd4
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=1245bcd1e80000

[Edward Adam Davis]

please test task hung in bdev_release

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git 8c9660f65153

diff --git a/block/bdev.c b/block/bdev.c

index 6f73b02d549c..05abc096518f 100644
--- a/block/bdev.c
+++ b/block/bdev.c
@@ -887,6 +887,7 @@ struct bdev_handle *bdev_open_by_dev(dev_t dev, blk_mode_t mode, void *holder,

}
}
mutex_unlock(&disk->open_mutex);
+ printk("out om, b: %p, disk: %p, %s\n", bdev, disk, __func__);

if (unblock_events)
disk_unblock_events(disk);

@@ -900,6 +901,7 @@ struct bdev_handle *bdev_open_by_dev(dev_t dev, blk_mode_t mode, void *holder,

if (holder)
bd_abort_claiming(bdev, holder);
mutex_unlock(&disk->open_mutex);
+ printk("out om, b: %p, %s\n", bdev, __func__);
disk_unblock_events(disk);
put_blkdev:
blkdev_put_no_open(bdev);

@@ -964,6 +966,7 @@ void bdev_release(struct bdev_handle *handle)

if (atomic_read(&bdev->bd_openers) == 1)
sync_blockdev(bdev);

+ printk("nxt om, b: %p, dk: %p, %s\n", bdev, disk, __func__);
mutex_lock(&disk->open_mutex);
bdev_yield_write_access(bdev, handle->mode);

@@ -982,6 +985,7 @@ void bdev_release(struct bdev_handle *handle)

else
blkdev_put_whole(bdev);
mutex_unlock(&disk->open_mutex);
+ printk("out om, b: %p, dk: %p, %s\n", bdev, disk, __func__);

module_put(disk->fops->owner);
blkdev_put_no_open(bdev);

diff --git a/block/partitions/core.c b/block/partitions/core.c
index f47ffcfdfcec..e48c26513f4d 100644
--- a/block/partitions/core.c
+++ b/block/partitions/core.c
@@ -698,6 +698,7 @@ int bdev_disk_changed(struct gendisk *disk, bool invalidate)

if (get_capacity(disk)) {
ret = blk_add_partitions(disk);
+ printk("r: %d, disk: %p, %s\n", ret, disk, __func__);
if (ret == -EAGAIN)
goto rescan;
} else if (invalidate) {
@@ -708,6 +709,7 @@ int bdev_disk_changed(struct gendisk *disk, bool invalidate)
kobject_uevent(&disk_to_dev(disk)->kobj, KOBJ_CHANGE);
}

+ printk("disk: %p, %s\n", disk, __func__);
return ret;
}
/*

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
INFO: task hung in bdev_release

INFO: task syz-executor.0:5482 blocked for more than 143 seconds.

Not tainted 6.7.0-rc2-next-20231124-syzkaller-dirty #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

task:syz-executor.0 state:D stack:27872 pid:5482 tgid:5481 ppid:5420 flags:0x00004006

Call Trace:
<TASK>
context_switch kernel/sched/core.c:5399 [inline]
__schedule+0xf15/0x5c00 kernel/sched/core.c:6726
__schedule_loop kernel/sched/core.c:6801 [inline]
schedule+0xe7/0x270 kernel/sched/core.c:6816
schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:6873
__mutex_lock_common kernel/locking/mutex.c:679 [inline]
__mutex_lock+0x5b4/0x9c0 kernel/locking/mutex.c:747

bdev_release+0xe6/0xac0 block/bdev.c:970

blkdev_release+0x37/0x50 block/fops.c:616
__fput+0x270/0xbb0 fs/file_table.c:394

task_work_run+0x14c/0x240 kernel/task_work.c:180
get_signal+0x105a/0x2770 kernel/signal.c:2669
arch_do_signal_or_restart+0x90/0x7e0 arch/x86/kernel/signal.c:309
exit_to_user_mode_loop kernel/entry/common.c:168 [inline]
exit_to_user_mode_prepare+0x11e/0x240 kernel/entry/common.c:204
__syscall_exit_to_user_mode_work kernel/entry/common.c:285 [inline]
syscall_exit_to_user_mode+0x1e/0x60 kernel/entry/common.c:296
do_syscall_64+0x4d/0x110 arch/x86/entry/common.c:88
entry_SYSCALL_64_after_hwframe+0x62/0x6a

RIP: 0033:0x7f175807cae9
RSP: 002b:00007f1758e0c0c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: 0000000000000000 RBX: 00007f175819bf80 RCX: 00007f175807cae9

RDX: 0000000000000000 RSI: 000000000000ab03 RDI: 0000000000000005

RBP: 00007f17580c847a R08: 0000000000000000 R09: 0000000000000000

R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000

R13: 000000000000000b R14: 00007f175819bf80 R15: 00007fff4338e6d8

</TASK>

Showing all locks held in the system:

1 lock held by khungtaskd/29:
#0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:301 [inline]
#0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:747 [inline]
#0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x75/0x340 kernel/locking/lockdep.c:6613

5 locks held by kworker/u4:5/135:
1 lock held by klogd/4502:

2 locks held by getty/4814:

#0: ffff888026fee0a0 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x24/0x80 drivers/tty/tty_ldisc.c:243
#1: ffffc90002f062f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0xfc4/0x1490 drivers/tty/n_tty.c:2201
1 lock held by udevd/5408:
#0: ffff88801d6e54c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_open_by_dev+0x27c/0xf20 block/bdev.c:857

1 lock held by syz-executor.0/5482:

#0: ffff88801d6e54c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_release+0xe6/0xac0 block/bdev.c:970
1 lock held by syz-executor.0/5794:
#0: ffff88801d6e54c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_open_by_dev+0x27c/0xf20 block/bdev.c:857
1 lock held by syz-executor.0/5810:
#0: ffff88801d6e54c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_open_by_dev+0x27c/0xf20 block/bdev.c:857

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 29 Comm: khungtaskd Not tainted 6.7.0-rc2-next-20231124-syzkaller-dirty #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xd9/0x1b0 lib/dump_stack.c:106
nmi_cpu_backtrace+0x277/0x390 lib/nmi_backtrace.c:113
nmi_trigger_cpumask_backtrace+0x299/0x300 lib/nmi_backtrace.c:62
trigger_all_cpu_backtrace include/linux/nmi.h:160 [inline]
check_hung_uninterruptible_tasks kernel/hung_task.c:222 [inline]
watchdog+0xf86/0x1210 kernel/hung_task.c:379
kthread+0x2c1/0x3a0 kernel/kthread.c:389
ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242
</TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1

CPU: 1 PID: 34 Comm: kworker/u4:2 Not tainted 6.7.0-rc2-next-20231124-syzkaller-dirty #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023

Workqueue: events_unbound cfg80211_wiphy_work
RIP: 0010:check_wait_context kernel/locking/lockdep.c:4704 [inline]
RIP: 0010:__lock_acquire+0x1063/0x3b10 kernel/locking/lockdep.c:5086
Code: 08 84 d2 0f 85 4b 28 00 00 44 8b 25 6b 2d b1 0d 45 85 e4 0f 84 f6 17 00 00 90 e9 a9 fe ff ff 41 bf 02 00 00 00 e9 30 f5 ff ff <31> db e9 d1 f4 ff ff 8b 4c 24 08 49 8d 5e 22 85 c9 0f 85 53 f8 ff
RSP: 0018:ffffc90000aaf200 EFLAGS: 00000046
RAX: 0000000000000001 RBX: 00000000ffffffff RCX: 0000000000000001
RDX: 0000000000000000 RSI: 1ffff11002a5ec7f RDI: ffff8880152f5940
RBP: dffffc0000000000 R08: 0000000000000004 R09: fffffbfff23e49e8
R10: ffffffff91f24f47 R11: 0000000000000004 R12: ffff8880152f63f9
R13: ffff8880152f6400 R14: ffff8880152f64a0 R15: 0000000000000000

FS: 0000000000000000(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 00007ffe3c726fa0 CR3: 000000000cd78000 CR4: 00000000003506f0

DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<NMI>
</NMI>
<TASK>
lock_acquire kernel/locking/lockdep.c:5753 [inline]
lock_acquire+0x1b1/0x530 kernel/locking/lockdep.c:5718
rcu_lock_acquire include/linux/rcupdate.h:301 [inline]
rcu_read_lock include/linux/rcupdate.h:747 [inline]

pfn_valid include/linux/mmzone.h:2028 [inline]
__virt_addr_valid+0x199/0x580 arch/x86/mm/physaddr.c:65
kasan_addr_to_slab+0xd/0x80 mm/kasan/common.c:36
__kasan_record_aux_stack+0xe/0xd0 mm/kasan/generic.c:492
kvfree_call_rcu+0x70/0xbe0 kernel/rcu/tree.c:3400
cfg80211_update_known_bss+0x802/0xa60 net/wireless/scan.c:1783
__cfg80211_bss_update+0x1ba/0x24b0 net/wireless/scan.c:1827
cfg80211_inform_single_bss_frame_data+0x771/0xf40 net/wireless/scan.c:2905
cfg80211_inform_bss_frame_data+0xbf/0x290 net/wireless/scan.c:2936
ieee80211_bss_info_update+0x300/0x8f0 net/mac80211/scan.c:205
ieee80211_rx_bss_info net/mac80211/ibss.c:1098 [inline]
ieee80211_rx_mgmt_probe_beacon net/mac80211/ibss.c:1577 [inline]
ieee80211_ibss_rx_queued_mgmt+0x198a/0x3140 net/mac80211/ibss.c:1604

ieee80211_iface_process_skb net/mac80211/iface.c:1589 [inline]
ieee80211_iface_work+0xa67/0xda0 net/mac80211/iface.c:1643
cfg80211_wiphy_work+0x24e/0x330 net/wireless/core.c:435

process_one_work+0x8a4/0x15f0 kernel/workqueue.c:2633
process_scheduled_works kernel/workqueue.c:2706 [inline]
worker_thread+0x8b6/0x1290 kernel/workqueue.c:2787
kthread+0x2c1/0x3a0 kernel/kthread.c:389
ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242
</TASK>


Tested on:

commit: 8c9660f6 Add linux-next specific files for 20231124
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git

console output: https://syzkaller.appspot.com/x/log.txt?x=16e7da1ee80000

kernel config: https://syzkaller.appspot.com/x/.config?x=ca1e8655505e280
dashboard link: https://syzkaller.appspot.com/bug?extid=4da851837827326a7cd4
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=10f501e1e80000

[Edward Adam Davis]

please test task hung in bdev_release

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git 8c9660f65153

diff --git a/block/bdev.c b/block/bdev.c

index 6f73b02d549c..9fdf2dbc450e 100644
--- a/block/bdev.c
+++ b/block/bdev.c
@@ -130,12 +130,14 @@ static void set_init_blocksize(struct block_device *bdev)
unsigned int bsize = bdev_logical_block_size(bdev);
loff_t size = i_size_read(bdev->bd_inode);

+ printk("s: %llu, %s\n", size, __func__);
while (bsize < PAGE_SIZE) {
if (size & bsize)
break;
bsize <<= 1;
}
bdev->bd_inode->i_blkbits = blksize_bits(bsize);
+ printk("out s: %llu, %s\n", size, __func__);
}

int set_blocksize(struct block_device *bdev, int size)
@@ -870,6 +872,7 @@ struct bdev_handle *bdev_open_by_dev(dev_t dev, blk_mode_t mode, void *holder,

if (ret)
goto put_module;
bdev_claim_write_access(bdev, mode);

+ printk("%p, h: %p, %s\n", bdev, holder, __func__);

if (holder) {
bd_finish_claiming(bdev, holder, hops);

@@ -887,6 +890,7 @@ struct bdev_handle *bdev_open_by_dev(dev_t dev, blk_mode_t mode, void *holder,

}
}
mutex_unlock(&disk->open_mutex);
+ printk("out om, b: %p, disk: %p, %s\n", bdev, disk, __func__);

if (unblock_events)
disk_unblock_events(disk);

@@ -900,6 +904,7 @@ struct bdev_handle *bdev_open_by_dev(dev_t dev, blk_mode_t mode, void *holder,

if (holder)
bd_abort_claiming(bdev, holder);
mutex_unlock(&disk->open_mutex);
+ printk("out om, b: %p, %s\n", bdev, __func__);
disk_unblock_events(disk);
put_blkdev:
blkdev_put_no_open(bdev);

@@ -964,6 +969,7 @@ void bdev_release(struct bdev_handle *handle)

if (atomic_read(&bdev->bd_openers) == 1)
sync_blockdev(bdev);

+ printk("nxt om, b: %p, dk: %p, %s\n", bdev, disk, __func__);
mutex_lock(&disk->open_mutex);
bdev_yield_write_access(bdev, handle->mode);

@@ -982,6 +988,7 @@ void bdev_release(struct bdev_handle *handle)

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
INFO: task hung in bdev_release

INFO: task syz-executor.0:5519 blocked for more than 143 seconds.

Not tainted 6.7.0-rc2-next-20231124-syzkaller-dirty #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

task:syz-executor.0 state:D stack:28208 pid:5519 tgid:5519 ppid:5421 flags:0x00004006

Call Trace:
<TASK>
context_switch kernel/sched/core.c:5399 [inline]
__schedule+0xf15/0x5c00 kernel/sched/core.c:6726
__schedule_loop kernel/sched/core.c:6801 [inline]
schedule+0xe7/0x270 kernel/sched/core.c:6816
schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:6873
__mutex_lock_common kernel/locking/mutex.c:679 [inline]
__mutex_lock+0x5b4/0x9c0 kernel/locking/mutex.c:747

bdev_release+0xe6/0xac0 block/bdev.c:973

blkdev_release+0x37/0x50 block/fops.c:616
__fput+0x270/0xbb0 fs/file_table.c:394

__fput_sync+0x47/0x50 fs/file_table.c:475
__do_sys_close fs/open.c:1591 [inline]
__se_sys_close fs/open.c:1576 [inline]
__x64_sys_close+0x86/0xf0 fs/open.c:1576
do_syscall_x64 arch/x86/entry/common.c:51 [inline]
do_syscall_64+0x40/0x110 arch/x86/entry/common.c:82
entry_SYSCALL_64_after_hwframe+0x62/0x6a

RIP: 0033:0x7fc4db47b9da
RSP: 002b:00007fff600ff100 EFLAGS: 00000293 ORIG_RAX: 0000000000000003
RAX: ffffffffffffffda RBX: 0000000000000006 RCX: 00007fc4db47b9da

RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000005

RBP: 00007fc4db59d980 R08: 0000001b2e860000 R09: 0000000000000001
R10: 00007fff601ac080 R11: 0000000000000293 R12: 000000000001c90d
R13: ffffffffffffffff R14: 00007fc4db000000 R15: 000000000001c5cc

</TASK>

Showing all locks held in the system:
1 lock held by khungtaskd/29:
#0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:301 [inline]
#0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:747 [inline]
#0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x75/0x340 kernel/locking/lockdep.c:6613

2 locks held by getty/4821:
#0: ffff8880272c20a0 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x24/0x80 drivers/tty/tty_ldisc.c:243

#1: ffffc90002f062f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0xfc4/0x1490 drivers/tty/n_tty.c:2201

1 lock held by udevd/5406:
#0: ffff88801da974c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_open_by_dev+0x27c/0xed0 block/bdev.c:859
1 lock held by syz-executor.0/5519:
#0: ffff88801da974c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_release+0xe6/0xac0 block/bdev.c:973

1 lock held by syz-executor.0/5806:

#0: ffff88801da974c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_open_by_dev+0x27c/0xed0 block/bdev.c:859
1 lock held by syz-executor.0/5823:
#0: ffff88801da974c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_open_by_dev+0x27c/0xed0 block/bdev.c:859

=============================================

NMI backtrace for cpu 1
CPU: 1 PID: 29 Comm: khungtaskd Not tainted 6.7.0-rc2-next-20231124-syzkaller-dirty #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xd9/0x1b0 lib/dump_stack.c:106
nmi_cpu_backtrace+0x277/0x390 lib/nmi_backtrace.c:113
nmi_trigger_cpumask_backtrace+0x299/0x300 lib/nmi_backtrace.c:62
trigger_all_cpu_backtrace include/linux/nmi.h:160 [inline]
check_hung_uninterruptible_tasks kernel/hung_task.c:222 [inline]
watchdog+0xf86/0x1210 kernel/hung_task.c:379
kthread+0x2c1/0x3a0 kernel/kthread.c:389
ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242
</TASK>

Sending NMI from CPU 1 to CPUs 0:

NMI backtrace for cpu 0

CPU: 0 PID: 48 Comm: kworker/u4:3 Not tainted 6.7.0-rc2-next-20231124-syzkaller-dirty #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023
Workqueue: events_unbound cfg80211_wiphy_work

RIP: 0010:find_held_lock+0x9/0x110 kernel/locking/lockdep.c:5236
Code: 0d a5 77 00 e9 18 ff ff ff e8 03 a5 77 00 4c 8d 44 24 20 eb 95 66 66 2e 0f 1f 84 00 00 00 00 00 90 41 57 44 8d 7a ff 49 63 c7 <41> 56 49 89 ce 48 8d 04 80 41 55 49 89 f5 41 54 55 89 d5 53 48 8d
RSP: 0018:ffffc90000b8eff8 EFLAGS: 00000006
RAX: 0000000000000004 RBX: 0000000000000003 RCX: ffffc90000b8f058
RDX: 0000000000000005 RSI: ffff8880b9840860 RDI: ffff888019edbb80
RBP: 1ffff92000171e03 R08: 0000000000000000 R09: fffffbfff1e32732
R10: ffffffff8f193997 R11: 0000000000000004 R12: ffff8880b9840860
R13: 0000000000000005 R14: ffff888019edc638 R15: 0000000000000004
FS: 0000000000000000(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000

CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 00007ffd39e31e4c CR3: 000000000cd78000 CR4: 00000000003506f0

DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<NMI>
</NMI>
<TASK>

__lock_release kernel/locking/lockdep.c:5428 [inline]
lock_release+0x1fd/0x6a0 kernel/locking/lockdep.c:5773
local_lock_release include/linux/local_lock_internal.h:38 [inline]
___slab_alloc+0x98b/0x1700 mm/slub.c:3139
__slab_alloc.constprop.0+0x56/0xa0 mm/slub.c:3297
__slab_alloc_node mm/slub.c:3350 [inline]
slab_alloc_node mm/slub.c:3442 [inline]
__kmem_cache_alloc_node+0x131/0x310 mm/slub.c:3491
__do_kmalloc_node mm/slab_common.c:1006 [inline]
__kmalloc+0x49/0x90 mm/slab_common.c:1020
kmalloc include/linux/slab.h:604 [inline]
kzalloc include/linux/slab.h:721 [inline]
ieee802_11_parse_elems_full+0xee/0x13a0 net/mac80211/util.c:1628

ieee802_11_parse_elems_crc net/mac80211/ieee80211_i.h:2288 [inline]
ieee802_11_parse_elems net/mac80211/ieee80211_i.h:2295 [inline]

ieee80211_inform_bss+0x120/0x1150 net/mac80211/scan.c:79
rdev_inform_bss+0xf9/0x440 net/wireless/rdev-ops.h:418
cfg80211_inform_single_bss_frame_data+0x7b7/0xf40 net/wireless/scan.c:2909

cfg80211_inform_bss_frame_data+0xbf/0x290 net/wireless/scan.c:2936
ieee80211_bss_info_update+0x300/0x8f0 net/mac80211/scan.c:205
ieee80211_rx_bss_info net/mac80211/ibss.c:1098 [inline]
ieee80211_rx_mgmt_probe_beacon net/mac80211/ibss.c:1577 [inline]
ieee80211_ibss_rx_queued_mgmt+0x198a/0x3140 net/mac80211/ibss.c:1604
ieee80211_iface_process_skb net/mac80211/iface.c:1589 [inline]
ieee80211_iface_work+0xa67/0xda0 net/mac80211/iface.c:1643
cfg80211_wiphy_work+0x24e/0x330 net/wireless/core.c:435
process_one_work+0x8a4/0x15f0 kernel/workqueue.c:2633
process_scheduled_works kernel/workqueue.c:2706 [inline]
worker_thread+0x8b6/0x1290 kernel/workqueue.c:2787
kthread+0x2c1/0x3a0 kernel/kthread.c:389
ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242
</TASK>


Tested on:

commit: 8c9660f6 Add linux-next specific files for 20231124
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git

console output: https://syzkaller.appspot.com/x/log.txt?x=13446c06e80000

kernel config: https://syzkaller.appspot.com/x/.config?x=ca1e8655505e280
dashboard link: https://syzkaller.appspot.com/bug?extid=4da851837827326a7cd4
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=165e748ae80000

[Edward Adam Davis]

please test task hung in bdev_release

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git 8c9660f65153

diff --git a/block/bdev.c b/block/bdev.c

index 6f73b02d549c..95be32bc8dea 100644
--- a/block/bdev.c
+++ b/block/bdev.c
@@ -938,6 +938,7 @@ struct bdev_handle *bdev_open_by_path(const char *path, blk_mode_t mode,
if (error)
return ERR_PTR(error);

+ printk("dev: %u, path: %s, holder: %p, %s\n", dev, path, holder, __func__);
handle = bdev_open_by_dev(dev, mode, holder, hops);
if (!IS_ERR(handle) && (mode & BLK_OPEN_WRITE) &&
bdev_read_only(handle->bdev)) {
@@ -964,6 +965,7 @@ void bdev_release(struct bdev_handle *handle)

if (atomic_read(&bdev->bd_openers) == 1)
sync_blockdev(bdev);

+ printk("nxt om, b: %p, dk: %p, %s\n", bdev, disk, __func__);
mutex_lock(&disk->open_mutex);
bdev_yield_write_access(bdev, handle->mode);

@@ -982,6 +984,7 @@ void bdev_release(struct bdev_handle *handle)

else
blkdev_put_whole(bdev);
mutex_unlock(&disk->open_mutex);
+ printk("out om, b: %p, dk: %p, %s\n", bdev, disk, __func__);

module_put(disk->fops->owner);
blkdev_put_no_open(bdev);

@@ -1092,7 +1095,8 @@ void sync_bdevs(bool wait)
old_inode = inode;
bdev = I_BDEV(inode);

- mutex_lock(&bdev->bd_disk->open_mutex);
+ printk("in, b: %p, dk: %p, %s\n", bdev, bdev->bd_disk, __func__);
+ mutex_lock(&bdev->bd_disk->sync_mutex);
if (!atomic_read(&bdev->bd_openers)) {
; /* skip */
} else if (wait) {
@@ -1106,7 +1110,8 @@ void sync_bdevs(bool wait)
} else {
filemap_fdatawrite(inode->i_mapping);
}
- mutex_unlock(&bdev->bd_disk->open_mutex);
+ mutex_unlock(&bdev->bd_disk->sync_mutex);
+ printk("out, b: %p, dk: %p, %s\n", bdev, bdev->bd_disk, __func__);

spin_lock(&blockdev_superblock->s_inode_list_lock);
}
diff --git a/block/genhd.c b/block/genhd.c
index 13db3a7943d8..da44d4739915 100644
--- a/block/genhd.c
+++ b/block/genhd.c
@@ -1355,6 +1355,7 @@ struct gendisk *__alloc_disk_node(struct request_queue *q, int node_id,

disk->node_id = node_id;
mutex_init(&disk->open_mutex);
+ mutex_init(&disk->sync_mutex);
xa_init(&disk->part_tbl);
if (xa_insert(&disk->part_tbl, 0, disk->part0, GFP_KERNEL))
goto out_destroy_part_tbl;
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index b6414e1e645b..c302df0caebf 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -475,10 +475,11 @@ static enum blk_eh_timer_return nbd_xmit_timeout(struct request *req)
*/
struct nbd_sock *nsock = config->socks[cmd->index];
cmd->retries++;
- dev_info(nbd_to_dev(nbd), "Possible stuck request %p: control (%s@%llu,%uB). Runtime %u seconds\n",
+ dev_info(nbd_to_dev(nbd), "Possible stuck request %p: control (%s@%llu,%uB). Runtime %u seconds, nbd: %p, d: %p\n",
req, nbdcmd_to_ascii(req_to_nbd_cmd_type(req)),
(unsigned long long)blk_rq_pos(req) << 9,
- blk_rq_bytes(req), (req->timeout / HZ) * cmd->retries);
+ blk_rq_bytes(req), (req->timeout / HZ) * cmd->retries,
+ nbd, nbd->disk);

mutex_lock(&nsock->tx_lock);
if (cmd->cookie != nsock->cookie) {
@@ -1425,6 +1426,7 @@ static int nbd_start_device(struct nbd_device *nbd)

args->index = i;
queue_work(nbd->recv_workq, &args->work);
}
+ printk("bs: %lld, blks: %lld, %s\n", config->bytesize, nbd_blksize(config), __func__);
return nbd_set_size(nbd, config->bytesize, nbd_blksize(config));
}

diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 3f8a21cd9233..31c6a7d6a74c 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -161,6 +161,7 @@ struct gendisk {
#define GD_OWNS_QUEUE 6

struct mutex open_mutex; /* open/close mutex */
+ struct mutex sync_mutex; /* sync mutex */
unsigned open_partitions; /* number of open partitions */

struct backing_dev_info *bdi;

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
INFO: task hung in bdev_release

INFO: task syz-executor.0:5520 blocked for more than 143 seconds.

Not tainted 6.7.0-rc2-next-20231124-syzkaller-dirty #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

task:syz-executor.0 state:D stack:28464 pid:5520 tgid:5519 ppid:5421 flags:0x00004006

Call Trace:
<TASK>
context_switch kernel/sched/core.c:5399 [inline]
__schedule+0xf15/0x5c00 kernel/sched/core.c:6726
__schedule_loop kernel/sched/core.c:6801 [inline]
schedule+0xe7/0x270 kernel/sched/core.c:6816
schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:6873
__mutex_lock_common kernel/locking/mutex.c:679 [inline]
__mutex_lock+0x5b4/0x9c0 kernel/locking/mutex.c:747

bdev_release+0xe6/0xac0 block/bdev.c:969

blkdev_release+0x37/0x50 block/fops.c:616
__fput+0x270/0xbb0 fs/file_table.c:394

task_work_run+0x14c/0x240 kernel/task_work.c:180
get_signal+0x105a/0x2770 kernel/signal.c:2669
arch_do_signal_or_restart+0x90/0x7e0 arch/x86/kernel/signal.c:309
exit_to_user_mode_loop kernel/entry/common.c:168 [inline]
exit_to_user_mode_prepare+0x11e/0x240 kernel/entry/common.c:204
__syscall_exit_to_user_mode_work kernel/entry/common.c:285 [inline]
syscall_exit_to_user_mode+0x1e/0x60 kernel/entry/common.c:296
do_syscall_64+0x4d/0x110 arch/x86/entry/common.c:88
entry_SYSCALL_64_after_hwframe+0x62/0x6a

RIP: 0033:0x7f13f867cae9
RSP: 002b:00007f13f79fe0c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: 0000000000000000 RBX: 00007f13f879bf80 RCX: 00007f13f867cae9

RDX: 0000000000000000 RSI: 000000000000ab03 RDI: 0000000000000005

RBP: 00007f13f86c847a R08: 0000000000000000 R09: 0000000000000000

R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000

R13: 000000000000000b R14: 00007f13f879bf80 R15: 00007fffee00fa68

</TASK>

Showing all locks held in the system:
1 lock held by khungtaskd/29:
#0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:301 [inline]
#0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:747 [inline]
#0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x75/0x340 kernel/locking/lockdep.c:6613

2 locks held by getty/4815:
#0: ffff88802b51a0a0 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x24/0x80 drivers/tty/tty_ldisc.c:243
#1: ffffc900015c72f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0xfc4/0x1490 drivers/tty/n_tty.c:2201
1 lock held by udevd/5416:
#0: ffff888143f0e4c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_open_by_dev+0x27c/0xed0 block/bdev.c:857
1 lock held by syz-executor.0/5520:
#0: ffff888143f0e4c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_release+0xe6/0xac0 block/bdev.c:969
1 lock held by syz-executor.0/5795:
#0: ffff888143f0e4c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_open_by_dev+0x27c/0xed0 block/bdev.c:857
1 lock held by syz-executor.0/5813:
#0: ffff888143f0e4c8 (&disk->open_mutex){+.+.}-{3:3}, at: bdev_open_by_dev+0x27c/0xed0 block/bdev.c:857

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 29 Comm: khungtaskd Not tainted 6.7.0-rc2-next-20231124-syzkaller-dirty #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xd9/0x1b0 lib/dump_stack.c:106
nmi_cpu_backtrace+0x277/0x390 lib/nmi_backtrace.c:113
nmi_trigger_cpumask_backtrace+0x299/0x300 lib/nmi_backtrace.c:62
trigger_all_cpu_backtrace include/linux/nmi.h:160 [inline]
check_hung_uninterruptible_tasks kernel/hung_task.c:222 [inline]
watchdog+0xf86/0x1210 kernel/hung_task.c:379
kthread+0x2c1/0x3a0 kernel/kthread.c:389
ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242
</TASK>

Sending NMI from CPU 0 to CPUs 1:

NMI backtrace for cpu 1

CPU: 1 PID: 76 Comm: kworker/u4:5 Not tainted 6.7.0-rc2-next-20231124-syzkaller-dirty #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023

Workqueue: bat_events batadv_nc_worker
RIP: 0010:lockdep_enabled kernel/locking/lockdep.c:116 [inline]
RIP: 0010:lock_release+0xb7/0x6a0 kernel/locking/lockdep.c:5766
Code: 00 89 db be 08 00 00 00 48 89 d8 48 c1 e8 06 48 8d 3c c5 90 36 19 8f e8 77 bb 76 00 48 0f a3 1d 6f c7 b0 0d 0f 82 0c 04 00 00 <48> c7 c3 58 6a 19 8f 48 b8 00 00 00 00 00 fc ff df 48 89 da 48 c1
RSP: 0018:ffffc90001597ae0 EFLAGS: 00000202
RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffffffff81686f19
RDX: 0000000000000000 RSI: ffffffff8b2f2100 RDI: ffffffff8ca75a68
RBP: 1ffff920002b2f5e R08: 0000000000000000 R09: fffffbfff1e326d2
R10: ffffffff8f193697 R11: 0000000000000002 R12: ffffffff8cfacf60
R13: 0000000000000000 R14: 000000000003bccc R15: 0000000000000001
FS: 0000000000000000(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000

CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 00005622f5959680 CR3: 000000000cd78000 CR4: 00000000003506f0

DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<NMI>
</NMI>
<TASK>

rcu_lock_release include/linux/rcupdate.h:306 [inline]
rcu_read_unlock include/linux/rcupdate.h:780 [inline]
batadv_nc_purge_orig_hash net/batman-adv/network-coding.c:412 [inline]
batadv_nc_worker+0x8f3/0x10e0 net/batman-adv/network-coding.c:719

process_one_work+0x8a4/0x15f0 kernel/workqueue.c:2633
process_scheduled_works kernel/workqueue.c:2706 [inline]
worker_thread+0x8b6/0x1290 kernel/workqueue.c:2787
kthread+0x2c1/0x3a0 kernel/kthread.c:389
ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242
</TASK>


Tested on:

commit: 8c9660f6 Add linux-next specific files for 20231124
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git

console output: https://syzkaller.appspot.com/x/log.txt?x=119557c1e80000

kernel config: https://syzkaller.appspot.com/x/.config?x=ca1e8655505e280
dashboard link: https://syzkaller.appspot.com/bug?extid=4da851837827326a7cd4
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=10731cd1e80000

[Edward Adam Davis]

please test task hung in bdev_release

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git 8c9660f65153

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index b6414e1e645b..3a00ae9b4867 100644

--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -475,10 +475,11 @@ static enum blk_eh_timer_return nbd_xmit_timeout(struct request *req)
*/
struct nbd_sock *nsock = config->socks[cmd->index];
cmd->retries++;
- dev_info(nbd_to_dev(nbd), "Possible stuck request %p: control (%s@%llu,%uB). Runtime %u seconds\n",
+ dev_info(nbd_to_dev(nbd), "Possible stuck request %p: control (%s@%llu,%uB). Runtime %u seconds, nbd: %p, d: %p\n",
req, nbdcmd_to_ascii(req_to_nbd_cmd_type(req)),
(unsigned long long)blk_rq_pos(req) << 9,
- blk_rq_bytes(req), (req->timeout / HZ) * cmd->retries);
+ blk_rq_bytes(req), (req->timeout / HZ) * cmd->retries,
+ nbd, nbd->disk);

mutex_lock(&nsock->tx_lock);
if (cmd->cookie != nsock->cookie) {

@@ -733,7 +734,7 @@ static int nbd_read_reply(struct nbd_device *nbd, struct socket *sock,
if (result < 0) {
if (!nbd_disconnected(nbd->config))
dev_err(disk_to_dev(nbd->disk),
- "Receive control failed (result %d)\n", result);
+ "Receive control failed (result %d), nbd: %p, d: %p\n", result, nbd, nbd->disk);
return result;
}

@@ -1394,6 +1395,9 @@ static int nbd_start_device(struct nbd_device *nbd)
}
set_bit(NBD_RT_HAS_PID_FILE, &config->runtime_flags);

+ if (num_connections == 1 && !nbd->tag_set.timeout)
+ nbd->tag_set.timeout = HZ * 1024;
+
nbd_dev_dbg_init(nbd);
for (i = 0; i < num_connections; i++) {
struct recv_thread_args *args;
@@ -1424,6 +1428,7 @@ static int nbd_start_device(struct nbd_device *nbd)
args->nsock = config->socks[i];

args->index = i;
queue_work(nbd->recv_workq, &args->work);

+ printk("%p, %p, bs: %lld, blks: %lld, c: %d, %s\n", nbd, nbd->disk, config->bytesize, nbd_blksize(config), num_connections, __func__);

[syzbot]

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-and-tested-by: syzbot+4da851...@syzkaller.appspotmail.com

Tested on:

commit: 8c9660f6 Add linux-next specific files for 20231124
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git

console output: https://syzkaller.appspot.com/x/log.txt?x=1636278ee80000

kernel config: https://syzkaller.appspot.com/x/.config?x=ca1e8655505e280
dashboard link: https://syzkaller.appspot.com/bug?extid=4da851837827326a7cd4
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=132ec6e1e80000

Note: testing is done by a robot and is best-effort only.

[Edward Adam Davis]

If the nbd timeout value is not set before calling nbd_start_device_ioctl(),
so the socket will be disabled disconnection, which will cause the timer to
only be reset and cause this 143 second timeout issue.

The solution I have provided here is to set the default timeout value for nbd in
nbd_start_device() to avoid problems from occurring.

Reported-and-tested-by: syzbot+4da851...@syzkaller.appspotmail.com
Signed-off-by: Edward Adam Davis <ead...@qq.com>
---
drivers/block/nbd.c | 3 +++
1 file changed, 3 insertions(+)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index b6414e1e645b..31656364d8a3 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -1394,6 +1394,9 @@ static int nbd_start_device(struct nbd_device *nbd)

}
set_bit(NBD_RT_HAS_PID_FILE, &config->runtime_flags);

+ if (num_connections == 1 && !nbd->tag_set.timeout)
+ nbd->tag_set.timeout = HZ * 1024;
+
nbd_dev_dbg_init(nbd);
for (i = 0; i < num_connections; i++) {
struct recv_thread_args *args;

--
2.43.0