[v5.15] possible deadlock in kernfs_remove

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: 6e2fd6534337 Linux 5.15.213
git tree: linux-5.15.y
console output: https://syzkaller.appspot.com/x/log.txt?x=1178bbb9580000
kernel config: https://syzkaller.appspot.com/x/.config?x=f161cbc9aef65db0
dashboard link: https://syzkaller.appspot.com/bug?extid=b2a56f035552cb55baaf
compiler: Debian clang version 22.1.8 (++20260613092233+e80beda6e255-1~exp1~20260613092250.77), Debian LLD 22.1.8

Unfortunately, I don't have any reproducer for this issue yet.

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/d7b46606b13d/disk-6e2fd653.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/da120fdeb081/vmlinux-6e2fd653.xz
kernel image: https://storage.googleapis.com/syzbot-assets/3c2643457e3f/bzImage-6e2fd653.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+b2a56f...@syzkaller.appspotmail.com

usb 5-1: No valid video chain found.
======================================================
WARNING: possible circular locking dependency detected
syzkaller #0 Not tainted
------------------------------------------------------
kworker/0:8/4232 is trying to acquire lock:
ffff8880623babd0 (kn->active#829){++++}-{0:0}, at: kernfs_remove+0x1d/0x30 fs/kernfs/dir.c:1406

but task is already holding lock:
ffffc9000311fd00 ((work_completion)(&mddev->del_work)){+.+.}-{0:0}, at: process_one_work+0x79e/0xff0 kernel/workqueue.c:2285

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #4 ((work_completion)(&mddev->del_work)){+.+.}-{0:0}:
process_one_work+0x7ba/0xff0 kernel/workqueue.c:2286
worker_thread+0xad7/0x12a0 kernel/workqueue.c:2457
kthread+0x42e/0x520 kernel/kthread.c:334
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:287

-> #3 ((wq_completion)md_misc){+.+.}-{0:0}:
flush_workqueue+0x16a/0x1390 kernel/workqueue.c:2830
md_alloc+0x2e/0xc10 drivers/md/md.c:5720
blk_probe_dev block/genhd.c:685 [inline]
blk_request_module+0x26e/0x290 block/genhd.c:-1
blkdev_get_no_open+0x38/0x1d0 block/bdev.c:736
blkdev_get_by_dev+0x77/0xa50 block/bdev.c:803
swsusp_check+0xa1/0x2b0 kernel/power/swap.c:1526
software_resume+0xc6/0x3b0 kernel/power/hibernate.c:982
resume_store+0xe4/0x130 kernel/power/hibernate.c:1184
kernfs_fop_write_iter+0x3a8/0x500 fs/kernfs/file.c:296
call_write_iter include/linux/fs.h:2173 [inline]
new_sync_write fs/read_write.c:507 [inline]
vfs_write+0x748/0xd70 fs/read_write.c:594
ksys_write+0x153/0x260 fs/read_write.c:647
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x4c/0xa0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x66/0xd0

-> #2 (major_names_lock){+.+.}-{3:3}:
__mutex_lock_common+0x1e5/0x2400 kernel/locking/mutex.c:596
__mutex_lock kernel/locking/mutex.c:729 [inline]
mutex_lock_nested+0x17/0x20 kernel/locking/mutex.c:743
blk_probe_dev block/genhd.c:682 [inline]
blk_request_module+0x31/0x290 block/genhd.c:698
blkdev_get_no_open+0x38/0x1d0 block/bdev.c:736
blkdev_get_by_dev+0x77/0xa50 block/bdev.c:803
lock_rdev drivers/md/md.c:2576 [inline]
md_import_device+0x3e2/0x810 drivers/md/md.c:3774
md_add_new_disk+0x3fa/0x1960 drivers/md/md.c:6790
md_ioctl+0x236f/0x26f0 drivers/md/md.c:7770
blkdev_ioctl+0x347/0x6f0 block/ioctl.c:604
block_ioctl+0xac/0xf0 block/fops.c:496
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:874 [inline]
__se_sys_ioctl+0xfa/0x170 fs/ioctl.c:860
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x4c/0xa0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x66/0xd0

-> #1 (&mddev->reconfig_mutex){+.+.}-{3:3}:
__mutex_lock_common+0x1e5/0x2400 kernel/locking/mutex.c:596
__mutex_lock kernel/locking/mutex.c:729 [inline]
mutex_lock_interruptible_nested+0x17/0x20 kernel/locking/mutex.c:765
mddev_lock drivers/md/md.h:543 [inline]
serialize_policy_store+0x11d/0x2e0 drivers/md/md.c:5525
md_attr_store+0xdd/0x130 drivers/md/md.c:5636
kernfs_fop_write_iter+0x3a8/0x500 fs/kernfs/file.c:296
call_write_iter include/linux/fs.h:2173 [inline]
new_sync_write fs/read_write.c:507 [inline]
vfs_write+0x748/0xd70 fs/read_write.c:594
ksys_write+0x153/0x260 fs/read_write.c:647
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x4c/0xa0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x66/0xd0

-> #0 (kn->active#829){++++}-{0:0}:
check_prev_add kernel/locking/lockdep.c:3053 [inline]
check_prevs_add kernel/locking/lockdep.c:3172 [inline]
validate_chain kernel/locking/lockdep.c:3788 [inline]
__lock_acquire+0x2c66/0x7b50 kernel/locking/lockdep.c:5012
lock_acquire+0x19e/0x400 kernel/locking/lockdep.c:5623
kernfs_drain fs/kernfs/dir.c:478 [inline]
__kernfs_remove+0x5da/0xcb0 fs/kernfs/dir.c:1372
kernfs_remove+0x1d/0x30 fs/kernfs/dir.c:1406
__kobject_del+0xe0/0x2e0 lib/kobject.c:628
kobject_del+0x41/0x60 lib/kobject.c:651
mddev_delayed_delete+0x27/0x30 drivers/md/md.c:5678
process_one_work+0x867/0xff0 kernel/workqueue.c:2310
worker_thread+0xad7/0x12a0 kernel/workqueue.c:2457
kthread+0x42e/0x520 kernel/kthread.c:334
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:287

other info that might help us debug this:

Chain exists of:
kn->active#829 --> (wq_completion)md_misc --> (work_completion)(&mddev->del_work)

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock((work_completion)(&mddev->del_work));
lock((wq_completion)md_misc);
lock((work_completion)(&mddev->del_work));
lock(kn->active#829);

*** DEADLOCK ***

2 locks held by kworker/0:8/4232:
#0: ffff888017a6cd38 ((wq_completion)md_misc){+.+.}-{0:0}, at: process_one_work+0x75c/0xff0 kernel/workqueue.c:-1
#1: ffffc9000311fd00 ((work_completion)(&mddev->del_work)){+.+.}-{0:0}, at: process_one_work+0x79e/0xff0 kernel/workqueue.c:2285

stack backtrace:
CPU: 0 PID: 4232 Comm: kworker/0:8 Not tainted syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/24/2026
Workqueue: md_misc mddev_delayed_delete
Call Trace:
<TASK>
dump_stack_lvl+0x188/0x250 lib/dump_stack.c:106
check_noncircular+0x296/0x330 kernel/locking/lockdep.c:2133
check_prev_add kernel/locking/lockdep.c:3053 [inline]
check_prevs_add kernel/locking/lockdep.c:3172 [inline]
validate_chain kernel/locking/lockdep.c:3788 [inline]
__lock_acquire+0x2c66/0x7b50 kernel/locking/lockdep.c:5012
lock_acquire+0x19e/0x400 kernel/locking/lockdep.c:5623
kernfs_drain fs/kernfs/dir.c:478 [inline]
__kernfs_remove+0x5da/0xcb0 fs/kernfs/dir.c:1372
kernfs_remove+0x1d/0x30 fs/kernfs/dir.c:1406
__kobject_del+0xe0/0x2e0 lib/kobject.c:628
kobject_del+0x41/0x60 lib/kobject.c:651
mddev_delayed_delete+0x27/0x30 drivers/md/md.c:5678
process_one_work+0x867/0xff0 kernel/workqueue.c:2310
worker_thread+0xad7/0x12a0 kernel/workqueue.c:2457
kthread+0x42e/0x520 kernel/kthread.c:334
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:287
</TASK>
(NULL device *): Create Association LS failed: Association Allocation Failed


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup