[Kernel Bug] possible deadlock in vhci_send_frame

[Longxing Li]

Dear Linux kernel developers and maintainers,

We would like to report a new kernel bug found by our tool. possible
deadlock in vhci_send_frame. Details are as follows.

Kernel commit: v5.15.189
Kernel config: see attachment
report: see attachment
C repro and Syz repro: see attachment

We are currently analyzing the root cause. We will provide further
updates in this thread as soon as we have more information.

Best regards,
Longxing Li

======================================================
WARNING: possible circular locking dependency detected
5.15.189 #1 Not tainted
------------------------------------------------------
kworker/u3:2/7652 is trying to acquire lock:
ffff88801c5ea518 (&data->open_mutex){+.+.}-{3:3}, at:
vhci_send_frame+0xb0/0x120 drivers/bluetooth/hci_vhci.c:71

but task is already holding lock:
ffffc90001effda0 ((work_completion)(&hdev->cmd_work)){+.+.}-{0:0}, at:
process_one_work+0x8f5/0x1530 kernel/workqueue.c:2285

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #3 ((work_completion)(&hdev->cmd_work)){+.+.}-{0:0}:
__flush_work+0x12d/0xc10 kernel/workqueue.c:3090
hci_dev_do_open+0xab6/0x1b40 net/bluetooth/hci_core.c:1626
hci_power_on+0x133/0x680 net/bluetooth/hci_core.c:2265
process_one_work+0x9db/0x1530 kernel/workqueue.c:2310
worker_thread+0x686/0x1180 kernel/workqueue.c:2457
kthread+0x3d0/0x4c0 kernel/kthread.c:334
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:287

-> #2 (&hdev->req_lock){+.+.}-{3:3}:
__mutex_lock_common kernel/locking/mutex.c:596 [inline]
__mutex_lock+0x183/0x1300 kernel/locking/mutex.c:729
hci_dev_do_close+0x5d/0x1250 net/bluetooth/hci_core.c:1737
hci_rfkill_set_block+0x1be/0x210 net/bluetooth/hci_core.c:2235
rfkill_set_block+0x200/0x550 net/rfkill/core.c:345
rfkill_sync_work+0x8a/0xc0 net/rfkill/core.c:1030
process_one_work+0x9db/0x1530 kernel/workqueue.c:2310
worker_thread+0x686/0x1180 kernel/workqueue.c:2457
kthread+0x3d0/0x4c0 kernel/kthread.c:334
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:287

-> #1 (rfkill_global_mutex){+.+.}-{3:3}:
__mutex_lock_common kernel/locking/mutex.c:596 [inline]
__mutex_lock+0x183/0x1300 kernel/locking/mutex.c:729
rfkill_register+0x36/0xb00 net/rfkill/core.c:1045
hci_register_dev+0x440/0xd90 net/bluetooth/hci_core.c:3960
__vhci_create_device+0x2c8/0x5e0 drivers/bluetooth/hci_vhci.c:129
vhci_create_device drivers/bluetooth/hci_vhci.c:153 [inline]
vhci_get_user drivers/bluetooth/hci_vhci.c:210 [inline]
vhci_write+0x2c0/0x460 drivers/bluetooth/hci_vhci.c:290
call_write_iter include/linux/fs.h:2172 [inline]
new_sync_write+0x4b2/0x680 fs/read_write.c:507
vfs_write+0x7d9/0xb40 fs/read_write.c:594
ksys_write+0x13a/0x260 fs/read_write.c:647
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x34/0x80 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x66/0xd0

-> #0 (&data->open_mutex){+.+.}-{3:3}:
check_prev_add kernel/locking/lockdep.c:3053 [inline]
check_prevs_add kernel/locking/lockdep.c:3172 [inline]
validate_chain kernel/locking/lockdep.c:3788 [inline]
__lock_acquire+0x293e/0x5590 kernel/locking/lockdep.c:5012
lock_acquire kernel/locking/lockdep.c:5623 [inline]
lock_acquire+0x1a8/0x4c0 kernel/locking/lockdep.c:5588
__mutex_lock_common kernel/locking/mutex.c:596 [inline]
__mutex_lock+0x183/0x1300 kernel/locking/mutex.c:729
vhci_send_frame+0xb0/0x120 drivers/bluetooth/hci_vhci.c:71
hci_send_frame+0x1c0/0x400 net/bluetooth/hci_core.c:4256
hci_cmd_work+0x223/0x3b0 net/bluetooth/hci_core.c:5201
process_one_work+0x9db/0x1530 kernel/workqueue.c:2310
worker_thread+0x686/0x1180 kernel/workqueue.c:2457
kthread+0x3d0/0x4c0 kernel/kthread.c:334
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:287

other info that might help us debug this:

Chain exists of:
&data->open_mutex --> &hdev->req_lock --> (work_completion)(&hdev->cmd_work)

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock((work_completion)(&hdev->cmd_work));
lock(&hdev->req_lock);
lock((work_completion)(&hdev->cmd_work));
lock(&data->open_mutex);

*** DEADLOCK ***

2 locks held by kworker/u3:2/7652:
#0: ffff888057be9938 ((wq_completion)hci0#3){+.+.}-{0:0}, at:
arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
#0: ffff888057be9938 ((wq_completion)hci0#3){+.+.}-{0:0}, at:
arch_atomic_long_set include/linux/atomic/atomic-long.h:41 [inline]
#0: ffff888057be9938 ((wq_completion)hci0#3){+.+.}-{0:0}, at:
atomic_long_set include/linux/atomic/atomic-instrumented.h:1198
[inline]
#0: ffff888057be9938 ((wq_completion)hci0#3){+.+.}-{0:0}, at:
set_work_data kernel/workqueue.c:635 [inline]
#0: ffff888057be9938 ((wq_completion)hci0#3){+.+.}-{0:0}, at:
set_work_pool_and_clear_pending kernel/workqueue.c:662 [inline]
#0: ffff888057be9938 ((wq_completion)hci0#3){+.+.}-{0:0}, at:
process_one_work+0x8c1/0x1530 kernel/workqueue.c:2281
#1: ffffc90001effda0
((work_completion)(&hdev->cmd_work)){+.+.}-{0:0}, at:
process_one_work+0x8f5/0x1530 kernel/workqueue.c:2285

stack backtrace:
CPU: 0 PID: 7652 Comm: kworker/u3:2 Not tainted 5.15.189 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
Workqueue: hci0 hci_cmd_work
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xfc/0x174 lib/dump_stack.c:106
check_noncircular+0x277/0x310 kernel/locking/lockdep.c:2133
check_prev_add kernel/locking/lockdep.c:3053 [inline]
check_prevs_add kernel/locking/lockdep.c:3172 [inline]
validate_chain kernel/locking/lockdep.c:3788 [inline]
__lock_acquire+0x293e/0x5590 kernel/locking/lockdep.c:5012
lock_acquire kernel/locking/lockdep.c:5623 [inline]
lock_acquire+0x1a8/0x4c0 kernel/locking/lockdep.c:5588
__mutex_lock_common kernel/locking/mutex.c:596 [inline]
__mutex_lock+0x183/0x1300 kernel/locking/mutex.c:729
vhci_send_frame+0xb0/0x120 drivers/bluetooth/hci_vhci.c:71
hci_send_frame+0x1c0/0x400 net/bluetooth/hci_core.c:4256
hci_cmd_work+0x223/0x3b0 net/bluetooth/hci_core.c:5201
process_one_work+0x9db/0x1530 kernel/workqueue.c:2310
worker_thread+0x686/0x1180 kernel/workqueue.c:2457
kthread+0x3d0/0x4c0 kernel/kthread.c:334
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:287
</TASK>

==================================================================
https://drive.google.com/file/d/13nr1pPRHCrZqYxz08ngNiCvmcRe-d_oK/view?usp=drive_link

https://drive.google.com/file/d/11Ox0bimeaa3r8BaGqhXBBymrs03Haz8o/view?usp=drive_link

https://drive.google.com/file/d/1yA-bbVeMCxS_p7ZjDH97NK4iI_1CLeAz/view?usp=drive_link

https://drive.google.com/file/d/1XqJJtZ0jKV_oFm8M6SuQV2YAuKEqWHbn/view?usp=drive_link