[v5.15] possible deadlock in smc_release

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: 3a5928702e71 Linux 5.15.167
git tree: linux-5.15.y
console output: https://syzkaller.appspot.com/x/log.txt?x=11476107980000
kernel config: https://syzkaller.appspot.com/x/.config?x=171882977b524c53
dashboard link: https://syzkaller.appspot.com/bug?extid=793d678d565a6f4300f4
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
userspace arch: arm64
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=12a7e19f980000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=13922a27980000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/22ee27cb312d/disk-3a592870.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/90bf6a3e3d20/vmlinux-3a592870.xz
kernel image: https://storage.googleapis.com/syzbot-assets/096dd2c73ac3/Image-3a592870.gz.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+793d67...@syzkaller.appspotmail.com

======================================================
WARNING: possible circular locking dependency detected
5.15.167-syzkaller #0 Not tainted
------------------------------------------------------
syz-executor946/4016 is trying to acquire lock:
ffff0000ca799e90 ((work_completion)(&new_smc->smc_listen_work)){+.+.}-{0:0}, at: __flush_work+0xd0/0x1c0 kernel/workqueue.c:3090

but task is already holding lock:
ffff0000ca798120 (sk_lock-AF_SMC/1){+.+.}-{0:0}, at: smc_release+0x1f4/0x574

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (sk_lock-AF_SMC/1){+.+.}-{0:0}:
lock_sock_nested+0xec/0x1ec net/core/sock.c:3250
smc_listen_out+0x8c/0x324 net/smc/af_smc.c:1480
smc_listen_out_connected net/smc/af_smc.c:1500 [inline]
smc_listen_decline net/smc/af_smc.c:1535 [inline]
smc_listen_work+0x47c/0xe60 net/smc/af_smc.c:1988
process_one_work+0x790/0x11b8 kernel/workqueue.c:2310
worker_thread+0x910/0x1034 kernel/workqueue.c:2457
kthread+0x37c/0x45c kernel/kthread.c:334
ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:870

-> #0 ((work_completion)(&new_smc->smc_listen_work)){+.+.}-{0:0}:
check_prev_add kernel/locking/lockdep.c:3053 [inline]
check_prevs_add kernel/locking/lockdep.c:3172 [inline]
validate_chain kernel/locking/lockdep.c:3788 [inline]
__lock_acquire+0x32d4/0x7638 kernel/locking/lockdep.c:5012
lock_acquire+0x240/0x77c kernel/locking/lockdep.c:5623
__flush_work+0xf8/0x1c0 kernel/workqueue.c:3090
__cancel_work_timer+0x3ec/0x548 kernel/workqueue.c:3181
cancel_work_sync+0x24/0x38 kernel/workqueue.c:3217
smc_clcsock_release+0x64/0xec net/smc/smc_close.c:29
__smc_release+0x388/0x540 net/smc/af_smc.c:171
smc_close_non_accepted+0xdc/0x298 net/smc/af_smc.c:1426
smc_close_cleanup_listen net/smc/smc_close.c:45 [inline]
smc_close_active+0x908/0xc8c net/smc/smc_close.c:219
__smc_release+0x9c/0x540 net/smc/af_smc.c:147
smc_release+0x26c/0x574 net/smc/af_smc.c:214
__sock_release net/socket.c:649 [inline]
sock_close+0xb8/0x1fc net/socket.c:1336
__fput+0x1c4/0x800 fs/file_table.c:280
____fput+0x20/0x30 fs/file_table.c:308
task_work_run+0x130/0x1e4 kernel/task_work.c:188
exit_task_work include/linux/task_work.h:33 [inline]
do_exit+0x670/0x20bc kernel/exit.c:874
do_group_exit+0x110/0x268 kernel/exit.c:996
__do_sys_exit_group kernel/exit.c:1007 [inline]
__se_sys_exit_group kernel/exit.c:1005 [inline]
__wake_up_parent+0x0/0x60 kernel/exit.c:1005
__invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:52
el0_svc_common+0x138/0x258 arch/arm64/kernel/syscall.c:142
do_el0_svc+0x58/0x14c arch/arm64/kernel/syscall.c:181
el0_svc+0x7c/0x1f0 arch/arm64/kernel/entry-common.c:608
el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:626
el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584

other info that might help us debug this:

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock(sk_lock-AF_SMC/1);
lock((work_completion)(&new_smc->smc_listen_work));
lock(sk_lock-AF_SMC/1);
lock((work_completion)(&new_smc->smc_listen_work));

*** DEADLOCK ***

2 locks held by syz-executor946/4016:
#0: ffff0000e1c8d010 (&sb->s_type->i_mutex_key#11){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:789 [inline]
#0: ffff0000e1c8d010 (&sb->s_type->i_mutex_key#11){+.+.}-{3:3}, at: __sock_release net/socket.c:648 [inline]
#0: ffff0000e1c8d010 (&sb->s_type->i_mutex_key#11){+.+.}-{3:3}, at: sock_close+0x80/0x1fc net/socket.c:1336
#1: ffff0000ca798120 (sk_lock-AF_SMC/1){+.+.}-{0:0}, at: smc_release+0x1f4/0x574

stack backtrace:
CPU: 1 PID: 4016 Comm: syz-executor946 Not tainted 5.15.167-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/06/2024
Call trace:
dump_backtrace+0x0/0x530 arch/arm64/kernel/stacktrace.c:152
show_stack+0x2c/0x3c arch/arm64/kernel/stacktrace.c:216
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x108/0x170 lib/dump_stack.c:106
dump_stack+0x1c/0x58 lib/dump_stack.c:113
print_circular_bug+0x150/0x1b8 kernel/locking/lockdep.c:2011
check_noncircular+0x2cc/0x378 kernel/locking/lockdep.c:2133
check_prev_add kernel/locking/lockdep.c:3053 [inline]
check_prevs_add kernel/locking/lockdep.c:3172 [inline]
validate_chain kernel/locking/lockdep.c:3788 [inline]
__lock_acquire+0x32d4/0x7638 kernel/locking/lockdep.c:5012
lock_acquire+0x240/0x77c kernel/locking/lockdep.c:5623
__flush_work+0xf8/0x1c0 kernel/workqueue.c:3090
__cancel_work_timer+0x3ec/0x548 kernel/workqueue.c:3181
cancel_work_sync+0x24/0x38 kernel/workqueue.c:3217
smc_clcsock_release+0x64/0xec net/smc/smc_close.c:29
__smc_release+0x388/0x540 net/smc/af_smc.c:171
smc_close_non_accepted+0xdc/0x298 net/smc/af_smc.c:1426
smc_close_cleanup_listen net/smc/smc_close.c:45 [inline]
smc_close_active+0x908/0xc8c net/smc/smc_close.c:219
__smc_release+0x9c/0x540 net/smc/af_smc.c:147
smc_release+0x26c/0x574 net/smc/af_smc.c:214
__sock_release net/socket.c:649 [inline]
sock_close+0xb8/0x1fc net/socket.c:1336
__fput+0x1c4/0x800 fs/file_table.c:280
____fput+0x20/0x30 fs/file_table.c:308
task_work_run+0x130/0x1e4 kernel/task_work.c:188
exit_task_work include/linux/task_work.h:33 [inline]
do_exit+0x670/0x20bc kernel/exit.c:874
do_group_exit+0x110/0x268 kernel/exit.c:996
__do_sys_exit_group kernel/exit.c:1007 [inline]
__se_sys_exit_group kernel/exit.c:1005 [inline]
__wake_up_parent+0x0/0x60 kernel/exit.c:1005
__invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:52
el0_svc_common+0x138/0x258 arch/arm64/kernel/syscall.c:142
do_el0_svc+0x58/0x14c arch/arm64/kernel/syscall.c:181
el0_svc+0x7c/0x1f0 arch/arm64/kernel/entry-common.c:608
el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:626
el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup