[v5.15] possible deadlock in rds_message_put
0 views
Skip to first unread message
syzbot
May 15, 2023, 3:00:54 AM5/15/23
to syzkaller...@googlegroups.com
Hello,
syzbot found the following issue on:
HEAD commit: b0ece631f84a Linux 5.15.111
git tree: linux-5.15.y
console output: https://syzkaller.appspot.com/x/log.txt?x=129201dc280000
kernel config: https://syzkaller.appspot.com/x/.config?x=db3750b003ff805e
dashboard link: https://syzkaller.appspot.com/bug?extid=4f1f1d5a96ae0676cf91
compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2
userspace arch: arm64
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=13651562280000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=1107a5ca280000
Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/eded75b6c3f9/disk-b0ece631.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/20e5ae802010/vmlinux-b0ece631.xz
kernel image: https://storage.googleapis.com/syzbot-assets/05d665c869f3/Image-b0ece631.gz.xz
IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+4f1f1d...@syzkaller.appspotmail.com
======================================================
WARNING: possible circular locking dependency detected
5.15.111-syzkaller #0 Not tainted
------------------------------------------------------
syz-executor298/4087 is trying to acquire lock:
ffff0000d021d900 (&rm->m_rs_lock){....}-{2:2}, at: rds_message_purge net/rds/message.c:138 [inline]
ffff0000d021d900 (&rm->m_rs_lock){....}-{2:2}, at: rds_message_put+0x138/0xa0c net/rds/message.c:180
but task is already holding lock:
ffff0000cdc9cbf0 (&rs->rs_recv_lock){....}-{2:2}, at: rds_clear_recv_queue+0x40/0x23c net/rds/recv.c:761
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #1 (&rs->rs_recv_lock){....}-{2:2}:
__raw_read_lock_irqsave include/linux/rwlock_api_smp.h:159 [inline]
_raw_read_lock_irqsave+0x138/0x1e8 kernel/locking/spinlock.c:236
rds_wake_sk_sleep+0x34/0xc8 net/rds/af_rds.c:109
rds_send_remove_from_sock+0x1a4/0x7e0 net/rds/send.c:634
rds_send_path_drop_acked+0x394/0x3f4 net/rds/send.c:710
rds_tcp_write_space+0x1a8/0x5a4 net/rds/tcp_send.c:198
tcp_new_space net/ipv4/tcp_input.c:5443 [inline]
tcp_check_space+0x128/0x774 net/ipv4/tcp_input.c:5462
tcp_data_snd_check net/ipv4/tcp_input.c:5471 [inline]
tcp_rcv_established+0xac4/0x1f40 net/ipv4/tcp_input.c:5966
tcp_v4_do_rcv+0x340/0xc70 net/ipv4/tcp_ipv4.c:1726
sk_backlog_rcv include/net/sock.h:1057 [inline]
__release_sock+0x1a8/0x408 net/core/sock.c:2690
release_sock+0x68/0x270 net/core/sock.c:3231
tcp_sock_set_cork+0x100/0x188 net/ipv4/tcp.c:3232
rds_tcp_xmit_path_complete+0x7c/0x8c net/rds/tcp_send.c:52
rds_send_xmit+0x19c0/0x238c net/rds/send.c:422
rds_sendmsg+0x160c/0x1bcc net/rds/send.c:1382
sock_sendmsg_nosec net/socket.c:704 [inline]
sock_sendmsg net/socket.c:724 [inline]
____sys_sendmsg+0x584/0x870 net/socket.c:2412
___sys_sendmsg+0x214/0x294 net/socket.c:2466
__sys_sendmsg net/socket.c:2495 [inline]
__do_sys_sendmsg net/socket.c:2504 [inline]
__se_sys_sendmsg net/socket.c:2502 [inline]
__arm64_sys_sendmsg+0x1ac/0x25c net/socket.c:2502
__invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:52
el0_svc_common+0x138/0x258 arch/arm64/kernel/syscall.c:142
do_el0_svc+0x58/0x14c arch/arm64/kernel/syscall.c:181
el0_svc+0x7c/0x1f0 arch/arm64/kernel/entry-common.c:596
el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:614
el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584
-> #0 (&rm->m_rs_lock){....}-{2:2}:
check_prev_add kernel/locking/lockdep.c:3053 [inline]
check_prevs_add kernel/locking/lockdep.c:3172 [inline]
validate_chain kernel/locking/lockdep.c:3787 [inline]
__lock_acquire+0x32cc/0x7620 kernel/locking/lockdep.c:5011
lock_acquire+0x240/0x77c kernel/locking/lockdep.c:5622
__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0xc4/0x14c kernel/locking/spinlock.c:162
rds_message_purge net/rds/message.c:138 [inline]
rds_message_put+0x138/0xa0c net/rds/message.c:180
rds_loop_inc_free+0x20/0x30 net/rds/loop.c:115
rds_inc_put+0x100/0x198 net/rds/recv.c:82
rds_clear_recv_queue+0x1b0/0x23c net/rds/recv.c:767
rds_release+0xbc/0x2e4 net/rds/af_rds.c:73
__sock_release net/socket.c:649 [inline]
sock_close+0xb8/0x1fc net/socket.c:1317
__fput+0x30c/0x7f0 fs/file_table.c:280
____fput+0x20/0x30 fs/file_table.c:308
task_work_run+0x130/0x1e4 kernel/task_work.c:164
exit_task_work include/linux/task_work.h:32 [inline]
do_exit+0x688/0x2134 kernel/exit.c:872
do_group_exit+0x110/0x268 kernel/exit.c:994
get_signal+0x634/0x1550 kernel/signal.c:2889
do_signal arch/arm64/kernel/signal.c:890 [inline]
do_notify_resume+0x3d0/0x32b8 arch/arm64/kernel/signal.c:943
prepare_exit_to_user_mode arch/arm64/kernel/entry-common.c:133 [inline]
exit_to_user_mode arch/arm64/kernel/entry-common.c:138 [inline]
el0_svc+0xfc/0x1f0 arch/arm64/kernel/entry-common.c:597
el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:614
el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584
other info that might help us debug this:
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(&rs->rs_recv_lock);
lock(&rm->m_rs_lock);
lock(&rs->rs_recv_lock);
lock(&rm->m_rs_lock);
*** DEADLOCK ***
2 locks held by syz-executor298/4087:
#0: ffff0000e033d610 (&sb->s_type->i_mutex_key#11){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:787 [inline]
#0: ffff0000e033d610 (&sb->s_type->i_mutex_key#11){+.+.}-{3:3}, at: __sock_release net/socket.c:648 [inline]
#0: ffff0000e033d610 (&sb->s_type->i_mutex_key#11){+.+.}-{3:3}, at: sock_close+0x80/0x1fc net/socket.c:1317
#1: ffff0000cdc9cbf0 (&rs->rs_recv_lock){....}-{2:2}, at: rds_clear_recv_queue+0x40/0x23c net/rds/recv.c:761
stack backtrace:
CPU: 1 PID: 4087 Comm: syz-executor298 Not tainted 5.15.111-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/14/2023
Call trace:
dump_backtrace+0x0/0x530 arch/arm64/kernel/stacktrace.c:152
show_stack+0x2c/0x3c arch/arm64/kernel/stacktrace.c:216
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x108/0x170 lib/dump_stack.c:106
dump_stack+0x1c/0x58 lib/dump_stack.c:113
print_circular_bug+0x150/0x1b8 kernel/locking/lockdep.c:2011
check_noncircular+0x2cc/0x378 kernel/locking/lockdep.c:2133
check_prev_add kernel/locking/lockdep.c:3053 [inline]
check_prevs_add kernel/locking/lockdep.c:3172 [inline]
validate_chain kernel/locking/lockdep.c:3787 [inline]
__lock_acquire+0x32cc/0x7620 kernel/locking/lockdep.c:5011
lock_acquire+0x240/0x77c kernel/locking/lockdep.c:5622
__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0xc4/0x14c kernel/locking/spinlock.c:162
rds_message_purge net/rds/message.c:138 [inline]
rds_message_put+0x138/0xa0c net/rds/message.c:180
rds_loop_inc_free+0x20/0x30 net/rds/loop.c:115
rds_inc_put+0x100/0x198 net/rds/recv.c:82
rds_clear_recv_queue+0x1b0/0x23c net/rds/recv.c:767
rds_release+0xbc/0x2e4 net/rds/af_rds.c:73
__sock_release net/socket.c:649 [inline]
sock_close+0xb8/0x1fc net/socket.c:1317
__fput+0x30c/0x7f0 fs/file_table.c:280
____fput+0x20/0x30 fs/file_table.c:308
task_work_run+0x130/0x1e4 kernel/task_work.c:164
exit_task_work include/linux/task_work.h:32 [inline]
do_exit+0x688/0x2134 kernel/exit.c:872
do_group_exit+0x110/0x268 kernel/exit.c:994
get_signal+0x634/0x1550 kernel/signal.c:2889
do_signal arch/arm64/kernel/signal.c:890 [inline]
do_notify_resume+0x3d0/0x32b8 arch/arm64/kernel/signal.c:943
prepare_exit_to_user_mode arch/arm64/kernel/entry-common.c:133 [inline]
exit_to_user_mode arch/arm64/kernel/entry-common.c:138 [inline]
el0_svc+0xfc/0x1f0 arch/arm64/kernel/entry-common.c:597
el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:614
el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584
---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.
syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
If the bug is already fixed, let syzbot know by replying with:
#syz fix: exact-commit-title
If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.
If you want to change bug's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)
If the bug is a duplicate of another bug, reply with:
#syz dup: exact-subject-of-another-report
If you want to undo deduplication, reply with:
#syz undup
syzbot found the following issue on:
HEAD commit: b0ece631f84a Linux 5.15.111
git tree: linux-5.15.y
console output: https://syzkaller.appspot.com/x/log.txt?x=129201dc280000
kernel config: https://syzkaller.appspot.com/x/.config?x=db3750b003ff805e
dashboard link: https://syzkaller.appspot.com/bug?extid=4f1f1d5a96ae0676cf91
compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2
userspace arch: arm64
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=13651562280000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=1107a5ca280000
Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/eded75b6c3f9/disk-b0ece631.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/20e5ae802010/vmlinux-b0ece631.xz
kernel image: https://storage.googleapis.com/syzbot-assets/05d665c869f3/Image-b0ece631.gz.xz
IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+4f1f1d...@syzkaller.appspotmail.com
======================================================
WARNING: possible circular locking dependency detected
5.15.111-syzkaller #0 Not tainted
------------------------------------------------------
syz-executor298/4087 is trying to acquire lock:
ffff0000d021d900 (&rm->m_rs_lock){....}-{2:2}, at: rds_message_purge net/rds/message.c:138 [inline]
ffff0000d021d900 (&rm->m_rs_lock){....}-{2:2}, at: rds_message_put+0x138/0xa0c net/rds/message.c:180
but task is already holding lock:
ffff0000cdc9cbf0 (&rs->rs_recv_lock){....}-{2:2}, at: rds_clear_recv_queue+0x40/0x23c net/rds/recv.c:761
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #1 (&rs->rs_recv_lock){....}-{2:2}:
__raw_read_lock_irqsave include/linux/rwlock_api_smp.h:159 [inline]
_raw_read_lock_irqsave+0x138/0x1e8 kernel/locking/spinlock.c:236
rds_wake_sk_sleep+0x34/0xc8 net/rds/af_rds.c:109
rds_send_remove_from_sock+0x1a4/0x7e0 net/rds/send.c:634
rds_send_path_drop_acked+0x394/0x3f4 net/rds/send.c:710
rds_tcp_write_space+0x1a8/0x5a4 net/rds/tcp_send.c:198
tcp_new_space net/ipv4/tcp_input.c:5443 [inline]
tcp_check_space+0x128/0x774 net/ipv4/tcp_input.c:5462
tcp_data_snd_check net/ipv4/tcp_input.c:5471 [inline]
tcp_rcv_established+0xac4/0x1f40 net/ipv4/tcp_input.c:5966
tcp_v4_do_rcv+0x340/0xc70 net/ipv4/tcp_ipv4.c:1726
sk_backlog_rcv include/net/sock.h:1057 [inline]
__release_sock+0x1a8/0x408 net/core/sock.c:2690
release_sock+0x68/0x270 net/core/sock.c:3231
tcp_sock_set_cork+0x100/0x188 net/ipv4/tcp.c:3232
rds_tcp_xmit_path_complete+0x7c/0x8c net/rds/tcp_send.c:52
rds_send_xmit+0x19c0/0x238c net/rds/send.c:422
rds_sendmsg+0x160c/0x1bcc net/rds/send.c:1382
sock_sendmsg_nosec net/socket.c:704 [inline]
sock_sendmsg net/socket.c:724 [inline]
____sys_sendmsg+0x584/0x870 net/socket.c:2412
___sys_sendmsg+0x214/0x294 net/socket.c:2466
__sys_sendmsg net/socket.c:2495 [inline]
__do_sys_sendmsg net/socket.c:2504 [inline]
__se_sys_sendmsg net/socket.c:2502 [inline]
__arm64_sys_sendmsg+0x1ac/0x25c net/socket.c:2502
__invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:52
el0_svc_common+0x138/0x258 arch/arm64/kernel/syscall.c:142
do_el0_svc+0x58/0x14c arch/arm64/kernel/syscall.c:181
el0_svc+0x7c/0x1f0 arch/arm64/kernel/entry-common.c:596
el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:614
el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584
-> #0 (&rm->m_rs_lock){....}-{2:2}:
check_prev_add kernel/locking/lockdep.c:3053 [inline]
check_prevs_add kernel/locking/lockdep.c:3172 [inline]
validate_chain kernel/locking/lockdep.c:3787 [inline]
__lock_acquire+0x32cc/0x7620 kernel/locking/lockdep.c:5011
lock_acquire+0x240/0x77c kernel/locking/lockdep.c:5622
__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0xc4/0x14c kernel/locking/spinlock.c:162
rds_message_purge net/rds/message.c:138 [inline]
rds_message_put+0x138/0xa0c net/rds/message.c:180
rds_loop_inc_free+0x20/0x30 net/rds/loop.c:115
rds_inc_put+0x100/0x198 net/rds/recv.c:82
rds_clear_recv_queue+0x1b0/0x23c net/rds/recv.c:767
rds_release+0xbc/0x2e4 net/rds/af_rds.c:73
__sock_release net/socket.c:649 [inline]
sock_close+0xb8/0x1fc net/socket.c:1317
__fput+0x30c/0x7f0 fs/file_table.c:280
____fput+0x20/0x30 fs/file_table.c:308
task_work_run+0x130/0x1e4 kernel/task_work.c:164
exit_task_work include/linux/task_work.h:32 [inline]
do_exit+0x688/0x2134 kernel/exit.c:872
do_group_exit+0x110/0x268 kernel/exit.c:994
get_signal+0x634/0x1550 kernel/signal.c:2889
do_signal arch/arm64/kernel/signal.c:890 [inline]
do_notify_resume+0x3d0/0x32b8 arch/arm64/kernel/signal.c:943
prepare_exit_to_user_mode arch/arm64/kernel/entry-common.c:133 [inline]
exit_to_user_mode arch/arm64/kernel/entry-common.c:138 [inline]
el0_svc+0xfc/0x1f0 arch/arm64/kernel/entry-common.c:597
el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:614
el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584
other info that might help us debug this:
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(&rs->rs_recv_lock);
lock(&rm->m_rs_lock);
lock(&rs->rs_recv_lock);
lock(&rm->m_rs_lock);
*** DEADLOCK ***
2 locks held by syz-executor298/4087:
#0: ffff0000e033d610 (&sb->s_type->i_mutex_key#11){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:787 [inline]
#0: ffff0000e033d610 (&sb->s_type->i_mutex_key#11){+.+.}-{3:3}, at: __sock_release net/socket.c:648 [inline]
#0: ffff0000e033d610 (&sb->s_type->i_mutex_key#11){+.+.}-{3:3}, at: sock_close+0x80/0x1fc net/socket.c:1317
#1: ffff0000cdc9cbf0 (&rs->rs_recv_lock){....}-{2:2}, at: rds_clear_recv_queue+0x40/0x23c net/rds/recv.c:761
stack backtrace:
CPU: 1 PID: 4087 Comm: syz-executor298 Not tainted 5.15.111-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/14/2023
Call trace:
dump_backtrace+0x0/0x530 arch/arm64/kernel/stacktrace.c:152
show_stack+0x2c/0x3c arch/arm64/kernel/stacktrace.c:216
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x108/0x170 lib/dump_stack.c:106
dump_stack+0x1c/0x58 lib/dump_stack.c:113
print_circular_bug+0x150/0x1b8 kernel/locking/lockdep.c:2011
check_noncircular+0x2cc/0x378 kernel/locking/lockdep.c:2133
check_prev_add kernel/locking/lockdep.c:3053 [inline]
check_prevs_add kernel/locking/lockdep.c:3172 [inline]
validate_chain kernel/locking/lockdep.c:3787 [inline]
__lock_acquire+0x32cc/0x7620 kernel/locking/lockdep.c:5011
lock_acquire+0x240/0x77c kernel/locking/lockdep.c:5622
__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0xc4/0x14c kernel/locking/spinlock.c:162
rds_message_purge net/rds/message.c:138 [inline]
rds_message_put+0x138/0xa0c net/rds/message.c:180
rds_loop_inc_free+0x20/0x30 net/rds/loop.c:115
rds_inc_put+0x100/0x198 net/rds/recv.c:82
rds_clear_recv_queue+0x1b0/0x23c net/rds/recv.c:767
rds_release+0xbc/0x2e4 net/rds/af_rds.c:73
__sock_release net/socket.c:649 [inline]
sock_close+0xb8/0x1fc net/socket.c:1317
__fput+0x30c/0x7f0 fs/file_table.c:280
____fput+0x20/0x30 fs/file_table.c:308
task_work_run+0x130/0x1e4 kernel/task_work.c:164
exit_task_work include/linux/task_work.h:32 [inline]
do_exit+0x688/0x2134 kernel/exit.c:872
do_group_exit+0x110/0x268 kernel/exit.c:994
get_signal+0x634/0x1550 kernel/signal.c:2889
do_signal arch/arm64/kernel/signal.c:890 [inline]
do_notify_resume+0x3d0/0x32b8 arch/arm64/kernel/signal.c:943
prepare_exit_to_user_mode arch/arm64/kernel/entry-common.c:133 [inline]
exit_to_user_mode arch/arm64/kernel/entry-common.c:138 [inline]
el0_svc+0xfc/0x1f0 arch/arm64/kernel/entry-common.c:597
el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:614
el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584
---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.
syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
If the bug is already fixed, let syzbot know by replying with:
#syz fix: exact-commit-title
If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.
If you want to change bug's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)
If the bug is a duplicate of another bug, reply with:
#syz dup: exact-subject-of-another-report
If you want to undo deduplication, reply with:
#syz undup
Reply all
Reply to author
Forward
0 new messages