[syzbot] WARNING: suspicious RCU usage in bpf_sk_reuseport_detach

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: 94ce3b64c62d net/tls: Use RCU API to access tls_ctx->netdev
git tree: net
console+strace: https://syzkaller.appspot.com/x/log.txt?x=14641e15080000
kernel config: https://syzkaller.appspot.com/x/.config?x=53da55f2bdeb0d4c
dashboard link: https://syzkaller.appspot.com/bug?extid=24bcff6e82ce253f23ec
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=106c89fd080000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=17ead885080000

The issue was bisected to:

commit f72c38fad234759fe943cb2e40bf3d0f7de1d4d9
Author: Edward Cree <ecree....@gmail.com>
Date: Wed Jul 20 18:33:48 2022 +0000

sfc: hook up ef100 representor TX

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=125bf9fd080000
final oops: https://syzkaller.appspot.com/x/report.txt?x=115bf9fd080000
console output: https://syzkaller.appspot.com/x/log.txt?x=165bf9fd080000

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+24bcff...@syzkaller.appspotmail.com
Fixes: f72c38fad234 ("sfc: hook up ef100 representor TX")

=============================
WARNING: suspicious RCU usage
5.19.0-syzkaller-05408-g94ce3b64c62d #0 Not tainted
-----------------------------
include/net/sock.h:592 suspicious rcu_dereference_check() usage!

other info that might help us debug this:


rcu_scheduler_active = 2, debug_locks = 1
4 locks held by syz-executor334/3611:
#0: ffff888073b7be10 (&sb->s_type->i_mutex_key#10){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:760 [inline]
#0: ffff888073b7be10 (&sb->s_type->i_mutex_key#10){+.+.}-{3:3}, at: __sock_release+0x86/0x280 net/socket.c:649
#1: ffffc900014e5c28 (&table->hash[i].lock){+...}-{2:2}, at: spin_lock_bh include/linux/spinlock.h:354 [inline]
#1: ffffc900014e5c28 (&table->hash[i].lock){+...}-{2:2}, at: udp_lib_unhash net/ipv4/udp.c:2014 [inline]
#1: ffffc900014e5c28 (&table->hash[i].lock){+...}-{2:2}, at: udp_lib_unhash+0x1d5/0x730 net/ipv4/udp.c:2004
#2: ffffffff8d7a9a78 (reuseport_lock){+...}-{2:2}, at: spin_lock_bh include/linux/spinlock.h:354 [inline]
#2: ffffffff8d7a9a78 (reuseport_lock){+...}-{2:2}, at: reuseport_detach_sock+0x22/0x4a0 net/core/sock_reuseport.c:346
#3: ffff888145f9a0b8 (clock-AF_INET){++..}-{2:2}, at: bpf_sk_reuseport_detach+0x26/0x190 kernel/bpf/reuseport_array.c:26

stack backtrace:
CPU: 1 PID: 3611 Comm: syz-executor334 Not tainted 5.19.0-syzkaller-05408-g94ce3b64c62d #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/22/2022
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:106
__rcu_dereference_sk_user_data_with_flags include/net/sock.h:592 [inline]
bpf_sk_reuseport_detach+0x156/0x190 kernel/bpf/reuseport_array.c:27
reuseport_detach_sock+0x8c/0x4a0 net/core/sock_reuseport.c:362
udp_lib_unhash net/ipv4/udp.c:2016 [inline]
udp_lib_unhash+0x210/0x730 net/ipv4/udp.c:2004
sk_common_release+0xba/0x390 net/core/sock.c:3600
inet_release+0x12e/0x280 net/ipv4/af_inet.c:428
__sock_release+0xcd/0x280 net/socket.c:650
sock_close+0x18/0x20 net/socket.c:1365
__fput+0x277/0x9d0 fs/file_table.c:320
task_work_run+0xdd/0x1a0 kernel/task_work.c:177
exit_task_work include/linux/task_work.h:38 [inline]
do_exit+0xade/0x29d0 kernel/exit.c:795
do_group_exit+0xd2/0x2f0 kernel/exit.c:925
__do_sys_exit_group kernel/exit.c:936 [inline]
__se_sys_exit_group kernel/exit.c:934 [inline]
__x64_sys_exit_group+0x3a/0x50 kernel/exit.c:934
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7fe407d09699
Code: Unable to access opcode bytes at RIP 0x7fe407d0966f.
RSP: 002b:00007ffc0ff152a8 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
For information about bisection process see: https://goo.gl/tpsmEJ#bisection
syzbot can test patches for this issue, for details see:
https://goo.gl/tpsmEJ#testing-patches

[Daniel Borkmann]

[ +Hawkins ]

On 8/15/22 12:59 PM, syzbot wrote:
> Hello,
>
> syzbot found the following issue on:
>
> HEAD commit: 94ce3b64c62d net/tls: Use RCU API to access tls_ctx->netdev
> git tree: net
> console+strace: https://syzkaller.appspot.com/x/log.txt?x=14641e15080000
> kernel config: https://syzkaller.appspot.com/x/.config?x=53da55f2bdeb0d4c
> dashboard link: https://syzkaller.appspot.com/bug?extid=24bcff6e82ce253f23ec
> compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=106c89fd080000
> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=17ead885080000
>
> The issue was bisected to:
>
> commit f72c38fad234759fe943cb2e40bf3d0f7de1d4d9
> Author: Edward Cree <ecree....@gmail.com>
> Date: Wed Jul 20 18:33:48 2022 +0000
>
> sfc: hook up ef100 representor TX

Looks rather related to:

commit 2a0133723f9ebeb751cfce19f74ec07e108bef1f
Author: Hawkins Jiawei <yin3...@gmail.com>
Date: Fri Aug 5 15:48:34 2022 +0800

net: fix refcount bug in sk_psock_get (2)

[Jakub Kicinski]

On Mon, 15 Aug 2022 14:28:58 +0200 Daniel Borkmann wrote:
> [ +Hawkins ]
>
> On 8/15/22 12:59 PM, syzbot wrote:
> > Hello,
> >
> > syzbot found the following issue on:
> >
> > HEAD commit: 94ce3b64c62d net/tls: Use RCU API to access tls_ctx->netdev
> > git tree: net
> > console+strace: https://syzkaller.appspot.com/x/log.txt?x=14641e15080000
> > kernel config: https://syzkaller.appspot.com/x/.config?x=53da55f2bdeb0d4c
> > dashboard link: https://syzkaller.appspot.com/bug?extid=24bcff6e82ce253f23ec
> > compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
> > syz repro: https://syzkaller.appspot.com/x/repro.syz?x=106c89fd080000
> > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=17ead885080000
> >
> > The issue was bisected to:
> >
> > commit f72c38fad234759fe943cb2e40bf3d0f7de1d4d9
> > Author: Edward Cree <ecree....@gmail.com>
> > Date: Wed Jul 20 18:33:48 2022 +0000
> >
> > sfc: hook up ef100 representor TX
>
> Looks rather related to:
>
> commit 2a0133723f9ebeb751cfce19f74ec07e108bef1f
> Author: Hawkins Jiawei <yin3...@gmail.com>
> Date: Fri Aug 5 15:48:34 2022 +0800
>
> net: fix refcount bug in sk_psock_get (2)

Indeed, looks like the reuseport clearing needs to be some
approximation of rcu_dereference_protected().

[Hillf Danton]

On Mon, 15 Aug 2022 03:59:26 -0700

> syzbot found the following issue on:
>
> HEAD commit: 94ce3b64c62d net/tls: Use RCU API to access tls_ctx->netdev
> git tree: net
> console+strace: https://syzkaller.appspot.com/x/log.txt?x=14641e15080000
> kernel config: https://syzkaller.appspot.com/x/.config?x=53da55f2bdeb0d4c
> dashboard link: https://syzkaller.appspot.com/bug?extid=24bcff6e82ce253f23ec
> compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=106c89fd080000
> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=17ead885080000

Add rcu lock to queisce thw warning.

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git 94ce3b64c62d

--- x/kernel/bpf/reuseport_array.c
+++ r/kernel/bpf/reuseport_array.c
@@ -24,6 +24,7 @@ void bpf_sk_reuseport_detach(struct sock
struct sock __rcu **socks;

write_lock_bh(&sk->sk_callback_lock);
+ rcu_read_lock();
socks = __rcu_dereference_sk_user_data_with_flags(sk, SK_USER_DATA_BPF);
if (socks) {
WRITE_ONCE(sk->sk_user_data, NULL);
@@ -35,6 +36,7 @@ void bpf_sk_reuseport_detach(struct sock
*/
RCU_INIT_POINTER(*socks, NULL);
}
+ rcu_read_unlock();
write_unlock_bh(&sk->sk_callback_lock);
}

--

[syzbot]

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-and-tested-by: syzbot+24bcff...@syzkaller.appspotmail.com

Tested on:

commit: 94ce3b64 net/tls: Use RCU API to access tls_ctx->netdev
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
console output: https://syzkaller.appspot.com/x/log.txt?x=11d25985080000

kernel config: https://syzkaller.appspot.com/x/.config?x=53da55f2bdeb0d4c
dashboard link: https://syzkaller.appspot.com/bug?extid=24bcff6e82ce253f23ec
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2

patch: https://syzkaller.appspot.com/x/patch.diff?x=174e6b35080000

Note: testing is done by a robot and is best-effort only.

[syzbot]

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-and-tested-by: syzbot+24bcff...@syzkaller.appspotmail.com

Tested on:

commit: 94ce3b64 net/tls: Use RCU API to access tls_ctx->netdev
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git
console output: https://syzkaller.appspot.com/x/log.txt?x=11bf00cb080000

kernel config: https://syzkaller.appspot.com/x/.config?x=53da55f2bdeb0d4c
dashboard link: https://syzkaller.appspot.com/bug?extid=24bcff6e82ce253f23ec
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2

patch: https://syzkaller.appspot.com/x/patch.diff?x=171d056b080000

[syzbot]

Auto-closing this bug as obsolete.
No recent activity, existing reproducers are no longer triggering the issue.