[syzbot] [kernel?] INFO: rcu detected stall in schedule (6)

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: a785fd28d31f Merge tag 'for-6.5-rc5-tag' of git://git.kern..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=167d8e17a80000
kernel config: https://syzkaller.appspot.com/x/.config?x=3e670757e16affb
dashboard link: https://syzkaller.appspot.com/bug?extid=77195ae75047f1438785
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=15ec36ada80000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/f025e375131a/disk-a785fd28.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/d38bc269b3fb/vmlinux-a785fd28.xz
kernel image: https://storage.googleapis.com/syzbot-assets/5eb32eee65b8/bzImage-a785fd28.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+77195a...@syzkaller.appspotmail.com

rcu: INFO: rcu_preempt self-detected stall on CPU
rcu: 0-....: (1 GPs behind) idle=816c/1/0x4000000000000000 softirq=8838/8840 fqs=5242
rcu: hardirqs softirqs csw/system
rcu: number: 0 0 0
rcu: cputime: 29521 22957 0 ==> 52480(ms)
rcu: (t=10500 jiffies g=5877 q=1892 ncpus=2)
CPU: 0 PID: 15 Comm: ksoftirqd/0 Not tainted 6.5.0-rc5-syzkaller-00362-ga785fd28d31f #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/26/2023
RIP: 0010:write_comp_data+0x7/0x90 kernel/kcov.c:230
Code: 81 e2 00 01 ff 00 75 10 65 48 8b 04 25 80 b9 03 00 48 8b 80 f8 15 00 00 c3 66 2e 0f 1f 84 00 00 00 00 00 65 8b 05 c1 b6 7d 7e <49> 89 f1 89 c6 49 89 d2 81 e6 00 01 00 00 49 89 f8 65 48 8b 14 25
RSP: 0018:ffffc90000007d58 EFLAGS: 00000293
RAX: 0000000000000102 RBX: ffff8880638b6394 RCX: ffffffff88595887
RDX: 0000000000000000 RSI: 0000000000000008 RDI: 0000000000000004
RBP: 0000000000000008 R08: 0000000000000004 R09: 0000000000000008
R10: 0000000000000010 R11: 0000000000000000 R12: 0000000000000010
R13: ffff88807ce5ab60 R14: 0000000000000000 R15: 0000000000000001
FS: 0000000000000000(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000020000600 CR3: 000000000c776000 CR4: 0000000000350ef0
Call Trace:
<IRQ>
taprio_dequeue_tc_priority+0x247/0x4b0 net/sched/sch_taprio.c:808
taprio_dequeue+0x12e/0x5f0 net/sched/sch_taprio.c:868
dequeue_skb net/sched/sch_generic.c:292 [inline]
qdisc_restart net/sched/sch_generic.c:397 [inline]
__qdisc_run+0x1c4/0x19d0 net/sched/sch_generic.c:415
qdisc_run include/net/pkt_sched.h:125 [inline]
qdisc_run include/net/pkt_sched.h:122 [inline]
net_tx_action+0x71e/0xc80 net/core/dev.c:5049
__do_softirq+0x218/0x965 kernel/softirq.c:553
invoke_softirq kernel/softirq.c:427 [inline]
__irq_exit_rcu kernel/softirq.c:632 [inline]
irq_exit_rcu+0xb7/0x120 kernel/softirq.c:644
sysvec_apic_timer_interrupt+0x93/0xc0 arch/x86/kernel/apic/apic.c:1109
</IRQ>
<TASK>
asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:645
RIP: 0010:finish_task_switch.isra.0+0x220/0xc90 kernel/sched/core.c:5253
Code: 0a 00 00 44 8b 0d c0 01 52 0d 45 85 c9 0f 85 c0 01 00 00 48 89 df e8 ff f8 ff ff e8 9a dd 32 00 fb 65 48 8b 1c 25 80 b9 03 00 <48> 8d bb e0 15 00 00 48 b8 00 00 00 00 00 fc ff df 48 89 fa 48 c1
RSP: 0018:ffffc90000147c88 EFLAGS: 00000202
RAX: 000000000006247f RBX: ffff888015e5bb80 RCX: 1ffffffff1d56e19
RDX: 0000000000000000 RSI: ffffffff8a6c80a0 RDI: ffffffff8ac811c0
RBP: ffffc90000147cd0 R08: 0000000000000001 R09: 0000000000000001
R10: ffffffff8eaba257 R11: 0000000000000000 R12: ffff8880b983d118
R13: ffff88802bd49dc0 R14: 0000000000000000 R15: ffff8880b983c3c0
context_switch kernel/sched/core.c:5384 [inline]
__schedule+0xee9/0x59f0 kernel/sched/core.c:6710
schedule+0xe7/0x1b0 kernel/sched/core.c:6786
smpboot_thread_fn+0x2ee/0xa00 kernel/smpboot.c:160
kthread+0x33a/0x430 kernel/kthread.c:389
ret_from_fork+0x2c/0x70 arch/x86/kernel/process.c:145
ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:304
</TASK>


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the bug is already fixed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.

If you want to overwrite bug's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the bug is a duplicate of another bug, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup

[Hillf Danton]

On Thu, 17 Aug 2023 02:25:46 -0700

> HEAD commit: a785fd28d31f Merge tag 'for-6.5-rc5-tag' of git://git.kern..
> git tree: upstream

> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=15ec36ada80000

Check if taprio_dequeue is rcu hog.

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git a785fd28d31f

--- x/net/sched/sch_taprio.c
+++ y/net/sched/sch_taprio.c
@@ -787,6 +787,7 @@ static struct sk_buff *taprio_dequeue_tc
int num_tc = netdev_get_num_tc(dev);
struct sk_buff *skb;
int tc;
+ int loop = 0;

for (tc = num_tc - 1; tc >= 0; tc--) {
int first_txq = q->cur_txq[tc];
@@ -805,6 +806,8 @@ static struct sk_buff *taprio_dequeue_tc

if (skb)
return skb;
+ if (loop++ > 50)
+ return NULL;
} while (q->cur_txq[tc] != first_txq);
}

--

[syzbot]

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-and-tested-by: syzbot+77195a...@syzkaller.appspotmail.com

Tested on:

commit: a785fd28 Merge tag 'for-6.5-rc5-tag' of git://git.kern..
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
console output: https://syzkaller.appspot.com/x/log.txt?x=16ec1d9ba80000

kernel config: https://syzkaller.appspot.com/x/.config?x=3e670757e16affb
dashboard link: https://syzkaller.appspot.com/bug?extid=77195ae75047f1438785
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=15fb4440680000

Note: testing is done by a robot and is best-effort only.

[syzbot]

syzbot has bisected this issue to:

commit c2368b19807affd7621f7c4638cd2e17fec13021
Author: Jiri Pirko <ji...@nvidia.com>
Date: Fri Jul 29 07:10:35 2022 +0000

net: devlink: introduce "unregistering" mark and use it during devlinks iteration

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=12ed7a99680000
start commit: a785fd28d31f Merge tag 'for-6.5-rc5-tag' of git://git.kern..
git tree: upstream
final oops: https://syzkaller.appspot.com/x/report.txt?x=11ed7a99680000
console output: https://syzkaller.appspot.com/x/log.txt?x=16ed7a99680000

kernel config: https://syzkaller.appspot.com/x/.config?x=3e670757e16affb
dashboard link: https://syzkaller.appspot.com/bug?extid=77195ae75047f1438785

syz repro: https://syzkaller.appspot.com/x/repro.syz?x=15ec36ada80000

Reported-by: syzbot+77195a...@syzkaller.appspotmail.com
Fixes: c2368b19807a ("net: devlink: introduce "unregistering" mark and use it during devlinks iteration")

For information about bisection process see: https://goo.gl/tpsmEJ#bisection

[syzbot]

Auto-closing this bug as obsolete.
No recent activity, existing reproducers are no longer triggering the issue.