[v6.1] INFO: rcu detected stall in kthreadd
0 views
Skip to first unread message
syzbot
May 15, 2024, 3:20:24 AMMay 15
to syzkaller...@googlegroups.com
Hello,
syzbot found the following issue on:
HEAD commit: 909ba1f1b414 Linux 6.1.90
git tree: linux-6.1.y
console output: https://syzkaller.appspot.com/x/log.txt?x=17571cf4980000
kernel config: https://syzkaller.appspot.com/x/.config?x=3be6d6f79b879a67
dashboard link: https://syzkaller.appspot.com/bug?extid=c0932c504bc682ee3a87
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
Unfortunately, I don't have any reproducer for this issue yet.
Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/63178de7cba7/disk-909ba1f1.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/25dec90d8126/vmlinux-909ba1f1.xz
kernel image: https://storage.googleapis.com/syzbot-assets/25509ea1c6cd/bzImage-909ba1f1.xz
IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+c0932c...@syzkaller.appspotmail.com
rcu: INFO: rcu_preempt self-detected stall on CPU
rcu: 1-...!: (1 GPs behind) idle=3e44/1/0x4000000000000000 softirq=19017/19018 fqs=8
(t=10500 jiffies g=22873 q=63 ncpus=2)
rcu: rcu_preempt kthread starved for 10484 jiffies! g22873 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt state:R running task stack:25368 pid:16 ppid:2 flags:0x00004000
Call Trace:
<TASK>
context_switch kernel/sched/core.c:5245 [inline]
__schedule+0x142d/0x4550 kernel/sched/core.c:6558
schedule+0xbf/0x180 kernel/sched/core.c:6634
schedule_timeout+0x1b9/0x300 kernel/time/timer.c:1965
rcu_gp_fqs_loop+0x2d2/0x1150 kernel/rcu/tree.c:1706
rcu_gp_kthread+0xa3/0x3b0 kernel/rcu/tree.c:1905
kthread+0x28d/0x320 kernel/kthread.c:376
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
</TASK>
rcu: Stack dump where RCU GP kthread last ran:
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 5807 Comm: syz-executor.2 Not tainted 6.1.90-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024
RIP: 0010:__kasan_check_read+0x6/0x10 mm/kasan/shadow.c:31
Code: 41 5e 41 5f 5d c3 48 c7 c7 f5 d1 90 8c eb 0a 48 c7 c7 2d d2 90 8c 4c 89 fe e8 96 ae a6 08 31 db eb d7 cc cc 89 f6 48 8b 0c 24 <31> d2 e9 d3 ec ff ff 0f 1f 00 89 f6 48 8b 0c 24 ba 01 00 00 00 e9
RSP: 0018:ffffc9000364f900 EFLAGS: 00000046
RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffff8191519b
RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffffffff8e738228
RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000555556c48480(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f74c6fa7984 CR3: 000000007bb3c000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<NMI>
</NMI>
<TASK>
instrument_atomic_read include/linux/instrumented.h:72 [inline]
_test_bit include/asm-generic/bitops/instrumented-non-atomic.h:141 [inline]
cpumask_test_cpu include/linux/cpumask.h:444 [inline]
cpu_online include/linux/cpumask.h:1030 [inline]
trace_irq_disable include/trace/events/preemptirq.h:36 [inline]
trace_hardirqs_off_finish+0x7b/0x1a0 kernel/trace/trace_preemptirq.c:67
irqentry_enter+0x33/0x50 kernel/entry/common.c:382
sysvec_irq_work+0xa/0xb0 arch/x86/kernel/irq_work.c:17
asm_sysvec_irq_work+0x16/0x20 arch/x86/include/asm/idtentry.h:679
RIP: 0010:finish_task_switch+0x1d3/0x810 kernel/sched/core.c:5124
Code: 37 0b 00 48 83 c4 08 4c 89 f7 e8 98 30 00 00 0f 1f 44 00 00 4c 89 f7 e8 7b d0 35 09 e8 36 26 32 00 fb 49 8d bc 24 f8 15 00 00 <48> 89 f8 48 c1 e8 03 49 bd 00 00 00 00 00 fc ff df 42 0f b6 04 28
RSP: 0018:ffffc9000364fa08 EFLAGS: 00000286
RAX: 8e3622f37e23f700 RBX: ffff88807be0d974 RCX: ffffffff91c93103
RDX: dffffc0000000000 RSI: ffffffff8aec01c0 RDI: ffff888077ceef38
RBP: ffffc9000364fa50 R08: dffffc0000000000 R09: ffffed1017307541
R10: 0000000000000000 R11: dffffc0000000001 R12: ffff888077ced940
R13: 1ffff110173076eb R14: ffff8880b983aa00 R15: ffff8880b983b758
context_switch kernel/sched/core.c:5248 [inline]
__schedule+0x1435/0x4550 kernel/sched/core.c:6558
schedule+0xbf/0x180 kernel/sched/core.c:6634
do_nanosleep+0x192/0x5f0 kernel/time/hrtimer.c:2045
hrtimer_nanosleep+0x24d/0x490 kernel/time/hrtimer.c:2098
__do_sys_clock_nanosleep kernel/time/posix-timers.c:1308 [inline]
__se_sys_clock_nanosleep+0x323/0x3b0 kernel/time/posix-timers.c:1285
do_syscall_x64 arch/x86/entry/common.c:51 [inline]
do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:81
entry_SYSCALL_64_after_hwframe+0x68/0xd2
RIP: 0033:0x7f91d8ca9175
Code: 24 0c 89 3c 24 48 89 4c 24 18 e8 f6 b9 ff ff 4c 8b 54 24 18 48 8b 54 24 10 41 89 c0 8b 74 24 0c 8b 3c 24 b8 e6 00 00 00 0f 05 <44> 89 c7 48 89 04 24 e8 4f ba ff ff 48 8b 04 24 48 83 c4 28 f7 d8
RSP: 002b:00007ffe7e3e6e80 EFLAGS: 00000293 ORIG_RAX: 00000000000000e6
RAX: ffffffffffffffda RBX: 000000000000002f RCX: 00007f91d8ca9175
RDX: 00007ffe7e3e6ec0 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 00007ffe7e3e6f4c R08: 0000000000000000 R09: 0000000000000010
R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000032
R13: 000000000002f113 R14: 000000000002ef5e R15: 0000000000000009
</TASK>
CPU: 1 PID: 2 Comm: kthreadd Not tainted 6.1.90-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024
RIP: 0010:finish_task_switch+0x1d3/0x810 kernel/sched/core.c:5124
Code: 37 0b 00 48 83 c4 08 4c 89 f7 e8 98 30 00 00 0f 1f 44 00 00 4c 89 f7 e8 7b d0 35 09 e8 36 26 32 00 fb 49 8d bc 24 f8 15 00 00 <48> 89 f8 48 c1 e8 03 49 bd 00 00 00 00 00 fc ff df 42 0f b6 04 28
RSP: 0018:ffffc90000077a88 EFLAGS: 00000282
RAX: c16ebc9c6a0af700 RBX: ffff88805e808034 RCX: ffffffff91c93103
RDX: dffffc0000000000 RSI: ffffffff8aec01c0 RDI: ffff88814006b3b8
RBP: ffffc90000077ad0 R08: dffffc0000000000 R09: ffffed1017327541
R10: 0000000000000000 R11: dffffc0000000001 R12: ffff888140069dc0
R13: 1ffff110173276eb R14: ffff8880b993aa00 R15: ffff8880b993b758
FS: 0000000000000000(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f91d990ff00 CR3: 0000000056260000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<IRQ>
</IRQ>
<TASK>
context_switch kernel/sched/core.c:5248 [inline]
__schedule+0x1435/0x4550 kernel/sched/core.c:6558
preempt_schedule_irq+0xf7/0x1c0 kernel/sched/core.c:6870
irqentry_exit+0x53/0x80 kernel/entry/common.c:439
asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:653
RIP: 0010:preempt_count_add+0x2/0x180 kernel/sched/core.c:5698
Code: 0c 24 80 e1 07 80 c1 03 38 c1 7c 97 48 8b 3c 24 e8 43 b2 83 00 eb 8c e9 f2 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 66 90 41 57 <41> 56 53 89 fb 48 c7 c0 60 31 c9 91 48 c1 e8 03 49 bf 00 00 00 00
RSP: 0018:ffffc90000077e28 EFLAGS: 00000202
RAX: 0000000000000001 RBX: 1ffff1102800d3b8 RCX: ffffffff8a933926
RDX: 0000000000000000 RSI: 0000000000000008 RDI: 0000000000000001
RBP: ffffc90000077f48 R08: dffffc0000000000 R09: ffffed102800d3b9
R10: 0000000000000000 R11: dffffc0000000001 R12: ffff888140069dc0
R13: ffff8880779a1b00 R14: ffff88814006b038 R15: dffffc0000000000
schedule+0xb8/0x180 kernel/sched/core.c:6633
kthreadd+0x420/0x760 kernel/kthread.c:734
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
</TASK>
---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.
syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title
If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)
If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report
If you want to undo deduplication, reply with:
#syz undup
syzbot found the following issue on:
HEAD commit: 909ba1f1b414 Linux 6.1.90
git tree: linux-6.1.y
console output: https://syzkaller.appspot.com/x/log.txt?x=17571cf4980000
kernel config: https://syzkaller.appspot.com/x/.config?x=3be6d6f79b879a67
dashboard link: https://syzkaller.appspot.com/bug?extid=c0932c504bc682ee3a87
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
Unfortunately, I don't have any reproducer for this issue yet.
Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/63178de7cba7/disk-909ba1f1.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/25dec90d8126/vmlinux-909ba1f1.xz
kernel image: https://storage.googleapis.com/syzbot-assets/25509ea1c6cd/bzImage-909ba1f1.xz
IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+c0932c...@syzkaller.appspotmail.com
rcu: INFO: rcu_preempt self-detected stall on CPU
rcu: 1-...!: (1 GPs behind) idle=3e44/1/0x4000000000000000 softirq=19017/19018 fqs=8
(t=10500 jiffies g=22873 q=63 ncpus=2)
rcu: rcu_preempt kthread starved for 10484 jiffies! g22873 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt state:R running task stack:25368 pid:16 ppid:2 flags:0x00004000
Call Trace:
<TASK>
context_switch kernel/sched/core.c:5245 [inline]
__schedule+0x142d/0x4550 kernel/sched/core.c:6558
schedule+0xbf/0x180 kernel/sched/core.c:6634
schedule_timeout+0x1b9/0x300 kernel/time/timer.c:1965
rcu_gp_fqs_loop+0x2d2/0x1150 kernel/rcu/tree.c:1706
rcu_gp_kthread+0xa3/0x3b0 kernel/rcu/tree.c:1905
kthread+0x28d/0x320 kernel/kthread.c:376
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
</TASK>
rcu: Stack dump where RCU GP kthread last ran:
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 5807 Comm: syz-executor.2 Not tainted 6.1.90-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024
RIP: 0010:__kasan_check_read+0x6/0x10 mm/kasan/shadow.c:31
Code: 41 5e 41 5f 5d c3 48 c7 c7 f5 d1 90 8c eb 0a 48 c7 c7 2d d2 90 8c 4c 89 fe e8 96 ae a6 08 31 db eb d7 cc cc 89 f6 48 8b 0c 24 <31> d2 e9 d3 ec ff ff 0f 1f 00 89 f6 48 8b 0c 24 ba 01 00 00 00 e9
RSP: 0018:ffffc9000364f900 EFLAGS: 00000046
RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffff8191519b
RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffffffff8e738228
RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000555556c48480(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f74c6fa7984 CR3: 000000007bb3c000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<NMI>
</NMI>
<TASK>
instrument_atomic_read include/linux/instrumented.h:72 [inline]
_test_bit include/asm-generic/bitops/instrumented-non-atomic.h:141 [inline]
cpumask_test_cpu include/linux/cpumask.h:444 [inline]
cpu_online include/linux/cpumask.h:1030 [inline]
trace_irq_disable include/trace/events/preemptirq.h:36 [inline]
trace_hardirqs_off_finish+0x7b/0x1a0 kernel/trace/trace_preemptirq.c:67
irqentry_enter+0x33/0x50 kernel/entry/common.c:382
sysvec_irq_work+0xa/0xb0 arch/x86/kernel/irq_work.c:17
asm_sysvec_irq_work+0x16/0x20 arch/x86/include/asm/idtentry.h:679
RIP: 0010:finish_task_switch+0x1d3/0x810 kernel/sched/core.c:5124
Code: 37 0b 00 48 83 c4 08 4c 89 f7 e8 98 30 00 00 0f 1f 44 00 00 4c 89 f7 e8 7b d0 35 09 e8 36 26 32 00 fb 49 8d bc 24 f8 15 00 00 <48> 89 f8 48 c1 e8 03 49 bd 00 00 00 00 00 fc ff df 42 0f b6 04 28
RSP: 0018:ffffc9000364fa08 EFLAGS: 00000286
RAX: 8e3622f37e23f700 RBX: ffff88807be0d974 RCX: ffffffff91c93103
RDX: dffffc0000000000 RSI: ffffffff8aec01c0 RDI: ffff888077ceef38
RBP: ffffc9000364fa50 R08: dffffc0000000000 R09: ffffed1017307541
R10: 0000000000000000 R11: dffffc0000000001 R12: ffff888077ced940
R13: 1ffff110173076eb R14: ffff8880b983aa00 R15: ffff8880b983b758
context_switch kernel/sched/core.c:5248 [inline]
__schedule+0x1435/0x4550 kernel/sched/core.c:6558
schedule+0xbf/0x180 kernel/sched/core.c:6634
do_nanosleep+0x192/0x5f0 kernel/time/hrtimer.c:2045
hrtimer_nanosleep+0x24d/0x490 kernel/time/hrtimer.c:2098
__do_sys_clock_nanosleep kernel/time/posix-timers.c:1308 [inline]
__se_sys_clock_nanosleep+0x323/0x3b0 kernel/time/posix-timers.c:1285
do_syscall_x64 arch/x86/entry/common.c:51 [inline]
do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:81
entry_SYSCALL_64_after_hwframe+0x68/0xd2
RIP: 0033:0x7f91d8ca9175
Code: 24 0c 89 3c 24 48 89 4c 24 18 e8 f6 b9 ff ff 4c 8b 54 24 18 48 8b 54 24 10 41 89 c0 8b 74 24 0c 8b 3c 24 b8 e6 00 00 00 0f 05 <44> 89 c7 48 89 04 24 e8 4f ba ff ff 48 8b 04 24 48 83 c4 28 f7 d8
RSP: 002b:00007ffe7e3e6e80 EFLAGS: 00000293 ORIG_RAX: 00000000000000e6
RAX: ffffffffffffffda RBX: 000000000000002f RCX: 00007f91d8ca9175
RDX: 00007ffe7e3e6ec0 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 00007ffe7e3e6f4c R08: 0000000000000000 R09: 0000000000000010
R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000032
R13: 000000000002f113 R14: 000000000002ef5e R15: 0000000000000009
</TASK>
CPU: 1 PID: 2 Comm: kthreadd Not tainted 6.1.90-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024
RIP: 0010:finish_task_switch+0x1d3/0x810 kernel/sched/core.c:5124
Code: 37 0b 00 48 83 c4 08 4c 89 f7 e8 98 30 00 00 0f 1f 44 00 00 4c 89 f7 e8 7b d0 35 09 e8 36 26 32 00 fb 49 8d bc 24 f8 15 00 00 <48> 89 f8 48 c1 e8 03 49 bd 00 00 00 00 00 fc ff df 42 0f b6 04 28
RSP: 0018:ffffc90000077a88 EFLAGS: 00000282
RAX: c16ebc9c6a0af700 RBX: ffff88805e808034 RCX: ffffffff91c93103
RDX: dffffc0000000000 RSI: ffffffff8aec01c0 RDI: ffff88814006b3b8
RBP: ffffc90000077ad0 R08: dffffc0000000000 R09: ffffed1017327541
R10: 0000000000000000 R11: dffffc0000000001 R12: ffff888140069dc0
R13: 1ffff110173276eb R14: ffff8880b993aa00 R15: ffff8880b993b758
FS: 0000000000000000(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f91d990ff00 CR3: 0000000056260000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<IRQ>
</IRQ>
<TASK>
context_switch kernel/sched/core.c:5248 [inline]
__schedule+0x1435/0x4550 kernel/sched/core.c:6558
preempt_schedule_irq+0xf7/0x1c0 kernel/sched/core.c:6870
irqentry_exit+0x53/0x80 kernel/entry/common.c:439
asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:653
RIP: 0010:preempt_count_add+0x2/0x180 kernel/sched/core.c:5698
Code: 0c 24 80 e1 07 80 c1 03 38 c1 7c 97 48 8b 3c 24 e8 43 b2 83 00 eb 8c e9 f2 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 66 90 41 57 <41> 56 53 89 fb 48 c7 c0 60 31 c9 91 48 c1 e8 03 49 bf 00 00 00 00
RSP: 0018:ffffc90000077e28 EFLAGS: 00000202
RAX: 0000000000000001 RBX: 1ffff1102800d3b8 RCX: ffffffff8a933926
RDX: 0000000000000000 RSI: 0000000000000008 RDI: 0000000000000001
RBP: ffffc90000077f48 R08: dffffc0000000000 R09: ffffed102800d3b9
R10: 0000000000000000 R11: dffffc0000000001 R12: ffff888140069dc0
R13: ffff8880779a1b00 R14: ffff88814006b038 R15: dffffc0000000000
schedule+0xb8/0x180 kernel/sched/core.c:6633
kthreadd+0x420/0x760 kernel/kthread.c:734
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
</TASK>
---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.
syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title
If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)
If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report
If you want to undo deduplication, reply with:
#syz undup
Reply all
Reply to author
Forward
0 new messages