[v5.15] INFO: rcu detected stall in kvm_vcpu_release
0 views
Skip to first unread message
syzbot
May 24, 2024, 8:50:27 PMMay 24
to syzkaller...@googlegroups.com
Hello,
syzbot found the following issue on:
HEAD commit: 83655231580b Linux 5.15.159
git tree: linux-5.15.y
console output: https://syzkaller.appspot.com/x/log.txt?x=15c74fb2980000
kernel config: https://syzkaller.appspot.com/x/.config?x=ea870d7ee19c0a5c
dashboard link: https://syzkaller.appspot.com/bug?extid=041606b95d0537b167df
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
Unfortunately, I don't have any reproducer for this issue yet.
Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/d5444b7453a0/disk-83655231.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/76ae736a43a3/vmlinux-83655231.xz
kernel image: https://storage.googleapis.com/syzbot-assets/2540e00ba747/bzImage-83655231.xz
IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+041606...@syzkaller.appspotmail.com
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: Tasks blocked on level-0 rcu_node (CPUs 0-1): P3699/2:b..l
(detected by 1, t=10502 jiffies, g=6265, q=325)
task:syz-executor.3 state:R running task stack:25592 pid: 3699 ppid: 3524 flags:0x00004002
Call Trace:
<TASK>
context_switch kernel/sched/core.c:5030 [inline]
__schedule+0x12c4/0x45b0 kernel/sched/core.c:6376
preempt_schedule_irq+0xf7/0x1c0 kernel/sched/core.c:6780
irqentry_exit+0x53/0x80 kernel/entry/common.c:432
asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:lock_acquire+0x252/0x4f0 kernel/locking/lockdep.c:5627
Code: 2b 00 74 08 4c 89 f7 e8 fc 7c 67 00 f6 44 24 61 02 0f 85 84 01 00 00 41 f7 c7 00 02 00 00 74 01 fb 48 c7 44 24 40 0e 36 e0 45 <4b> c7 44 25 00 00 00 00 00 43 c7 44 25 09 00 00 00 00 43 c7 44 25
RSP: 0018:ffffc900044cf940 EFLAGS: 00000206
RAX: 0000000000000001 RBX: 1ffff92000899f34 RCX: 1ffff92000899ed4
RDX: dffffc0000000000 RSI: ffffffff8a8b3c20 RDI: ffffffff8ad8f680
RBP: ffffc900044cfa88 R08: dffffc0000000000 R09: fffffbfff1f7f019
R10: 0000000000000000 R11: dffffc0000000001 R12: 1ffff92000899f30
R13: dffffc0000000000 R14: ffffc900044cf9a0 R15: 0000000000000246
rcu_lock_acquire+0x20/0x30 include/linux/rcupdate.h:312
rcu_read_lock include/linux/rcupdate.h:739 [inline]
percpu_ref_tryget_many include/linux/percpu-refcount.h:241 [inline]
percpu_ref_tryget+0x10/0x140 include/linux/percpu-refcount.h:266
css_tryget include/linux/cgroup.h:355 [inline]
get_mem_cgroup_from_objcg+0xa3/0x120 mm/memcontrol.c:2812
obj_cgroup_uncharge_pages mm/memcontrol.c:2989 [inline]
__memcg_kmem_uncharge_page+0x73/0x130 mm/memcontrol.c:3080
free_pages_prepare mm/page_alloc.c:1332 [inline]
free_pcp_prepare mm/page_alloc.c:1391 [inline]
free_unref_page_prepare+0x11d/0xcf0 mm/page_alloc.c:3317
free_unref_page+0x95/0x2d0 mm/page_alloc.c:3396
__vunmap+0x8d4/0xa20 mm/vmalloc.c:2621
memslot_rmap_free arch/x86/kvm/x86.c:11747 [inline]
kvm_arch_free_memslot+0x3c/0x150 arch/x86/kvm/x86.c:11756
kvm_free_memslot arch/x86/kvm/../../../virt/kvm/kvm_main.c:934 [inline]
kvm_free_memslots arch/x86/kvm/../../../virt/kvm/kvm_main.c:948 [inline]
kvm_destroy_vm arch/x86/kvm/../../../virt/kvm/kvm_main.c:1254 [inline]
kvm_put_kvm+0x1183/0x1840 arch/x86/kvm/../../../virt/kvm/kvm_main.c:1283
kvm_vcpu_release+0x53/0x60 arch/x86/kvm/../../../virt/kvm/kvm_main.c:3643
__fput+0x3bf/0x890 fs/file_table.c:280
task_work_run+0x129/0x1a0 kernel/task_work.c:164
tracehook_notify_resume include/linux/tracehook.h:189 [inline]
exit_to_user_mode_loop+0x106/0x130 kernel/entry/common.c:181
exit_to_user_mode_prepare+0xb1/0x140 kernel/entry/common.c:214
__syscall_exit_to_user_mode_work kernel/entry/common.c:296 [inline]
syscall_exit_to_user_mode+0x5d/0x240 kernel/entry/common.c:307
do_syscall_64+0x47/0xb0 arch/x86/entry/common.c:86
entry_SYSCALL_64_after_hwframe+0x66/0xd0
RIP: 0033:0x7f54f7d95dda
RSP: 002b:00007ffdf039fec0 EFLAGS: 00000293 ORIG_RAX: 0000000000000003
RAX: 0000000000000000 RBX: 000000000000000b RCX: 00007f54f7d95dda
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 000000000000000a
RBP: 00007f54f7ec7980 R08: 0000001b30320000 R09: 0000000000001291
R10: 000000008120b6f7 R11: 0000000000000293 R12: 000000000000fad1
R13: 00007f54f7ec5f8c R14: 00007ffdf039ffc0 R15: 0000000000000032
</TASK>
rcu: rcu_preempt kthread starved for 9924 jiffies! g6265 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1
rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt state:R running task stack:27288 pid: 15 ppid: 2 flags:0x00004000
Call Trace:
<TASK>
context_switch kernel/sched/core.c:5030 [inline]
__schedule+0x12c4/0x45b0 kernel/sched/core.c:6376
schedule+0x11b/0x1f0 kernel/sched/core.c:6459
schedule_timeout+0x1b9/0x300 kernel/time/timer.c:1914
rcu_gp_fqs_loop+0x2bf/0x1080 kernel/rcu/tree.c:1972
rcu_gp_kthread+0xa4/0x360 kernel/rcu/tree.c:2145
kthread+0x3f6/0x4f0 kernel/kthread.c:334
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:300
</TASK>
rcu: Stack dump where RCU GP kthread last ran:
NMI backtrace for cpu 1
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.15.159-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024
Call Trace:
<IRQ>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x1e3/0x2d0 lib/dump_stack.c:106
nmi_cpu_backtrace+0x46a/0x4a0 lib/nmi_backtrace.c:111
nmi_trigger_cpumask_backtrace+0x181/0x2a0 lib/nmi_backtrace.c:62
trigger_single_cpu_backtrace include/linux/nmi.h:166 [inline]
rcu_check_gp_kthread_starvation+0x1d2/0x240 kernel/rcu/tree_stall.h:487
print_other_cpu_stall+0x137a/0x14d0 kernel/rcu/tree_stall.h:592
check_cpu_stall kernel/rcu/tree_stall.h:745 [inline]
rcu_pending kernel/rcu/tree.c:3932 [inline]
rcu_sched_clock_irq+0xa38/0x1150 kernel/rcu/tree.c:2619
update_process_times+0x196/0x200 kernel/time/timer.c:1818
tick_sched_handle kernel/time/tick-sched.c:254 [inline]
tick_sched_timer+0x386/0x550 kernel/time/tick-sched.c:1473
__run_hrtimer kernel/time/hrtimer.c:1686 [inline]
__hrtimer_run_queues+0x55b/0xcf0 kernel/time/hrtimer.c:1750
hrtimer_interrupt+0x392/0x980 kernel/time/hrtimer.c:1812
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1085 [inline]
__sysvec_apic_timer_interrupt+0x139/0x470 arch/x86/kernel/apic/apic.c:1102
sysvec_apic_timer_interrupt+0x8c/0xb0 arch/x86/kernel/apic/apic.c:1096
</IRQ>
<TASK>
asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:native_save_fl arch/x86/include/asm/irqflags.h:22 [inline]
RIP: 0010:arch_local_save_flags arch/x86/include/asm/irqflags.h:70 [inline]
RIP: 0010:arch_irqs_disabled arch/x86/include/asm/irqflags.h:132 [inline]
RIP: 0010:acpi_safe_halt drivers/acpi/processor_idle.c:110 [inline]
RIP: 0010:acpi_idle_do_entry+0x10f/0x340 drivers/acpi/processor_idle.c:570
Code: ed 58 f7 48 83 e3 08 0f 85 0a 01 00 00 4c 8d 74 24 20 e8 24 69 5f f7 0f 1f 44 00 00 e8 1a e9 58 f7 0f 00 2d 33 aa bb 00 fb f4 <4c> 89 f3 48 c1 eb 03 42 80 3c 3b 00 74 08 4c 89 f7 e8 8b b7 a2 f7
RSP: 0018:ffffc90000d67b00 EFLAGS: 000002d3
RAX: ffffffff8a2773a6 RBX: 0000000000000000 RCX: ffff88813fe68000
RDX: 0000000000000000 RSI: ffffffff8a8b2980 RDI: ffffffff8ad8f680
RBP: ffffc90000d67b90 R08: ffffffff8186dcf0 R09: ffffed1027fcd001
R10: 0000000000000000 R11: dffffc0000000001 R12: 1ffff920001acf60
R13: ffff888145e52004 R14: ffffc90000d67b20 R15: dffffc0000000000
acpi_idle_enter+0x352/0x4f0 drivers/acpi/processor_idle.c:705
cpuidle_enter_state+0x521/0xef0 drivers/cpuidle/cpuidle.c:237
cpuidle_enter+0x59/0x90 drivers/cpuidle/cpuidle.c:351
call_cpuidle kernel/sched/idle.c:158 [inline]
cpuidle_idle_call kernel/sched/idle.c:239 [inline]
do_idle+0x3e4/0x670 kernel/sched/idle.c:306
cpu_startup_entry+0x14/0x20 kernel/sched/idle.c:403
start_secondary+0x371/0x500 arch/x86/kernel/smpboot.c:281
secondary_startup_64_no_verify+0xb1/0xbb
</TASK>
---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.
syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title
If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)
If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report
If you want to undo deduplication, reply with:
#syz undup
syzbot found the following issue on:
HEAD commit: 83655231580b Linux 5.15.159
git tree: linux-5.15.y
console output: https://syzkaller.appspot.com/x/log.txt?x=15c74fb2980000
kernel config: https://syzkaller.appspot.com/x/.config?x=ea870d7ee19c0a5c
dashboard link: https://syzkaller.appspot.com/bug?extid=041606b95d0537b167df
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
Unfortunately, I don't have any reproducer for this issue yet.
Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/d5444b7453a0/disk-83655231.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/76ae736a43a3/vmlinux-83655231.xz
kernel image: https://storage.googleapis.com/syzbot-assets/2540e00ba747/bzImage-83655231.xz
IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+041606...@syzkaller.appspotmail.com
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: Tasks blocked on level-0 rcu_node (CPUs 0-1): P3699/2:b..l
(detected by 1, t=10502 jiffies, g=6265, q=325)
task:syz-executor.3 state:R running task stack:25592 pid: 3699 ppid: 3524 flags:0x00004002
Call Trace:
<TASK>
context_switch kernel/sched/core.c:5030 [inline]
__schedule+0x12c4/0x45b0 kernel/sched/core.c:6376
preempt_schedule_irq+0xf7/0x1c0 kernel/sched/core.c:6780
irqentry_exit+0x53/0x80 kernel/entry/common.c:432
asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:lock_acquire+0x252/0x4f0 kernel/locking/lockdep.c:5627
Code: 2b 00 74 08 4c 89 f7 e8 fc 7c 67 00 f6 44 24 61 02 0f 85 84 01 00 00 41 f7 c7 00 02 00 00 74 01 fb 48 c7 44 24 40 0e 36 e0 45 <4b> c7 44 25 00 00 00 00 00 43 c7 44 25 09 00 00 00 00 43 c7 44 25
RSP: 0018:ffffc900044cf940 EFLAGS: 00000206
RAX: 0000000000000001 RBX: 1ffff92000899f34 RCX: 1ffff92000899ed4
RDX: dffffc0000000000 RSI: ffffffff8a8b3c20 RDI: ffffffff8ad8f680
RBP: ffffc900044cfa88 R08: dffffc0000000000 R09: fffffbfff1f7f019
R10: 0000000000000000 R11: dffffc0000000001 R12: 1ffff92000899f30
R13: dffffc0000000000 R14: ffffc900044cf9a0 R15: 0000000000000246
rcu_lock_acquire+0x20/0x30 include/linux/rcupdate.h:312
rcu_read_lock include/linux/rcupdate.h:739 [inline]
percpu_ref_tryget_many include/linux/percpu-refcount.h:241 [inline]
percpu_ref_tryget+0x10/0x140 include/linux/percpu-refcount.h:266
css_tryget include/linux/cgroup.h:355 [inline]
get_mem_cgroup_from_objcg+0xa3/0x120 mm/memcontrol.c:2812
obj_cgroup_uncharge_pages mm/memcontrol.c:2989 [inline]
__memcg_kmem_uncharge_page+0x73/0x130 mm/memcontrol.c:3080
free_pages_prepare mm/page_alloc.c:1332 [inline]
free_pcp_prepare mm/page_alloc.c:1391 [inline]
free_unref_page_prepare+0x11d/0xcf0 mm/page_alloc.c:3317
free_unref_page+0x95/0x2d0 mm/page_alloc.c:3396
__vunmap+0x8d4/0xa20 mm/vmalloc.c:2621
memslot_rmap_free arch/x86/kvm/x86.c:11747 [inline]
kvm_arch_free_memslot+0x3c/0x150 arch/x86/kvm/x86.c:11756
kvm_free_memslot arch/x86/kvm/../../../virt/kvm/kvm_main.c:934 [inline]
kvm_free_memslots arch/x86/kvm/../../../virt/kvm/kvm_main.c:948 [inline]
kvm_destroy_vm arch/x86/kvm/../../../virt/kvm/kvm_main.c:1254 [inline]
kvm_put_kvm+0x1183/0x1840 arch/x86/kvm/../../../virt/kvm/kvm_main.c:1283
kvm_vcpu_release+0x53/0x60 arch/x86/kvm/../../../virt/kvm/kvm_main.c:3643
__fput+0x3bf/0x890 fs/file_table.c:280
task_work_run+0x129/0x1a0 kernel/task_work.c:164
tracehook_notify_resume include/linux/tracehook.h:189 [inline]
exit_to_user_mode_loop+0x106/0x130 kernel/entry/common.c:181
exit_to_user_mode_prepare+0xb1/0x140 kernel/entry/common.c:214
__syscall_exit_to_user_mode_work kernel/entry/common.c:296 [inline]
syscall_exit_to_user_mode+0x5d/0x240 kernel/entry/common.c:307
do_syscall_64+0x47/0xb0 arch/x86/entry/common.c:86
entry_SYSCALL_64_after_hwframe+0x66/0xd0
RIP: 0033:0x7f54f7d95dda
RSP: 002b:00007ffdf039fec0 EFLAGS: 00000293 ORIG_RAX: 0000000000000003
RAX: 0000000000000000 RBX: 000000000000000b RCX: 00007f54f7d95dda
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 000000000000000a
RBP: 00007f54f7ec7980 R08: 0000001b30320000 R09: 0000000000001291
R10: 000000008120b6f7 R11: 0000000000000293 R12: 000000000000fad1
R13: 00007f54f7ec5f8c R14: 00007ffdf039ffc0 R15: 0000000000000032
</TASK>
rcu: rcu_preempt kthread starved for 9924 jiffies! g6265 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1
rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt state:R running task stack:27288 pid: 15 ppid: 2 flags:0x00004000
Call Trace:
<TASK>
context_switch kernel/sched/core.c:5030 [inline]
__schedule+0x12c4/0x45b0 kernel/sched/core.c:6376
schedule+0x11b/0x1f0 kernel/sched/core.c:6459
schedule_timeout+0x1b9/0x300 kernel/time/timer.c:1914
rcu_gp_fqs_loop+0x2bf/0x1080 kernel/rcu/tree.c:1972
rcu_gp_kthread+0xa4/0x360 kernel/rcu/tree.c:2145
kthread+0x3f6/0x4f0 kernel/kthread.c:334
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:300
</TASK>
rcu: Stack dump where RCU GP kthread last ran:
NMI backtrace for cpu 1
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.15.159-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024
Call Trace:
<IRQ>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x1e3/0x2d0 lib/dump_stack.c:106
nmi_cpu_backtrace+0x46a/0x4a0 lib/nmi_backtrace.c:111
nmi_trigger_cpumask_backtrace+0x181/0x2a0 lib/nmi_backtrace.c:62
trigger_single_cpu_backtrace include/linux/nmi.h:166 [inline]
rcu_check_gp_kthread_starvation+0x1d2/0x240 kernel/rcu/tree_stall.h:487
print_other_cpu_stall+0x137a/0x14d0 kernel/rcu/tree_stall.h:592
check_cpu_stall kernel/rcu/tree_stall.h:745 [inline]
rcu_pending kernel/rcu/tree.c:3932 [inline]
rcu_sched_clock_irq+0xa38/0x1150 kernel/rcu/tree.c:2619
update_process_times+0x196/0x200 kernel/time/timer.c:1818
tick_sched_handle kernel/time/tick-sched.c:254 [inline]
tick_sched_timer+0x386/0x550 kernel/time/tick-sched.c:1473
__run_hrtimer kernel/time/hrtimer.c:1686 [inline]
__hrtimer_run_queues+0x55b/0xcf0 kernel/time/hrtimer.c:1750
hrtimer_interrupt+0x392/0x980 kernel/time/hrtimer.c:1812
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1085 [inline]
__sysvec_apic_timer_interrupt+0x139/0x470 arch/x86/kernel/apic/apic.c:1102
sysvec_apic_timer_interrupt+0x8c/0xb0 arch/x86/kernel/apic/apic.c:1096
</IRQ>
<TASK>
asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:native_save_fl arch/x86/include/asm/irqflags.h:22 [inline]
RIP: 0010:arch_local_save_flags arch/x86/include/asm/irqflags.h:70 [inline]
RIP: 0010:arch_irqs_disabled arch/x86/include/asm/irqflags.h:132 [inline]
RIP: 0010:acpi_safe_halt drivers/acpi/processor_idle.c:110 [inline]
RIP: 0010:acpi_idle_do_entry+0x10f/0x340 drivers/acpi/processor_idle.c:570
Code: ed 58 f7 48 83 e3 08 0f 85 0a 01 00 00 4c 8d 74 24 20 e8 24 69 5f f7 0f 1f 44 00 00 e8 1a e9 58 f7 0f 00 2d 33 aa bb 00 fb f4 <4c> 89 f3 48 c1 eb 03 42 80 3c 3b 00 74 08 4c 89 f7 e8 8b b7 a2 f7
RSP: 0018:ffffc90000d67b00 EFLAGS: 000002d3
RAX: ffffffff8a2773a6 RBX: 0000000000000000 RCX: ffff88813fe68000
RDX: 0000000000000000 RSI: ffffffff8a8b2980 RDI: ffffffff8ad8f680
RBP: ffffc90000d67b90 R08: ffffffff8186dcf0 R09: ffffed1027fcd001
R10: 0000000000000000 R11: dffffc0000000001 R12: 1ffff920001acf60
R13: ffff888145e52004 R14: ffffc90000d67b20 R15: dffffc0000000000
acpi_idle_enter+0x352/0x4f0 drivers/acpi/processor_idle.c:705
cpuidle_enter_state+0x521/0xef0 drivers/cpuidle/cpuidle.c:237
cpuidle_enter+0x59/0x90 drivers/cpuidle/cpuidle.c:351
call_cpuidle kernel/sched/idle.c:158 [inline]
cpuidle_idle_call kernel/sched/idle.c:239 [inline]
do_idle+0x3e4/0x670 kernel/sched/idle.c:306
cpu_startup_entry+0x14/0x20 kernel/sched/idle.c:403
start_secondary+0x371/0x500 arch/x86/kernel/smpboot.c:281
secondary_startup_64_no_verify+0xb1/0xbb
</TASK>
---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.
syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title
If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)
If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report
If you want to undo deduplication, reply with:
#syz undup
Reply all
Reply to author
Forward
0 new messages