INFO: rcu detected stall in run_timer_softirq

[syzbot]

Hello,

syzbot found the following crash on:

HEAD commit: 84f5ad46 Linux 4.14.162
git tree: linux-4.14.y
console output: https://syzkaller.appspot.com/x/log.txt?x=16756115e00000
kernel config: https://syzkaller.appspot.com/x/.config?x=2eeb394ad524f8c4
dashboard link: https://syzkaller.appspot.com/bug?extid=a00590c16bd367965f90
compiler: gcc (GCC) 9.0.0 20181231 (experimental)

Unfortunately, I don't have any reproducer for this crash yet.

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+a00590...@syzkaller.appspotmail.com

INFO: rcu_sched detected stalls on CPUs/tasks:
(detected by 0, t=10502 jiffies, g=21089, c=21088, q=21)
All QSes seen, last rcu_sched kthread activity 10495
(4295011588-4295001093), jiffies_till_next_fqs=1, root ->qsmask 0x0
syz-executor.0 R running task 26736 2465 8332 0x0000000c
Call Trace:
<IRQ>
sched_show_task kernel/sched/core.c:5168 [inline]
sched_show_task.cold+0x2f0/0x351 kernel/sched/core.c:5143
print_other_cpu_stall kernel/rcu/tree.c:1501 [inline]
check_cpu_stall kernel/rcu/tree.c:1616 [inline]
__rcu_pending kernel/rcu/tree.c:3390 [inline]
rcu_pending kernel/rcu/tree.c:3452 [inline]
rcu_check_callbacks.cold+0xcb1/0xd0a kernel/rcu/tree.c:2792
update_process_times+0x31/0x70 kernel/time/timer.c:1590
tick_sched_handle+0x85/0x160 kernel/time/tick-sched.c:161
tick_sched_timer+0x43/0x130 kernel/time/tick-sched.c:1219
__run_hrtimer kernel/time/hrtimer.c:1223 [inline]
__hrtimer_run_queues+0x270/0xbc0 kernel/time/hrtimer.c:1287
hrtimer_interrupt+0x1d8/0x5d0 kernel/time/hrtimer.c:1321
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1075 [inline]
smp_apic_timer_interrupt+0x11c/0x5e0 arch/x86/kernel/apic/apic.c:1100
apic_timer_interrupt+0x96/0xa0 arch/x86/entry/entry_64.S:792
RIP: 0010:__raw_spin_unlock_irq include/linux/spinlock_api_smp.h:169
[inline]
RIP: 0010:_raw_spin_unlock_irq+0x54/0x90 kernel/locking/spinlock.c:200
RSP: 0018:ffff8880aec07df0 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff10
RAX: 1ffffffff0fe2d2b RBX: ffff888055e6e828 RCX: 1ffff1100a53597e
RDX: dffffc0000000000 RSI: ffff8880529acbd0 RDI: ffff8880529acb7c
RBP: ffff8880aec07df8 R08: ffff8880529ac300 R09: ffff8880529acbf0
R10: 0000000000000000 R11: 0000000000000000 R12: ffff8880aec239c0
R13: ffff888055e6e7c0 R14: ffff8880aec239c0 R15: dffffc0000000000
expire_timers kernel/time/timer.c:1317 [inline]
__run_timers kernel/time/timer.c:1636 [inline]
__run_timers kernel/time/timer.c:1604 [inline]
run_timer_softirq+0x5a5/0x1520 kernel/time/timer.c:1649
__do_softirq+0x244/0x9a0 kernel/softirq.c:288
invoke_softirq kernel/softirq.c:368 [inline]
irq_exit+0x160/0x1b0 kernel/softirq.c:409
exiting_irq arch/x86/include/asm/apic.h:648 [inline]
smp_apic_timer_interrupt+0x146/0x5e0 arch/x86/kernel/apic/apic.c:1102
apic_timer_interrupt+0x96/0xa0 arch/x86/entry/entry_64.S:792
</IRQ>
RIP: 0010:__sanitizer_cov_trace_pc+0x41/0x60 kernel/kcov.c:87
RSP: 0018:ffff888055edf3b8 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff10
RAX: 0000000000040000 RBX: ffffffff88340fe0 RCX: ffffc90007e34000
RDX: 000000000003ffff RSI: ffffffff830ea4ec RDI: ffffffff87049140
RBP: ffff888055edf3b8 R08: 0000000000000000 R09: 0000000000000003
R10: 0000000000000000 R11: ffff8880529ac300 R12: 0000000000000001
R13: ffffffff87049100 R14: ffffffff87049140 R15: 0000000000000000
check_preemption_disabled+0x1c/0x250 lib/smp_processor_id.c:14
debug_smp_processor_id+0x1c/0x20 lib/smp_processor_id.c:57
rcu_lockdep_current_cpu_online kernel/rcu/tree.c:1186 [inline]
rcu_lockdep_current_cpu_online+0x3a/0x140 kernel/rcu/tree.c:1177
rcu_read_lock_sched_held+0x97/0x130 kernel/rcu/update.c:113
trace_ext4_mark_inode_dirty include/trace/events/ext4.h:248 [inline]
ext4_mark_inode_dirty+0x6bd/0x860 fs/ext4/inode.c:5919
ext4_dirty_inode+0x73/0xa0 fs/ext4/inode.c:5959
__mark_inode_dirty+0x54c/0x1040 fs/fs-writeback.c:2144
mark_inode_dirty include/linux/fs.h:2019 [inline]
dquot_alloc_space include/linux/quotaops.h:315 [inline]
dquot_alloc_block include/linux/quotaops.h:332 [inline]
ext4_xattr_block_set+0x192c/0x2b00 fs/ext4/xattr.c:1996
ext4_xattr_set_handle+0x85a/0xda0 fs/ext4/xattr.c:2406
__ext4_set_acl+0x2ef/0x510 fs/ext4/acl.c:214
ext4_set_acl+0x20e/0x400 fs/ext4/acl.c:254
set_posix_acl fs/posix_acl.c:874 [inline]
set_posix_acl+0x1e0/0x290 fs/posix_acl.c:857
posix_acl_xattr_set+0x88/0x120 fs/posix_acl.c:892
__vfs_setxattr+0xd8/0x130 fs/xattr.c:150
__vfs_setxattr_noperm+0x102/0x3c0 fs/xattr.c:181
vfs_setxattr+0xc5/0xf0 fs/xattr.c:224
setxattr+0x1de/0x350 fs/xattr.c:453
path_setxattr+0x11f/0x140 fs/xattr.c:472
SYSC_setxattr fs/xattr.c:487 [inline]
SyS_setxattr+0x3b/0x50 fs/xattr.c:483
do_syscall_64+0x1e8/0x640 arch/x86/entry/common.c:292
entry_SYSCALL_64_after_hwframe+0x42/0xb7
RIP: 0033:0x45af49
RSP: 002b:00007f321aec4c78 EFLAGS: 00000246 ORIG_RAX: 00000000000000bc
RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 000000000045af49
RDX: 0000000020000c40 RSI: 00000000200005c0 RDI: 0000000020000580
RBP: 000000000075bf20 R08: 0000000000000000 R09: 0000000000000000
R10: 000000000000005c R11: 0000000000000246 R12: 00007f321aec56d4
R13: 00000000004c1ca0 R14: 00000000004e5cd0 R15: 00000000ffffffff
rcu_sched kthread starved for 10495 jiffies! g21089 c21088 f0x2
RCU_GP_WAIT_FQS(3) ->state=0x0 ->cpu=1
rcu_sched R running task 29824 9 2 0x80000000
Call Trace:
context_switch kernel/sched/core.c:2808 [inline]
__schedule+0x7b8/0x1cd0 kernel/sched/core.c:3384
schedule+0x92/0x1c0 kernel/sched/core.c:3428
schedule_timeout+0x43e/0xe10 kernel/time/timer.c:1746
rcu_gp_kthread+0xbf4/0x1ec0 kernel/rcu/tree.c:2255
kthread+0x319/0x430 kernel/kthread.c:232
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:404


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

[syzbot]

Hello,

syzbot found the following crash on:

HEAD commit: db5b9190 Linux 4.19.96
git tree: linux-4.19.y
console output: https://syzkaller.appspot.com/x/log.txt?x=1748cad1e00000
kernel config: https://syzkaller.appspot.com/x/.config?x=7a24894346f71303
dashboard link: https://syzkaller.appspot.com/bug?extid=086954cc3593d0e10490

compiler: gcc (GCC) 9.0.0 20181231 (experimental)

Unfortunately, I don't have any reproducer for this crash yet.

IMPORTANT: if you fix the bug, please add the following tag to the commit:

Reported-by: syzbot+086954...@syzkaller.appspotmail.com

audit: type=1400 audit(1579267083.127:175): avc: denied { map } for
pid=20097 comm="syz-executor.5"
path=2F616E6F6E5F6875676570616765202864656C6574656429 dev="hugetlbfs"
ino=259286 scontext=unconfined_u:system_r:insmod_t:s0-s0:c0.c1023
tcontext=unconfined_u:object_r:hugetlbfs_t:s0 tclass=file permissive=1
rcu: INFO: rcu_preempt self-detected stall on CPU
rcu: 0-...!: (1 GPs behind) idle=83a/1/0x4000000000000004
softirq=185952/185953 fqs=6
rcu: (t=10501 jiffies g=225217 q=871)
rcu: rcu_preempt kthread starved for 10399 jiffies! g225217 f0x0
RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1
rcu: RCU grace-period kthread stack dump:
rcu_preempt R running task 29072 10 2 0x80000000
Call Trace:
context_switch kernel/sched/core.c:2826 [inline]
__schedule+0x866/0x1dc0 kernel/sched/core.c:3515
schedule+0x92/0x1c0 kernel/sched/core.c:3559
schedule_timeout+0x4db/0xfc0 kernel/time/timer.c:1806
rcu_gp_kthread+0xd5c/0x2190 kernel/rcu/tree.c:2202
kthread+0x354/0x420 kernel/kthread.c:246
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:415
NMI backtrace for cpu 0
CPU: 0 PID: 20121 Comm: syz-executor.5 Not tainted 4.19.96-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
<IRQ>
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x197/0x210 lib/dump_stack.c:118
nmi_cpu_backtrace.cold+0x63/0xa4 lib/nmi_backtrace.c:101
nmi_trigger_cpumask_backtrace+0x1b0/0x1f8 lib/nmi_backtrace.c:62
arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38
trigger_single_cpu_backtrace include/linux/nmi.h:164 [inline]
rcu_dump_cpu_stacks+0x189/0x1d5 kernel/rcu/tree.c:1340
print_cpu_stall kernel/rcu/tree.c:1478 [inline]
check_cpu_stall kernel/rcu/tree.c:1550 [inline]
__rcu_pending kernel/rcu/tree.c:3293 [inline]
rcu_pending kernel/rcu/tree.c:3336 [inline]
rcu_check_callbacks.cold+0x5e3/0xd90 kernel/rcu/tree.c:2682
update_process_times+0x32/0x80 kernel/time/timer.c:1638
tick_sched_handle+0xa2/0x190 kernel/time/tick-sched.c:164
tick_sched_timer+0x47/0x130 kernel/time/tick-sched.c:1274
__run_hrtimer kernel/time/hrtimer.c:1401 [inline]
__hrtimer_run_queues+0x33b/0xdc0 kernel/time/hrtimer.c:1463
hrtimer_interrupt+0x314/0x770 kernel/time/hrtimer.c:1521
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1067 [inline]
smp_apic_timer_interrupt+0x111/0x550 arch/x86/kernel/apic/apic.c:1092
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:893

RIP: 0010:__raw_spin_unlock_irq include/linux/spinlock_api_smp.h:169
[inline]

RIP: 0010:_raw_spin_unlock_irq+0x54/0x90 kernel/locking/spinlock.c:192
Code: c0 d8 5b f2 88 48 ba 00 00 00 00 00 fc ff df 48 c1 e8 03 80 3c 10 00
75 33 48 83 3d 1d 63 9c 01 00 74 20 fb 66 0f 1f 44 00 00 <bf> 01 00 00 00
e8 b2 e7 f2 f9 65 8b 05 fb f4 ab 78 85 c0 74 06 41
RSP: 0018:ffff8880ae807dd0 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff13
RAX: 1ffffffff11e4b7b RBX: ffff88808935a590 RCX: 1ffff1100498456c
RDX: dffffc0000000000 RSI: ffff888024c22b40 RDI: ffff888024c22b3c
RBP: ffff8880ae807dd8 R08: ffff888024c222c0 R09: ffff888024c22b60
R10: 0000000000000000 R11: 0000000000000000 R12: ffff8880ae823a00
R13: dffffc0000000000 R14: ffffffff85d9f470 R15: ffff8880ae823a00
expire_timers kernel/time/timer.c:1362 [inline]
__run_timers kernel/time/timer.c:1684 [inline]
__run_timers kernel/time/timer.c:1652 [inline]
run_timer_softirq+0x644/0x16a0 kernel/time/timer.c:1697
__do_softirq+0x25c/0x921 kernel/softirq.c:292
invoke_softirq kernel/softirq.c:372 [inline]
irq_exit+0x180/0x1d0 kernel/softirq.c:412
exiting_irq arch/x86/include/asm/apic.h:536 [inline]
smp_apic_timer_interrupt+0x13b/0x550 arch/x86/kernel/apic/apic.c:1094
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:893
</IRQ>
RIP: 0010:arch_local_irq_restore arch/x86/include/asm/paravirt.h:789
[inline]
RIP: 0010:lock_release+0x47a/0xa30 kernel/locking/lockdep.c:3925
Code: 00 00 00 00 00 48 c1 e8 03 80 3c 10 00 0f 85 a7 03 00 00 48 83 3d 2d
14 a0 07 00 0f 84 65 02 00 00 48 8b bd 68 ff ff ff 57 9d <0f> 1f 44 00 00
48 b8 00 00 00 00 00 fc ff df 48 c7 04 03 00 00 00
RSP: 0018:ffff88820b6ff5c0 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff13
RAX: 1ffffffff11e4b79 RBX: 1ffff110416dfebe RCX: 1ffff11004984566
RDX: dffffc0000000000 RSI: 0000000000000002 RDI: 0000000000000286
RBP: ffff88820b6ff678 R08: ffff888024c222c0 R09: 0000000000000001
R10: ffffed1015d04732 R11: ffff8880ae823993 R12: ffff888024c222c0
R13: ffffffff81a8c5c3 R14: 0000000000000003 R15: ffff88820b6ff650
rcu_lock_release include/linux/rcupdate.h:247 [inline]
rcu_read_unlock include/linux/rcupdate.h:681 [inline]
__unlock_page_memcg+0x70/0x100 mm/memcontrol.c:1953
unlock_page_memcg+0x2c/0x40 mm/memcontrol.c:1962
page_remove_file_rmap mm/rmap.c:1249 [inline]
page_remove_rmap+0x57b/0x12c0 mm/rmap.c:1300
zap_pte_range mm/memory.c:1338 [inline]
zap_pmd_range mm/memory.c:1440 [inline]
zap_pud_range mm/memory.c:1469 [inline]
zap_p4d_range mm/memory.c:1490 [inline]
unmap_page_range+0xda2/0x1fb0 mm/memory.c:1511
unmap_single_vma+0x19d/0x300 mm/memory.c:1556
unmap_vmas+0xae/0x180 mm/memory.c:1586
exit_mmap+0x2c2/0x530 mm/mmap.c:3091
__mmput kernel/fork.c:1005 [inline]
mmput+0x15f/0x4c0 kernel/fork.c:1026
exit_mm kernel/exit.c:546 [inline]
do_exit+0xb1e/0x30d0 kernel/exit.c:867
do_group_exit+0x135/0x370 kernel/exit.c:983
get_signal+0x3ec/0x1fc0 kernel/signal.c:2578
do_signal+0x95/0x1960 arch/x86/kernel/signal.c:821
exit_to_usermode_loop+0x244/0x2c0 arch/x86/entry/common.c:163
prepare_exit_to_usermode arch/x86/entry/common.c:198 [inline]
syscall_return_slowpath arch/x86/entry/common.c:271 [inline]
do_syscall_64+0x53d/0x620 arch/x86/entry/common.c:296
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x45aff9
Code: ad b6 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7
48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff
ff 0f 83 7b b6 fb ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007f63eac1bc78 EFLAGS: 00000246 ORIG_RAX: 0000000000000009
RAX: 0000000020000000 RBX: 00007f63eac1c6d4 RCX: 000000000045aff9
RDX: 0000000000000000 RSI: 0000000000ff5000 RDI: 0000000020000000
RBP: 000000000075bf20 R08: ffffffffffffffff R09: 0000000000000000
R10: 000000000005c831 R11: 0000000000000246 R12: 00000000ffffffff
R13: 0000000000000710 R14: 00000000004c8474 R15: 000000000075bf2c
netlink: 20 bytes leftover after parsing attributes in process
`syz-executor.1'.

[syzbot]

Auto-closing this bug as obsolete.
Crashes did not happen for a while, no reproducer and no activity.

[syzbot]