[syzbot] INFO: rcu detected stall in gc

syzbot

unread,

Mar 20, 2022, 8:02:23 AM3/20/22

to linux-...@vger.kernel.org, syzkall...@googlegroups.com, tg...@linutronix.de

Hello,

syzbot found the following issue on:

HEAD commit: 91265a6da44d Add linux-next specific files for 20220303
git tree: linux-next
console output: https://syzkaller.appspot.com/x/log.txt?x=157605d5700000
kernel config: https://syzkaller.appspot.com/x/.config?x=617f79440a35673a
dashboard link: https://syzkaller.appspot.com/bug?extid=eec403943a2a2455adaa
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=143195d9700000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=170736a9700000

Bisection is inconclusive: the issue happens on the oldest tested release.

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=141bc691700000
final oops: https://syzkaller.appspot.com/x/report.txt?x=161bc691700000
console output: https://syzkaller.appspot.com/x/log.txt?x=121bc691700000

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+eec403...@syzkaller.appspotmail.com

rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 0-...!: (1 GPs behind) idle=a59/1/0x4000000000000000 softirq=5468/5472 fqs=5
(detected by 1, t=10502 jiffies, g=4825, q=140)
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 3644 Comm: kworker/0:1 Not tainted 5.17.0-rc6-next-20220303-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: events_power_efficient gc_worker
RIP: 0010:hlock_class kernel/locking/lockdep.c:240 [inline]
RIP: 0010:__lock_acquire+0x1460/0x56c0 kernel/locking/lockdep.c:5056
Code: 0f b7 db be 08 00 00 00 48 89 d8 48 c1 f8 06 48 8d 3c c5 80 59 01 90 e8 fe 54 67 00 48 0f a3 1d f6 ed a3 0e 0f 83 2c 06 00 00 <48> 8d 1c 5b 48 c1 e3 06 48 81 c3 a0 5d 01 90 48 8d 7b 40 48 b8 00
RSP: 0018:ffffc90000007be0 EFLAGS: 00000047
RAX: 0000000000000001 RBX: 0000000000000029 RCX: ffffffff815d6b82
RDX: fffffbfff2002b31 RSI: 0000000000000008 RDI: ffffffff90015980
RBP: ffff88801bc4a842 R08: 0000000000000000 R09: ffffffff90015987
R10: fffffbfff2002b30 R11: 0000000000000001 R12: ffff88801bc4a820
R13: ffff88801bc49d40 R14: 0000000000000001 R15: 3d0b7b89c89fff91
FS: 0000000000000000(0000) GS:ffff8880b9c00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000020000600 CR3: 0000000076bbe000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<IRQ>
lock_acquire kernel/locking/lockdep.c:5672 [inline]
lock_acquire+0x1ab/0x510 kernel/locking/lockdep.c:5637
__raw_spin_lock_irq include/linux/spinlock_api_smp.h:119 [inline]
_raw_spin_lock_irq+0x32/0x50 kernel/locking/spinlock.c:170
__run_hrtimer kernel/time/hrtimer.c:1689 [inline]
__hrtimer_run_queues+0x243/0xe50 kernel/time/hrtimer.c:1749
hrtimer_interrupt+0x31c/0x790 kernel/time/hrtimer.c:1811
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1086 [inline]
__sysvec_apic_timer_interrupt+0x146/0x530 arch/x86/kernel/apic/apic.c:1103
sysvec_apic_timer_interrupt+0x8e/0xc0 arch/x86/kernel/apic/apic.c:1097
</IRQ>
<TASK>
asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:preempt_count arch/x86/include/asm/preempt.h:27 [inline]
RIP: 0010:check_kcov_mode kernel/kcov.c:166 [inline]
RIP: 0010:__sanitizer_cov_trace_pc+0x0/0x60 kernel/kcov.c:200
Code: 48 89 ef 5d e9 51 f3 4a 00 5d be 03 00 00 00 e9 96 96 74 02 66 0f 1f 44 00 00 48 8b be b0 01 00 00 e8 b4 ff ff ff 31 c0 c3 90 <65> 8b 05 29 49 89 7e 89 c1 48 8b 34 24 81 e1 00 01 00 00 65 48 8b
RSP: 0018:ffffc90003ccfc30 EFLAGS: 00000293
RAX: 0000000000000000 RBX: 0000000000000200 RCX: 0000000000000000
RDX: ffff88801bc49d40 RSI: ffffffff87821df7 RDI: 0000000000000003
RBP: 0000000000000000 R08: 0000000000000000 R09: ffffffff90015a17
R10: ffffffff87821ded R11: 0000000000000001 R12: dffffc0000000000
R13: 0000000000000000 R14: 0000000000040000 R15: 0000000000040000
__seqprop_spinlock_sequence include/linux/seqlock.h:277 [inline]
nf_conntrack_get_ht include/net/netfilter/nf_conntrack.h:331 [inline]
gc_worker+0x24d/0x12b0 net/netfilter/nf_conntrack_core.c:1441
process_one_work+0x996/0x1610 kernel/workqueue.c:2289
worker_thread+0x665/0x1080 kernel/workqueue.c:2436
kthread+0x2e9/0x3a0 kernel/kthread.c:376
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
</TASK>
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.570 msecs
rcu: rcu_preempt kthread starved for 10492 jiffies! g4825 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1
rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt state:R running task stack:28752 pid: 16 ppid: 2 flags:0x00004000
Call Trace:
<TASK>
context_switch kernel/sched/core.c:5043 [inline]
__schedule+0xa94/0x4910 kernel/sched/core.c:6352
schedule+0xd2/0x1f0 kernel/sched/core.c:6424
schedule_timeout+0x14a/0x2a0 kernel/time/timer.c:1881
rcu_gp_fqs_loop+0x186/0x810 kernel/rcu/tree.c:1999
rcu_gp_kthread+0x1de/0x320 kernel/rcu/tree.c:2172
kthread+0x2e9/0x3a0 kernel/kthread.c:376
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
</TASK>
rcu: Stack dump where RCU GP kthread last ran:
NMI backtrace for cpu 1
CPU: 1 PID: 45 Comm: kworker/u4:2 Not tainted 5.17.0-rc6-next-20220303-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: events_unbound toggle_allocation_gate
Call Trace:
<IRQ>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:106
nmi_cpu_backtrace.cold+0x47/0x144 lib/nmi_backtrace.c:111
nmi_trigger_cpumask_backtrace+0x1e6/0x230 lib/nmi_backtrace.c:62
trigger_single_cpu_backtrace include/linux/nmi.h:164 [inline]
rcu_check_gp_kthread_starvation.cold+0x1fb/0x200 kernel/rcu/tree_stall.h:516
print_other_cpu_stall kernel/rcu/tree_stall.h:621 [inline]
check_cpu_stall kernel/rcu/tree_stall.h:767 [inline]
rcu_pending kernel/rcu/tree.c:3960 [inline]
rcu_sched_clock_irq+0x21ae/0x22a0 kernel/rcu/tree.c:2660
update_process_times+0x16d/0x200 kernel/time/timer.c:1785
tick_sched_handle+0x9b/0x180 kernel/time/tick-sched.c:243
tick_sched_timer+0xee/0x120 kernel/time/tick-sched.c:1473
__run_hrtimer kernel/time/hrtimer.c:1685 [inline]
__hrtimer_run_queues+0x1c0/0xe50 kernel/time/hrtimer.c:1749
hrtimer_interrupt+0x31c/0x790 kernel/time/hrtimer.c:1811
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1086 [inline]
__sysvec_apic_timer_interrupt+0x146/0x530 arch/x86/kernel/apic/apic.c:1103
sysvec_apic_timer_interrupt+0x8e/0xc0 arch/x86/kernel/apic/apic.c:1097
</IRQ>
<TASK>
asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:check_kcov_mode+0xf/0x40 kernel/kcov.c:166
Code: 7c 24 08 e8 b3 9b 4b 00 e9 61 fd ff ff cc cc cc cc cc cc cc cc cc cc cc cc cc cc 65 8b 05 79 4d 89 7e 89 c2 81 e2 00 01 00 00 <a9> 00 01 ff 00 74 10 31 c0 85 d2 74 15 8b 96 ac 15 00 00 85 d2 74
RSP: 0018:ffffc90000b679c8 EFLAGS: 00000246
RAX: 0000000000000001 RBX: ffff8880b9c42380 RCX: 0000000000000001
RDX: 0000000000000000 RSI: ffff888016df8000 RDI: 0000000000000003
RBP: 0000000000000003 R08: 0000000000000000 R09: 0000000000000001
R10: ffffffff816d99aa R11: 0000000000000000 R12: ffffed1017388471
R13: 0000000000000000 R14: ffff8880b9c42388 R15: 0000000000000001
write_comp_data kernel/kcov.c:221 [inline]
__sanitizer_cov_trace_const_cmp4+0x1c/0x70 kernel/kcov.c:287
csd_lock_wait kernel/smp.c:443 [inline]
smp_call_function_many_cond+0x50a/0xc90 kernel/smp.c:972
on_each_cpu_cond_mask+0x56/0xa0 kernel/smp.c:1138
on_each_cpu include/linux/smp.h:71 [inline]
text_poke_sync arch/x86/kernel/alternative.c:1146 [inline]
text_poke_bp_batch+0x3e9/0x6b0 arch/x86/kernel/alternative.c:1387
text_poke_flush arch/x86/kernel/alternative.c:1504 [inline]
text_poke_flush arch/x86/kernel/alternative.c:1501 [inline]
text_poke_finish+0x16/0x30 arch/x86/kernel/alternative.c:1511
arch_jump_label_transform_apply+0x13/0x20 arch/x86/kernel/jump_label.c:146
jump_label_update+0x32f/0x410 kernel/jump_label.c:830
static_key_enable_cpuslocked+0x1b1/0x260 kernel/jump_label.c:177
static_key_enable+0x16/0x20 kernel/jump_label.c:190
toggle_allocation_gate mm/kfence/core.c:735 [inline]
toggle_allocation_gate+0x100/0x390 mm/kfence/core.c:727
process_one_work+0x996/0x1610 kernel/workqueue.c:2289
worker_thread+0x665/0x1080 kernel/workqueue.c:2436
kthread+0x2e9/0x3a0 kernel/kthread.c:376
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
</TASK>
----------------
Code disassembly (best guess):
0: 0f b7 db movzwl %bx,%ebx
3: be 08 00 00 00 mov $0x8,%esi
8: 48 89 d8 mov %rbx,%rax
b: 48 c1 f8 06 sar $0x6,%rax
f: 48 8d 3c c5 80 59 01 lea -0x6ffea680(,%rax,8),%rdi
16: 90
17: e8 fe 54 67 00 callq 0x67551a
1c: 48 0f a3 1d f6 ed a3 bt %rbx,0xea3edf6(%rip) # 0xea3ee1a
23: 0e
24: 0f 83 2c 06 00 00 jae 0x656
* 2a: 48 8d 1c 5b lea (%rbx,%rbx,2),%rbx <-- trapping instruction
2e: 48 c1 e3 06 shl $0x6,%rbx
32: 48 81 c3 a0 5d 01 90 add $0xffffffff90015da0,%rbx
39: 48 8d 7b 40 lea 0x40(%rbx),%rdi
3d: 48 rex.W
3e: b8 .byte 0xb8

---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
For information about bisection process see: https://goo.gl/tpsmEJ#bisection
syzbot can test patches for this issue, for details see:
https://goo.gl/tpsmEJ#testing-patches

Hillf Danton

unread,

Mar 20, 2022, 11:22:09 AM3/20/22

to syzbot, linux-...@vger.kernel.org, syzkall...@googlegroups.com

On Sun, 20 Mar 2022 05:02:23 -0700

To fix rcu stall, move CPU hog out of the section under rcu_read_lock.

Hillf

#syz test: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/ 91265a6da44d

--- x/net/netfilter/nf_conntrack_core.c
+++ y/net/netfilter/nf_conntrack_core.c
@@ -1435,6 +1435,8 @@ static void gc_worker(struct work_struct
struct hlist_nulls_head *ct_hash;
struct hlist_nulls_node *n;
struct nf_conn *tmp;
+ int bailout = 0;
+ unsigned long start;

rcu_read_lock();

@@ -1444,10 +1446,15 @@ static void gc_worker(struct work_struct
break;
}

+ start = jiffies;
hlist_nulls_for_each_entry_rcu(h, n, &ct_hash[i], hnnode) {
struct nf_conntrack_net *cnet;
struct net *net;

+ if (start != jiffies) {
+ bailout = 1;
+ break;
+ }
tmp = nf_ct_tuplehash_to_ctrack(h);

if (test_bit(IPS_OFFLOAD_BIT, &tmp->status)) {
@@ -1496,6 +1503,8 @@ static void gc_worker(struct work_struct
next_run = 0;
break;
}
+ if (bailout)
+ i--;
} while (i < hashsz);

if (gc_work->exiting)
--

syzbot

unread,

Mar 20, 2022, 11:33:07 AM3/20/22

to hda...@sina.com, linux-...@vger.kernel.org, syzkall...@googlegroups.com

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
INFO: rcu detected stall in corrupted

rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 1-... } 2642 jiffies s: 2057 root: 0x2/.
rcu: blocking rcu_node structures (internal RCU debug):
Task dump for CPU 1:
task:syz-executor212 state:R running task stack:26424 pid: 4080 ppid: 4063 flags:0x0000000e
Call Trace:
<TASK>
</TASK>

Tested on:

commit: 91265a6d Add linux-next specific files for 20220303
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/
console output: https://syzkaller.appspot.com/x/log.txt?x=13f1becb700000

kernel config: https://syzkaller.appspot.com/x/.config?x=617f79440a35673a
dashboard link: https://syzkaller.appspot.com/bug?extid=eec403943a2a2455adaa
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2

patch: https://syzkaller.appspot.com/x/patch.diff?x=11d0caeb700000

Hillf Danton

unread,

Mar 20, 2022, 9:21:00 PM3/20/22

to syzbot, linux-...@vger.kernel.org, syzkall...@googlegroups.com

On Sun, 20 Mar 2022 08:33:07 -0700

To see what syz-executor212 was doing while the rcu stall was detected,
serialize the hash_resize and gc worker using nf_conntrack_mutex, given
no sense to do gc in parallel to the resize path.

Hillf

#syz test: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/ 91265a6da44d

--- x/net/netfilter/nf_conntrack_core.c
+++ y/net/netfilter/nf_conntrack_core.c

@@ -1430,6 +1430,7 @@ static void gc_worker(struct work_struct
if (gc_work->early_drop)
nf_conntrack_max95 = nf_conntrack_max / 100u * 95u;

+ mutex_lock(&nf_conntrack_mutex);
do {
struct nf_conntrack_tuple_hash *h;
struct hlist_nulls_head *ct_hash;
@@ -1498,6 +1499,8 @@ static void gc_worker(struct work_struct
}
} while (i < hashsz);

+ mutex_unlock(&nf_conntrack_mutex);
+
if (gc_work->exiting)
return;

--

syzbot

unread,

Mar 20, 2022, 9:32:13 PM3/20/22

to hda...@sina.com, linux-...@vger.kernel.org, syzkall...@googlegroups.com

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
INFO: rcu detected stall in corrupted

IPv6: ADDRCONF(NETDEV_CHANGE): veth0_to_batadv: link becomes ready
IPv6: ADDRCONF(NETDEV_CHANGE): veth1_to_batadv: link becomes ready
rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 0-... } 2628 jiffies s: 2065 root: 0x1/.

rcu: blocking rcu_node structures (internal RCU debug):

Task dump for CPU 0:
task:syz-executor354 state:R running task stack:27224 pid: 4078 ppid: 4061 flags:0x0000400e

Call Trace:
<TASK>
</TASK>

Tested on:

commit: 91265a6d Add linux-next specific files for 20220303
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/

console output: https://syzkaller.appspot.com/x/log.txt?x=16355271700000

kernel config: https://syzkaller.appspot.com/x/.config?x=617f79440a35673a
dashboard link: https://syzkaller.appspot.com/bug?extid=eec403943a2a2455adaa
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2

patch: https://syzkaller.appspot.com/x/patch.diff?x=17b1f271700000

Hillf Danton

unread,

Mar 20, 2022, 11:15:49 PM3/20/22

to syzbot, linux-...@vger.kernel.org, syzkall...@googlegroups.com

On Sun, 20 Mar 2022 18:32:12 -0700

What is weird in the original report after another look is

> > INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.570 msecs
> > rcu: rcu_preempt kthread starved for 10492 jiffies! g4825 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1

> > .......snip....

CPU1 was detected starved waiting for IPI echo with preempt disabled.
Is it evidence for hardware glitch wrt IPI?

> > smp_call_function_many_cond+0x50a/0xc90 kernel/smp.c:972
> > on_each_cpu_cond_mask+0x56/0xa0 kernel/smp.c:1138
> > on_each_cpu include/linux/smp.h:71 [inline]
> > text_poke_sync arch/x86/kernel/alternative.c:1146 [inline]
> > text_poke_bp_batch+0x3e9/0x6b0 arch/x86/kernel/alternative.c:1387
> > text_poke_flush arch/x86/kernel/alternative.c:1504 [inline]
> > text_poke_flush arch/x86/kernel/alternative.c:1501 [inline]
> > text_poke_finish+0x16/0x30 arch/x86/kernel/alternative.c:1511
> > arch_jump_label_transform_apply+0x13/0x20 arch/x86/kernel/jump_label.c:146
> > jump_label_update+0x32f/0x410 kernel/jump_label.c:830
> > static_key_enable_cpuslocked+0x1b1/0x260 kernel/jump_label.c:177
> > static_key_enable+0x16/0x20 kernel/jump_label.c:190
> > toggle_allocation_gate mm/kfence/core.c:735 [inline]
> > toggle_allocation_gate+0x100/0x390 mm/kfence/core.c:727
> > process_one_work+0x996/0x1610 kernel/workqueue.c:2289
> > worker_thread+0x665/0x1080 kernel/workqueue.c:2436
> > kthread+0x2e9/0x3a0 kernel/kthread.c:376
> > ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
> > </TASK>

See what will come up with the CPU hog in nfct removed.

Hillf

#syz test: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/ 91265a6da44d

--- x/net/netfilter/nf_conntrack_core.c
+++ y/net/netfilter/nf_conntrack_core.c

@@ -2558,6 +2558,7 @@ int nf_conntrack_hash_resize(unsigned in

if (!hashsize)
return -EINVAL;
+ return -EINVAL;

hash = nf_ct_alloc_hashtable(&hashsize, 1);
if (!hash)
--

syzbot

unread,

Mar 20, 2022, 11:23:13 PM3/20/22

to hda...@sina.com, linux-...@vger.kernel.org, syzkall...@googlegroups.com

Hello,

syzbot tried to test the proposed patch but the build/boot failed:

failed to create VM pool: failed to create GCE image: create image operation failed: &{Code:PERMISSIONS_ERROR Location: Message:Required 'read' permission for 'disks/ci-upstream-linux-next-kasan-gce-root-test-job-test-job-image.tar.gz' ForceSendFields:[] NullFields:[]}.

Tested on:

commit: 91265a6d Add linux-next specific files for 20220303
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/

kernel config: https://syzkaller.appspot.com/x/.config?x=617f79440a35673a
dashboard link: https://syzkaller.appspot.com/bug?extid=eec403943a2a2455adaa
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2

patch: https://syzkaller.appspot.com/x/patch.diff?x=17ce6425700000

Hillf Danton

unread,

Mar 21, 2022, 2:31:06 AM3/21/22

to syzbot, linux-...@vger.kernel.org, syzkall...@googlegroups.com

On Sun, 20 Mar 2022 20:23:12 -0700

See what will come up if it is reproduced upstream with the nfct
CPU hog disabled, particularly wrt any reaction to IPI.

#syz test: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master

--- x/net/netfilter/nf_conntrack_core.c
+++ y/net/netfilter/nf_conntrack_core.c
@@ -2552,6 +2552,8 @@ int nf_conntrack_hash_resize(unsigned in

if (!hashsize)
return -EINVAL;
+ else
+ return -ENOMEM;

syzbot

unread,

Mar 21, 2022, 2:43:09 AM3/21/22

to hda...@sina.com, linux-...@vger.kernel.org, syzkall...@googlegroups.com

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:

BUG: soft lockup in smp_call_function

watchdog: BUG: soft lockup - CPU#0 stuck for 143s! [kworker/u4:5:1244]
Modules linked in:
irq event stamp: 595274
hardirqs last enabled at (595273): [<ffffffff89600c02>] asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
hardirqs last disabled at (595274): [<ffffffff894c14ab>] sysvec_apic_timer_interrupt+0xb/0xc0 arch/x86/kernel/apic/apic.c:1097
softirqs last enabled at (588534): [<ffffffff81474343>] invoke_softirq kernel/softirq.c:432 [inline]
softirqs last enabled at (588534): [<ffffffff81474343>] __irq_exit_rcu+0x123/0x180 kernel/softirq.c:637
softirqs last disabled at (588525): [<ffffffff81474343>] invoke_softirq kernel/softirq.c:432 [inline]
softirqs last disabled at (588525): [<ffffffff81474343>] __irq_exit_rcu+0x123/0x180 kernel/softirq.c:637
CPU: 0 PID: 1244 Comm: kworker/u4:5 Not tainted 5.17.0-syzkaller-00083-gf443e374ae13-dirty #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: events_unbound toggle_allocation_gate

RIP: 0010:__sanitizer_cov_trace_pc+0x5c/0x60 kernel/kcov.c:210
Code: 82 80 15 00 00 83 f8 02 75 20 48 8b 8a 88 15 00 00 8b 92 84 15 00 00 48 8b 01 48 83 c0 01 48 39 c2 76 07 48 89 34 c1 48 89 01 <c3> 0f 1f 00 41 55 41 54 49 89 fc 55 48 bd eb 83 b5 80 46 86 c8 61
RSP: 0018:ffffc90005a3f9e8 EFLAGS: 00000293
RAX: 0000000000000000 RBX: ffff8880b9d3fec0 RCX: 0000000000000000
RDX: ffff88801cf21d00 RSI: ffffffff816d3654 RDI: 0000000000000003

RBP: 0000000000000003 R08: 0000000000000000 R09: 0000000000000001

R10: ffffffff816d367a R11: 0000000000000000 R12: ffffed10173a7fd9
R13: 0000000000000001 R14: ffff8880b9d3fec8 R15: 0000000000000001

FS: 0000000000000000(0000) GS:ffff8880b9c00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 00007ffe833fd960 CR3: 000000000b88e000 CR4: 00000000003506f0

DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:

<TASK>
rep_nop arch/x86/include/asm/vdso/processor.h:13 [inline]
cpu_relax arch/x86/include/asm/vdso/processor.h:18 [inline]
csd_lock_wait kernel/smp.c:440 [inline]
smp_call_function_many_cond+0x4e4/0xc90 kernel/smp.c:969
on_each_cpu_cond_mask+0x56/0xa0 kernel/smp.c:1135
on_each_cpu include/linux/smp.h:71 [inline]
text_poke_sync arch/x86/kernel/alternative.c:1112 [inline]
text_poke_bp_batch+0x21d/0x6f0 arch/x86/kernel/alternative.c:1300
text_poke_flush arch/x86/kernel/alternative.c:1470 [inline]
text_poke_flush arch/x86/kernel/alternative.c:1467 [inline]
text_poke_finish+0x16/0x30 arch/x86/kernel/alternative.c:1477

arch_jump_label_transform_apply+0x13/0x20 arch/x86/kernel/jump_label.c:146
jump_label_update+0x32f/0x410 kernel/jump_label.c:830
static_key_enable_cpuslocked+0x1b1/0x260 kernel/jump_label.c:177
static_key_enable+0x16/0x20 kernel/jump_label.c:190
toggle_allocation_gate mm/kfence/core.c:735 [inline]
toggle_allocation_gate+0x100/0x390 mm/kfence/core.c:727

process_one_work+0x9ac/0x1650 kernel/workqueue.c:2307
worker_thread+0x657/0x1110 kernel/workqueue.c:2454
kthread+0x2e9/0x3a0 kernel/kthread.c:377
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
</TASK>
Sending NMI from CPU 0 to CPUs 1:

NMI backtrace for cpu 1

CPU: 1 PID: 4081 Comm: syz-executor201 Not tainted 5.17.0-syzkaller-00083-gf443e374ae13-dirty #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011

RIP: 0010:_raw_spin_lock_irqsave+0x3/0x50 kernel/locking/spinlock.c:161
Code: 31 d2 31 f6 e8 fe 9b 0e f8 48 89 ef 58 5d e9 04 0f 0f f8 e8 bf 0a 30 f8 eb c9 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 41 54 55 <48> 89 fd 9c 41 5c fa 41 f7 c4 00 02 00 00 75 36 bf 01 00 00 00 e8
RSP: 0018:ffffc90000dc0d40 EFLAGS: 00000002
RAX: 0000000000001399 RBX: 0000000000058308 RCX: 0000000000000000
RDX: 61c8864680b583eb RSI: ffffffff89ae6860 RDI: ffffffff906f8610
RBP: ffffffff906f8608 R08: ffffffff906f8610 R09: 0000000000000001
R10: ffffffff81681c55 R11: 0000000000000000 R12: dffffc0000000000
R13: ffffffff89ae6860 R14: 1ffff920001b81ad R15: ffff888140782b40
FS: 0000555555b0b300(0000) GS:ffff8880b9d00000(0000) knlGS:0000000000000000

CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 0000000020000600 CR3: 000000001e6c1000 CR4: 00000000003506e0

DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<IRQ>

debug_object_deactivate lib/debugobjects.c:735 [inline]
debug_object_deactivate+0x101/0x300 lib/debugobjects.c:723
debug_hrtimer_deactivate kernel/time/hrtimer.c:425 [inline]
debug_deactivate kernel/time/hrtimer.c:481 [inline]
__run_hrtimer kernel/time/hrtimer.c:1653 [inline]
__hrtimer_run_queues+0x3f8/0xe50 kernel/time/hrtimer.c:1749

hrtimer_interrupt+0x31c/0x790 kernel/time/hrtimer.c:1811
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1086 [inline]
__sysvec_apic_timer_interrupt+0x146/0x530 arch/x86/kernel/apic/apic.c:1103
sysvec_apic_timer_interrupt+0x8e/0xc0 arch/x86/kernel/apic/apic.c:1097
</IRQ>
<TASK>
asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638

RIP: 0010:__raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:152 [inline]
RIP: 0010:_raw_spin_unlock_irqrestore+0x38/0x70 kernel/locking/spinlock.c:194
Code: 74 24 10 e8 6a 92 0e f8 48 89 ef e8 c2 10 0f f8 81 e3 00 02 00 00 75 25 9c 58 f6 c4 02 75 2d 48 85 db 74 01 fb bf 01 00 00 00 <e8> 73 c8 01 f8 65 8b 05 9c 76 b3 76 85 c0 74 0a 5b 5d c3 e8 b0 04
RSP: 0018:ffffc9000289f1e8 EFLAGS: 00000206
RAX: 0000000000000012 RBX: 0000000000000200 RCX: 1ffffffff2001bce
RDX: 0000000000000000 RSI: 0000000000000202 RDI: 0000000000000001
RBP: ffff888140782ae8 R08: 0000000000000001 R09: ffffffff8ffc5a07
R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000000
R13: ffff888140782ae8 R14: 0000000000000246 R15: ffff888140782800
spin_unlock_irqrestore include/linux/spinlock.h:404 [inline]
taprio_change+0x2f0c/0x4050 net/sched/sch_taprio.c:1606
taprio_init+0x52e/0x670 net/sched/sch_taprio.c:1738
qdisc_create.constprop.0+0x44a/0x10f0 net/sched/sch_api.c:1253
tc_modify_qdisc+0x4c5/0x1a00 net/sched/sch_api.c:1660
rtnetlink_rcv_msg+0x413/0xb80 net/core/rtnetlink.c:5596
netlink_rcv_skb+0x153/0x420 net/netlink/af_netlink.c:2494
netlink_unicast_kernel net/netlink/af_netlink.c:1317 [inline]
netlink_unicast+0x539/0x7e0 net/netlink/af_netlink.c:1343
netlink_sendmsg+0x904/0xe00 net/netlink/af_netlink.c:1919
sock_sendmsg_nosec net/socket.c:705 [inline]
sock_sendmsg+0xcf/0x120 net/socket.c:725
____sys_sendmsg+0x6e8/0x810 net/socket.c:2413
___sys_sendmsg+0xf3/0x170 net/socket.c:2467
__sys_sendmsg+0xe5/0x1b0 net/socket.c:2496
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7f5f768be729
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 81 15 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 c0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffe833c99c8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00007f5f768be729
RDX: 0000000000000000 RSI: 00000000200007c0 RDI: 0000000000000004
RBP: 0000000000000000 R08: 00007ffe833c99f0 R09: 00007ffe833c99f0
R10: 00007ffe833c99f0 R11: 0000000000000246 R12: 00007ffe833c99ec
R13: 00007ffe833c9a00 R14: 00007ffe833c9a40 R15: 0000000000000000
</TASK>
----------------
Code disassembly (best guess), 1 bytes skipped:
0: 80 15 00 00 83 f8 02 adcb $0x2,-0x77d0000(%rip) # 0xf8830007
7: 75 20 jne 0x29
9: 48 8b 8a 88 15 00 00 mov 0x1588(%rdx),%rcx
10: 8b 92 84 15 00 00 mov 0x1584(%rdx),%edx
16: 48 8b 01 mov (%rcx),%rax
19: 48 83 c0 01 add $0x1,%rax
1d: 48 39 c2 cmp %rax,%rdx
20: 76 07 jbe 0x29
22: 48 89 34 c1 mov %rsi,(%rcx,%rax,8)
26: 48 89 01 mov %rax,(%rcx)
* 29: c3 retq <-- trapping instruction
2a: 0f 1f 00 nopl (%rax)
2d: 41 55 push %r13
2f: 41 54 push %r12
31: 49 89 fc mov %rdi,%r12
34: 55 push %rbp
35: 48 bd eb 83 b5 80 46 movabs $0x61c8864680b583eb,%rbp
3c: 86 c8 61

Tested on:

commit: f443e374 Linux 5.17
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=157ae133700000
kernel config: https://syzkaller.appspot.com/x/.config?x=19ca6f72fd444749

dashboard link: https://syzkaller.appspot.com/bug?extid=eec403943a2a2455adaa
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2

patch: https://syzkaller.appspot.com/x/patch.diff?x=15e78333700000

Hillf Danton

unread,

Mar 21, 2022, 4:08:58 AM3/21/22

to syzbot, tg...@linutronix.de, Dmitry Vyukov, Paul E . McKenney, linux-...@vger.kernel.org, syzkall...@googlegroups.com

Hi Paul and tglx

On Sun, 20 Mar 2022 23:43:07 -0700

Given soft lockup detected on CPU0 waiting for IPI echo, is this report
the evidence for hardware glitch wrt handling IPI? IOW can we rule out
software ones in this case?

Thanks
Hillf

Paul E. McKenney

unread,

Mar 21, 2022, 5:01:42 PM3/21/22

to Hillf Danton, syzbot, tg...@linutronix.de, Dmitry Vyukov, linux-...@vger.kernel.org, syzkall...@googlegroups.com

There are several possible causes:

1. Hardware/firmware delays as you say.
2. Some part of the kernel disabling irqs for too long.
3. Some specific IPI handler is running for too long.
4. So many IPIs are being sent to this CPU that it cannot keep up.

Tracing should help distinguish between these.

Thanx, Paul

Hillf Danton

unread,

Jul 31, 2022, 3:46:45 AM7/31/22

to syzbot, linux-...@vger.kernel.org, syzkall...@googlegroups.com

On Sun, 20 Mar 2022 05:02:23 -0700

> syzbot found the following issue on:
>
> HEAD commit: 91265a6da44d Add linux-next specific files for 20220303
> git tree: linux-next
> console output: https://syzkaller.appspot.com/x/log.txt?x=157605d5700000
> kernel config: https://syzkaller.appspot.com/x/.config?x=617f79440a35673a
> dashboard link: https://syzkaller.appspot.com/bug?extid=eec403943a2a2455adaa
> compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=143195d9700000
> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=170736a9700000
>
> Bisection is inconclusive: the issue happens on the oldest tested release.
>
> bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=141bc691700000
> final oops: https://syzkaller.appspot.com/x/report.txt?x=161bc691700000
> console output: https://syzkaller.appspot.com/x/log.txt?x=121bc691700000
>
> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> Reported-by: syzbot+eec403...@syzkaller.appspotmail.com
>
> rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
> rcu: 0-...!: (1 GPs behind) idle=a59/1/0x4000000000000000 softirq=5468/5472 fqs=5
> (detected by 1, t=10502 jiffies, g=4825, q=140)
> Sending NMI from CPU 1 to CPUs 0:
> NMI backtrace for cpu 0
> CPU: 0 PID: 3644 Comm: kworker/0:1 Not tainted 5.17.0-rc6-next-20220303-syzkaller #0

Print the hrtimer hog.

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git 91265a6da44d

diff -pur m/kernel/time/hrtimer.c n/kernel/time/hrtimer.c
--- m/kernel/time/hrtimer.c 2022-07-31 15:34:48.596885000 +0800
+++ n/kernel/time/hrtimer.c 2022-07-31 15:37:26.171641100 +0800
@@ -1646,6 +1646,7 @@ static void __run_hrtimer(struct hrtimer
{
enum hrtimer_restart (*fn)(struct hrtimer *);
bool expires_in_hardirq;
+ unsigned long ts;
int restart;

lockdep_assert_held(&cpu_base->lock);
@@ -1681,9 +1682,13 @@ static void __run_hrtimer(struct hrtimer
raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
trace_hrtimer_expire_entry(timer, now);
expires_in_hardirq = lockdep_hrtimer_enter(timer);
+ ts = jiffies + 4;

restart = fn(timer);

+ WARN_ONCE(time_after(jiffies, ts),
+ "On CPU%u hrtimer %ps took more than 4 ticks\n", cpu_base->cpu, fn);
+
lockdep_hrtimer_exit(expires_in_hardirq);
trace_hrtimer_expire_exit(timer);
raw_spin_lock_irq(&cpu_base->lock);
--

syzbot

unread,

Jul 31, 2022, 4:03:16 AM7/31/22

to hda...@sina.com, linux-...@vger.kernel.org, syzkall...@googlegroups.com

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:

WARNING in __hrtimer_run_queues

------------[ cut here ]------------
On CPU1 hrtimer tick_sched_timer took more than 4 ticks
WARNING: CPU: 1 PID: 0 at kernel/time/hrtimer.c:1689 __run_hrtimer kernel/time/hrtimer.c:1689 [inline]
WARNING: CPU: 1 PID: 0 at kernel/time/hrtimer.c:1689 __hrtimer_run_queues+0xe3c/0x1000 kernel/time/hrtimer.c:1754
Modules linked in:
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.17.0-rc6-next-20220303-syzkaller-dirty #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/22/2022
RIP: 0010:__run_hrtimer kernel/time/hrtimer.c:1689 [inline]
RIP: 0010:__hrtimer_run_queues+0xe3c/0x1000 kernel/time/hrtimer.c:1754
Code: fa 48 c1 ea 03 0f b6 04 02 84 c0 74 08 3c 03 0f 8e b7 01 00 00 41 8b 77 40 48 c7 c7 a0 68 ce 89 48 8b 54 24 08 e8 50 82 b0 07 <0f> 0b e9 62 f4 ff ff e8 c8 42 5c 00 e9 5c f6 ff ff 48 8b 7c 24 18
RSP: 0018:ffffc900001e0e30 EFLAGS: 00010082
RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000
RDX: ffff888011a21d40 RSI: ffffffff81602878 RDI: fffff5200003c1b8
RBP: 0000000000000001 R08: 0000000000000000 R09: 0000000000000000
R10: ffffffff815fd23e R11: 0000000000000000 R12: ffff8880b9d2afa0
R13: 0000000000000000 R14: ffff8880b9d2a680 R15: ffff8880b9d2a600
FS: 0000000000000000(0000) GS:ffff8880b9d00000(0000) knlGS:0000000000000000

CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 00007fba30aa0097 CR3: 000000002354c000 CR4: 00000000003506e0

DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<IRQ>

hrtimer_interrupt+0x31c/0x790 kernel/time/hrtimer.c:1816

local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1086 [inline]
__sysvec_apic_timer_interrupt+0x146/0x530 arch/x86/kernel/apic/apic.c:1103
sysvec_apic_timer_interrupt+0x8e/0xc0 arch/x86/kernel/apic/apic.c:1097
</IRQ>
<TASK>
asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638

RIP: 0010:native_save_fl arch/x86/include/asm/irqflags.h:29 [inline]
RIP: 0010:arch_local_save_flags arch/x86/include/asm/irqflags.h:70 [inline]
RIP: 0010:arch_irqs_disabled arch/x86/include/asm/irqflags.h:130 [inline]
RIP: 0010:acpi_safe_halt drivers/acpi/processor_idle.c:116 [inline]
RIP: 0010:acpi_idle_do_entry+0x1c6/0x250 drivers/acpi/processor_idle.c:556
Code: 89 de e8 4d e6 18 f8 84 db 75 ac e8 64 e2 18 f8 e8 1f 2b 1f f8 eb 0c e8 58 e2 18 f8 0f 00 2d 91 ed d1 00 e8 4c e2 18 f8 fb f4 <9c> 5b 81 e3 00 02 00 00 fa 31 ff 48 89 de e8 c7 e4 18 f8 48 85 db
RSP: 0018:ffffc90000177d18 EFLAGS: 00000293
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
RDX: ffff888011a21d40 RSI: ffffffff896045f4 RDI: 0000000000000000
RBP: ffff888016545864 R08: 0000000000000001 R09: 0000000000000001
R10: ffffffff817f7138 R11: 0000000000000000 R12: 0000000000000001
R13: ffff888016545800 R14: ffff888016545864 R15: ffff8880199b0004
acpi_idle_enter+0x361/0x500 drivers/acpi/processor_idle.c:692
cpuidle_enter_state+0x1b1/0xc80 drivers/cpuidle/cpuidle.c:237
cpuidle_enter+0x4a/0xa0 drivers/cpuidle/cpuidle.c:351
call_cpuidle kernel/sched/idle.c:158 [inline]
cpuidle_idle_call kernel/sched/idle.c:239 [inline]
do_idle+0x3e8/0x590 kernel/sched/idle.c:306
cpu_startup_entry+0x14/0x20 kernel/sched/idle.c:403
start_secondary+0x265/0x340 arch/x86/kernel/smpboot.c:272
secondary_startup_64_no_verify+0xc3/0xcb
</TASK>
----------------
Code disassembly (best guess):
0: 89 de mov %ebx,%esi
2: e8 4d e6 18 f8 callq 0xf818e654
7: 84 db test %bl,%bl
9: 75 ac jne 0xffffffb7
b: e8 64 e2 18 f8 callq 0xf818e274
10: e8 1f 2b 1f f8 callq 0xf81f2b34
15: eb 0c jmp 0x23
17: e8 58 e2 18 f8 callq 0xf818e274
1c: 0f 00 2d 91 ed d1 00 verw 0xd1ed91(%rip) # 0xd1edb4
23: e8 4c e2 18 f8 callq 0xf818e274
28: fb sti
29: f4 hlt
* 2a: 9c pushfq <-- trapping instruction
2b: 5b pop %rbx
2c: 81 e3 00 02 00 00 and $0x200,%ebx
32: fa cli
33: 31 ff xor %edi,%edi
35: 48 89 de mov %rbx,%rsi
38: e8 c7 e4 18 f8 callq 0xf818e504
3d: 48 85 db test %rbx,%rbx

Tested on:

commit: 91265a6d Add linux-next specific files for 20220303
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
console output: https://syzkaller.appspot.com/x/log.txt?x=11658846080000

kernel config: https://syzkaller.appspot.com/x/.config?x=617f79440a35673a
dashboard link: https://syzkaller.appspot.com/bug?extid=eec403943a2a2455adaa
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2

patch: https://syzkaller.appspot.com/x/patch.diff?x=161e281e080000

Hillf Danton

unread,

Jul 31, 2022, 4:16:00 AM7/31/22

to syzbot, linux-...@vger.kernel.org, syzkall...@googlegroups.com

On Sun, 20 Mar 2022 05:02:23 -0700

> syzbot found the following issue on:
>
> HEAD commit: 91265a6da44d Add linux-next specific files for 20220303
> git tree: linux-next
> console output: https://syzkaller.appspot.com/x/log.txt?x=157605d5700000
> kernel config: https://syzkaller.appspot.com/x/.config?x=617f79440a35673a
> dashboard link: https://syzkaller.appspot.com/bug?extid=eec403943a2a2455adaa
> compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=143195d9700000
> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=170736a9700000
>
> Bisection is inconclusive: the issue happens on the oldest tested release.
>
> bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=141bc691700000
> final oops: https://syzkaller.appspot.com/x/report.txt?x=161bc691700000
> console output: https://syzkaller.appspot.com/x/log.txt?x=121bc691700000
>
> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> Reported-by: syzbot+eec403...@syzkaller.appspotmail.com
>
> rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
> rcu: 0-...!: (1 GPs behind) idle=a59/1/0x4000000000000000 softirq=5468/5472 fqs=5
> (detected by 1, t=10502 jiffies, g=4825, q=140)
> Sending NMI from CPU 1 to CPUs 0:
> NMI backtrace for cpu 0
> CPU: 0 PID: 3644 Comm: kworker/0:1 Not tainted 5.17.0-rc6-next-20220303-syzkaller #0

Print the hrtimer hog.

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git 91265a6da44d

diff -pur m/kernel/time/hrtimer.c n/kernel/time/hrtimer.c
--- m/kernel/time/hrtimer.c 2022-07-31 15:34:48.596885000 +0800

+++ n/kernel/time/hrtimer.c 2022-07-31 16:04:47.190229000 +0800

@@ -1646,6 +1646,7 @@ static void __run_hrtimer(struct hrtimer
{
enum hrtimer_restart (*fn)(struct hrtimer *);
bool expires_in_hardirq;
+ unsigned long ts;
int restart;

lockdep_assert_held(&cpu_base->lock);
@@ -1681,9 +1682,13 @@ static void __run_hrtimer(struct hrtimer
raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
trace_hrtimer_expire_entry(timer, now);
expires_in_hardirq = lockdep_hrtimer_enter(timer);
+ ts = jiffies + 4;

restart = fn(timer);

+ if (time_after(jiffies, ts))
+ pr_warn("On CPU%u hrtimer %ps took more than 4 ticks\n", cpu_base->cpu, fn);

syzbot

unread,

Jul 31, 2022, 4:31:09 AM7/31/22

to hda...@sina.com, linux-...@vger.kernel.org, syzkall...@googlegroups.com

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:

INFO: rcu detected stall in corrupted

rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 0-... } 2654 jiffies s: 2049 root: 0x1/.

rcu: blocking rcu_node structures (internal RCU debug):
Task dump for CPU 0:

task:syz-executor.0 state:R running task stack:27224 pid: 4073 ppid: 4053 flags:0x0000400e

Call Trace:
<TASK>
</TASK>

Tested on:

commit: 91265a6d Add linux-next specific files for 20220303
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git

console output: https://syzkaller.appspot.com/x/log.txt?x=11978a82080000

kernel config: https://syzkaller.appspot.com/x/.config?x=617f79440a35673a
dashboard link: https://syzkaller.appspot.com/bug?extid=eec403943a2a2455adaa
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2

patch: https://syzkaller.appspot.com/x/patch.diff?x=14ff72fe080000

Hillf Danton

unread,

Jul 31, 2022, 5:48:17 AM7/31/22

to syzbot, linux-...@vger.kernel.org, syzkall...@googlegroups.com

On Sun, 20 Mar 2022 05:02:23 -0700

> syzbot found the following issue on:
>
> HEAD commit: 91265a6da44d Add linux-next specific files for 20220303
> git tree: linux-next
> console output: https://syzkaller.appspot.com/x/log.txt?x=157605d5700000
> kernel config: https://syzkaller.appspot.com/x/.config?x=617f79440a35673a
> dashboard link: https://syzkaller.appspot.com/bug?extid=eec403943a2a2455adaa
> compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=143195d9700000
> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=170736a9700000
>
> Bisection is inconclusive: the issue happens on the oldest tested release.
>
> bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=141bc691700000
> final oops: https://syzkaller.appspot.com/x/report.txt?x=161bc691700000
> console output: https://syzkaller.appspot.com/x/log.txt?x=121bc691700000
>
> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> Reported-by: syzbot+eec403...@syzkaller.appspotmail.com
>
> rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
> rcu: 0-...!: (1 GPs behind) idle=a59/1/0x4000000000000000 softirq=5468/5472 fqs=5
> (detected by 1, t=10502 jiffies, g=4825, q=140)
> Sending NMI from CPU 1 to CPUs 0:
> NMI backtrace for cpu 0
> CPU: 0 PID: 3644 Comm: kworker/0:1 Not tainted 5.17.0-rc6-next-20220303-syzkaller #0

Print the hrtimer hog.

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git 91265a6da44d

diff -pur m/kernel/time/hrtimer.c n/kernel/time/hrtimer.c
--- m/kernel/time/hrtimer.c 2022-07-31 15:34:48.596885000 +0800

+++ n/kernel/time/hrtimer.c 2022-07-31 17:45:29.015729400 +0800

@@ -1646,6 +1646,7 @@ static void __run_hrtimer(struct hrtimer
{
enum hrtimer_restart (*fn)(struct hrtimer *);
bool expires_in_hardirq;
+ unsigned long ts;
int restart;

lockdep_assert_held(&cpu_base->lock);

@@ -1681,9 +1682,16 @@ static void __run_hrtimer(struct hrtimer

raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
trace_hrtimer_expire_entry(timer, now);
expires_in_hardirq = lockdep_hrtimer_enter(timer);

+ ts = jiffies + 10;

restart = fn(timer);

+ WARN_ONCE(time_after(jiffies, ts),

+ "On CPU%u hrtimer %ps took more than 10 ticks\n", cpu_base->cpu, fn);
+/*

+ if (time_after(jiffies, ts))
+ pr_warn("On CPU%u hrtimer %ps took more than 4 ticks\n", cpu_base->cpu, fn);

+*/

syzbot

unread,

Jul 31, 2022, 6:09:17 AM7/31/22

to hda...@sina.com, linux-...@vger.kernel.org, syzkall...@googlegroups.com

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
INFO: rcu detected stall in corrupted

rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:

rcu: 0-...!: (1 GPs behind) idle=cad/1/0x4000000000000000 softirq=7023/7025 fqs=2
(detected by 1, t=10502 jiffies, g=7257, q=133)

Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0

CPU: 0 PID: 4073 Comm: syz-executor.0 Not tainted 5.17.0-rc6-next-20220303-syzkaller-dirty #0
Harclient_loop: send disconnect: Broken pipe

Tested on:

commit: 91265a6d Add linux-next specific files for 20220303
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git

console output: https://syzkaller.appspot.com/x/log.txt?x=166fe82e080000

kernel config: https://syzkaller.appspot.com/x/.config?x=617f79440a35673a
dashboard link: https://syzkaller.appspot.com/bug?extid=eec403943a2a2455adaa
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2

patch: https://syzkaller.appspot.com/x/patch.diff?x=15c240c1080000

syzbot

unread,

Sep 24, 2023, 6:59:19 AM9/24/23

to b...@vger.kernel.org, core...@netfilter.org, da...@davemloft.net, dvy...@google.com, edum...@google.com, f...@strlen.de, gauta...@gmail.com, hda...@sina.com, j...@mojatatu.com, ji...@resnulli.us, kad...@netfilter.org, ku...@kernel.org, lesli...@gmail.com, linux-...@vger.kernel.org, mohitbh...@gmail.com, net...@vger.kernel.org, netfilt...@vger.kernel.org, pab...@redhat.com, pa...@netfilter.org, pau...@kernel.org, sdp.s...@gmail.com, syzkall...@googlegroups.com, tahi...@nitk.edu.in, tg...@linutronix.de, vsaicha...@gmail.com, xiyou.w...@gmail.com

syzbot has bisected this issue to:

commit ec97ecf1ebe485a17cd8395a5f35e6b80b57665a
Author: Mohit P. Tahiliani <tahi...@nitk.edu.in>
Date: Wed Jan 22 18:22:33 2020 +0000

net: sched: add Flow Queue PIE packet scheduler

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=15c5748e680000
start commit: d4a7ce642100 igc: Fix Kernel Panic during ndo_tx_timeout c..
git tree: net
final oops: https://syzkaller.appspot.com/x/report.txt?x=17c5748e680000
console output: https://syzkaller.appspot.com/x/log.txt?x=13c5748e680000
kernel config: https://syzkaller.appspot.com/x/.config?x=77b9a3cf8f44c6da
dashboard link: https://syzkaller.appspot.com/bug?extid=eec403943a2a2455adaa
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1504b511a80000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=137bf931a80000

Reported-by: syzbot+eec403...@syzkaller.appspotmail.com
Fixes: ec97ecf1ebe4 ("net: sched: add Flow Queue PIE packet scheduler")

Hillf Danton

unread,

Sep 25, 2023, 7:03:22 AM9/25/23

to syzbot, linux-...@vger.kernel.org, syzkall...@googlegroups.com

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master

syzbot

unread,

Sep 25, 2023, 7:23:14 AM9/25/23

to hda...@sina.com, linux-...@vger.kernel.org, syzkall...@googlegroups.com

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
INFO: rcu detected stall in corrupted

rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: {
0-....
} 3076 jiffies s: 5117 root: 0x1/.

rcu: blocking rcu_node structures (internal RCU debug):

Sending NMI from CPU 1 to CPUs 0:

Node 0 active_anon:43180kB inactive_anon:1968kB active_file:0kB inactive_file:207652kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:28180kB dirty:60kB writeback:0kB shmem:3552kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:10776kB pagetables:2324kB sec_pagetables:0kB all_unreclaimable? no
Node 1 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:4kB writeback:0kB shmem:1536kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:16kB pagetables:0kB sec_pagetables:0kB all_unreclaimable? no
Node 0 DMA free:15360kB boost:0kB min:200kB low:248kB high:296kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 2611 2612 2612 2612
Node 0 DMA32 free:1783712kB boost:0kB min:35396kB low:44244kB high:53092kB reserved_highatomic:0KB active_anon:43144kB inactive_anon:1964kB active_file:0kB inactive_file:206836kB unevictable:1536kB writepending:60kB present:3129332kB managed:2678260kB mlocked:0kB bounce:0kB free_pcp:12452kB local_pcp:8152kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 Normal free:4kB boost:0kB min:8kB low:8kB high:8kB reserved_highatomic:0KB active_anon:36kB inactive_anon:4kB active_file:0kB inactive_file:816kB unevictable:0kB writepending:0kB present:1048576kB managed:872kB mlocked:0kB bounce:0kB free_pcp:12kB local_pcp:12kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 1 Normal free:3927548kB boost:0kB min:54496kB low:68120kB high:81744kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB writepending:4kB present:4194304kB managed:4117312kB mlocked:0kB bounce:0kB free_pcp:15296kB local_pcp:7136kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15360kB
Node 0 DMA32: 1384*4kB (UME) 946*8kB (UME) 215*16kB (UME) 94*32kB (UME) 43*64kB (UME) 19*128kB (UME) 3*256kB (UM) 2*512kB (ME) 4*1024kB (UME) 2*2048kB (ME) 427*4096kB (M) = 1783712kB
Node 0 Normal: 1*4kB (M) 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4kB
Node 1 Normal: 193*4kB (UE) 55*8kB (UE) 38*16kB (UE) 77*32kB (UE) 27*64kB (UME) 5*128kB (UE) 2*256kB (UM) 3*512kB (UME) 1*1024kB (U) 1*2048kB (U) 956*4096kB (M) = 3927548kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 0 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 1 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
53203 total pagecache pages
0 pages in swap cache
Free swap = 124996kB
Total swap = 124996kB
2097051 pages RAM
0 pages HighMem/MovableOnly
394100 pages reserved
0 pages cma reserved
Mem-Info:
active_anon:10795 inactive_anon:492 isolated_anon:0
active_file:0 inactive_file:51931 isolated_file:0
unevictable:768 dirty:16 writeback:0
slab_reclaimable:9708 slab_unreclaimable:98313
mapped:7045 shmem:1272 pagetables:581
sec_pagetables:0 bounce:0
kernel_misc_reclaimable:0
free:1431656 free_pcp:6940 free_cma:0
Node 0 active_anon:43180kB inactive_anon:1968kB active_file:0kB inactive_file:207652kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:28180kB dirty:60kB writeback:0kB shmem:3552kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:10776kB pagetables:2324kB sec_pagetables:0kB all_unreclaimable? no
Node 1 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:4kB writeback:0kB shmem:1536kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:16kB pagetables:0kB sec_pagetables:0kB all_unreclaimable? no
Node 0 DMA free:15360kB boost:0kB min:200kB low:248kB high:296kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 2611 2612 2612 2612
Node 0 DMA32 free:1783712kB boost:0kB min:35396kB low:44244kB high:53092kB reserved_highatomic:0KB active_anon:43144kB inactive_anon:1964kB active_file:0kB inactive_file:206836kB unevictable:1536kB writepending:60kB present:3129332kB managed:2678260kB mlocked:0kB bounce:0kB free_pcp:12452kB local_pcp:4300kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 Normal free:4kB boost:0kB min:8kB low:8kB high:8kB reserved_highatomic:0KB active_anon:36kB inactive_anon:4kB active_file:0kB inactive_file:816kB unevictable:0kB writepending:0kB present:1048576kB managed:872kB mlocked:0kB bounce:0kB free_pcp:12kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 1 Normal free:3927548kB boost:0kB min:54496kB low:68120kB high:81744kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB writepending:4kB present:4194304kB managed:4117312kB mlocked:0kB bounce:0kB free_pcp:15296kB local_pcp:8160kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15360kB
Node 0 DMA32: 1384*4kB (UME) 946*8kB (UME) 215*16kB (UME) 94*32kB (UME) 43*64kB (UME) 19*128kB (UME) 3*256kB (UM) 2*512kB (ME) 4*1024kB (UME) 2*2048kB (ME) 427*4096kB (M) = 1783712kB
Node 0 Normal: 1*4kB (M) 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4kB
Node 1 Normal: 193*4kB (UE) 55*8kB (UE) 38*16kB (UE) 77*32kB (UE) 27*64kB (UME) 5*128kB (UE) 2*256kB (UM) 3*512kB (UME) 1*1024kB (U) 1*2048kB (U) 956*4096kB (M) = 3927548kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 0 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 1 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
53203 total pagecache pages
0 pages in swap cache
Free swap = 124996kB
Total swap = 124996kB
2097051 pages RAM
0 pages HighMem/MovableOnly
394100 pages reserved
0 pages cma reserved
Mem-Info:
active_anon:10795 inactive_anon:492 isolated_anon:0
active_file:0 inactive_file:51931 isolated_file:0
unevictable:768 dirty:16 writeback:0
slab_reclaimable:9708 slab_unreclaimable:98313
mapped:7045 shmem:1272 pagetables:581
sec_pagetables:0 bounce:0
kernel_misc_reclaimable:0
free:1431656 free_pcp:6940 free_cma:0
Node 0 active_anon:43180kB inactive_anon:1968kB active_file:0kB inactive_file:207652kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:28180kB dirty:60kB writeback:0kB shmem:3552kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:10776kB pagetables:2324kB sec_pagetables:0kB all_unreclaimable? no
Node 1 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:4kB writeback:0kB shmem:1536kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:16kB pagetables:0kB sec_pagetables:0kB all_unreclaimable? no
Node 0 DMA free:15360kB boost:0kB min:200kB low:248kB high:296kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 2611 2612 2612 2612
Node 0 DMA32 free:1783712kB boost:0kB min:35396kB low:44244kB high:53092kB reserved_highatomic:0KB active_anon:43144kB inactive_anon:1964kB active_file:0kB inactive_file:206836kB unevictable:1536kB writepending:60kB present:3129332kB managed:2678260kB mlocked:0kB bounce:0kB free_pcp:12452kB local_pcp:8152kB free_cma:0kB

NMI backtrace for cpu 0

CPU: 0 PID: 6059 Comm: syz-executor.4 Not tainted 6.6.0-rc3-syzkaller-g6465e260f487 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/04/2023
RIP: 0010:io_serial_in+0x87/0xa0 drivers/tty/serial/8250/8250_port.c:418
Code: d1 eb fc 48 8d 7d 40 44 89 e1 48 b8 00 00 00 00 00 fc ff df 48 89 fa d3 e3 48 c1 ea 03 80 3c 02 00 75 16 66 03 5d 40 89 da ec <5b> 0f b6 c0 5d 41 5c c3 e8 bc 86 41 fd eb a6 e8 15 87 41 fd eb e3
RSP: 0000:ffffc900000072d0 EFLAGS: 00000002
RAX: dffffc0000000000 RBX: 00000000000003fd RCX: 0000000000000000
RDX: 00000000000003fd RSI: ffffffff849bf500 RDI: ffffffff92560d40
RBP: ffffffff92560d00 R08: 0000000000000001 R09: 000000000000001f
R10: 0000000000000000 R11: 205d304320202020 R12: 0000000000000000
R13: 0000000000000020 R14: fffffbfff24ac1fa R15: dffffc0000000000
FS: 0000555556b1a480(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000

CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 00007f18fc2d7693 CR3: 0000000028e0a000 CR4: 00000000003506f0

DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:

<NMI>
</NMI>
<IRQ>
serial_in drivers/tty/serial/8250/8250.h:117 [inline]
serial_lsr_in drivers/tty/serial/8250/8250.h:139 [inline]
wait_for_lsr+0xd7/0x180 drivers/tty/serial/8250/8250_port.c:2086
serial8250_console_fifo_write drivers/tty/serial/8250/8250_port.c:3371 [inline]
serial8250_console_write+0xce5/0x1060 drivers/tty/serial/8250/8250_port.c:3449
console_emit_next_record kernel/printk/printk.c:2910 [inline]
console_flush_all+0x4eb/0xfb0 kernel/printk/printk.c:2966
console_unlock+0x10c/0x260 kernel/printk/printk.c:3035
vprintk_emit+0x17f/0x5f0 kernel/printk/printk.c:2307
vprintk+0x7b/0x90 kernel/printk/printk_safe.c:45
_printk+0xc8/0x100 kernel/printk/printk.c:2332
show_free_areas+0x1260/0x21b0 mm/show_mem.c:308
__show_mem+0x34/0x150 mm/show_mem.c:409
k_spec drivers/tty/vt/keyboard.c:667 [inline]
k_spec+0xea/0x140 drivers/tty/vt/keyboard.c:656
kbd_keycode drivers/tty/vt/keyboard.c:1524 [inline]
kbd_event+0xcc8/0x17c0 drivers/tty/vt/keyboard.c:1543
input_to_handler+0x382/0x4c0 drivers/input/input.c:132
input_pass_values.part.0+0x52f/0x7a0 drivers/input/input.c:161
input_pass_values drivers/input/input.c:148 [inline]
input_event_dispose+0x5ee/0x770 drivers/input/input.c:378
input_handle_event+0x11c/0xd80 drivers/input/input.c:406
input_repeat_key+0x251/0x340 drivers/input/input.c:2263
call_timer_fn+0x19e/0x580 kernel/time/timer.c:1700
expire_timers kernel/time/timer.c:1751 [inline]
__run_timers+0x764/0xb10 kernel/time/timer.c:2022
run_timer_softirq+0x58/0xd0 kernel/time/timer.c:2035
__do_softirq+0x218/0x965 kernel/softirq.c:553
invoke_softirq kernel/softirq.c:427 [inline]
__irq_exit_rcu kernel/softirq.c:632 [inline]
irq_exit_rcu+0xb7/0x120 kernel/softirq.c:644
sysvec_apic_timer_interrupt+0x93/0xc0 arch/x86/kernel/apic/apic.c:1074
</IRQ>
<TASK>
asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:645
RIP: 0010:percpu_counter_add_batch+0xe6/0x1f0 lib/percpu_counter.c:103
Code: e8 bf de 33 fd 48 85 db 75 42 e8 25 e3 33 fd 9c 5b 81 e3 00 02 00 00 31 ff 48 89 de e8 a3 de 33 fd 48 85 db 0f 85 bd 00 00 00 <48> 83 c4 18 5b 5d 41 5c 41 5d 41 5e 41 5f e9 f7 e2 33 fd e8 f2 e2
RSP: 0000:ffffc9000a177af8 EFLAGS: 00000293

RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000

RDX: ffff888014f38000 RSI: ffffffff8453e402 RDI: 0000000000000007
RBP: ffff888017f63d90 R08: 0000000000000007 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 000000000000000f
R13: 0000000000000001 R14: 0000607f4641c7f8 R15: 0000000000000020
percpu_counter_add include/linux/percpu_counter.h:69 [inline]
add_mm_counter include/linux/mm.h:2552 [inline]
set_pte_range+0x3b3/0x9d0 mm/memory.c:4355
filemap_map_order0_folio mm/filemap.c:3552 [inline]
filemap_map_pages+0x455/0x1310 mm/filemap.c:3598
do_fault_around mm/memory.c:4525 [inline]
do_read_fault mm/memory.c:4558 [inline]
do_fault mm/memory.c:4705 [inline]
do_pte_missing mm/memory.c:3669 [inline]
handle_pte_fault mm/memory.c:4978 [inline]
__handle_mm_fault+0x291c/0x3e10 mm/memory.c:5119
handle_mm_fault+0x478/0xa00 mm/memory.c:5284
do_user_addr_fault+0x30b/0x1000 arch/x86/mm/fault.c:1364
handle_page_fault arch/x86/mm/fault.c:1505 [inline]
exc_page_fault+0x5c/0xd0 arch/x86/mm/fault.c:1561
asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:570
RIP: 0033:0x7f18fc26f46d
Code: ff 31 c0 c3 44 31 c0 44 29 c0 c3 66 90 48 83 fa 01 7e 5e c4 41 01 ef ff 89 f8 09 f0 c1 e0 14 3d 00 00 00 f8 0f 87 c8 03 00 00 <c5> fe 6f 07 c5 fd 74 0e c5 85 74 d0 c5 ed df c9 c5 fd d7 c9 48 83
RSP: 002b:00007ffde748fd68 EFLAGS: 00010207
RAX: 000000006f300000 RBX: 00007f18fc200000 RCX: 00007f18fbe00000
RDX: 0000000000000007 RSI: 00007f18fc2c82f1 RDI: 00007f18fc2d7693
RBP: 0000000000000000 R08: 00007f18fc200000 R09: 0000000000000001
R10: 00007ffde74a2080 R11: 0000000001531dae R12: 00007f18fbe00058
R13: 000000000000104a R14: 00007f18fbe00000 R15: 00007f18fbe00060
</TASK>
lowmem_reserve[]: 0 0 0 0 0
Node 0 Normal free:4kB boost:0kB min:8kB low:8kB high:8kB reserved_highatomic:0KB active_anon:36kB inactive_anon:4kB active_file:0kB inactive_file:816kB unevictable:0kB writepending:0kB present:1048576kB managed:872kB mlocked:0kB bounce:0kB free_pcp:12kB local_pcp:12kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 1 Normal free:3927548kB boost:0kB min:54496kB low:68120kB high:81744kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB writepending:4kB present:4194304kB managed:4117312kB mlocked:0kB bounce:0kB free_pcp:15296kB local_pcp:7136kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15360kB
Node 0 DMA32: 1384*4kB (UME) 946*8kB (UME) 215*16kB (UME) 94*32kB (UME) 43*64kB (UME) 19*128kB (UME) 3*256kB (UM) 2*512kB (ME) 4*1024kB (UME) 2*2048kB (ME) 427*4096kB (M) = 1783712kB
Node 0 Normal: 1*4kB (M) 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4kB
Node 1 Normal: 193*4kB (UE) 55*8kB (UE) 38*16kB (UE) 77*32kB (UE) 27*64kB (UME) 5*128kB (UE) 2*256kB (UM) 3*512kB (UME) 1*1024kB (U) 1*2048kB (U) 956*4096kB (M) = 3927548kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 0 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 1 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
53203 total pagecache pages
0 pages in swap cache
Free swap = 124996kB
Total swap = 124996kB
2097051 pages RAM
0 pages HighMem/MovableOnly
394100 pages reserved
0 pages cma reserved
Mem-Info:
active_anon:10770 inactive_anon:492 isolated_anon:0
active_file:0 inactive_file:51931 isolated_file:0
unevictable:768 dirty:16 writeback:0
slab_reclaimable:9708 slab_unreclaimable:98402
mapped:7045 shmem:1272 pagetables:556
sec_pagetables:0 bounce:0
kernel_misc_reclaimable:0
free:1431656 free_pcp:6909 free_cma:0
Node 0 active_anon:43080kB inactive_anon:1968kB active_file:0kB inactive_file:207652kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:28180kB dirty:60kB writeback:0kB shmem:3552kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:10776kB pagetables:2224kB sec_pagetables:0kB all_unreclaimable? no
Node 1 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:4kB writeback:0kB shmem:1536kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:16kB pagetables:0kB sec_pagetables:0kB all_unreclaimable? no
Node 0 DMA free:15360kB boost:0kB min:200kB low:248kB high:296kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 2611 2612 2612 2612
Node 0 DMA32 free:1783712kB boost:0kB min:35396kB low:44244kB high:53092kB reserved_highatomic:0KB active_anon:43044kB inactive_anon:1964kB active_file:0kB inactive_file:206836kB unevictable:1536kB writepending:60kB present:3129332kB managed:2678260kB mlocked:0kB bounce:0kB free_pcp:12328kB local_pcp:4176kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 Normal free:4kB boost:0kB min:8kB low:8kB high:8kB reserved_highatomic:0KB active_anon:36kB inactive_anon:4kB active_file:0kB inactive_file:816kB unevictable:0kB writepending:0kB present:1048576kB managed:872kB mlocked:0kB bounce:0kB free_pcp:12kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 1 Normal free:3927548kB boost:0kB min:54496kB low:68120kB high:81744kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB writepending:4kB present:4194304kB managed:4117312kB mlocked:0kB bounce:0kB free_pcp:15296kB local_pcp:8160kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15360kB
Node 0 DMA32: 1384*4kB (UME) 946*8kB (UME) 215*16kB (UME) 94*32kB (UME) 43*64kB (UME) 19*128kB (UME) 3*256kB (UM) 2*512kB (ME) 4*1024kB (UME) 2*2048kB (ME) 427*4096kB (M) = 1783712kB
Node 0 Normal: 1*4kB (M) 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4kB
Node 1 Normal: 193*4kB (UE) 55*8kB (UE) 38*16kB (UE) 77*32kB (UE) 27*64kB (UME) 5*128kB (UE) 2*256kB (UM) 3*512kB (UME) 1*1024kB (U) 1*2048kB (U) 956*4096kB (M) = 3927548kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 0 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 1 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
53203 total pagecache pages
0 pages in swap cache
Free swap = 124996kB
Total swap = 124996kB
2097051 pages RAM
0 pages HighMem/MovableOnly
394100 pages reserved
0 pages cma reserved
Mem-Info:
active_anon:10770 inactive_anon:492 isolated_anon:0
active_file:0 inactive_file:51931 isolated_file:0
unevictable:768 dirty:16 writeback:0
slab_reclaimable:9708 slab_unreclaimable:98402
mapped:7045 shmem:1272 pagetables:556
sec_pagetables:0 bounce:0
kernel_misc_reclaimable:0
free:1431656 free_pcp:6907 free_cma:0
Node 0 active_anon:42980kB inactive_anon:1968kB active_file:0kB inactive_file:207652kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:28180kB dirty:60kB writeback:0kB shmem:3552kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:10776kB pagetables:2224kB sec_pagetables:0kB all_unreclaimable? no
Node 1 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:4kB writeback:0kB shmem:1536kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:16kB pagetables:0kB sec_pagetables:0kB all_unreclaimable? no
Node 0 DMA free:15360kB boost:0kB min:200kB low:248kB high:296kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 2611 2612 2612 2612
Node 0 DMA32 free:1783712kB boost:0kB min:35396kB low:44244kB high:53092kB reserved_highatomic:0KB active_anon:42944kB inactive_anon:1964kB active_file:0kB inactive_file:206836kB unevictable:1536kB writepending:60kB present:3129332kB managed:2678260kB mlocked:0kB bounce:0kB free_pcp:12588kB local_pcp:8152kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 Normal free:4kB boost:0kB min:8kB low:8kB high:8kB reserved_highatomic:0KB active_anon:36kB inactive_anon:4kB active_file:0kB inactive_file:816kB unevictable:0kB writepending:0kB present:1048576kB managed:872kB mlocked:0kB bounce:0kB free_pcp:12kB local_pcp:12kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 1 Normal free:3927548kB boost:0kB min:54496kB low:68120kB high:81744kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB writepending:4kB present:4194304kB managed:4117312kB mlocked:0kB bounce:0kB free_pcp:15296kB local_pcp:7136kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15360kB
Node 0 DMA32: 1384*4kB (UME) 946*8kB (UME) 215*16kB (UME) 94*32kB (UME) 43*64kB (UME) 19*128kB (UME) 3*256kB (UM) 2*512kB (ME) 4*1024kB (UME) 2*2048kB (ME) 427*4096kB (M) = 1783712kB
Node 0 Normal: 1*4kB (M) 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4kB
Node 1 Normal: 193*4kB (UE) 55*8kB (UE) 38*16kB (UE) 77*32kB (UE) 27*64kB (UME) 5*128kB (UE) 2*256kB (UM) 3*512kB (UME) 1*1024kB (U) 1*2048kB (U) 956*4096kB (M) = 3927548kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 0 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 1 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
53203 total pagecache pages
0 pages in swap cache
Free swap = 124996kB
Total swap = 124996kB
2097051 pages RAM
0 pages HighMem/MovableOnly
394100 pages reserved
0 pages cma reserved
Mem-Info:
active_anon:10745 inactive_anon:492 isolated_anon:0
active_file:0 inactive_file:51931 isolated_file:0
unevictable:768 dirty:16 writeback:0
slab_reclaimable:9708 slab_unreclaimable:98392
mapped:7045 shmem:1272 pagetables:556
sec_pagetables:0 bounce:0
kernel_misc_reclaimable:0
free:1431656 free_pcp:6974 free_cma:0
Node 0 active_anon:42980kB inactive_anon:1968kB active_file:0kB inactive_file:207652kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:28180kB dirty:60kB writeback:0kB shmem:3552kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:10776kB pagetables:2224kB sec_pagetables:0kB all_unreclaimable? no
Node 1 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:4kB writeback:0kB shmem:1536kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:16kB pagetables:0kB sec_pagetables:0kB all_unreclaimable? no
Node 0 DMA free:15360kB boost:0kB min:200kB low:248kB high:296kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 2611 2612 2612 2612
Node 0 DMA32 free:1783712kB boost:0kB min:35396kB low:44244kB high:53092kB reserved_highatomic:0KB active_anon:42944kB inactive_anon:1964kB active_file:0kB inactive_file:206836kB unevictable:1536kB writepending:60kB present:3129332kB managed:2678260kB mlocked:0kB bounce:0kB free_pcp:12588kB local_pcp:4436kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 Normal free:4kB boost:0kB min:8kB low:8kB high:8kB reserved_highatomic:0KB active_anon:36kB inactive_anon:4kB active_file:0kB inactive_file:816kB unevictable:0kB writepending:0kB present:1048576kB managed:872kB mlocked:0kB bounce:0kB free_pcp:12kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 1 Normal free:3927548kB boost:0kB min:54496kB low:68120kB high:81744kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB writepending:4kB present:4194304kB managed:4117312kB mlocked:0kB bounce:0kB free_pcp:15296kB local_pcp:8160kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15360kB
Node 0 DMA32: 1384*4kB (UME) 946*8kB (UME) 215*16kB (UME) 94*32kB (UME) 43*64kB (UME) 19*128kB (UME) 3*256kB (UM) 2*512kB (ME) 4*1024kB (UME) 2*2048kB (ME) 427*4096kB (M) = 1783712kB
Node 0 Normal: 1*4kB (M) 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4kB
Node 1 Normal: 193*4kB (UE) 55*8kB (UE) 38*16kB (UE) 77*32kB (UE) 27*64kB (UME) 5*128kB (UE) 2*256kB (UM) 3*512kB (UME) 1*1024kB (U) 1*2048kB (U) 956*4096kB (M) = 3927548kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 0 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 1 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
53203 total pagecache pages
0 pages in swap cache
Free swap = 124996kB
Total swap = 124996kB
2097051 pages RAM
0 pages HighMem/MovableOnly
394100 pages reserved
0 pages cma reserved
Mem-Info:
active_anon:10745 inactive_anon:492 isolated_anon:0
active_file:0 inactive_file:51931 isolated_file:0
unevictable:768 dirty:16 writeback:0
slab_reclaimable:9708 slab_unreclaimable:98392
mapped:7045 shmem:1272 pagetables:556
sec_pagetables:0 bounce:0
kernel_misc_reclaimable:0
free:1431656 free_pcp:6974 free_cma:0
Node 0 active_anon:42880kB inactive_anon:1968kB active_file:0kB inactive_file:207652kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:28180kB dirty:60kB writeback:0kB shmem:3552kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:10748kB pagetables:2124kB sec_pagetables:0kB all_unreclaimable? no
Node 1 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:4kB writeback:0kB shmem:1536kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:16kB pagetables:0kB sec_pagetables:0kB all_unreclaimable? no
Node 0 DMA free:15360kB boost:0kB min:200kB low:248kB high:296kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 2611 2612 2612 2612
Node 0 DMA32 free:1783712kB boost:0kB min:35396kB low:44244kB high:53092kB reserved_highatomic:0KB active_anon:42844kB inactive_anon:1964kB active_file:0kB inactive_file:206836kB unevictable:1536kB writepending:60kB present:3129332kB managed:2678260kB mlocked:0kB bounce:0kB free_pcp:12812kB local_pcp:8152kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 Normal free:4kB boost:0kB min:8kB low:8kB high:8kB reserved_highatomic:0KB active_anon:36kB inactive_anon:4kB active_file:0kB inactive_file:816kB unevictable:0kB writepending:0kB present:1048576kB managed:872kB mlocked:0kB bounce:0kB free_pcp:12kB local_pcp:12kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 1 Normal free:3927548kB boost:0kB min:54496kB low:68120kB high:81744kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB writepending:4kB present:4194304kB managed:4117312kB mlocked:0kB bounce:0kB free_pcp:15296kB local_pcp:7136kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15360kB
Node 0 DMA32: 1384*4kB (UME) 946*8kB (UME) 215*16kB (UME) 94*32kB (UME) 43*64kB (UME) 19*128kB (UME) 3*256kB (UM) 2*512kB (ME) 4*1024kB (UME) 2*2048kB (ME) 427*4096kB (M) = 1783712kB
Node 0 Normal: 1*4kB (M) 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4kB
Node 1 Normal: 193*4kB (UE) 55*8kB (UE) 38*16kB (UE) 77*32kB (UE) 27*64kB (UME) 5*128kB (UE) 2*256kB (UM) 3*512kB (UME) 1*1024kB (U) 1*2048kB (U) 956*4096kB (M) = 3927548kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 0 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 1 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
53203 total pagecache pages
0 pages in swap cache
Free swap = 124996kB
Total swap = 124996kB
2097051 pages RAM
0 pages HighMem/MovableOnly
394100 pages reserved
0 pages cma reserved
Mem-Info:
active_anon:10720 inactive_anon:492 isolated_anon:0
active_file:0 inactive_file:51931 isolated_file:0
unevictable:768 dirty:16 writeback:0
slab_reclaimable:9708 slab_unreclaimable:98392
mapped:7045 shmem:1272 pagetables:531
sec_pagetables:0 bounce:0
kernel_misc_reclaimable:0
free:1431656 free_pcp:7030 free_cma:0
Node 0 active_anon:42880kB inactive_anon:1968kB active_file:0kB inactive_file:207652kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:28180kB dirty:60kB writeback:0kB shmem:3552kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:10748kB pagetables:2124kB sec_pagetables:0kB all_unreclaimable? no
Node 1 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:4kB writeback:0kB shmem:1536kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:16kB pagetables:0kB sec_pagetables:0kB all_unreclaimable? no
Node 0 DMA free:15360kB boost:0kB min:200kB low:248kB high:296kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 2611 2612 2612 2612
Node 0 DMA32 free:1783712kB boost:0kB min:35396kB low:44244kB high:53092kB reserved_highatomic:0KB active_anon:42844kB inactive_anon:1964kB active_file:0kB inactive_file:206836kB unevictable:1536kB writepending:60kB present:3129332kB managed:2678260kB mlocked:0kB bounce:0kB free_pcp:12812kB local_pcp:4660kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 Normal free:4kB boost:0kB min:8kB low:8kB high:8kB reserved_highatomic:0KB active_anon:36kB inactive_anon:4kB active_file:0kB inactive_file:816kB unevictable:0kB writepending:0kB present:1048576kB managed:872kB mlocked:0kB bounce:0kB free_pcp:12kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 1 Normal free:3927548kB boost:0kB min:54496kB low:68120kB high:81744kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB writepending:4kB present:4194304kB managed:4117312kB mlocked:0kB bounce:0kB free_pcp:15296kB local_pcp:8160kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15360kB
Node 0 DMA32: 1384*4kB (UME) 946*8kB (UME) 215*16kB (UME) 94*32kB (UME) 43*64kB (UME) 19*128kB (UME) 3*256kB (UM) 2*512kB (ME) 4*1024kB (UME) 2*2048kB (ME) 427*4096kB (M) = 1783712kB
Node 0 Normal: 1*4kB (M) 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4kB
Node 1 Normal: 193*4kB (UE) 55*8kB (UE) 38*16kB (UE) 77*32kB (UE) 27*64kB (UME) 5*128kB (UE) 2*256kB (UM) 3*512kB (UME) 1*1024kB (U) 1*2048kB (U) 956*4096kB (M) = 3927548kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 0 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 1 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
53203 total pagecache pages
0 pages in swap cache
Free swap = 124996kB
Total swap = 124996kB
2097051 pages RAM
0 pages HighMem/MovableOnly
394100 pages reserved
0 pages cma reserved
Mem-Info:
active_anon:10720 inactive_anon:492 isolated_anon:0
active_file:0 inactive_file:51931 isolated_file:0
unevictable:768 dirty:16 writeback:0
slab_reclaimable:9708 slab_unreclaimable:98420
mapped:7045 shmem:1272 pagetables:531
sec_pagetables:0 bounce:0
kernel_misc_reclaimable:0
free:1431656 free_pcp:7022 free_cma:0
Node 0 active_anon:42880kB inactive_anon:1968kB active_file:0kB inactive_file:207652kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:28180kB dirty:60kB writeback:0kB shmem:3552kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:10748kB pagetables:2124kB sec_pagetables:0kB all_unreclaimable? no
Node 1 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:4kB writeback:0kB shmem:1536kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:16kB pagetables:0kB sec_pagetables:0kB all_unreclaimable? no
Node 0 DMA free:15360kB boost:0kB min:200kB low:248kB high:296kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 2611 2612 2612 2612
Node 0 DMA32 free:1783712kB boost:0kB min:35396kB low:44244kB high:53092kB reserved_highatomic:0KB active_anon:42844kB inactive_anon:1964kB active_file:0kB inactive_file:206836kB unevictable:1536kB writepending:60kB present:3129332kB managed:2678260kB mlocked:0kB bounce:0kB free_pcp:13004kB local_pcp:8152kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 Normal free:4kB boost:0kB min:8kB low:8kB high:8kB reserved_highatomic:0KB active_anon:36kB inactive_anon:4kB active_file:0kB inactive_file:816kB unevictable:0kB writepending:0kB present:1048576kB managed:872kB mlocked:0kB bounce:0kB free_pcp:12kB local_pcp:12kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 1 Normal free:3927548kB boost:0kB min:54496kB low:68120kB high:81744kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB writepending:4kB present:4194304kB managed:4117312kB mlocked:0kB bounce:0kB free_pcp:15296kB local_pcp:7136kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15360kB
Node 0 DMA32: 1384*4kB (UME) 946*8kB (UME) 215*16kB (UME) 94*32kB (UME) 43*64kB (UME) 19*128kB (UME) 3*256kB (UM) 2*512kB (ME) 4*1024kB (UME) 2*2048kB (ME) 427*4096kB (M) = 1783712kB
Node 0 Normal: 1*4kB (M) 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4kB
Node 1 Normal: 193*4kB (UE) 55*8kB (UE) 38*16kB (UE) 77*32kB (UE) 27*64kB (UME) 5*128kB (UE) 2*256kB (UM) 3*512kB (UME) 1*1024kB (U) 1*2048kB (U) 956*4096kB (M) = 3927548kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 0 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 1 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
53203 total pagecache pages
0 pages in swap cache
Free swap = 124996kB
Total swap = 124996kB
2097051 pages RAM
0 pages HighMem/MovableOnly
394100 pages reserved
0 pages cma reserved
Mem-Info:
active_anon:10720 inactive_anon:492 isolated_anon:0
active_file:0 inactive_file:51931 isolated_file:0
unevictable:768 dirty:16 writeback:0
slab_reclaimable:9708 slab_unreclaimable:98356
mapped:7045 shmem:1272 pagetables:531
sec_pagetables:0 bounce:0
kernel_misc_reclaimable:0
free:1431656 free_pcp:7078 free_cma:0
Node 0 active_anon:42880kB inactive_anon:1968kB active_file:0kB inactive_file:207652kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:28180kB dirty:60kB writeback:0kB shmem:3552kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:10748kB pagetables:2124kB sec_pagetables:0kB all_unreclaimable? no
Node 1 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:4kB writeback:0kB shmem:1536kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:16kB pagetables:0kB sec_pagetables:0kB all_unreclaimable? no
Node 0 DMA free:15360kB boost:0kB min:200kB low:248kB high:296kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 2611 2612 2612 2612
Node 0 DMA32 free:1783712kB boost:0kB min:35396kB low:44244kB high:53092kB reserved_highatomic:0KB active_anon:42844kB inactive_anon:1964kB active_file:0kB inactive_file:206836kB unevictable:1536kB writepending:60kB present:3129332kB managed:2678260kB mlocked:0kB bounce:0kB free_pcp:13004kB local_pcp:4852kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 Normal free:4kB boost:0kB min:8kB low:8kB high:8kB reserved_highatomic:0KB active_anon:36kB inactive_anon:4kB active_file:0kB inactive_file:816kB unevictable:0kB writepending:0kB present:1048576kB managed:872kB mlocked:0kB bounce:0kB free_pcp:12kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 1 Normal free:3927548kB boost:0kB min:54496kB low:68120kB high:81744kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB writepending:4kB present:4194304kB managed:4117312kB mlocked:0kB bounce:0kB free_pcp:15296kB local_pcp:8160kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15360kB
Node 0 DMA32: 1384*4kB (UME) 946*8kB (UME) 215*16kB (UME) 94*32kB (UME) 43*64kB (UME) 19*128kB (UME) 3*256kB (UM) 2*512kB (ME) 4*1024kB (UME) 2*2048kB (ME) 427*4096kB (M) = 1783712kB
Node 0 Normal: 1*4kB (M) 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4kB
Node 1 Normal: 193*4kB (UE) 55*8kB (UE) 38*16kB (UE) 77*32kB (UE) 27*64kB (UME) 5*128kB (UE) 2*256kB (UM) 3*512kB (UME) 1*1024kB (U) 1*2048kB (U) 956*4096kB (M) = 3927548kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 0 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 1 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
53203 total pagecache pages
0 pages in swap cache
Free swap = 124996kB
Total swap = 124996kB
2097051 pages RAM
0 pages HighMem/MovableOnly
394100 pages reserved
0 pages cma reserved
Mem-Info:
active_anon:10720 inactive_anon:492 isolated_anon:0
active_file:0 inactive_file:51931 isolated_file:0
unevictable:768 dirty:16 writeback:0
slab_reclaimable:9708 slab_unreclaimable:98356
mapped:7045 shmem:1272 pagetables:531
sec_pagetables:0 bounce:0
kernel_misc_reclaimable:0
free:1431656 free_pcp:7078 free_cma:0
Node 0 active_anon:42880kB inactive_anon:1968kB active_file:0kB inactive_file:207652kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:28180kB dirty:60kB writeback:0kB shmem:3552kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:10720kB pagetables:2124kB sec_pagetables:0kB all_unreclaimable? no
Node 1 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:4kB writeback:0kB shmem:1536kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:16kB pagetables:0kB sec_pagetables:0kB all_unreclaimable? no
Node 0 DMA free:15360kB boost:0kB min:200kB low:248kB high:296kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 2611 2612 2612 2612
Node 0 DMA32 free:1783712kB boost:0kB min:35396kB low:44244kB high:53092kB reserved_highatomic:0KB active_anon:42844kB inactive_anon:1964kB active_file:0kB inactive_file:206836kB unevictable:1536kB writepending:60kB present:3129332kB managed:2678260kB mlocked:0kB bounce:0kB free_pcp:13704kB local_pcp:8152kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 Normal free:4kB boost:0kB min:8kB low:8kB high:8kB reserved_highatomic:0KB active_anon:36kB inactive_anon:4kB active_file:0kB inactive_file:816kB unevictable:0kB writepending:0kB present:1048576kB managed:872kB mlocked:0kB bounce:0kB free_pcp:12kB local_pcp:12kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 1 Normal free:3927548kB boost:0kB min:54496kB low:68120kB high:81744kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB writepending:4kB present:4194304kB managed:4117312kB mlocked:0kB bounce:0kB free_pcp:15296kB local_pcp:7136kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15360kB
Node 0 DMA32: 1384*4kB (UME) 946*8kB (UME) 215*16kB (UME) 94*32kB (UME) 43*64kB (UME) 19*128kB (UME) 3*256kB (UM) 2*512kB (ME) 4*1024kB (UME) 2*2048kB (ME) 427*4096kB (M) = 1783712kB
Node 0 Normal: 1*4kB (M) 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4kB
Node 1 Normal: 193*4kB (UE) 55*8kB (UE) 38*16kB (UE) 77*32kB (UE) 27*64kB (UME) 5*128kB (UE) 2*256kB (UM) 3*512kB (UME) 1*1024kB (U) 1*2048kB (U) 956*4096kB (M) = 3927548kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 0 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 1 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
53203 total pagecache pages
0 pages in swap cache
Free swap = 124996kB
Total swap = 124996kB
2097051 pages RAM
0 pages HighMem/MovableOnly
394100 pages reserved
0 pages cma reserved
Mem-Info:
active_anon:10720 inactive_anon:492 isolated_anon:0
active_file:0 inactive_file:51931 isolated_file:0
unevictable:768 dirty:16 writeback:0
slab_reclaimable:9708 slab_unreclaimable:98172
mapped:7045 shmem:1272 pagetables:531
sec_pagetables:0 bounce:0
kernel_misc_reclaimable:0
free:1431656 free_pcp:7253 free_cma:0
Node 0 active_anon:42880kB inactive_anon:1968kB active_file:0kB inactive_file:207652kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:28180kB dirty:60kB writeback:0kB shmem:3552kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:10720kB pagetables:2124kB sec_pagetables:0kB all_unreclaimable? no
Node 1 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:4kB writeback:0kB shmem:1536kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:16kB pagetables:0kB sec_pagetables:0kB all_unreclaimable? no
Node 0 DMA free:15360kB boost:0kB min:200kB low:248kB high:296kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 2611 2612 2612 2612
Node 0 DMA32 free:1783712kB boost:0kB min:35396kB low:44244kB high:53092kB reserved_highatomic:0KB active_anon:42844kB inactive_anon:1964kB active_file:0kB inactive_file:206836kB unevictable:1536kB writepending:60kB present:3129332kB managed:2678260kB mlocked:0kB bounce:0kB free_pcp:13704kB local_pcp:5552kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 Normal free:4kB boost:0kB min:8kB low:8kB high:8kB reserved_highatomic:0KB active_anon:36kB inactive_anon:4kB active_file:0kB inactive_file:816kB unevictable:0kB writepending:0kB present:1048576kB managed:872kB mlocked:0kB bounce:0kB free_pcp:12kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 1 Normal free:3927548kB boost:0kB min:54496kB low:68120kB high:81744kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB writepending:4kB present:4194304kB managed:4117312kB mlocked:0kB bounce:0kB free_pcp:15296kB local_pcp:8160kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15360kB
Node 0 DMA32: 1384*4kB (UME) 946*8kB (UME) 215*16kB (UME) 94*32kB (UME) 43*64kB (UME) 19*128kB (UME) 3*256kB (UM) 2*512kB (ME) 4*1024kB (UME) 2*2048kB (ME) 427*4096kB (M) = 1783712kB
Node 0 Normal: 1*4kB (M) 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4kB
Node 1 Normal: 193*4kB (UE) 55*8kB (UE) 38*16kB (UE) 77*32kB (UE) 27*64kB (UME) 5*128kB (UE) 2*256kB (UM) 3*512kB (UME) 1*1024kB (U) 1*2048kB (U) 956*4096kB (M) = 3927548kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 0 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 1 hugepages_total=2 hugepages_free=2 hugepages_surp=0 hugepages_size=2048kB
53203 total pagecache pages
0 pages in swap cache
Free swap = 124996kB
Total swap = 124996kB
2097051 pages RAM
0 pages HighMem/MovableOnly
394100 pages reserved
0 pages cma reserved
Mem-Info:
active_anon:10720 inactive_anon:492 isolated_anon:0
active_file:0 inactive_file:51931 isolated_file:0
unevictable:768 dirty:16 writeback:0
slab_reclaimable:9708 slab_unreclaimable:98172
mapped:7045 shmem:1272 pagetables:531
sec_pagetables:0 bounce:0
kernel_misc_reclaimable:0
free:1431656 free_pcp:7253 free_cma:0
Node 0 active_anon:42880kB inactive_anon:1968kB active_file:0kB inactive_file:207652kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:28180kB dirty:60kB writeback:0kB shmem:3552kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:10692kB pagetables:2124kB sec_pagetables:0kB all_unreclaimable? no
Node 1 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:72kB unevictable:1536kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:4kB writeback:0kB shmem:1536kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:16kB pagetables:0kB sec_pagetables:0kB all_unreclaimable? no
Node 0 DMA free:15360kB boost:0kB min:200kB low:248kB high:296kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 2611 2612 2612 2612

Tested on:

commit: 6465e260 Linux 6.6-rc3
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master
console output: https://syzkaller.appspot.com/x/log.txt?x=1098761e680000
kernel config: https://syzkaller.appspot.com/x/.config?x=bb54ecdfa197f132
dashboard link: https://syzkaller.appspot.com/bug?extid=eec403943a2a2455adaa
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40

Note: no patches were applied.

Hillf Danton

unread,

Sep 25, 2023, 7:55:20 AM9/25/23

to syzbot, linux-...@vger.kernel.org, syzkall...@googlegroups.com

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master

--- x/drivers/input/input.c
+++ y/drivers/input/input.c
@@ -149,6 +149,8 @@ static void input_pass_values(struct inp

if (!count)
return;
+ if (count > 5)
+ count = 5;

rcu_read_lock();

--

syzbot

unread,

Sep 25, 2023, 8:23:28 AM9/25/23

to hda...@sina.com, linux-...@vger.kernel.org, syzkall...@googlegroups.com

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-and-tested-by: syzbot+eec403...@syzkaller.appspotmail.com

Tested on:

commit: 6465e260 Linux 6.6-rc3
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master

console output: https://syzkaller.appspot.com/x/log.txt?x=1690b7fe680000

kernel config: https://syzkaller.appspot.com/x/.config?x=bb54ecdfa197f132
dashboard link: https://syzkaller.appspot.com/bug?extid=eec403943a2a2455adaa
compiler: gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=133c9afa680000

Note: testing is done by a robot and is best-effort only.

syzbot

unread,

Mar 6, 2024, 5:38:04 PMMar 6

to b...@vger.kernel.org, core...@netfilter.org, da...@davemloft.net, dvy...@google.com, edum...@google.com, f...@strlen.de, gauta...@gmail.com, hda...@sina.com, j...@mojatatu.com, ji...@resnulli.us, kad...@netfilter.org, ku...@kernel.org, lesli...@gmail.com, linux-...@vger.kernel.org, michal...@intel.com, mohitbh...@gmail.com, net...@vger.kernel.org, netfilt...@vger.kernel.org, pab...@redhat.com, pa...@netfilter.org, pau...@kernel.org, sdp.s...@gmail.com, syzkall...@googlegroups.com, tahi...@nitk.edu.in, tg...@linutronix.de, vsaicha...@gmail.com, xiyou.w...@gmail.com

syzbot suspects this issue was fixed by commit:

commit 8c21ab1bae945686c602c5bfa4e3f3352c2452c5
Author: Eric Dumazet <edum...@google.com>
Date: Tue Aug 29 12:35:41 2023 +0000

net/sched: fq_pie: avoid stalls in fq_pie_timer()

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=1338df0e180000

start commit: d4a7ce642100 igc: Fix Kernel Panic during ndo_tx_timeout c..
git tree: net

kernel config: https://syzkaller.appspot.com/x/.config?x=77b9a3cf8f44c6da
dashboard link: https://syzkaller.appspot.com/bug?extid=eec403943a2a2455adaa
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1504b511a80000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=137bf931a80000

If the result looks correct, please mark the issue as fixed by replying with:

#syz fix: net/sched: fq_pie: avoid stalls in fq_pie_timer()

Reply all

Reply to author

Forward

[syzbot] INFO: rcu detected stall in gc_worker (3)

syzbot

Hillf Danton

syzbot

Hillf Danton

syzbot

Hillf Danton

syzbot

Hillf Danton

syzbot

Hillf Danton

Paul E. McKenney

Hillf Danton

syzbot

Hillf Danton

syzbot

Hillf Danton

syzbot

syzbot

Hillf Danton

syzbot

Hillf Danton

syzbot

syzbot