possible deadlock in pie_timer
7 views
Skip to first unread message
syzbot
Feb 3, 2022, 4:38:28 AM2/3/22
to syzkaller...@googlegroups.com
Hello,
syzbot found the following issue on:
HEAD commit: 3f8a27f9e27b Linux 4.19.211
git tree: linux-4.19.y
console output: https://syzkaller.appspot.com/x/log.txt?x=1153b9cc700000
kernel config: https://syzkaller.appspot.com/x/.config?x=9b9277b418617afe
dashboard link: https://syzkaller.appspot.com/bug?extid=de0e3bbaa3f13a456d32
compiler: gcc version 10.2.1 20210110 (Debian 10.2.1-6)
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=10c9cbb7b00000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=16578942700000
IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+de0e3b...@syzkaller.appspotmail.com
======================================================
WARNING: possible circular locking dependency detected
4.19.211-syzkaller #0 Not tainted
------------------------------------------------------
swapper/1/0 is trying to acquire lock:
00000000eeabeb81 (&qdisc_rx_lock){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline]
00000000eeabeb81 (&qdisc_rx_lock){+.-.}, at: pie_timer+0x92/0x740 net/sched/sch_pie.c:433
but task is already holding lock:
000000006bcb8190 ((&q->adapt_timer)){+.-.}, at: lockdep_copy_map include/linux/lockdep.h:168 [inline]
000000006bcb8190 ((&q->adapt_timer)){+.-.}, at: call_timer_fn+0xc9/0x700 kernel/time/timer.c:1328
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #1 ((&q->adapt_timer)){+.-.}:
qdisc_destroy+0x180/0x790 net/sched/sch_generic.c:983
sfb_change+0x2c9/0xb30 net/sched/sch_sfb.c:527
qdisc_change net/sched/sch_api.c:1240 [inline]
tc_modify_qdisc+0xf6a/0x1a80 net/sched/sch_api.c:1543
rtnetlink_rcv_msg+0x453/0xb80 net/core/rtnetlink.c:4782
netlink_rcv_skb+0x160/0x440 net/netlink/af_netlink.c:2463
netlink_unicast_kernel net/netlink/af_netlink.c:1325 [inline]
netlink_unicast+0x4d5/0x690 net/netlink/af_netlink.c:1351
netlink_sendmsg+0x6c3/0xc50 net/netlink/af_netlink.c:1917
sock_sendmsg_nosec net/socket.c:651 [inline]
sock_sendmsg+0xc3/0x120 net/socket.c:661
___sys_sendmsg+0x7bb/0x8e0 net/socket.c:2227
__sys_sendmsg net/socket.c:2265 [inline]
__do_sys_sendmsg net/socket.c:2274 [inline]
__se_sys_sendmsg net/socket.c:2272 [inline]
__x64_sys_sendmsg+0x132/0x220 net/socket.c:2272
do_syscall_64+0xf9/0x620 arch/x86/entry/common.c:293
entry_SYSCALL_64_after_hwframe+0x49/0xbe
-> #0 (&qdisc_rx_lock){+.-.}:
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
spin_lock include/linux/spinlock.h:329 [inline]
pie_timer+0x92/0x740 net/sched/sch_pie.c:433
call_timer_fn+0x177/0x700 kernel/time/timer.c:1338
expire_timers+0x243/0x4e0 kernel/time/timer.c:1375
__run_timers kernel/time/timer.c:1696 [inline]
run_timer_softirq+0x21c/0x670 kernel/time/timer.c:1709
__do_softirq+0x265/0x980 kernel/softirq.c:292
invoke_softirq kernel/softirq.c:372 [inline]
irq_exit+0x215/0x260 kernel/softirq.c:412
exiting_irq arch/x86/include/asm/apic.h:536 [inline]
smp_apic_timer_interrupt+0x136/0x550 arch/x86/kernel/apic/apic.c:1098
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:894
native_safe_halt+0xe/0x10 arch/x86/include/asm/irqflags.h:60
arch_safe_halt arch/x86/include/asm/paravirt.h:94 [inline]
default_idle+0x49/0x310 arch/x86/kernel/process.c:557
cpuidle_idle_call kernel/sched/idle.c:153 [inline]
do_idle+0x2ec/0x4b0 kernel/sched/idle.c:263
cpu_startup_entry+0xc5/0xe0 kernel/sched/idle.c:369
start_secondary+0x435/0x5c0 arch/x86/kernel/smpboot.c:271
secondary_startup_64+0xa4/0xb0 arch/x86/kernel/head_64.S:243
other info that might help us debug this:
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock((&q->adapt_timer));
lock(&qdisc_rx_lock);
lock((&q->adapt_timer));
lock(&qdisc_rx_lock);
*** DEADLOCK ***
1 lock held by swapper/1/0:
#0: 000000006bcb8190 ((&q->adapt_timer)){+.-.}, at: lockdep_copy_map include/linux/lockdep.h:168 [inline]
#0: 000000006bcb8190 ((&q->adapt_timer)){+.-.}, at: call_timer_fn+0xc9/0x700 kernel/time/timer.c:1328
stack backtrace:
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.19.211-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
<IRQ>
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x1fc/0x2ef lib/dump_stack.c:118
print_circular_bug.constprop.0.cold+0x2d7/0x41e kernel/locking/lockdep.c:1222
check_prev_add kernel/locking/lockdep.c:1866 [inline]
check_prevs_add kernel/locking/lockdep.c:1979 [inline]
validate_chain kernel/locking/lockdep.c:2420 [inline]
__lock_acquire+0x30c9/0x3ff0 kernel/locking/lockdep.c:3416
lock_acquire+0x170/0x3c0 kernel/locking/lockdep.c:3908
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
spin_lock include/linux/spinlock.h:329 [inline]
pie_timer+0x92/0x740 net/sched/sch_pie.c:433
call_timer_fn+0x177/0x700 kernel/time/timer.c:1338
expire_timers+0x243/0x4e0 kernel/time/timer.c:1375
__run_timers kernel/time/timer.c:1696 [inline]
run_timer_softirq+0x21c/0x670 kernel/time/timer.c:1709
__do_softirq+0x265/0x980 kernel/softirq.c:292
invoke_softirq kernel/softirq.c:372 [inline]
irq_exit+0x215/0x260 kernel/softirq.c:412
exiting_irq arch/x86/include/asm/apic.h:536 [inline]
smp_apic_timer_interrupt+0x136/0x550 arch/x86/kernel/apic/apic.c:1098
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:894
</IRQ>
RIP: 0010:native_safe_halt+0xe/0x10 arch/x86/include/asm/irqflags.h:61
Code: 48 89 df e8 f4 20 7f f9 e9 2e ff ff ff 48 89 df e8 e7 20 7f f9 eb 82 90 90 90 90 90 e9 07 00 00 00 0f 00 2d 14 43 4e 00 fb f4 <c3> 90 e9 07 00 00 00 0f 00 2d 04 43 4e 00 f4 c3 90 90 41 56 41 55
RSP: 0018:ffff8880b5a9fd40 EFLAGS: 00000282 ORIG_RAX: ffffffffffffff13
RAX: 1ffffffff13e3054 RBX: dffffc0000000000 RCX: 0000000000000000
RDX: dffffc0000000000 RSI: 0000000000000001 RDI: ffff8880b5a86c44
RBP: 0000000000000001 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff89f18290
R13: 1ffff11016b53fb2 R14: 0000000000000000 R15: 0000000000000000
arch_safe_halt arch/x86/include/asm/paravirt.h:94 [inline]
default_idle+0x49/0x310 arch/x86/kernel/process.c:557
cpuidle_idle_call kernel/sched/idle.c:153 [inline]
do_idle+0x2ec/0x4b0 kernel/sched/idle.c:263
cpu_startup_entry+0xc5/0xe0 kernel/sched/idle.c:369
start_secondary+0x435/0x5c0 arch/x86/kernel/smpboot.c:271
secondary_startup_64+0xa4/0xb0 arch/x86/kernel/head_64.S:243
----------------
Code disassembly (best guess):
0: 48 89 df mov %rbx,%rdi
3: e8 f4 20 7f f9 callq 0xf97f20fc
8: e9 2e ff ff ff jmpq 0xffffff3b
d: 48 89 df mov %rbx,%rdi
10: e8 e7 20 7f f9 callq 0xf97f20fc
15: eb 82 jmp 0xffffff99
17: 90 nop
18: 90 nop
19: 90 nop
1a: 90 nop
1b: 90 nop
1c: e9 07 00 00 00 jmpq 0x28
21: 0f 00 2d 14 43 4e 00 verw 0x4e4314(%rip) # 0x4e433c
28: fb sti
29: f4 hlt
* 2a: c3 retq <-- trapping instruction
2b: 90 nop
2c: e9 07 00 00 00 jmpq 0x38
31: 0f 00 2d 04 43 4e 00 verw 0x4e4304(%rip) # 0x4e433c
38: f4 hlt
39: c3 retq
3a: 90 nop
3b: 90 nop
3c: 41 56 push %r14
3e: 41 55 push %r13
---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.
syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
syzbot can test patches for this issue, for details see:
https://goo.gl/tpsmEJ#testing-patches
syzbot found the following issue on:
HEAD commit: 3f8a27f9e27b Linux 4.19.211
git tree: linux-4.19.y
console output: https://syzkaller.appspot.com/x/log.txt?x=1153b9cc700000
kernel config: https://syzkaller.appspot.com/x/.config?x=9b9277b418617afe
dashboard link: https://syzkaller.appspot.com/bug?extid=de0e3bbaa3f13a456d32
compiler: gcc version 10.2.1 20210110 (Debian 10.2.1-6)
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=10c9cbb7b00000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=16578942700000
IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+de0e3b...@syzkaller.appspotmail.com
======================================================
WARNING: possible circular locking dependency detected
4.19.211-syzkaller #0 Not tainted
------------------------------------------------------
swapper/1/0 is trying to acquire lock:
00000000eeabeb81 (&qdisc_rx_lock){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline]
00000000eeabeb81 (&qdisc_rx_lock){+.-.}, at: pie_timer+0x92/0x740 net/sched/sch_pie.c:433
but task is already holding lock:
000000006bcb8190 ((&q->adapt_timer)){+.-.}, at: lockdep_copy_map include/linux/lockdep.h:168 [inline]
000000006bcb8190 ((&q->adapt_timer)){+.-.}, at: call_timer_fn+0xc9/0x700 kernel/time/timer.c:1328
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #1 ((&q->adapt_timer)){+.-.}:
qdisc_destroy+0x180/0x790 net/sched/sch_generic.c:983
sfb_change+0x2c9/0xb30 net/sched/sch_sfb.c:527
qdisc_change net/sched/sch_api.c:1240 [inline]
tc_modify_qdisc+0xf6a/0x1a80 net/sched/sch_api.c:1543
rtnetlink_rcv_msg+0x453/0xb80 net/core/rtnetlink.c:4782
netlink_rcv_skb+0x160/0x440 net/netlink/af_netlink.c:2463
netlink_unicast_kernel net/netlink/af_netlink.c:1325 [inline]
netlink_unicast+0x4d5/0x690 net/netlink/af_netlink.c:1351
netlink_sendmsg+0x6c3/0xc50 net/netlink/af_netlink.c:1917
sock_sendmsg_nosec net/socket.c:651 [inline]
sock_sendmsg+0xc3/0x120 net/socket.c:661
___sys_sendmsg+0x7bb/0x8e0 net/socket.c:2227
__sys_sendmsg net/socket.c:2265 [inline]
__do_sys_sendmsg net/socket.c:2274 [inline]
__se_sys_sendmsg net/socket.c:2272 [inline]
__x64_sys_sendmsg+0x132/0x220 net/socket.c:2272
do_syscall_64+0xf9/0x620 arch/x86/entry/common.c:293
entry_SYSCALL_64_after_hwframe+0x49/0xbe
-> #0 (&qdisc_rx_lock){+.-.}:
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
spin_lock include/linux/spinlock.h:329 [inline]
pie_timer+0x92/0x740 net/sched/sch_pie.c:433
call_timer_fn+0x177/0x700 kernel/time/timer.c:1338
expire_timers+0x243/0x4e0 kernel/time/timer.c:1375
__run_timers kernel/time/timer.c:1696 [inline]
run_timer_softirq+0x21c/0x670 kernel/time/timer.c:1709
__do_softirq+0x265/0x980 kernel/softirq.c:292
invoke_softirq kernel/softirq.c:372 [inline]
irq_exit+0x215/0x260 kernel/softirq.c:412
exiting_irq arch/x86/include/asm/apic.h:536 [inline]
smp_apic_timer_interrupt+0x136/0x550 arch/x86/kernel/apic/apic.c:1098
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:894
native_safe_halt+0xe/0x10 arch/x86/include/asm/irqflags.h:60
arch_safe_halt arch/x86/include/asm/paravirt.h:94 [inline]
default_idle+0x49/0x310 arch/x86/kernel/process.c:557
cpuidle_idle_call kernel/sched/idle.c:153 [inline]
do_idle+0x2ec/0x4b0 kernel/sched/idle.c:263
cpu_startup_entry+0xc5/0xe0 kernel/sched/idle.c:369
start_secondary+0x435/0x5c0 arch/x86/kernel/smpboot.c:271
secondary_startup_64+0xa4/0xb0 arch/x86/kernel/head_64.S:243
other info that might help us debug this:
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock((&q->adapt_timer));
lock(&qdisc_rx_lock);
lock((&q->adapt_timer));
lock(&qdisc_rx_lock);
*** DEADLOCK ***
1 lock held by swapper/1/0:
#0: 000000006bcb8190 ((&q->adapt_timer)){+.-.}, at: lockdep_copy_map include/linux/lockdep.h:168 [inline]
#0: 000000006bcb8190 ((&q->adapt_timer)){+.-.}, at: call_timer_fn+0xc9/0x700 kernel/time/timer.c:1328
stack backtrace:
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.19.211-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
<IRQ>
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x1fc/0x2ef lib/dump_stack.c:118
print_circular_bug.constprop.0.cold+0x2d7/0x41e kernel/locking/lockdep.c:1222
check_prev_add kernel/locking/lockdep.c:1866 [inline]
check_prevs_add kernel/locking/lockdep.c:1979 [inline]
validate_chain kernel/locking/lockdep.c:2420 [inline]
__lock_acquire+0x30c9/0x3ff0 kernel/locking/lockdep.c:3416
lock_acquire+0x170/0x3c0 kernel/locking/lockdep.c:3908
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
spin_lock include/linux/spinlock.h:329 [inline]
pie_timer+0x92/0x740 net/sched/sch_pie.c:433
call_timer_fn+0x177/0x700 kernel/time/timer.c:1338
expire_timers+0x243/0x4e0 kernel/time/timer.c:1375
__run_timers kernel/time/timer.c:1696 [inline]
run_timer_softirq+0x21c/0x670 kernel/time/timer.c:1709
__do_softirq+0x265/0x980 kernel/softirq.c:292
invoke_softirq kernel/softirq.c:372 [inline]
irq_exit+0x215/0x260 kernel/softirq.c:412
exiting_irq arch/x86/include/asm/apic.h:536 [inline]
smp_apic_timer_interrupt+0x136/0x550 arch/x86/kernel/apic/apic.c:1098
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:894
</IRQ>
RIP: 0010:native_safe_halt+0xe/0x10 arch/x86/include/asm/irqflags.h:61
Code: 48 89 df e8 f4 20 7f f9 e9 2e ff ff ff 48 89 df e8 e7 20 7f f9 eb 82 90 90 90 90 90 e9 07 00 00 00 0f 00 2d 14 43 4e 00 fb f4 <c3> 90 e9 07 00 00 00 0f 00 2d 04 43 4e 00 f4 c3 90 90 41 56 41 55
RSP: 0018:ffff8880b5a9fd40 EFLAGS: 00000282 ORIG_RAX: ffffffffffffff13
RAX: 1ffffffff13e3054 RBX: dffffc0000000000 RCX: 0000000000000000
RDX: dffffc0000000000 RSI: 0000000000000001 RDI: ffff8880b5a86c44
RBP: 0000000000000001 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff89f18290
R13: 1ffff11016b53fb2 R14: 0000000000000000 R15: 0000000000000000
arch_safe_halt arch/x86/include/asm/paravirt.h:94 [inline]
default_idle+0x49/0x310 arch/x86/kernel/process.c:557
cpuidle_idle_call kernel/sched/idle.c:153 [inline]
do_idle+0x2ec/0x4b0 kernel/sched/idle.c:263
cpu_startup_entry+0xc5/0xe0 kernel/sched/idle.c:369
start_secondary+0x435/0x5c0 arch/x86/kernel/smpboot.c:271
secondary_startup_64+0xa4/0xb0 arch/x86/kernel/head_64.S:243
----------------
Code disassembly (best guess):
0: 48 89 df mov %rbx,%rdi
3: e8 f4 20 7f f9 callq 0xf97f20fc
8: e9 2e ff ff ff jmpq 0xffffff3b
d: 48 89 df mov %rbx,%rdi
10: e8 e7 20 7f f9 callq 0xf97f20fc
15: eb 82 jmp 0xffffff99
17: 90 nop
18: 90 nop
19: 90 nop
1a: 90 nop
1b: 90 nop
1c: e9 07 00 00 00 jmpq 0x28
21: 0f 00 2d 14 43 4e 00 verw 0x4e4314(%rip) # 0x4e433c
28: fb sti
29: f4 hlt
* 2a: c3 retq <-- trapping instruction
2b: 90 nop
2c: e9 07 00 00 00 jmpq 0x38
31: 0f 00 2d 04 43 4e 00 verw 0x4e4304(%rip) # 0x4e433c
38: f4 hlt
39: c3 retq
3a: 90 nop
3b: 90 nop
3c: 41 56 push %r14
3e: 41 55 push %r13
---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.
syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
syzbot can test patches for this issue, for details see:
https://goo.gl/tpsmEJ#testing-patches
Reply all
Reply to author
Forward
0 new messages