kernel panic: stack is corrupted in lock_acquire

18 views
Skip to first unread message

syzbot

unread,
Jan 4, 2019, 6:17:04 AM1/4/19
to syzkaller-upst...@googlegroups.com
Hello,

syzbot found the following crash on:

HEAD commit: 645ff1e8e704 Merge branch 'for-linus' of git://git.kernel...
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=175cda9f400000
kernel config: https://syzkaller.appspot.com/x/.config?x=7308e68273924137
dashboard link: https://syzkaller.appspot.com/bug?extid=881ce26f0a76c7598ed4
compiler: gcc (GCC) 9.0.0 20181231 (experimental)
CC: [linux-...@vger.kernel.org tg...@linutronix.de]

Unfortunately, I don't have any reproducer for this crash yet.

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+881ce2...@syzkaller.appspotmail.com

Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in:
lock_acquire+0x50c/0x570 kernel/locking/lockdep.c:3845
CPU: 1 PID: 8981 Comm: syz-executor2 Not tainted 4.20.0+ #8
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:

======================================================
WARNING: possible circular locking dependency detected
4.20.0+ #8 Not tainted
------------------------------------------------------
syz-executor2/8981 is trying to acquire lock:
0000000038875520 ((console_sem).lock){-.-.}, at: down_trylock+0x13/0x70
kernel/locking/semaphore.c:136

but task is already holding lock:
00000000244bdc8b (hrtimer_bases.lock){-.-.}, at:
hrtimer_interrupt+0xff/0x770 kernel/time/hrtimer.c:1490

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #4 (hrtimer_bases.lock){-.-.}:
__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0x95/0xcd kernel/locking/spinlock.c:152
lock_hrtimer_base.isra.0+0x75/0x130 kernel/time/hrtimer.c:165
hrtimer_start_range_ns+0x120/0xda0 kernel/time/hrtimer.c:1104
hrtimer_start_expires include/linux/hrtimer.h:409 [inline]
start_rt_bandwidth kernel/sched/rt.c:70 [inline]
inc_rt_group kernel/sched/rt.c:1147 [inline]
inc_rt_tasks kernel/sched/rt.c:1191 [inline]
__enqueue_rt_entity kernel/sched/rt.c:1261 [inline]
enqueue_rt_entity kernel/sched/rt.c:1305 [inline]
enqueue_task_rt+0x95b/0x1010 kernel/sched/rt.c:1335
enqueue_task+0xb9/0x380 kernel/sched/core.c:730
__sched_setscheduler+0xe32/0x1fe0 kernel/sched/core.c:4336
_sched_setscheduler+0x218/0x340 kernel/sched/core.c:4373
sched_setscheduler+0xe/0x10 kernel/sched/core.c:4388
watchdog_dev_init+0x109/0x1db drivers/watchdog/watchdog_dev.c:1144
watchdog_init+0x81/0x294 drivers/watchdog/watchdog_core.c:355
do_one_initcall+0x129/0x937 init/main.c:889
do_initcall_level init/main.c:957 [inline]
do_initcalls init/main.c:965 [inline]
do_basic_setup init/main.c:983 [inline]
kernel_init_freeable+0x4db/0x5ca init/main.c:1136
kernel_init+0x12/0x1c5 init/main.c:1056
ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:352

-> #3 (&rt_b->rt_runtime_lock){-.-.}:
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2f/0x40 kernel/locking/spinlock.c:144
__enable_runtime kernel/sched/rt.c:786 [inline]
rq_online_rt+0xb4/0x390 kernel/sched/rt.c:2178
set_rq_online.part.0+0xe7/0x140 kernel/sched/core.c:5666
set_rq_online kernel/sched/core.c:5772 [inline]
sched_cpu_activate+0x29e/0x430 kernel/sched/core.c:5765
cpuhp_invoke_callback+0x2f6/0x2110 kernel/cpu.c:168
cpuhp_thread_fun+0x496/0x8a0 kernel/cpu.c:685
smpboot_thread_fn+0x6ab/0xa10 kernel/smpboot.c:164
kthread+0x357/0x430 kernel/kthread.c:246
ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:352

-> #2 (&rq->lock){-.-.}:
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2f/0x40 kernel/locking/spinlock.c:144
rq_lock kernel/sched/sched.h:1149 [inline]
task_fork_fair+0xb5/0x7a0 kernel/sched/fair.c:10058
sched_fork+0x437/0xb90 kernel/sched/core.c:2359
copy_process+0x1ff6/0x8730 kernel/fork.c:1893
_do_fork+0x1a9/0x1170 kernel/fork.c:2222
kernel_thread+0x34/0x40 kernel/fork.c:2281
rest_init+0x28/0x37b init/main.c:409
arch_call_rest_init+0xe/0x1b
start_kernel+0x882/0x8bd init/main.c:741
x86_64_start_reservations+0x29/0x2b arch/x86/kernel/head64.c:470
x86_64_start_kernel+0x77/0x7b arch/x86/kernel/head64.c:451
secondary_startup_64+0xa4/0xb0 arch/x86/kernel/head_64.S:243

-> #1 (&p->pi_lock){-.-.}:
__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0x95/0xcd kernel/locking/spinlock.c:152
try_to_wake_up+0xb9/0x1480 kernel/sched/core.c:1965
wake_up_process+0x10/0x20 kernel/sched/core.c:2129
__up.isra.0+0x1c0/0x2a0 kernel/locking/semaphore.c:262
up+0x13e/0x1c0 kernel/locking/semaphore.c:187
__up_console_sem+0xb7/0x1c0 kernel/printk/printk.c:236
console_unlock+0x778/0x11e0 kernel/printk/printk.c:2426
vprintk_emit+0x370/0x960 kernel/printk/printk.c:1931
vprintk_default+0x28/0x30 kernel/printk/printk.c:1958
vprintk_func+0x7e/0x189 kernel/printk/printk_safe.c:398
printk+0xba/0xed kernel/printk/printk.c:1991
kauditd_printk_skb kernel/audit.c:545 [inline]
kauditd_hold_skb.cold+0x3f/0x4e kernel/audit.c:578
kauditd_send_queue+0x133/0x170 kernel/audit.c:741
kauditd_thread+0x808/0xb80 kernel/audit.c:867
kthread+0x357/0x430 kernel/kthread.c:246
ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:352

-> #0 ((console_sem).lock){-.-.}:

other info that might help us debug this:

Chain exists of:
(console_sem).lock --> &rt_b->rt_runtime_lock --> hrtimer_bases.lock

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock(hrtimer_bases.lock);
lock(&rt_b->rt_runtime_lock);
lock(hrtimer_bases.lock);
lock((console_sem).lock);

*** DEADLOCK ***

4 locks held by syz-executor2/8981:
#0: 000000002aa54ad6 (rcu_read_lock){....}, at: __skb_unlink
include/linux/skbuff.h:1925 [inline]
#0: 000000002aa54ad6 (rcu_read_lock){....}, at: __skb_dequeue
include/linux/skbuff.h:1941 [inline]
#0: 000000002aa54ad6 (rcu_read_lock){....}, at:
process_backlog+0x195/0x750 net/core/dev.c:5921
#1: 000000002aa54ad6 (rcu_read_lock){....}, at: __skb_pull
include/linux/skbuff.h:2158 [inline]
#1: 000000002aa54ad6 (rcu_read_lock){....}, at:
ip_local_deliver_finish+0x13a/0x390 net/ipv4/ip_input.c:231
#2: 00000000244bdc8b (hrtimer_bases.lock){-.-.}, at:
hrtimer_interrupt+0xff/0x770 kernel/time/hrtimer.c:1490
#3: 0000000076d3ae53 (tk_core.seq){----}, at: hrtimer_update_base
kernel/time/hrtimer.c:578 [inline]
#3: 0000000076d3ae53 (tk_core.seq){----}, at:
hrtimer_interrupt+0x13d/0x770 kernel/time/hrtimer.c:1491

stack backtrace:
CPU: 1 PID: 8981 Comm: syz-executor2 Not tainted 4.20.0+ #8
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
Shutting down cpus with NMI
Kernel Offset: disabled
Rebooting in 86400 seconds..


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#bug-status-tracking for how to communicate with
syzbot.

Dmitry Vyukov

unread,
Jan 4, 2019, 6:23:50 AM1/4/19
to syzbot, Stefano Brivio, 'Dmitry Vyukov' via syzkaller-upstream-moderation
On Fri, Jan 4, 2019 at 12:17 PM syzbot
<syzbot+881ce2...@syzkaller.appspotmail.com> wrote:
>
> Hello,
>
> syzbot found the following crash on:
>
> HEAD commit: 645ff1e8e704 Merge branch 'for-linus' of git://git.kernel...
> git tree: upstream
> console output: https://syzkaller.appspot.com/x/log.txt?x=175cda9f400000
> kernel config: https://syzkaller.appspot.com/x/.config?x=7308e68273924137
> dashboard link: https://syzkaller.appspot.com/bug?extid=881ce26f0a76c7598ed4
> compiler: gcc (GCC) 9.0.0 20181231 (experimental)
> CC: [linux-...@vger.kernel.org tg...@linutronix.de]
>
> Unfortunately, I don't have any reproducer for this crash yet.
>
> IMPORTANT: if you fix the bug, please add the following tag to the commit:
> Reported-by: syzbot+881ce2...@syzkaller.appspotmail.com

Presumably a random manifestation of the stack overflow/corruption:

#syz dup: kernel panic: stack is corrupted in udp4_lib_lookup2

See https://groups.google.com/forum/#!msg/syzkaller-bugs/vr87kmG5qRI/31nOcuVsFgAJ
> --
> You received this message because you are subscribed to the Google Groups "syzkaller-upstream-moderation" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller-upstream-m...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/syzkaller-upstream-moderation/000000000000ca6680057ea00476%40google.com.
> For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages