[v6.1] possible deadlock in bpf_trace_run4

0 views
Skip to first unread message

syzbot

unread,
Feb 7, 2024, 7:12:38 PMFeb 7
to syzkaller...@googlegroups.com
Hello,

syzbot found the following issue on:

HEAD commit: f1bb70486c9c Linux 6.1.77
git tree: linux-6.1.y
console output: https://syzkaller.appspot.com/x/log.txt?x=14c30be4180000
kernel config: https://syzkaller.appspot.com/x/.config?x=39447811cb133e7e
dashboard link: https://syzkaller.appspot.com/bug?extid=26c6c4c12ed034bca593
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40

Unfortunately, I don't have any reproducer for this issue yet.

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/f93cb7e9dad2/disk-f1bb7048.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/22703d1d86ee/vmlinux-f1bb7048.xz
kernel image: https://storage.googleapis.com/syzbot-assets/4129725af309/bzImage-f1bb7048.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+26c6c4...@syzkaller.appspotmail.com

------------[ cut here ]------------
======================================================
WARNING: possible circular locking dependency detected
6.1.77-syzkaller #0 Not tainted
------------------------------------------------------
syz-executor.1/15025 is trying to acquire lock:
ffffffff8d0068d8 ((console_sem).lock){-.-.}-{2:2}, at: down_trylock+0x1c/0xa0 kernel/locking/semaphore.c:139

but task is already holding lock:
ffffffff8d188678 (trace_printk_lock){....}-{2:2}, at: ____bpf_trace_printk kernel/trace/bpf_trace.c:390 [inline]
ffffffff8d188678 (trace_printk_lock){....}-{2:2}, at: bpf_trace_printk+0x151/0x200 kernel/trace/bpf_trace.c:376

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #3 (trace_printk_lock){....}-{2:2}:
lock_acquire+0x1f8/0x5a0 kernel/locking/lockdep.c:5662
__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0xd1/0x120 kernel/locking/spinlock.c:162
____bpf_trace_printk kernel/trace/bpf_trace.c:390 [inline]
bpf_trace_printk+0x151/0x200 kernel/trace/bpf_trace.c:376
0xffffffffa0001b07
bpf_dispatcher_nop_func include/linux/bpf.h:989 [inline]
__bpf_prog_run include/linux/filter.h:600 [inline]
bpf_prog_run include/linux/filter.h:607 [inline]
__bpf_trace_run kernel/trace/bpf_trace.c:2275 [inline]
bpf_trace_run4+0x253/0x470 kernel/trace/bpf_trace.c:2316
__traceiter_sched_switch+0x91/0xc0 include/trace/events/sched.h:222
trace_sched_switch include/trace/events/sched.h:222 [inline]
__schedule+0x2116/0x4550 kernel/sched/core.c:6555
preempt_schedule_common+0x83/0xd0 kernel/sched/core.c:6727
preempt_schedule+0xd9/0xe0 kernel/sched/core.c:6751
preempt_schedule_thunk+0x16/0x18 arch/x86/entry/thunk_64.S:34
try_to_wake_up+0x87e/0x12e0 kernel/sched/core.c:4234
wake_up_process kernel/sched/core.c:4362 [inline]
wake_up_q+0x8b/0xd0 kernel/sched/core.c:1007
futex_wake+0x4ea/0x590 kernel/futex/waitwake.c:184
do_futex+0x375/0x490 kernel/futex/syscalls.c:111
__do_sys_futex kernel/futex/syscalls.c:183 [inline]
__se_sys_futex+0x3d7/0x460 kernel/futex/syscalls.c:164
do_syscall_x64 arch/x86/entry/common.c:51 [inline]
do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:81
entry_SYSCALL_64_after_hwframe+0x63/0xcd

-> #2 (&rq->__lock){-.-.}-{2:2}:
lock_acquire+0x1f8/0x5a0 kernel/locking/lockdep.c:5662
_raw_spin_lock_nested+0x2d/0x40 kernel/locking/spinlock.c:378
raw_spin_rq_lock_nested+0x26/0x140 kernel/sched/core.c:537
raw_spin_rq_lock kernel/sched/sched.h:1354 [inline]
rq_lock kernel/sched/sched.h:1644 [inline]
task_fork_fair+0x5d/0x350 kernel/sched/fair.c:11863
sched_cgroup_fork+0x374/0x400 kernel/sched/core.c:4686
copy_process+0x2442/0x4060 kernel/fork.c:2384
kernel_clone+0x222/0x920 kernel/fork.c:2682
user_mode_thread+0x12e/0x190 kernel/fork.c:2758
rest_init+0x23/0x300 init/main.c:696
start_kernel+0x0/0x53f init/main.c:891
start_kernel+0x496/0x53f init/main.c:1138
secondary_startup_64_no_verify+0xcf/0xdb

-> #1 (&p->pi_lock){-.-.}-{2:2}:
lock_acquire+0x1f8/0x5a0 kernel/locking/lockdep.c:5662
__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0xd1/0x120 kernel/locking/spinlock.c:162
try_to_wake_up+0xad/0x12e0 kernel/sched/core.c:4112
up+0x6e/0x90 kernel/locking/semaphore.c:191
__up_console_sem+0x11a/0x1e0 kernel/printk/printk.c:260
__console_unlock kernel/printk/printk.c:2662 [inline]
console_unlock+0x591/0x7c0 kernel/printk/printk.c:2873
vprintk_emit+0x523/0x740 kernel/printk/printk.c:2268
_printk+0xd1/0x111 kernel/printk/printk.c:2293
__ext4_msg+0x2d3/0x2e0 fs/ext4/super.c:938
ext4_check_feature_compatibility+0x13d/0x800 fs/ext4/super.c:4606
__ext4_fill_super fs/ext4/super.c:5123 [inline]
ext4_fill_super+0x3cf9/0x8b50 fs/ext4/super.c:5664
get_tree_bdev+0x3fe/0x620 fs/super.c:1355
vfs_get_tree+0x88/0x270 fs/super.c:1562
do_new_mount+0x2ba/0xb40 fs/namespace.c:3051
do_mount fs/namespace.c:3394 [inline]
__do_sys_mount fs/namespace.c:3602 [inline]
__se_sys_mount+0x2d5/0x3c0 fs/namespace.c:3579
do_syscall_x64 arch/x86/entry/common.c:51 [inline]
do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:81
entry_SYSCALL_64_after_hwframe+0x63/0xcd

-> #0 ((console_sem).lock){-.-.}-{2:2}:
check_prev_add kernel/locking/lockdep.c:3090 [inline]
check_prevs_add kernel/locking/lockdep.c:3209 [inline]
validate_chain+0x1661/0x5950 kernel/locking/lockdep.c:3825
__lock_acquire+0x125b/0x1f80 kernel/locking/lockdep.c:5049
lock_acquire+0x1f8/0x5a0 kernel/locking/lockdep.c:5662
__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0xd1/0x120 kernel/locking/spinlock.c:162
down_trylock+0x1c/0xa0 kernel/locking/semaphore.c:139
__down_trylock_console_sem+0x105/0x250 kernel/printk/printk.c:243
console_trylock kernel/printk/printk.c:2615 [inline]
console_trylock_spinning kernel/printk/printk.c:1867 [inline]
vprintk_emit+0x1ee/0x740 kernel/printk/printk.c:2267
_printk+0xd1/0x111 kernel/printk/printk.c:2293
__warn_printk+0x233/0x350 kernel/panic.c:715
format_decode+0x12d2/0x1f10 lib/vsprintf.c:2661
bstr_printf+0x130/0x10c0 lib/vsprintf.c:3234
____bpf_trace_printk kernel/trace/bpf_trace.c:391 [inline]
bpf_trace_printk+0x186/0x200 kernel/trace/bpf_trace.c:376
bpf_prog_12183cdb1cd51dab+0x2e/0x32
bpf_dispatcher_nop_func include/linux/bpf.h:989 [inline]
__bpf_prog_run include/linux/filter.h:600 [inline]
bpf_prog_run include/linux/filter.h:607 [inline]
bpf_test_run+0x40f/0x8b0 net/bpf/test_run.c:402
bpf_prog_test_run_skb+0xaf1/0x13a0 net/bpf/test_run.c:1180
bpf_prog_test_run+0x32f/0x3a0 kernel/bpf/syscall.c:3670
__sys_bpf+0x3eb/0x6c0 kernel/bpf/syscall.c:5023
__do_sys_bpf kernel/bpf/syscall.c:5109 [inline]
__se_sys_bpf kernel/bpf/syscall.c:5107 [inline]
__x64_sys_bpf+0x78/0x90 kernel/bpf/syscall.c:5107
do_syscall_x64 arch/x86/entry/common.c:51 [inline]
do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:81
entry_SYSCALL_64_after_hwframe+0x63/0xcd

other info that might help us debug this:

Chain exists of:
(console_sem).lock --> &rq->__lock --> trace_printk_lock

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock(trace_printk_lock);
lock(&rq->__lock);
lock(trace_printk_lock);
lock((console_sem).lock);

*** DEADLOCK ***

2 locks held by syz-executor.1/15025:
#0: ffffffff8d12a740 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:318 [inline]
#0: ffffffff8d12a740 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:759 [inline]
#0: ffffffff8d12a740 (rcu_read_lock){....}-{1:2}, at: bpf_test_timer_enter+0x19/0x140 net/bpf/test_run.c:36
#1: ffffffff8d188678 (trace_printk_lock){....}-{2:2}, at: ____bpf_trace_printk kernel/trace/bpf_trace.c:390 [inline]
#1: ffffffff8d188678 (trace_printk_lock){....}-{2:2}, at: bpf_trace_printk+0x151/0x200 kernel/trace/bpf_trace.c:376

stack backtrace:
CPU: 0 PID: 15025 Comm: syz-executor.1 Not tainted 6.1.77-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/25/2024
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x1e3/0x2cb lib/dump_stack.c:106
check_noncircular+0x2fa/0x3b0 kernel/locking/lockdep.c:2170
check_prev_add kernel/locking/lockdep.c:3090 [inline]
check_prevs_add kernel/locking/lockdep.c:3209 [inline]
validate_chain+0x1661/0x5950 kernel/locking/lockdep.c:3825
__lock_acquire+0x125b/0x1f80 kernel/locking/lockdep.c:5049
lock_acquire+0x1f8/0x5a0 kernel/locking/lockdep.c:5662
__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0xd1/0x120 kernel/locking/spinlock.c:162
down_trylock+0x1c/0xa0 kernel/locking/semaphore.c:139
__down_trylock_console_sem+0x105/0x250 kernel/printk/printk.c:243
console_trylock kernel/printk/printk.c:2615 [inline]
console_trylock_spinning kernel/printk/printk.c:1867 [inline]
vprintk_emit+0x1ee/0x740 kernel/printk/printk.c:2267
_printk+0xd1/0x111 kernel/printk/printk.c:2293
__warn_printk+0x233/0x350 kernel/panic.c:715
format_decode+0x12d2/0x1f10 lib/vsprintf.c:2661
bstr_printf+0x130/0x10c0 lib/vsprintf.c:3234
____bpf_trace_printk kernel/trace/bpf_trace.c:391 [inline]
bpf_trace_printk+0x186/0x200 kernel/trace/bpf_trace.c:376
bpf_prog_12183cdb1cd51dab+0x2e/0x32
bpf_dispatcher_nop_func include/linux/bpf.h:989 [inline]
__bpf_prog_run include/linux/filter.h:600 [inline]
bpf_prog_run include/linux/filter.h:607 [inline]
bpf_test_run+0x40f/0x8b0 net/bpf/test_run.c:402
bpf_prog_test_run_skb+0xaf1/0x13a0 net/bpf/test_run.c:1180
bpf_prog_test_run+0x32f/0x3a0 kernel/bpf/syscall.c:3670
__sys_bpf+0x3eb/0x6c0 kernel/bpf/syscall.c:5023
__do_sys_bpf kernel/bpf/syscall.c:5109 [inline]
__se_sys_bpf kernel/bpf/syscall.c:5107 [inline]
__x64_sys_bpf+0x78/0x90 kernel/bpf/syscall.c:5107
do_syscall_x64 arch/x86/entry/common.c:51 [inline]
do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:81
entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7fe26a87dda9
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 e1 20 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fe26b5ed0c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
RAX: ffffffffffffffda RBX: 00007fe26a9abf80 RCX: 00007fe26a87dda9
RDX: 0000000000000028 RSI: 0000000020000080 RDI: 000000000000000a
RBP: 00007fe26a8ca47a R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 000000000000000b R14: 00007fe26a9abf80 R15: 00007ffdc15d5288
</TASK>
Please remove unsupported % in format string
WARNING: CPU: 0 PID: 15025 at lib/vsprintf.c:2661 format_decode+0x12d2/0x1f10 lib/vsprintf.c:2661
Modules linked in:
CPU: 0 PID: 15025 Comm: syz-executor.1 Not tainted 6.1.77-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/25/2024
RIP: 0010:format_decode+0x12d2/0x1f10 lib/vsprintf.c:2661
Code: da 03 01 48 b8 00 00 00 00 00 fc ff df 41 0f b6 04 07 84 c0 0f 85 30 0c 00 00 41 0f be 36 48 c7 c7 20 7b 07 8c e8 ae b5 cd f6 <0f> 0b e9 01 fa ff ff 48 8b 4c 24 18 80 e1 07 38 c1 0f 8c e3 ed ff
RSP: 0018:ffffc9000769f6a0 EFLAGS: 00010046
RAX: 19c0aa0477ca6600 RBX: 00000000ffffffdb RCX: 0000000000040000
RDX: ffffc90004a17000 RSI: 000000000003ffff RDI: 0000000000040000
RBP: ffffc9000769f790 R08: ffffffff81527d8e R09: ffffed1017304f1c
R10: 0000000000000000 R11: dffffc0000000001 R12: ffff0a00ffffff00
R13: ffff0000ffffff00 R14: ffffc9000769f9ac R15: 1ffff92000ed3f35
FS: 00007fe26b5ed6c0(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000001b33421000 CR3: 0000000023329000 CR4: 00000000003506f0
Call Trace:
<TASK>
bstr_printf+0x130/0x10c0 lib/vsprintf.c:3234
____bpf_trace_printk kernel/trace/bpf_trace.c:391 [inline]
bpf_trace_printk+0x186/0x200 kernel/trace/bpf_trace.c:376
bpf_prog_12183cdb1cd51dab+0x2e/0x32
bpf_dispatcher_nop_func include/linux/bpf.h:989 [inline]
__bpf_prog_run include/linux/filter.h:600 [inline]
bpf_prog_run include/linux/filter.h:607 [inline]
bpf_test_run+0x40f/0x8b0 net/bpf/test_run.c:402
bpf_prog_test_run_skb+0xaf1/0x13a0 net/bpf/test_run.c:1180
bpf_prog_test_run+0x32f/0x3a0 kernel/bpf/syscall.c:3670
__sys_bpf+0x3eb/0x6c0 kernel/bpf/syscall.c:5023
__do_sys_bpf kernel/bpf/syscall.c:5109 [inline]
__se_sys_bpf kernel/bpf/syscall.c:5107 [inline]
__x64_sys_bpf+0x78/0x90 kernel/bpf/syscall.c:5107
do_syscall_x64 arch/x86/entry/common.c:51 [inline]
do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:81
entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7fe26a87dda9
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 e1 20 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fe26b5ed0c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
RAX: ffffffffffffffda RBX: 00007fe26a9abf80 RCX: 00007fe26a87dda9
RDX: 0000000000000028 RSI: 0000000020000080 RDI: 000000000000000a
RBP: 00007fe26a8ca47a R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 000000000000000b R14: 00007fe26a9abf80 R15: 00007ffdc15d5288
</TASK>


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup
Reply all
Reply to author
Forward
0 new messages