possible deadlock in perf_trace_init

6 views
Skip to first unread message

syzbot

unread,
Apr 24, 2019, 2:48:06 PM4/24/19
to syzkaller...@googlegroups.com
Hello,

syzbot found the following crash on:

HEAD commit: 68d7a45e Linux 4.14.113
git tree: linux-4.14.y
console output: https://syzkaller.appspot.com/x/log.txt?x=16872fc4a00000
kernel config: https://syzkaller.appspot.com/x/.config?x=dbf1fde4d7489e1c
dashboard link: https://syzkaller.appspot.com/bug?extid=b0ccd80ce35eb840e66d
compiler: gcc (GCC) 9.0.0 20181231 (experimental)

Unfortunately, I don't have any reproducer for this crash yet.

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+b0ccd8...@syzkaller.appspotmail.com

======================================================
WARNING: possible circular locking dependency detected
4.14.113 #3 Not tainted
------------------------------------------------------
syz-executor.3/27782 is trying to acquire lock:
(event_mutex){+.+.}, at: [<ffffffff8162bf98>] perf_trace_init+0x58/0xaa0
kernel/trace/trace_event_perf.c:216

but task is already holding lock:
(&cpuctx_mutex/1){+.+.}, at: [<ffffffff816bfd90>]
perf_event_ctx_lock_nested+0x150/0x2c0 kernel/events/core.c:1235

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #5 (&cpuctx_mutex/1){+.+.}:
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3994
__mutex_lock_common kernel/locking/mutex.c:756 [inline]
__mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
mutex_lock_double kernel/events/core.c:9904 [inline]
__perf_event_ctx_lock_double kernel/events/core.c:9963 [inline]
SYSC_perf_event_open+0x1225/0x24c0 kernel/events/core.c:10225
SyS_perf_event_open+0x34/0x40 kernel/events/core.c:9983
do_syscall_64+0x1eb/0x630 arch/x86/entry/common.c:289
entry_SYSCALL_64_after_hwframe+0x42/0xb7

-> #4 (&cpuctx_mutex){+.+.}:
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3994
__mutex_lock_common kernel/locking/mutex.c:756 [inline]
__mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
perf_event_init_cpu+0xc2/0x170 kernel/events/core.c:11223
perf_event_init+0x2d8/0x31a kernel/events/core.c:11270
start_kernel+0x3b9/0x700 init/main.c:621
x86_64_start_reservations+0x29/0x2b arch/x86/kernel/head64.c:380
x86_64_start_kernel+0x77/0x7b arch/x86/kernel/head64.c:361
secondary_startup_64+0xa5/0xb0 arch/x86/kernel/head_64.S:240

-> #3 (pmus_lock){+.+.}:
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3994
__mutex_lock_common kernel/locking/mutex.c:756 [inline]
__mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
perf_event_init_cpu+0x2f/0x170 kernel/events/core.c:11217
cpuhp_invoke_callback+0x1ec/0x1ac0 kernel/cpu.c:184
cpuhp_up_callbacks kernel/cpu.c:572 [inline]
_cpu_up+0x228/0x530 kernel/cpu.c:1134
do_cpu_up+0x121/0x150 kernel/cpu.c:1169
cpu_up+0x1b/0x20 kernel/cpu.c:1177
smp_init+0x157/0x173 kernel/smp.c:578
kernel_init_freeable+0x30e/0x538 init/main.c:1067
kernel_init+0x12/0x162 init/main.c:999
ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:402

-> #2 (cpu_hotplug_lock.rw_sem){++++}:
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3994
percpu_down_read_preempt_disable include/linux/percpu-rwsem.h:36
[inline]
percpu_down_read include/linux/percpu-rwsem.h:59 [inline]
cpus_read_lock+0x3d/0xc0 kernel/cpu.c:295
static_key_slow_inc+0x13/0x30 kernel/jump_label.c:123
tracepoint_add_func kernel/tracepoint.c:223 [inline]
tracepoint_probe_register_prio+0x4dc/0x6d0 kernel/tracepoint.c:283
tracepoint_probe_register+0x2b/0x40 kernel/tracepoint.c:304
trace_event_reg+0x277/0x330 kernel/trace/trace_events.c:305
perf_trace_event_reg kernel/trace/trace_event_perf.c:122 [inline]
perf_trace_event_init kernel/trace/trace_event_perf.c:197 [inline]
perf_trace_init+0x450/0xaa0 kernel/trace/trace_event_perf.c:221
perf_tp_event_init+0x7d/0xf0 kernel/events/core.c:8106
perf_try_init_event+0x170/0x210 kernel/events/core.c:9342
perf_init_event kernel/events/core.c:9380 [inline]
perf_event_alloc.part.0+0xd48/0x2540 kernel/events/core.c:9640
perf_event_alloc kernel/events/core.c:9993 [inline]
SYSC_perf_event_open+0xa2d/0x24c0 kernel/events/core.c:10097
SyS_perf_event_open+0x34/0x40 kernel/events/core.c:9983
do_syscall_64+0x1eb/0x630 arch/x86/entry/common.c:289
entry_SYSCALL_64_after_hwframe+0x42/0xb7

-> #1 (tracepoints_mutex){+.+.}:
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3994
__mutex_lock_common kernel/locking/mutex.c:756 [inline]
__mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
tracepoint_probe_register_prio+0x36/0x6d0 kernel/tracepoint.c:279
tracepoint_probe_register+0x2b/0x40 kernel/tracepoint.c:304
trace_event_reg+0x277/0x330 kernel/trace/trace_events.c:305
perf_trace_event_reg kernel/trace/trace_event_perf.c:122 [inline]
perf_trace_event_init kernel/trace/trace_event_perf.c:197 [inline]
perf_trace_init+0x450/0xaa0 kernel/trace/trace_event_perf.c:221
perf_tp_event_init+0x7d/0xf0 kernel/events/core.c:8106
perf_try_init_event+0x170/0x210 kernel/events/core.c:9342
perf_init_event kernel/events/core.c:9380 [inline]
perf_event_alloc.part.0+0xd48/0x2540 kernel/events/core.c:9640
perf_event_alloc kernel/events/core.c:9993 [inline]
SYSC_perf_event_open+0xa2d/0x24c0 kernel/events/core.c:10097
SyS_perf_event_open+0x34/0x40 kernel/events/core.c:9983
do_syscall_64+0x1eb/0x630 arch/x86/entry/common.c:289
entry_SYSCALL_64_after_hwframe+0x42/0xb7

-> #0 (event_mutex){+.+.}:
check_prev_add kernel/locking/lockdep.c:1901 [inline]
check_prevs_add kernel/locking/lockdep.c:2018 [inline]
validate_chain kernel/locking/lockdep.c:2460 [inline]
__lock_acquire+0x2c89/0x45e0 kernel/locking/lockdep.c:3487
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3994
__mutex_lock_common kernel/locking/mutex.c:756 [inline]
__mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
perf_trace_init+0x58/0xaa0 kernel/trace/trace_event_perf.c:216
perf_tp_event_init+0x7d/0xf0 kernel/events/core.c:8106
perf_try_init_event+0xec/0x210 kernel/events/core.c:9342
perf_init_event kernel/events/core.c:9380 [inline]
perf_event_alloc.part.0+0xd48/0x2540 kernel/events/core.c:9640
perf_event_alloc kernel/events/core.c:9993 [inline]
SYSC_perf_event_open+0xa2d/0x24c0 kernel/events/core.c:10097
SyS_perf_event_open+0x34/0x40 kernel/events/core.c:9983
do_syscall_64+0x1eb/0x630 arch/x86/entry/common.c:289
entry_SYSCALL_64_after_hwframe+0x42/0xb7

other info that might help us debug this:

Chain exists of:
event_mutex --> &cpuctx_mutex --> &cpuctx_mutex/1

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock(&cpuctx_mutex/1);
lock(&cpuctx_mutex);
lock(&cpuctx_mutex/1);
lock(event_mutex);

*** DEADLOCK ***

2 locks held by syz-executor.3/27782:
#0: (&pmus_srcu){....}, at: [<ffffffff816c4b98>]
perf_event_alloc.part.0+0xba8/0x2540 kernel/events/core.c:9636
#1: (&cpuctx_mutex/1){+.+.}, at: [<ffffffff816bfd90>]
perf_event_ctx_lock_nested+0x150/0x2c0 kernel/events/core.c:1235

stack backtrace:
CPU: 0 PID: 27782 Comm: syz-executor.3 Not tainted 4.14.113 #3
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:17 [inline]
dump_stack+0x138/0x19c lib/dump_stack.c:53
print_circular_bug.isra.0.cold+0x1cc/0x28f kernel/locking/lockdep.c:1258
check_prev_add kernel/locking/lockdep.c:1901 [inline]
check_prevs_add kernel/locking/lockdep.c:2018 [inline]
validate_chain kernel/locking/lockdep.c:2460 [inline]
__lock_acquire+0x2c89/0x45e0 kernel/locking/lockdep.c:3487
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3994
__mutex_lock_common kernel/locking/mutex.c:756 [inline]
__mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
perf_trace_init+0x58/0xaa0 kernel/trace/trace_event_perf.c:216
perf_tp_event_init+0x7d/0xf0 kernel/events/core.c:8106
perf_try_init_event+0xec/0x210 kernel/events/core.c:9342
perf_init_event kernel/events/core.c:9380 [inline]
perf_event_alloc.part.0+0xd48/0x2540 kernel/events/core.c:9640
perf_event_alloc kernel/events/core.c:9993 [inline]
SYSC_perf_event_open+0xa2d/0x24c0 kernel/events/core.c:10097
SyS_perf_event_open+0x34/0x40 kernel/events/core.c:9983
do_syscall_64+0x1eb/0x630 arch/x86/entry/common.c:289
entry_SYSCALL_64_after_hwframe+0x42/0xb7
RIP: 0033:0x458d99
RSP: 002b:00007f332228ec78 EFLAGS: 00000246 ORIG_RAX: 000000000000012a
RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 0000000000458d99
RDX: 0000000000000000 RSI: ffffffffffffffff RDI: 0000000020000200
RBP: 000000000073bf00 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000003 R11: 0000000000000246 R12: 00007f332228f6d4
R13: 00000000004c5628 R14: 00000000004d97f8 R15: 00000000ffffffff
caif:caif_disconnect_client(): nothing to disconnect
caif:caif_disconnect_client(): nothing to disconnect
sctp_setsockopt_delayed_ack: 34 callbacks suppressed
sctp: [Deprecated]: syz-executor.2 (pid 28010) Use of struct
sctp_assoc_value in delayed_ack socket option.
Use struct sctp_sack_info instead
sctp: [Deprecated]: syz-executor.0 (pid 28002) Use of struct
sctp_assoc_value in delayed_ack socket option.
Use struct sctp_sack_info instead
caif:caif_disconnect_client(): nothing to disconnect
caif:caif_disconnect_client(): nothing to disconnect
sctp: [Deprecated]: syz-executor.0 (pid 28044) Use of struct
sctp_assoc_value in delayed_ack socket option.
Use struct sctp_sack_info instead
sctp: [Deprecated]: syz-executor.2 (pid 28049) Use of struct
sctp_assoc_value in delayed_ack socket option.
Use struct sctp_sack_info instead
sctp: [Deprecated]: syz-executor.1 (pid 28089) Use of struct
sctp_assoc_value in delayed_ack socket option.
Use struct sctp_sack_info instead
sctp: [Deprecated]: syz-executor.1 (pid 28103) Use of struct
sctp_assoc_value in delayed_ack socket option.
Use struct sctp_sack_info instead
sctp: [Deprecated]: syz-executor.1 (pid 28103) Use of struct
sctp_assoc_value in delayed_ack socket option.
Use struct sctp_sack_info instead
sctp: [Deprecated]: syz-executor.2 (pid 28090) Use of struct
sctp_assoc_value in delayed_ack socket option.
Use struct sctp_sack_info instead
sctp: [Deprecated]: syz-executor.0 (pid 28095) Use of struct
sctp_assoc_value in delayed_ack socket option.
Use struct sctp_sack_info instead
sctp: [Deprecated]: syz-executor.1 (pid 28134) Use of struct
sctp_assoc_value in delayed_ack socket option.
Use struct sctp_sack_info instead
could not allocate digest TFM handle sha256_mb
could not allocate digest TFM handle sha256_mb
sctp_setsockopt_delayed_ack: 46 callbacks suppressed
sctp: [Deprecated]: syz-executor.0 (pid 28474) Use of struct
sctp_assoc_value in delayed_ack socket option.
Use struct sctp_sack_info instead
sctp: [Deprecated]: syz-executor.2 (pid 28481) Use of struct
sctp_assoc_value in delayed_ack socket option.
Use struct sctp_sack_info instead
sctp: [Deprecated]: syz-executor.1 (pid 28513) Use of struct
sctp_assoc_value in delayed_ack socket option.
Use struct sctp_sack_info instead
sctp: [Deprecated]: syz-executor.1 (pid 28535) Use of struct
sctp_assoc_value in delayed_ack socket option.
Use struct sctp_sack_info instead
sctp: [Deprecated]: syz-executor.0 (pid 28506) Use of struct
sctp_assoc_value in delayed_ack socket option.
Use struct sctp_sack_info instead


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

syzbot

unread,
Aug 20, 2019, 4:20:07 PM8/20/19
to syzkaller...@googlegroups.com
syzbot has found a reproducer for the following crash on:

HEAD commit: 45f092f9 Linux 4.14.139
git tree: linux-4.14.y
console output: https://syzkaller.appspot.com/x/log.txt?x=16a05f4a600000
kernel config: https://syzkaller.appspot.com/x/.config?x=56ab4cf14cc8892d
dashboard link: https://syzkaller.appspot.com/bug?extid=b0ccd80ce35eb840e66d
compiler: gcc (GCC) 9.0.0 20181231 (experimental)
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1385d2f2600000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+b0ccd8...@syzkaller.appspotmail.com

IPv6: ADDRCONF(NETDEV_CHANGE): hsr0: link becomes ready
IPv6: ADDRCONF(NETDEV_UP): vxcan1: link is not ready
8021q: adding VLAN 0 to HW filter on device batadv0
IPv6: ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready
======================================================
WARNING: possible circular locking dependency detected
4.14.139 #35 Not tainted
------------------------------------------------------
syz-executor.0/6539 is trying to acquire lock:
(event_mutex){+.+.}, at: [<ffffffff8162b4c8>] perf_trace_init+0x58/0xaa0
kernel/trace/trace_event_perf.c:216

but task is already holding lock:
(&cpuctx_mutex/1){+.+.}, at: [<ffffffff816bb420>]
perf_event_ctx_lock_nested+0x150/0x2c0 kernel/events/core.c:1235

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #5 (&cpuctx_mutex/1){+.+.}:
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3991
__mutex_lock_common kernel/locking/mutex.c:756 [inline]
__mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
mutex_lock_double kernel/events/core.c:9904 [inline]
__perf_event_ctx_lock_double kernel/events/core.c:9963 [inline]
SYSC_perf_event_open+0x134c/0x2610 kernel/events/core.c:10225
SyS_perf_event_open+0x34/0x40 kernel/events/core.c:9983
do_syscall_64+0x1e8/0x640 arch/x86/entry/common.c:292
entry_SYSCALL_64_after_hwframe+0x42/0xb7

-> #4 (&cpuctx_mutex){+.+.}:
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3991
__mutex_lock_common kernel/locking/mutex.c:756 [inline]
__mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
perf_event_init_cpu+0xc2/0x170 kernel/events/core.c:11223
perf_event_init+0x2d8/0x31a kernel/events/core.c:11270
start_kernel+0x3b6/0x6fd init/main.c:621
x86_64_start_reservations+0x29/0x2b arch/x86/kernel/head64.c:381
x86_64_start_kernel+0x77/0x7b arch/x86/kernel/head64.c:362
secondary_startup_64+0xa5/0xb0 arch/x86/kernel/head_64.S:240

-> #3 (pmus_lock){+.+.}:
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3991
__mutex_lock_common kernel/locking/mutex.c:756 [inline]
__mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
perf_event_init_cpu+0x2f/0x170 kernel/events/core.c:11217
cpuhp_invoke_callback+0x1ea/0x1ab0 kernel/cpu.c:184
cpuhp_up_callbacks kernel/cpu.c:572 [inline]
_cpu_up+0x228/0x530 kernel/cpu.c:1134
do_cpu_up+0x121/0x150 kernel/cpu.c:1169
cpu_up+0x1b/0x20 kernel/cpu.c:1177
smp_init+0x157/0x170 kernel/smp.c:578
kernel_init_freeable+0x30b/0x532 init/main.c:1067
kernel_init+0x12/0x162 init/main.c:999
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:404

-> #2 (cpu_hotplug_lock.rw_sem){++++}:
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3991
percpu_down_read_preempt_disable include/linux/percpu-rwsem.h:36
[inline]
percpu_down_read include/linux/percpu-rwsem.h:59 [inline]
cpus_read_lock+0x3d/0xc0 kernel/cpu.c:295
static_key_slow_inc+0x13/0x30 kernel/jump_label.c:123
tracepoint_add_func kernel/tracepoint.c:223 [inline]
tracepoint_probe_register_prio+0x4d6/0x6d0 kernel/tracepoint.c:283
tracepoint_probe_register+0x2b/0x40 kernel/tracepoint.c:304
trace_event_reg+0x277/0x330 kernel/trace/trace_events.c:305
perf_trace_event_reg kernel/trace/trace_event_perf.c:122 [inline]
perf_trace_event_init kernel/trace/trace_event_perf.c:197 [inline]
perf_trace_init+0x449/0xaa0 kernel/trace/trace_event_perf.c:221
perf_tp_event_init+0x7d/0xf0 kernel/events/core.c:8106
perf_try_init_event+0x164/0x200 kernel/events/core.c:9342
perf_init_event kernel/events/core.c:9380 [inline]
perf_event_alloc.part.0+0xd90/0x25b0 kernel/events/core.c:9640
perf_event_alloc kernel/events/core.c:9993 [inline]
SYSC_perf_event_open+0xad1/0x2610 kernel/events/core.c:10097
SyS_perf_event_open+0x34/0x40 kernel/events/core.c:9983
do_syscall_64+0x1e8/0x640 arch/x86/entry/common.c:292
entry_SYSCALL_64_after_hwframe+0x42/0xb7

-> #1 (tracepoints_mutex){+.+.}:
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3991
__mutex_lock_common kernel/locking/mutex.c:756 [inline]
__mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
tracepoint_probe_register_prio+0x36/0x6d0 kernel/tracepoint.c:279
tracepoint_probe_register+0x2b/0x40 kernel/tracepoint.c:304
trace_event_reg+0x277/0x330 kernel/trace/trace_events.c:305
perf_trace_event_reg kernel/trace/trace_event_perf.c:122 [inline]
perf_trace_event_init kernel/trace/trace_event_perf.c:197 [inline]
perf_trace_init+0x449/0xaa0 kernel/trace/trace_event_perf.c:221
perf_tp_event_init+0x7d/0xf0 kernel/events/core.c:8106
perf_try_init_event+0x164/0x200 kernel/events/core.c:9342
perf_init_event kernel/events/core.c:9380 [inline]
perf_event_alloc.part.0+0xd90/0x25b0 kernel/events/core.c:9640
perf_event_alloc kernel/events/core.c:9993 [inline]
SYSC_perf_event_open+0xad1/0x2610 kernel/events/core.c:10097
SyS_perf_event_open+0x34/0x40 kernel/events/core.c:9983
do_syscall_64+0x1e8/0x640 arch/x86/entry/common.c:292
entry_SYSCALL_64_after_hwframe+0x42/0xb7

-> #0 (event_mutex){+.+.}:
check_prev_add kernel/locking/lockdep.c:1901 [inline]
check_prevs_add kernel/locking/lockdep.c:2018 [inline]
validate_chain kernel/locking/lockdep.c:2460 [inline]
__lock_acquire+0x2cb3/0x4620 kernel/locking/lockdep.c:3487
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3991
__mutex_lock_common kernel/locking/mutex.c:756 [inline]
__mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
perf_trace_init+0x58/0xaa0 kernel/trace/trace_event_perf.c:216
perf_tp_event_init+0x7d/0xf0 kernel/events/core.c:8106
perf_try_init_event+0xe6/0x200 kernel/events/core.c:9342
perf_init_event kernel/events/core.c:9380 [inline]
perf_event_alloc.part.0+0xd90/0x25b0 kernel/events/core.c:9640
perf_event_alloc kernel/events/core.c:9993 [inline]
SYSC_perf_event_open+0xad1/0x2610 kernel/events/core.c:10097
SyS_perf_event_open+0x34/0x40 kernel/events/core.c:9983
do_syscall_64+0x1e8/0x640 arch/x86/entry/common.c:292
entry_SYSCALL_64_after_hwframe+0x42/0xb7

other info that might help us debug this:

Chain exists of:
event_mutex --> &cpuctx_mutex --> &cpuctx_mutex/1

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock(&cpuctx_mutex/1);
lock(&cpuctx_mutex);
lock(&cpuctx_mutex/1);
lock(event_mutex);

*** DEADLOCK ***

2 locks held by syz-executor.0/6539:
#0: (&pmus_srcu){....}, at: [<ffffffff816bf9ea>]
perf_event_alloc.part.0+0xbaa/0x25b0 kernel/events/core.c:9636
#1: (&cpuctx_mutex/1){+.+.}, at: [<ffffffff816bb420>]
perf_event_ctx_lock_nested+0x150/0x2c0 kernel/events/core.c:1235

stack backtrace:
CPU: 1 PID: 6539 Comm: syz-executor.0 Not tainted 4.14.139 #35
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:17 [inline]
dump_stack+0x138/0x19c lib/dump_stack.c:53
print_circular_bug.isra.0.cold+0x1cc/0x28f kernel/locking/lockdep.c:1258
check_prev_add kernel/locking/lockdep.c:1901 [inline]
check_prevs_add kernel/locking/lockdep.c:2018 [inline]
validate_chain kernel/locking/lockdep.c:2460 [inline]
__lock_acquire+0x2cb3/0x4620 kernel/locking/lockdep.c:3487
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3991
__mutex_lock_common kernel/locking/mutex.c:756 [inline]
__mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
perf_trace_init+0x58/0xaa0 kernel/trace/trace_event_perf.c:216
perf_tp_event_init+0x7d/0xf0 kernel/events/core.c:8106
perf_try_init_event+0xe6/0x200 kernel/events/core.c:9342
perf_init_event kernel/events/core.c:9380 [inline]
perf_event_alloc.part.0+0xd90/0x25b0 kernel/events/core.c:9640
perf_event_alloc kernel/events/core.c:9993 [inline]
SYSC_perf_event_open+0xad1/0x2610 kernel/events/core.c:10097
SyS_perf_event_open+0x34/0x40 kernel/events/core.c:9983
do_syscall_64+0x1e8/0x640 arch/x86/entry/common.c:292
entry_SYSCALL_64_after_hwframe+0x42/0xb7
RIP: 0033:0x459829
RSP: 002b:00007f2c7d17ec78 EFLAGS: 00000246 ORIG_RAX: 000000000000012a
RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 0000000000459829
RDX: 0000000000000000 RSI: ffffffffffffffff RDI: 0000000020000200
RBP: 000000000075bf20 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000003 R11: 0000000000000246 R12: 00007f2c7d17f6d4
R13: 00000000004c6684 R14: 00000000004db6b8 R15: 00000000ffffffff

syzbot

unread,
Aug 25, 2019, 7:17:08 PM8/25/19
to syzkaller...@googlegroups.com
syzbot has found a reproducer for the following crash on:

HEAD commit: b5260801 Linux 4.14.140
git tree: linux-4.14.y
console output: https://syzkaller.appspot.com/x/log.txt?x=139f439c600000
kernel config: https://syzkaller.appspot.com/x/.config?x=a1845e2543d718e1
dashboard link: https://syzkaller.appspot.com/bug?extid=b0ccd80ce35eb840e66d
compiler: gcc (GCC) 9.0.0 20181231 (experimental)
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=132754a2600000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=104d57a6600000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+b0ccd8...@syzkaller.appspotmail.com

audit: type=1400 audit(1566774842.651:36): avc: denied { map } for
pid=6820 comm="syz-executor004" path="/root/syz-executor004254867"
dev="sda1" ino=16482 scontext=unconfined_u:system_r:insmod_t:s0-s0:c0.c1023
tcontext=unconfined_u:object_r:user_home_t:s0 tclass=file permissive=1
======================================================
WARNING: possible circular locking dependency detected
4.14.140 #36 Not tainted
------------------------------------------------------
syz-executor004/6820 is trying to acquire lock:
(event_mutex){+.+.}, at: [<ffffffff8162b8d8>] perf_trace_init+0x58/0xaa0
kernel/trace/trace_event_perf.c:216

but task is already holding lock:
(&cpuctx_mutex/1){+.+.}, at: [<ffffffff816bbe80>]
perf_event_ctx_lock_nested+0x150/0x2c0 kernel/events/core.c:1235

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #5 (&cpuctx_mutex/1){+.+.}:
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3991
__mutex_lock_common kernel/locking/mutex.c:756 [inline]
__mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
mutex_lock_double kernel/events/core.c:9904 [inline]
__perf_event_ctx_lock_double kernel/events/core.c:9963 [inline]
SYSC_perf_event_open+0x134c/0x2690 kernel/events/core.c:10225
SyS_perf_event_open+0x34/0x40 kernel/events/core.c:9983
do_syscall_64+0x1e8/0x640 arch/x86/entry/common.c:292
entry_SYSCALL_64_after_hwframe+0x42/0xb7

-> #4 (&cpuctx_mutex){+.+.}:
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3991
__mutex_lock_common kernel/locking/mutex.c:756 [inline]
__mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
perf_event_init_cpu+0xc2/0x170 kernel/events/core.c:11223
perf_event_init+0x2d8/0x31a kernel/events/core.c:11270
start_kernel+0x3b6/0x6fd init/main.c:621
x86_64_start_reservations+0x29/0x2b arch/x86/kernel/head64.c:381
x86_64_start_kernel+0x77/0x7b arch/x86/kernel/head64.c:362
secondary_startup_64+0xa5/0xb0 arch/x86/kernel/head_64.S:240

-> #3 (pmus_lock){+.+.}:
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3991
__mutex_lock_common kernel/locking/mutex.c:756 [inline]
__mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
perf_event_init_cpu+0x2f/0x170 kernel/events/core.c:11217
cpuhp_invoke_callback+0x1ea/0x1ab0 kernel/cpu.c:184
cpuhp_up_callbacks kernel/cpu.c:572 [inline]
_cpu_up+0x228/0x530 kernel/cpu.c:1134
do_cpu_up+0x121/0x150 kernel/cpu.c:1169
cpu_up+0x1b/0x20 kernel/cpu.c:1177
smp_init+0x157/0x170 kernel/smp.c:578
kernel_init_freeable+0x30b/0x532 init/main.c:1067
kernel_init+0x12/0x162 init/main.c:999
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:404

-> #2 (cpu_hotplug_lock.rw_sem){++++}:
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3991
percpu_down_read_preempt_disable include/linux/percpu-rwsem.h:36
[inline]
percpu_down_read include/linux/percpu-rwsem.h:59 [inline]
cpus_read_lock+0x3d/0xc0 kernel/cpu.c:295
static_key_slow_inc+0x13/0x30 kernel/jump_label.c:123
tracepoint_add_func kernel/tracepoint.c:223 [inline]
tracepoint_probe_register_prio+0x4d6/0x6d0 kernel/tracepoint.c:283
tracepoint_probe_register+0x2b/0x40 kernel/tracepoint.c:304
trace_event_reg+0x277/0x330 kernel/trace/trace_events.c:305
perf_trace_event_reg kernel/trace/trace_event_perf.c:122 [inline]
perf_trace_event_init kernel/trace/trace_event_perf.c:197 [inline]
perf_trace_init+0x449/0xaa0 kernel/trace/trace_event_perf.c:221
perf_tp_event_init+0x7d/0xf0 kernel/events/core.c:8106
perf_try_init_event+0x164/0x200 kernel/events/core.c:9342
perf_init_event kernel/events/core.c:9380 [inline]
perf_event_alloc.part.0+0xd90/0x25b0 kernel/events/core.c:9640
perf_event_alloc kernel/events/core.c:9993 [inline]
SYSC_perf_event_open+0xad1/0x2690 kernel/events/core.c:10097
SyS_perf_event_open+0x34/0x40 kernel/events/core.c:9983
do_syscall_64+0x1e8/0x640 arch/x86/entry/common.c:292
entry_SYSCALL_64_after_hwframe+0x42/0xb7

-> #1 (tracepoints_mutex){+.+.}:
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3991
__mutex_lock_common kernel/locking/mutex.c:756 [inline]
__mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
tracepoint_probe_register_prio+0x36/0x6d0 kernel/tracepoint.c:279
tracepoint_probe_register+0x2b/0x40 kernel/tracepoint.c:304
trace_event_reg+0x277/0x330 kernel/trace/trace_events.c:305
perf_trace_event_reg kernel/trace/trace_event_perf.c:122 [inline]
perf_trace_event_init kernel/trace/trace_event_perf.c:197 [inline]
perf_trace_init+0x449/0xaa0 kernel/trace/trace_event_perf.c:221
perf_tp_event_init+0x7d/0xf0 kernel/events/core.c:8106
perf_try_init_event+0x164/0x200 kernel/events/core.c:9342
perf_init_event kernel/events/core.c:9380 [inline]
perf_event_alloc.part.0+0xd90/0x25b0 kernel/events/core.c:9640
perf_event_alloc kernel/events/core.c:9993 [inline]
SYSC_perf_event_open+0xad1/0x2690 kernel/events/core.c:10097
SyS_perf_event_open+0x34/0x40 kernel/events/core.c:9983
do_syscall_64+0x1e8/0x640 arch/x86/entry/common.c:292
entry_SYSCALL_64_after_hwframe+0x42/0xb7

-> #0 (event_mutex){+.+.}:
check_prev_add kernel/locking/lockdep.c:1901 [inline]
check_prevs_add kernel/locking/lockdep.c:2018 [inline]
validate_chain kernel/locking/lockdep.c:2460 [inline]
__lock_acquire+0x2cb3/0x4620 kernel/locking/lockdep.c:3487
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3991
__mutex_lock_common kernel/locking/mutex.c:756 [inline]
__mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
perf_trace_init+0x58/0xaa0 kernel/trace/trace_event_perf.c:216
perf_tp_event_init+0x7d/0xf0 kernel/events/core.c:8106
perf_try_init_event+0xe6/0x200 kernel/events/core.c:9342
perf_init_event kernel/events/core.c:9380 [inline]
perf_event_alloc.part.0+0xd90/0x25b0 kernel/events/core.c:9640
perf_event_alloc kernel/events/core.c:9993 [inline]
SYSC_perf_event_open+0xad1/0x2690 kernel/events/core.c:10097
SyS_perf_event_open+0x34/0x40 kernel/events/core.c:9983
do_syscall_64+0x1e8/0x640 arch/x86/entry/common.c:292
entry_SYSCALL_64_after_hwframe+0x42/0xb7

other info that might help us debug this:

Chain exists of:
event_mutex --> &cpuctx_mutex --> &cpuctx_mutex/1

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock(&cpuctx_mutex/1);
lock(&cpuctx_mutex);
lock(&cpuctx_mutex/1);
lock(event_mutex);

*** DEADLOCK ***

2 locks held by syz-executor004/6820:
#0: (&pmus_srcu){....}, at: [<ffffffff816bf89a>]
perf_event_alloc.part.0+0xbaa/0x25b0 kernel/events/core.c:9636
#1: (&cpuctx_mutex/1){+.+.}, at: [<ffffffff816bbe80>]
perf_event_ctx_lock_nested+0x150/0x2c0 kernel/events/core.c:1235

stack backtrace:
CPU: 0 PID: 6820 Comm: syz-executor004 Not tainted 4.14.140 #36
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:17 [inline]
dump_stack+0x138/0x197 lib/dump_stack.c:53
print_circular_bug.isra.0.cold+0x1cc/0x28f kernel/locking/lockdep.c:1258
check_prev_add kernel/locking/lockdep.c:1901 [inline]
check_prevs_add kernel/locking/lockdep.c:2018 [inline]
validate_chain kernel/locking/lockdep.c:2460 [inline]
__lock_acquire+0x2cb3/0x4620 kernel/locking/lockdep.c:3487
lock_acquire+0x16f/0x430 kernel/locking/lockdep.c:3991
__mutex_lock_common kernel/locking/mutex.c:756 [inline]
__mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
perf_trace_init+0x58/0xaa0 kernel/trace/trace_event_perf.c:216
perf_tp_event_init+0x7d/0xf0 kernel/events/core.c:8106
perf_try_init_event+0xe6/0x200 kernel/events/core.c:9342
perf_init_event kernel/events/core.c:9380 [inline]
perf_event_alloc.part.0+0xd90/0x25b0 kernel/events/core.c:9640
perf_event_alloc kernel/events/core.c:9993 [inline]
SYSC_perf_event_open+0xad1/0x2690 kernel/events/core.c:10097
SyS_perf_event_open+0x34/0x40 kernel/events/core.c:9983
do_syscall_64+0x1e8/0x640 arch/x86/entry/common.c:292
entry_SYSCALL_64_after_hwframe+0x42/0xb7
RIP: 0033:0x440459
RSP: 002b:00007ffc0f5e1f08 EFLAGS: 00000246 ORIG_RAX: 000000000000012a

Reply all
Reply to author
Forward
0 new messages