possible deadlock in lock_timer_base

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: d7a03a44 Add linux-next specific files for 20201223
git tree: linux-next
console output: https://syzkaller.appspot.com/x/log.txt?x=155fb7b7500000
kernel config: https://syzkaller.appspot.com/x/.config?x=ea251547cbb170c5
dashboard link: https://syzkaller.appspot.com/bug?extid=8983d6d4f7df556be565
compiler: gcc (GCC) 10.1.0-syz 20200507

Unfortunately, I don't have any reproducer for this issue yet.

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+8983d6...@syzkaller.appspotmail.com

======================================================
WARNING: possible circular locking dependency detected
5.10.0-next-20201223-syzkaller #0 Not tainted
------------------------------------------------------
syz-executor.0/9817 is trying to acquire lock:
ffffffff8b48b998 (allocation_wait.lock){..-.}-{2:2}, at: __wake_up_common_lock+0xb4/0x130 kernel/sched/wait.c:137

but task is already holding lock:
ffff8880b9e24898 (&base->lock){-.-.}-{2:2}, at: lock_timer_base+0x5a/0x1f0 kernel/time/timer.c:944

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #3 (&base->lock){-.-.}-{2:2}:
__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0x39/0x50 kernel/locking/spinlock.c:159
lock_timer_base+0x5a/0x1f0 kernel/time/timer.c:944
__mod_timer+0x398/0xe30 kernel/time/timer.c:1017
__queue_delayed_work+0x1a7/0x270 kernel/workqueue.c:1655
queue_delayed_work_on+0xd1/0xe0 kernel/workqueue.c:1680
psi_task_change+0x142/0x220 kernel/sched/psi.c:807
psi_enqueue kernel/sched/stats.h:82 [inline]
enqueue_task kernel/sched/core.c:1592 [inline]
activate_task kernel/sched/core.c:1615 [inline]
wake_up_new_task+0x9f2/0xdd0 kernel/sched/core.c:3834
kernel_clone+0x1f0/0xab0 kernel/fork.c:2489
kernel_thread+0xb5/0xf0 kernel/fork.c:2517
rest_init+0x23/0x388 init/main.c:687
start_kernel+0x470/0x491 init/main.c:1064
secondary_startup_64_no_verify+0xb0/0xbb

-> #2 (&rq->lock){-.-.}-{2:2}:
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:151
rq_lock kernel/sched/sched.h:1318 [inline]
task_fork_fair+0x74/0x4d0 kernel/sched/fair.c:10767
sched_fork+0x3fc/0xbd0 kernel/sched/core.c:3764
copy_process+0x1d61/0x6fc0 kernel/fork.c:2070
kernel_clone+0xe7/0xab0 kernel/fork.c:2465
kernel_thread+0xb5/0xf0 kernel/fork.c:2517
rest_init+0x23/0x388 init/main.c:687
start_kernel+0x470/0x491 init/main.c:1064
secondary_startup_64_no_verify+0xb0/0xbb

-> #1 (&p->pi_lock){-.-.}-{2:2}:
__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0x39/0x50 kernel/locking/spinlock.c:159
try_to_wake_up+0x98/0x13b0 kernel/sched/core.c:3335
autoremove_wake_function+0x12/0x140 kernel/sched/wait.c:404
__wake_up_common+0x147/0x650 kernel/sched/wait.c:108
__wake_up_common_lock+0xd0/0x130 kernel/sched/wait.c:138
__kfence_alloc+0xa0/0xbc0 mm/kfence/core.c:710
kfence_alloc include/linux/kfence.h:108 [inline]
slab_alloc_node mm/slub.c:2826 [inline]
kmem_cache_alloc_node_trace+0x28d/0x3d0 mm/slub.c:2953
kmalloc_node include/linux/slab.h:570 [inline]
kzalloc_node include/linux/slab.h:693 [inline]
alloc_worker kernel/workqueue.c:1826 [inline]
create_worker+0xfe/0x6d0 kernel/workqueue.c:1923
maybe_create_worker kernel/workqueue.c:2094 [inline]
manage_workers kernel/workqueue.c:2146 [inline]
worker_thread+0xaef/0x1120 kernel/workqueue.c:2393
kthread+0x3b1/0x4a0 kernel/kthread.c:292
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:296

-> #0 (allocation_wait.lock){..-.}-{2:2}:
check_prev_add kernel/locking/lockdep.c:2868 [inline]
check_prevs_add kernel/locking/lockdep.c:2993 [inline]
validate_chain kernel/locking/lockdep.c:3608 [inline]
__lock_acquire+0x2ade/0x5500 kernel/locking/lockdep.c:4832
lock_acquire kernel/locking/lockdep.c:5437 [inline]
lock_acquire+0x29d/0x740 kernel/locking/lockdep.c:5402
__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0x39/0x50 kernel/locking/spinlock.c:159
__wake_up_common_lock+0xb4/0x130 kernel/sched/wait.c:137
__kfence_alloc+0xa0/0xbc0 mm/kfence/core.c:710
kfence_alloc include/linux/kfence.h:108 [inline]
slab_alloc_node mm/slub.c:2826 [inline]
slab_alloc mm/slub.c:2911 [inline]
kmem_cache_alloc+0x259/0x470 mm/slub.c:2916
kmem_cache_zalloc include/linux/slab.h:672 [inline]
fill_pool+0x264/0x5c0 lib/debugobjects.c:171
__debug_object_init+0x7a/0xd10 lib/debugobjects.c:560
debug_object_init lib/debugobjects.c:615 [inline]
debug_object_activate+0x32c/0x3e0 lib/debugobjects.c:701
debug_timer_activate kernel/time/timer.c:727 [inline]
__mod_timer+0x77d/0xe30 kernel/time/timer.c:1048
__queue_delayed_work+0x1a7/0x270 kernel/workqueue.c:1655
queue_delayed_work_on+0xd1/0xe0 kernel/workqueue.c:1680
queue_delayed_work include/linux/workqueue.h:522 [inline]
schedule_delayed_work include/linux/workqueue.h:626 [inline]
unaccount_event kernel/events/core.c:4703 [inline]
_free_event+0xf49/0x1300 kernel/events/core.c:4812
put_event kernel/events/core.c:4935 [inline]
perf_event_release_kernel+0xa24/0xe00 kernel/events/core.c:5050
perf_release+0x33/0x40 kernel/events/core.c:5060
__fput+0x283/0x920 fs/file_table.c:280
task_work_run+0xdd/0x190 kernel/task_work.c:140
tracehook_notify_resume include/linux/tracehook.h:189 [inline]
exit_to_user_mode_loop kernel/entry/common.c:174 [inline]
exit_to_user_mode_prepare+0x249/0x250 kernel/entry/common.c:201
__syscall_exit_to_user_mode_work kernel/entry/common.c:291 [inline]
syscall_exit_to_user_mode+0x19/0x50 kernel/entry/common.c:302
entry_SYSCALL_64_after_hwframe+0x44/0xa9

other info that might help us debug this:

Chain exists of:
allocation_wait.lock --> &rq->lock --> &base->lock

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock(&base->lock);
lock(&rq->lock);
lock(&base->lock);
lock(allocation_wait.lock);

*** DEADLOCK ***

1 lock held by syz-executor.0/9817:
#0: ffff8880b9e24898 (&base->lock){-.-.}-{2:2}, at: lock_timer_base+0x5a/0x1f0 kernel/time/timer.c:944

stack backtrace:
CPU: 0 PID: 9817 Comm: syz-executor.0 Not tainted 5.10.0-next-20201223-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:79 [inline]
dump_stack+0x107/0x163 lib/dump_stack.c:120
check_noncircular+0x25f/0x2e0 kernel/locking/lockdep.c:2117
check_prev_add kernel/locking/lockdep.c:2868 [inline]
check_prevs_add kernel/locking/lockdep.c:2993 [inline]
validate_chain kernel/locking/lockdep.c:3608 [inline]
__lock_acquire+0x2ade/0x5500 kernel/locking/lockdep.c:4832
lock_acquire kernel/locking/lockdep.c:5437 [inline]
lock_acquire+0x29d/0x740 kernel/locking/lockdep.c:5402
__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0x39/0x50 kernel/locking/spinlock.c:159
__wake_up_common_lock+0xb4/0x130 kernel/sched/wait.c:137
__kfence_alloc+0xa0/0xbc0 mm/kfence/core.c:710
kfence_alloc include/linux/kfence.h:108 [inline]
slab_alloc_node mm/slub.c:2826 [inline]
slab_alloc mm/slub.c:2911 [inline]
kmem_cache_alloc+0x259/0x470 mm/slub.c:2916
kmem_cache_zalloc include/linux/slab.h:672 [inline]
fill_pool+0x264/0x5c0 lib/debugobjects.c:171
__debug_object_init+0x7a/0xd10 lib/debugobjects.c:560
debug_object_init lib/debugobjects.c:615 [inline]
debug_object_activate+0x32c/0x3e0 lib/debugobjects.c:701
debug_timer_activate kernel/time/timer.c:727 [inline]
__mod_timer+0x77d/0xe30 kernel/time/timer.c:1048
__queue_delayed_work+0x1a7/0x270 kernel/workqueue.c:1655
queue_delayed_work_on+0xd1/0xe0 kernel/workqueue.c:1680
queue_delayed_work include/linux/workqueue.h:522 [inline]
schedule_delayed_work include/linux/workqueue.h:626 [inline]
unaccount_event kernel/events/core.c:4703 [inline]
_free_event+0xf49/0x1300 kernel/events/core.c:4812
put_event kernel/events/core.c:4935 [inline]
perf_event_release_kernel+0xa24/0xe00 kernel/events/core.c:5050
perf_release+0x33/0x40 kernel/events/core.c:5060
__fput+0x283/0x920 fs/file_table.c:280
task_work_run+0xdd/0x190 kernel/task_work.c:140
tracehook_notify_resume include/linux/tracehook.h:189 [inline]
exit_to_user_mode_loop kernel/entry/common.c:174 [inline]
exit_to_user_mode_prepare+0x249/0x250 kernel/entry/common.c:201
__syscall_exit_to_user_mode_work kernel/entry/common.c:291 [inline]
syscall_exit_to_user_mode+0x19/0x50 kernel/entry/common.c:302
entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x417bf1
Code: 75 14 b8 03 00 00 00 0f 05 48 3d 01 f0 ff ff 0f 83 a4 1a 00 00 c3 48 83 ec 08 e8 0a fc ff ff 48 89 04 24 b8 03 00 00 00 0f 05 <48> 8b 3c 24 48 89 c2 e8 53 fc ff ff 48 89 d0 48 83 c4 08 48 3d 01
RSP: 002b:00007ffe977a18a0 EFLAGS: 00000293 ORIG_RAX: 0000000000000003
RAX: 0000000000000000 RBX: 0000000000000004 RCX: 0000000000417bf1
RDX: 0000000000000000 RSI: 0000000080000000 RDI: 0000000000000003
RBP: 0000000000000001 R08: 0000000000000ab3 R09: 00000000e0282ab7
R10: 00007ffe977a1980 R11: 0000000000000293 R12: 000000000119ca00
R13: 000000000119ca00 R14: 00000000000003e8 R15: 000000000119bf8c


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

[Hillf Danton]

Sat, 02 Jan 2021 22:59:11 -0800

To avoid ttwu, open code wait_event_timeout().

--- a/mm/kfence/core.c
+++ b/mm/kfence/core.c
@@ -94,9 +94,6 @@ DEFINE_STATIC_KEY_FALSE(kfence_allocatio
/* Gates the allocation, ensuring only one succeeds in a given period. */
static atomic_t allocation_gate = ATOMIC_INIT(1);

-/* Wait queue to wake up allocation-gate timer task. */
-static DECLARE_WAIT_QUEUE_HEAD(allocation_wait);
-
/* Statistics counters for debugfs. */
enum kfence_counter_id {
KFENCE_COUNTER_ALLOCATED,
@@ -586,6 +583,8 @@ late_initcall(kfence_debugfs_init);
static struct delayed_work kfence_timer;
static void toggle_allocation_gate(struct work_struct *work)
{
+ unsigned long end;
+
if (!READ_ONCE(kfence_enabled))
return;

@@ -596,7 +595,11 @@ static void toggle_allocation_gate(struc
* Await an allocation. Timeout after 1 second, in case the kernel stops
* doing allocations, to avoid stalling this worker task for too long.
*/
- wait_event_timeout(allocation_wait, atomic_read(&allocation_gate) != 0, HZ);
+ for (end = jiffies + HZ; jiffies < end;) {
+ if (atomic_read(&allocation_gate) != 0)
+ break;
+ schedule_timeout(1);
+ }

/* Disable static key and reset timer. */
static_branch_disable(&kfence_allocation_key);
@@ -707,7 +710,6 @@ void *__kfence_alloc(struct kmem_cache *
*/
if (atomic_read(&allocation_gate) || atomic_inc_return(&allocation_gate) > 1)
return NULL;
- wake_up(&allocation_wait);

if (!READ_ONCE(kfence_enabled))
return NULL;

[Marco Elver]

Thanks, I'll send a patch.

We need set_current_state(TASK_UNINTERRUPTIBLE) here, otherwise
schedule_timeout() has no effect. I'll fix this...

> + schedule_timeout(1);
> + }

__set_current_state(TASK_RUNNING);

> /* Disable static key and reset timer. */
> static_branch_disable(&kfence_allocation_key);
> @@ -707,7 +710,6 @@ void *__kfence_alloc(struct kmem_cache *
> */
> if (atomic_read(&allocation_gate) || atomic_inc_return(&allocation_gate) > 1)
> return NULL;
> - wake_up(&allocation_wait);
>
> if (!READ_ONCE(kfence_enabled))
> return NULL;
>

> --
> You received this message because you are subscribed to the Google Groups "syzkaller-bugs" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller-bug...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/syzkaller-bugs/20210103094904.14908-1-hdanton%40sina.com.

[syzbot]

syzbot has found a reproducer for the following issue on:

HEAD commit: 35c3e2791756 Revert "net: Re-use and set mono_delivery_tim..
git tree: net
console output: https://syzkaller.appspot.com/x/log.txt?x=10569181180000
kernel config: https://syzkaller.appspot.com/x/.config?x=6fb1be60a193d440
dashboard link: https://syzkaller.appspot.com/bug?extid=8983d6d4f7df556be565
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=13d9fa4e180000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=137afac9180000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/26b55a26fc12/disk-35c3e279.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/6f39fa55c828/vmlinux-35c3e279.xz
kernel image: https://storage.googleapis.com/syzbot-assets/e1e0501539e6/bzImage-35c3e279.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+8983d6...@syzkaller.appspotmail.com

=====================================================

WARNING: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected
6.8.0-syzkaller-05228-g35c3e2791756 #0 Not tainted
-----------------------------------------------------
rcu_preempt/16 [HC0[0]:SC0[2]:HE0:SE0] is trying to acquire:
ffff888021c65020 (&htab->buckets[i].lock){+...}-{2:2}, at: spin_lock_bh include/linux/spinlock.h:356 [inline]
ffff888021c65020 (&htab->buckets[i].lock){+...}-{2:2}, at: sock_hash_delete_elem+0xb0/0x300 net/core/sock_map.c:939

and this task is already holding:
ffff8880b952a758
(&base->lock){-.-.}-{2:2}, at: lock_timer_base+0x112/0x240 kernel/time/timer.c:1051
which would create a new lock dependency:
(&base->lock){-.-.}-{2:2} -> (
&htab->buckets[i].lock){+...}-{2:2}

but this new dependency connects a HARDIRQ-irq-safe lock:

(&base->lock){-.-.}-{2:2}

... which became HARDIRQ-irq-safe at:
lock_acquire+0x1e4/0x530 kernel/locking/lockdep.c:5754
__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0xd5/0x120 kernel/locking/spinlock.c:162
lock_timer_base+0x112/0x240 kernel/time/timer.c:1051
add_timer_on+0x1e5/0x5c0 kernel/time/timer.c:1366
handle_irq_event_percpu kernel/irq/handle.c:195 [inline]
handle_irq_event+0xad/0x1f0 kernel/irq/handle.c:210
handle_level_irq+0x3c5/0x6e0 kernel/irq/chip.c:648
generic_handle_irq_desc include/linux/irqdesc.h:161 [inline]
handle_irq arch/x86/kernel/irq.c:238 [inline]
__common_interrupt+0x13a/0x230 arch/x86/kernel/irq.c:257
common_interrupt+0xa5/0xd0 arch/x86/kernel/irq.c:247
asm_common_interrupt+0x26/0x40 arch/x86/include/asm/idtentry.h:693
__raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:152 [inline]
_raw_spin_unlock_irqrestore+0xd8/0x140 kernel/locking/spinlock.c:194
__setup_irq+0x1277/0x1cf0 kernel/irq/manage.c:1818
request_threaded_irq+0x2ab/0x380 kernel/irq/manage.c:2202
request_irq include/linux/interrupt.h:168 [inline]
setup_default_timer_irq+0x25/0x60 arch/x86/kernel/time.c:70
x86_late_time_init+0x66/0xc0 arch/x86/kernel/time.c:94
start_kernel+0x3f3/0x500 init/main.c:1039
x86_64_start_reservations+0x2a/0x30 arch/x86/kernel/head64.c:509
x86_64_start_kernel+0x99/0xa0 arch/x86/kernel/head64.c:490
common_startup_64+0x13e/0x147

to a HARDIRQ-irq-unsafe lock:
(&htab->buckets[i].lock){+...}-{2:2}

... which became HARDIRQ-irq-unsafe at:
...
lock_acquire+0x1e4/0x530 kernel/locking/lockdep.c:5754
__raw_spin_lock_bh include/linux/spinlock_api_smp.h:126 [inline]
_raw_spin_lock_bh+0x35/0x50 kernel/locking/spinlock.c:178
spin_lock_bh include/linux/spinlock.h:356 [inline]
sock_hash_free+0x164/0x820 net/core/sock_map.c:1154
bpf_map_free_deferred+0xe6/0x110 kernel/bpf/syscall.c:734
process_one_work kernel/workqueue.c:3254 [inline]
process_scheduled_works+0xa00/0x1770 kernel/workqueue.c:3335
worker_thread+0x86d/0xd70 kernel/workqueue.c:3416
kthread+0x2f0/0x390 kernel/kthread.c:388
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:243

other info that might help us debug this:

Possible interrupt unsafe locking scenario:

CPU0 CPU1
---- ----
lock(&htab->buckets[i].lock
);
local_irq_disable();
lock(&base->lock);
lock(&htab->buckets[i].lock
);
<Interrupt>
lock(&base->lock);

*** DEADLOCK ***

2 locks held by rcu_preempt/16:
#0:
ffff8880b952a758
(&base->lock){-.-.}-{2:2}, at: lock_timer_base+0x112/0x240 kernel/time/timer.c:1051
#1: ffffffff8e131920
(rcu_read_lock
){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:298 [inline]
){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:750 [inline]
){....}-{1:2}, at: __bpf_trace_run kernel/trace/bpf_trace.c:2380 [inline]
){....}-{1:2}, at: bpf_trace_run2+0x114/0x420 kernel/trace/bpf_trace.c:2420

the dependencies between HARDIRQ-irq-safe lock and the holding lock:
-> (&base->lock){-.-.}-{2:2} {
IN-HARDIRQ-W at:
lock_acquire+0x1e4/0x530 kernel/locking/lockdep.c:5754
__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0xd5/0x120 kernel/locking/spinlock.c:162
lock_timer_base+0x112/0x240 kernel/time/timer.c:1051
add_timer_on+0x1e5/0x5c0 kernel/time/timer.c:1366
handle_irq_event_percpu kernel/irq/handle.c:195 [inline]
handle_irq_event+0xad/0x1f0 kernel/irq/handle.c:210
handle_level_irq+0x3c5/0x6e0 kernel/irq/chip.c:648
generic_handle_irq_desc include/linux/irqdesc.h:161 [inline]
handle_irq arch/x86/kernel/irq.c:238 [inline]
__common_interrupt+0x13a/0x230 arch/x86/kernel/irq.c:257
common_interrupt+0xa5/0xd0 arch/x86/kernel/irq.c:247
asm_common_interrupt+0x26/0x40 arch/x86/include/asm/idtentry.h:693
__raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:152 [inline]
_raw_spin_unlock_irqrestore+0xd8/0x140 kernel/locking/spinlock.c:194
__setup_irq+0x1277/0x1cf0 kernel/irq/manage.c:1818
request_threaded_irq+0x2ab/0x380 kernel/irq/manage.c:2202
request_irq include/linux/interrupt.h:168 [inline]
setup_default_timer_irq+0x25/0x60 arch/x86/kernel/time.c:70
x86_late_time_init+0x66/0xc0 arch/x86/kernel/time.c:94
start_kernel+0x3f3/0x500 init/main.c:1039
x86_64_start_reservations+0x2a/0x30 arch/x86/kernel/head64.c:509
x86_64_start_kernel+0x99/0xa0 arch/x86/kernel/head64.c:490
common_startup_64+0x13e/0x147
IN-SOFTIRQ-W at:
lock_acquire+0x1e4/0x530 kernel/locking/lockdep.c:5754
__raw_spin_lock_irq include/linux/spinlock_api_smp.h:119 [inline]
_raw_spin_lock_irq+0xd3/0x120 kernel/locking/spinlock.c:170
__run_timer_base+0x103/0x8e0 kernel/time/timer.c:2418
run_timer_base kernel/time/timer.c:2428 [inline]
run_timer_softirq+0x67/0x170 kernel/time/timer.c:2436
__do_softirq+0x2be/0x943 kernel/softirq.c:554
invoke_softirq kernel/softirq.c:428 [inline]
__irq_exit_rcu+0xf2/0x1c0 kernel/softirq.c:633
irq_exit_rcu+0x9/0x30 kernel/softirq.c:645
common_interrupt+0xaa/0xd0 arch/x86/kernel/irq.c:247
asm_common_interrupt+0x26/0x40 arch/x86/include/asm/idtentry.h:693
console_flush_all+0x9cd/0xec0
console_unlock+0x13b/0x4d0 kernel/printk/printk.c:3025
vprintk_emit+0x509/0x720 kernel/printk/printk.c:2292
_printk+0xd5/0x120 kernel/printk/printk.c:2317
cpu_select_mitigations+0x3c/0xa0 arch/x86/kernel/cpu/bugs.c:148
arch_cpu_finalize_init+0x20/0xa0 arch/x86/kernel/cpu/common.c:2325
start_kernel+0x402/0x500 init/main.c:1043
x86_64_start_reservations+0x2a/0x30 arch/x86/kernel/head64.c:509
x86_64_start_kernel+0x99/0xa0 arch/x86/kernel/head64.c:490
common_startup_64+0x13e/0x147
INITIAL USE
at:
lock_acquire+0x1e4/0x530 kernel/locking/lockdep.c:5754
__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0xd5/0x120 kernel/locking/spinlock.c:162
lock_timer_base+0x112/0x240 kernel/time/timer.c:1051
__mod_timer+0x1ca/0xeb0 kernel/time/timer.c:1132
queue_delayed_work_on+0x15a/0x260 kernel/workqueue.c:2595
queue_delayed_work include/linux/workqueue.h:620 [inline]
crng_reseed+0xe7/0x220 drivers/char/random.c:258
random_init+0x1a9/0x300 drivers/char/random.c:901
start_kernel+0x253/0x500 init/main.c:991
x86_64_start_reservations+0x2a/0x30 arch/x86/kernel/head64.c:509
x86_64_start_kernel+0x99/0xa0 arch/x86/kernel/head64.c:490
common_startup_64+0x13e/0x147
}
... key at: [<ffffffff945023c0>] init_timer_cpu.__key+0x0/0x20

the dependencies between the lock to be acquired
and HARDIRQ-irq-unsafe lock:
->
(&htab->buckets[i].lock
){+...}-{2:2} {
HARDIRQ-ON-W at:
lock_acquire+0x1e4/0x530 kernel/locking/lockdep.c:5754
__raw_spin_lock_bh include/linux/spinlock_api_smp.h:126 [inline]
_raw_spin_lock_bh+0x35/0x50 kernel/locking/spinlock.c:178
spin_lock_bh include/linux/spinlock.h:356 [inline]
sock_hash_free+0x164/0x820 net/core/sock_map.c:1154
bpf_map_free_deferred+0xe6/0x110 kernel/bpf/syscall.c:734
process_one_work kernel/workqueue.c:3254 [inline]
process_scheduled_works+0xa00/0x1770 kernel/workqueue.c:3335
worker_thread+0x86d/0xd70 kernel/workqueue.c:3416
kthread+0x2f0/0x390 kernel/kthread.c:388
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:243
INITIAL USE
at:
lock_acquire+0x1e4/0x530 kernel/locking/lockdep.c:5754
__raw_spin_lock_bh include/linux/spinlock_api_smp.h:126 [inline]
_raw_spin_lock_bh+0x35/0x50 kernel/locking/spinlock.c:178
spin_lock_bh include/linux/spinlock.h:356 [inline]
sock_hash_free+0x164/0x820 net/core/sock_map.c:1154
bpf_map_free_deferred+0xe6/0x110 kernel/bpf/syscall.c:734
process_one_work kernel/workqueue.c:3254 [inline]
process_scheduled_works+0xa00/0x1770 kernel/workqueue.c:3335
worker_thread+0x86d/0xd70 kernel/workqueue.c:3416
kthread+0x2f0/0x390 kernel/kthread.c:388
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:243
}
... key at: [<ffffffff94882300>] sock_hash_alloc.__key+0x0/0x20
... acquired at:
lock_acquire+0x1e4/0x530 kernel/locking/lockdep.c:5754
__raw_spin_lock_bh include/linux/spinlock_api_smp.h:126 [inline]
_raw_spin_lock_bh+0x35/0x50 kernel/locking/spinlock.c:178
spin_lock_bh include/linux/spinlock.h:356 [inline]
sock_hash_delete_elem+0xb0/0x300 net/core/sock_map.c:939
bpf_prog_2c29ac5cdc6b1842+0x42/0x46
bpf_dispatcher_nop_func include/linux/bpf.h:1234 [inline]
__bpf_prog_run include/linux/filter.h:657 [inline]
bpf_prog_run include/linux/filter.h:664 [inline]
__bpf_trace_run kernel/trace/bpf_trace.c:2381 [inline]
bpf_trace_run2+0x204/0x420 kernel/trace/bpf_trace.c:2420
trace_timer_start include/trace/events/timer.h:52 [inline]
enqueue_timer+0x396/0x550 kernel/time/timer.c:663
internal_add_timer kernel/time/timer.c:688 [inline]
__mod_timer+0xa0e/0xeb0 kernel/time/timer.c:1183
schedule_timeout+0x1b9/0x310 kernel/time/timer.c:2571
rcu_gp_fqs_loop+0x2df/0x1370 kernel/rcu/tree.c:1663
rcu_gp_kthread+0xa7/0x3b0 kernel/rcu/tree.c:1862
kthread+0x2f0/0x390 kernel/kthread.c:388
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:243


stack backtrace:
CPU: 1 PID: 16 Comm: rcu_preempt Not tainted 6.8.0-syzkaller-05228-g35c3e2791756 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 02/29/2024
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x1e7/0x2e0 lib/dump_stack.c:106
print_bad_irq_dependency kernel/locking/lockdep.c:2626 [inline]
check_irq_usage kernel/locking/lockdep.c:2865 [inline]
check_prev_add kernel/locking/lockdep.c:3138 [inline]
check_prevs_add kernel/locking/lockdep.c:3253 [inline]
validate_chain+0x4dc7/0x58e0 kernel/locking/lockdep.c:3869
__lock_acquire+0x1346/0x1fd0 kernel/locking/lockdep.c:5137
lock_acquire+0x1e4/0x530 kernel/locking/lockdep.c:5754
__raw_spin_lock_bh include/linux/spinlock_api_smp.h:126 [inline]
_raw_spin_lock_bh+0x35/0x50 kernel/locking/spinlock.c:178
spin_lock_bh include/linux/spinlock.h:356 [inline]
sock_hash_delete_elem+0xb0/0x300 net/core/sock_map.c:939
bpf_prog_2c29ac5cdc6b1842+0x42/0x46
bpf_dispatcher_nop_func include/linux/bpf.h:1234 [inline]
__bpf_prog_run include/linux/filter.h:657 [inline]
bpf_prog_run include/linux/filter.h:664 [inline]
__bpf_trace_run kernel/trace/bpf_trace.c:2381 [inline]
bpf_trace_run2+0x204/0x420 kernel/trace/bpf_trace.c:2420
trace_timer_start include/trace/events/timer.h:52 [inline]
enqueue_timer+0x396/0x550 kernel/time/timer.c:663
internal_add_timer kernel/time/timer.c:688 [inline]
__mod_timer+0xa0e/0xeb0 kernel/time/timer.c:1183
schedule_timeout+0x1b9/0x310 kernel/time/timer.c:2571
rcu_gp_fqs_loop+0x2df/0x1370 kernel/rcu/tree.c:1663
rcu_gp_kthread+0xa7/0x3b0 kernel/rcu/tree.c:1862
kthread+0x2f0/0x390 kernel/kthread.c:388
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:243
</TASK>


---
If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.

[Sven Eckelmann]

On Tuesday, 19 March 2024 11:33:17 CET syzbot wrote:
> syzbot has found a reproducer for the following issue on:
>
> HEAD commit: 35c3e2791756 Revert "net: Re-use and set mono_delivery_tim..
> git tree: net
> console output: https://syzkaller.appspot.com/x/log.txt?x=10569181180000
> kernel config: https://syzkaller.appspot.com/x/.config?x=6fb1be60a193d440
> dashboard link: https://syzkaller.appspot.com/bug?extid=8983d6d4f7df556be565
> compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=13d9fa4e180000
> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=137afac9180000
>
> Downloadable assets:
> disk image: https://storage.googleapis.com/syzbot-assets/26b55a26fc12/disk-35c3e279.raw.xz
> vmlinux: https://storage.googleapis.com/syzbot-assets/6f39fa55c828/vmlinux-35c3e279.xz
> kernel image: https://storage.googleapis.com/syzbot-assets/e1e0501539e6/bzImage-35c3e279.xz
>
> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> Reported-by: syzbot+8983d6...@syzkaller.appspotmail.com

Sorry, this is a little bit off-topic. But how does sysbot figure out the
subsystems (like "[batman?]"). Because neither the reproducer nor the
backtrace nor the console output mention anything batman-adv related.

Kind regards,
Sven

[Aleksandr Nogikh]

Hi Sven,

On Tue, Mar 19, 2024 at 2:19 PM Sven Eckelmann <sv...@narfation.org> wrote:
>
> On Tuesday, 19 March 2024 11:33:17 CET syzbot wrote:
> > syzbot has found a reproducer for the following issue on:
> >

< ... >

>
> Sorry, this is a little bit off-topic. But how does sysbot figure out the
> subsystems (like "[batman?]"). Because neither the reproducer nor the
> backtrace nor the console output mention anything batman-adv related.

Syzbot looks at several crash reports to determine the bug subsystems
and in this case one of those crashes was pointing to
net/batman-adv/multicast.c:

https://syzkaller.appspot.com/text?tag=CrashReport&x=15afccb3280000

--
Aleksandr

>
> Kind regards,
> Sven
>
> --

[Hillf Danton]

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git ea80e3ed09ab

--- x/net/core/sock_map.c
+++ y/net/core/sock_map.c
@@ -932,11 +932,12 @@ static long sock_hash_delete_elem(struct
struct bpf_shtab_bucket *bucket;
struct bpf_shtab_elem *elem;
int ret = -ENOENT;
+ unsigned long flags;

hash = sock_hash_bucket_hash(key, key_size);
bucket = sock_hash_select_bucket(htab, hash);

- spin_lock_bh(&bucket->lock);
+ spin_lock_irqsave(&bucket->lock, flags);
elem = sock_hash_lookup_elem_raw(&bucket->head, hash, key, key_size);
if (elem) {
hlist_del_rcu(&elem->node);
@@ -944,7 +945,7 @@ static long sock_hash_delete_elem(struct
sock_hash_free_elem(htab, elem);
ret = 0;
}
- spin_unlock_bh(&bucket->lock);
+ spin_unlock_irqrestore(&bucket->lock, flags);
return ret;
}

@@ -1143,6 +1144,8 @@ static void sock_hash_free(struct bpf_ma
*/
synchronize_rcu();
for (i = 0; i < htab->buckets_num; i++) {
+ unsigned long flags;
+
bucket = sock_hash_select_bucket(htab, i);

/* We are racing with sock_hash_delete_from_link to
@@ -1151,11 +1154,11 @@ static void sock_hash_free(struct bpf_ma
* exists, psock exists and holds a ref to socket. That
* lets us to grab a socket ref too.
*/
- spin_lock_bh(&bucket->lock);
+ spin_lock_irqsave(&bucket->lock, flags);
hlist_for_each_entry(elem, &bucket->head, node)
sock_hold(elem->sk);
hlist_move_list(&bucket->head, &unlink_list);
- spin_unlock_bh(&bucket->lock);
+ spin_unlock_irqrestore(&bucket->lock, flags);

/* Process removed entries out of atomic context to
* block for socket lock before deleting the psock's
--

[syzbot]

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-and-tested-by: syzbot+8983d6...@syzkaller.appspotmail.com

Tested on:

commit: ea80e3ed net: ethernet: mtk_eth_soc: fix PPE hanging i..
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git
console output: https://syzkaller.appspot.com/x/log.txt?x=12583185180000

kernel config: https://syzkaller.appspot.com/x/.config?x=6fb1be60a193d440
dashboard link: https://syzkaller.appspot.com/bug?extid=8983d6d4f7df556be565
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=12f9eaa5180000

Note: testing is done by a robot and is best-effort only.