[syzbot] [bpf?] WARNING: locking bug in bpf_map_put

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: f9f24ca362a4 Add linux-next specific files for 20241031
git tree: linux-next
console+strace: https://syzkaller.appspot.com/x/log.txt?x=14886630580000
kernel config: https://syzkaller.appspot.com/x/.config?x=328572ed4d152be9
dashboard link: https://syzkaller.appspot.com/bug?extid=d2adb332fe371b0595e3
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=174432a7980000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=14ffe55f980000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/eb84549dd6b3/disk-f9f24ca3.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/beb29bdfa297/vmlinux-f9f24ca3.xz
kernel image: https://storage.googleapis.com/syzbot-assets/8881fe3245ad/bzImage-f9f24ca3.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+d2adb3...@syzkaller.appspotmail.com

=============================
[ BUG: Invalid wait context ]
6.12.0-rc5-next-20241031-syzkaller #0 Not tainted
-----------------------------
syz-executor304/5844 is trying to lock:
ffffffff8e9ba4b8 (map_idr_lock){+...}-{3:3}, at: bpf_map_free_id kernel/bpf/syscall.c:468 [inline]
ffffffff8e9ba4b8 (map_idr_lock){+...}-{3:3}, at: bpf_map_put+0x9a/0x380 kernel/bpf/syscall.c:902
other info that might help us debug this:
context-{5:5}
2 locks held by syz-executor304/5844:
#0: ffffffff8e939f20 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire include/linux/rcupdate.h:337 [inline]
#0: ffffffff8e939f20 (rcu_read_lock){....}-{1:3}, at: rcu_read_lock include/linux/rcupdate.h:849 [inline]
#0: ffffffff8e939f20 (rcu_read_lock){....}-{1:3}, at: map_delete_elem+0x338/0x5c0 kernel/bpf/syscall.c:1777
#1: ffff88807b870410 (&htab->lockdep_key){....}-{2:2}, at: htab_lock_bucket+0x1a4/0x370 kernel/bpf/hashtab.c:167
stack backtrace:
CPU: 1 UID: 0 PID: 5844 Comm: syz-executor304 Not tainted 6.12.0-rc5-next-20241031-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:94 [inline]
dump_stack_lvl+0x241/0x360 lib/dump_stack.c:120
print_lock_invalid_wait_context kernel/locking/lockdep.c:4826 [inline]
check_wait_context kernel/locking/lockdep.c:4898 [inline]
__lock_acquire+0x15a8/0x2100 kernel/locking/lockdep.c:5176
lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5849
__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0xd5/0x120 kernel/locking/spinlock.c:162
bpf_map_free_id kernel/bpf/syscall.c:468 [inline]
bpf_map_put+0x9a/0x380 kernel/bpf/syscall.c:902
htab_put_fd_value kernel/bpf/hashtab.c:911 [inline]
free_htab_elem+0xbb/0x460 kernel/bpf/hashtab.c:946
htab_map_delete_elem+0x576/0x6b0 kernel/bpf/hashtab.c:1438
map_delete_elem+0x431/0x5c0 kernel/bpf/syscall.c:1778
__sys_bpf+0x598/0x810 kernel/bpf/syscall.c:5745
__do_sys_bpf kernel/bpf/syscall.c:5861 [inline]
__se_sys_bpf kernel/bpf/syscall.c:5859 [inline]
__x64_sys_bpf+0x7c/0x90 kernel/bpf/syscall.c:5859
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f9ad03385e9
Code: 48 83 c4 28 c3 e8 37 17 00 00 0f 1f 80 00 00 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffd14d58828 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
RAX: ffffffffffffffda RBX: 00007ffd14d589f8 RCX: 00007f9ad03385e9
RDX: 0000000000000020 RSI: 0000000020000300 RDI: 0000000000000003
R


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup

[Edward Adam Davis]

#syz test: upstream master

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
WARNING: locking bug in bpf_map_put

=============================
[ BUG: Invalid wait context ]

6.12.0-rc5-syzkaller-00291-g05b92660cdfe #0 Not tainted
-----------------------------
syz.0.15/6647 is trying to lock:
ffffffff8e9b57d8 (map_idr_lock){+...}-{3:3}, at: bpf_map_free_id kernel/bpf/syscall.c:380 [inline]
ffffffff8e9b57d8 (map_idr_lock){+...}-{3:3}, at: bpf_map_put+0x9a/0x380 kernel/bpf/syscall.c:808

other info that might help us debug this:
context-{5:5}

2 locks held by syz.0.15/6647:
#0: ffffffff8e937de0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire include/linux/rcupdate.h:337 [inline]
#0: ffffffff8e937de0 (rcu_read_lock){....}-{1:3}, at: rcu_read_lock include/linux/rcupdate.h:849 [inline]
#0: ffffffff8e937de0 (rcu_read_lock){....}-{1:3}, at: map_delete_elem+0x338/0x5c0 kernel/bpf/syscall.c:1677
#1: ffff88807caca0f8 (&htab->lockdep_key){....}-{2:2}, at: htab_lock_bucket+0x1a4/0x370 kernel/bpf/hashtab.c:167
stack backtrace:
CPU: 0 UID: 0 PID: 6647 Comm: syz.0.15 Not tainted 6.12.0-rc5-syzkaller-00291-g05b92660cdfe #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:94 [inline]
dump_stack_lvl+0x241/0x360 lib/dump_stack.c:120

print_lock_invalid_wait_context kernel/locking/lockdep.c:4802 [inline]
check_wait_context kernel/locking/lockdep.c:4874 [inline]
__lock_acquire+0x15a8/0x2100 kernel/locking/lockdep.c:5152
lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5825

__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0xd5/0x120 kernel/locking/spinlock.c:162

bpf_map_free_id kernel/bpf/syscall.c:380 [inline]
bpf_map_put+0x9a/0x380 kernel/bpf/syscall.c:808

htab_put_fd_value kernel/bpf/hashtab.c:911 [inline]
free_htab_elem+0xbb/0x460 kernel/bpf/hashtab.c:946
htab_map_delete_elem+0x576/0x6b0 kernel/bpf/hashtab.c:1438

map_delete_elem+0x431/0x5c0 kernel/bpf/syscall.c:1678
__sys_bpf+0x598/0x810 kernel/bpf/syscall.c:5644
__do_sys_bpf kernel/bpf/syscall.c:5760 [inline]
__se_sys_bpf kernel/bpf/syscall.c:5758 [inline]
__x64_sys_bpf+0x7c/0x90 kernel/bpf/syscall.c:5758

do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f

RIP: 0033:0x7ff3cef7e719
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ff3cfda7038 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
RAX: ffffffffffffffda RBX: 00007ff3cf135f80 RCX: 00007ff3cef7e719

RDX: 0000000000000020 RSI: 0000000020000300 RDI: 0000000000000003

RBP: 00007ff3ceff132e R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000000 R14: 00007ff3cf135f80 R15: 00007ffd7e2203e8
</TASK>


Tested on:

commit: 05b92660 Merge tag 'pci-v6.12-fixes-2' of git://git.ke..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=145a9630580000
kernel config: https://syzkaller.appspot.com/x/.config?x=a6b0d046ffd0e9c1

dashboard link: https://syzkaller.appspot.com/bug?extid=d2adb332fe371b0595e3
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40

Note: no patches were applied.

[Edward Adam Davis]

there is a raw_spin_lock and spin_lock:

raw_spin_lock(&b->raw_lock);
spin_lock(&map_idr_lock);
spin_unlock(&map_idr_lock);
raw_spin_unlock(&b->raw_lock);

#syz test: upstream master

diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index a8f1808a1ca5..4891904ee1fc 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -54,7 +54,7 @@ DEFINE_PER_CPU(int, bpf_prog_active);
static DEFINE_IDR(prog_idr);
static DEFINE_SPINLOCK(prog_idr_lock);
static DEFINE_IDR(map_idr);
-static DEFINE_SPINLOCK(map_idr_lock);
+static DEFINE_RAW_SPINLOCK(map_idr_lock);
static DEFINE_IDR(link_idr);
static DEFINE_SPINLOCK(link_idr_lock);

@@ -352,11 +352,11 @@ static int bpf_map_alloc_id(struct bpf_map *map)
int id;

idr_preload(GFP_KERNEL);
- spin_lock_bh(&map_idr_lock);
+ raw_spin_lock_bh(&map_idr_lock);
id = idr_alloc_cyclic(&map_idr, map, 1, INT_MAX, GFP_ATOMIC);
if (id > 0)
map->id = id;
- spin_unlock_bh(&map_idr_lock);
+ raw_spin_unlock_bh(&map_idr_lock);
idr_preload_end();

if (WARN_ON_ONCE(!id))
@@ -377,12 +377,12 @@ void bpf_map_free_id(struct bpf_map *map)
if (!map->id)
return;

- spin_lock_irqsave(&map_idr_lock, flags);
+ raw_spin_lock_irqsave(&map_idr_lock, flags);

idr_remove(&map_idr, map->id);
map->id = 0;

- spin_unlock_irqrestore(&map_idr_lock, flags);
+ raw_spin_unlock_irqrestore(&map_idr_lock, flags);
}

#ifdef CONFIG_MEMCG
@@ -1479,9 +1479,9 @@ struct bpf_map *__bpf_map_inc_not_zero(struct bpf_map *map, bool uref)

struct bpf_map *bpf_map_inc_not_zero(struct bpf_map *map)
{
- spin_lock_bh(&map_idr_lock);
+ raw_spin_lock_bh(&map_idr_lock);
map = __bpf_map_inc_not_zero(map, false);
- spin_unlock_bh(&map_idr_lock);
+ raw_spin_unlock_bh(&map_idr_lock);

return map;
}
@@ -4278,11 +4278,37 @@ static int bpf_obj_get_next_id(const union bpf_attr *attr,
return err;
}

+static int bpf_obj_get_next_id_raw(const union bpf_attr *attr,
+ union bpf_attr __user *uattr,
+ struct idr *idr,
+ raw_spinlock_t *lock)
+{
+ u32 next_id = attr->start_id;
+ int err = 0;
+
+ if (CHECK_ATTR(BPF_OBJ_GET_NEXT_ID) || next_id >= INT_MAX)
+ return -EINVAL;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ next_id++;
+ raw_spin_lock_bh(lock);
+ if (!idr_get_next(idr, &next_id))
+ err = -ENOENT;
+ raw_spin_unlock_bh(lock);
+
+ if (!err)
+ err = put_user(next_id, &uattr->next_id);
+
+ return err;
+}
+
struct bpf_map *bpf_map_get_curr_or_next(u32 *id)
{
struct bpf_map *map;

- spin_lock_bh(&map_idr_lock);
+ raw_spin_lock_bh(&map_idr_lock);
again:
map = idr_get_next(&map_idr, id);
if (map) {
@@ -4292,7 +4318,7 @@ struct bpf_map *bpf_map_get_curr_or_next(u32 *id)
goto again;
}
}
- spin_unlock_bh(&map_idr_lock);
+ raw_spin_unlock_bh(&map_idr_lock);

return map;
}
@@ -4378,13 +4404,13 @@ static int bpf_map_get_fd_by_id(const union bpf_attr *attr)
if (f_flags < 0)
return f_flags;

- spin_lock_bh(&map_idr_lock);
+ raw_spin_lock_bh(&map_idr_lock);
map = idr_find(&map_idr, id);
if (map)
map = __bpf_map_inc_not_zero(map, true);
else
map = ERR_PTR(-ENOENT);
- spin_unlock_bh(&map_idr_lock);
+ raw_spin_unlock_bh(&map_idr_lock);

if (IS_ERR(map))
return PTR_ERR(map);
@@ -5656,7 +5682,7 @@ static int __sys_bpf(enum bpf_cmd cmd, bpfptr_t uattr, unsigned int size)
&prog_idr, &prog_idr_lock);
break;
case BPF_MAP_GET_NEXT_ID:
- err = bpf_obj_get_next_id(&attr, uattr.user,
+ err = bpf_obj_get_next_id_raw(&attr, uattr.user,
&map_idr, &map_idr_lock);
break;
case BPF_BTF_GET_NEXT_ID:
Upstream-Status: Pending

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:

WARNING: locking bug in radix_tree_node_alloc

=============================
[ BUG: Invalid wait context ]

6.12.0-rc5-syzkaller-00299-g11066801dd4b-dirty #0 Not tainted
-----------------------------
syz.0.15/6617 is trying to lock:
ffff8880b8643cc0 (&c->lock){-.-.}-{3:3}, at: local_lock_acquire include/linux/local_lock_internal.h:29 [inline]
ffff8880b8643cc0 (&c->lock){-.-.}-{3:3}, at: ___slab_alloc+0x265/0x14b0 mm/slub.c:3695

other info that might help us debug this:
context-{5:5}

2 locks held by syz.0.15/6617:
#0: ffff8880b863d678 (lock){+.+.}-{3:3}, at: local_lock_acquire include/linux/local_lock_internal.h:29 [inline]
#0: ffff8880b863d678 (lock){+.+.}-{3:3}, at: __radix_tree_preload+0x80/0x860 lib/radix-tree.c:334
#1: ffffffff8e9b57d8 (map_idr_lock){+...}-{2:2}, at: bpf_map_alloc_id+0x21/0xe0 kernel/bpf/syscall.c:355
stack backtrace:
CPU: 0 UID: 0 PID: 6617 Comm: syz.0.15 Not tainted 6.12.0-rc5-syzkaller-00299-g11066801dd4b-dirty #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:94 [inline]
dump_stack_lvl+0x241/0x360 lib/dump_stack.c:120
print_lock_invalid_wait_context kernel/locking/lockdep.c:4802 [inline]
check_wait_context kernel/locking/lockdep.c:4874 [inline]
__lock_acquire+0x15a8/0x2100 kernel/locking/lockdep.c:5152
lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5825

local_lock_acquire include/linux/local_lock_internal.h:29 [inline]
___slab_alloc+0x27e/0x14b0 mm/slub.c:3695
__slab_alloc+0x58/0xa0 mm/slub.c:3908
__slab_alloc_node mm/slub.c:3961 [inline]
slab_alloc_node mm/slub.c:4122 [inline]
kmem_cache_alloc_noprof+0x1c1/0x2a0 mm/slub.c:4141
radix_tree_node_alloc+0x8b/0x3c0 lib/radix-tree.c:276
idr_get_free+0x296/0xab0 lib/radix-tree.c:1506
idr_alloc_u32+0x195/0x330 lib/idr.c:46
idr_alloc_cyclic+0x106/0x300 lib/idr.c:125
bpf_map_alloc_id+0x40/0xe0 kernel/bpf/syscall.c:356
map_create+0xdc3/0x11c0 kernel/bpf/syscall.c:1398
__sys_bpf+0x6c8/0x810 kernel/bpf/syscall.c:5661
__do_sys_bpf kernel/bpf/syscall.c:5786 [inline]
__se_sys_bpf kernel/bpf/syscall.c:5784 [inline]
__x64_sys_bpf+0x7c/0x90 kernel/bpf/syscall.c:5784

do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f

RIP: 0033:0x7fe23117e719

Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48

RSP: 002b:00007fe231ee3038 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
RAX: ffffffffffffffda RBX: 00007fe231335f80 RCX: 00007fe23117e719
RDX: 0000000000000048 RSI: 00000000200004c0 RDI: 0000000000000000
RBP: 00007fe2311f132e R08: 0000000000000000 R09: 0000000000000000

R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000

R13: 0000000000000001 R14: 00007fe231335f80 R15: 00007fff2658b528
</TASK>


Tested on:

commit: 11066801 Merge tag 'linux_kselftest-fixes-6.12-rc6' of..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=143a5630580000

kernel config: https://syzkaller.appspot.com/x/.config?x=a6b0d046ffd0e9c1
dashboard link: https://syzkaller.appspot.com/bug?extid=d2adb332fe371b0595e3
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=12ce3340580000

[Edward Adam Davis]

there is a raw_spin_lock and spin_lock:

raw_spin_lock(&b->raw_lock);
spin_lock(&map_idr_lock);
spin_unlock(&map_idr_lock);
raw_spin_unlock(&b->raw_lock);

#syz test: upstream master

diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c
index b14b87463ee0..c2e0b26c8053 100644
--- a/kernel/bpf/hashtab.c
+++ b/kernel/bpf/hashtab.c
@@ -78,7 +78,7 @@
*/
struct bucket {
struct hlist_nulls_head head;
- raw_spinlock_t raw_lock;
+ spinlock_t lock;
};

#define HASHTAB_MAP_LOCK_COUNT 8
@@ -140,8 +140,8 @@ static void htab_init_buckets(struct bpf_htab *htab)

for (i = 0; i < htab->n_buckets; i++) {
INIT_HLIST_NULLS_HEAD(&htab->buckets[i].head, i);
- raw_spin_lock_init(&htab->buckets[i].raw_lock);
- lockdep_set_class(&htab->buckets[i].raw_lock,
+ spin_lock_init(&htab->buckets[i].lock);
+ lockdep_set_class(&htab->buckets[i].lock,
&htab->lockdep_key);
cond_resched();
}
@@ -164,7 +164,7 @@ static inline int htab_lock_bucket(const struct bpf_htab *htab,
return -EBUSY;
}

- raw_spin_lock(&b->raw_lock);
+ spin_lock(&b->lock);
*pflags = flags;

return 0;
@@ -175,7 +175,7 @@ static inline void htab_unlock_bucket(const struct bpf_htab *htab,
unsigned long flags)
{
hash = hash & min_t(u32, HASHTAB_MAP_LOCK_MASK, htab->n_buckets - 1);
- raw_spin_unlock(&b->raw_lock);
+ spin_unlock(&b->lock);
__this_cpu_dec(*(htab->map_locked[hash]));
local_irq_restore(flags);
preempt_enable();

[syzbot]

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-by: syzbot+d2adb3...@syzkaller.appspotmail.com
Tested-by: syzbot+d2adb3...@syzkaller.appspotmail.com

Tested on:

commit: 11066801 Merge tag 'linux_kselftest-fixes-6.12-rc6' of..
git tree: upstream

console output: https://syzkaller.appspot.com/x/log.txt?x=15802987980000

kernel config: https://syzkaller.appspot.com/x/.config?x=a6b0d046ffd0e9c1
dashboard link: https://syzkaller.appspot.com/bug?extid=d2adb332fe371b0595e3
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40

patch: https://syzkaller.appspot.com/x/patch.diff?x=1395b55f980000

Note: testing is done by a robot and is best-effort only.

[syzbot]

syzbot has bisected this issue to:

commit 560af5dc839eef08a273908f390cfefefb82aa04
Author: Sebastian Andrzej Siewior <big...@linutronix.de>
Date: Wed Oct 9 15:45:03 2024 +0000

lockdep: Enable PROVE_RAW_LOCK_NESTING with PROVE_LOCKING.

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=122a4740580000
start commit: f9f24ca362a4 Add linux-next specific files for 20241031
git tree: linux-next
final oops: https://syzkaller.appspot.com/x/report.txt?x=112a4740580000
console output: https://syzkaller.appspot.com/x/log.txt?x=162a4740580000

kernel config: https://syzkaller.appspot.com/x/.config?x=328572ed4d152be9
dashboard link: https://syzkaller.appspot.com/bug?extid=d2adb332fe371b0595e3

syz repro: https://syzkaller.appspot.com/x/repro.syz?x=174432a7980000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=14ffe55f980000

Reported-by: syzbot+d2adb3...@syzkaller.appspotmail.com
Fixes: 560af5dc839e ("lockdep: Enable PROVE_RAW_LOCK_NESTING with PROVE_LOCKING.")

For information about bisection process see: https://goo.gl/tpsmEJ#bisection

[Sebastian Andrzej Siewior]

This is due to raw_spinlock_t in bucket::lock and the acquired
spinlock_t underneath. Would it would to move free part outside of the
locked section?

diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c
index b14b87463ee04..1d8d09fdd2da5 100644
--- a/kernel/bpf/hashtab.c
+++ b/kernel/bpf/hashtab.c
@@ -824,13 +824,14 @@ static bool htab_lru_map_delete_node(void *arg, struct bpf_lru_node *node)
hlist_nulls_for_each_entry_rcu(l, n, head, hash_node)
if (l == tgt_l) {
hlist_nulls_del_rcu(&l->hash_node);
- check_and_free_fields(htab, l);
bpf_map_dec_elem_count(&htab->map);
break;
}

htab_unlock_bucket(htab, b, tgt_l->hash, flags);

+ if (l == tgt_l)
+ check_and_free_fields(htab, l);
return l == tgt_l;
}

@@ -1181,14 +1182,18 @@ static long htab_map_update_elem(struct bpf_map *map, void *key, void *value,
* concurrent search will find it before old elem
*/
hlist_nulls_add_head_rcu(&l_new->hash_node, head);
- if (l_old) {
+ if (l_old)
hlist_nulls_del_rcu(&l_old->hash_node);
+ htab_unlock_bucket(htab, b, hash, flags);
+
+ if (l_old) {
if (!htab_is_prealloc(htab))
free_htab_elem(htab, l_old);
else
check_and_free_fields(htab, l_old);
}
- ret = 0;
+ return 0;
+
err:
htab_unlock_bucket(htab, b, hash, flags);
return ret;
@@ -1433,14 +1438,15 @@ static long htab_map_delete_elem(struct bpf_map *map, void *key)

l = lookup_elem_raw(head, hash, key, key_size);

- if (l) {
+ if (l)
hlist_nulls_del_rcu(&l->hash_node);
- free_htab_elem(htab, l);
- } else {
+ else
ret = -ENOENT;
- }

htab_unlock_bucket(htab, b, hash, flags);
+
+ if (l)
+ free_htab_elem(htab, l);
return ret;
}

@@ -1647,14 +1653,16 @@ static int __htab_map_lookup_and_delete_elem(struct bpf_map *map, void *key,
}

hlist_nulls_del_rcu(&l->hash_node);
- if (!is_lru_map)
- free_htab_elem(htab, l);
}

htab_unlock_bucket(htab, b, hash, bflags);

- if (is_lru_map && l)
- htab_lru_push_free(htab, l);
+ if (l) {
+ if (is_lru_map)
+ htab_lru_push_free(htab, l);
+ else
+ free_htab_elem(htab, l);
+ }

return ret;
}
@@ -1851,15 +1859,12 @@ __htab_map_lookup_and_delete_batch(struct bpf_map *map,

/* bpf_lru_push_free() will acquire lru_lock, which
* may cause deadlock. See comments in function
- * prealloc_lru_pop(). Let us do bpf_lru_push_free()
- * after releasing the bucket lock.
+ * prealloc_lru_pop(). htab_lru_push_free() may allocate
+ * sleeping locks. Let us do bpf_lru_push_free() after
+ * releasing the bucket lock.
*/
- if (is_lru_map) {
- l->batch_flink = node_to_free;
- node_to_free = l;
- } else {
- free_htab_elem(htab, l);
- }
+ l->batch_flink = node_to_free;
+ node_to_free = l;
}
dst_key += key_size;
dst_val += value_size;
@@ -1871,7 +1876,10 @@ __htab_map_lookup_and_delete_batch(struct bpf_map *map,
while (node_to_free) {
l = node_to_free;
node_to_free = node_to_free->batch_flink;
- htab_lru_push_free(htab, l);
+ if (is_lru_map)
+ htab_lru_push_free(htab, l);
+ else
+ free_htab_elem(htab, l);
}

next_batch:

[Hou Tao]

Hi,

On 11/5/2024 12:28 AM, Sebastian Andrzej Siewior wrote:
> On 2024-11-03 18:29:04 [-0800], syzbot wrote:
>> syzbot has bisected this issue to:
>>
>> commit 560af5dc839eef08a273908f390cfefefb82aa04
>> Author: Sebastian Andrzej Siewior <big...@linutronix.de>
>> Date: Wed Oct 9 15:45:03 2024 +0000
>>
>> lockdep: Enable PROVE_RAW_LOCK_NESTING with PROVE_LOCKING.
>>
>> bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=122a4740580000
>> start commit: f9f24ca362a4 Add linux-next specific files for 20241031
>> git tree: linux-next
>> final oops: https://syzkaller.appspot.com/x/report.txt?x=112a4740580000
>> console output: https://syzkaller.appspot.com/x/log.txt?x=162a4740580000
>> kernel config: https://syzkaller.appspot.com/x/.config?x=328572ed4d152be9
>> dashboard link: https://syzkaller.appspot.com/bug?extid=d2adb332fe371b0595e3
>> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=174432a7980000
>> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=14ffe55f980000
>>
>> Reported-by: syzbot+d2adb3...@syzkaller.appspotmail.com
>> Fixes: 560af5dc839e ("lockdep: Enable PROVE_RAW_LOCK_NESTING with PROVE_LOCKING.")
>>
>> For information about bisection process see: https://goo.gl/tpsmEJ#bisection
> This is due to raw_spinlock_t in bucket::lock and the acquired
> spinlock_t underneath. Would it would to move free part outside of the
> locked section?

I think moving free_htab_elem() after htab_unlock_bucket() is OK. But
the fix below is not enough, and there is some corn cases for
pre-allocated element . I had written a patch for the problem a few day
ago because the problem can be easily reproduced by running test_maps. I
am also writing a selftest patch for it.  I could post the patch and the
selftest patch if you are OK with it.

> .

[syzbot]

Auto-closing this bug as obsolete.
No recent activity, existing reproducers are no longer triggering the issue.