[syzbot] WARNING in cpuset_attach

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: c40e8341e3b3 Merge tag 'cgroup-for-6.0-rc2-fixes' of git:/..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=167bc60b080000
kernel config: https://syzkaller.appspot.com/x/.config?x=911efaff115942bb
dashboard link: https://syzkaller.appspot.com/bug?extid=29d3a3b4d86c8136ad9e
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2

Unfortunately, I don't have any reproducer for this issue yet.

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+29d3a3...@syzkaller.appspotmail.com

------------[ cut here ]------------
WARNING: CPU: 0 PID: 3684 at kernel/cpu.c:347 lockdep_assert_cpus_held kernel/cpu.c:347 [inline]
WARNING: CPU: 0 PID: 3684 at kernel/cpu.c:347 lockdep_assert_cpus_held+0xbd/0xe0 kernel/cpu.c:336
Modules linked in:
CPU: 0 PID: 3684 Comm: kvm-nx-lpage-re Not tainted 6.0.0-rc2-syzkaller-00054-gc40e8341e3b3 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/22/2022
RIP: 0010:lockdep_assert_cpus_held kernel/cpu.c:347 [inline]
RIP: 0010:lockdep_assert_cpus_held+0xbd/0xe0 kernel/cpu.c:336
Code: e8 38 0c 34 00 be ff ff ff ff 48 c7 c7 70 64 e3 8b e8 37 71 37 08 31 ff 89 c3 89 c6 e8 dc 08 34 00 85 db 75 d5 e8 13 0c 34 00 <0f> 0b eb cc 48 c7 c7 44 66 de 8d e8 d3 73 80 00 e9 62 ff ff ff 48
RSP: 0018:ffffc90004777c28 EFLAGS: 00010293
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
RDX: ffff88801f243b00 RSI: ffffffff8147ff0d RDI: 0000000000000005
RBP: ffff88814b0a4800 R08: 0000000000000005 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffffc90004777de0
R13: ffffffff8bfdc2a0 R14: 1ffff920008eef8c R15: ffffc90004777e04
FS: 0000000000000000(0000) GS:ffff8880b9b00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000020833000 CR3: 000000001bddc000 CR4: 00000000003526e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<TASK>
cpuset_attach+0x92/0x520 kernel/cgroup/cpuset.c:2292
cgroup_migrate_execute+0xbc7/0x1220 kernel/cgroup/cgroup.c:2584
cgroup_attach_task+0x416/0x7c0 kernel/cgroup/cgroup.c:2875
cgroup_attach_task_all+0xde/0x140 kernel/cgroup/cgroup-v1.c:70
kvm_vm_worker_thread+0xdd/0x5a0 arch/x86/kvm/../../../virt/kvm/kvm_main.c:5979
kthread+0x2e4/0x3a0 kernel/kthread.c:376
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:306
</TASK>


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

[syzbot]

syzbot has found a reproducer for the following issue on:

HEAD commit: c40e8341e3b3 Merge tag 'cgroup-for-6.0-rc2-fixes' of git:/..
git tree: upstream

console+strace: https://syzkaller.appspot.com/x/log.txt?x=169937b5080000

kernel config: https://syzkaller.appspot.com/x/.config?x=911efaff115942bb
dashboard link: https://syzkaller.appspot.com/bug?extid=29d3a3b4d86c8136ad9e
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2

syz repro: https://syzkaller.appspot.com/x/repro.syz?x=10823695080000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=16dab02b080000

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+29d3a3...@syzkaller.appspotmail.com

------------[ cut here ]------------

WARNING: CPU: 0 PID: 3614 at kernel/cpu.c:347 lockdep_assert_cpus_held kernel/cpu.c:347 [inline]
WARNING: CPU: 0 PID: 3614 at kernel/cpu.c:347 lockdep_assert_cpus_held+0xbd/0xe0 kernel/cpu.c:336
Modules linked in:
CPU: 1 PID: 3614 Comm: vhost-3613 Not tainted 6.0.0-rc2-syzkaller-00054-gc40e8341e3b3 #0

Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/22/2022
RIP: 0010:lockdep_assert_cpus_held kernel/cpu.c:347 [inline]
RIP: 0010:lockdep_assert_cpus_held+0xbd/0xe0 kernel/cpu.c:336
Code: e8 38 0c 34 00 be ff ff ff ff 48 c7 c7 70 64 e3 8b e8 37 71 37 08 31 ff 89 c3 89 c6 e8 dc 08 34 00 85 db 75 d5 e8 13 0c 34 00 <0f> 0b eb cc 48 c7 c7 44 66 de 8d e8 d3 73 80 00 e9 62 ff ff ff 48

RSP: 0018:ffffc9000377fbe8 EFLAGS: 00010293

RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000

RDX: ffff888025c19d80 RSI: ffffffff8147ff0d RDI: 0000000000000005
RBP: ffff888075905000 R08: 0000000000000005 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffffc9000377fda0
R13: ffffffff8bfdc2a0 R14: 1ffff920006eff84 R15: ffffc9000377fdc4
FS: 0000000000000000(0000) GS:ffff8880b9a00000(0000) knlGS:0000000000000000

CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 0000000020000150 CR3: 0000000025a0b000 CR4: 0000000000350ef0

Call Trace:
<TASK>
cpuset_attach+0x92/0x520 kernel/cgroup/cpuset.c:2292
cgroup_migrate_execute+0xbc7/0x1220 kernel/cgroup/cgroup.c:2584
cgroup_attach_task+0x416/0x7c0 kernel/cgroup/cgroup.c:2875
cgroup_attach_task_all+0xde/0x140 kernel/cgroup/cgroup-v1.c:70

vhost_attach_cgroups_work+0x3a/0x90 drivers/vhost/vhost.c:523
vhost_worker+0x23d/0x3d0 drivers/vhost/vhost.c:364

[Tetsuo Handa]

syzbot is hitting percpu_rwsem_assert_held(&cpu_hotplug_lock) warning at
cpuset_attach() [1], for commit 4f7e7236435ca0ab ("cgroup: Fix
threadgroup_rwsem <-> cpus_read_lock() deadlock") missed that
cpuset_attach() is also called from cgroup_attach_task_all().
Add cpus_read_lock() like what cgroup_procs_write_start() does.

Link: https://syzkaller.appspot.com/bug?extid=29d3a3b4d86c8136ad9e [1]
Reported-by: syzbot <syzbot+29d3a3...@syzkaller.appspotmail.com>
Signed-off-by: Tetsuo Handa <penguin...@I-love.SAKURA.ne.jp>
Fixes: 4f7e7236435ca0ab ("cgroup: Fix threadgroup_rwsem <-> cpus_read_lock() deadlock")
---
Waiting for test result from syzbot. Should we remove "static" from
cgroup_attach_lock() and call from cgroup_attach_task_all() ?

kernel/cgroup/cgroup-v1.c | 2 ++
1 file changed, 2 insertions(+)

diff --git a/kernel/cgroup/cgroup-v1.c b/kernel/cgroup/cgroup-v1.c
index 2ade21b54dc4..ff6a8099eb2a 100644
--- a/kernel/cgroup/cgroup-v1.c
+++ b/kernel/cgroup/cgroup-v1.c
@@ -59,6 +59,7 @@ int cgroup_attach_task_all(struct task_struct *from, struct task_struct *tsk)
int retval = 0;

mutex_lock(&cgroup_mutex);
+ cpus_read_lock();
percpu_down_write(&cgroup_threadgroup_rwsem);
for_each_root(root) {
struct cgroup *from_cgrp;
@@ -72,6 +73,7 @@ int cgroup_attach_task_all(struct task_struct *from, struct task_struct *tsk)
break;
}
percpu_up_write(&cgroup_threadgroup_rwsem);
+ cpus_read_unlock();
mutex_unlock(&cgroup_mutex);

return retval;
--
2.18.4

[syzbot]

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-and-tested-by: syzbot+29d3a3...@syzkaller.appspotmail.com

Tested on:

commit: c40e8341 Merge tag 'cgroup-for-6.0-rc2-fixes' of git:/..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=166fc1f3080000

kernel config: https://syzkaller.appspot.com/x/.config?x=911efaff115942bb
dashboard link: https://syzkaller.appspot.com/bug?extid=29d3a3b4d86c8136ad9e
compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2

patch: https://syzkaller.appspot.com/x/patch.diff?x=1575be1d080000

Note: testing is done by a robot and is best-effort only.

[Tejun Heo]

On Thu, Aug 25, 2022 at 05:38:38PM +0900, Tetsuo Handa wrote:
> syzbot is hitting percpu_rwsem_assert_held(&cpu_hotplug_lock) warning at
> cpuset_attach() [1], for commit 4f7e7236435ca0ab ("cgroup: Fix
> threadgroup_rwsem <-> cpus_read_lock() deadlock") missed that
> cpuset_attach() is also called from cgroup_attach_task_all().
> Add cpus_read_lock() like what cgroup_procs_write_start() does.
>
> Link: https://syzkaller.appspot.com/bug?extid=29d3a3b4d86c8136ad9e [1]
> Reported-by: syzbot <syzbot+29d3a3...@syzkaller.appspotmail.com>
> Signed-off-by: Tetsuo Handa <penguin...@I-love.SAKURA.ne.jp>
> Fixes: 4f7e7236435ca0ab ("cgroup: Fix threadgroup_rwsem <-> cpus_read_lock() deadlock")

Applied to cgroup/for-6.0-fixes and sent pull request.

> Waiting for test result from syzbot. Should we remove "static" from
> cgroup_attach_lock() and call from cgroup_attach_task_all() ?

Yeah, that probably is better. Can you spin up a follow up patch?

Thanks.

--
tejun