[PATCH] sched/auto_group: fix consume memory even if add 'noautogroup' in the cmdline

[Wanpeng Li]

We have a server which have 200 CPUs and 8G memory, there is auto_group creation
which will almost consume 12MB memory even if add 'noautogroup' in the kernel
boot parameter. In addtion, SLUB per cpu partial caches freeing that is local to
a processor which requires the taking of locks at the price of more indeterminism
in the latency of the free. This patch fix it by check noautogroup earlier to avoid
free after unnecessary memory consumption.

cat /sys/kernel/slab/kmalloc-512/alloc_calls
18000 .alloc_fair_sched_group+0xec/0x1e0 age=2579/19587/286617 pid=1-8462 cpus=0-1,5,9,21,26,29,41,61,
69,73,76-77,89,92-93,97,101,109,121,125,133,141,145,149,153,161,185 nodes=1
cat /sys/kernel/slab/kmalloc-192/alloc_calls
18000 .alloc_fair_sched_group+0x110/0x1e0 age=2637/19654/286688 pid=1-8462 cpus=0-1,5,9,21,26,29,41,61,
69,73,76-77,89,92-93,97,101,109,121,125,133,141,145,149,153,161,185 nodes=1

Signed-off-by: Wanpeng Li <liw...@linux.vnet.ibm.com>
---
kernel/sched/auto_group.c | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/kernel/sched/auto_group.c b/kernel/sched/auto_group.c
index 4a07353..748ebc9 100644
--- a/kernel/sched/auto_group.c
+++ b/kernel/sched/auto_group.c
@@ -145,15 +145,11 @@ autogroup_move_group(struct task_struct *p, struct autogroup *ag)

p->signal->autogroup = autogroup_kref_get(ag);

- if (!ACCESS_ONCE(sysctl_sched_autogroup_enabled))
- goto out;
-
t = p;
do {
sched_move_task(t);
} while_each_thread(p, t);

-out:
unlock_task_sighand(p, &flags);
autogroup_kref_put(prev);
}
@@ -161,7 +157,12 @@ out:
/* Allocates GFP_KERNEL, cannot be called under any spinlock */
void sched_autogroup_create_attach(struct task_struct *p)
{
- struct autogroup *ag = autogroup_create();
+ struct autogroup *ag;
+
+ if (!ACCESS_ONCE(sysctl_sched_autogroup_enabled))
+ return;
+
+ ag = autogroup_create();

autogroup_move_group(p, ag);
/* drop extra reference added by autogroup_create() */
--
1.8.3.2

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

[Peter Zijlstra]

On Fri, Dec 27, 2013 at 06:22:31PM +0800, Wanpeng Li wrote:
> We have a server which have 200 CPUs and 8G memory, there is auto_group creation

I'm hoping that is 8T, otherwise that's a severely under provisioned
system, that's a mere 40M per cpu, does that even work?

> which will almost consume 12MB memory even if add 'noautogroup' in the kernel
> boot parameter. In addtion, SLUB per cpu partial caches freeing that is local to
> a processor which requires the taking of locks at the price of more indeterminism
> in the latency of the free. This patch fix it by check noautogroup earlier to avoid
> free after unnecessary memory consumption.

That's just a bad changelog. It fails to explain the actual problem and
it babbles about unrelated things like SLUB details.

Also, I'm not entirely sure what the intention was of this code, I've so
far tried to ignore the entire autogroup fest...

It looks like it creates and maintains the entire autogroup hierarchy,
such that if you at runtime enable the sysclt and move tasks 'back' to
the root cgroup you get the autogroup behaviour.

Was this intended? Mike?

This patch obviously breaks that.

[Mike Galbraith]

On Mon, 2014-01-06 at 13:17 +0100, Peter Zijlstra wrote:
> On Fri, Dec 27, 2013 at 06:22:31PM +0800, Wanpeng Li wrote:
> > We have a server which have 200 CPUs and 8G memory, there is auto_group creation
>
> I'm hoping that is 8T, otherwise that's a severely under provisioned
> system, that's a mere 40M per cpu, does that even work?
>
> > which will almost consume 12MB memory even if add 'noautogroup' in the kernel
> > boot parameter. In addtion, SLUB per cpu partial caches freeing that is local to
> > a processor which requires the taking of locks at the price of more indeterminism
> > in the latency of the free. This patch fix it by check noautogroup earlier to avoid
> > free after unnecessary memory consumption.
>
> That's just a bad changelog. It fails to explain the actual problem and
> it babbles about unrelated things like SLUB details.
>
> Also, I'm not entirely sure what the intention was of this code, I've so
> far tried to ignore the entire autogroup fest...
>
> It looks like it creates and maintains the entire autogroup hierarchy,
> such that if you at runtime enable the sysclt and move tasks 'back' to
> the root cgroup you get the autogroup behaviour.
>
> Was this intended? Mike?

Yeah, it was intended that autogroups always exist if you config it in.
We could make is such that noautogroup makes it irreversibly off/dead.

People with 200 ram starved CPUs can turn it off in their .config too :)

-Mike