GPU scheduling cgroup controller

[Tvrtko Ursulin]

Hi all,

Heads up to the ChromeOS community regarding a DRM cgroup controller proposal I have started working on recently - see https://lists.freedesktop.org/archives/dri-devel/2022-November/379548.html.

Tl;dr; as ChromeOS uses window focus status (background/foreground) to on-the-fly re-configure the CPU controller for the cgroup where ARC lives (and other VMs), I am trying to sketch out the same approach for GPU scheduling.

It would be the same semantics as the CPU controller, albeit a bit less strict in what can achieve. Despite that, with the i915 driver I can show that putting a GPU demanding ARC container in the background could significantly reduce the impact on the foreground rendering. Which should improve UI/graphics latency for the foreground window. For instance glxgears running un-vsynced and unfettered, in parallel to a 60fps workload:

474 frames in 5.0 seconds = 94.779 FPS
519 frames in 5.0 seconds = 103.740 FPS
518 frames in 5.0 seconds = 103.338 FPS
469 frames in 5.0 seconds = 93.788 FPS
482 frames in 5.0 seconds = 96.311 FPS

Versus same glxgears instance getting de-prioritized on the GPU after exceeding its GPU share, once its cgroup weight gets decreased:

369 frames in 5.0 seconds = 73.444 FPS
373 frames in 5.0 seconds = 74.434 FPS
389 frames in 5.0 seconds = 77.794 FPS
357 frames in 5.0 seconds = 71.393 FPS
308 frames in 5.0 seconds = 61.414 FPS

So far I've been sketching this out on Linux, but now I am moving into wiring up a ChromeOS prototype.

Controller is also designed in a way that any DRM driver which can query GPU utilisation and have some sort of scheduling priority control could implement the same approach.

I would be keen to know if ChromeOS would be interested in such capability.

Regards,

Tvrtko

[Tvrtko Ursulin]

Actually perhaps I should have sent this to ChromiumOS? Adding to Cc now.

[Robert Kroeger]

Can you write some kind of design document for this and share it with us?

Rob.

--

Robert Kroeger
rjkr...@google.com

[Tvrtko Ursulin]

Hi Rob,

I can try - what kind of design doc you have in mind - more than what I have in the cover letter, which is basically copied from one of the patch contents?

The design pretty much follows the CPU controller weights. GPU time budget is then hierarchically split through the tree and group which is over budget gets a signal from the controller to throttle down. Any driver which has support for exposing GPU utilisation in fdinfo (i915, msm, amdgpu, and etnaviv I think soon could get it) and supports some sort of scheduling priority could in theory implement this. My RFC only does it for i915.

Otherwise I am currently trying to bring up a proof of concept integration into a full ChromeOS build. I am targeting Volteer since that is what I have hardware wise. Maybe this week, if all is well, I get something up and running.

Regards,

Tvrtko

[Tvrtko Ursulin]

Interestingly, the CrOS build I did last week behaves differently in this respect that the one I experimented on a few months ago. Previously background ARC windows have been kept running, but now they appear to get paused when Chrome is the foreground window. Doubly interesting if there is 2nd ARC window, like the Play Store, then the background ARC window is kept running.

I am hunting through git log to find the rationale for this and where it is implemented.