Medium term Fair-value / usage across queues / namespaces?
13 views
Skip to first unread message
John Singleton
Apr 21, 2022, 9:56:37 PM4/21/22
to volcano-sh
Hi, new community member here, looking to evaluate volcano for my companies scheduler for future scaling (targeting 10,000 cores), to replace our current clunky Airflow implementation.
A key requirement is 'fair usage over time', i.e.
* We have 4 teams with 25% weight each.
* Last week only team A was the only team submitting jobs (and thus their queue would claim the entire cluster resources)
* This week all teams have pending workloads
We would expect that Team A would face a penalty, such that the queue weights would be (e.g 10/30/30/30)
Looking at documentation (e.g. https://github.com/volcano-sh/volcano/blob/master/docs/design/fairshare.md) that seems to be distributing the resources in a queue by namespace.
So ...
Is there an explicit feature to penalise previous users to tend to the target weights over time? If so then how is it implemented / configured?
If not then is this a planned feature?
And it looks like namespace weightings will get applied by queue - or is it simpler to just have one namespace = one queue?
* We have 4 teams with 25% weight each.
* Last week only team A was the only team submitting jobs (and thus their queue would claim the entire cluster resources)
* This week all teams have pending workloads
We would expect that Team A would face a penalty, such that the queue weights would be (e.g 10/30/30/30)
Looking at documentation (e.g. https://github.com/volcano-sh/volcano/blob/master/docs/design/fairshare.md) that seems to be distributing the resources in a queue by namespace.
So ...
Is there an explicit feature to penalise previous users to tend to the target weights over time? If so then how is it implemented / configured?
If not then is this a planned feature?
And it looks like namespace weightings will get applied by queue - or is it simpler to just have one namespace = one queue?
Reply all
Reply to author
Forward
0 new messages