Pod Scheduling Readiness

[sunil kalva]

Subject: Before scheduling a POD, we want to gate them and approved by an external QuotaManager

Problem Statement:  We’ve implemented an external quota manager responsible for reviewing all incoming POD requests for capacity/quota requirements. Only once these requests receive approval from the quota manager are they considered eligible for scheduling. Consequently, we intend to leverage the pods schedulingGates feature to realize this functionality.

It seems that Volcano is not adhering to this feature. It proceeds to schedule pods without honoring schedulingGates flag. This behavior is in contrast to the default Kubernetes kube-scheduler, which functions as expected.

Even though these pods have the schedulingGates added and were assigned to a node by the Volcano scheduler, it appears that they are subsequently rejected by the Kubernetes API server. I believe this error is occurring after the execution of the Volcano scheduler.

Failed to bind pod <namsepace/pod-name> to node <node> : &errors.StatusError{ErrStatus:v1.Status{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ListMeta:v1.ListMeta{SelfLink:"", ResourceVersion:"", Continue:"", RemainingItemCount:(*int64)(nil)}, Status:"Failure", Message:"Operation cannot be fulfilled on pods/binding \”app-name\”: pod pod-name has non-empty .spec.schedulingGates", Reason:"Conflict", Details:(*v1.StatusDetails)(0x140002188a0), Code:409}}

Please suggest how can we have this with volcano, If not can i add this feature in volcano and send the PR ? (edited) 

Reference :

1. https://kubernetes.io/docs/concepts/scheduling-eviction/pod-scheduling-readiness/,

2. https://kubernetes.io/blog/2022/12/26/pod-scheduling-readiness-alpha/#:~:text=Under%20the%20hood%2C%20scheduling%20gates,beginning%20of%20each%20scheduling%20cycle.

[William Wang]

Hi skalva,

Thank you for pointing out this issue. It is reasonable to support pod-scheduling-readiness in Volcano.  I'll open a issue in Volcano and add it in  the pipeline :)

Best regards,

William