IMPORTANT: we are planning breaking/non-backward-compatible changes to some alpha Kubernetes scheduling features in 1.6
David Oppenheimer
tl;dr: If you are currently using any of the following alpha scheduling features -- node affinity/anti-affinity, inter-pod affinity/anti-affinity, taints/tolerations, or multi-scheduler -- they will stop working when you upgrade to 1.6 unless you take action. All of these features will be available in 1.6, but we are moving them from alpha to beta, and with that the way they are requested/specified is moving from annotations to object fields. So after you upgrade to 1.6, you will need to set the corresponding object fields if you want to continue to get the same scheduling behavior. Already-running pods will not be affected.
Details:
For Kubernetes 1.6 the SIGs were asked to focus on code health and stability. sig-scheduling is doing a large amount of work in both areas. Most of this work is not user-visible, but one critical part of the work that will impact users is that we are moving a number of features from alpha to beta:
In each of these cases, the feature was previously requested using an annotation. As part of moving these features to beta, they will now be specified using a field in the corresponding object. In particular:
Node affinity/anti-affinity and inter-pod affinity/anti-affinity will be specified using the Affinity field of PodSpec instead of using the scheduler.alpha.kubernetes.io/affinity annotation on Pod
Taints will be specified using the Taints field of NodeSpec instead of using the scheduler.alpha.kubernetes.io/taints annotation on Node
Tolerations will be specified using the Tolerations field of PodSpec instead of using the scheduler.alpha.kubernetes.io/tolerations annotation on Pod
The name of the scheduler responsible for scheduling a pod will be specified using the SchedulerName field of PodSpec instead of using the scheduler.alpha.kubernetes.io/name annotation on Pod. (As before, you only need to set this field if you want a pod to be processed by a scheduler other than the default Kubernetes scheduler.)
We are not maintaining backward compatibility. Once you upgrade to 1.6, the annotations will no longer have any effect. (You can still create and delete the annotations, of course. It is also safe to leave them in place, though they will have no effect when you are running 1.6 and later.) Pods that are already running will not be affected, but pod scheduling after the upgrade will only consider the beta field, not the old alpha annotation.
What this means for upgrading 1.5 -> 1.6 / downgrading 1.6 -> 1.5
When upgrading to 1.6, you will need to
Add Affinity to the pod template in your ReplicaSet, Job, etc. objects where today you are using the scheduler.alpha.kubernetes.io/affinity annotation in the pod template
Add Taints to the NodeSpec of any Node objects where today you are using the scheduler.alpha.kubernetes.io/taints annotation on the Node
Add Tolerations to the pod templates where today you are using the scheduler.alpha.kubernetes.io/tolerations annotation
Add SchedulerName to the pod templates where today you are using the scheduler.alpha.kubernetes.io/name annotation
You can leave the annotations in place (i.e. not delete them) until you are sure you won’t need to roll back to 1.5.
If you roll back to 1.5, only the annotations will affect scheduling while you are running 1.5. Once you have rolled back to 1.5, you should run the cluster/update-storage-objects.sh script, which will clear the new fields by touching every object (the annotations will be unaffected). This will ensure that when you later roll forward to 1.6, the Affinity, Taints, Tolerations, and SchedulerName fields will be cleared. If you don’t run this script, then when you roll forward to 1.6, only the objects that you happened to touch while you are running 1.5 will have the fields cleared, and the ones you didn't touch will have the values that were set when you were running 1.6 before downgrading.
We will update the documentation for these features and will have release notes reflecting the changes, but I wanted to make sure to send an announcement about this well ahead of time to warn folks.
Thanks,
David (on behalf of sig-scheduling)
Brian Grant
--
You received this message because you are subscribed to the Google Groups "Kubernetes developer/contributor discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-dev+unsubscribe@googlegroups.com.
To post to this group, send email to kubernetes-dev@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-dev/CAOU1bzeYz_wq2s9bxymAH0U%2B8x%3DG2c6vBRaE5VE3Hozx8rAgCg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.