Kubernetes v1.31.0-alpha.1 is live!

171 views
Skip to first unread message

Adolfo García Veytia

unread,
Jun 12, 2024, 4:42:37 PMJun 12
to d...@kubernetes.io, kubernete...@kubernetes.io

Kubernetes Community,

Kubernetes v1.31.0-alpha.1 has been built and pushed using Golang version 1.22.4.

The release notes have been updated in CHANGELOG-1.31.md, with a pointer to them on GitHub:

v1.31.0-alpha.1

Downloads for v1.31.0-alpha.1

Source Code

filename sha512 hash
kubernetes.tar.gz c3d3b7c0f58866a09006b47ba0e7677c95451c0c5b727963ec2bb318fcf0fd94a75f14e51485dacbcf34fab2879325216d9723162e2039d09344ab75b8313fad
kubernetes-src.tar.gz 16e46516d52f89b9bf623e90bab4d17708b540d67c153c0f81c42a4f6bb335f549b5c451c71701aeeb279ee3f60f1379df98bfab4d24db33a2ff7ef23b70c943

Client Binaries

filename sha512 hash
kubernetes-client-darwin-amd64.tar.gz 219fc2cfcd6da50693eca80209e6d6c7b1331c79c059126766ebdbb5dac56e8efb277bc39d0c32a4d1f4bf51445994c91ce27f291bccdda7859b4be666b2452f
kubernetes-client-darwin-arm64.tar.gz 054897580442e027c4d0c5c67769e0f98f464470147abb981b200358bcf13b134eac166845350f2e2c8460df3577982f18eafad3be698cfee6e5a4a2e088f0d3
kubernetes-client-linux-386.tar.gz a783ba568bbe28e0ddddcbd2c16771f2354786bcc5de4333e9d0a73a1027a8a45c2cc58c69b740db83fec12647e93df2536790df5e191d96dea914986b717ee6
kubernetes-client-linux-amd64.tar.gz f0f39dc1f8cf5dd6029afccae904cd082ed3a4da9283a4506311b0f820e50bdbe9370aaa784f382ec5cbfaa7b115ce34578801080443380f8e606fad225467f0
kubernetes-client-linux-arm.tar.gz 744b69d0b0a40d8fbcb8bd582ee36da3682e189c33a780de01b320cf07eac0b215e6051f6f57ea34b9417423d0d4a42df85d72753226d53b5fe59411b096335d
kubernetes-client-linux-arm64.tar.gz ebec17b4e81bfbd1789e2889435929c38976c5f054d093b964a12cf82c173a1d31c976db51c8a64bf822c17ef4ae41cef1a252bb53143094effe730601e63fe5
kubernetes-client-linux-ppc64le.tar.gz 0b5602ec8c9c0afafe4f7c05590bdf8176ec158abb8b52e0bea026eb937945afc52aadeb4d1547fff0883b29e1aec0b92fbbae2e950a0cffa870db21674cef9e
kubernetes-client-linux-s390x.tar.gz 21b37221c9259e0c7a3fee00f4de20fbebe435755313ed0887d44989e365a67eff0450eda836e93fccf11395c89c9702a17dc494d51633f48c7bb9afe94253c4
kubernetes-client-windows-386.tar.gz 9e261d3ce6d640e8d43f7761777ea7d62cc0b37e709a96a1e5b691bd7fc6023804dc599edadac351dc9f9107c43bd5d6b962050a3363e5d1037036e4ab51a2ed
kubernetes-client-windows-amd64.tar.gz 53606a24ff85e011fd142a2e3b6c8cda058c77afdab6698eb488ab456bf41d299ca442c50482e00535ea6453472d987de6fd75f952febc5a33e46bb5cdf9c0ee
kubernetes-client-windows-arm64.tar.gz f29dd44241d3363eecdcf7063cec5e6516698434c5494e922ee561b3553fbd214075cb0f4832dfadad7a894a3b9df9ee94bb4adb542feda2314d05b1b7b71f78

Server Binaries

filename sha512 hash
kubernetes-server-linux-amd64.tar.gz 55b2c9cacb14c2a7b657079e1b2620c0580e3a01d91b0bd3f1e8b1a70e4bb59c4c361eb8aad425734fd579d6e944aedc7695082cb640a3af902dff623a565714
kubernetes-server-linux-arm64.tar.gz 24422b969931754c7a72876d1d3ad773bdbdb42bb53ca8d2020b7987a03d20136ad5693c1aa81515b94e3ab71ed486c4b09a9d99b3ef4a7a78d8cd742f7cf9fd
kubernetes-server-linux-ppc64le.tar.gz 76b6cc096ed38e0d08c1af6ee0715e0a29674eb990ee9675abb3bb9345c70469ca25b62b7babc9afdd6628d1970817d36b66a7b5890572cb0bc9519735c78599
kubernetes-server-linux-s390x.tar.gz 4b5a1660e1acfe3e2cb03097608c9c3c7ceedd80c9b71c22ac7572db49598d6e9bff903c8415236947ea1ba14f9595a6bbc178f15959191b92805ce5b01063c3

Node Binaries

filename sha512 hash
kubernetes-node-linux-amd64.tar.gz 98b402d2cb135af8b2d328ae452fae832e4bfe9e5ab471f277fe134411a46c5493d62def5f5af1259c977bd94b90ce8c8d5e9ba8ee1c7b7fe991316510d09e71
kubernetes-node-linux-arm64.tar.gz 052a7ccb8ed370451d883b64cd219b803141eaef4a8498ee45c61d09eff1364b7c4d5785bc8247c9a806dee5887d53abe44e645ada2d45349a0163c3e229decd
kubernetes-node-linux-ppc64le.tar.gz 32a2cc80b367fb6a447d1b674eed220b13e03662f453c155b1752ccef72ccd55503ca73267cf782472e58771a57efc68eee4cb47520e09e6987a7183329d20fa
kubernetes-node-linux-s390x.tar.gz d358de45ae5566b534c9751e7acf0e577e73646d556b444020ee75a731e488ca467df1bfbc5c6a9b3e967f0ea9586bf82657cb22d569a2df69b317671dc6bcae
kubernetes-node-windows-amd64.tar.gz 95c8962439485920c0d50d85ffa037cc4dacaa61392894394759d4d9efb2525d6e1b4e6177c72eed5f55511b6f9c279795601744a1a2da2ee3cb3b518ac31c8a

Container Images

All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name.

name architectures
registry.k8s.io/conformance:v1.31.0-alpha.1 amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-apiserver:v1.31.0-alpha.1 amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-controller-manager:v1.31.0-alpha.1 amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-proxy:v1.31.0-alpha.1 amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-scheduler:v1.31.0-alpha.1 amd64, arm64, ppc64le, s390x
registry.k8s.io/kubectl:v1.31.0-alpha.1 amd64, arm64, ppc64le, s390x

Changelog since v1.30.0

Urgent Upgrade Notes

(No, really, you MUST read this before you upgrade)

  • Kubelet flag --keep-terminated-pod-volumes was removed. This flag was deprecated in 2017. (#122082, @carlory) [SIG Apps, Node, Storage and Testing]

Changes by Kind

Deprecation

  • CephFS volume plugin ( kubernetes.io/cephfs) was removed in this release and the cephfs volume type became non-functional. Alternative is to use CephFS CSI driver (https://github.com/ceph/ceph-csi/) in your Kubernetes Cluster. A re-deployment of your application is required to use the new driver if you were using kubernetes.io/cephfs volume plugin before upgrading cluster version to 1.31+. (#124544, @carlory) [SIG Node, Scalability, Storage and Testing]

  • CephRBD volume plugin ( kubernetes.io/rbd) was removed in this release. And its csi migration support was also removed, so the rbd volume type became non-functional. Alternative is to use RBD CSI driver (https://github.com/ceph/ceph-csi/) in your Kubernetes Cluster. A re-deployment of your application is required to use the new driver if you were using kubernetes.io/rbd volume plugin before upgrading cluster version to 1.31+. (#124546, @carlory) [SIG Node, Scalability, Scheduling, Storage and Testing]

  • Kube-scheduler deprecated all non-csi volumelimit plugins and removed those from defaults plugins.

    • AzureDiskLimits
    • CinderLimits
    • EBSLimits
    • GCEPDLimits

    The NodeVolumeLimits plugin can handle the same functionality as the above plugins since the above volume types are migrated to CSI. Please remove those plugins and replace them with the NodeVolumeLimits plugin if you explicitly use those plugins in the scheduler config. Those plugins will be removed in the release 1.32. (#124500, @carlory) [SIG Scheduling and Storage]

  • Kubeadm: deprecated the kubeadm RootlessControlPlane feature gate (previously alpha), given that the core K8s UserNamespacesSupport feature gate graduated to Beta in 1.30. Once core Kubernetes support for user namespaces is generally available and kubeadm has started to support running the control plane in userns pods, the kubeadm RootlessControlPlane feature gate will be removed entirely. Until kubeadm supports the userns functionality out of the box, users can continue using the deprecated RootlessControlPlane feature gate, or opt-in UserNamespacesSupport by using kubeadm patches on the static pod manifests. (#124997, @neolit123) [SIG Cluster Lifecycle]

  • Kubeadm: mark the sub-phase of 'init kubelet-finilize' called 'experimental-cert-rotation' as deprecated and print a warning if it is used directly; it will be removed in a future release. Add a replacement sub-phase 'enable-client-cert-rotation'. (#124419, @neolit123) [SIG Cluster Lifecycle]

  • Remove k8s.io/legacy-cloud-providers from staging (#124767, @carlory) [SIG API Machinery, Cloud Provider and Release]

  • Removed legacy cloud provider integration code (undoing a previous reverted commit) (#124886, @carlory) [SIG Cloud Provider and Release]

API Change

  • Added the feature gates StrictCostEnforcementForVAP and StrictCostEnforcementForWebhooks to enforce the strct cost calculation for CEL extended libraries. It is strongly recommended to turn on the feature gates as early as possible. (#124675, @cici37) [SIG API Machinery, Auth, Node and Testing]

  • Component-base/logs: when compiled with Go >= 1.21, component-base will automatically configure the slog default logger together with initializing klog. (#120696, @pohly) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Storage and Testing]

  • DRA: client-side validation of a ResourceHandle would have accepted a missing DriverName, whereas server-side validation then would have raised an error. (#124075, @pohly) [SIG Apps]

  • Fix Deep Copy issue in getting controller reference (#124116, @HiranmoyChowdhury) [SIG API Machinery and Release]

  • Fix the comment for the Job's managedBy field (#124793, @mimowo) [SIG API Machinery and Apps]

  • Fixes a 1.30.0 regression in openapi descriptions of imagePullSecrets and hostAliases fields to mark the fields used as keys in those lists as either defaulted or required. (#124553, @pmalek) [SIG API Machinery]

  • Graduate MatchLabelKeys/MismatchLabelKeys feature in PodAffinity/PodAntiAffinity to Beta (#123638, @sanposhiho) [SIG API Machinery, Apps, Scheduling and Testing]

  • Graduated the DisableNodeKubeProxyVersion feature gate to beta. By default, the kubelet no longer attempts to set the .status.kubeProxyVersion field for its associated Node. (#123845, @HirazawaUi) [SIG API Machinery, Cloud Provider, Network, Node and Testing]

  • Improved scheduling performance when many nodes, and prefilter returns 1-2 nodes (e.g. daemonset)

    For developers of out-of-tree PostFilter plugins, note that the semantics of NodeToStatusMap are changing: A node with an absent value in the NodeToStatusMap should be interpreted as having an UnschedulableAndUnresolvable status (#125197, @gabesaba) [SIG Scheduling]

  • K8s.io/apimachinery/pkg/util/runtime: new calls support handling panics and errors in the context where they occur. PanicHandlers and ErrorHandlers now must accept a context parameter for that. Log output is structured instead of unstructured. (#121970, @pohly) [SIG API Machinery and Instrumentation]

  • Kube-apiserver: the --encryption-provider-config file is now loaded with strict deserialization, which fails if the config file contains duplicate or unknown fields. This protects against accidentally running with config files that are malformed, mis-indented, or have typos in field names, and getting unexpected behavior. When --encryption-provider-config-automatic-reload is used, new encryption config files that contain typos after the kube-apiserver is running are treated as invalid and the last valid config is used. (#124912, @enj) [SIG API Machinery and Auth]

  • Kube-controller-manager removes deprecated command flags: --volume-host-cidr-denylist and --volume-host-allow-local-loopback (#124017, @carlory) [SIG API Machinery, Apps, Cloud Provider and Storage]

  • Kube-controller-manager: the horizontal-pod-autoscaler-upscale-delay and horizontal-pod-autoscaler-downscale-delay flags have been removed (deprecated and non-functional since v1.12) (#124948, @SataQiu) [SIG API Machinery, Apps and Autoscaling]

  • Support fine-grained supplemental groups policy (KEP-3619), which enables fine-grained control for supplementary groups in the first container processes. You can choose whether to include groups defined in the container image(/etc/groups) for the container's primary uid or not. (#117842, @everpeace) [SIG API Machinery, Apps and Node]

  • The kube-proxy nodeportAddresses / --nodeport-addresses option now accepts the value "primary", meaning to only listen for NodePort connections on the node's primary IPv4 and/or IPv6 address (according to the Node object). This is strongly recommended, if you were not previously using --nodeport-addresses, to avoid surprising behavior.

    (This behavior is enabled by default with the nftables backend; you would need to explicitly request --nodeport-addresses 0.0.0.0/0,::/0 there to get the traditional "listen on all interfaces" behavior.) (#123105, @danwinship) [SIG API Machinery, Network and Windows]

Feature

  • Add --keep-* flags to kubectl debug, which enables to control the removal of probes, labels, annotations and initContainers from copy pod. (#123149, @mochizuki875) [SIG CLI and Testing]

  • Add apiserver.latency.k8s.io/apf-queue-wait annotation to the audit log to record the time spent waiting in apf queue (#123919, @hakuna-matatah) [SIG API Machinery]

  • Add the WatchList method to the rest client in client-go. When used, it establishes a stream to obtain a consistent snapshot of data from the server. This method is meant to be used by the generated client. (#122657, @p0lyn0mial) [SIG API Machinery]

  • Added cri-client staging repository. (#123797, @saschagrunert) [SIG API Machinery, Node, Release and Testing]

  • Added flag to kubectl logs called --all-pods to get all pods from a object that uses a pod selector. (#124732, @cmwylie19) [SIG CLI and Testing]

  • Added ports autocompletion for kubectl port-foward command (#124683, @TessaIO) [SIG CLI]

  • Added support for building Windows kube-proxy container image. A container image for kube-proxy on Windows can now be built with the command make release-images KUBE_BUILD_WINDOWS=y. The Windows kube-proxy image can be used with Windows Host Process Containers. (#109939, @claudiubelu) [SIG Windows]

  • Adds completion for kubectl set image. (#124592, @ah8ad3) [SIG CLI]

  • Allow creating ServiceAccount tokens bound to Node objects. This allows users to bind a service account token's validity to a named Node object, similar to Pod bound tokens. Use with kubectl create token <serviceaccount-name> --bound-object-kind=Node --bound-object-node=<node-name>. (#125238, @munnerz) [SIG Auth and CLI]

  • CEL default compatibility environment version to updated to 1.30 so that the extended libraries added before 1.30 is available to use. (#124779, @cici37) [SIG API Machinery]

  • CEL expressions and additionalProperties are now allowed to be used under nested quantifiers in CRD schemas (#124381, @alexzielenski) [SIG API Machinery]

  • CEL: add name formats library (#123572, @alexzielenski) [SIG API Machinery]

  • Checking etcd version to warn about deprecated etcd versions if ConsistentListFromCache is enabled. (#124612, @ah8ad3) [SIG API Machinery]

  • Client-go/reflector: warns when the bookmark event for initial events hasn't been received (#124614, @p0lyn0mial) [SIG API Machinery]

  • Custom resource field selectors are now in beta and enabled by default. Check out https://github.com/kubernetes/enhancements/issues/4358 for more details. (#124681, @jpbetz) [SIG API Machinery, Auth and Testing]

  • Dependencies: start using registry.k8s.io/pause:3.10 (#125112, @neolit123) [SIG CLI, Cloud Provider, Cluster Lifecycle, Node, Release, Testing and Windows]

  • Graduated support for CDI device IDs to general availability. The DevicePluginCDIDevices feature gate is now enabled unconditionally. (#123315, @bart0sh) [SIG Node]

  • Kube-apiserver: http/2 serving can be disabled with a --disable-http2-serving flag (#122176, @slashpai) [SIG API Machinery]

  • Kube-proxy's nftables mode (--proxy-mode=nftables) is now beta and available by default.

    FIXME ADD MORE HERE BEFORE THE RELEASE, DOCS LINKS AND STUFF (#124383, @danwinship) [SIG Cloud Provider and Network]

  • Kube-scheduler implements scheduling hints for the CSILimit plugin. The scheduling hints allow the scheduler to retry scheduling a Pod that was previously rejected by the CSILimit plugin if a deleted pod has a PVC from the same driver. (#121508, @utam0k) [SIG Scheduling and Storage]

  • Kube-scheduler implements scheduling hints for the InterPodAffinity plugin. The scheduling hints allow the scheduler to retry scheduling a Pod that was previously rejected by the InterPodAffinity plugin if create/delete/update a related Pod or a node which matches the pod affinity. (#122471, @nayihz) [SIG Scheduling and Testing]

  • Kubeadm: during "upgrade" , if the "etcd.yaml" static pod does not need upgrade, still consider rotating the etcd certificates and restarting the etcd static pod if the "kube-apiserver.yaml" manifest is to be upgraded and if certificate renewal is not disabled. (#124688, @neolit123) [SIG Cluster Lifecycle]

  • Kubeadm: enhance the "patches" functionality to be able to patch coredns deployment. The new patch target is called "corednsdeployment" (e.g. patch file "corednsdeployment+json.json"). This makes it possible to apply custom patches to coredns deployment during "init" and "upgrade". (#124820, @SataQiu) [SIG Cluster Lifecycle]

  • Kubeadm: mark the flag "--experimental-output' as deprecated (it will be removed in a future release) and add a new flag '--output" that serves the same purpose. Affected commands are - "kubeadm config images list", "kubeadm token list", "kubeadm upgade plan", "kubeadm certs check-expiration". (#124393, @carlory) [SIG Cluster Lifecycle]

  • Kubeadm: switch to using the new etcd endpoints introduced in 3.5.11 - /livez (for liveness probe) and /readyz (for readyness and startup probe). With this change it is no longer possible to deploy a custom etcd version older than 3.5.11 with kubeadm 1.31. If so, please upgrade. (#124465, @neolit123) [SIG Cluster Lifecycle]

  • Kubeadm: switched kubeadm to start using the CRI client library instead of shelling out of the crictl binary for actions against a CRI endpoint. The kubeadm deb/rpm packages will continue to install the cri-tools package for one more release, but in you must adapt your scripts to install crictl manually from https://github.com/kubernetes-sigs/cri-tools/releases or a different location.

    The kubeadm package will stop depending on the cri-tools package in Kubernetes 1.32, which means that installing kubeadm will no longer automatically ensure installation of crictl. (#124685, @saschagrunert) [SIG Cluster Lifecycle]

  • Kubeadm: use output/v1alpha3 to print structural output for the commands "kubeadm config images list" and "kubeadm token list". (#124464, @carlory) [SIG Cluster Lifecycle]

  • Kubelet server can now dynamically load certificate files (#124574, @zhangweikop) [SIG Auth and Node]

  • Kubelet will not restart the container when fields other than image in the pod spec change. (#124220, @HirazawaUi) [SIG Node]

  • Kubemark: adds two flags, --kube-api-qps and --kube-api-burst (#124147, @devincd) [SIG Scalability]

  • Kubernetes is now built with go 1.22.3 (#124828, @cpanato) [SIG Release and Testing]

  • Kubernetes is now built with go 1.22.4 (#125363, @cpanato) [SIG Architecture, Cloud Provider, Release, Storage and Testing]

  • Pause: add a -v flag to the Windows variant of the pause binary, which prints the version of pause and exits. The Linux pause already has the flag. (#125067, @neolit123) [SIG Windows]

  • Promoted generateName retries to beta, and made the NameGenerationRetries feature gate enabled by default. You can read https://kep.k8s.io/4420 for more details. (#124673, @jpbetz) [SIG API Machinery]

  • Scheduler changes its logic of calculating evaluatedNodes from "contains the number of nodes that filtered out by PreFilterResult and Filter plugins" to "the number of nodes filtered out by Filter plugins only". (#124735, @AxeZhan) [SIG Scheduling]

  • Services implement a field selector for the ClusterIP and Type fields. Kubelet uses the fieldselector on Services to avoid watching for Headless Services and reduce the memory consumption. (#123905, @aojea) [SIG Apps, Node and Testing]

  • The iptables mode of kube-proxy now tracks accepted packets that are destined for node-ports on localhost by introducing kubeproxy_iptables_localhost_nodeports_accepted_packets_total metric. This will help users to identify if they rely on iptables.localhostNodePorts feature and ulitmately help them to migrate from iptables to nftables. (#125015, @aroradaman) [SIG Instrumentation, Network and Testing]

  • The iptables mode of kube-proxy now tracks packets that are wrongfully marked invalid by conntrack and subsequently dropped by introducing kubeproxy_iptables_ct_state_invalid_dropped_packets_total metric (#122812, @aroradaman) [SIG Instrumentation, Network and Testing]

  • The name of CEL optional type has been changed from optional to optional_type. (#124328, @jiahuif) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Network and Node]

  • The scheduler implements QueueingHint in TaintToleration plugin, which enhances the throughput of scheduling. (#124287, @sanposhiho) [SIG Scheduling and Testing]

  • The sidecar finish time will be accounted when calculating the job's finish time. (#124942, @AxeZhan) [SIG Apps]

  • This PR adds tracing support to the kubelet's read-only endpoint, which currently does not have tracing. It makes use the WithPublicEndpoint option to prevent callers from influencing sampling decisions. (#121770, @frzifus) [SIG Node]

  • Users can traverse all the pods that are in the scheduler and waiting in the permit stage through method IterateOverWaitingPods. In other words, all waitingPods in scheduler can be obtained from any profiles. Before this commit, each profile could only obtain waitingPods within that profile. (#124926, @kerthcet) [SIG Scheduling]

Failing Test

Bug or Regression

  • .status.terminating field now gets correctly tracked for deleted active Pods when a Job fails. (#125175, @dejanzele) [SIG Apps and Testing]
  • Added an extra line between two different key value pairs under data when running kubectl describe configmap (#123597, @siddhantvirus) [SIG CLI]
  • Allow parameter to be set along with proto file path (#124281, @fulviodenza) [SIG API Machinery]
  • Cel: converting a quantity value into a quantity value failed. (#123669, @pohly) [SIG API Machinery]
  • Client-go/tools/record.Broadcaster: fixed automatic shutdown on WithContext cancellation (#124635, @pohly) [SIG API Machinery]
  • Do not remove the "batch.kubernetes.io/job-tracking" finalizer from a Pod, in a corner case scenario, when the Pod is controlled by an API object which is not a batch Job (e.g. when the Pod is controlled by a custom CRD). (#124798, @mimowo) [SIG Apps and Testing]
  • Drop additional rule requirement (cronjobs/finalizers) in the roles who use kubectl create cronjobs to be backwards compatible (#124883, @ardaguclu) [SIG CLI]
  • Emition of RecreatingFailedPod and RecreatingTerminatedPod events has been removed from stateful set lifecycle. (#123809, @atiratree) [SIG Apps and Testing]
  • Endpointslices mirrored from Endpoints by the EndpointSliceMirroring controller were not reconciled if modified (#124131, @zyjhtangtang) [SIG Apps and Network]
  • Ensure daemonset controller to count old unhealthy pods towards max unavailable budget (#123233, @marshallbrekka) [SIG Apps]
  • Fix "-kube-test-repo-list" e2e flag may not take effect (#123587, @huww98) [SIG API Machinery, Apps, Autoscaling, CLI, Network, Node, Scheduling, Storage, Testing and Windows]
  • Fix a race condition in kube-controller-manager and scheduler caused by a bug in transforming informer happening when objects were accessed during Resync operation by making the transforming function idempotent. (#124352, @wojtek-t) [SIG API Machinery and Scheduling]
  • Fix a race condition in transforming informer happening when objects were accessed during Resync operation (#124344, @wojtek-t) [SIG API Machinery]
  • Fix kubelet on Windows fails if a pod has SecurityContext with RunAsUser (#125040, @carlory) [SIG Storage, Testing and Windows]
  • Fix throughput when scheduling daemonset pods to reach 300 pods/s, if the configured qps allows it. (#124714, @sanposhiho) [SIG Scheduling]
  • Fix: the resourceclaim controller forgot to wait for podSchedulingSynced and templatesSynced (#124589, @carlory) [SIG Apps and Node]
  • Fixed EDITOR/KUBE_EDITOR with double-quoted paths with spaces when on Windows cmd.exe. (#112104, @oldium) [SIG CLI and Windows]
  • Fixed a bug in the JSON frame reader that could cause it to retain a reference to the underlying array of the byte slice passed to Read. (#123620, @benluddy) [SIG API Machinery]
  • Fixed a bug in the scheduler where it would crash when prefilter returns a non-existent node. (#124933, @AxeZhan) [SIG Scheduling and Testing]
  • Fixed a bug where kubectl describe incorrectly displayed NetworkPolicy port ranges (showing only the starting port). (#123316, @jcaamano) [SIG CLI]
  • Fixed a regression where kubelet --hostname-override no longer worked correctly with an external cloud provider. (#124516, @danwinship) [SIG Node]
  • Fixed an issue that prevents the linking of trace spans for requests that are proxied through kube-aggregator. (#124189, @toddtreece) [SIG API Machinery]
  • Fixed bug where kubectl get with --sort-by flag does not sort strings alphanumerically. (#124514, @brianpursley) [SIG CLI]
  • Fixed the format of the error indicating that a user does not have permission on the object referenced by paramRef in ValidatingAdmissionPolicyBinding. (#124653, @m1kola) [SIG API Machinery]
  • Fixes a bug where hard evictions due to resource pressure would let the pod have the full termination grace period, instead of shutting down instantly. This bug also affected force deleted pods. Both cases now get a termination grace period of 1 second. (#124063, @olyazavr) [SIG Node]
  • Fixes a missing status. prefix on custom resource validation error messages. (#123822, @JoelSpeed) [SIG API Machinery]
  • Improved scheduling latency when many gated pods (#124618, @gabesaba) [SIG Scheduling and Testing]
  • Job: Fix a bug that the SuccessCriteriaMet could be added to the Job with successPolicy regardless of the featureGate enabling (#125429, @tenzen-y) [SIG Apps]
  • Kube-apiserver: fixes a 1.28 regression printing pods with invalid initContainer status (#124906, @liggitt) [SIG Node]
  • Kubeadm: allow 'kubeadm init phase certs sa' to accept the '--config' flag. (#125396, @Kavinraja-G) [SIG Cluster Lifecycle]
  • Kubeadm: don't mount /etc/pki in kube-apisever and kube-controller-manager pods as an additional Linux system CA location. Mount /etc/pki/ca-trust and /etc/pki/tls/certs instead. /etc/ca-certificate, /usr/share/ca-certificates, /usr/local/share/ca-certificates and /etc/ssl/certs continue to be mounted. (#124361, @neolit123) [SIG Cluster Lifecycle]
  • Kubeadm: during kubelet health checks, respect the healthz address:port configured in the KubeletConfiguration instead of hardcoding localhost:10248. (#125265, @neolit123) [SIG Cluster Lifecycle]
  • Kubeadm: during the preflight check "CreateJob" of "kubeadm upgrade", check if there are no nodes where a Pod can schedule. If there are none, show a warning and skip this preflight check. This can happen in single node clusters where the only node was drained. (#124503, @neolit123) [SIG Cluster Lifecycle]
  • Kubeadm: fix a regression where the KubeletConfiguration is not properly downloaded during "kubeadm upgrade" commands from the kube-system/kubelet-config ConfigMap, resulting in the local '/var/lib/kubelet/config.yaml' file being written as a defaulted config. (#124480, @neolit123) [SIG Cluster Lifecycle]
  • Kubeadm: fixed a bug where the PublicKeysECDSA feature gate was not respected when generating kubeconfig files. (#125388, @neolit123) [SIG Cluster Lifecycle]
  • Kubeadm: improve the "IsPriviledgedUser" preflight check to not fail on certain Windows setups. (#124665, @neolit123) [SIG Cluster Lifecycle]
  • Kubeadm: stop storing the ResolverConfig in the global KubeletConfiguration and instead set it dynamically for each node (#124038, @SataQiu) [SIG Cluster Lifecycle]
  • Kubectl support both:
    • kubectl create secret docker-registry --from-file=<path/to/.docker/config.json>
    • kubectl create secret docker-registry --from-file=.dockerconfigjson=<path/to/.docker/config.json> (#119589, @carlory) [SIG CLI]
  • Kubectl: Show the Pod phase in the STATUS column as 'Failed' or 'Succeeded' when the Pod is terminated (#122038, @lowang-bh) [SIG CLI]
  • Kubelet no longer crashes when a DRA driver returns a nil as part of the Node(Un)PrepareResources response instead of an empty struct (did not affect drivers written in Go, first showed up with a driver written in Rust). (#124091, @bitoku) [SIG Node]
  • Make kubectl find kubectl-create-subcommand plugins also when positional arguments exists, e.g. kubectl create subcommand arg. (#124123, @sttts) [SIG CLI]
  • Removed admission plugin PersistentVolumeLabel. Please use https://github.com/kubernetes-sigs/cloud-pv-admission-labeler instead if you need a similar functionality. (#124505, @jsafrane) [SIG API Machinery, Auth and Storage]
  • StatefulSet autodelete will respect controlling owners on PVC claims as described in https://github.com/kubernetes/enhancements/pull/4375 (#122499, @mattcary) [SIG Apps and Testing]
  • The "fake" clients generated by client-gen now have the same semantics on error as the real clients; in particular, a failed Get(), Create(), etc, no longer returns nil. (It now returns a pointer to a zero-valued object, like the real clients do.) This will break some downstream unit tests that were testing result == nil rather than err != nil, and in some cases may expose bugs in the underlying code that were hidden by the incorrect unit tests. (#122892, @danwinship) [SIG API Machinery, Auth, Cloud Provider, Instrumentation and Storage]
  • The Service LoadBalancer controller was not correctly considering the service.Status new IPMode field and excluding the Ports when comparing if the status has changed, causing that changes in these fields may not update the service.Status correctly (#125225, @aojea) [SIG Apps, Cloud Provider and Network]
  • The nftables kube-proxy mode now has its own metrics rather than reporting metrics with "iptables" in their names. (#124557, @danwinship) [SIG Network and Windows]
  • Updated description of default values for --healthz-bind-address and --metrics-bind-address parameters (#123545, @yangjunmyfm192085) [SIG Network]

Other (Cleanup or Flake)

  • ACTION-REQUIRED: DRA drivers using the v1alpha2 kubelet gRPC API are no longer supported and need to be updated. (#124316, @pohly) [SIG Node and Testing]
  • Build etcd image v3.5.13 (#124026, @liangyuanpeng) [SIG API Machinery and Etcd]
  • Build etcd image v3.5.14 (#125235, @humblec) [SIG API Machinery]
  • CSI spec support has been lifted to v1.9.0 in this release (#125150, @humblec) [SIG Storage and Testing]
  • E2e.test and e2e_node.test: tests which depend on alpha or beta feature gates now have Feature:Alpha or Feature:Beta as Ginkgo labels. The inline text is [Alpha] or [Beta], as before. (#124350, @pohly) [SIG Testing]
  • Etcd: Update to v3.5.13 (#124027, @liangyuanpeng) [SIG API Machinery, Cloud Provider, Cluster Lifecycle, Etcd and Testing]
  • Expose apiserver_watch_cache_resource_version metric to simplify debugging problems with watchcache. (#125377, @wojtek-t) [SIG API Machinery and Instrumentation]
  • Fixed a typo in the help text for the pod_scheduling_sli_duration_seconds metric in kube-scheduler (#124221, @arturhoo) [SIG Instrumentation, Scheduling and Testing]
  • Job-controller: the JobReadyPods feature flag has been removed (deprecated since v1.31) (#125168, @kaisoz) [SIG Apps]
  • Kubeadm: improve the warning message about the NodeSwap check which kubeadm performs on preflight. (#125157, @carlory) [SIG Cluster Lifecycle]
  • Kubeadm: only enable the klog flags that are still supported for kubeadm, rather than hiding the unwanted flags. This means that the previously unrecommended hidden flags about klog (including --alsologtostderr, --log-backtrace-at, --log-dir, --logtostderr, --log-file, --log-file-max-size, --one-output, --skip-log-headers, --stderrthreshold and --vmodule) are no longer allowed to be used. (#125179, @SataQiu) [SIG Cluster Lifecycle]
  • Kubeadm: remove the EXPERIMENTAL tag from the phase "kubeadm join control-plane-prepare download-certs". (#124374, @neolit123) [SIG Cluster Lifecycle]
  • Kubeadm: remove the deprecated and NO-OP "kubeadm join control-plane-join update-status" phase. (#124373, @neolit123) [SIG Cluster Lifecycle]
  • Kubeadm: removed the deprecated output.kubeadm.k8s.io/v1alpha2 API for structured output. Please use v1alpha3 instead. (#124496, @carlory) [SIG Cluster Lifecycle]
  • Kubeadm: the deprecated UpgradeAddonsBeforeControlPlane featuregate has been removed, upgrade of the CoreDNS and kube-proxy addons will not be triggered until all the control plane instances have been upgraded. (#124715, @SataQiu) [SIG Cluster Lifecycle]
  • Kubeadm: the global --rootfs flag is now considered non-experimental. (#124375, @neolit123) [SIG Cluster Lifecycle]
  • Kubectl describe service and ingress will now use endpointslices instead of endpoints (#124598, @aroradaman) [SIG CLI and Network]
  • Kubelet flags --iptables-masquerade-bit and --iptables-drop-bit were deprecated in v1.28 and have now been removed entirely. (#122363, @carlory) [SIG Network and Node]
  • Migrated the pkg/proxy to use contextual logging. (#122979, @fatsheep9146) [SIG Network and Scalability]
  • Moved remote CRI implementation from kubelet to k8s.io/cri-client repository. (#124634, @saschagrunert) [SIG Node, Release and Testing]
  • Remove GA ServiceNodePortStaticSubrange feature gate (#124738, @xuzhenglun) [SIG Network]
  • Removed generally available feature gate CSINodeExpandSecret. (#124462, @carlory) [SIG Storage]
  • Removed generally available feature gate ConsistentHTTPGetHandlers. (#124463, @carlory) [SIG Node]
  • Removes ENABLE_CLIENT_GO_WATCH_LIST_ALPHA environmental variable from the reflector. To activate the feature set KUBE_FEATURE_WatchListClient environmental variable or a corresponding command line option (this works only binaries that explicitly expose it). (#122791, @p0lyn0mial) [SIG API Machinery and Testing]
  • Removing the last remaining in-tree gcp cloud provider and credential provider. Please use the external cloud provider and credential provider from https://github.com/kubernetes/cloud-provider-gcp instead. (#124519, @dims) [SIG API Machinery, Apps, Auth, Autoscaling, Cloud Provider, Instrumentation, Network, Node, Scheduling, Storage and Testing]
  • Scheduler framework: PreBind implementations are now allowed to return Pending and Unschedulable status codes. (#125360, @pohly) [SIG Scheduling]
  • The feature gate "DefaultHostNetworkHostPortsInPodTemplates" has been removed. This behavior was deprecated in v1.28, and has had no reports of trouble since. (#124417, @thockin) [SIG Apps]
  • The feature gate "SkipReadOnlyValidationGCE" has been removed. This gate has been active for 2 releases with no reports of issues (and was such a niche thing, we didn't expect any). (#124210, @thockin) [SIG Apps]
  • The kube-scheduler exposes /livez and /readz for health checks that are in compliance with https://kubernetes.io/docs/reference/using-api/health-checks/#api-endpoints-for-health (#118148, @linxiulei) [SIG API Machinery, Scheduling and Testing]
  • The kubelet is no longer able to recover from device manager state file older than 1.20. If the proper recommended upgrade flow is followed, there should be no issue. (#123398, @ffromani) [SIG Node and Testing]
  • Update CNI Plugins to v1.5.0 (#125113, @bzsuni) [SIG Cloud Provider, Network, Node and Testing]
  • Updated cni-plugins to v1.4.1. (#123894, @saschagrunert) [SIG Cloud Provider, Node and Testing]
  • Updated cri-tools to v1.30.0. (#124364, @saschagrunert) [SIG Cloud Provider, Node and Release]

Dependencies

Added

Changed

Removed


Contributors, the CHANGELOG-1.31.md has been bootstrapped with v1.31.0-alpha.1 release notes and you may edit now as needed.



Published by your Kubernetes Release Managers. 

-- 
Adolfo García Veytia
uServers Comunicaciones
<adolfo...@uservers.net>
Reply all
Reply to author
Forward
0 new messages