Kubernetes v1.33.0 is live!

124 views
Skip to first unread message

Matteo Bianchi

unread,
Apr 23, 2025, 1:51:25 PMApr 23
to d...@kubernetes.io
Kubernetes Community,

Kubernetes v1.33.0 has been built and pushed using Golang version 1.24.2.

The release notes have been updated in CHANGELOG-1.33.md, with a pointer to them on GitHub:

v1.33.0

Documentation

Downloads for v1.33.0

Source Code

filenamesha512 hash
kubernetes.tar.gzd325cf208bec566b03ce9a3e56972f430243b46cad086ef9094d7e89e7ebab22e4e7869ad87c8bcb95370c4bcc6d43ca0fdff20c7f668c7db31122af6ef5fcb5
kubernetes-src.tar.gz0460b3327ef3ede807924e63da19ee78608c0ed1eebe80b9f4f201d26e1e1072d2902b4648db3d289069d0ad7707d4b37362eaf6a45e1f8c3687185ca8e83884

Client Binaries

filenamesha512 hash
kubernetes-client-darwin-amd64.tar.gza12e25581fd3716aa0db3ce5524ba7ae9a6e0606b92454c6c12c9b32b2900d17db2a85355c6f6d9bf6fa32ec1a1466df9501e5ab3510f5d8ae4193aafa0ba8f8
kubernetes-client-darwin-arm64.tar.gz7faacc4eda215101b8497c598e2e5ee8cd7889013b5888f17bc933f7785484e880a47c9e46504783cf503068f3462b21eecfa8a30a0f53c4a671633f528d0fa6
kubernetes-client-linux-386.tar.gz09e64479bfe760718685b0dddc060ee34e3efce029b1374254ffa09717148300692ee12e265fd1622746794d91aa7d407f258cab14905437c15e9876b47a24c5
kubernetes-client-linux-amd64.tar.gz23031beed988f77fa759d03c81f6e66ad39666e08ae56f1d8120c95b834dd06cb9d0d8aafc99152c8e4e880c000d613a0a560e985e81751cae91b445001096dd
kubernetes-client-linux-arm.tar.gz4ce625f861eab1f98c6fb39b93a1a9a50e669f31f65d713344aa36f8d00012cbb35a4d85ed9a15deffc42329e32d32b8b469f8f801e0232d9de50c768bbd058e
kubernetes-client-linux-arm64.tar.gzba722521450771a326103bffc6095496620f67d2eceda233d006b02209277818a5a960903b0902ffaa055a6700b43505010066008e858a8197f8eeaf156fc814
kubernetes-client-linux-ppc64le.tar.gz26ebdc9f21ea90177c8503606373ca7cd62dc034c3c1886f8a9c4fe3822d70e53e51088cbddf09922fc81d4670af67e9c7d1cea920ed9d536f460cc8451c02f0
kubernetes-client-linux-s390x.tar.gzba44c74096ec228362c37a47388e612736021c7d8a0c26b21af6c4970b2c2b4b6abd20561775a2425965ad158599fd7605da6a9ef1ec851fb5b53554be180977
kubernetes-client-windows-386.tar.gz74a065c301e18cf9a403e7f6976310d2d6cd99406194ad5f92bb270d2f2aadf8a8a3d0ac66a4528d4f43183ad43baf07dedbecca448293c3fa91f2c888af5118
kubernetes-client-windows-amd64.tar.gz89b3447b137780de65da653b6724ec7ccf9cdffe9e6b228d87f2b58060e51c15fb83f7b7ae6b70d3dbdbe7164d71f70650a81f37e47bad3c980a02092003aa32
kubernetes-client-windows-arm64.tar.gzb9cbfa357d48388aaff2565a85ad094e4b9642894b2fe2c565b9bb093ca007116b883463aa378ca8ac5993c1d5c4a581b9d8fe1ad4c4098fcf3c807c0bc67e32

Server Binaries

filenamesha512 hash
kubernetes-server-linux-amd64.tar.gz487aea4b3e1066b4d7644b44195e8ca0d55bde4807d5c96d6fc020661b14cf356aebe1e3fd7c1f841ba1b5a0be9da097dfaf117f05b821f75dd0aa29cd99fb70
kubernetes-server-linux-arm64.tar.gz7ebebcb44435a18050beefbde7c6d2d36d86fee8908514b3f3e0925a93e0791193613c7b19f2a359b2330f0cb62ca39e1bfd9628ae6b9d713c5dcd21857ae845
kubernetes-server-linux-ppc64le.tar.gz07a93cac90368ed216caaf1ea3885051b2ec1843de90fea5464cc8f666aecc11519fad32a83b7989f8fd3d6fe3862060a23859398a3287c2f782c03dd134f4d8
kubernetes-server-linux-s390x.tar.gzad3b3ad780f62944d0d6778461f0e8b81ae66391fa8eb666bac05cff95b22dd669ddd1917045240c54070313b1f6d81ed1868df084f6b4f46e8b1b49b5c0ae67

Node Binaries

filenamesha512 hash
kubernetes-node-linux-amd64.tar.gz053b44d2fbf7e71d2bf4766448bfe755775bc33ab26f56e2b5a4c3d07981d75fc45d8c5f6ae6f4508fb5aff803000709c9ac8e9d7a5797d37b34be24c2a1975e
kubernetes-node-linux-arm64.tar.gzb367dabfd6697479c1e50f977898f479210588855202f0ea6e2f29ad435a9174e88c387e21e2495af8fa412faf5ac858706bbb88f20217d93b1e529fdc57c5d6
kubernetes-node-linux-ppc64le.tar.gz99a907d19183e9e50a6043acfc2fbf239a6ecf39707831fe563dda3cbadca3b9d11a6bbfcb9050f725713b7a9679421958a2e52ec549f823dd40fdaef34f6d02
kubernetes-node-linux-s390x.tar.gz52f802417f4ced7e82c3e24b54e9315ced590a8c9fdee63efb7820734fa6216551cf2683c907b3c211b5e19fe978f33ef1d6f85d58c10008930375fcb5f08231
kubernetes-node-windows-amd64.tar.gz61ef82babea9d7f3f19dcc208dd692f65cdfc3cfd01d3e5c6c35897c6e2a1ae05952162f5e9dba08d87a49abdc27d102392619c5902238ef16fd44d44fbf5c9f

Container Images

All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name.

namearchitectures
registry.k8s.io/conformance:v1.33.0amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-apiserver:v1.33.0amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-controller-manager:v1.33.0amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-proxy:v1.33.0amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-scheduler:v1.33.0amd64, arm64, ppc64le, s390x
registry.k8s.io/kubectl:v1.33.0amd64, arm64, ppc64le, s390x

Changelog since v1.32.0

Urgent Upgrade Notes

(No, really, you MUST read this before you upgrade)

  • Added the ability to reduce both the initial delay and the maximum delay accrued between container restarts for a node for containers in CrashLoopBackOff across the cluster to the recommended values of 1s initial delay and 60s maximum delay. To set this for a node, turn on the feature gate ReduceDefaultCrashLoopBackOffDecay. If you are also using the feature gate KubeletCrashLoopBackOffMax with a configured per-node CrashLoopBackOff.MaxContainerRestartPeriod, the effective kubelet configuration will follow the conflict resolution policy described further in the documentation here. (#130711, @lauralorenz) [SIG Node and Testing]
  • [Action Required] CSI drivers that call IsLikelyNotMountPoint should not assume false means that the path is a mount point. Each CSI driver needs to make sure correct usage of return value of IsLikelyNotMountPoint because if the file is an irregular file but not a mount point is acceptable (#129370, @andyzhangx) [SIG Storage and Windows]
  • Fixed the behavior of the KUBE_PROXY_NFTABLES_SKIP_KERNEL_VERSION_CHECK environment variable in the nftables proxier. The kernel version check is now skipped only when this variable is explicitly set to a non-empty value. To skip the check, set the KUBE_PROXY_NFTABLES_SKIP_KERNEL_VERSION_CHECK environment variable. (#130401, @ryota-sakamoto)
  • Renamed UpdatePodTolerations action type to UpdatePodToleration. Action required for custom plugin developers to update their code to follow the rename. (#129023, @zhifei92) [SIG Scheduling and Testing]

Changes by Kind

Deprecation

  • The EndpointSlice hints field has graduated to GA. The beta annotation service.kubernetes.io/topology-mode is now considered deprecated and will not graduate to GA. It remains operational for backward compatibility. Users are encouraged to use the spec.trafficDistribution field in the Service API for topology-aware routing configuration. (#130742, @gauravkghildiyal) [SIG Network]
  • The StorageCapacityScoring feature gate was added to score nodes by available storage capacity. It's in alpha and disabled by default. The VolumeCapacityPriority alpha feature was replaced with this, and the default behavior was changed. The VolumeCapacityPriority preferred a node with the least allocatable, but the StorageCapacityScoring preferred a node with the maximum allocatable. See KEP-4049 for details. (#128184, @cupnes) [SIG Scheduling, Storage and Testing]
  • The WatchFromStorageWithoutResourceVersion feature was deprecated and can no longer be enabled. (#129930, @serathius)
  • The pod status.resize field is now deprecated and will no longer be set. The status of a pod resize will be exposed under two new conditions: PodResizeInProgress and PodResizePending instead. (#130733, @natasha41575) [SIG API Machinery, Apps, CLI, Node, Scheduling and Testing]
  • The v1 Endpoints API is now officially deprecated (though still fully supported). The API will not be removed, but all users should use the EndpointSlice API instead. (#130098, @danwinship) [SIG API Machinery and Network]

API Change

  • A new alpha feature gate, MutableCSINodeAllocatableCount, has been introduced.

    When this feature gate is enabled, the CSINode.Spec.Drivers[*].Allocatable.Count field becomes mutable, and a new field, NodeAllocatableUpdatePeriodSeconds, is available in the CSIDriver object. This allows periodic updates to a node's reported allocatable volume capacity, preventing stateful pods from becoming stuck due to outdated information that kube-scheduler relies on. (#130007, @torredil) [SIG Apps, Node, Scheduling and Storage]

  • Added feature gate DRAPartitionableDevices, when enabled, Dynamic Resource Allocation support partitionable devices allocation. (#130764, @cici37) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Scheduling, Storage and Testing]

  • Added DRA support for a "one-of" prioritized list of selection criteria to satisfy a device request in a resource claim. (#128586, @mortent) [SIG API Machinery, Apps, Etcd, Node, Scheduling and Testing]

  • Added a /flagz endpoint for kubelet endpoint (#128857, @zhifei92) [SIG Architecture, Instrumentation and Node]

  • Added a new tolerance field to HorizontalPodAutoscaler, overriding the cluster-wide default. Enabled via the HPAConfigurableTolerance alpha feature gate. (#130797, @jm-franc) [SIG API Machinery, Apps, Autoscaling, Etcd, Node, Scheduling and Testing]

  • Added support for configuring custom stop signals with a new StopSignal container lifecycle (#130556, @sreeram-venkitesh) [SIG API Machinery, Apps, Node and Testing]

  • Added support for in-place vertical scaling of Pods with sidecars (containers defined within initContainers where the restartPolicy is set to Always). (#128367, @vivzbansal) [SIG API Machinery, Apps, CLI, Node, Scheduling and Testing]

  • CPUManager Policy Options support is GA (#130535, @ffromani) [SIG API Machinery, Node and Testing]

  • Changed the Pod API to support hugepage resources at spec level for pod-level resources. (#130577, @KevinTMtz) [SIG Apps, CLI, Node, Scheduling, Storage and Testing]

  • DRA API: The maximum number of pods that can use the same ResourceClaim is now 256 instead of 32. Downgrading a cluster where this relaxed limit is in use to Kubernetes 1.32.0 is not supported, as version 1.32.0 would refuse to update ResourceClaims with more than 32 entries in the status.reservedFor field. (#129543, @pohly) [SIG API Machinery, Node and Testing]

  • DRA: CEL expressions using attribute strings exceeded the cost limit because their cost estimation was incomplete. (#129661, @pohly) [SIG Node]

  • DRA: Device taints enable DRA drivers or admins to mark device as unusable, which prevents allocating them. Pods may also get evicted at runtime if a device becomes unusable, depending on the severity of the taint and whether the claim tolerates the taint. (#130447, @pohly) [SIG API Machinery, Apps, Architecture, Auth, Etcd, Instrumentation, Node, Scheduling and Testing]

  • DRA: Starting Kubernetes 1.33, only users with access to an admin namespace with the kubernetes.io/dra-admin-access label are authorized to create ResourceClaim or ResourceClaimTemplate objects with the adminAccess field in this admin namespace if they want to and only they can reference these ResourceClaims or ResourceClaimTemplates in their pod or deployment specs. (#130225, @ritazh) [SIG API Machinery, Apps, Auth, Node and Testing]

  • DRA: when asking for "All" devices on a node, Kubernetes <= 1.32 proceeded to schedule pods onto nodes with no devices by not allocating any devices for those pods. Kubernetes 1.33 changes that to only picking nodes which have at least one device. Users who want the "proceed with scheduling also without devices" semantic can use the upcoming prioritized list feature with one sub-request for "all" devices and a second alternative with "count: 0". (#129560, @bart0sh) [SIG API Machinery and Node]

  • Expanded the on-disk kubelet credential provider configuration to allow an optional tokenAttribute field to be configured. When it is set, the kubelet will provision a token with the given audience bound to the current pod and its service account. This KSA token along with required annotations on the KSA defined in configuration will be sent to the credential provider plugin via its standard input (along with the image information that is already sent today). The KSA annotations to be sent are configurable in the kubelet credential provider configuration. (#128372, @aramase) [SIG API Machinery, Auth, Node and Testing]

  • Fixed the example validation rule in godoc:

    When configuring a JWT authenticator:

    If username.expression uses 'claims.email', then 'claims.email_verified' must be used in username.expression or extra[].valueExpression or claimValidationRules[].expression. An example claim validation rule expression that matches the validation automatically applied when username.claim is set to 'email' is 'claims.?email_verified.orValue(true) == true'. By explicitly comparing the value to true, we let type-checking see the result will be a boolean, and to make sure a non-boolean email_verified claim will be caught at runtime. (#130875, @aramase) [SIG Auth and Release]

  • For the InPlacePodVerticalScaling feature, the API server will no longer set the resize status to Proposed upon receiving a resize request. (#130574, @natasha41575) [SIG Apps, Node and Testing]

  • Graduate the MatchLabelKeys (MismatchLabelKeys) feature in PodAffinity (PodAntiAffinity) to GA (#130463, @sanposhiho) [SIG API Machinery, Apps, Node, Scheduling and Testing]

  • Graduated image volume sources to beta:

    • Allowed subPath/subPathExpr for image volumes
    • Added kubelet metrics kubelet_image_volume_requested_total, kubelet_image_volume_mounted_succeed_total and kubelet_image_volume_mounted_errors_total (#130135, @saschagrunert) [SIG API Machinery, Apps, Node and Testing]

  • Implemented a new status field, .status.terminatingReplicas, for Deployments and ReplicaSets to track terminating pods. The new field is present when the DeploymentPodReplacementPolicy feature gate is enabled. (#128546, @atiratree) [SIG API Machinery, Apps and Testing]

  • Implemented validation for NodeSelectorRequirement values in Kubernetes when creating pods. (#128212, @AxeZhan) [SIG Apps and Scheduling]

  • Improved how the API server responds to list requests where the response format negotiates to Protobuf. List responses in Protobuf are marshalled one element at the time, drastically reducing memory needed to serve large collections. Streaming list responses can be disabled via the StreamingCollectionEncodingToProtobuf feature gate. (#129407, @serathius) [SIG API Machinery, Apps, Architecture, Auth, CLI, Cloud Provider, Network, Node, Release, Scheduling, Storage and Testing]

  • InPlacePodVerticalScaling: Memory limits cannot be decreased unless the memory resize restart policy is set to RestartContainer. Container resizePolicy is no longer mutable. (#130183, @tallclair) [SIG Apps and Node]

  • Introduced API type coordination.k8s.io/v1beta1/LeaseCandidate CoordinatedLeaderElection feature moves to Beta (#130751, @Jefftree) [SIG API Machinery, Etcd and Testing]

  • Introduced API type coordination.k8s.io/v1beta1/LeaseCandidate (#130291, @Jefftree) [SIG API Machinery, Etcd and Testing]

  • It introduces a new scope name VolumeAttributesClass.

    It matches all PVC objects that have the volume attributes class mentioned.

    If you want to limit the count of PVCs that have a specific volume attributes class. In that case, you can create a quota object with the scope name VolumeAttributesClass and a matchExpressions that match the volume attributes class. (#124360, @carlory) [SIG API Machinery, Apps and Testing]

  • KEP-3857: Recursive Read-only (RRO) mounts: promote to GA (#130116, @AkihiroSuda) [SIG Apps, Node and Testing]

  • kubectl: Added alpha support for customizing kubectl behavior using preferences from a kuberc file, separate from kubeconfig. (#125230, @ardaguclu) [SIG API Machinery, CLI and Testing]

  • kubelet: added KubeletConfiguration.subidsPerPod. (#130028, @AkihiroSuda) [SIG API Machinery and Node]

  • Kubernetes components that accepted X.509 client certificate authentication now read the user UID from a certificate subject name RDN with object ID 1.3.6.1.4.1.57683.2. An RDN with this object ID had to contain a string value and appear no more than once in the certificate subject. Reading the user UID from this RDN could be disabled by setting the beta feature gate AllowParsingUserUIDFromCertAuth to false(until the feature gate graduated to GA). (#127897, @modulitos) [SIG API Machinery, Auth and Testing]

  • MergeDefaultEvictionSettings indicates that defaults for the evictionHard, evictionSoft, evictionSoftGracePeriod, and evictionMinimumReclaim fields should be merged into values specified for those fields in this configuration. Signals specified in this configuration take precedence. Signals not specified in this configuration inherit their defaults. (#127577, @vaibhav2107) [SIG API Machinery and Node]

  • New configuration is introduced to the kubelet that allows it to track container images and the list of authentication information that leads to their successful pulls. This data is persisted across reboots of the host and restarts of the kubelet.

    The kubelet ensures any image requiring credential verification is always pulled if authentication information from an image pull is not yet present, thus enforcing authentication / re-authentication. This means an image pull might be attempted even in cases where a pod requests the IfNotPresent image pull policy, and might lead to the pod not starting if its pull policy is Never and is unable to present authentication information that led to a previous successful pull of the image it is requesting. (#128152, @stlaz) [SIG API Machinery, Architecture, Auth, Node and Testing]

  • Promoted JobSuccessPolicy E2E to Conformance (#130658, @tenzen-y) [SIG API Machinery, Apps, Architecture and Testing]

  • Promoted NodeInclusionPolicyInPodTopologySpread to Stable in v1.33 (#130920, @kerthcet) [SIG Apps, Node, Scheduling and Testing]

  • Promoted the JobSuccessPolicy to Stable. (#130536, @tenzen-y) [SIG API Machinery, Apps, Architecture and Testing]

  • Promoted the Job's JobBackoffLimitPerIndex feature-gate to stable. (#130061, @mimowo) [SIG API Machinery, Apps, Architecture and Testing]

  • Promoted the feature gate AnyVolumeDataSource to GA. (#129770, @sunnylovestiramisu) [SIG Apps, Storage and Testing]

  • Removed general available feature gate CPUManager. (#129296, @carlory) [SIG API Machinery, Node and Testing]

  • Removed general available feature-gate PDBUnhealthyPodEvictionPolicy. (#129500, @carlory) [SIG API Machinery, Apps and Auth]

  • Start reporting swap capacity as part of node.status.nodeSystemInfo. (#129954, @iholder101) [SIG API Machinery, Apps and Node]

  • Graduated the MultiCIDRServiceAllocator feature gate to stable, and the DisableAllocatorDualWrite feature gate to beta (disabled by default). Action required for Kubernetes cluster administrators and for distributions that manage the cluster Service CIDR. Kubernetes now allows users to define the cluster Service CIDR via an API object: ServiceCIDR. Distributions or administrators of Kubernetes may want to control that new Service CIDRs added to the cluster do not overlap with other networks on the cluster, that only belong to a specific range of IPs. Administrators may also prefer to retain the existing behavior of only having one ServiceCIDR per cluster. You can use ValidatingAdmissionPolicy to achieve this. (#128971, @aojea) [SIG Apps, Architecture, Auth, CLI, Etcd, Network, Release and Testing]

  • The ClusterTrustBundle API is moving to v1beta1. In order for the ClusterTrustBundleProjection feature to work on the kubelet side, the ClusterTrustBundle API must be available at v1beta1 version and the ClusterTrustBundleProjection feature gate must be enabled. If the API becomes later after kubelet started running, restart the kubelet to enable the feature. (#128499, @stlaz) [SIG API Machinery, Apps, Auth, Etcd, Node, Storage and Testing]

  • The Service trafficDistribution field, including the PreferClose option, has graduated to GA. Services that do not have the field configured will continue to operate with their existing behavior. Refer to the documentation https://kubernetes.io/docs/concepts/services-networking/service/#traffic-distribution for more details. (#130673, @gauravkghildiyal) [SIG Apps, Network and Testing]

  • The feature gate InPlacePodVerticalScalingAllocatedStatus is deprecated and no longer used. The AllocatedResources field in ContainerStatus is now guarded by the InPlacePodVerticalScaling feature gate. (#130880, @tallclair) [SIG CLI, Node and Scheduling]

  • The kube-controller-manager will set the observedGeneration field on pod conditions when the PodObservedGenerationTracking feature gate is set. (#130650, @natasha41575) [SIG API Machinery, Apps, Node, Scheduling, Storage, Testing and Windows]

  • The kube-scheduler will set the observedGeneration field on pod conditions when the PodObservedGenerationTracking feature gate is set. (#130649, @natasha41575) [SIG Node, Scheduling and Testing]

  • The kubelet will set the observedGeneration field on pod conditions when the PodObservedGenerationTracking feature gate is set. (#130573, @natasha41575) [SIG Apps, Node, Scheduling, Storage, Testing and Windows]

  • The minimum value validation of ReplicationController's replicas and minReadySeconds fields have been migrated to declarative validation. The requiredness of both fields is also declaratively validated. If the DeclarativeValidation feature gate is enabled, mismatches with existing validation are reported via metrics. If the DeclarativeValidationTakeover feature gate is enabled, declarative validation is the primary source of errors for migrated fields. (#130725, @jpbetz) [SIG API Machinery, Apps, Architecture, CLI, Cluster Lifecycle, Instrumentation, Network, Node and Storage]

  • The resource.k8s.io/v1beta1 API is deprecated and will be removed in 1.36. Use v1beta2 instead. (#129970, @mortent) [SIG API Machinery, Apps, Auth, Etcd, Node, Scheduling and Testing]

  • Validation now requires new StatefulSets with a .spec.serviceName field value to pass DNS1123 validation. Previously created StatefulSets with an invalid .spec.serviceName field value could not create any pods, and should be deleted.

    • Published OpenAPI for the StatefulSet schema is corrected to indicate the .spec.serviceName is optional. (#130233, @soltysh) [SIG API Machinery, Apps and Testing]

  • When the PreferSameTrafficDistribution feature gate is enabled, a new trafficDistribution value PreferSameNode is available, which attempts to always route Service connections to an endpoint on the same node as the client. Additionally, PreferSameZone is introduced as an alias for PreferClose. (#130844, @danwinship) [SIG API Machinery, Apps, Network and Windows]

  • When the PodObservedGenerationTracking feature gate was set, the kubelet populated status.observedGeneration to reflect the latest metadata.generation it observed for the pod. (#130352, @natasha41575) [SIG API Machinery, Apps, CLI, Node, Release, Scheduling, Storage, Testing and Windows]

  • When the StrictIPCIDRValidation feature gate is enabled, Kubernetes will be slightly stricter about what values will be accepted as IP addresses and network address ranges (“CIDR blocks”).

    In particular, octets within IPv4 addresses are not allowed to have any leading 0s, and IPv4-mapped IPv6 values (e.g. ::ffff:192.168.0.1) are forbidden. These sorts of values can potentially cause security problems when different components interpret the same string as referring to different IP addresses (as in CVE-2021-29923).

    This tightening applies only to fields in built-in API kinds, and not to custom resource kinds, values in Kubernetes configuration files, or command-line arguments.

    (When the feature gate is disabled, creating an object with such an invalid IP or CIDR value will result in a warning from the API server about the fact that it will be rejected in the future.) (#122550, #128786, @danwinship) [SIG API Machinery, Apps, Network, Node, Scheduling and Testing]

  • apidiscovery.k8s.io/v2beta1 API group is disabled by default (#130347, @Jefftree) [SIG API Machinery and Testing]

  • kubectl apply now coerces null values for labels and annotations in manifests to empty string values, consistent with typed JSON metadata decoding, rather than dropping all labels and annotations (#129257, @liggitt) [SIG API Machinery]

Feature

  • Added ListFromCacheSnapshot feature gate that allows apiserver to serve LISTs with exact RV and continuations from cache (#130423, @serathius) [SIG API Machinery, Etcd and Testing]
  • Added Pressure Stall Information (PSI) metrics to node metrics. (#130701, @roycaihw) [SIG Node and Testing]
  • Added Windows Server, Version 2025 for windows-servercore-cache test image (#130935, @aramase) [SIG Testing and Windows]
  • Added metrics to expose the main known reasons for resource alignment errors (#129950, @ffromani) [SIG Node and Testing]
  • Added SchedulerPopFromBackoffQ feature gate that is in beta and enabled by default. Improved scheduling queue behavior by popping pods from the backoffQ when the activeQ is empty. This allows to process potentially schedulable pods ASAP, eliminating a penalty effect of the backoff queue. (#130772, @macsko) [SIG Scheduling and Testing]
  • Added apiserver.latency.k8s.io/authentication annotation to the audit log to record the time spent authenticating slow requests. Also added apiserver.latency.k8s.io/authorization annotation to record the time spent authorizing slow requests. (#130571, @hakuna-matatah)
  • Added a /flagz endpoint for kube-proxy (#128985, @yongruilin) [SIG Instrumentation and Network]
  • Added a /status endpoint for kube-proxy (#128989, @Henrywu573) [SIG Instrumentation and Network]
  • Added a /statusz HTTP endpoint to the kube-scheduler. (#128818, @yongruilin) [SIG Architecture, Instrumentation, Scheduling and Testing]
  • Added a /statusz HTTP endpoint to the kubelet. (#128811, @zhifei92) [SIG Architecture, Instrumentation and Node]
  • Added a /statusz endpoint for kube-controller-manager (#128991, @Henrywu573) [SIG API Machinery, Cloud Provider, Instrumentation and Testing]
  • Added a /statusz endpoint for kube-scheduler (#128987, @Henrywu573) [SIG Instrumentation, Scheduling and Testing]
  • Added a mechanism that calculates a digest of etcd and the watch cache every 5 minutes and exposes it as the apiserver_storage_digest metric. (#130475, @serathius) [SIG API Machinery, Instrumentation and Testing]
  • Added a new CLI flag --emulation-forward-compatible Added a new CLI --runtime-config-emulation-forward-compatible (#130354, @siyuanfoundation) [SIG API Machinery, Etcd and Testing]
  • Added a new option strict-cpu-reservation for CPU Manager static policy. When this option is enabled, CPU cores in reservedSystemCPUs will be strictly used for system daemons and interrupt processing no longer available for any workload. (#130290, @psasnal) [SIG Node and Testing]
  • Added an alpha feature gate OrderedNamespaceDeletion. When enabled, the pods resources are deleted before all other resources during namespace deletion. (#130035, @cici37) [SIG API Machinery, Apps and Testing]
  • Added e2e tests for volume group snapshots. (#128972, @manishym) [SIG Cloud Provider, Storage and Testing]
  • Added unit test helpers to validate CEL and patterns in CustomResourceDefinitions. (#129028, @sttts)
  • Added validation of containerLogMaxFiles within kubelet configuration files. (#129072, @kannon92)
  • Adding resource completion in kubectl debug command (#130033, @ardaguclu) [SIG CLI]
  • Adds a /flagz endpoint for kube-controller-manager endpoint (#128824, @yongruilin) [SIG API Machinery and Instrumentation]
  • Allowed ImageVolume for Restricted PSA profiles. (#130394, @Barakmor1)
  • Allowed dynamic configuration of the service account name and audience that the kubelet could request a token for, as part of the node audience restriction feature. (#130485, @aramase) [SIG Auth and Testing]
  • Automatically copy topology.k8s.io/zone, topology.k8s.io/region and kubernetes.io/hostname labels from Node objects to Pods when they are scheduled to a node (via the pods/binding endpoint) to allow applications that need to be explicitly aware of their assigned node topology to access this information via the downward API, rather than requiring permission to get node objects (exposing the entire API surface of the Node object to otherwise unprivileged workloads). (#127092, @munnerz) [SIG API Machinery, Node and Testing]
  • Bumped ProcMountType feature to on by default beta (#130798, @haircommander) [SIG Node]
  • Calculated pod resources are now cached when adding pods to NodeInfo in the scheduler framework, improving performance when processing unschedulable pods. (#129635, @macsko) [SIG Scheduling]
  • cel-go has been bumped to v0.23.2. (#129844, @cici37) [SIG API Machinery, Auth, Cloud Provider and Node]
  • Changed metadata management for Pods to populate .metadata.generation on writes. New pods will have a metadata.generation of 1; updates to mutable fields in the Pod .spec will result in metadata.generation being incremented by 1. (#130181, @natasha41575) [SIG Apps, Node and Testing]
  • DRA: Starting Kubernetes 1.33, regular users with namespaced cluster edit role assigned have read permission to resourceclaims, resourceclaims/status,resourceclaimtemplates. And write permission for resourceclaims, resourceclaimtemplates. (#130738, @ritazh) [SIG Auth]
  • DRAResourceClaimDeviceStatus is now turned on by default allowing DRA-Drivers to report device status data for each allocated device. (#130814, @LionelJouin) [SIG Network and Node]
  • DistributeCPUsAcrossNUMA policy option is promoted to Beta. (#130541, @swatisehgal) [SIG Node]
  • Enabled the OrderedNamespaceDeletion feature gate by default. (#130507, @cici37) [SIG API Machinery and Apps]
  • Enabled user namespaces support (feature gate UserNamespacesSupport) by default. (#130138, @rata) [SIG Node and Testing]
  • Endpoints resources created by the Endpoints controller now include a label indicating this. Users who manually create Endpoints can also add this label, but they should consider using EndpointSlices instead. (#130564, @danwinship) [SIG Apps and Network]
  • Errors returned by apiserver from uninitialized cache will include last error from etcd (#130899, @serathius) [SIG API Machinery and Testing]
  • Errors that occur during pod resize actuation will now surface in the PodResizeInProgress condition. (#130902, @natasha41575)
  • Extended the kube-apiserver loopback client certificate validity to 14 months to align with the updated Kubernetes support lifecycle. (#130047, @HirazawaUi) [SIG API Machinery and Auth]
  • Extended the schema of the kube-proxy healthz and livez HTTP endpoints to incorporate information about the corresponding IP family. (#129271, @aroradaman) [SIG Network and Windows]
  • Fixed SELinuxWarningController defaults when running kube-controller-manager in a container. (#130037, @jsafrane) [SIG Apps and Storage]
  • Fixed a bug to ensure container-level swap metrics are collected. (#129486, @iholder101) [SIG Node and Testing]
  • git-repo volume plugin has been disabled by default, with the option to turn it back (#129923, @vinayakankugoyal)
  • Graduated the WinDSR feature in the kube-proxy to beta. The WinDSR feature gate is now enabled by default. (#130876, @rzlink) [SIG Windows]
  • Graduated the asynchronous preemption feature in the scheduler to beta. Now the feature flag (SchedulerAsyncPreemption) is enabled by default. (#130550, @sanposhiho) [SIG Scheduling]
  • Graduated BtreeWatchCache feature gate to GA. (#129934, @serathius)
  • Graduated the DisableNodeKubeProxyVersion feature gate to enable by default, the kubelet no longer attempts to set the .status.kubeProxyVersion field for its associated Node. (#129713, @HirazawaUi) [SIG Node]
  • Graduated the KubeletFineGrainedAuthz feature gate to beta; the gate is now enabled by default. (#129656, @vinayakankugoyal) [SIG Auth, CLI, Node, Storage and Testing]
  • If scheduling fails on PreBind or Bind, scheduler will retry the failed pod immediately after backoff time, regardless of the reason for failing. In this case EventsToRegister (QHints) will not be taken into consideration before retry. (#130189, @ania-borowiec) [SIG Scheduling]
  • Implemented full support for contextual logging in client-go/rest. BackoffManagerWithContext was used instead of BackoffManager to ensure that the caller could interrupt the sleep. (#127709, @pohly) [SIG API Machinery, Architecture, Auth, Cloud Provider, Instrumentation, Network and Node]
  • Improved how the API server responds to list requests where the response format negotiates to JSON. List responses in JSON are marshalled one element at a time, drastically reducing the memory needed to serve large collections. Streaming list responses can be disabled via the StreamingJSONListEncoding feature gate. (#129334, @serathius) [SIG API Machinery, Architecture and Release]
  • Improved scheduling performance of pods with required topology spreading. (#129119, @macsko) [SIG Scheduling]
  • Introduced the LegacySidecarContainers feature gate enabling the legacy code path that predates the SidecarContainers feature. This temporary feature gate is disabled by default, only available in v1.33, and will be removed in v1.34. (#130058, @gjkim42) [SIG Node]
  • KEP-3619: fine-grained supplemental groups policy is graduated to Beta. Note that kubelet now rejects pods with .spec.securityContext.supplementalGroupsPolicy: Strict when scheduled to the node that does not support the feature (.status.features.supplementalGroupsPolicy: false). (#130210, @everpeace) [SIG Apps, Node and Testing]
  • kube-apiserver: Promoted the ServiceAccountTokenNodeBinding feature gate general availability. It is now locked to enabled. (#129591, @liggitt) [SIG Auth and Testing]
  • kube-apiserver: the StorageObjectInUseProtection admission plugin added the kubernetes.io/vac-protection finalizer to the given VolumeAttributesClass object when it is created if the feature-gate VolumeAttributesClass is turned on and storage.k8s.io/v1beta1 is enabled. (#130553, @Phaow) [SIG Storage and Testing]
  • kubeadm: kubeadm upgrade plan now supports --etcd-upgrade flag to control whether the etcd upgrade plan should be displayed. Add an EtcdUpgrade field into UpgradeConfiguration.Plan for v1beta4. (#130023, @SataQiu) [SIG Cluster Lifecycle]
  • kubeadm: Added preflight check for cp on Linux nodes and xcopy on Windows nodes. These binaries are required for kubeadm to work properly. (#130045, @carlory)
  • kubeadm: Improved kubeadm init and kubeadm join to provide consistent error messages when the kubelet failed or when failed to wait for control plane components. (#130040, @HirazawaUi)
  • kubeadm: Promoted the feature gate ControlPlaneKubeletLocalMode to Beta. By default, kubeadm will use the local kube-apiserver endpoint for the kubelet when creating a cluster with kubeadm init or when joining control plane nodes with kubeadm join. Enabling the feature gate also affects the kubeadm init phase kubeconfig kubelet phase, where the flag --control-plane-endpoint no longer affects the generated kubeconfig Server field, but the flag --apiserver-advertise-address can now be used for the same purpose. (#129956, @chrischdi)
  • kubeadm: graduated the WaitForAllControlPlaneComponents feature gate to Beta. When checking the health status of a control plane component, make sure that the address and port defined as arguments in the respective component's static Pod manifest are used. (#129620, @neolit123) [SIG Cluster Lifecycle]
  • kubeadm: if the NodeLocalCRISocket feature gate is enabled, remove the kubeadm.alpha.kubernetes.io/cri-socket annotation from a given node on kubeadm upgrade. (#129279, @HirazawaUi) [SIG Cluster Lifecycle and Testing]
  • kubeadm: if the NodeLocalCRISocket feature gate is enabled, remove the flag --container-runtime-endpoint from the /var/lib/kubelet/kubeadm-flags.env file on kubeadm upgrade. (#129278, @HirazawaUi) [SIG Cluster Lifecycle]
  • kubeadm: removed preflight check for nsenter on Linux nodes kubeadm: added preflight check for losetup on Linux nodes. It's required by kubelet for keeping a block device opened. (#129450, @carlory) [SIG Cluster Lifecycle]
  • kubeadm: removed the feature gate EtcdLearnerMode which graduated to GA in 1.32. (#129589, @neolit123) [SIG Cluster Lifecycle]
  • kubelet + DRA: For DRA driver plugins (and only for those!), the kubelet now supports a rolling update with maxSurge > 0 in the driver's DaemonSet. A DRA driver must support this, which can be done via the k8s.io/dynamic-resource-allocation/kubeletplugin helper package. (#129832, @pohly) [SIG Node, Storage and Testing]
  • Kubernetes is now built with Go 1.24.2 (#131369, @ameukam) [SIG Release and Testing]
  • NodeRestriction admission now validates that the audience value, the kubelet requested a service account token for, is part of the pod spec volume. The kube-apiserver featuregate ServiceAccountNodeAudienceRestriction is enabled by default in 1.33. (#130017, @aramase)
  • Pod resource checkpointing is now tracked by the allocated_pods_state and actuated_pods_state files, replacing the previously used pod_status_manager_state. (#130599, @tallclair)
  • PodLifecycleSleepAction is now turned on by default allowing users to create containers with sleep lifecycle action with a duration of zero seconds (#130621, @sreeram-venkitesh) [SIG Node]
  • Promoted RelaxedDNSSearchValidation to beta, allowing for Pod search domains to be a single dot "." or contain an underscore "_". (#130128, @adrianmoisey) [SIG Apps and Network]
  • Promoted in-place Pod vertical scaling to beta. The InPlacePodVerticalScaling feature gate is now enabled by default. (#130905, @tallclair) [SIG Node]
  • Promoted kubectl --subresource flag to stable. (#130238, @soltysh)
  • Promoted the CRDValidationRatcheting feature gate to GA in 1.33 (#130013, @yongruilin) [SIG API Machinery]
  • Promoted the feature gate CSIMigrationPortworx to GA. If your applications are using Portworx volumes, please make sure that the corresponding Portworx CSI driver is installed on your cluster before upgrading to 1.31 or later because all operations for the in-tree portworxVolume type are redirected to the pxd.portworx.com CSI driver when the feature gate is enabled. (#129297, @gohilankit) [SIG Storage]
  • Promoted the feature gate HonorPVReclaimPolicy to GA. (#129583, @carlory) [SIG Apps, Storage and Testing]
  • Respect the incoming trace context for authenticated requests to the kube-apiserver for APIServer tracing. (#127053, @dashpole) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Instrumentation, Network, Node and Testing]
  • SELinuxChangePolicy and SELinuxMount graduated to Beta. SELinuxMount stays off by default. (#130544, @jsafrane) [SIG Auth, Node and Storage]
  • Scheduling Framework exposes NodeInfo to the ScorePlugin. (#130537, @saintube) [SIG Scheduling, Storage and Testing]
  • The RemoteRequestHeaderUID feature moves to beta and is now enabled by default. This makes the kube-apiserver propagate UIDs in the X-Remote-Uid header in requests to the aggregated API servers. The header is not honored by default for incoming requests, but that can be enabled by setting the --requestheader-uid-headers flag explicitly. (#130560, @stlaz) [SIG API Machinery, Auth and Testing]
  • The DeclarativeValidation feature gate is enabled by default. When enabled, mismatches with existing hand written validation is reported via metrics. The DeclarativeValidationTakeover feature gate remains disabled by default. While disabled, validation errors produced by hand written validation are always return to the caller. To switch to declarative validation is primary source of errors for migrated fields, enable this feature gate. (#130728, @jpbetz) [SIG API Machinery]
  • The SidecarContainers feature has graduated to GA. 'SidecarContainers' feature gate was locked to default value and will be removed in v1.36. If you were setting this feature gate explicitly, please remove it now. (#129731, @gjkim42) [SIG Apps, Node, Scheduling and Testing]
  • The nftables mode of kube-proxy is now GA. (The iptables mode remains the default; you can select the nftables mode by passing --proxy-mode nftables or using a config file with mode: nftables. See the kube-proxy documentation for more details.) (#129653, @danwinship) [SIG Network]
  • Updated /version response to report binary version information separate from compatibility version (#130019, @yongruilin) [SIG API Machinery, Architecture, Release and Testing]
  • Upgraded the kubectl autoscale subcommand to use autoscaling/v2 rather than autoscaling/v1 APIs. The command now attempts to use the autoscaling/v2 API first. If the autoscaling/v2 API is unavailable or an error occurs, it falls back to the autoscaling/v1 API. (#128950, @googs1025) [SIG Autoscaling and CLI]
  • User namespaces support (feature gate UserNamespacesSupport) is now enabled by (#130138, @rata) [SIG Node and Testing]
  • Various controllers that write out IP address or CIDR values to API objects now ensure that they always write out the values in canonical form. (#130101, @danwinship) [SIG Apps, Network and Node]
  • kubeproxy_conntrack_reconciler_deleted_entries_total metric can be used to track cumulative sum of conntrack flows cleared by reconciler. (#130204, @aroradaman)
  • kubeproxy_conntrack_reconciler_sync_duration_seconds metric can now be used to track conntrack reconciliation latency. (#130200, @aroradaman)
  • The StorageCapacityScoring feature gate was added to score nodes by available storage capacity. It's in alpha and disabled by default. The VolumeCapacityPriority alpha feature was replaced with this, and the default behavior was changed. The VolumeCapacityPriority preferred a node with the least allocatable, but the StorageCapacityScoring preferred a node with the maximum allocatable. See KEP-4049 for details. (#128184, @cupnes) [SIG Scheduling, Storage and Testing]

Documentation

Bug or Regression

  • --feature-gate=InOrderInformers (default on), causes informers to process watch streams in order as opposed to grouping updates for the same item close together. Binaries embedding client-go, but not wiring the featuregates can disable by setting the KUBE_FEATURE_InOrderInformers=false. (#129568, @deads2k) [SIG API Machinery]

  • Added a validation for the revisionHistoryLimit field in the .spec of a StatefulSet, to prevent it from being set to a negative value. (#129017, @ardaguclu)

  • Added progress tracking for volume permission and ownership changes. (#130398, @gnufied) [SIG Node and Storage]

  • Changed the signature of PublishResources() for ResourceSlices to accept a resourceslice.DriverResources parameter instead of a Resources parameter. (#129142, @googs1025) [SIG Node and Testing]

  • DRA: the explanation for why a pod which wasn't using ResourceClaims was unscheduleable included a useless "no new claims to deallocate" when it was unscheduleable for some other reasons. (#129823, @googs1025) [SIG Node and Scheduling]

  • Disabled InPlace Pod Resize for Swap enabled containers that does not have memory ResizePolicy as RestartContainer (#130831, @ajaysundark) [SIG Node and Testing]

  • Enabled ratcheting validation on status subresources for CustomResourceDefinitions. (#129506, @JoelSpeed)

  • Fix: Adopted go1.23 behavior change in mount point parsing on Windows. (#129368, @andyzhangx) [SIG Storage and Windows]

  • Fixed CVE-2024-51744. (#128621, @kmala) [SIG Auth, Cloud Provider and Node]

  • Fixed kubectl wait --for=create behavior with label selectors, to properly wait for resources with matching labels to appear. (#128662, @omerap12) [SIG CLI and Testing]

  • Fixed a bug in HorizontalPodAutoscaler. HPAs with ContainerResource metrics no longer return an error when container metrics are missing. Instead they use the same logic as Resource metrics to perform calculations. (#127193, @DP19) [SIG Apps and Autoscaling]

  • Fixed a bug in the exclusive assignment availability check for the InPlacePodVerticalScalingExclusiveCPUs feature gate. (#130559, @esotsal)

  • Fixed a bug where adding an ephemeral container to a pod which references a new secret or config map doesn't give the pod access to that new secret or config map. (#114984, @cslink) (#129670, @cslink) [SIG Auth]

  • Fixed a bug where kube-apiserver could emit a subsequent watch event even if the previous event failed to decrypt and was not emitted. (#131020, @wojtek-t) [SIG API Machinery and Etcd]

  • Fixed a bug where the kube-proxy EndpointSliceCache memory experienced a leak. (#128929, @orange30)

  • Fixed a data race that could occur when a single Go type was serialized to CBOR concurrently for the first time within a program. (#129170, @benluddy) [SIG API Machinery]

  • Fixed a panic in kube-controller-manager handling StatefulSet objects when revisionHistoryLimit is negative. (#129301, @ardaguclu)

  • Fixed a regression in 1.32 that prevented pods with postStart hooks from starting. (#129946, @alex-petrov-vt)

  • Fixed a regression in 1.32 where nodes could fail to report status and renew serving certificates after the kubelet restarted. (#130348, @aojea)

  • Fixed a regression with the ServiceAccountNodeAudienceRestriction feature where azureFile volumes encountered 'failed to get service account token attributes' errors. (#129993, @aramase) [SIG Auth and Testing]

  • Fixed a storage bug related to multipath. iSCSI and Fibre Channel devices attached to nodes via multipath now resolve correctly when partitioned. (#128086, @RomanBednar)

  • Fixed a test failure in TestSetVolumeOwnershipOwner for fsGroup=3000 and symlink cases in volume_linux_test.go. The tests were failing due to invalid ownership verification and the issue has been resolved by adjusting file permission change handling, ensuring correct behavior when run as root. (#130616, @gnufied)

  • Fixed an issue in register-gen where imports for k8s.io/apimachinery/pkg/runtime and k8s.io/apimachinery/pkg/runtime/schema were missing. (#129307, @LionelJouin) [SIG API Machinery]

  • Fixed an issue in the CEL CIDR library where subnets contained within another CIDR were incorrectly rejected as not being contained. (#130450, @JoelSpeed)

  • Fixed an issue where kubelet would unmount volumes of running pods upon restart if the referenced PVC was being deleted by the user. (#130335, @carlory) [SIG Node, Storage and Testing]

  • Fixed an issue where pods did not correctly have a pending phase after the node reboot. (#128516, @gjkim42) [SIG Node and Testing]

  • Fixed an issue with Kubernetes-style sidecar containers (in other words: init containers with an Always restart policy) and Services. Before the fix, named ports exposed by a sidecar could not be accessed using a Service. (#128850, @toVersus) [SIG Network and Testing]

  • Fixed compressed kubelet log file permissions to use uncompressed kubelet log file permissions. (#129893, @simonfogliato) [SIG Node]

  • Fixed in-tree to CSI migration for Portworx volumes, in clusters where Portworx security feature is enabled (it's a Portworx feature, not Kubernetes feature). It required secret data from the secret mentioned in-tree SC, to be passed in CSI requests which was not happening before this fix. (#129630, @gohilankit) [SIG Storage]

  • Fixed a rare and sporadic network issues that occurred when the host was under heavy load. (#130256, @adrianmoisey)

  • Fixed the bug where Events failed to be created when the referenced object name was not a valid Event name. Now, a UUID is used as the name instead of the referenced object name and the timestamp suffix. (#129790, @aojea)

  • Fixed a 1.32 regression kube-proxy, when using a Service with External or LoadBalancer IPs on UDP services , was consuming a large amount of CPU because it was not filtering by the Service destination port and trying to delete all the UDP entries associated to the service. (#130484, @aojea) [SIG Network]

  • Implemented logging and event recording for probe results with an Unknown status in the kubelet's prober module. This helped improve the diagnosis and monitoring of cases where container probes returned an Unknown result, enhancing the observability and reliability of health checks. (#125901, @jralmaraz)

  • Improved reboot event reporting. The kubelet will only emit one reboot Event when a server-level reboot is detected, even if the kubelet cannot write its status to the associated Node (which triggers a retry). (#129151, @rphillips) [SIG Node]

  • Includes WebSockets HTTPS proxy support (#129872, @seans3) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Instrumentation, Network and Node]

  • kube-apiserver: --service-account-max-token-expiration can now be used in combination with an external token signer --service-account-signing-endpoint, as long as the --service-account-max-token-expiration is not longer than the external token signer's max expiration. (#129816, @sambdavidson) [SIG API Machinery and Auth]

  • kube-apiserver: Fixed a bug where the ResourceQuota admission plugin did not respect any scope changes when a resource was updated, such as setting or unsetting the terminationGracePeriodSeconds field of an existing pod. (#130060, @carlory) [SIG API Machinery, Scheduling and Testing]

  • kube-apiserver: shortening the grace period during a pod deletion no longer moves the metadata.deletionTimestamp into the past (#122646, @liggitt) [SIG API Machinery]

  • kube-proxy: Fixed a potential memory leak that could occur in clusters with a high volume of UDP workflows. (#130032, @aroradaman)

  • kubeadm: Avoided loading the file passed to --kubeconfig during kubeadm init phases more than once. (#129006, @kokes)

  • kubeadm: fixed a bug where an image is not pulled if there is an error with the sandbox image from CRI. (#129594, @neolit123) [SIG Cluster Lifecycle]

  • kubeadm: fixed a bug where the node.skipPhases in UpgradeConfiguration is not respected by the kubeadm upgrade node subcommand. (#129452, @SataQiu)

  • kubeadm: fixed panic when no UpgradeConfiguration was found in the config file. (#130202, @SataQiu)

  • kubeadm: fixed the bug where the v1beta4 Timeouts.EtcdAPICall field was not respected in etcd client operations, and the default timeout of 2 minutes was always used. (#129859, @neolit123) [SIG Cluster Lifecycle]

  • kubeadm: if an addon is disabled in the ClusterConfiguration, skip it during upgrade. (#129418, @neolit123) [SIG Cluster Lifecycle]

  • kubeadm: make sure that it is possible to health check the kube-apiserver when it has --anonymous-auth=false set and the WaitForAllControlPlaneComponents feature gate is enabled. (#131036, @neolit123) [SIG Cluster Lifecycle]

  • kubeadm: run kernel version and OS version preflight checks for kubeadm upgrade. (#129401, @pacoxu)

  • Provides an additional function argument to directly specify the version for the tools that the consumers wished to use. (#129658, @unmarshall)

  • Removed a warning related to Linux user namespaces and kernel version. Previously, if the feature gate UserNamespacesSupport was enabled, the kubelet warned when detecting a Linux kernel version earlier than 6.3.0. While user namespace support generally requires kernel 6.3 or newer, it can also work on older kernels. (#130243, @rata)

  • Removed the limitation on exposing port 10250 externally using a Service. (#129174, @RyanAoh) [SIG Apps and Network]

  • Resolved a performance regression in default 1.31+ configurations, related to the ConsistentListFromCache feature, where rapid create/update API requests across different namespaces encounter increased latency. (#130113, @AwesomePatrol)

  • Revised scheduling behavior to correctly handle nominated node changes. Trigger rescheduling of pods if necessary when pods with nominated node names got deleted or nominated on a different node. (#129058, @dom4ha) [SIG Scheduling, Storage and Testing]

  • The /flagz endpoint in kube-apiserver now correctly returns parsed flag values when the ComponentFlagz feature-gate is enabled. (#130328, @richabanker) [SIG API Machinery and Instrumentation]

  • The BalancedAllocation plugin now skips all best-effort (zero-requested) pods. (#130260, @Bowser1704)

  • The following roles have had Watch added to them (prefixed with system:controller:):

    • cronjob-controller
    • endpoint-controller
    • endpointslice-controller
    • endpointslicemirroring-controller
    • horizontal-pod-autoscaler
    • node-controller
    • pod-garbage-collector
    • storage-version-migrator-controller (#130405, @kariya-mitsuru) [SIG Auth]

  • The response from kube-apiserver's /flagz endpoint would respond correctly with parsed flags value. (#129996, @yongruilin) [SIG API Machinery, Architecture, Instrumentation and Testing]

  • When cpu-manager-policy=static is configured, containers meeting the qualifications for static cpu assignment (i.e. Containers with integer CPU requests in pods with Guaranteed QOS) will not have cfs quota enforced. Because this fix changes a long-established behavior, users observing a regressions can use the DisableCPUQuotaWithExclusiveCPUs feature gate (enabled by default) to restore the previous behavior. Please file an issue if you encounter problems and have to use the Feature Gate. (#127525, @scott-grimes) [SIG Node and Testing]

  • When using the Alpha DRAResourceClaimDeviceStatus feature, IP address values in the NetworkDeviceData are now validated more strictly. (#129219, @danwinship) [SIG Network]

  • YAML input that might previously have been misinterpreted as JSON is now correctly accepted. (#130666, @thockin)

  • [kubectl] Improved the describe output for projected volume sources to clearly indicate whether Secret and ConfigMap entries are optional. (#129457, @gshaibi) [SIG CLI]

  • kube-apiserver: Fixes an issue updating the default ServiceCIDR API object and creating dual-stack Service API objects when --service-cluster-ip-range flag passed to kube-apiserver is changed from single-stack to dual-stack. (#131263, @aojea) [SIG API Machinery, Network and Testing]

Other (Cleanup or Flake)

    1. kube-apiserver: removed the deprecated the --cloud-provider and --cloud-config CLI parameters.
    2. removed generally available feature-gate DisableCloudProviders and DisableKubeletCloudCredentialProviders (#130162, @carlory) [SIG API Machinery, Cloud Provider, Node and Testing]
  • Added metrics to capture CPU distribution across NUMA nodes (#130491, @swatisehgal) [SIG Node and Testing]
  • Add metrics to track allocation of Uncore (aka last-level aka L3) Cache blocks (#130133, @ffromani) [SIG Node and Testing]
  • Changed the dependency version for CoreDNS. Kubernetes tools now install CoreDNS v1.12.0. (#128926, @bzsuni) [SIG Cloud Provider and Cluster Lifecycle]
  • Changed the error message displayed when a pod is trying to attach a volume that does not match the label/selector from "x node(s) had volume node affinity conflict" to "x node(s) didn't match PersistentVolume's node affinity". (#129887, @rhrmo) [SIG Scheduling and Storage]
  • client-gen now sorts input group/versions to ensure stable output generation even with unsorted inputs (#130626, @BenTheElder) [SIG API Machinery]
  • e2e framework: framework.WithFeatureGate [Alpha], [Beta] and [Feature:OffByDefault] tags are now set 1:1 with Alpha, Beta, Feature:OffByDefault Ginkgo labels, replacingFeature:Alpha and Feature:Beta labels. BetaOffByDefault is also added as a Ginkgo label only for off-by-default beta features (#130908, @BenTheElder) [SIG Testing]
  • E2e.test: [Feature:OffByDefault] was added to test names when specifying a feature gate that is not enabled by default. (#130655, @BenTheElder) [SIG Auth and Testing]
  • Extended the schema of kube-proxy's metrics / endpoints to incorporate information about the corresponding IP family. (#129173, @aroradaman) [SIG Network and Windows]
  • Fixed a linting issue in TestNodeDeletionReleaseCIDR. (#128856, @adrianmoisey) [SIG Apps and Network]
  • Flipped StorageNamespaceIndex feature gate to false and deprecated it. (#129933, @serathius)
  • Implemented logging for failed transactions and the full table in kube-proxy with nftables when using log level 4 or higher. Logging is rate-limited to one entry every 24 hours to avoid performance issues. (#128886, @npinaeva)
  • Implemented the scheduler_cache_size metric. Additionally, the scheduler_scheduler_cache_size metric is now deprecated in favor of scheduler_cache_size, and will be removed in v1.34. (#128810, @googs1025)
  • kube-apiserver: Inactive serving code is removed for authentication.k8s.io/v1alpha1 APIs (#129186, @liggitt) [SIG Auth and Testing]
  • kubeadm: Use generic terminology in logs instead of direct mentions of YAML/JSON (#130345, @HirazawaUi)
  • kubeadm: removed preflight check for ip, iptables, ethtool and tc on Linux nodes. kubelet and kube-proxy will continue to report iptables errors if its usage is required. The tools ip, ethtool and tc had legacy usage in the kubelet but are no longer required. (#129131, @pacoxu) [SIG Cluster Lifecycle]
  • kubeadm: removed preflight check for touch on Linux nodes. (#129317, @carlory) [SIG Cluster Lifecycle]
  • kubelet no longer logs multiple errors when running on a system with no iptables binaries installed. (#129826, @danwinship) [SIG Network and Node]
  • Reduced log verbosity for high-frequency, low-value log entries in Job, IPAM, and ReplicaSet controllers by adjusting them to V(2), V(4) and V(4) respectively. This change minimizes log noise while maintaining access to these logs when needed. (#130591, @fmuyassarov) [SIG Apps and Network]
  • Removed alpha support for Windows HostNetwork containers. (#130250, @marosset) [SIG Network, Node and Windows]
  • Removed general available feature gate PersistentVolumeLastPhaseTransitionTime. (#129295, @carlory) [SIG Storage]
  • Removed general available feature-gate AppArmor. (#129375, @carlory) [SIG Auth and Node]
  • Removed generally available feature gate KubeProxyDrainingTerminatingNodes. (#129692, @alexanderConstantinescu) [SIG Network]
  • Removed generally available feature-gate AppArmorFields. (#129497, @carlory) [SIG Node]
  • Removed support for v1alpha1 version of ValidatingAdmissionPolicy and ValidatingAdmissionPolicyBinding API kinds. (#129207, @Jefftree) [SIG Etcd and Testing]
  • Removed the JobPodFailurePolicy feature gate, which graduated to GA in 1.31 and was unconditionally enabled. (#129498, @carlory)
  • Removed the deprecated pod_scheduling_duration_seconds metric. Users need to migrate to pod_scheduling_sli_duration_seconds. (#128906, @sanposhiho) [SIG Instrumentation and Scheduling]
  • Renamed some metrics related to CoreDNS, see the README for v1.11.0 of CoreDNS. (#129232, @DamianSawicki)
  • Show a warning message to inform users that the debug container's capabilities granted by debugging profile may not work as expected if a non-root user is specified in target Pod's .Spec.SecurityContext.RunAsUser field. (#127696, @mochizuki875) [SIG CLI and Testing]
  • The SeparateCacheWatchRPC feature gate is deprecated and disabled by default. (#129929, @serathius) [SIG API Machinery]
  • Renamed coredns metrics, see https://github.com/coredns/coredns/blob/v1.11.0/plugin/forward/README.md#metrics. (#129175, @DamianSawicki) [SIG Cloud Provider]
  • Updated CNI plugins to v1.6.2. (#129776, @saschagrunert) [SIG Cloud Provider, Node and Testing]
  • Updated cri-tools to v1.32.0. (#129116, @saschagrunert)
  • Updated the etcd client library to v3.5.21 (#131103, @ahrtr) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Etcd, Instrumentation, Network, Node and Storage]
  • kube-apiserver disables the beta WatchList feature by default in 1.33 in favor of the StreamingCollectionEncodingToJSON and StreamingCollectionEncodingToProtobuf features.kube-controller-manager no longer opts into enabling the WatchListClient feature in 1.33. (#131359, @deads2k) [SIG API Machinery]

Dependencies

Added

Changed

Removed


Contributors, the CHANGELOG-1.33.md has been bootstrapped with v1.33.0 release notes and you may edit now as needed.



Published by your Kubernetes Release Managers.

Reply all
Reply to author
Forward
0 new messages