Kubernetes v1.28.0-beta.0 has been built and pushed using Golang version 1.20.6.
The release notes have been updated in CHANGELOG-1.28.md, with a pointer to them on GitHub:
| filename | sha512 hash |
|---|---|
| kubernetes.tar.gz | 5a4575157380ff5dd66fd87e7045f2f57ed0db59513bfef57ee768a7a98f855faa06503a7480e77cdf5128fe66461a6c91f0705f8148347f903342f45b65f8da |
| kubernetes-src.tar.gz | 8efebf779daf168929dd2798d2e52750d09339a17256091b4315b1de82167b26388725a60bc3525468b1a23656932faf1c45ba6957df0bd2b3f48bafc8b62138 |
| filename | sha512 hash |
|---|---|
| kubernetes-client-darwin-amd64.tar.gz | 956bd780cb36815ae8969e345d4f6004740167f3e5e3a1d1b1deda254ac2b167371b7c9e79497bd01e3f11d2e2f8e1c35f8fdc3114f08324470635cff1efab20 |
| kubernetes-client-darwin-arm64.tar.gz | 68ebd65edb40e23c5e70b6666fea34a774ccd66313dc884e28a8032cfd8c166c1a2dc66c635a61b0d568fe825b208bfd32a040e3eded536f71617acac625f3ea |
| kubernetes-client-linux-386.tar.gz | 0291c01019e47af1631f3f90f46aa92e4572301fe2411586fb2d69291fdf113a8ff78531f51530d05c6113e28e0e69e23f1d2e2143f832f6b8f77a133b09e493 |
| kubernetes-client-linux-amd64.tar.gz | bd9a98bd35ed9bb113399e50456da7008629e060381a7f6b9071fd7bcd498cb252da2824376662a4d4d4cae637feb90f3768414751e7ec0339c6c9711f6aceda |
| kubernetes-client-linux-arm.tar.gz | 311b3c1a42d68fa337f6e8b2290817196c97024535ee94f2180971e09ca78c9037b41fca48c1eda1fa75079631ea8805f0c6a173e35fe4a9762d13bf3bdf1c58 |
| kubernetes-client-linux-arm64.tar.gz | 039491fddc63e279821ff5287cada111b5c94dc0389d697adc2503f77905a74cda1a67dd52d4b42be0141896d04cfb18f3dfc0c387620d9a9fb3a7f2aa5b399e |
| kubernetes-client-linux-ppc64le.tar.gz | 70e4011dce6777c511a69872a1e015b3a2137cea100cd6b988946ea227bd05827f4223a44c9d24433043cc414d3746465603dc4de2e84128e7689712dd29b00d |
| kubernetes-client-linux-s390x.tar.gz | 51ea51fbcdcf7f2c9c4c97b6f28ceb42753e52ccf69318c05befa77b94d9c17e871061b4d03dbc12632663efed161424b3dba97ac6df46cf27d6deb1a0c011fb |
| kubernetes-client-windows-386.tar.gz | e069121289664d53a6570d90ac2d66911d18cf8c47461f5e8e2cd52ddd651a0e119327d23455b9c273e979378379d27e5cf102c7f8b8c98871c9ef9c7e790e52 |
| kubernetes-client-windows-amd64.tar.gz | f19a06ad641a282b9895673a0628cc937222b53d9b852fd7543de01294d403100d0bcd4659fbf73bf8cdc55a8e3c7f494991db2b9f4d8bc63446b7810232d3aa |
| kubernetes-client-windows-arm64.tar.gz | 7bd6a6d6ae7b487aa35c162532e9b59e184aac0fb6b65c61b2af06217007f78aeb5d0bed039983c534358152155739a6e30f533488c49a13d3c81d54eea6b8c6 |
| filename | sha512 hash |
|---|---|
| kubernetes-server-linux-amd64.tar.gz | 7cd83dc6e7013ef8638965fb3e30f8ba122f045d987029da345c185662bea824fc6a3fae34ff549c457638daf703833312893180538552d194fcc7f4fb0642fd |
| kubernetes-server-linux-arm64.tar.gz | 07e9304f6864f4334b9710ad1a130044c9b1349bf9e47e5c9857b688322abe5babf45a95cdbad3f2650b5447c11864edc3d50fa86de5d485e84730260efbdc8e |
| kubernetes-server-linux-ppc64le.tar.gz | 84f7c2b7021bd136d0c9013b349d63102433f7485552a0f19cb61f4630f256aef1f99a54f9da14b5d6c242778488c5539e0c358b9e421aafaf746ce783773e9b |
| kubernetes-server-linux-s390x.tar.gz | 80263d622d219ae687ea24d9b552c5e6fcb920edc6adc73fe5c742cbb34db08a045f52babb0e6c5acfab98616b9f9f2c87150db9a04f5799836b8c8fd0709f31 |
| filename | sha512 hash |
|---|---|
| kubernetes-node-linux-amd64.tar.gz | bb24b87e1971fde06882d4a2b91b2663ee9daad1e1f8f31f457eaac235b26466ec2413d947d06803b8fc9c356e56e77f7ed31b1f021eae0fdb3df426bc610717 |
| kubernetes-node-linux-arm64.tar.gz | 91aa0771cf8f6615655aeda2484d967f53dbe10cee7ac724be23570d5ca60e3fe11e354cd8b715b882ba20534dab67fee505cb2cd6df1c90d124f778eadff67b |
| kubernetes-node-linux-ppc64le.tar.gz | e30e0efdd8ee02b3ee9c4da1e106a41ffdff7606401f44647e6bd03ebf4ef1900bf7c7d5a2382412a0e2b4bd7013e04e3baa08637d4dbe2b2993fcdab7e2378d |
| kubernetes-node-linux-s390x.tar.gz | e011016917d19bc5b84a76899a3eb6d7e2a6bf270e2d799ba77a9c90daabaf2055655ed0b6a62f2b9e5edfbd4b902e6e4a2408d2dc5c63a19a706d3f838a3864 |
| kubernetes-node-windows-amd64.tar.gz | b9f365607de7112b2e62458462f1261905b1731d40c125db228bbc5aa48cc108872552db3c873702df0607bc0eb1996a1dca16cd27526f002bf1842e2eefc4ef |
All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name.
Changed kubectl version default output to be identical to what kubectl version --short printed,
and remove --short flag entirely. (#116720, @soltysh) [SIG CLI and Testing]
Deprecated support for CSI migration of Ceph RBD volumes.
Users who were relying on Kubernetes' ability to migrate to an out-of-tree storage driver should complete that migration before the support for it is removed. (#118303, @carlory) [SIG Storage]
KMSv1 is deprecated and will only receive security updates going forward. Use KMSv2 instead. Set --feature-gates=KMSv1=true to use the deprecated KMSv1 feature. (#119007, @aramase) [SIG API Machinery and Auth]
The deprecated flag --lock-object-namespace and --lock-object-name have been removed from kube-scheduler. Please use --leader-elect-resource-namespace and --leader-elect-resource-name or ComponentConfig instead to configure those parameters. (#119130, @SataQiu) [SIG Scheduling]
ContainerAllocateResponse. This field maps to the CDIDevice field in the CRI protocol. (#118254, @elezar) [SIG Node and Testing]batch.kubernetes.io/cronjob-scheduled-timestamp to Job objects scheduled from CronJobs. (#118137, @helayoty) [SIG Apps]reason and fieldPath into CRD validation rules to allow users to specify reason and field path when validation failed. (#118041, @cici37) [SIG API Machinery]namespaceObject
variable with expressions. (#118267, @cici37) [SIG API Machinery and Testing]AdmissionWebhookMatchCondition feature to beta (#119380, @a-hilaly) [SIG API Machinery]BETA stability. (#119264, @logicalhan) [SIG API Machinery, Architecture, Cluster Lifecycle and Instrumentation]resizeStatus enum from pvc.Status and replaced with AllocatedResourceStatus (#116335, @gnufied) [SIG API Machinery, Apps, Auth, Node, Storage and Testing]statefulset.kubernetes.io/pod-index. (#119232, @danielvegamyhre) [SIG Apps]<pod>-<claim name>, but a random suffix will avoid name collisions. (#117351, @pohly) [SIG API Machinery, Apps, Auth, Node, Scheduling and Testing]A ValidatingAdmissionPolicy now has its messageExpression field checked against resolved types. (#119209, @jiahuif) [SIG API Machinery]
Add ConsistentListFromCache feature gate that allows apiserver to serve consistent lists from cache (#118508, @serathius) [SIG API Machinery, Instrumentation and Testing]
Add full cgroup v2 swap support for both Limited and Unlimited swap.
When LimitedSwap is enabled the swap limit would be automatically calculated for Burstable QoS pods. For Best-Effort / Guaranteed QoS pods, swap would be disabled.
Containers with memory requests equal to their memory limits also won't have swap access, and it is a way to opt-out of swap for a single container.
The formula for the swap limit for Burstable QoS pods is:
(<memory-request>/<node-memory-capacity>)*<node-swap-capacity>.
Support for cgroup v1 is removed. (#118764, @iholder101) [SIG Node and Testing]
Add handling for pods in podgc for PodReplacementPolicy or PodDisruption (#118772, @kannon92) [SIG Apps and Testing]
Add reason to metric attachdetach_controller_forced_detaches in the attach detach controller. (#119185, @xing-yang) [SIG Apps and Storage]
Add swap to stats to Summary API and Prometheus endpoints (stats/summary and /metrics/resource). (#118865, @iholder101) [SIG Node and Testing]
Added a new command line argument --interactive to kubectl. The new command line argument lets a user confirm deletion requests per resource interactively. (#114530, @ardaguclu) [SIG CLI and Testing]
Added a new feature gate, SchedulerQueueingHints (enabled by default).
The new feature gate activates a framework for fine-grained filtering of events related to scheduler plugins.
In this release, no default scheduling plugins make use of the hinting framework, so you should not expect any behavior changes. (#119328, @sanposhiho) [SIG Scheduling]
Adds apiserver_admission_match_condition_evaluation_seconds and apiserver_admission_match_condition_exclusions_total metrics (#119311, @ivelichkovich) [SIG API Machinery]
Bump distroless-iptables to 0.2.6 based on Go 1.20.6 (#119365, @xmudrii) [SIG Testing]
CEL authorizer checks no longer raise runtime errors. Calls to "check" will always return a decision object and the authorization error (if any) can be accessed within expressions using the new decision methods "errored" and "error". (#118804, @benluddy) [SIG API Machinery]
CRI: expose commit memory bytes in container stats specific to Windows (#119238, @kiashok) [SIG Node and Windows]
Cloud controller manager's node controller now emits timing metrics for initial Node synchronization. These metrics measure the delay between the creation of a new Node and the node controller's initial management actions, such as removing the cloud provider taint. These metrics should be consulted when setting cloud controller manager's --concurrent-node-syncs flag. (#119241, @cartermckinnon) [SIG Cloud Provider and Instrumentation]
Faster scheduling when ResourceClaims are involved (#119078, @pohly) [SIG Node and Scheduling]
Graduate the ProbeTerminationGracePeriod feature gate to GA (#114307, @rphillips) [SIG Apps and Node]
Hashing of KeyID in Logs
This release adds a feature to hash the KeyID values in the logs. The KeyID values are sensitive information that should not be exposed in plain text in the logs. By hashing the KeyID values, we can protect the confidentiality of the data while still being able to log the necessary information. (#118988, @nilekhc) [SIG API Machinery, Auth and Testing]
Implement alpha support for a drop-in kubelet configuration directory (#119390, @sohankunkerkar) [SIG Node]
In the course of admitting a single request, the ValidatingAdmissionPolicy plugin will perform no more than one authorization check per unique authorizer expression. All evaluations of identical authorizer expressions will produce the same decision. (#116443, @benluddy) [SIG API Machinery and Testing]
Kube-controller-manager: the dynamic resource controller steps in when a pod got created such that the scheduler ignores it (i.e. spec.nodeName is set) and then takes care of triggering delayed resource claim allocation and/or reserving a claim for the pod. (#118209, @pohly) [SIG API Machinery, Apps, Auth, Node and Testing]
Kube-proxy service health returns http header "X-Load-Balancing-Endpoint-Weight" with number of local endpoints. The same information is still available in response body JSON payload.LocalEndpoints. (#118999, @cezarygerard) [SIG Network]
Kubelet: plugins for dynamic resource allocation may use the v1alpha3 API instead of v1alpha2 if they want to do prepare/unprepare operations in batches. (#119012, @pohly) [SIG Node and Testing]
Kubelet: security of dynamic resource allocation was enhanced by limiting node access to those objects that are needed on the node. (#116254, @pohly) [SIG Auth and Testing]
Kubernetes is now built with Go 1.20.6 (#119324, @xmudrii) [SIG API Machinery, Auth, Cloud Provider, Release and Testing]
Migrate pkg/controller/endpoint to contextual logging (#116755, @my-git9) [SIG Apps, Instrumentation and Network]
Migrated the EndpointSlice and EndpointSliceMirroring controllers (within kube-controller-manager) to use contextual logging. (#115295, @Namanl2001) [SIG API Machinery, Apps, Network and Testing]
Move non-graceful node shutdown to GA. (#118228, @carlory) [SIG Apps, Storage and Testing]
New CEL Library functions to support Kubernetes Quantities. (#118803, @alexzielenski) [SIG API Machinery]
New Metrics Added for Encryption Configuration Controller
This release adds new metrics to the Encryption Configuration Controller to help monitor the automatic reloading of encryption configuration. The new metrics include:
apiserver_encryption_config_controller_automatic_reload_failures_total: Total number of failed automatic reloads of encryption configuration.apiserver_encryption_config_controller_automatic_reload_success_total: Total number of successful automatic reloads of encryption configuration.apiserver_encryption_config_controller_automatic_reload_last_timestamp_seconds: Timestamp of the last successful or failed automatic reload of encryption configuration.These metrics can be used to monitor the health of the Encryption Configuration Controller and to troubleshoot any issues that may arise during automatic reloading of encryption configuration. (#119008, @nilekhc) [SIG API Machinery, Auth and Instrumentation]
New staging repo has been created for the EndpointSlice reconciler. (#118953, @mskrocki) [SIG Apps, Network and Release]
Promote the following apiserver flowcontrol metrics to Beta:
apiserver_flowcontrol_request_wait_duration_seconds apiserver_flowcontrol_current_executing_seats apiserver_flowcontrol_nominal_limit_seats apiserver_flowcontrol_rejected_requests_total apiserver_flowcontrol_dispatched_requests_total apiserver_flowcontrol_current_inqueue_requests apiserver_flowcontrol_current_executing_requests (#119110, @andrewsykim) [SIG API Machinery and Instrumentation]
Replace apiserver_storage_db_total_size_in_bytes with apiserver_storage_size_bytes metric (#118812, @serathius) [SIG API Machinery, Instrumentation and Testing]
The apiserver debug endpoint /debug/api_priority_and_fairness/dump_requests has been extended to dump executing requests as well as queued ones. A column for StartTime has been added to the returned table, with the queued requests having a StartTime of "0001-01-01T00:00:00Z". The executing requests have a RequestIndexInQueue of -1, and the QueueIndex is also -1 for priority levels without queues. (#119009, @MikeSpreitzer) [SIG API Machinery]
The scheduler skips the PodTopologySpread Score plugin when nothing to do with the Pod. It will affect some metrics values related to the PodTopologySpread Score plugin. (#118608, @utam0k) [SIG Scheduling]
TopologyManagerPolicyOptions feature-flag is promoted to beta and enabled by default. (#118816, @PiotrProkop) [SIG Node]
Update kube-apiserver's priority & fairness work estimator such that 'max seats' is MIN(0.15 x nominalCL, nomincalCL / handSize) (#118601, @andrewsykim) [SIG API Machinery]
ValidatingAdmissionPolicy type checking now correctly handles authorizer variable. (#118540, @jiahuif) [SIG API Machinery]
With the KubeletCgroupDriverFromCRI feature gate enabled and sufficiently new version of a container
runtime, kubelet automatically detects the cgroup driver config from the container runtime, eliminating
the need to specify the cgroupDriver configuration option (or --cgroup-driver` flag) of kubelet. (#118770, @marquiz) [SIG Node]
[Kube-proxy]: implement connection draining for terminating nodes, KEP-3836 (#116470, @alexanderConstantinescu) [SIG Network]
force_delete_pods_total and force_delete_pod_errors_total metrics count all pod deletion behaviors. (#118480, @carlory) [SIG Apps]
kubectl port-forward, when used with a Deployment, could connect to a terminating pod even when a running pod is also available. (#119256, @brianpursley) [SIG CLI]NewVolumeManagerReconstruction feature enabled and a CSI volume present in /var/lib/kubelet/pods. (#117804, @jsafrane) [SIG Node and Storage]sync_proxy_rules_iptables_total metric has now reverted back
to its pre-1.27 behavior of tracking the total number of iptables rules that
kube-proxy is responsible for, rather than only counting the number of rules
that it re-synced on the last sync. The new sync_proxy_rules_iptables_last
metric now gives the latter number. (#119140, @danwinship) [SIG Network]apiserver_flowcontrol_request_concurrency_limit has been deprecated and will be removed in a future release. It is a duplicate of apiserver_flowcontrol_nominal_limit_seats (introduced in release 1.26) but has an outdated name and had an outdated HELP string. (#118959, @MikeSpreitzer) [SIG API Machinery]kube-controller-manager) to use contextual logging. (#119147, @mengjiao-liu) [SIG API Machinery, Apps, Instrumentation and Testing]kube-controller-manager to use contextual logging. kube-controller-manager is now converted completely. (#119250, @pohly) [SIG API Machinery, Apps, Cloud Provider, Instrumentation, Network, Storage and Testing]
Contributors, the
CHANGELOG-1.28.md has been bootstrapped with
v1.28.0-beta.0 release notes and you may edit now as needed.
Published by your
Kubernetes Release
Managers.