k8s v1.13.0 is live!

430 views
Skip to first unread message

Doug MacEachern

unread,
Dec 3, 2018, 8:18:56 PM12/3/18
to kuberne...@googlegroups.com, kubernete...@googlegroups.com, dmace...@vmware.com
Kubernetes team,

Kubernetes v1.13.0 has been built and pushed.

The release notes have been updated in CHANGELOG-1.13.md with a pointer to it on github:

v1.13.0

Documentation

Downloads for v1.13.0

filename sha512 hash
kubernetes.tar.gz 7b6a81c9f1b852b1e889c1b62281569a4b8853c79e5675b0910d941dfa7863c97f244f6d607aae3faf60bccd596dedb9d136b7fffeae199876e780904fd9f31e
kubernetes-src.tar.gz 844b9fbba21374dd190c8f12dd0e5b3303dd2cd7ad25f241d6f7e46f74adf6987afad021553521d4f479c19d87aa8d4d5be77ac7a6715d31a9187a5bab3b397b

Client Binaries

filename sha512 hash
kubernetes-client-darwin-386.tar.gz 0c010351acb660a75122feb876c9887d46ec2cb466872dd073b7f5b26fdadd96888a350e01606f2ae43606a5a4ab2d9309441f4357cee924b19688f9b02c55dc
kubernetes-client-darwin-amd64.tar.gz c2c40bd202900124f4e9458b067a1e1fc040030dc84ce9bcc6a5beb263de05892c16f3bdafb8d854e343e71f086207f390fd0b60f6e32e770c73294b053da6e4
kubernetes-client-linux-386.tar.gz 5f5449be103b103d72a4e2b1028ab014cf7f74781166327f2ae284e4f5ecb539f6b60f36b8f7c7be0ae43dfb30661b2672dd93a1fa7e26d6c67498672674bf12
kubernetes-client-linux-amd64.tar.gz 61a6cd3b1fb34507e0b762a45da09d88e34921985970a2ba594e0e5af737d94c966434b4e9f8e84fb73a0aeb5fa3e557344cd2eb902bf73c67d4b4bff33c6831
kubernetes-client-linux-arm.tar.gz dd5591e2b88c347759a138c4d2436a0f5252341d0e8c9fbab16b8f151e2744cbdd0c8583555a451425bc471f11b688ce568d9245caf8a278cbac2b343fdead89
kubernetes-client-linux-arm64.tar.gz 894ed30261598ebf3485f3575e95f85e3c353f4d834bf9a6ea53b265427704b43fba5403fbc4d522b3f02afb08e6afaae200af1fe57996291a7c74398ec2fe17
kubernetes-client-linux-ppc64le.tar.gz 6c26c807fc730ea736fda75dc57ac73395ba78bb828fffeee18b385be550d8f3ba2bbc27a52a8f15bcbbe68218c7945d9fb725e6759c117422bc0a632c110670
kubernetes-client-linux-s390x.tar.gz 41e6e972de77c0bde22fdd779ea64e731b60f32e97e78a024f33fc3e33a3b364b7f77ece7d3c64ad85b7f8fe7c8fc6d6892098a3362d1fe01ebf3d551fe2bf37
kubernetes-client-windows-386.tar.gz 442229e5030452901b924a94e7a879d4085597a4f201a5b3fc5ac9806cab5830c836cfa7a33e8f1693fe2e8badc4047bf227d7fb00c537fb1fb4cb7639de5455
kubernetes-client-windows-amd64.tar.gz a11a8e8e732e7292781b9cb1de6e3e41683f95fb3fefc2b1a7b5fb1f064a0d80c0833876d931675135778457d81de9ed2e81caee4b3eb27d9f23c7b722b17442

Server Binaries

filename sha512 hash
kubernetes-server-linux-amd64.tar.gz a8e3d457e5bcc1c09eeb66111e8dd049d6ba048c3c0fa90a61814291afdcde93f1c6dbb07beef090d1d8a9958402ff843e9af23ae9f069c17c0a7c6ce4034686
kubernetes-server-linux-arm.tar.gz 4e17494767000256775e4dd33c0a9b2d152bd4b5fba9f343b6dfeb5746ff34e400a8e0aaf2153476453225ef57e4bb1ae3635416ab18f9e4dabf4e5cc82f8aaa
kubernetes-server-linux-arm64.tar.gz 0ddd0cf0ff56cebfa89efb1972cc2bc6916e824c2af56cfd330ac5638c8918eaf3c60d05714b220dbf4f896160eded123beeba42f5be55fe434a43d04508d86a
kubernetes-server-linux-ppc64le.tar.gz b93828560224e812ed21b57fea5458fa8560745cfec96fc1677b258393c00e208ad9b99467b575e74e01699ffd75f03f5793675032e7306cba7208c1afb53c8d
kubernetes-server-linux-s390x.tar.gz 154d565329d5ba52cdb7c3d43d8854b7a9b8e34803c4df6b3e6ae74c1a6e255c78e6559b7546b9158df0e3f7931bbdaf43407d95cd875c79f5cce960bb9882dd

Node Binaries

filename sha512 hash
kubernetes-node-linux-amd64.tar.gz 9d18ba5f0c3b09edcf29397a496a1e908f4906087be3792989285630d7bcbaf6cd3bdd7b07dace439823885acc808637190f5eaa240b7b4580acf277b67bb553
kubernetes-node-linux-arm.tar.gz 959b04ff7b8690413e01bffeabaab2119794dedf06b7aae1743e49988f797cb7e6ff12e1a91af2d4c5f664414f3aa4bd9020521c6a21c1196c194d12a6f7fe08
kubernetes-node-linux-arm64.tar.gz b5c18e8c9e28cf276067c871446720d86b6f162e22c3a5e9343cdbc6857baa6961d09a6908b6acd1bbd132c2e2e526377676babf77b8d3bfb36f8711827c105a
kubernetes-node-linux-ppc64le.tar.gz 63e3504d3b115fdf3396968afafd1107b98e5a1a15b7c042a87f5a9cffbdc274f7b06b07ce90eb51876cfffd57cf7f20180bad7e9f9762af577e51f4f13d2f7a
kubernetes-node-linux-s390x.tar.gz 21c5c2721febf7fddeada9569f3ecbd059267e5d2cc325d98fb74faf1ae9e9e15899750225a1fc7c25feef96e7705b1456cb489f4882b9eb10e78bd0f590d019
kubernetes-node-windows-amd64.tar.gz 3e73d3ecff14b4c85a71bb6cf91b1ab7d9c3075c64bd5ce6863562ab17bf808b0cbc33ddd25346d25040649c1ad89745796afd218190886b54f1d8acc17896e4

Kubernetes 1.13 Release Notes

Security Content

  • CVE-2018-1002105, a critical security issue in the Kubernetes API Server, is resolved in v1.13.0 (and in v1.10.11, v1.11.5, and v1.12.3). We recommend all clusters running previous versions update to one of these releases immediately. See issue #71411 for details.

Urgent Upgrade Notes

(No, really, you MUST do this before you upgrade)

Before upgrading to Kubernetes 1.13, you must keep the following in mind:

  • kube-apiserver
  • The deprecated etcd2 storage backend has been removed. Before upgrading a kube-apiserver using --storage-backend=etcd2, etcd v2 data must be migrated to the v3 storage backend, and kube-apiserver invocations changed to use --storage-backend=etcd3. Please consult the installation procedure used to set up etcd for specific migration instructions. Backups prior to upgrade are always a good practice, but since the etcd2 to etcd3 migration is not reversible, an etcd backup prior to migration is essential.
  • The deprecated --etcd-quorum-read flag has been removed. Quorum reads are now always enabled when fetching data from etcd. Remove the --etcd-quorum-read flag from kube-apiserver invocations before upgrading.
  • kube-controller-manager
  • The deprecated --insecure-experimental-approve-all-kubelet-csrs-for-group flag has been removed.
  • kubelet
  • The deprecated --google-json-key flag has been removed. Remove the --google-json-key flag from kubelet invocations before upgrading. (#69354, @yujuhong)
  • DaemonSet pods now make use of scheduling features that require kubelets to be at 1.11 or above. Ensure all kubelets in the cluster are at 1.11 or above before upgrading kube-controller-manager to 1.13.
  • The schema for the alpha CSINodeInfo CRD has been split into spec and status fields, and new fields status.available and status.volumePluginMechanism added. Clusters using the previous alpha schema must delete and recreate the CRD using the new schema. (#70515, @davidz627)
  • kube-scheduler dropped support for configuration files with apiVersion componentconfig/v1alpha1. Ensure kube-scheduler is configured using command-line flags or a configuration file with apiVersion kubescheduler.config.k8s.io/v1alpha1 before upgrading to 1.13.
  • kubectl
  • The deprecated command run-container has been removed. Invocations should use kubectl run instead (#70728, @Pingan2017)
  • client-go releases will no longer have bootstrap (k8s.io/client-go/tools/bootstrap) related code. Any reference to it will break. Please redirect all references to k8s.io/bootstrap instead. (#67356, @yliaog)
  • Kubernetes cannot distinguish between GCE Zonal PDs and Regional PDs with the same name. To workaround this issue, precreate PDs with unique names. PDs that are dynamically provisioned do not encounter this issue. (#70716, @msau42)

Known Issues

  • If kubelet plugin registration for a driver fails, kubelet will not retry. The driver must delete and recreate the driver registration socket in order to force kubelet to attempt registration again. Restarting only the driver container may not be sufficient to trigger recreation of the socket, instead a pod restart may be required. (#71487)
  • In some cases, a Flex volume resize may leave a PVC with erroneous Resizing condition even after volume has been successfully expanded. Users may choose to delete the condition, but it is not required. (#71470)
  • The CSI driver-registrar external sidecar container v1.0.0-rc2 is known to take up to 1 minute to start in some cases. We expect this issue to be resolved in a future release of the sidecar container. For verification, please see the release notes of future releases of the external sidecar container. (#76)
  • When using IPV6-only, be sure to use proxy-mode=iptables as proxy-mode=ipvs is known to not work. (#68437)

Deprecations

  • kube-apiserver
  • The --service-account-api-audiences flag is deprecated in favor of --api-audiences. The old flag is accepted with a warning but will be removed in a future release. (#70105, @mikedanese)
  • The --experimental-encryption-provider-config flag is deprecated in favor of --encryption-provider-config. The old flag is accepted with a warning but will be removed in 1.14. (#71206, @stlaz)
  • As part of graduating the etcd encryption feature to beta, the configuration file referenced by --encryption-provider-config now uses kind: EncryptionConfiguration and apiVersion: apiserver.config.k8s.io/v1. Support for kind: EncryptionConfig and apiVersion: v1 is deprecated and will be removed in a future release. (#67383, @stlaz)
  • The --deserialization-cache-size flag is deprecated, and will be removed in a future release. The flag is inactive since the etcd2 storage backend was removed. (#69842, @liggitt)
  • The Node authorization mode no longer allows kubelets to delete their Node API objects (prior to 1.11, in rare circumstances related to cloudprovider node ID changes, kubelets would attempt to delete/recreate their Node object at startup) (#71021, @liggitt)
  • The built-in system:csi-external-provisioner and system:csi-external-attacher cluster roles are deprecated and will not be auto-created in a future release. CSI deployments should provide their own RBAC role definitions with required permissions. (#69868, @pohly)
  • The built-in system:aws-cloud-provider cluster role is deprecated and will not be auto-created in a future release. Deployments using the AWS cloud provider should grant required permissions to the aws-cloud-provider service account in the kube-system namespace as part of deployment. (#66635, @wgliang)
  • kubelet
  • Use of the beta plugin registration directory {kubelet_root_dir}/plugins/ for registration of external drivers via the kubelet plugin registration protocol is deprecated in favor of {kubelet_root_dir}/plugins_registry/. Support for the old directory is planned to be removed in v1.15. Device plugin and CSI storage drivers should switch to the new directory prior to v1.15. Only CSI storage drivers that support 0.x versions of the CSI API are allowed in the old directory. (#70494 by @RenaudWasTaken and #71314 by @saad-ali)
  • With the release of the CSI 1.0 API, support for CSI drivers using 0.3 and older releases of the CSI API is deprecated, and is planned to be removed in Kubernetes v1.15. CSI drivers should be updated to support the CSI 1.0 API, and deployed in the new kubelet plugin registration directory ({kubelet_root_dir}/plugins_registry/) once all nodes in the cluster are at 1.13 or higher (#71020 and #71314, both by @saad-ali)
  • Use of the --node-labels flag to set labels under the kubernetes.io/ and k8s.io/ prefix will be subject to restriction by the NodeRestriction admission plugin in future releases. See admission plugin documentation for allowed labels. (#68267, @liggitt)
  • kube-scheduler
  • The alpha critical pod annotation (scheduler.alpha.kubernetes.io/critical-pod) is deprecated. Pod priority should be used instead to mark pods as critical. (#70298, @bsalamat)
  • The following features are now GA, and the associated feature gates are deprecated and will be removed in a future release:
  • CSIPersistentVolume
  • GCERegionalPersistentDisk
  • KubeletPluginsWatcher
  • VolumeScheduling
  • kubeadm
  • The DynamicKubeletConfig feature gate is deprecated. The functionality is still accessible by using the kubeadm alpha kubelet enable-dynamic command.
  • The command kubeadm config print-defaults is deprecated in favor of kubeadm config print init-defaults and kubeadm config print join-defaults (#69617, @rosti)
  • support for the v1alpha3 configuration file format is deprecated and will be removed in 1.14. Use kubeadm config migrate to migrate v1alpha3 configuration files to v1beta1, which provides improvements in image repository management, addons configuration, and other areas. The documentation for v1beta1 can be found here: https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta1
  • The node.status.volumes.attached.devicePath field is deprecated for CSI volumes and will not be set in future releases (#71095, @msau42)
  • kubectl
  • The kubectl convert command is deprecated and will be removed in a future release (#70820, @seans3)
  • Support for passing unknown provider names to the E2E test binaries is deprecated and will be removed in a future release. Use --provider=skeleton (no ssh access) or --provider=local (local cluster with ssh) instead. (#70141, @pohly)

Major Themes

SIG API Machinery

For the 1.13 release, SIG API Machinery is happy to announce that the dry-run functionality is now beta.

SIG Auth

With this release we've made several important enhancements to core SIG Auth areas. In the authorization category, we've further reduced Kubelet privileges by restricting node self-updates of labels to a whitelisted selection and by disallowing kubelets from deleting their Node API object. In authentication, we added alpha-level support for automounting improved service account tokens through projected volumes. We also enabled audience validation in TokenReview for the new tokens for improved scoping. Under audit logging, the new alpha-level "dynamic audit configuration" adds support for dynamically registering webhooks to receive a stream of audit events< /a>. Finally, we've enhanced secrets protection by graduating etcd encryption out of experimental.

SIG AWS

In v1.13 we worked on tighter integrations of Kubernetes API objects with AWS services. These include three out-of-tree alpha feature releases:

1) Alpha for AWS ALB (Application Load Balancer) integration to Kubernetes Ingress resources.
2) Alpha for CSI specification 0.3 integration to AWS EBS (Elastic Block Store)
3) Alpha for the cloudprovider-aws cloud controller manager binary. Additionally we added aws-k8s-tester, deployer interface for kubetest, to the test-infra repository. This plugin allowed us to integrate Prow to the 3 subprojects defined above in order to provide CI signal for all 3 features. The CI signal is visible here under SIG-AWS.

For detailed release notes on the three alpha features from SIG AWS, please refer to the following Changelogs:

SIG Azure

For 1.13 SIG Azure was focused on adding additional Azure Disk support for Ultra SSD, Standard SSD, and Premium Azure Files. Azure Availability Zones and cross resource group nodes were also moved from Alpha to Beta in 1.13.

SIG Big Data

During the 1.13 release cycle, SIG Big Data has been focused on community engagements relating to 3rd-party project integrations with Kubernetes. There have been no impacts on the 1.13 release.

SIG CLI

Over the course of 1.13 release SIG CLI mostly focused on stabilizing the items we’ve been working on over the past releases such as server-side printing and its support in kubectl, as well as finishing kubectl diff which is based on server-side dry-run feature. We’ve continued separating kubectl code to prepare for extraction out of main repository. Finally, thanks to the awesome support and feedback from community we’ve managed to promote the new plugin mechanism to Beta.

SIG Cloud Provider

For v1.13, SIG Cloud Provider has been focused on stabilizing the common APIs and interfaces consumed by cloud providers today. This involved auditing the cloud provider APIs for anything that should be deprecated as well as adding changes where necessary. In addition, SIG Cloud Provider has begun exploratory work around having a “cloud provider” e2e test suite which can be used to test common cloud provider functionalities with resources such as nodes and load balancers.

We are also continuing our long running effort to extract all the existing cloud providers that live in k8s.io/kubernetes into their own respective repos. Along with this migration, we are slowly transitioning users to use the cloud-controller-manager for any cloud provider features instead of the kube-controller-manager.

SIG Cluster Lifecycle

For 1.13 SIG Cluster Lifecycle is pleased to announce the long awaited promotion of kubeadm to stable GA, and the promotion of kubeadm’s configuration API to v1beta1.
In this release the SIG again focused on further improving the user experience on cluster creation and also fixing a number of bugs and other assorted improvements.

Some notable changes in kubeadm since Kubernetes 1.12:

  • kubeadm’s configuration API is now v1beta1. The new configuration format provides improvements in - image repository management, addons configuration, and other areas. We encourage v1alpha3 users to migrate to this configuration API using kubeadm config migrate, as v1alpha3 will be removed in 1.14. The documentation for v1beta1 can be found here: https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta1
  • kubeadm has graduated kubeadm alpha phase commands to kubeadm init phase. This means that the phases of creating a control-plane node are now tightly integrated as part of the init command. Alpha features, not yet ready for GA are still kept under kubeadm alpha and we appreciate feedback on them.
  • kubeadm init and kubeadm init phase now have a --image-repository flag, improving support for environments with limited access to official kubernetes repository.
  • The DynamicKubeletConfig and SelfHosting functionality was moved outside of kubeadm init and feature gates and is now exposed under kubeadm alpha.
  • Kubeadm init phase certs now support the --csr-only option, simplifying custom CA creation.
  • kubeadm join --experimental-control-plane now automatically adds a new etcd member for local etcd mode, further simplifying required tasks for HA clusters setup.
  • Improvements were made to kubeadm reset related to cleaning etcd and notifying the user about the state of iptables.
  • kubeadm commands now print warnings if input YAML documents contain unknown or duplicate fields.
  • kubeadm now properly recognizes Docker 18.09.0 and newer, but still treats 18.06 as the default supported version.
  • kubeadm now automatically sets the --pod-infra-container-image flag when starting the kubelet.

SIG IBM Cloud

The IBM Cloud SIG was focused on defining its charter and working towards moving its cloud provider code to an external repository with a goal to have this work done by the end of Kubernetes 1.14 release cycle. In the SIG meetings, we also made sure to share updates on the latest Kubernetes developments in the IBM Cloud like the availability of Kubernetes v1.12.2 in the IBM Cloud Kubernetes Service (IKS). The SIG updates were provided in the Kubernetes community weekly call and at the KubeCon China 2018.

SIG Multicluster

Moving Federation v2 from Alpha towards Beta has been the focus of our effort over the past quarter. To this end we engaged with end users, and successfully enlisted additional contributors from companies including IBM, Amadeus, Cisco and others. Federation v2 provides a suite of decoupled API’s and re-usable components for building multi-cluster control planes. We plan to start releasing Beta components in late 2018. In addition, more minor updates were made to our cluster-registry and multi-cluster ingress sub-projects.

SIG Network

For 1.13, the areas of focus were in IPv6, DNS improvements and some smaller items:
CoreDNS is now the default cluster DNS passing all of the scale/resource usage tests
Node-local DNS cache feature is available in Alpha. This feature deploys a lightweight DNS caching Daemonset that avoids the conntrack and converts queries from UDP to more reliable TCP.
PodReady++ feature now has kubectl CLI support.

Progress was made towards finalizing the IPv6 dual stack support KEP and support for topological routing of services.

SIG Node

SIG Node focused on stability and performance improvements in the 1.13 release. A new alpha feature is introduced to improve the mechanism that nodes heartbeat back to the control plane. The NodeLease feature results in the node using a Lease resource in the kube-node-lease namespace that is renewed periodically. The NodeStatus that was used previously to heartbeat back to the control plane is only updated when it changes. This reduces load on the control plane for large clusters. The Kubelet plugin registration mechanism, which enables automatic discovery of external plugins (including CSI and device plugins) has been promoted to stable in this release (introduced as alpha in 1.11 and promoted to beta in 1.12).

SIG Openstack

The major theme for the SIG OpenStack release is the work-in-progress for removing the in-tree provider. This work, being done in conjunction with SIG Cloud Provider, is focusing on moving internal APIs that the OpenStack (and other providers) depends upon to staging to guarantee API stability. This work also included abstracting the in-tree Cinder API and refactoring code to the external Cinder provider to remove additional Cinder volume provider code.

Additional work was also done to implement an OpenStack driver for the Cluster API effort lead by SIG Cluster Lifecycle. For the external Cloud-Provider-OpenStack code, the SIG largely focused on bug fixes and updates to match K8s 1.13 development.

SIG Scalability

SIG Scalability has mostly focused on stability and deflaking our tests, investing into framework for writing scalability tests (ClusterLoader v2) with a goal to migrate all tests to it by the end of 2018 and on the work towards extending definition of Kubernetes scalability by providing more/better user-friendly SLIs/SLOs.

SIG Scheduling

SIG Scheduling has mostly focused on stability in 1.13 and has postponed some of the major features to the next versions. There are still two notable changes: 1. TaintBasedEviction is moved to Beta and will be enabled by default. With this feature enabled, condition taints are automatically added to the nodes and pods can add tolerations for them if needed. 2. Pod critical annotation is deprecated. Pods should use pod priority instead of the annotation.

It is worth noting again that kube-scheduler will use apiVersion kubescheduler.config.k8s.io/v1alpha1 instead of componentconfig/v1alpha1 in its configuration files in 1.13.

SIG Service Catalog

The Service Plan Defaults feature is still under active development.
We continue to improve the UX for the svcat CLI, specifically filling in gaps for the new Namespaced Service Broker feature.

SIG Storage

Over the last year, SIG Storage has been focused on adding support for the Container Storage Interface (CSI) to Kubernetes. The specification recently moved to 1.0, and on the heels of this achievement, Kubernetes v1.13 moves CSI support for PersistentVolumes to GA.

With CSI the Kubernetes volume layer becomes truly extensible, allowing third party storage developers to write drivers making their storage systems available in Kubernetes without having to touch the core code.

CSI was first introduction as alpha in Kubernetes v1.9 and moved to beta in Kubernetes v1.10.

You can find a list of sample and production drivers in the CSI Documentation.

SIG Storage also moves support for Block Volumes to beta (introduced as alpha in v1.9) and support for Topology Aware Volume Scheduling to stable (introduced as alpha in v1.9 and promoted to beta in 1.10).

SIG UI

The migration to the newest version of Angular is still under active development as it is most important thing on the roadmap at the moment. We are getting closer to to the new release. We continue fixing bugs and adding other improvements.

SIG VMWare

Major focus for SIG VMware for this release is the work on moving internal APIs that the vSphere provider depends upon to staging to guarantee API stability. This work is being done in conjunction with SIG Cloud Provider and includes the creation of a brand new vsphere-csi plugin to replace the current volume functionalities in-tree.

Additional work was also done to implement a vSphere provider for the Cluster API effort lead by SIG Cluster Lifecycle. For the out-of-tree vSphere cloud provider, the SIG largely focused on bug fixes and updates to match K8s 1.13 development.

SIG Windows

SIG Windows focused on improving reliability for Windows and Kubernetes support

New Features

  • kubelet: When node lease feature is enabled, kubelet reports node status to api server only if there is some change or it didn't report over last report interval. (#69753, @wangzhen127)
  • vSphereVolume implements Raw Block Volume Support (#68761, @fanzhangio)
  • CRD supports multi-version Schema, Subresources and AdditionalPrintColumns (NOTE that CRDs created prior to 1.13 populated the top-level additionalPrinterColumns field by default. To apply an updated that changes to per-version additionalPrinterColumns, the top-level additionalPrinterColumns field must be explicitly set to null). (#70211, @roycaihw)
  • New addon in addon manager that automatically installs CSI CRDs if CSIDriverRegistry or CSINodeInfo feature gates are true. (#70193, @saad-ali)
  • Delegated authorization can now allow unrestricted access for system:masters like the main kube-apiserver (#70671, @deads2k)
  • Added dns capabilities for Windows CNI plugins: (#67435, @feiskyer)
  • kube-apiserver: --audit-webhook-version and --audit-log-version now default to audit.k8s.io/v1 if unspecified (#70476, @charrywanganthony)
  • kubeadm: timeoutForControlPlane is introduced as part of the API Server config, that controls the timeout for the wait for control plane to be up. Default value is 4 minutes. (#70480, @rosti)
  • --api-audiences now defaults to the --service-account-issuer if the issuer is provided but the API audience is not. (#70308, @mikedanese)
  • Added support for projected volume in describe function (#70158, @WanLinghao)
  • kubeadm now automatically creates a new stacked etcd member when joining a new control plane node (does not applies to external etcd) (#69486, @fabriziopandini)
  • Display the usage of ephemeral-storage when using kubectl describe node (#70268, @Pingan2017)
  • Added functionality to enable br_netfilter and ip_forward for debian packages to improve kubeadm support for CRI runtime besides Docker. (#70152, @ashwanikhemani)
  • Added regions ap-northeast-3 and eu-west-3 to the list of well known AWS regions. (#70252, @nckturner)
  • kubeadm: Implemented preflight check to ensure that number of CPUs (#70048, @bart0sh)
  • CoreDNS is now the default DNS server in kube-up deployments. (#69883, @chrisohaver)
  • Opt out of chowning and chmoding from kubectl cp. (#69573, @bjhaid)
  • Failed to provision volume with StorageClass "azurefile-premium": failed to create share andy-mg1121-dynamic-pvc-1a7b2813-d1b7-11e8-9e96-000d3a03e16b in account f7228f99bcde411e8ba4900: failed to create file share, err: storage: service returned error: StatusCode=400, ErrorCode=InvalidHeaderValue, ErrorMessage=The value for one of the HTTP headers is not in the correct format. (#69718, @andyzhangx)
  • TaintBasedEvictions feature is promoted to beta. (#69824, @Huang-Wei)
  • Fixed https://github.com/kubernetes/client-go/issues/478 by adding support for JSON Patch in client-go/dynamic/fake (#69330, @vaikas-google)
  • Dry-run is promoted to Beta and will be enabled by default. (#69644, @apelisse)
  • kubectl get priorityclass now prints value column by default. (#69431, @Huang-Wei)
  • Added a new container based image for running e2e tests (#69368, @dims)
  • The LC_ALL and LC_MESSAGES env vars can now be used to set desired locale for kubectl while keeping LANG unchanged. (#69500, @m1kola)
  • NodeLifecycleController: Now node lease renewal is treated as the heartbeat signal from the node, in addition to NodeStatus Update. (#69241, @wangzhen127)
  • Added dynamic shared informers to write generic, non-generated controllers (#69308, @p0lyn0mial)
  • Upgraded to etcd 3.3 client (#69322, @jpbetz)
  • It is now possible to use named ports in the kubectl port-forward command (#69477, @m1kola)
  • kubectl wait now supports condition value checks other than true using --for condition=available=false (#69295, @deads2k)
  • Updated defaultbackend image to 1.5. Users should concentrate on updating scripts to the new version. (#69120, @aledbf)
  • Bumped Dashboard version to v1.10.0 (#68450, @jeefy)
  • Added env variables to control CPU requests of kube-controller-manager and kube-scheduler. (#68823, @loburm)
  • PodSecurityPolicy objects now support a MayRunAs rule for fsGroup and supplementalGroups options. This allows specifying ranges of allowed GIDs for pods/containers without forcing a default GID the way MustRunAs does. This means that a container to which such a policy applies to won't use any fsGroup/supplementalGroup GID if not explicitly specified, yet a specified GID must still fall in the GID range according to the policy. (#65135, @stlaz)
  • Upgrade Stackdriver Logging Agent addon image to 0.6-1.6.0-1 to use Fluentd v1.2. This provides nanoseconds timestamp granularity for logs. (#70954, @qingling128)
  • When the BoundServiceAccountTokenVolumes Alpha feature is enabled, ServiceAccount volumes now use a projected volume source and their names have the prefix "kube-api-access". (#69848, @mikedanese)
  • Raw block volume support is promoted to beta, and enabled by default. This is accessible via the volumeDevices container field in pod specs, and the volumeMode field in persistent volume and persistent volume claims definitions. (#71167, @msau42)
  • TokenReview now supports audience validation of tokens with audiences other than the kube-apiserver. (#62692, @mikedanese)
  • StatefulSet is supported in kubectl autoscale command (#71103, @Pingan2017)
  • Kubernetes v1.13 moves support for Container Storage Interface to GA. As part of this move Kubernetes now supports CSI v1.0.0 and deprecates support for CSI 0.3 and older releases. Older CSI drivers must be updated to CSI 1.0 and moved to the new kubelet plugin registration directory in order to work with Kubernetes 1.15+. (#71020, @saad-ali)
  • Added option to create CSRs instead of certificates for kubeadm init phase certs and kubeadm alpha certs renew (#70809, @liztio)
  • Added a kubelet socket which serves an grpc service containing the devices used by containers on the node. (#70508, @dashpole)
  • Added DynamicAuditing feature which allows for the configuration of audit webhooks through the use of an AuditSink API object. (#67257, @pbarker)
  • The kube-apiserver's healthz now takes in an optional query parameter which allows you to disable health checks from causing healthz failures. (#70676, @logicalhan)
  • Introduced support for running a nodelocal dns cache. It is disabled by default, can be enabled by setting KUBE_ENABLE_NODELOCAL_DNS=true (#70555, @prameshj)
  • Added readiness gates in extended output for pods (#70775, @freehan)
  • Added Ready column and improve human-readable output of Deployments and StatefulSets (#70466, @Pingan2017)
  • Added kubelet_container_log_size_bytes metric representing the log file size of a container. (#70749, @brancz)
  • NodeLifecycleController: When node lease feature is enabled, node lease will be deleted when the corresponding node is deleted. (#70034, @wangzhen127)
  • GCERegionalPersistentDisk feature is GA now! (#70716, @jingxu97)
  • Added secure port 10259 to the kube-scheduler (enabled by default) and deprecate old insecure port 10251. Without further flags self-signed certs are created on startup in memory. (#69663, @sttts)

Release Notes From SIGs

SIG API Machinery

  • The OwnerReferencesPermissionEnforcement admission plugin now checks authorization for the correct scope (namespaced or cluster-scoped) of the owner resource type. Previously, it always checked permissions at the same scope as the child resource. (#70389, @caesarxuchao)
  • OpenAPI spec now correctly marks delete request's body parameter as optional (#70032, @iamneha)
  • The rules for incrementing metadata.generation of custom resources changed: (#69059, @caesarxuchao)
  • If the custom resource participates the spec/status convention, the metadata.generation of the CR increments when there is any change, except for the changes to the metadata or the changes to the status.
  • If the custom resource does not participate the spec/status convention, the metadata.generation of the CR increments when there is any change to the CR, except for changes to the metadata.
  • A custom resource is considered to participate the spec/status convention if and only if the "CustomResourceSubresources" feature gate is turned on and the CRD has .spec.subresources.status={}.
  • Fixed patch/update operations on multi-version custom resources (#70087, @liggitt)
  • Reduced memory utilization of admission webhook metrics by removing resource related labels. (#69895, @jpbetz)
  • Kubelet can now parse PEM file containing both TLS certificate and key in arbitrary order. Previously key was always required to be first. (#69536, @awly)
  • Code-gen: Removed lowercasing for project imports (#68484, @jsturtevant)
  • Fixed client cert setup in delegating authentication logic (#69430, @DirectXMan12)
  • OpenAPI spec and API reference now reflect dryRun query parameter for POST/PUT/PATCH operations (#69359, @roycaihw)
  • Fixed the sample-apiserver so that its BanFlunder admission plugin can be used. (#68417, @MikeSpreitzer)
  • APIService availability related to networking glitches are corrected faster (#68678, @deads2k)
  • Fixed an issue with stuck connections handling error responses (#71412, @liggitt)
  • apiserver: fixed handling and logging of panics in REST handlers (#71076, @liggitt)
  • kube-controller-manager no longer removes ownerReferences from ResourceQuota objects (#70035, @liggitt)
  • "unfinished_work_microseconds" is added to the workqueue metrics; it can be used to detect stuck worker threads. (kube-controller-manager runs many workqueues.) (#70884, @lavalamp)
  • Timeouts set in ListOptions for clients are also be respected locally (#70998, @deads2k)
  • Added support for CRD conversion webhook (#67006, @mbohlool)
  • client-go: fixed sending oversized data frames to spdystreams in remotecommand.NewSPDYExecutor (#70999, @liggitt)
  • Fixed missing flags in -controller-manager --help. (#71298, @stewart-yu)
  • Fixed missing flags in kube-apiserver --help. (#70204, @imjching)
  • The caBundle and service fields in admission webhook API objects now correctly indicate they are optional (#70138, @liggitt)
  • Fixed an issue with stuck connections handling error responses (#71419, @liggitt)
  • kube-controller-manager and cloud-controller-manager now hold generated serving certificates in-memory unless a writeable location is specified with --cert-dir (#69884, @liggitt)
  • CCM server will not listen insecurely if secure port is specified (#68982, @aruneli)
  • List operations against the API now return internal server errors instead of partially complete lists when a value cannot be transformed from storage. The updated behavior is consistent with all other operations that require transforming data from storage such as watch and get. (#69399, @mikedanese)

SIG Auth

  • API Server can be configured to reject requests that cannot be audit-logged. (#65763, @x13n)
  • Go clients created from a kubeconfig that specifies a TokenFile now periodically reload the token from the specified file. (#70606, @mikedanese)
  • When --rotate-server-certificates is enabled, kubelet will no longer request a new certificate on startup if the current certificate on disk is satisfactory. (#69991, @agunnerson-ibm)
  • Added dynamic audit configuration api (#67547, @pbarker)
  • Added ability to control primary GID of containers through Pod Spec and PodSecurityPolicy (#67802, @krmayankk)
  • kube-apiserver: the NodeRestriction admission plugin now prevents kubelets from modifying Node labels prefixed with node-restriction.kubernetes.io/. The node-restriction.kubernetes.io/ label prefix is reserved for cluster administrators to use for labeling Node objects to target workloads to nodes in a way that kubelets cannot modify or spoof. (#68267, @liggitt)

SIG Autoscaling

SIG AWS

  • service.beta.kubernetes.io/aws-load-balancer-internal now supports true and false values, previously it only supported non-empty strings (#69436, @mcrute)
  • Added service.beta.kubernetes.io/aws-load-balancer-security-groups annotation to set the security groups to the AWS ELB to be the only ones specified in the annotation in case this is present (does not add 0.0.0.0/0). (#62774, @Raffo)

SIG Azure

  • Ensured orphan public IPs on Azure deleted when service recreated with the same name. (#70463, @feiskyer)
  • Improved Azure instance metadata handling by adding caches. (#70353, @feiskyer)
  • Corrected check for non-Azure managed nodes with the Azure cloud provider (#70135, @marc-sensenich)
  • Fixed azure disk attach/detach failed forever issue (#71377, @andyzhangx)
  • DisksAreAttached --> getNodeDataDisks--> GetDataDisks --> getVirtualMachine --> vmCache.Get (#71495, @andyzhangx)

SIG CLI

  • kubectl apply can now change a deployment strategy from rollout to recreate without explicitly clearing the rollout-related fields (#70436, @liggitt)
  • The kubectl plugin list command now displays discovered plugin paths in the same order as they are found in a user's PATH variable. (#70443, @juanvallejo)
  • kubectl get no longer exits before printing all of its results if an error is found (#70311, @juanvallejo)
  • Fixed a runtime error occuring when sorting the output of kubectl get with empty results (#70740, @mfpierre)
  • kubectl: support multiple arguments for cordon/uncordon and drain (#68655, @goodluckbot)
  • Fixed ability for admin/edit/view users to see controller revisions, needed for kubectl rollout commands (#70699, @liggitt)
  • kubectl rollout undo now returns errors when attempting to rollback a deployment to a non-existent revision (#70039, @liggitt)
  • kubectl run now generates apps/v1 deployments by default (#71006, @liggitt)
  • The "kubectl cp" command now supports path shortcuts (../) in remote paths. (#65189, @juanvallejo)
  • Fixed dry-run output in kubectl apply --prune (#69344, @zegl)
  • The kubectl wait command must handle when a watch returns an error vs closing by printing out the error and retrying the watch. (#69389, @smarterclayton)
  • kubectl: support multiple arguments for cordon/uncordon and drain (#68655, @goodluckbot)

SIG Cloud Provider

SIG Cluster Lifecycle

  • kubeadm: Updates version of CoreDNS to 1.2.6 (#70796, @detiber)
  • kubeadm: Validate kubeconfig files in case of external CA mode. (#70537, @yagonobre)
  • kubeadm: The writable config file option for extra volumes is renamed to readOnly with a reversed meaning. With readOnly defaulted to false (as in pod specs). (#70495, @rosti)
  • kubeadm: Multiple API server endpoints support upon join is removed as it is now redundant. (#69812, @rosti)
  • kubeadm reset now cleans up custom etcd data path (#70003, @yagonobre)
  • kubeadm: Fixed unnecessary upgrades caused by undefined order of Volumes and VolumeMounts in manifests (#70027, @bart0sh)
  • kubeadm: Fixed node join taints. (#69846, @andrewrynhard)
  • Fixed cluster autoscaler addon permissions so it can access batch/job. (#69858, @losipiuk)
  • kubeadm: JoinConfiguration now houses the discovery options in a nested Discovery structure, which in turn has a couple of other nested structures to house more specific options (BootstrapTokenDiscovery and FileDiscovery) (#67763, @rosti)
  • kubeadm: Fixed a possible scenario where kubeadm can pull much newer control-plane images (#69301, @neolit123)
  • kubeadm now allows mixing of init/cluster and join configuration in a single YAML file (although a warning gets printed in this case). (#69426, @rosti)
  • kubeadm: Added a v1beta1 API. (#69289, @fabriziopandini)
  • kubeadm init correctly uses --node-name and --cri-socket when --config option is also used (#71323, @bart0sh)
  • kubeadm: Always pass spec.nodeName as --hostname-override for kube-proxy (#71283, @Klaven)
  • kubeadm join correctly uses --node-name and --cri-socket when --config option is also used (#71270, @bart0sh)
  • kubeadm now supports the --image-repository flag for customizing what registry to pull images from (#71135, @luxas)
  • kubeadm: The writable config file option for extra volumes is renamed to readOnly with a reversed meaning. With readOnly defaulted to false (as in pod specs). (#70495, @rosti)
  • kubeadm: Multiple API server endpoints support upon join is removed as it is now redundant. (#69812, @rosti)
  • kubeadm: JoinConfiguration now houses the discovery options in a nested Discovery structure, which in turn has a couple of other nested structures to house more specific options (BootstrapTokenDiscovery and FileDiscovery) (#67763, @rosti)
  • kubeadm: Added a v1beta1 API. (#69289, @fabriziopandini)
  • kubeadm: Use advertise-client-urls instead of listen-client-urls as and etcd-servers options for apiserver. (#69827, @tomkukral)
  • Kubeadm now respects the custom image registry configuration across joins and upgrades. Kubeadm passes the custom registry to the kubelet for a custom pause container. (#70603, @chuckha)
  • kubeadm reset now outputs instructions about manual iptables rules cleanup. (#70874, @rdodev)
  • kubeadm: remove the AuditPolicyConfiguration feature gate (#70807, @Klaven)
  • kubeadm pre-pulls Etcd image only if external Etcd is not used and (#70743, @bart0sh)
  • kubeadm: UnifiedControlPlaneImage is replaced by UseHyperKubeImage boolean value. (#70793, @rosti)
  • For kube-up and derived configurations, CoreDNS will honor master taints, for consistency with kube-dns behavior. (#70868, @justinsb)
  • Recognize newer docker versions without -ce/-ee suffix: 18.09.0 (#71001, @thomas-riccardi)
  • Any external provider should be aware the cloud-provider interface should be imported from :- (#68310, @cheftako)
  • Fixed 'kubeadm upgrade' infinite loop waiting for pod restart (#69886, @bart0sh)
  • Bumped addon-manager to v8.8 (#69337, @MrHohn)
  • GCE: Filter out spammy audit logs from cluster autoscaler. (#70696, @loburm)
  • GCE: Enable by default audit logging truncating backend. (#68288, @loburm)
  • Bumped cluster-proportional-autoscaler to 1.3.0 (#69338, @MrHohn)
  • Updated defaultbackend to v1.5 (#69334, @bowei)

SIG GCP

  • Added tolerations for Stackdriver Logging and Metadata Agents. (#69737, @qingling128)
  • Enabled insertId generation, and updated Stackdriver Logging Agent image to 0.5-1.5.36-1-k8s. This help reduce log duplication and guarantee log order. (#68920, @qingling128)
  • Updated crictl to v1.12.0 (#69033, @feiskyer)

SIG Network

  • Corrected family type (inet6) for ipsets in ipv6-only clusters (#68436, @uablrek)
  • kube-proxy argument hostname-override can be used to override hostname defined in the configuration file (#69340, @stevesloka)
  • CoreDNS correctly implements DNS spec for Services with externalNames that look like IP addresses. Kube-dns does not follow the spec for the same case, resulting in a behavior change when moving from Kube-dns to CoreDNS. See: coredns/coredns#2324
  • IPVS proxier now set net/ipv4/vs/conn_reuse_mode to 0 by default, which will highly improve IPVS proxier performance. (#71114, @Lion-Wei)
  • CoreDNS is now version 1.2.6 (#70799, @rajansandeep)
  • Addon configuration is introduced in the kubeadm config API, while feature flag CoreDNS is now deprecated. (#70024, @fabriziopandini)

SIG Node

  • Fixed a bug in previous releases where a pod could be placed inside another pod's cgroup when specifying --cgroup-root (#70678, @dashpole)
  • Optimized calculating stats when only CPU and Memory stats are returned from Kubelet stats/summary http endpoint. (#68841, @krzysztof-jastrzebski)
  • kubelet now supports log-file option to write logs directly to a specific file (#70917, @dims)
  • Do not detach volume if mount in progress (#71145, @gnufied)
  • The runtimeHandler field on the RuntimeClass resource now accepts the empty string. (#69550, @tallclair)
  • kube-apiserver: fixes procMount field incorrectly being marked as required in openapi schema (#69694, @jessfraz)

SIG OpenStack

  • Fixed cloud-controller-manager crash when using OpenStack provider and PersistentVolume initializing controller (#70459, @mvladev)

SIG Release

  • Use debian-base instead of busybox as base image for server images (#70245, @ixdy)
  • Images for cloud-controller-manager, kube-apiserver, kube-controller-manager, and kube-scheduler now contain a minimal /etc/nsswitch.conf and should respect /etc/hosts for lookups (#69238, @BenTheElder)

SIG Scheduling

  • Added metrics for volume scheduling operations (#59529, @wackxu)
  • Improved memory use and performance when processing large numbers of pods containing tolerations (#65350, @liggitt)
  • Fixed a bug in the scheduler that could cause the scheduler to go to an infinite loop when all nodes in a zone are removed. (#69758, @bsalamat)
  • Clear pod binding cache on bind error to make sure stale pod binding cache will not be used. (#71212, @cofyc)
  • Fixed a scheduler panic due to internal cache inconsistency (#71063, @Huang-Wei)
  • Report kube-scheduler unhealthy if leader election is deadlocked. (#71085, @bsalamat)
  • Fixed a potential bug that scheduler preempts unnecessary pods. (#70898, @Huang-Wei)

SIG Storage

  • Fixed CSI volume limits not showing up in node's capacity and allocatable (#70540, @gnufied)
  • CSI drivers now have access to mountOptions defined on the storage class when attaching volumes. (#67898, @bswartz)
  • change default azure file mount permission to 0777 (#69854, @andyzhangx)
  • Fixed subpath in containerized kubelet. (#69565, @jsafrane)
  • Fixed panic on iSCSI volume tear down. (#69140, @jsafrane)
  • CSIPersistentVolume feature, i.e. PersistentVolumes with CSIPersistentVolumeSource, is GA. (#69929, @jsafrane)
  • Fixed CSIDriver API object to allow missing fields. (#69331, @jsafrane)
  • Flex volume plugins now support expandvolume (to increase underlying volume capacity) and expanfs (resize filesystem) commands that Flex plugin authors can implement to support expanding in use Flex PersistentVolumes (#67851, @aniket-s-kulkarni)
  • Enabled AttachVolumeLimit feature (#69225, @gnufied)
  • The default storage class annotation for the storage addons has been changed to use the GA variant (#68345, @smelchior)
  • GlusterFS PersistentVolumes sources can now reference endpoints in any namespace using the spec.glusterfs.endpointsNamespace field. Ensure all kubelets are upgraded to 1.13+ before using this capability. (#60195, @humblec)
  • Fixed GetVolumeLimits log flushing issue (#69558, @andyzhangx)
  • The MountPropagation feature is unconditionally enabled in v1.13, and can no longer be disabled. (#68230, @bertinatto)

SIG Windows

  • kubelet --system-reserved and --kube-reserved are supported now on Windows nodes (#69960, @feiskyer)
  • Windows runtime endpoints is now switched to npipe:////./pipe/dockershim from tcp://localhost:3735. (#69516, @feiskyer)
  • Fixed service issues with named targetPort for Windows (#70076, @feiskyer)
  • Handle Windows named pipes in host mounts. (#69484, @ddebroy)
  • Fixed inconsistency in windows kernel proxy when updating HNS policy. (#68923, @delulu)

External Dependencies

  • Default etcd server is unchanged at v3.2.24 since Kubernetes 1.12. (#68318)
  • The list of validated docker versions remain unchanged at 1.11.1, 1.12.1, 1.13.1, 17.03, 17.06, 17.09, 18.06 since Kubernetes 1.12. (#68495)
  • The default Go version was updated to 1.11.2. (#70665)
  • The minimum supported Go version was updated to 1.11.2 (#69386)
  • CNI is unchanged at v0.6.0 since Kubernetes 1.10 (#51250)
  • CSI is updated to 1.0.0. Pre-1.0.0 API support is now deprecated. (#71020])
  • The dashboard add-on has been updated to v1.10.0. (#68450)
  • Heapster remains at v1.6.0-beta, but is now retired in Kubernetes 1.13 (#67074)
  • Cluster Autoscaler has been upgraded to v1.13.0 (#71513)
  • kube-dns is unchanged at v1.14.13 since Kubernetes 1.12 (#68900)
  • Influxdb is unchanged at v1.3.3 since Kubernetes 1.10 (#53319)
  • Grafana is unchanged at v4.4.3 since Kubernetes 1.10 (#53319)
  • Kibana has been upgraded to v6.3.2. (#67582)
  • CAdvisor has been updated to v0.32.0 (#70964)
  • fluentd-gcp-scaler has been updated to v0.5.0 (#68837)
  • Fluentd in fluentd-elasticsearch is unchanged at v1.2.4 since Kubernetes 1.11 (#67434)
  • fluentd-elasticsearch has been updated to v2.2.1 (#68012)
  • The fluent-plugin-kubernetes_metadata_filter plugin in fluentd-elasticsearch is unchanged at 2.0.0 since Kubernetes 1.12 (#67544)
  • fluentd-gcp has been updated to v3.2.0 (#70954)
  • OIDC authentication is unchanged at coreos/go-oidc v2 since Kubernetes 1.10 (#58544)
  • Calico was updated to v3.3.1 (#70932)
  • Upgraded crictl on GCE to v1.12.0 (#69033)
  • CoreDNS has been updated to v1.2.6 (#70799)
  • event-exporter has been updated to v0.2.3 (#67691)
  • Es-image remains unchanged at Elasticsearch 6.3.2 since Kubernetes 1.12 (#67484)
  • metrics-server remains unchanged at v0.3.1 since Kubernetes 1.12 (#68746)
  • GLBC remains unchanged at v1.2.3 since Kubernetes 1.12 (#66793)
  • Ingress-gce remains unchanged at v1.2.3 since Kubernetes 1.12 (#66793)
  • ip-masq-agen remains unchanged at v2.1.1 since Kubernetes 1.12 (#67916)
  • v1.13.0-rc.2
  • v1.13.0-rc.1
  • v1.13.0-beta.2
  • v1.13.0-beta.1
  • v1.13.0-alpha.3
  • v1.13.0-alpha.2
  • v1.13.0-alpha.1


Leads, the CHANGELOG-1.13.md has been bootstrapped with v1.13.0 release notes and you may edit now as needed.


Published by anago, the Kubernetes Release Tool

Reply all
Reply to author
Forward
0 new messages