k8s v1.14.0 is live!

133 views
Skip to first unread message

K8s-Anago

unread,
Mar 25, 2019, 5:59:21 PM3/25/19
to kuberne...@googlegroups.com, kubernete...@googlegroups.com, hh...@pivotal.io
Kubernetes team,

Kubernetes v1.14.0 has been built and pushed.

The release notes have been updated in CHANGELOG-1.14.md with a pointer to it on github:

v1.14.0

Documentation

Downloads for v1.14.0

filename sha512 hash
kubernetes.tar.gz 0ad264a46f185a9ff4db0393508a9598dab146f438b2cfdc7527592eb422870b8f26ade7ed089359c06741d998fcd730f897eae261f922c1a26d9fdc034d270d
kubernetes-src.tar.gz c5175439decc1c5f54254572bfec3c9f61f39d6bd1cbc28d1f771f8f931b98f0c305f1871618ce7e9de9cf3bf8227e19dcf985a7e017c74d0d7ab4005b3dbd59

Client Binaries

filename sha512 hash
kubernetes-client-darwin-386.tar.gz 68bdba50a2b0be755e73e34ffc758fd419940adace096b1ddebd44a0eae2c7cdaed984965ea8f2145c1cab0be47bd6c72c2aeb73e51d449bfeb9ce1854b6c562
kubernetes-client-darwin-amd64.tar.gz 255bd93082b3ac5d69bd4e45c75c9f19efee50ad6add50837ff2987ce16cbcc485fad334c980b17f69e5a344ee50548e206f747441ad4a045aa65746c79d10ca
kubernetes-client-linux-386.tar.gz 2bd115ad2503fdfe5482e4592fcc0c8a2aee36be5205220a13c8050cd1e55dd3c08377425dbe5a03e4ffd21cf603c739ec4eaf3e5b2514a725d095df46f25d98
kubernetes-client-linux-amd64.tar.gz a551adf8019b17fce5aff2b379fab3627588978a2d628b64ba1af6f3be1b435322368b00dd04fa739d01c341420016b93239cc0d4601cee86706d81d78cb4d7f
kubernetes-client-linux-arm.tar.gz 24e771cd4074786330e07f5537259a28d0932102639326230d9161f12a8dc545638a55bc252771eb4e21e95e2c7f0918dc1238ac4dc70d3b8b33f093da7123ab
kubernetes-client-linux-arm64.tar.gz 16204f2345ab3523bbe3c868f04806a97c111d940b2594aaff67cf73b4259040c7770d5b0e7bdb7ffd7389f87e5f090ae875bd0f192b07582f59a01a1df32f5b
kubernetes-client-linux-ppc64le.tar.gz cd9ce829d585dd3331c53d35015d4017026d5efd24b9bc2f342995245628598c98bd8b1f1d706b196a7b3046a44049d4aba6efb4b1000722bfd055bd8a662f1f
kubernetes-client-linux-s390x.tar.gz 482c0a8e53b27f8922f58d89fb81842ddd9c3ffd120e635838992dc97d535e46b42e7d8c439cb739b7c1d63c8eed27d7e3bcac7126a6a96e56cc13d52f396328
kubernetes-client-windows-386.tar.gz 4446d666f999e979a7245e1b7ebf4817f7bd23aa247a38853a63b9cda473c7d4c2d376a2fd0df13ba15b740bf6b458cac14bd03dbf5a8151fc230e40c08294cf
kubernetes-client-windows-amd64.tar.gz 97f4789f21d10fd3df446e55bc489472dcd534c623bb40dc3cb20fe1edd74c1a89a50ce7caa4e5e0536f3b22d8698060bfe8c46f4adbd0e507349412e52664e8

Server Binaries

filename sha512 hash
kubernetes-server-linux-amd64.tar.gz 25739802a641517a8bbb933b69000a943e8dd38e616b8778149dd0138737abacf377683da2ff35fdd0bbb305b88bc8fc711df20a2585720a43bb674ef36b034f
kubernetes-server-linux-arm.tar.gz c1dbba77a4ff5661eb36c55182a753b88ccc9b89ca31e162b06672126743cfea115b2f8ea8658b12344c36df17958e310c1b8efbdd7800f44f013e1e6f10477d
kubernetes-server-linux-arm64.tar.gz ad346bbe2a053c1106b51e5125698737dc7b76fa3bf439e14d4b4ba1c262678fede9c507c1098aac6e14d2c742c526c8d257fefa95dd3bbb1dff959e1dc7b9aa
kubernetes-server-linux-ppc64le.tar.gz 49f9bd1c751620ecf4b5c152f287d72b36abca21fd1dfe99443d984473c6efa051a910de585c42f5447ef7c18d7dbd905a66c4f09ca6025f45e63f5e96e3ca2f
kubernetes-server-linux-s390x.tar.gz d6be847f2a0358755a69dea26181e5fc1a80ac4939b8b04a3875e1f6693553cad562452bfad21b2e380ddda1839ab846122bc3339d8bec0971f218f6e8f6dce9

Node Binaries

filename sha512 hash
kubernetes-node-linux-amd64.tar.gz 75dc99919d1084d7d471a53ab60c743dc399145c99e83f37c6ba3c241b2c0b2ecc2c0d1b94690ff912e2a15b7c5595aa1d2d24c2fc439e06d85ff0246fb43b89
kubernetes-node-linux-arm.tar.gz 49013a4f01be8086fff332099d94903082688b9b295d2f34468462656da4709360025e9d84b069410c608977ef803079af09af1f1e2678af7cb64e0fc02e9c9d
kubernetes-node-linux-arm64.tar.gz f8c0cb0c089cd1d1977c049002620b8cf748d193c1b76dd1d3aac01ff9273549c06a1e3dfe983dc40a95ee8b0719908e0cdf86ce17359b5f1b2426f2c55799a1
kubernetes-node-linux-ppc64le.tar.gz 48fc02c856a192388877189a43eb1cda531e548bb035f9dfe6a1e3c8d3bcbd0f8e14f29382da45702cb28a91126d13ede42bd6e9159e12ecbd387ca9a58f9a92
kubernetes-node-linux-s390x.tar.gz d7c5f52cf602fd0c0d0f72d4cfe1ceaa4bad70a42f37f21c103f17c3448ceb2396c1bfa521eeeb9eef5f3173d84e4268704a247edd826d765f65e9a29a4f7f72
kubernetes-node-windows-amd64.tar.gz 120afdebe844b06a7437bb9788c3e7ea4fc6352aa18cc6a00e70f44f54664f844429f138870bc15862579da632632dff2e7323be7f627d9c33585a11ad2bed6b

Kubernetes v1.14 Release Notes

1.14 What’s New

Support for Windows Nodes is Graduating to Stable (#116 )

  • Support for Windows Server 2019 for worker nodes and containers
  • Support for out of tree networking with Azure-CNI, OVN-Kubernetes and Flannel
  • Improved support for pods, service types, workload controllers and metrics/quotas to closely match the capabilities offered for Linux containers
    kubernetes/enhancements: #116 [kep]

Updated Plugin Mechanism for kubectl is Graduating to Stable (#579)

  • Extends functionality to kubectl to support extensions adding new commands as well as overriding specific subcommands (at any depth).
  • Documentation fixes
    kubernetes/enhancements: #579 [kep]

Durable Local Storage Management is Now GA (#121)

  • Makes locally attached (non-network attached) storage available as a persistent volume source.
  • Allows users to take advantage of the typically cheaper and improved performance of persistent local storage
    kubernetes/kubernetes: #73525, #74391, #74769
    kubernetes/enhancements: #121 [kep]

Pid Limiting is Graduating to Beta (#757)

  • Prevents a pod from starving pid resource
  • Ability to isolate pid resources pod-to-pod and node-to-pod
    kubernetes/kubernetes: #73651
    kubernetes/enhancements: #757 [kep]

Pod Priority and Preemption in Kubernetes (#564)

  • Pod priority and preemption enables Kubernetes scheduler to schedule more important Pods first and when cluster is out of resources, it removes less important pods to create room for more important ones. The importance is specified by priority.
    kubernetes/kubernetes: #73498, #73555, #74465
    kubernetes/enhancements: #564 [kep]

Pod Ready++ (#580)

  • Introduces extension point for external feedback on pod readiness.
    kubernetes/kubernetes: #74434,
    kubernetes/enhancements: #580 [kep]

Kubeadm: Automate certificate copy between control planes in HA setups

  • Joining control plane nodes to a HA cluster can now be simplified by enabling the optional automatic copy of certificates from an existing control plane node.
  • You can now use kubeadm init --experimental-upload-certs and kubeadm join --experimental-control-plane --certificate-key.
    kubernetes/kubeadm: #1373
    kubernetes/enhancements: #357 [kep]

Kubeadm: Expose the kubeadm join workflow as phases

  • The kubeadm join command can now be used in phases. Similar to the work that was done for kubeadm init in 1.13, in 1.14 the join phases can be now executed step-by-step/selectively using the kubeadm join phase sub-command. This makes it possible to further customize the workflow of joining nodes to the cluster.
    kubernetes/kubeadm: #1204
    kubernetes/enhancements: kep

Known Issues

  • There is a known issue coredns/coredns#2629 in CoreDNS 1.3.1, wherein if the Kubernetes API shuts down while CoreDNS is connected, CoreDNS will crash. The issue is fixed in CoreDNS 1.4.0 in coredns/coredns#2529.
  • Kubelet might fail to restart if an existing flexvolume mounted pvc contains a large number of directories, or is full. #75019

Urgent Upgrade Notes

(No, really, you MUST do this before you upgrade)

  • kube-apiserver:
  • Default RBAC policy no longer grants access to discovery and permission-checking APIs (used by kubectl auth can-i) to unauthenticated users. Upgraded clusters preserve prior behavior, but cluster administrators wishing to grant unauthenticated users access in new clusters will need to explicitly opt-in to expose the discovery and/or permission-checking APIs:
    • kubectl create clusterrolebinding anonymous-discovery --clusterrole=system:discovery --group=system:unauthenticated
    • kubectl create clusterrolebinding anonymous-access-review --clusterrole=system:basic-user --group=system:unauthenticated
  • The deprecated --storage-versions flag has been removed. The storage versions will always be the default value built-in the kube-apiserver binary. (#67678, @caesarxuchao)
  • The deprecated --repair-malformed-updates flag has been removed (#73663, @danielqsj)
  • The /swaggerapi/* schema docs, deprecated since 1.7, have been removed in favor of the /openapi/v2 schema docs. (#72924, @liggitt)
  • The /swagger.json and /swagger-2.0.0.pb-v1 schema documents, deprecated since v1.10, have been removed in favor of /openapi/v2 (#73148, @liggitt)
  • kube-apiserver now only aggregates openapi schemas from /openapi/v2 endpoints of aggregated API servers. The fallback to aggregate from /swagger.json has been removed. Ensure aggregated API servers provide schema information via /openapi/v2 (available since v1.10). (#73441, @roycaihw)
  • The OpenAPI definitions with the prefix "io.k8s.kubernetes.pkg" (deprecated since 1.9) have been removed. (#74596, @sttts)

  • The ValidateProxyRedirects feature was promoted to Beta and enabled by default. This feature restricts redirect-following from the apiserver to same-host redirects. If nodes are configured to respond to CRI streaming requests on a different host interface than what the apiserver makes requests on (only the case if not using the built-in dockershim & setting the kubelet flag --redirect-container-streaming=true), then these requests will be broken. In that case, the feature can be temporarily disabled until the node configuration is corrected. We suggest setting --redirect-container-streaming=false on the kubelet to avoid issues.(#72552, @tallclair)

  • kubectl

  • The deprecated --show-all flag to kubectl get has been removed (#69255, @Pingan2017)

  • kubelet
  • The deprecated --experimental-fail-swap-on flag has been removed (#69552, @Pingan2017)

  • Health check (liveness & readiness) probes using an HTTPGetAction will no longer follow redirects to different hostnames from the original probe request. Instead, these non-local redirects will be treated as a Success (the documented behavior). In this case an event with reason "ProbeWarning" will be generated, indicating that the redirect was ignored. If you were previously relying on the redirect to run health checks against different endpoints, you will need to perform the healthcheck logic outside the Kubelet, for instance by proxying the external endpoint rather than redirecting to it. (#75416, @tallclair)

  • client-go
  • The deprecated versionless API group accessors (like clientset.Apps()) have been removed. Use an explicit version instead (like clientset.AppsV1()) (#74422, @liggitt)

  • The disk-cached discovery client is moved from k8s.io/client-go/discovery to k8s.io/client-go/discovery/cached/disk.
    The memory-cached discovery client is moved from k8s.io/client-go/discovery/cached to k8s.io/client-go/discovery/cached/memory.
    (#72214, @caesarxuchao)

  • kubeadm

  • kubeadm alpha preflight and kubeadm alpha preflight node are removed; you can now use kubeadm join phase preflight (#73718, @fabriziopandini)

  • The deprecated taints node.alpha.kubernetes.io/notReady and node.alpha.kubernetes.io/unreachable are no longer supported or adjusted. These uses should be replaced with node.kubernetes.io/not-ready and node.kubernetes.io/unreachable
    (#73001, @shivnagarajan)

  • Any Prometheus queries that match pod_name and container_name labels (e.g. cadvisor or kubelet probe metrics) should be updated to use pod and container instead. pod_name and container_name labels will be present alongside pod and container labels for one transitional release and removed in the future.
    (#69099, @ehashman)

Deprecations

  • kubectl
  • kubectl convert is deprecated and will be removed in v1.17.

  • The --export flag for the kubectl get command is deprecated and will be removed in v1.18. (#73787, @soltysh)

  • kubelet
  • OS and Arch information is now recorded in kubernetes.io/os and kubernetes.io/arch labels on Node objects. The previous labels (beta.kubernetes.io/os and beta.kubernetes.io/arch) are still recorded, but are deprecated and targeted for removal in v1.18. (#73333, @yujuhong)

  • The --containerized flag is deprecated and will be removed in a future release (#74267, @dims)

  • hyperkube
  • The --make-symlinks flag is deprecated and will be removed in a future release. (#74975, @dims)
  • API
  • Ingress resources are now available via networking.k8s.io/v1beta1. Ingress resources in extensions/v1beta1 are deprecated and will no longer be served in v1.18. Existing persisted data is available via the new API group/version (#74057, @liggitt)
  • NetworkPolicy resources will no longer be served from extensions/v1beta1 in v1.16. Migrate use to the networking.k8s.io/v1 API, available since v1.8. Existing persisted data can be retrieved via the networking.k8s.io/v1 API.
  • PodSecurityPolicy resources will no longer be served from extensions/v1beta1 in v1.16. Migrate to the policy/v1beta1 API, available since v1.10. Existing persisted data can be retrieved via the policy/v1beta1 API.
  • DaemonSet, Deployment, and ReplicaSet resources will no longer be served from extensions/v1beta1, apps/v1beta1, or apps/v1beta2 in v1.16. Migrate to the apps/v1 API, available since v1.9. Existing persisted data can be retrieved via the apps/v1 API.
  • PriorityClass resources have been promoted to scheduling.k8s.io/v1 with no changes. The scheduling.k8s.io/v1beta1 and scheduling.k8s.io/v1alpha1 versions are now deprecated and will stop being served by default in v1.17. (#73555, #74465, @bsalamat)
  • The export query parameter for list API calls is deprecated and will be removed in v1.18 (#73783, @deads2k)
  • The following features are now GA, and the associated feature gates are deprecated and will be removed in v1.15:
  • CustomPodDNS
  • HugePages
  • MountPropagation
  • PersistentLocalVolumes
  • CoreDNS: The following directives or keywords are deprecated and will be removed in v1.15:
  • upstream option of kubernetes plugin, becoming default behavior in v1.15.

  • proxy plugin replaced by forward plugin

Removed and deprecated metrics

Removed metrics

  • reflector_items_per_list
  • reflector_items_per_watch
  • reflector_last_resource_version
  • reflector_list_duration_seconds
  • reflector_lists_total
  • reflector_short_watches_total
  • reflector_watch_duration_seconds
  • reflector_watches_total

Deprecated metrics

  • rest_client_request_latency_seconds -> rest_client_request_duration_seconds
  • apiserver_proxy_tunnel_sync_latency_secs -> apiserver_proxy_tunnel_sync_duration_seconds
  • scheduler_scheduling_latency_seconds -> scheduler_scheduling_duration_seconds
  • kubelet_pod_worker_latency_microseconds -> kubelet_pod_worker_duration_seconds
  • kubelet_pod_start_latency_microseconds -> kubelet_pod_start_duration_seconds
  • kubelet_cgroup_manager_latency_microseconds -> kubelet_cgroup_manager_duration_seconds
  • kubelet_pod_worker_start_latency_microseconds -> kubelet_pod_worker_start_duration_seconds
  • kubelet_pleg_relist_latency_microseconds -> kubelet_pleg_relist_duration_seconds
  • kubelet_pleg_relist_interval_microseconds -> kubelet_pleg_relist_interval_seconds
  • kubelet_eviction_stats_age_microseconds -> kubelet_eviction_stats_age_seconds
  • kubelet_runtime_operations -> kubelet_runtime_operations_total
  • kubelet_runtime_operations_latency_microseconds -> kubelet_runtime_operations_duration_seconds
  • kubelet_runtime_operations_errors -> kubelet_runtime_operations_errors_total
  • kubelet_device_plugin_registration_count -> kubelet_device_plugin_registration_total
  • kubelet_device_plugin_alloc_latency_microseconds -> kubelet_device_plugin_alloc_duration_seconds
  • docker_operations -> docker_operations_total
  • docker_operations_latency_microseconds -> docker_operations_latency_seconds
  • docker_operations_errors -> docker_operations_errors_total
  • docker_operations_timeout -> docker_operations_timeout_total
  • network_plugin_operations_latency_microseconds -> network_plugin_operations_latency_seconds
  • sync_proxy_rules_latency_microseconds -> sync_proxy_rules_latency_seconds
  • apiserver_request_count -> apiserver_request_total
  • apiserver_request_latencies -> apiserver_request_latency_seconds
  • apiserver_request_latencies_summary -> apiserver_request_latency_seconds
  • apiserver_dropped_requests -> apiserver_dropped_requests_total
  • etcd_helper_cache_hit_count -> etcd_helper_cache_hit_total
  • etcd_helper_cache_miss_count -> etcd_helper_cache_miss_total
  • etcd_helper_cache_entry_count -> etcd_helper_cache_entry_total
  • etcd_request_cache_get_latencies_summary -> etcd_request_cache_get_latency_seconds
  • etcd_request_cache_add_latencies_summary -> etcd_request_cache_add_latency_seconds
  • etcd_request_latencies_summary -> etcd_request_latency_seconds
  • transformation_latencies_microseconds -> transformation_latencies_seconds
  • data_key_generation_latencies_microseconds -> data_key_generation_latencies_seconds

Notable Features

  • Increased the histogram resolution of the API server client certificate to accommodate short-lived (< 6h) client certificates. (#74806, @mxinden)
  • Updated to use golang 1.12 (#74632, @cblecker)
  • The RunAsGroup feature has been promoted to beta and enabled by default. PodSpec and PodSecurityPolicy objects can be used to control the primary GID of containers on supported container runtimes. (#73007, @krmayankk)
  • Added the same information to an init container as a standard container in a pod when using PodPresets. (#71479, @soggiest)
  • kube-conformance image will now run ginkgo with the --dryRun flag if the container is run with the environment variable E2E_DRYRUN set. (#74731, @johnSchnake)
  • Introduced dynamic volume provisioning shim for CSI migration (#73653, @ddebroy)
  • Applied resources from a directory containing kustomization.yaml (#74140, @Liujingfang1)
  • kubeadm: Allowed to download certificate secrets uploaded by init or upload-certs phase, allowing to transfer certificate secrets (certificates and keys) from the cluster to other master machines when creating HA deployments. (#74168, @ereslibre)
  • The --quiet option to kubectl run now suppresses resource deletion messages emitted when the --rm option is specified. (#73266, @awh)
  • Added Custom Resource support to kubectl autoscale (#72678, @rmohr)
  • Cinder volume limit can be now configured from node too (#74542, @gnufied)
  • It is now possible to combine the -f and -l flags in kubectl logs (#67573, @m1kola)
  • New conformance tests added for API Aggregation. (#63947, @jennybuckley)
  • Moved fluentd-elasticsearch addon images to community controlled location (#73819, @coffeepac)
  • Removed local etcd members from the etcd cluster when kubeadm reset (#74112, @pytimer)
  • kubeadm will now not fail preflight checks when running on >= 5.0 Linux kernel (#74355, @brb)
  • Optimized scheduler cache snapshot algorithm to improve scheduling throughput. (#74041, @bsalamat)
  • It is now possible to upload certificates required to join a new control-plane to kubeadm-certs secret using the flag --experimental-upload-certs on init or upload-certs phase. (#73907, @yagonobre)
    @RobertKrawitz)
  • kubectl auth reconcile now outputs details about what changes are being made (#71564, @liggitt)
  • Added Kustomize as a subcommand in kubectl (#73033, @Liujingfang1)
  • Added kubelet_node_name metrics. (#72910, @danielqsj)
  • Updated AWS SDK to v1.16.26 for ECR PrivateLink support (#73435, @micahhausler)
  • Expanded kubectl wait to work with more types of selectors. (#71746, @rctl)
    (#72832, @MrHohn)
  • Added configuration for AWS endpoint fine control: (#72245, @ampsingram)
  • The CoreDNS configuration now has the forward plugin for proxy in the default configuration instead of the proxy plugin. (#73267, @rajansandeep)
  • Added alpha field storageVersionHash to the discovery document for each resource. Its value must be treated as opaque by clients. Only equality comparison on the value is valid. (#73191, @caesarxuchao)
  • If you are running the cloud-controller-manager and you have the pvlabel.kubernetes.io alpha Initializer enabled, you must now enable PersistentVolume labeling using the PersistentVolumeLabel admission controller instead. You can do this by adding PersistentVolumeLabel in the --enable-admission-plugins kube-apiserver flag. (#73102, @andrewsykim)
  • kubectl supports copying files with wild card (#72641, @dixudx)
  • kubeadm now attempts to detect an installed CRI by its usual domain socket, so that --cri-socket can be omitted from the command line if Docker is not used and there is a single CRI installed. (#69366, @rosti)
  • Install CSINodeInfo and CSIDriver CRDs in the local cluster. (#72584, @xing-yang)
  • Node OS/arch labels are promoted to GA (#73048, @yujuhong)
  • Add support for max attach limit for Cinder (#72980, @gnufied)
  • Enable mTLS encription between etcd and kube-apiserver in GCE (#70144, @wenjiaswe)
  • Add ResourceVersion as a precondition for delete in order to ensure a delete fails if an unobserved change happens to an object. (#74040, @ajatprabha)
  • Support collecting pod logs under /var/log/pods/NAMESPACE_NAME_UID to stackdriver with k8s_pod resource type. (#74502, @Random-Liu)
  • Change CRI pod log directory from /var/log/pods/UID to /var/log/pods/NAMESPACE_NAME_UID. (#74441, @Random-Liu)
  • Promote RuntimeClass to beta, and enable by default. (#75003, @tallclair)
  • New dry_run metric label (indicating the value of the dryRun query parameters) into the metrics: (#74997, @jennybuckley)
  • GCE: bump COS image version to cos-beta-73-11647-64-0 (#75149, @yguo0905)
  • Alpha support for ephemeral CSI inline volumes that are embedded in pod specs. (#74086, @vladimirvivien)

API Changes

  • [CRI] Added a new field called runtime_handler into PodSandbox and PodSandboxStatus to track the RuntimeClass information of a pod. (#73833, @haiyanmeng)

Detailed Bug Fixes And Changes

API Machinery

  • client-go: PortForwarder.GetPorts() now contain correct local port if no local port was initially specified when setting up the port forwarder (#73676, @martin-helmich)
  • Fixed an issue with missing apiVersion/kind in object data sent to admission webhooks (#74448, @liggitt)
  • Prometheus metrics for crd_autoregister, crd_finalizer and crd_naming_condition_controller are exported. (#71767, @roycaihw)
  • Fix admission metrics in seconds. (#72343, @danielqsj)
  • When a watch is closed by an HTTP2 load balancer and we are told to go away, skip printing the message to stderr by default.
  • Speedup kubectl by >10 when calling out to kube-apiserver for discovery information. (#73345, @sttts)
  • Fix watch to not send the same set of events multiple times causing watcher to go back in time (#73845, @wojtek-t)
    (#73277, @smarterclayton)
  • Fix kube-apiserver not to create default/kubernetes service endpoints before it reports readiness via the /healthz and therefore is ready to serve requests. Also early during startup old endpoints are remove which might be left over from a previously crashed kube-apiserver. (#74668, @sttts)
  • Add a configuration field to shorten the timeout of validating/mutating admission webhook call. The timeout value must be between 1 and 30 seconds. Default to 30 seconds when unspecified. (#74562, @roycaihw)
  • The apiserver, including both the kube-apiserver and apiservers built with the generic apiserver library, will now return 413 RequestEntityTooLarge error if a json patch contains more than 10,000 operations. (#74000, @caesarxuchao)
  • Fixed an error processing watch events when running skewed apiservers (#73482, @liggitt)
  • jsonpath expressions containing [start:end:step] slice are now evaluated correctly (#73149, @liggitt)
  • metadata.deletionTimestamp is no longer moved into the future when issuing repeated DELETE requests against a resource containing a finalizer. (#73138, @liggitt)
  • Fix kube-apiserver not to create default/kubernetes service endpoints before it reports readiness via the /healthz and therefore is ready to serve requests. Also early during startup old endpoints are remove which might be left over from a previously crashed kube-apiserver. (#74668, @sttts)
  • watch.Until now works for long durations. (#67350, @tnozicka)
  • Added duration metric for CRD webhook converters. (#74376, @mbohlool)
  • Fix keymutex issues which may crash in some platforms. (#74348, @danielqsj)
  • Considerably reduced the CPU load in kube-apiserver while aggregating OpenAPI specifications from aggregated API servers. (#71223, @sttts)
  • Fix graceful apiserver shutdown to not drop outgoing bytes before the process terminates. (#72970, @sttts)

Apps

  • Adds deleting pods created by DaemonSet assigned to not existing nodes. (#73401, @krzysztof-jastrzebski)
  • Pod eviction now honors graceful deletion by default if no delete options are provided in the eviction request. (#72730, @liggitt)

Auth

  • Add kubectl auth can-i --list option, which allows users to know what actions they can do in specific namespaces. (#64820, @WanLinghao)
  • The rules field in RBAC Role and ClusterRole objects is now correctly reported as optional in the openapi schema. (#73250, @liggitt)
  • system:kube-controller-manager and system:kube-scheduler users are now permitted to perform delegated authentication/authorization checks by default RBAC policy (#72491, @liggitt)
  • Error messages returned in authentication webhook status responses are now correctly included in the apiserver log (#73595, @liggitt)
  • Fixed use of webhook admission plugins with multi-version custom resources (#74154, @mbohlool)

AWS

  • Prevent AWS Network Load Balancer security groups ingress rules to be deleted by ensuring target groups are tagged. (#73594, @masterzen)
  • AWS ELB health checks will now use HTTPS/SSL protocol for HTTPS/SSL backends. (#70309, @2rs2ts)

Azure

CLI

  • Fixed --help flag parsing (#74682, @soltysh)
  • Fixed a bug where kubectl describe cannot obtain the event messages for a static pod (#74156, @gaorong)
  • Fixed panic when performing a set env operation on a --local resource (#65636, @juanvallejo)
  • Missing directories listed in a user's PATH are no longer considered errors and are instead logged by the kubectl plugin list command when listing available plugins. (#73542, @juanvallejo)
  • Now users could get object info like:

bash a. kubectl get pod test-pod -o custom-columns=CONTAINER:.spec.containers[0:3].name b. kubectl get pod test-pod -o custom-columns=CONTAINER:.spec.containers[-2:].name

(#73063, @WanLinghao)

  • The kubectl api-resources command will no longer fail to display any resources on a single failure (#73035, @juanvallejo)
  • kubectl loads config file once and uses persistent client config (#71117, @dixudx)
  • Print SizeLimit of EmptyDir in kubectl describe pod outputs. (#69279, @dtaniwaki)
  • kubectl delete --all-namespaces is now a recognized flag. (#73716, @deads2k)

Cloud Provider

  • Fixed a bug that caused PV allocation on non-English vSphere installations to fail (#73115, @alvaroaleman)

Cluster Lifecycle

  • kubeadm: fixed nil pointer dereference caused by a bug in url parsing (#74454, @bart0sh)
  • CoreDNS adds readinessProbe which prevents loadbalancing to unready pods, and also allows rolling updates to work as expected. (#74137, @rajansandeep)
  • kubeadm no longer allows using v1alpha3 configs for anything else than converting them to v1beta1. (#74025, @rosti)
  • kubeadm: allow the usage of --kubeconfig-dir and --config flags on kubeadm init (#73998, @yagonobre)
  • kubeadm: all master components are now exclusively relying on the PriorityClassName pod spec for annotating them as cluster critical components. Since scheduler.alpha.kubernetes.io/critical-pod annotation is no longer supported by Kubernetes 1.14 this annotation is no longer added to master components. (#73857, @ereslibre)
  • kubeadm no longer dumps backtrace if it fails to remove the running containers on reset. (#73951, @rosti)
  • kubeadm: fixed a bug in the underlying library for diff related to characters like '%' (#73941, @neolit123)
  • Scale max-inflight limits together with master VM sizes. (#73268, @wojtek-t)
  • kubeadm reset: fixed a crash caused by the absence of a configuration file (#73636, @bart0sh)
  • CoreDNS is now version 1.3.1 (#73610, @rajansandeep)
  • kubeadm: When certificates are present joining a new control plane make sure that they match at least the required SANs (#73093, @ereslibre)
  • kubeadm: add back --cert-dir option for kubeadm init phase certs sa (#73239, @mattkelly)
  • kubeadm: explicitly wait for etcd to have grown when joining a new control plane (#72984, @ereslibre)
  • kubeadm: pull images when joining a new control plane instance (#72870, @MalloZup)
  • Couldn't create a kubeconfig; the CA files couldn't be loaded: failed to load key: couldn't load the private key file /etc/kubernetes/pki/ca.key: open /etc/kubernetes/pki/ca.key: no such file or directory (#75431, @fabriziopandini)
  • Exit kube-proxy when configuration file changes (#59176, @dixudx)
  • kube-addon-manager was updated to v9.0, and now uses kubectl v1.13.2 and prunes workload resources via the apps/v1 API (#72978, @liggitt)
  • kubeadm: Allow certain certs/keys to be missing on the secret when transferring secrets using --experimental-upload-certs feature (#75415, @ereslibre)

GCP

  • Fixed liveness probe in fluentd-gcp cluster addon (#74522, @Pluies)
  • Reduce GCE log rotation check from 1 hour to every 5 minutes. Rotation policy is unchanged (new day starts, log file size > 100MB). (#72062, @jpbetz)

Network

Node

  • Fixed help message for --container-runtime-endpoint: only unix socket is support on Linux. (#74712, @feiskyer)
  • Image garbage collection no longer fails for images with only one tag but more than one repository associated. (#70647, @corvus-ch)
  • Re-issue Allocate grpc calls before starting a container that requests device-plugin resources if the cached state is missing. (#73824, @jiayingz)
  • [CRI] Add a new field called runtime_handler into PodSandbox and PodSandboxStatus to track the RuntimeClass information of a pod. (#73833, @haiyanmeng)
  • Kubelet now tries to stop containers in unknown state once before restart or remove. (#73802, @Random-Liu)
  • when pleg channel is full, discard events and record its count (#72709, @changyaowei)
  • Fixed the unexpected NotReady status when Node's iops is full if the runtime is dockershim. (#74389, @answer1991)
  • Fixed #73264 cpuPeriod was not reset, but used as set via flag, although it was disabled via alpha gate (#73342, @szuecs)
  • Update kubelet CLI summary documentation and generated webpage (#73256, @deitch)
  • Set a low oom_score_adj for containers in pods with system-critical priorities (#73758, @sjenning)
  • kubelet: Resolved hang/timeout issues when running large numbers of pods with unique ConfigMap/Secret references (#74755, @liggitt)
  • Events reported for container creation, start, and stop now report the container name in the message and are more consistently formatted. (#73892, @smarterclayton)
  • remove stale OutOfDisk condition from kubelet side (#72507, @dixudx)
  • Fixed the setting of NodeAddresses when using the vSphere CloudProvider and nodes that have multiple IP addresses. (#70805, @danwinship)
  • Fixed dockershim panic issues when deleting docker images. (#75367, @feiskyer)
  • Kubelet no longer watches ConfigMaps and Secrets for terminated pods, in worst scenario causing it to not be able to send other requests to kube-apiserver (#74809, @oxddr)
  • A new TaintNodesByCondition admission plugin taints newly created Node objects as "not ready", to fix a race condition that could cause pods to be scheduled on new nodes before their taints were updated to accurately reflect their reported conditions. This admission plugin is enabled by default if the TaintNodesByCondition feature is enabled. (#73097, @bsalamat)
  • kubelet now accepts pid=<number> in the --system-reserved and --kube-reserved options to ensure that the specified number of process IDs will be reserved for the system as a whole and for Kubernetes system daemons respectively. Please reference Kube Reserved and System Reserved in Reserve Compute Resources for System Daemons in the Kubernetes documentation for general discussion of resource reservation. To utilize this functionality, you must set the feature gate SupportNodePidsLimit=true (#73651

Scheduling

  • Improve fairness of the scheduling queue by placing pods which are attempted recently behind other pods with the same priority. (#73700, @denkensk)
  • Improve scheduler robustness to ensure that unschedulable pods are reconsidered for scheduling when appropriate. (#73700, #72558, @denkensk, #73078, @Huang-Wei)

Storage

  • Fixed scanning of failed iSCSI targets. (#74306, @jsafrane)
  • StorageOS volume plugin updated to fix an issue where volume mount succeeds even if request to mount via StorageOS API fails. (#69782, @darkowlzz)
  • Ensure directories on volumes are group-executable when using fsGroup (#73533, @mxey)
  • Update CSI version to 1.1 (#75391, @gnufied)
  • Ensure that volumes get provisioned based on the zone information provided in allowedTopologies. (#72731, @skarthiksrinivas)
  • Extends the VolumeSubpathEnvExpansion alpha feature to support environment variable expansion (#71351, @kevtaylor)
  • Fixed a bug that prevented deletion of dynamically provisioned volumes in Quobyte backends. (#68925, @casusbelli)

Testing

  • e2e storage tests run faster and are easier to read (#72434, @pohly)
  • e2e.test now rejects unknown --provider values instead of merely warning about them. An empty provider name is not accepted anymore and was replaced by skeleton (a provider with no special behavior). (#73402, @pohly)
  • Update to go1.11.5 (#73326, @ixdy)
  • Update to use go1.12.1 (#75413, @BenTheElder)
  • e2e tests that require SSH may be used against clusters that have nodes without external IP addresses by setting the environment variable KUBE_SSH_BASTION to the host:port of a machine that is allowed to SSH to those nodes. The same private key that the test would use is used for the bastion host. The test connects to the bastion and then tunnels another SSH connection to the node. (#72286, @smarterclayton)
  • PidPressure evicts pods from lowest priority to highest priority (#72844, @dashpole)
  • Split up the mondo kubernetes-test tarball into kubernetes-test-portable and kubernetes-test-{OS}-{ARCH} tarballs. (#74065, @ixdy)

VMware

Windows

Support for Windows nodes and Windows containers is going stable.

Support for Group Managed Service Accounts (GMSA) for Windows containers in Kubernetes. GMSA are a specific type of Active Directory account that provides automatic password management, simplified service principal name (SPN) management, and the ability to delegate the management to other administrators across multiple servers.

  • Fixed smb remount and unmount issues on Windows (#73661, @andyzhangx, #75087, @andyzhangx)
  • Add network stats for Windows nodes and containers (#74788, @feiskyer)
  • The new test [sig-network] DNS should provide /etc/hosts entries for the cluster [LinuxOnly] [Conformance] will validate the host entries set in the /etc/hosts file (pod's FQDN and hostname), which should be managed by Kubelet. (#72729, @bclau)
  • Allow the kubelet to pass Windows GMSA credentials down to Docker (#73726, @wk8)
  • Added kube-proxy support for overlay networking and DSR in Windows and new flags for network-name, source-vip, and enable-dsr. (#70896, @ksubrmnn)
  • windows: Ensure graceful termination when being run as windows service (#73292, @steffengy)
  • vSphere cloud provider correctly retrieves the VM's UUID when running on Windows (#71147, @benmoss)
  • Kubelet: add usageNanoCores from CRI stats provider (#73659, @feiskyer)
  • Introduced support for Windows nodes into the cluster bringup scripts for GCE. (#73442, @pjh)
  • Add network stats for Windows nodes and pods. (#70121, @feiskyer)
  • CoreDNS is only officially supported on Linux at this time. As such, when kubeadm is used to deploy this component into your kubernetes cluster, it will be restricted (using nodeSelectors) to run only on nodes with that operating system. This ensures that in clusters which include Windows nodes, the scheduler will not ever attempt to place CoreDNS pods on these machines, reducing setup latency and enhancing initial cluster stability. (#69940, @MarcPow)

External Dependencies

  • Default etcd server and client have been updated to v3.3.10. (#71615, #70168)
  • The list of validated docker versions has changed. 1.11.1 and 1.12.1 have been removed. The current list is 1.13.1, 17.03, 17.06, 17.09, 18.06, 18.09. (#72823, #72831)
  • The default Go version was updated to 1.12.1. (#75422)
  • CNI has been updated to v0.7.5 (#75455)
  • CSI has been updated to v1.1.0. (#75391)
  • The dashboard add-on has been updated to v1.10.1. (#72495)
  • Cluster Autoscaler has been updated to v1.14.0 (#75480)
  • kube-dns is unchanged at v1.14.13 since Kubernetes 1.12 (#68900)
  • Influxdb is unchanged at v1.3.3 since Kubernetes 1.10 (#53319)
  • Grafana is unchanged at v4.4.3 since Kubernetes 1.10 (#53319)
  • Kibana has been upgraded to v6.6.1. (#71251)
  • CAdvisor has been updated to v0.33.1 (#75140)
  • fluentd-gcp-scaler is unchanged at v0.5.0 since Kubernetes 1.13 (#68837)
  • Fluentd in fluentd-elasticsearch has been upgraded to v1.3.3 (#71180)
  • fluentd-elasticsearch has been updated to v2.4.0 (#71180)
  • The fluent-plugin-kubernetes_metadata_filter plugin in fluentd-elasticsearch has been updated to v2.1.6 (#71180)
  • fluentd-gcp is unchanged at v3.2.0 since Kubernetes 1.13 (#70954)
  • OIDC authentication is unchanged at coreos/go-oidc v2 since Kubernetes 1.10 (#58544)
  • Calico is unchanged at v3.3.1 since Kubernetes 1.13 (#70932)
  • crictl on GCE is unchanged at v1.12.0 since Kubernetes 1.13 (#69033)
  • CoreDNS has been updated to v1.3.1 (#73610)
  • event-exporter has been updated to v0.2.3 (#67691)
  • Es-image has been updated to Elasticsearch 6.6.1 (#71252)
  • metrics-server remains unchanged at v0.3.1 since Kubernetes 1.12 (#68746)
  • GLBC remains unchanged at v1.2.3 since Kubernetes 1.12 (#66793)
  • Ingress-gce remains unchanged at v1.2.3 since Kubernetes 1.12 (#66793)
  • ip-masq-agen remains unchanged at v2.1.1 since Kubernetes 1.12 (#67916)- v1.14.0-rc.1
  • v1.14.0-beta.2
  • v1.14.0-beta.1
  • v1.14.0-alpha.3
  • v1.14.0-alpha.2
  • v1.14.0-alpha.1


Leads, the CHANGELOG-1.14.md has been bootstrapped with v1.14.0 release notes and you may edit now as needed.


Published by anago, the Kubernetes Release Tool

Reply all
Reply to author
Forward
This conversation is locked
You cannot reply and perform actions on locked conversations.
0 new messages