ContextQMD
Back to Kubernetes

CHANGELOG 1.9

CHANGELOG/CHANGELOG-1.9.md

1.36.0313.7 KB
Original Source
<!-- BEGIN MUNGE: GENERATED_TOC --> <!-- END MUNGE: GENERATED_TOC --> <!-- NEW RELEASE NOTES ENTRY -->

v1.9.11

Documentation

Downloads for v1.9.11

filenamesha512 hash
kubernetes.tar.gz4f04d07e31ca49c033680cabb011de1d893db94213ee3d68cb4345be435012279682b299f2b44dce596bcfd6806a253e9544e263d5a7003ccbab92da8dcfed24
kubernetes-src.tar.gz49e1134c365d2a6662bc85e055960581909704508e67aad920c46662666c230430c44bd08ce242825a2bb316b3220fbcbdf40232d8c42491cb3aa27a2ce8def8

Client Binaries

filenamesha512 hash
kubernetes-client-darwin-386.tar.gz1136c55df76d99abaac0a9a7543a72d02af6f445434b21f708e7c146f073b700b03535389d9e4d8bff1985f8a71d4571a30c4c813346db06c6a38e37e720a558
kubernetes-client-darwin-amd64.tar.gza83ff68d1ead35885a5ac6f3f1457e53d67448210bdccd96bbede823e9c1d93de9344f54dd714bd5779d20ed306d42e174711fe4928ffc9881cd2975a39f387f
kubernetes-client-linux-386.tar.gz9a34cf0015b57d8c85f2f995a2710725748142c4eee9050af85f4295fce66f8287fba4da1825027a72d452e7d89081fb28be26935ce21f1d5d6ccfe29f6ad1c6
kubernetes-client-linux-amd64.tar.gzc5601d6564b7168dd93725609b9b46018243dd52eb4fad40f977da2af2fb8554250b0381bfd2edce846d38c8ff264c29bb8cd89f4988a89c630dc1ca124e0be8
kubernetes-client-linux-arm.tar.gz04b8db6de27450ed5f2df162df759cad0a35d4d3317d45b29c486f4b29c040551393788ddb18711bfcb7a8bdb6d530084641640a1e72955126050e50c2fade9d
kubernetes-client-linux-arm64.tar.gzd46a460a699b778d5bb2f743b077c8cb3ad430dd426b953b8a6cdddb35d6a6188692dd8449ad40e7a1b17e2c63ed8c240c50b85cd8fafd71c501b8f5f67eaa7b
kubernetes-client-linux-ppc64le.tar.gz846d095b236990030292b4abbfd279a7264355c8ad778aafb42c58c6d694227b3dba205ffd38146af055ee9e00280fac08c82acdff3228f3657b4838af8fbaf4
kubernetes-client-linux-s390x.tar.gzd4637eb0336c4333644ebfa5714b9b9edc50ee942c14e044f9300c3cf61353bbb88766f9cf1eedd042c7a6b8573a212345d73bde1a093e5e31c85cdf45132957
kubernetes-client-windows-386.tar.gz00b0b9c39ce4b05685d7047adb8b06269f439fef6b39c35c95c67bba7b948507c8ec947cafef660a7fa943e67aa9a1b72555250d5af6edc664053b8ed0365ca6
kubernetes-client-windows-amd64.tar.gzac5d530c61044ee03fa2b9aad02e66c5c4715236aeb8d3cf71d740ac41c8cc8973afd5b5d9c6b3ef3dd7e5a9e90b75b5219132809b8b5ccf6e14181ca7f3a57b

Server Binaries

filenamesha512 hash
kubernetes-server-linux-amd64.tar.gz0d1d6bf7a06375c3698a5a513de63c15201a064fd43f80913a1eccb4062f46184e952814e75e66dfafcfb449b6f21f8ea59dd49b3695b1172095d1016ae1c543
kubernetes-server-linux-arm.tar.gza63f310a8af4ec4262ecf9f5d133b55207163d55a7ac0ae1c4be9e97293ab26004d7deffcdda803e3db7401419f8c26834ef8c5c2f670e238633d3ca268aebf7
kubernetes-server-linux-arm64.tar.gze26ef5a4c72cf873894ba8a7cccfb179ecc9c8e6bbabe3f09b27bbbc83562074d45f67e1ed083b7489ea916061956b96de12c4336e5d8da69ea6e4c48648126f
kubernetes-server-linux-ppc64le.tar.gz5f5c7a29f47f91c5659e7dae9d1e67121fdaa7a375a2832ff94b52443c7b050d01a976a33a790ee237532711f311a00e3ea0c1e67f35851477cedb201e45b547
kubernetes-server-linux-s390x.tar.gz09e0bd18f193bc0fc3594d7899f74ebfb8635c695661c4f9ee79e0f497e24fdafeb612b6f15a928b720fec8da691f79c274c30d76937321958abc2e8b184c094

Node Binaries

filenamesha512 hash
kubernetes-node-linux-amd64.tar.gz078e2a040d147a768744b5caa495b14bc6d98c98555116229fd6824fa94568733c96efe830a613245e2dea77fd384847d7a8bf31035175b6b7db6b933dcc2a6b
kubernetes-node-linux-arm.tar.gz5a0f8027ed446e1cd564416dd669b36b324d2b30759be01c3746c460e37b1a48ff5f723868c6d6e06f0e2a4550b29e08fade38864b18464e43fa85ec32d9b5eb
kubernetes-node-linux-arm64.tar.gze45bcce3e4e7a2788d4e8dab032aaff30a57150d489c1d143780f5c17c1539ae8bc3c6090cfb08ff77feefec4605034412b87ae5376f2a7e83cc46b3db0a899c
kubernetes-node-linux-ppc64le.tar.gz411fc9eeee0fcf0b0c616c871aa6d1e8f102fe7b6c3cde8bbad89af64a2014342574ea9650dee55db15a4444cc7999d657448f7b65c7bfb0b0cfcc222b1a9a68
kubernetes-node-linux-s390x.tar.gzff6742741999851df1494467227f47c1e319004da05b7b0d1cdd09c602c64155de18863ea3289998f8dd204aac2c8a99529c84462eff257d6ddd65e5a62db527
kubernetes-node-windows-amd64.tar.gzb17127e1bae7521fe5f99e600bf7bf961beda6dea7a9934c39bf2f07f24379644b1226b139878fe355f826979178f487e02e3aef780aa08db125da4b2989aeda

Changelog since v1.9.10

Action Required

  • action required: the API server and client-go libraries have been fixed to support additional non-alpha-numeric characters in UserInfo "extra" data keys. Both should be updated in order to properly support extra data containing "/" characters or other characters disallowed in HTTP headers. (#65799, @dekkagaijin)

Other notable changes

  • Add fallbacks to ARM API when getting empty node IP from Azure IMDS (#69077, @feiskyer)
  • fix UnmountDevice failure on Windows (#68608, @andyzhangx)
  • adjusted http/2 buffer sizes for apiservers to prevent starvation issues between concurrent streams (#67902, @liggitt)
  • Bump up version number of debian-base, debian-hyperkube-base and debian-iptables. (#67026, @satyasm)
    • Also updates dependencies of users of debian-base.
    • debian-base version 0.3.1 is already available.
  • Update debian-iptables and hyperkube-base images to include CVE fixes. (#67365, @ixdy)
  • Immediately close the other side of the connection when proxying. (#67288, @MHBauer)
  • Fix potential panic when getting azure load balancer status (#68609, @feiskyer)
  • [fluentd-gcp-scaler addon] Bump fluentd-gcp-scaler to 0.4 to pick up security fixes. (#67691, @loburm)
    • [prometheus-to-sd addon] Bump prometheus-to-sd to 0.3.1 to pick up security fixes, bug fixes and new features.
    • [event-exporter addon] Bump event-exporter to 0.2.3 to pick up security fixes.
  • PVC may not be synced to controller local cache in time if PV is bound by external PV binder (e.g. kube-scheduler), double check if PVC is not found to prevent reclaiming PV wrongly. (#67062, @cofyc)
  • support cross resource group for azure file (#68117, @andyzhangx)
  • Use real device path in isDeviceOpen check on umount (#68674, @gnufied)
  • Bump ip-masq-agent to v2.1.1 (#67916, @MrHohn)
      • Update debian-iptables image for CVEs.
      • Change chain name to IP-MASQ to be compatible with the
    • pre-injected masquerade rules.
  • Add NoSchedule/NoExecute tolerations to ip-masq-agent, ensuring it to be scheduled in all nodes except master. (#66260, @tanshanshan)
  • set EnableHTTPSTrafficOnly in azure storage account creation (#64957, @andyzhangx)
  • Fix VMWare VM freezing bug by reverting #51066 (#67825, @nikopen)
  • [fluentd-gcp addon] Bump version of fluentd-gcp to 2.0.18 to include latest security fixes. (#67693, @loburm)
  • Return apiserver panics as 500 errors instead terminating the apiserver process. (#68001, @sttts)
  • fix issue that pull image failed from a cross-subscription Azure Container Registry by SP access (#66429, @andyzhangx)
  • kube-controller-manager can now start the quota controller when discovery results can only be partially determined. (#67433, @deads2k)
  • attachdetach controller attaches volumes immediately when Pod's PVCs are bound (#66863, @cofyc)
  • This fix prevents a GCE PD volume from being mounted if the udev device link is stale and tries to correct the link. (#66832, @msau42)
  • skip nodes that have a primary NIC in a 'Failed' provisioningState (#65412, @yastij)
  • A smaller fix for the apiserver panic. (#67552, @wgliang)
  • Allows extension API server to dynamically discover the requestheader CA certificate when the core API server doesn't use certificate based authentication for it's clients (#66394, @rtripat)
  • add external resource group support for azure disk (#64427, @andyzhangx)

v1.9.10

Documentation & Examples

Downloads for v1.9.10

filenamesha256 hash
kubernetes.tar.gz77dbeb124f399e7e7ae0d7e3111316e6325b66383fc0ce7c2505758616fb6cd8
kubernetes-src.tar.gzce2556081fa209deac0d32bdb95fb6cf6ef147b6e18f06cbb4dbbca0a97630a3

Client Binaries

filenamesha256 hash
kubernetes-client-darwin-386.tar.gz989994a73ba6009acc74e4e0c43f1ab5f353033df1012ee02568abac162c0ef6
kubernetes-client-darwin-amd64.tar.gze5ee33e2b85ee0968f625b931ebe6f87fc5d3f116af54b3457aee114e0fd3629
kubernetes-client-linux-386.tar.gza6c9bdb3e1b70f4c44461596124262464988f167da3acc00a5d50b853e725daa
kubernetes-client-linux-amd64.tar.gz94da1929a9ef1129cf23b85b690effeca1e1029696ed4e174b1c47747e6e40e2
kubernetes-client-linux-arm.tar.gze89b0e48be3b54cb3f598350fd357cea0e5d143e208d15394a0b54d17bf9dba0
kubernetes-client-linux-arm64.tar.gz79dd62511b2465797b1d50b4ff7894ac77fb7cd9a30437aa0c54db10dce2f3cc
kubernetes-client-linux-ppc64le.tar.gzc30b014a0630672eb07272782203b99390b653c8ccde339fbef5f3c2031d6a2a
kubernetes-client-linux-s390x.tar.gze408569f43fc8c93cb49bd122d61d06204d526d1f4855ccb8ea5f1ce6423f120
kubernetes-client-windows-386.tar.gzf92f23bada75b3128ec61294e8d342c5bf6e86bd4ac042a2d774ffbb119fd8be
kubernetes-client-windows-amd64.tar.gz85a23fd2e5766067d266744b4672d7c5d2816e4750569f0ce1bdb85a2834c3f3

Server Binaries

filenamesha256 hash
kubernetes-server-linux-amd64.tar.gzb3299f985084021984b1a5ed823348fa97d52a5b416dbf7b5b0c299988a8fdcf
kubernetes-server-linux-arm.tar.gz9cacacdaae444b37413070920f38c192112144cacf1cb4cb17cdff3e0ed9b7ff
kubernetes-server-linux-arm64.tar.gzc557bf0495bf68c83c02c78c59ba12524f22105f9fc6fc80849a78a4c6a695e1
kubernetes-server-linux-ppc64le.tar.gzc50f172c4824c897218c736d6d17c5699045ff45f3536a6d6fc2b8c4b2d60053
kubernetes-server-linux-s390x.tar.gz449289491af6e3aa47cf58472be3cd9f638dfe762a4d0525d7563352cfe8b7fc

Node Binaries

filenamesha256 hash
kubernetes-node-linux-amd64.tar.gz8a64921a4ab5a858be6c7a90b195cec732cc2d67d37c6195e88210044401d05d
kubernetes-node-linux-arm.tar.gzde68c2b616a601843eed932ea08ac186acdaa5e75488237227263c747ea009a8
kubernetes-node-linux-arm64.tar.gz8b6d61316e99c17d51ffd966c26d2034960947968a5a6840e547a1966a087ad7
kubernetes-node-linux-ppc64le.tar.gz471d3a21f9bd8e8a7f03f8d225c4a72d7305f888102556e0ba29e5ec86dc763e
kubernetes-node-linux-s390x.tar.gz67b3d7e13e01c55c64762b70f63d14a3bc7910a4a0df296e9f18b4c5271fa19d
kubernetes-node-windows-amd64.tar.gzd82e22e0ab2321961cc0daa7accd985e2288e0ab0598ee515bc477d64e91f2e5

Changelog since v1.9.9

Other notable changes

v1.9.9

Documentation & Examples

Downloads for v1.9.9

filenamesha256 hash
kubernetes.tar.gz846b3f42bfe3e176ac44c4e79826a68648d982a1b6ebc2af15c1424f7b8f42f2
kubernetes-src.tar.gz6978228a7b85a63ef126e3dfd9356d5516e59f2a4924022bc113590a167d93bb

Client Binaries

filenamesha256 hash
kubernetes-client-darwin-386.tar.gzaf053dd90b11c875d41613d00733f53ec0c129afc9ff456c8b5e395cc9c65719
kubernetes-client-darwin-amd64.tar.gz63a8add3288be2e1ac8f8a7e9bf14564ed19e2808e50d031a16befdd22e57f66
kubernetes-client-linux-386.tar.gze7ed4413060321ef6170a5d6fc872997d6f14441b3d2ca365e0a1b63a657bcc4
kubernetes-client-linux-amd64.tar.gz3bc3b2d864a8fe8aaaa98be9fbeb59f3ad5a175ef3b6033ec7fd99283fcebfa1
kubernetes-client-linux-arm.tar.gz9fbfe7ef1bc1e923f3eb6325a481f177ee20ca5fabad249b6b9013d9375ef880
kubernetes-client-linux-arm64.tar.gz3e5e85dfae384163bcd90913d4cfe3184fd3ea6742ad61d00d1427f875ab6ee6
kubernetes-client-linux-ppc64le.tar.gz9542b37a160fb5e9feb4eb10f2642cfaf98166aed86c6279f71c847e1d3855e4
kubernetes-client-linux-s390x.tar.gz5d3fb0d74b8f234452243dfe6643a5fa9eee6b7805367d50e34311d91048e4d4
kubernetes-client-windows-386.tar.gzfacac737ab5cb3164c5668f4dcd5e82d308fb1006be87d00cfc1fd2c8da8b239
kubernetes-client-windows-amd64.tar.gz656731e7b04e4e26d906221933f1d47ff8b00b8426c0ec62dfbae505303e142e

Server Binaries

filenamesha256 hash
kubernetes-server-linux-amd64.tar.gz11167f79e8778da1fd5c3ecd547eec24744f34b708da52d6be2eed832b7e3a7a
kubernetes-server-linux-arm.tar.gz0fed178dd970ab7c3f8d255128faa28c3e5ee5407337ec79df97ee13fb76b22e
kubernetes-server-linux-arm64.tar.gz92a5e739a27be3cc45edc3560d45ae524af718043a1b5d9694918718f815d222
kubernetes-server-linux-ppc64le.tar.gz9e24340fff59ea466d1215e68ffe2f752d0dadaa0dbf1820f58f3a0d7da86e76
kubernetes-server-linux-s390x.tar.gzfe4cdfb14dd639e79c83a563b18433051c2fcc6d6695f5d3e1e8479a757b9172

Node Binaries

filenamesha256 hash
kubernetes-node-linux-amd64.tar.gzb98abba759a0035af1cf3c8e89d929748bb638e0e89438790f085b35f1df5a93
kubernetes-node-linux-arm.tar.gzbbe91a19a65f1fa17ab7c58e770bc8b079037c408d0a63cbe8b2442f4a1c72e1
kubernetes-node-linux-arm64.tar.gz94faf97de649457d8e02f631a4118b868e24199ad01b88b3c2d268e91054205e
kubernetes-node-linux-ppc64le.tar.gz973dc2a30c8d1a989b23f1ff0153b95baba2667f890a0baefb39eaf7c952a40c
kubernetes-node-linux-s390x.tar.gz1107320849132f7e840a14cf5ec9d909005562b54578fbb69eba4f96d1de6960
kubernetes-node-windows-amd64.tar.gz618aa00d34e460ffdb963eabae58013a03ec72981fdabf34df7538fcb66a683c

Changelog since v1.9.8

Action Required

  • ACTION REQUIRED: Kubernetes JSON deserializer is now case-sensitive to restore compatibility with pre-1.8 servers. If your config files contains fields with wrong case, the config files will be now invalid. (#65298, @nikhita)
  • A cluster-autoscaler ClusterRole is added to cover only the functionality required by Cluster Autoscaler and avoid abusing system:cluster-admin role. (#64503, @kgolab)
    • action required: Cloud providers other than GCE might want to update their deployments or sample yaml files to reuse the role created via add-on.

Other notable changes

  • kubelet: fix hangs in updating Node status after network interruptions/changes between the kubelet and API server (#63492, @liggitt)
  • fix mount unmount failure for a Windows pod (#63272, @andyzhangx)
  • Fix concurrent map access panic (#65327, @dashpole)
    • Don't watch .mount cgroups to reduce number of inotify watches
    • Fix NVML initialization race condition
    • Fix brtfs disk metrics when using a subdirectory of a subvolume
  • Bump version of prometheus-to-sd to 0.2.6 to decrease log noise and include latest security patches. (#64964, @loburm)
  • fixes a memory leak in the kube-controller-manager observed when large numbers of pods with tolerations are created/deleted (#65339, @liggitt)
  • Fix kube-controller-manager panic while provisioning Azure security group rules (#64739, @feiskyer)
  • Fixes issue for readOnly subpath mounts for SELinux systems and when the volume mountPath already existed in the container image. (#64351, @msau42)
  • removed unsafe double RLock in cpumanager (#62464, @choury)
  • Fix container StartTime in the kubelet's stats/summary endpoint (#64229, @dashpole)
  • Fix issue of colliding nodePorts when the cluster has services with externalTrafficPolicy=Local (#64468, @nicksardo)
  • Fix incorrectly propagated ResourceVersion in ListRequests returning 0 items. (#64150, @wojtek-t)
  • GCE: Fix to make the built-in kubernetes service properly point to the master's load balancer address in clusters that use multiple master VMs. (#63696, @grosskur)
  • Add a way to pass extra arguments to etcd. (#63961, @mborsz)

v1.9.8

Documentation & Examples

Downloads for v1.9.8

filenamesha256 hash
kubernetes.tar.gzde31bcccfe99b88f54ff24147be0b6c4fbc4fe46b10f81d0f05294317070e221
kubernetes-src.tar.gzd349086847a22ee89dc1fba335741c670650c0b61c1648612f4a40bc0b90255e

Client Binaries

filenamesha256 hash
kubernetes-client-darwin-386.tar.gzde772fcc08bf14d21c96f55aad8812bac36b7ef61957f7dab5ba3ec0321b691f
kubernetes-client-darwin-amd64.tar.gz6aff6127b21e14009e3aa181fc1a5932868060a32ba291170cd10a54acc2e6f4
kubernetes-client-linux-386.tar.gz1d5429470fd7c8c451075fa1cdbcbf430510a80131ad52b45f51e5e941ffed05
kubernetes-client-linux-amd64.tar.gz70b66cc9cd17a184a38d4a8daa5e3410f10390eb7c8745418c004cda457e609c
kubernetes-client-linux-arm.tar.gz648cfde6590100621c1eaa2e98d072c03ba19239fbc9809d8f6b859f81d61fc1
kubernetes-client-linux-arm64.tar.gz0966e4070d2cad96610d2a533501d1ca116016fe6a76b865d052ea72dec60a19
kubernetes-client-linux-ppc64le.tar.gz12d4a5e572d7c8dcccd569c93b0d7eaffd8bbed0f44d78e08c062d90d3dd1eec
kubernetes-client-linux-s390x.tar.gz53f619ad8c74cbcc0a3000d007ac98f544f7724a566c0560bd2b3d1ab123b223
kubernetes-client-windows-386.tar.gz04bcaaecd9e72b5dcb0019314baeaa1b3915401ae82a73e616b8579fcfb8c5e7
kubernetes-client-windows-amd64.tar.gzec5d0b08daaaa101597a8bc8ff51727556ae16e26c1d71fe96dd07ba4ad63cbf

Server Binaries

filenamesha256 hash
kubernetes-server-linux-amd64.tar.gz13a410ad1de823a807474c29475640cb3160f78c865f74c339f9690107d6f819
kubernetes-server-linux-arm.tar.gz42f625c0c6bf370a6213e498c044d8ba62d757e365cd1ec292e4f7532c2360de
kubernetes-server-linux-arm64.tar.gz464fe315c21b29417b74f51c5d923787bc046d652089a6e11f3a1fd72ae66f74
kubernetes-server-linux-ppc64le.tar.gze62a7f5e2b5f9502da07b03d2c4b6670d31d320b7811e9dbf0cf7470cc2377c1
kubernetes-server-linux-s390x.tar.gz11323b78190600000c92167cbd9ac000f9561545876425a9cf3ba7eab69ab132

Node Binaries

filenamesha256 hash
kubernetes-node-linux-amd64.tar.gz34c699bab128eb5094a3c7ca21e728d9ab7b08fe98ee9156671c7a3228344329
kubernetes-node-linux-arm.tar.gzf8bb865a85d2cf2bfa0188cf250471ff6997b2b1e8bd750db39b635355568445
kubernetes-node-linux-arm64.tar.gz7be2b47820b82153a7cf39abc146a59595e7156209f29a86fa6a4878ea4840c1
kubernetes-node-linux-ppc64le.tar.gz1ee6402840c60b4519b8c785057e35880f78c2449e17695a82174f12144dd4e0
kubernetes-node-linux-s390x.tar.gza023e6357f05ed1314a107c6848b48b3b2cb5b562dd557b64720481288b588d8
kubernetes-node-windows-amd64.tar.gza75684e1da14453517ed7deab78453579e148341440a6b4afa5a17c59a72a459

Changelog since v1.9.7

Other notable changes

  • Fix a bug in scheduler cache by using Pod UID as the cache key instead of namespace/name (#61069, @anfernee)
  • Fix in vSphere Cloud Provider to handle upgrades from kubernetes version less than v1.9.4 to v1.9.4 and above. (#62919, @abrarshivani)
  • Fixes issue where subpath readOnly mounts failed (#63045, @msau42)
  • Fix in vSphere Cloud Provider to report disk is detach when VM is not found. (#62220, @abrarshivani)
  • corrects a race condition in bootstrapping aggregated cluster roles in new HA clusters (#63761, @liggitt)
  • Add MAX_PODS_PER_NODE env so that GCE/GKE user can use it to specify the default max pods per node for the cluster. IP_ALIAS_SIZE will be changed accordingly. Must have ip alias enabled. (#63451, @grayluck)
  • Fix user visible files creation for windows (#62375, @feiskyer)
  • Add ipset and udevadm to the hyperkube base image. (#61357, @rphillips)
  • Fixes bugs that make apiserver panic when aggregating valid but not well formed OpenAPI spec (#63626, @roycaihw)
  • Kubernetes version command line parameter in kubeadm has been updated to drop an unnecessary redirection from ci/latest.txt to ci-cross/latest.txt. Users should know exactly where the builds are stored on Google Cloud storage buckets from now on. For example for 1.9 and 1.10, users can specify ci/latest-1.9 and ci/latest-1.10 as the CI build jobs what build images correctly updates those. The CI jobs for master update the ci-cross/latest location, so if you are looking for latest master builds, then the correct parameter to use would be ci-cross/latest. (#63504, @dims)
  • Fix issue where on re-registration of device plugin, allocatable was not getting updated. This issue makes devices invisible to the Kubelet if device plugin restarts. Only work-around, if this fix is not there, is to restart the kubelet and then start device plugin. (#63118, @vikaschoudhary16)
  • GCE: Fix for internal load balancer management resulting in backend services with outdated instance group links. (#62887, @nicksardo)
  • Bugfix allowing use of IP-aliases with custom-mode network in GCE setup scripts. (#62172, @shyamjvs)

v1.9.7

Documentation & Examples

Downloads for v1.9.7

filenamesha256 hash
kubernetes.tar.gz617b534fca9c5b2ced9c8eb3298a41fc55413aa876dcfec09bd15488e488db92
kubernetes-src.tar.gz1f10552d4282afe71ebbf046b8c5d0392453f0193fc511c59ee67e3c8f8ab0dd

Client Binaries

filenamesha256 hash
kubernetes-client-darwin-386.tar.gz7c58efd0c8cdcbdedbf461d63d4b5ba9b131f6c7e659f2da1e2b7127e5607f3d
kubernetes-client-darwin-amd64.tar.gz17e7ebab62026fbff5453a47fe6d0ac540de72c0596f769dc4d337a0aa7341d6
kubernetes-client-linux-386.tar.gzb159500ff36fbc4230d541d0c8e596cc37abfb20ec78cc7b7923e9ef3b184b4e
kubernetes-client-linux-amd64.tar.gz55ddbf3aa9f0aff90305845c8c9314520bcc69bbac2d9dedaf10a2356f9fc178
kubernetes-client-linux-arm.tar.gzc829f10dbc19b18bd59b4dd88d9e60e565c50138b017a0a41addc8178e81074a
kubernetes-client-linux-arm64.tar.gzd3a58780a74a235f8c8eb13a88fc8d62c810ff815f78beb8870f4c302c82d0ac
kubernetes-client-linux-ppc64le.tar.gz11803507d1d7b404e6fe63a04025a73aa0b1456e2d81d52f4824faa1a2311853
kubernetes-client-linux-s390x.tar.gzf86120b75f324e92ea72a1d341b4b2c65df3a66eb5b035ce347e7bdfefb570b8
kubernetes-client-windows-386.tar.gz523b117ac9c0321cfb5bc21fbfc68f83b00d6ec10365ea3eb1bf7dd7b1385bc9
kubernetes-client-windows-amd64.tar.gz9390d5066ff21f8b542676cb5d8f0dacae4d1c722d374bdeb09be7fcc11d020e

Server Binaries

filenamesha256 hash
kubernetes-server-linux-amd64.tar.gz8ed839b8b3851f54c00c8fe283ad505cf1d2878d4b2797dcfce9adedf34fab02
kubernetes-server-linux-arm.tar.gz0989b9fcb666b18a4c49998f9316fcc79093ee047a6be21d821168d80d341129
kubernetes-server-linux-arm64.tar.gz9e64195da3ffe0745109871b2c05d6788d1e1cff2c8956169eae55057952ea88
kubernetes-server-linux-ppc64le.tar.gz6809f1d29ea9dfa694efbda37b4ffbe6238cf9cd323c9ab5cfe69c9e18a7e43b
kubernetes-server-linux-s390x.tar.gze6cfdcdfeb3f73421df2c011337bd8b508c991d085e3cb420f87ba1d2aa877e0

Node Binaries

filenamesha256 hash
kubernetes-node-linux-amd64.tar.gzc890c3ceab080829b101e7b33e5d841fc29b6d060b5811859fa39a8d4a63ec36
kubernetes-node-linux-arm.tar.gz1dbaabdba6b01410dba5e370c262004ad8b2a0907cb1b7bab260fef84a455b04
kubernetes-node-linux-arm64.tar.gzc230fed814125176b9d215b51ccc58541937c09261adb7db702e60fb7ee78450
kubernetes-node-linux-ppc64le.tar.gzddd153bcefbad5286cedfc5768134fbb0048af9f1e70cd90145ba72063291482
kubernetes-node-linux-s390x.tar.gz25473510b470ec8e4f01835a4f3db58b9278687e6eb5aa440e948fbf13666fa4
kubernetes-node-windows-amd64.tar.gz5968c2b75244353ac6736420abc2de3d8c1c458efb4122319302a0b889bb2a8f

Changelog since v1.9.6

Action Required

  • ACTION REQUIRED: In-place node upgrades to this release from versions 1.7.14, 1.8.9, and 1.9.4 are not supported if using subpath volumes with PVCs. Such pods should be drained from the node first. (#61373, @msau42)

Other notable changes

  • Update kube-dns to Version 1.14.10. Major changes: (#62676, @MrHohn)
      • Fix a bug in DNS resolution for externalName services
    • and PTR records that need to query from upstream nameserver.
  • fix the issue that default azure disk fsypte(ext4) does not work on Windows (#62250, @andyzhangx)
  • Ensure expected load balancer is selected for Azure (#62450, @feiskyer)
  • Fix error message regarding conversion of v1.ListOptions to samplecontroller.k8s.io/v1alpha1. (#57243, @munnerz)
  • Resolves forbidden error when the daemon-set-controller cluster role access controllerrevisions resources. (#62146, @frodenas)
  • fix nsenter GetFileType issue in containerized kubelet (#62467, @andyzhangx)
  • fix incompatible file type checking on Windows (#62154, @dixudx)
  • Fix kubelet PVC stale metrics (#59170, @cofyc)
  • Fix panic create/update CRD when mutating/validating webhook configured. (#61404, @hzxuzhonghu)
  • Introduced --http2-max-streams-per-connection command line flag on api-servers and set default to 1000 for aggregated API servers. (#60054, @MikeSpreitzer)
  • Fix GCE etcd scripts to pass in all required parameters for the etcd migration utility to correctly perform HA upgrades and downgrades (#61956, @jpbetz)
  • Update kube-dns to Version 1.14.9. Major changes: (#61908, @MrHohn)
      • Fix for kube-dns returns NXDOMAIN when not yet synced with apiserver.
      • Don't generate empty record for externalName service.
      • Add validation for upstreamNameserver port.
      • Update go version to 1.9.3.
  • Backport Cloud CIDR allocator fixes to 1.9 (#61797, @satyasm)
  • openstack cinder detach problem is fixed if nova is shutdowned (#56846, @zetaab)
  • Fixed kube-proxy to work correctly with iptables 1.6.2 and later. (#61734, @danwinship)
  • Ensure cloudprovider.InstanceNotFound is reported when the VM is not found on Azure (#61531, @feiskyer)
  • On AWS kubelet returns an error when started under conditions that do not allow it to work (AWS has not yet tagged the instance). (#60125, @vainu-arto)
  • Fix mounting of UNIX sockets(and other special files) in subpaths (#61480, @gnufied)
  • Fixed missing error checking that could cause kubelet to crash in a race condition. (#60962, @technicianted)
  • Bump Heapster to v1.5.2 (#61396, @kawych)
  • Fixed a race condition in k8s.io/client-go/tools/cache.SharedInformer that could violate the sequential delivery guarantee and cause panics on shutdown. (#59828, @krousey)
  • [fluentd-gcp addon] Update fluentd and event-exporter images to have the latest base image. (#61719, @crassirostris)
  • Support new NODE_OS_DISTRIBUTION 'custom' on GCE (#61235, @yguo0905)
  • Fixes a bug where character devices are not recognized by the kubelet (#60440, @andrewsykim)
  • Fixes storage e2e test failures in GKE regional clusters. (#61303, @verult)
  • fix the error prone account creation method of blob disk (#59739, @andyzhangx)
  • The webhook admission controller in a custom apiserver now works off-the-shelf. (#60995, @caesarxuchao)
  • fix azure file plugin failure issue on Windows after node restart (#60625, @andyzhangx)
  • Fixes the races around devicemanager Allocate() and endpoint deletion. (#60856, @jiayingz)

v1.9.6

Documentation & Examples

Downloads for v1.9.6

filenamesha256 hash
kubernetes.tar.gz254f14a1c0a160c92c974ce471d86f549ca5ba25e22c42b0008a9d07291f931f
kubernetes-src.tar.gz67b2cebdb264da1c2b2ef5f41d6252346d58d944a75f51ba890fe3077b4975ee

Client Binaries

filenamesha256 hash
kubernetes-client-darwin-386.tar.gzd0b53a07fc41fa7f460ee7ae6cf227d136729136a4b1b35b20f71e9388496246
kubernetes-client-darwin-amd64.tar.gz5f5ae3da3fc9e1bf5466accacf82e60d9708d638b9f8d8346f770e8bbeefa38f
kubernetes-client-linux-386.tar.gz9f421d9ec59ad5fd75fc1184d48c9b473646a91e3182ede8c2852a03103f2459
kubernetes-client-linux-amd64.tar.gz2b1ab65171bcd43a099d4f7d05d7804c737272270e83f633e1c14ceed9a99133
kubernetes-client-linux-arm.tar.gz85b9a0908a9bb82009f89069bce45c30aa61575651151886688f7333291fd09a
kubernetes-client-linux-arm64.tar.gz574669991c5fc4771fc2c61eb46b8ecc857b4ccb9a639ae2443442518d571234
kubernetes-client-linux-ppc64le.tar.gzf286c3417c31ab2f89c615786def5ce3040216ed1530a3e21de7844e551185f0
kubernetes-client-linux-s390x.tar.gz29789d1f5e2d0b9816d21e219d00d5f4619a20504378152ca964b6ee52aa7ae9
kubernetes-client-windows-386.tar.gz543bfb8e1c53c76a7939083d4bc3aaa4aec4e62edcbfea291b231bc864f387e9
kubernetes-client-windows-amd64.tar.gzbd534611db31a0c5a3c981212308dfa465a09cec7320aaa296761e4633c6d693

Server Binaries

filenamesha256 hash
kubernetes-server-linux-amd64.tar.gz2a65bbbd67084bf89021f9139f60412d32153bed52e41eb625a616bc504fc6a9
kubernetes-server-linux-arm.tar.gzae130e2bb8591e100a977652ea660d85baf4fcad2cd28719389e97ec686a869d
kubernetes-server-linux-arm64.tar.gz0914665076bf33e4d9685edfeccb6c08a906f99b0408c443113994048c2017e6
kubernetes-server-linux-ppc64le.tar.gz3eb5424b705ca286438a4d9dba15b9837f061cb24f9f86b337c95f0e8986cfbb
kubernetes-server-linux-s390x.tar.gz33b92f8bf1bdc286921ccb270624ce9ca4462cce089523de7af686788442e3db

Node Binaries

filenamesha256 hash
kubernetes-node-linux-amd64.tar.gz59fd08906a4bd2e1f5f2c8bd4098928a23add4a6e3ff5006cd28a0099cbc5dcb
kubernetes-node-linux-arm.tar.gz4465e4452dc1270f0ac9808d681d58d159f25b73a69a85f1206890378fd76925
kubernetes-node-linux-arm64.tar.gzad80ff26a583d8cfbd576151c9d3b71a6e345dfadb349f0529667aaaac39d562
kubernetes-node-linux-ppc64le.tar.gz21c4685a50758737c4466cbc6f221dee534f50ecf542d924558a00a08fc41db3
kubernetes-node-linux-s390x.tar.gzd9ccb9bc1acc55a168a48c9ab29ecba335aaabeff484e74823747174019dbc65
kubernetes-node-windows-amd64.tar.gza9529c35e258ddf0da7e585bb7106f2635178b07691763dde483c47f579754a9

Changelog since v1.9.5

Other notable changes

v1.9.5

Documentation & Examples

Downloads for v1.9.5

filenamesha256 hash
kubernetes.tar.gz72947d7ac9a6f5bfe9f98b3362ce176cfc4d7c35caa1cf974ca2fd6dbc8ad608
kubernetes-src.tar.gza02261fb0e1d70feb95af36d404ad247ee46103b84748a5fec8b906648f68e0f

Client Binaries

filenamesha256 hash
kubernetes-client-darwin-386.tar.gz5ee29f16801e5776e985dd2a02bebe0ba25079ba0a5d331115bc9cd2a5b1225c
kubernetes-client-darwin-amd64.tar.gz5309c71bae2f1a8133d5c41522b827c0905fdaf2122690388fa5e15f4898a1c9
kubernetes-client-linux-386.tar.gz8e75b94eb78187033583b9bd9e53fbe4f74b242b9a8f03ebc953db6806604ae4
kubernetes-client-linux-amd64.tar.gz3248cff16e96610166f8b1c7c2d5b2403fb4fd15bff5af6e935388105ca62bc4
kubernetes-client-linux-arm.tar.gzd7836eb1d723db98a3a1d9b4e5fe16f9ee5b3304b680bd78d158ebcea59fdcb3
kubernetes-client-linux-arm64.tar.gz5ce9642d2413c87ffe42a90ba4878b0793f0fd5e143e00640c31a92ae1d80ded
kubernetes-client-linux-ppc64le.tar.gz6fd223aa94d18abca50a1f7f9607a23fba487770eba4de0f696e9a6bebd565da
kubernetes-client-linux-s390x.tar.gzeadc49e8c33461b1290fcf07b49db639478837f3bb7330d50fe2516fcf919e1a
kubernetes-client-windows-386.tar.gzcd2a65af308864e1a6128c40e87ddc56ece6d20de7ecc7009f3356a534a0fcfb
kubernetes-client-windows-amd64.tar.gze1f14e4f4028f545a9ab1bb02ff47b0a80afcb4f3a6f15e7b141fe580440e15f

Server Binaries

filenamesha256 hash
kubernetes-server-linux-amd64.tar.gz189e299a990f3dd2be9d2ac6c8315ea43a0ce1ccfc5d9d6b8c3325537a90395f
kubernetes-server-linux-arm.tar.gz408187d6a9cd8fa76581cdd64e438804993d39aea4061fd879cb9f30ddebdbda
kubernetes-server-linux-arm64.tar.gz5b68c3160c5c5455049684ffd3fcbe2c8037a0e8c992d657ff6f42de9db5d602
kubernetes-server-linux-ppc64le.tar.gz224d6566030d7b5e3215a257a625bda51dfa9c02af27bdcb1fbec9c0a57d7749
kubernetes-server-linux-s390x.tar.gz46c07056949cea5bc8ce60fb6c2b8cefee1fe7dc7adc705aeb8ef8ad0d703738

Node Binaries

filenamesha256 hash
kubernetes-node-linux-amd64.tar.gz46e5d05356e9ea99877cb1b0ef033cdc5a5e87df5a5c45c5cbc1f54adb278c1d
kubernetes-node-linux-arm.tar.gz022044653f19fa662467aea85ce8586d59e5888b8863cf5d95ca3c70424906c9
kubernetes-node-linux-arm64.tar.gz89ddd363d6ee207a9e78211dac9a74771d4eaf80668377d880632e8b31fd61d4
kubernetes-node-linux-ppc64le.tar.gz77543ec335bab7c69eb0b5a651de7252ecf033120867cd45029f65e07e901027
kubernetes-node-linux-s390x.tar.gz02266ccf4c16cf46fe8c218d12ad8c2c4513457f1b64aeb9d1e6ce59fb92c8b9
kubernetes-node-windows-amd64.tar.gzadc21f94570496c6299d4bf002f1b749d4695f094027aeaf24deb82f9818214d

Changelog since v1.9.4

Other notable changes

  • gce: fixes race condition in ServiceController, where nodes weren't updated in the node sync loop, by updating TargetPools in the ensureExternalLoadBalancer call. (#58368, @MrHohn)
  • fix race condition issue when detaching azure disk (#60183, @andyzhangx)
  • Get parent dir via canonical absolute path when trying to judge mount-point (#58433, @yue9944882)
  • Set node external IP for azure node when disabling UseInstanceMetadata (#60959, @feiskyer)
  • Fixes potential deadlock when deleting CustomResourceDefinition for custom resources with finalizers (#60542, @liggitt)
  • Unauthorized requests will not match audit policy rules where users or groups are set. (#59398, @CaoShuFeng)
  • [fluentd-gcp addon] Fixed bug with reporting metrics in event-exporter (#60126, @serathius)
  • Restores the ability of older clients to delete and scale jobs with initContainers (#59880, @liggitt)
  • fixed foreground deletion of podtemplates (#60683, @nilebox)
  • Bug fix: Clusters with GCE feature 'DiskAlphaAPI' enabled were unable to dynamically provision GCE PD volumes. (#59447, @verult)
  • Fix a regression that prevented using subPath volume mounts with secret, configMap, projected, and downwardAPI volumes (#61080, @liggitt)

v1.9.4

Documentation & Examples

Downloads for v1.9.4

filenamesha256 hash
kubernetes.tar.gz45b6aa8adbf3cf9fe37ddf063400a984766363b31f4da6204c00d02815616ce4
kubernetes-src.tar.gz645819c4e479d80d4f489bb022e3332fcede8fcb8e4265245621547d0b5ac8a7

Client Binaries

filenamesha256 hash
kubernetes-client-darwin-386.tar.gz8a6401958fa52b0a56011a7650ca2ab06d95b6927826cbc4834287e313f7216b
kubernetes-client-darwin-amd64.tar.gz7217f3d72ee5a23a06703c262dc051728775615236b87fa53c45969152a92c8d
kubernetes-client-linux-386.tar.gze5bd0c3fe36accd55e11854e4893fdced22261ef70d6ad22f67f8a70efbfee57
kubernetes-client-linux-amd64.tar.gzdb6ec27f4541ef91f73a3dd173851fea06c17a1eac52b86641d6d639f3dfc2ea
kubernetes-client-linux-arm.tar.gz15e385d032165cade02a4969618c13fb8b3e1074c06318581b646bd93cfc7153
kubernetes-client-linux-arm64.tar.gza49b589cb08711714f70cbdf5fc2734a981746b0e29858c60bc83b4ca226a132
kubernetes-client-linux-ppc64le.tar.gz3aae4268fdc5a81f0593abf5161c172b0c88c57f61ff6515a7cde1c7b35afc21
kubernetes-client-linux-s390x.tar.gzfb28c3940f3ab905caafccf277e2fe8fcdda7627dd8b784f25ca400d0e2383a2
kubernetes-client-windows-386.tar.gz243e4dc67fe9a55824c4648ba77589bcb7e34850095056433e5f15238cd3da32
kubernetes-client-windows-amd64.tar.gzb4ce826e60550c4a6852f28537565e115392a979f8c91095fdc2b61c13431e9a

Server Binaries

filenamesha256 hash
kubernetes-server-linux-amd64.tar.gzef7ffabb220df9616d9a483a815c3182d44c868a5bbb9ad1b1297270d3f59fec
kubernetes-server-linux-arm.tar.gzbff92c29be844663c087f17c85d527d9cf458ddcdeee0926f74da48d0d980c31
kubernetes-server-linux-arm64.tar.gz568c8966ebe05164452e6ae152352dc6d05a56bfb9b5c1860f5a68c15b55c0bd
kubernetes-server-linux-ppc64le.tar.gz39a1918ae53022fb38c98a02b945ae6cd5607a3a71011b19395fbc759739a453
kubernetes-server-linux-s390x.tar.gz2ea1af6654d5b3178fd4738c7fe5d019e0b4ea5fab746ebe2daa559d661844c7

Node Binaries

filenamesha256 hash
kubernetes-node-linux-amd64.tar.gza8224a7bf2b1b9aeab80946cfcf0f68549b3972f41850d857487c912318780c8
kubernetes-node-linux-arm.tar.gz503ae9d552e94f0fd4fe44787f38c3fc67c47f84f8d3567cc043375962d295a3
kubernetes-node-linux-arm64.tar.gze6578a7929858ab4b411a0ff6ea2c2d0c78dfc94ea4e23210d1db92fee3a930f
kubernetes-node-linux-ppc64le.tar.gzc1269cb6a78fc5bb57b47ed24458e4bd17cad6c849b5951c919d39ff72bcb0da
kubernetes-node-linux-s390x.tar.gz06f2027b133e3617806b327c7a7248c75025146efc31ba2cabe9480ba3dc7fed
kubernetes-node-windows-amd64.tar.gz2304b297bd9aec7e4285c697db75a523ff9dbb2c00e3baccf90375d4fdab5b91

Changelog since v1.9.3

Other notable changes

  • Fixes CVE-2017-1002101 - See https://issue.k8s.io/60813 for details (#61045, @liggitt)
  • Fixes a case when Deployment with recreate strategy could get stuck on old failed Pod. (#60493, @tnozicka)
  • Build using go1.9.3. (#59012, @ixdy)
  • fix device name change issue for azure disk (#60346, @andyzhangx)
  • Changes secret, configMap, downwardAPI and projected volumes to mount read-only, instead of allowing applications to write data and then reverting it automatically. Until version 1.11, setting the feature gate ReadOnlyAPIDataVolumes=false will preserve the old behavior. (#58720, @joelsmith)
  • Add automatic etcd 3.2->3.1 and 3.1->3.0 minor version rollback support to gcr.io/google_container/etcd images. For HA clusters, all members must be stopped before performing a rollback. (#59298, @jpbetz)
  • Fix the bug where kubelet in the standalone mode would wait for the update from the apiserver source. (#59276, @roboll)
  • fix the create azure file pvc failure if there is no storage account in current resource group (#56557, @andyzhangx)
  • Increase allowed lag for ssh key sync loop in tunneler to allow for one failure (#60068, @wojtek-t)
  • Fixing a bug in OpenStack cloud provider, where dual stack deployments (IPv4 and IPv6) did not work well when using kubenet as the network plugin. (#59749, @zioproto)
  • Bugfix: vSphere Cloud Provider (VCP) does not need any special service account anymore. (#59440, @rohitjogvmw)
  • vSphere Cloud Provider supports VMs provisioned on vSphere v1.6.5 (#59519, @abrarshivani)
  • Allow node IPAM controller to configure IPAMFromCluster mode to use IP aliases instead of routes in GCP. (#59688, @jingax10)
  • Fixed an issue where Portworx volume driver wasn't passing namespace and annotations to the Portworx Create API. (#59607, @harsh-px)
  • Use a more reliable way to get total physical memory on windows nodes (#57124, @JiangtianLi)
  • Fix race causing apiserver crashes during etcd healthchecking (#60069, @wojtek-t)
  • return error if New-SmbGlobalMapping failed when mounting azure file on Windows (#59540, @andyzhangx)
  • Ensure Azure public IP removed after service deleted (#59340, @feiskyer)
  • Map correct vmset name for Azure internal load balancers (#59747, @feiskyer)
  • Node's providerID is following Azure resource ID format now when useInstanceMetadata is enabled (#59539, @feiskyer)

v1.9.3

Documentation & Examples

Downloads for v1.9.3

filenamesha256 hash
kubernetes.tar.gzb495325eacd1354514b20ef1f0b99c6a41277842fc93b6cf5c9cb6e8657c266f
kubernetes-src.tar.gzf99a016dc616be37e7fe161ff435335a2442ebcede622486e7a9cf0bacedb625

Client Binaries

filenamesha256 hash
kubernetes-client-darwin-386.tar.gz084dd17c182acbc1ee248ea9f9fc720224be6245f13d9904cd7ca44205eb38ed
kubernetes-client-darwin-amd64.tar.gzc6ae13f8da18322ca3651b289c8e48475839e6f4c741ae12342cd69bde467773
kubernetes-client-linux-386.tar.gz231d9255c11d38b88c6b7febe43d1ea9564c6b36b34cb905450c7beb7c46e051
kubernetes-client-linux-amd64.tar.gz2f509c05f0c4e1c1ac9e98879a1924f24546905349457904344d79dc639217be
kubernetes-client-linux-arm64.tar.gzd8fe5dc1bc80d5dfb60e811c0bfcd392b2761f76400fc4c48b17d4d4cd0aabf1
kubernetes-client-linux-arm.tar.gz7c084e01a97379256746ada2b980e36e727acc23aaa614d98e4e0a144faad37e
kubernetes-client-linux-ppc64le.tar.gz669629d372ebe169140238f106c6d97b53a1895f4ac8393147fbddddf83eeb47
kubernetes-client-linux-s390x.tar.gz1627933c04ba9a155ac63c0a9a90ada32badd618c2e2919d3044cd5039963cc4
kubernetes-client-windows-386.tar.gz68de0d599a5e09195479602390343a017296b3aa774b4a783455581e1065cc8d
kubernetes-client-windows-amd64.tar.gze8872561f33258a8509e90aa955c5b57d6b5d9a657864bf5002e21285a8f4792

Server Binaries

filenamesha256 hash
kubernetes-server-linux-amd64.tar.gz09ab78a1b091ce8affb39d5218ba780eb36bc8026d557ed6d5efcd5a51b7927a
kubernetes-server-linux-arm64.tar.gzf3e38a8ffae0b5f2ac0c776a0b4586630e8b258f2802237ebda4d612f6d41d9e
kubernetes-server-linux-arm.tar.gzeeba15fc5db374e6e1b66b846988422e751752d930e4c2c89f5a92de5f466010
kubernetes-server-linux-ppc64le.tar.gzce05d9cf268b213e9a57dcbb5f9d570c62e72a15f8af9e692f4a26a8f40d8df1
kubernetes-server-linux-s390x.tar.gz1ca63330add758e7638357eba79753d1af610ea5de8b082aa740ef4852abd51a

Node Binaries

filenamesha256 hash
kubernetes-node-linux-amd64.tar.gzc40f983c11f93752a40180cb719ddd473cbf07f43a3af5d2b575411c85b76f88
kubernetes-node-linux-arm64.tar.gz7a0c5c313d14d88bd11010d416c0614e7dc2362e78e1ffb65ee098bfe944b881
kubernetes-node-linux-arm.tar.gz7a3e288cb04e3fe5f2537645bd74a68d7b471c15c6eb51eb9d5e1ac6edfc7e9f
kubernetes-node-linux-ppc64le.tar.gz401f763112d20cf2c613d065beecd47387bb11d82a49fd2222a2ac38a4e06c20
kubernetes-node-linux-s390x.tar.gz9a6d921e1cef37dcbaac61be13a70410cd03bc26335b7730cce6d9d3c8506b22
kubernetes-node-windows-amd64.tar.gz0db90e50c23ef16e9bfa5a990647bd4299a809166a2a37764e880b1910feee49

Changelog since v1.9.2

Action Required

  • Bug fix: webhooks now do not skip cluster-scoped resources (#58185, @caesarxuchao)
    • Action required: Before upgrading your Kubernetes clusters, double check if you had configured webhooks for cluster-scoped objects (e.g., nodes, persistentVolume), these webhooks will start to take effect. Delete/modify the configs if that's not desirable.

Other notable changes

  • CustomResourceDefinitions: OpenAPI v3 validation schemas containing $refreferences are no longer permitted (valid references could not be constructed previously because property ids were not permitted either). Before upgrading, ensure CRD definitions do not include those $ref fields. (#58438, @carlory)
  • Ensure IP is set for Azure internal load balancer. (#59083, @feiskyer)
  • Configurable etcd quota backend bytes in GCE (#59259, @wojtek-t)
  • Updates Calico version to v2.6.7 (Fixed a bug where Felix would crash when parsing a NetworkPolicy with a named port. See https://github.com/projectcalico/calico/releases/tag/v2.6.7) (#59130, @caseydavenport)
  • Cluster Autoscaler 1.1.1 (details: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.1.1) (#59272, @mwielgus)
  • cloudprovider/openstack: fix bug the tries to use octavia client to query flip (#59075, @jrperritt)
  • Fixed a bug which caused the apiserver reboot failure in the presence of malfunctioning webhooks. (#59073, @caesarxuchao)
  • Configurable etcd compaction frequency in GCE (#59106, @wojtek-t)
  • Prevent kubelet from getting wedged if initialization of modules returns an error. (#59020, @brendandburns)
  • [GCE] Apiserver uses InternalIP as the most preferred kubelet address type by default. (#59019, @MrHohn)
  • Expose Metrics Server metrics via /metric endpoint. (#57456, @kawych)
  • Get windows kernel version directly from registry (#58498, @feiskyer)
  • Fixes a bug where kubelet crashes trying to free memory under memory pressure (#58574, @yastij)
  • Updated priority of mirror pod according to PriorityClassName. (#58485, @k82cn)
  • Access to externally managed IP addresses via the kube-apiserver service proxy subresource is no longer allowed by default. This can be re-enabled via the ServiceProxyAllowExternalIPs feature gate, but will be disallowed completely in 1.11 (#57265, @brendandburns)
  • Detach and clear bad disk URI (#58345, @rootfs)
  • Add apiserver metric for number of requests dropped because of inflight limit. (#58340, @gmarek)
  • Add apiserver metric for current inflight-request usage. (#58342, @gmarek)
  • kube-apiserver is changed to use SSH tunnels for webhook iff the webhook is not directly routable from apiserver's network environment. (#58644, @yguo0905)
  • Update Calico version to v2.6.6 (#58482, @tmjd)
  • Fix garbage collection and resource quota when the controller-manager uses --leader-elect=false (#57340, @jmcmeek)
  • kube-apiserver: fixes loading of --admission-control-config-file containing AdmissionConfiguration apiserver.k8s.io/v1alpha1 config object (#58441, @liggitt)
  • Fix a bug affecting nested data volumes such as secret, configmap, etc. (#57422, @joelsmith)
  • Reduce Metrics Server memory requirement (#58391, @kawych)
  • GCP: allow a master to not include a metadata concealment firewall rule (if it's not running the metadata proxy). (#58104, @ihmccreery)
  • Bump GCE metadata proxy to v0.1.9 to pick up security fixes. (#58221, @ihmccreery)
  • Fixes an issue where the resourceVersion of an object in a DELETE watch event was not the resourceVersion of the delete itself, but of the last update to the object. This could disrupt the ability of clients clients to re-establish watches properly. (#58547, @liggitt)
  • Fixed encryption key and encryption provider rotation (#58375, @liggitt)
  • Correctly handle transient connection reset errors on GET requests from client library. (#58520, @porridge)
  • Avoid controller-manager to crash when enabling IP alias for K8s cluster. (#58557, @jingax10)

v1.9.2

Documentation & Examples

Downloads for v1.9.2

filenamesha256 hash
kubernetes.tar.gz7a922d49b1194cb1b59b22cecb4eb1197f7c37250d4326410dc71aa5dc5ec8a2
kubernetes-src.tar.gz9f128809cdd442d71a13f7c61c7a0e03e832cf0c068a86184c1bcc9acdb78872

Client Binaries

filenamesha256 hash
kubernetes-client-darwin-386.tar.gz37d2dd1b1762f1040699584736bbc1a2392e94779a19061d477786bcce3d3f01
kubernetes-client-darwin-amd64.tar.gz42adc9762b30bfd3648323f9a8f350efeedec08a901997073f6d4244f7a16f78
kubernetes-client-linux-386.tar.gz5dde6c6388353376aaa0bd731b0366d9d2d11baee3746662b008e09d9618d55f
kubernetes-client-linux-amd64.tar.gzc45cf9e9d27b9d1bfc6d26f86856271fec6f8e7007f014597d27668f72f8c349
kubernetes-client-linux-arm64.tar.gz05c3810b00adcdbf7bc67671847f11e287da72f308cc704e5679e83564236fee
kubernetes-client-linux-arm.tar.gza9421d4627eb9eaa1e46cfd4276943e25b5b80e52db6945f173a2a45782ce42d
kubernetes-client-linux-ppc64le.tar.gzadc345ab050e09a3069a47e862c0ce88630a586905b33f6e5fd339005ceffbbf
kubernetes-client-linux-s390x.tar.gzfdff4b462e67569a4a1110b696d8af2c563e0a19e50a58a7b1a4346942b07993
kubernetes-client-windows-386.tar.gz1a82e8e4213153993a6e86e74120f62f95645952b223ed8586316358dd22a225
kubernetes-client-windows-amd64.tar.gza8648d4d3e0f85597bd57de87459a040ceab4c073d647027a70b0fba8862eab3

Server Binaries

filenamesha256 hash
kubernetes-server-linux-amd64.tar.gz2218fe0b939273b57ce00c7d5f3f7d2c34ebde5ae500ba2646eea6ba26c7c63d
kubernetes-server-linux-arm64.tar.gz3b4bc6cf91c3eaf37ef2b361dd77e838f0a8ca2b8cbb4dd42793c1fea5186b69
kubernetes-server-linux-arm.tar.gz73e77da0ddc951f791b5f7b73420ba0dbb141b3637cc48b4e916a41249e40ce3
kubernetes-server-linux-ppc64le.tar.gz860ba4ac773e4aff69dde781cac7ac1fb1824f2158155dfa49c50dd3acf0ab82
kubernetes-server-linux-s390x.tar.gz19e0fd7863e217b4cb67f91b56ceb5939ae677f523681bdf8ccac174f36f576d

Node Binaries

filenamesha256 hash
kubernetes-node-linux-amd64.tar.gzf86b7038dc89d79b277c5fba499f391c25f5aba8f5caa3119c05065f9917b6f9
kubernetes-node-linux-arm64.tar.gz87f40c37a3e359a9350a3bcbe0e27ad6e7dfa0d8ee5f6d2ecf061813423ffa73
kubernetes-node-linux-arm.tar.gzb73d879a03e7eba5543af0b56085ebb4919d401f6a06d4803517ddf606e8240e
kubernetes-node-linux-ppc64le.tar.gz26331e5d84d98fc3a94d2d55fd411159b2a79b6083758cea1dac36a0a4a44336
kubernetes-node-linux-s390x.tar.gzcbf52f3942965bb659d1f0f624e09ff01b2ee9f6e6217b3876c41600e1d4c711
kubernetes-node-windows-amd64.tar.gz70d59046a7c949d4fd4850ee57b1cd44dddfb041c548a21354ee30d7bfb1003d

Changelog since v1.9.1

Other notable changes

  • Fixes authentication problem faced during various vSphere operations. (#57978, @prashima)
  • The getSubnetIDForLB() should return subnet id rather than net id. (#58208, @FengyunPan)
  • Add cache for VM get operation in azure cloud provider (#57432, @karataliu)
  • Update kube-dns to Version 1.14.8 that includes only small changes to how Prometheus metrics are collected. (#57918, @rramkumar1)
  • Fixes a possible deadlock preventing quota from being recalculated (#58107, @ironcladlou)
  • Fixes a bug in Heapster deployment for google sink. (#57902, @kawych)
  • GCE: Allows existing internal load balancers to continue using an outdated subnetwork (#57861, @nicksardo)
  • Update etcd version to 3.1.11 (#57811, @xiangpengzhao)
  • fix device name change issue for azure disk: add remount logic (#57953, @andyzhangx)
  • calico-node addon tolerates all NoExecute and NoSchedule taints by default. (#57122, @caseydavenport)
  • Allow kubernetes components to react to SIGTERM signal and shutdown gracefully. (#57756, @mborsz)
  • Fixes controller manager crash in certain vSphere cloud provider environment. (#57286, @rohitjogvmw)
  • fix azure disk not available issue when device name changed (#57549, @andyzhangx)
  • GCE: support passing kube-scheduler policy config via SCHEDULER_POLICY_CONFIG (#57425, @yguo0905)

v1.9.1

Documentation & Examples

Downloads for v1.9.1

filenamesha256 hash
kubernetes.tar.gz0eece0e6c1f68535ea71b58b87e239019bb57fdd61118f3d7defa6bbf4fad5ee
kubernetes-src.tar.gz625ebb79412bd12feccf12e8b6a15d9c71ea681b571f34deaa59fe6c9ba55935

Client Binaries

filenamesha256 hash
kubernetes-client-darwin-386.tar.gz909556ed9b8445703d0124f2d8c1901b00afaba63a9123a4296be8663c3a2b2d
kubernetes-client-darwin-amd64.tar.gz71e191d99d3ac1426e23e087b8d0875e793e5615d3aa7ac1e175b250f9707c48
kubernetes-client-linux-386.tar.gz1c4e60c0c056a3300c7fcc9faccd1b1ea2b337e1360c20c5b1c25fdc47923cf0
kubernetes-client-linux-amd64.tar.gzfe8fe40148df404b33069931ea30937699758ed4611ef6baddb4c21b7b19db5e
kubernetes-client-linux-arm64.tar.gz921f5711b97f0b4de69784d9c79f95e80f75a550f28fc1f26597aa0ef6faa471
kubernetes-client-linux-arm.tar.gz77b010cadef98dc832a2f560afe15e57a675ed9fbc59ffad5e19878510997874
kubernetes-client-linux-ppc64le.tar.gz02aa71ddcbe8b711814af7287aac79de5d99c1c143c0d3af5e14b1ff195b8bdc
kubernetes-client-linux-s390x.tar.gz7e315024267306a620045d003785ecc8d7f2e763a6108ae806d5d384aa7552cc
kubernetes-client-windows-386.tar.gz99b2a81b7876498e119db4cb34c434b3790bc41cd882384037c1c1b18cba9f99
kubernetes-client-windows-amd64.tar.gzd89d303cbbf9e57e5a540277158e4d83ad18ca7402b5b54665f1378bb4528599

Server Binaries

filenamesha256 hash
kubernetes-server-linux-amd64.tar.gz5acf2527461419ba883ac352f7c36c3fa0b86a618dbede187054ad90fa233b0e
kubernetes-server-linux-arm64.tar.gze1f61b4dc6e0c9986e95ec25f876f9a89966215ee8cc7f4a3539ec391b217587
kubernetes-server-linux-arm.tar.gz441c45e16e63e9bdf99887a896a99b3a376af778cb778cc1d0e6afc505237200
kubernetes-server-linux-ppc64le.tar.gzc0175f02180d9c88028ee5ad4e3ea04af8a6741a97f4900b02615f7f83c4d1c5
kubernetes-server-linux-s390x.tar.gz2178150d31197ad7f59d44ffea37d682c2675b3a4ea2fc3fa1eaa0e768b993f7

Node Binaries

filenamesha256 hash
kubernetes-node-linux-amd64.tar.gzb8ff0ae693ecca4d55669c66786d6c585f8c77b41a270d65f8175eba8729663a
kubernetes-node-linux-arm64.tar.gzf0f63baaace463dc663c98cbc9a41e52233d1ef33410571ce3f3e78bd485787e
kubernetes-node-linux-arm.tar.gz554bdd11deaf390de85830c7c888dfd4d75d9de8ac147799df12993f27bde905
kubernetes-node-linux-ppc64le.tar.gz913af8ca8b258930e76fd3368acc83608e36e7e270638fa01a6e3be4f682d8bd
kubernetes-node-linux-s390x.tar.gz8192c1c80563230d727fab71514105571afa52cde8520b3d90af58e6daf0e19c
kubernetes-node-windows-amd64.tar.gz4408e6d741c6008044584c0d7235e608c596e836d51346ee773589d9b4589fdc

Changelog since v1.9.0

Other notable changes

  • Compare correct file names for volume detach operation (#57053, @prashima)
  • Fixed a garbage collection race condition where objects with ownerRefs pointing to cluster-scoped objects could be deleted incorrectly. (#57211, @liggitt)
  • Free up CPU and memory requested but unused by Metrics Server Pod Nanny. (#57252, @kawych)
  • Configurable liveness probe initial delays for etcd and kube-apiserver in GCE (#57749, @wojtek-t)
  • Fixed garbage collection hang (#57503, @liggitt)
  • GCE: Fixes ILB creation on automatic networks with manually created subnetworks. (#57351, @nicksardo)
  • Check for known manifests during preflight instead of only checking for non-empty manifests directory. (#57287, @mattkelly)
  • enable flexvolume on Windows node (#56921, @andyzhangx)
  • change default azure file/dir mode to 0755 (#56551, @andyzhangx)
  • fix incorrect error info when creating an azure file PVC failed (#56550, @andyzhangx)
  • Retry 'connection refused' errors when setting up clusters on GCE. (#57394, @mborsz)
  • Fixes issue creating docker secrets with kubectl 1.9 for accessing docker private registries. (#57463, @dims)
  • Fixes a bug where if an error was returned that was not an autorest.DetailedError we would return "not found", nil which caused nodes to go to NotReady state. (#57484, @brendandburns)
  • Fix Heapster configuration and Metrics Server configuration to enable overriding default resource requirements. (#56965, @kawych)

v1.9.0

Documentation & Examples

Downloads for v1.9.0

filenamesha256 hash
kubernetes.tar.gzd8a52a97382a418b69d46a8b3946bd95c404e03a2d50489d16b36517c9dbc7f4
kubernetes-src.tar.gz95d35ad7d274e5ed207674983c3e8ec28d8190c17e635ee922e2af8349fb031b

Client Binaries

filenamesha256 hash
kubernetes-client-darwin-386.tar.gz2646aa4badf9281b42b921c1e9e2ed235e1305d331423f252a3380396e0c383f
kubernetes-client-darwin-amd64.tar.gze76e69cf58399c10908afce8bb8d1f12cb8811de7b24e657e5f9fc80e7b9b6fb
kubernetes-client-linux-386.tar.gzbcd5ca428eb78fdaadbcf9ff78d9cbcbf70585a2d2582342a4460e55f3bbad13
kubernetes-client-linux-amd64.tar.gzba96c8e71dba68b1b3abcad769392fb4df53e402cb65ef25cd176346ee2c39e8
kubernetes-client-linux-arm64.tar.gz80ceae744fbbfc7759c3d95999075f98e5d86d80e53ea83d16fa8e849da4073d
kubernetes-client-linux-arm.tar.gz86b271e2518230f3502708cbe8f188a3a68b913c812247b8cc6fbb4c9f35f6c8
kubernetes-client-linux-ppc64le.tar.gz8b7506ab64ceb2ff470120432d7a6a93adf14e14e612b3c53b3c238d334b55e2
kubernetes-client-linux-s390x.tar.gzc066aa75a99c141410f9b9a78d230aff4a14dee472fe2b17729e902739798831
kubernetes-client-windows-386.tar.gza315535d6a64842a7c2efbf2bb876c0b73db7efd4c848812af07956c2446f526
kubernetes-client-windows-amd64.tar.gz5d2ba1f008253da1a784c8bb5266d026fb6fdac5d22133b51e86d348dbaff49b

Server Binaries

filenamesha256 hash
kubernetes-server-linux-amd64.tar.gza8d7be19e3b662681dc50dc0085ca12045979530a27d0200cf986ada3eff4d32
kubernetes-server-linux-arm64.tar.gz8ef6ad23c60a50b4255ff41db044b2f5922e2a4b0332303065d9e66688a0b026
kubernetes-server-linux-arm.tar.gz7cb99cf65553c9637ee6f55821ea3f778873a9912917ebbd6203e06d5effb055
kubernetes-server-linux-ppc64le.tar.gz529b0f45a0fc688aa624aa2b850f28807ce2be3ac1660189f20cd3ae864ac064
kubernetes-server-linux-s390x.tar.gz692f0c198da712f15ff93a4634c67f9105e3ec603240b50b51a84480ed63e987

Node Binaries

filenamesha256 hash
kubernetes-node-linux-amd64.tar.gz7ff3f526d1c4ec23516a65ecec3b947fd8f52d8c0605473b1a87159399dfeab1
kubernetes-node-linux-arm64.tar.gzfada290471467c341734a3cfff63cd0f867aad95623b67096029d76c459bde06
kubernetes-node-linux-arm.tar.gzded3640bef5f9701f7f622de4ed162cd2e5a968e80a6a56b843ba84a0b146fac
kubernetes-node-linux-ppc64le.tar.gza83ebe3b360d33c2190bffd5bf0e2c68268ca2c85e3b5295c1a71ddb517a4f90
kubernetes-node-linux-s390x.tar.gz1210efdf35ec5e0b2e96ff7e456e340684ff12dbea36aa255ac592ca7195e168
kubernetes-node-windows-amd64.tar.gz9961ad142abc7e769bbe962aeb30a014065fae83291a2d65bc2da91f04fbf185

1.9 Release Notes

Before updating to 1.9, you are strongly recommended to back up your etcd data. Consult the installation procedure you are using (kargo, kops, kube-up, kube-aws, kubeadm etc) for specific advice.

Some upgrade methods might upgrade etcd from 3.0 to 3.1 automatically when you upgrade from Kubernetes 1.8, unless you specify otherwise. Because etcd does not support downgrading, you'll need to either remain on etcd 3.1 or restore from a backup if you want to downgrade back to Kubernetes 1.8.

Introduction to 1.9.0

Kubernetes version 1.9 includes new features and enhancements, as well as fixes to identified issues. The release notes contain a brief overview of the important changes introduced in this release. The content is organized by Special Interest Group (SIG).

For initial installations, see the Setup topics in the Kubernetes documentation.

To upgrade to this release from a previous version, first take any actions required Before Upgrading.

For more information about this release and for the latest documentation, see the Kubernetes documentation.

Major themes

Kubernetes is developed by community members whose work is organized into Special Interest Groups, which provide the themes that guide their work. For the 1.9 release, these themes included:

API Machinery

Extensibility. SIG API Machinery added a new class of admission control webhooks (mutating), and brought the admission control webhooks to beta.

Apps

The core workloads API, which is composed of the DaemonSet, Deployment, ReplicaSet, and StatefulSet kinds, has been promoted to GA stability in the apps/v1 group version. As such, the apps/v1beta2 group version is deprecated, and all new code should use the kinds in the apps/v1 group version.

Auth

SIG Auth focused on extension-related authorization improvements. Permissions can now be added to the built-in RBAC admin/edit/view roles using cluster role aggregation. Webhook authorizers can now deny requests and short-circuit checking subsequent authorizers. Performance and usability of the beta PodSecurityPolicy feature was also improved.

AWS

In v1.9 SIG AWS has improved stability of EBS support across the board. If a Volume is “stuck” in the attaching state to a node for too long a unschedulable taint will be applied to the node, so a Kubernetes admin can take manual steps to correct the error. Users are encouraged to ensure they are monitoring for the taint, and should consider automatically terminating instances in this state.

In addition, support for NVMe disks has been added to Kubernetes, and a service of type LoadBalancer can now be backed with an NLB instead of an ELB (alpha).

Azure

SIG Azure worked on improvements in the cloud provider, including significant work on the Azure Load Balancer implementation.

Cluster Lifecycle

SIG Cluster Lifecycle has been focusing on improving kubeadm in order to bring it to GA in a future release, as well as developing the Cluster API. For kubeadm, most new features, such as support for CoreDNS, IPv6 and Dynamic Kubelet Configuration, have gone in as alpha features. We expect to graduate these features to beta and beyond in the next release. The initial Cluster API spec and GCE sample implementation were developed from scratch during this cycle, and we look forward to stabilizing them into something production-grade during 2018.

Instrumentation

In v1.9 we focused on improving stability of the components owned by the SIG, including Heapster, Custom Metrics API adapters for Prometheus, and Stackdriver.

Network

In v1.9 SIG Network has implemented alpha support for IPv6, and alpha support for CoreDNS as a drop-in replacement for kube-dns. Additionally, SIG Network has begun the deprecation process for the extensions/v1beta1 NetworkPolicy API in favor of the networking.k8s.io/v1 equivalent.

Node

SIG Node iterated on the ability to support more workloads with better performance and improved reliability. Alpha features were improved around hardware accelerator support, device plugins enablement, and cpu pinning policies to enable us to graduate these features to beta in a future release. In addition, a number of reliability and performance enhancements were made across the node to help operators in production.

OpenStack

In this cycle, SIG OpenStack focused on configuration simplification through smarter defaults and the use of auto-detection wherever feasible (Block Storage API versions, Security Groups) as well as updating API support, including:

  • Block Storage (Cinder) V3 is now supported.
  • Load Balancer (Octavia) V2 is now supported, in addition to Neutron LBaaS V2.
  • Neutron LBaas V1 support has been removed.

This work enables Kubernetes to take full advantage of the relevant services as exposed by OpenStack clouds. Refer to the Cloud Providers documentation for more information.

Storage

SIG Storage is responsible for storage and volume plugin components.

For the 1.9 release, SIG Storage made Kubernetes more pluggable and modular by introducing an alpha implementation of the Container Storage Interface (CSI). CSI will make installing new volume plugins as easy as deploying a pod, and enable third-party storage providers to develop their plugins without the need to add code to the core Kubernetes codebase.

The SIG also focused on adding functionality to the Kubernetes volume subsystem, such as alpha support for exposing volumes as block devices inside containers, extending the alpha volume-resizing support to more volume plugins, and topology-aware volume scheduling.

Windows

We are advancing support for Windows Server and Windows Server Containers to beta along with continued feature and functional advancements on both the Kubernetes and Windows platforms. This opens the door for many Windows-specific applications and workloads to run on Kubernetes, significantly expanding the implementation scenarios and the enterprise reach of Kubernetes.

Before Upgrading

Consider the following changes, limitations, and guidelines before you upgrade:

API Machinery

  • The admission API, which is used when the API server calls admission control webhooks, is moved from admission.v1alpha1 to admission.v1beta1. You must delete any existing webhooks before you upgrade your cluster, and update them to use the latest API. This change is not backward compatible.
  • The admission webhook configurations API, part of the admissionregistration API, is now at v1beta1. Delete any existing webhook configurations before you upgrade, and update your configuration files to use the latest API. For this and the previous change, see also the documentation.
  • A new ValidatingAdmissionWebhook is added (replacing GenericAdmissionWebhook) and is available in the generic API server. You must update your API server configuration file to pass the webhook to the --admission-control flag. (#55988, @caesarxuchao) (#54513, @deads2k)
  • The deprecated options --portal-net and --service-node-ports for the API server are removed. (#52547, @xiangpengzhao)

Auth

  • PodSecurityPolicy: A compatibility issue with the allowPrivilegeEscalation field that caused policies to start denying pods they previously allowed was fixed. If you defined PodSecurityPolicy objects using a 1.8.0 client or server and set allowPrivilegeEscalation to false, these objects must be reapplied after you upgrade. (#53443, @liggitt)
  • KMS: Alpha integration with GCP KMS was removed in favor of a future out-of-process extension point. Discontinue use of the GCP KMS integration and ensure data has been decrypted (or reencrypted with a different provider) before upgrading (#54759, @sakshamsharma)

CLI

  • Swagger 1.2 validation is removed for kubectl. The options --use-openapi and --schema-cache-dir are also removed because they are no longer needed. (#53232, @apelisse)

Cluster Lifecycle

  • You must either specify the --discovery-token-ca-cert-hash flag to kubeadm join, or opt out of the CA pinning feature using --discovery-token-unsafe-skip-ca-verification.
  • The default auto-detect behavior of the kubelet's --cloud-provider flag is removed.
    • You can manually set --cloud-provider=auto-detect, but be aware that this behavior will be removed completely in a future version.
    • Best practice for version 1.9 and future versions is to explicitly set a cloud-provider. See the documentation
  • The kubeadm --skip-preflight-checks flag is now deprecated and will be removed in a future release.
  • If you are using the cloud provider API to determine the external host address of the apiserver, set --external-hostname explicitly instead. The cloud provider detection has been deprecated and will be removed in the future (#54516, @dims)

Multicluster

  • Development of Kubernetes Federation has moved to github.com/kubernetes/federation. This move out of tree also means that Federation will begin releasing separately from Kubernetes. Impact:
    • Federation-specific behavior will no longer be included in kubectl
    • kubefed will no longer be released as part of Kubernetes
    • The Federation servers will no longer be included in the hyperkube binary and image. (#53816, @marun)

Node

  • The kubelet --network-plugin-dir flag is removed. This flag was deprecated in version 1.7, and is replaced with --cni-bin-dir. (#53564, @supereagle)
  • kubelet's --cloud-provider flag no longer defaults to "auto-detect". If you want cloud-provider support in kubelet, you must set a specific cloud-provider explicitly. (#53573, @dims)

Network

  • NetworkPolicy objects are now stored in etcd in v1 format. After you upgrade to version 1.9, make sure that all NetworkPolicy objects are migrated to v1. (#51955, @danwinship)
  • The API group/version for the kube-proxy configuration has changed from componentconfig/v1alpha1 to kubeproxy.config.k8s.io/v1alpha1. If you are using a config file for kube-proxy instead of the command line flags, you must change its apiVersion to kubeproxy.config.k8s.io/v1alpha1. (#53645, @xiangpengzhao)
  • The "ServiceNodeExclusion" feature gate must now be enabled for the alpha.service-controller.kubernetes.io/exclude-balancer annotation on nodes to be honored. (#54644, @brendandburns)

Scheduling

  • Taint key unreachable is now in GA.
  • Taint key notReady is changed to not-ready, and is also now in GA.
  • These changes are automatically updated for taints. Tolerations for these taints must be updated manually. Specifically, you must:
    • Change node.alpha.kubernetes.io/notReady to node.kubernetes.io/not-ready
    • Change node.alpha.kubernetes.io/unreachable to node.kubernetes.io/unreachable
  • The node.kubernetes.io/memory-pressure taint now respects the configured whitelist. To use it, you must add it to the whitelist.(#55251, @deads2k)
  • Refactor kube-scheduler configuration (#52428)
    • The kube-scheduler command now supports a --config flag which is the location of a file containing a serialized scheduler configuration. Most other kube-scheduler flags are now deprecated. (#52562, @ironcladlou)
  • Opaque integer resources (OIR), which were (deprecated in v1.8.), have been removed. (#55103, @ConnorDoyle)

Storage

  • [alpha] The LocalPersistentVolumes alpha feature now also requires the VolumeScheduling alpha feature. This is a breaking change, and the following changes are required:
    • The VolumeScheduling feature gate must also be enabled on kube-scheduler and kube-controller-manager components.
    • The NoVolumeNodeConflict predicate has been removed. For non-default schedulers, update your scheduler policy.
    • The CheckVolumeBinding predicate must be enabled in non-default schedulers. (#55039, @msau42)

OpenStack

  • Remove the LbaasV1 of OpenStack cloud provider, currently only support LbaasV2. (#52717, @FengyunPan)

Known Issues

This section contains a list of known issues reported in Kubernetes 1.9 release. The content is populated from the [v1.9.x known issues and FAQ accumulator] #57159.

  • If you are adding Windows Server Virtual Machines as nodes to your Kubernetes environment, there is a compatibility issue with certain virtualization products. Specifically the Windows version of the kubelet.exe calls GetPhysicallyInstalledSystemMemory to get the physical memory installed on Windows machines and reports it as part of node metrics to heapster. This API call fails for VMware and VirtualBox virtualization environments. This issue is not present in bare metal Windows deployments, in Hyper-V, or on some of the popular public cloud providers.

  • If you run kubectl get po while the API server in unreachable, a misleading error is returned: the server doesn't have a resource type "po". To work around this issue, specify the full resource name in the command instead of the abbreviation: kubectl get pods. This issue will be fixed in a future release.

For more information, see #57198.

  • Mutating and validating webhook configurations are continuously polled by the API server (once per second). This issue will be fixed in a future release.

For more information, see #56357.

  • Audit logging is slow because writes to the log are performed synchronously with requests to the log. This issue will be fixed in a future release.

For more information, see #53006.

  • Custom Resource Definitions (CRDs) are not properly deleted under certain conditions. This issue will be fixed in a future release.

For more information, see #56348.

  • API server times out after performing a rolling update of the etcd cluster. This issue will be fixed in a future release.

For more information, see #47131

  • If a namespaced resource is owned by a cluster scoped resource, and the namespaced dependent is processed before the cluster scoped owner has ever been observed by the garbage collector, the dependent will be erroneously deleted.

For more information, see #54940

Deprecations

This section provides an overview of deprecated API versions, options, flags, and arguments. Deprecated means that we intend to remove the capability from a future release. After removal, the capability will no longer work. The sections are organized by SIGs.

API Machinery

  • The kube-apiserver --etcd-quorum-read flag is deprecated and the ability to switch off quorum read will be removed in a future release. (#53795, @xiangpengzhao)
  • The /ui redirect in kube-apiserver is deprecated and will be removed in Kubernetes 1.10. (#53046, @maciaszczykm)
  • etcd2 as a backend is deprecated and support will be removed in Kubernetes 1.13 or 1.14.

Auth

  • Default controller-manager options for --cluster-signing-cert-file and --cluster-signing-key-file are deprecated and will be removed in a future release. (#54495, @mikedanese)
  • RBAC objects are now stored in etcd in v1 format. After upgrading to 1.9, ensure all RBAC objects (Roles, RoleBindings, ClusterRoles, ClusterRoleBindings) are at v1. v1alpha1 support is deprecated and will be removed in a future release. (#52950, @liggitt)

Cluster Lifecycle

  • kube-apiserver: --ssh-user and --ssh-keyfile are now deprecated and will be removed in a future release. Users of SSH tunnel functionality in Google Container Engine for the Master -> Cluster communication should plan alternate methods for bridging master and node networks. (#54433, @dims)
  • The kubeadm --skip-preflight-checks flag is now deprecated and will be removed in a future release.
  • If you are using the cloud provider API to determine the external host address of the apiserver, set --external-hostname explicitly instead. The cloud provider detection has been deprecated and will be removed in the future (#54516, @dims)

Network

  • The NetworkPolicy extensions/v1beta1 API is now deprecated and will be removed in a future release. This functionality has been migrated to a dedicated v1 API - networking.k8s.io/v1. v1beta1 Network Policies can be upgraded to the v1 API with the cluster/update-storage-objects.sh script. Documentation can be found here. (#56425, @cmluciano)

Storage

  • The volume.beta.kubernetes.io/storage-class annotation is deprecated. It will be removed in a future release. For the StorageClass API object, use v1, and in place of the annotation use v1.PersistentVolumeClaim.Spec.StorageClassName and v1.PersistentVolume.Spec.StorageClassName instead. (#53580, @xiangpengzhao)

Scheduling

  • The kube-scheduler command now supports a --config flag, which is the location of a file containing a serialized scheduler configuration. Most other kube-scheduler flags are now deprecated. (#52562, @ironcladlou)

Node

  • The kubelet's --enable-custom-metrics flag is now deprecated. (#54154, @mtaufen)

Notable Changes

Workloads API (apps/v1)

As announced with the release of version 1.8, the Kubernetes Workloads API is at v1 in version 1.9. This API consists of the DaemonSet, Deployment, ReplicaSet and StatefulSet kinds.

API Machinery

Admission Control

  • Admission webhooks are now in beta, and include the following:
    • Mutation support for admission webhooks. (#54892, @caesarxuchao)
    • Webhook admission now takes a config file that describes how to authenticate to webhook servers (#54414, @deads2k)
    • The dynamic admission webhook now supports a URL in addition to a service reference, to accommodate out-of-cluster webhooks. (#54889, @lavalamp)
    • Added namespaceSelector to externalAdmissionWebhook configuration to allow applying webhooks only to objects in the namespaces that have matching labels. (#54727, @caesarxuchao)
  • Metrics are added for monitoring admission plugins, including the new dynamic (webhook-based) ones. (#55183, @jpbetz)
  • The PodSecurityPolicy annotation kubernetes.io/psp on pods is set only once on create. (#55486, @sttts)

API & API server

  • Fixed a bug related to discovery information for scale subresources in the apps API group (#54683, @liggitt)
  • Fixed a bug that prevented client-go metrics from being registered in Prometheus. This bug affected multiple components. (#53434, @crassirostris)

Audit

  • Fixed a bug so that kube-apiserver now waits for open connections to finish before exiting. This fix provides graceful shutdown and ensures that the audit backend no longer drops events on shutdown. (#53695, @hzxuzhonghu)
  • Webhooks now always retry sending if a connection reset error is returned. (#53947, @crassirostris)

Custom Resources

  • Validation of resources defined by a Custom Resource Definition (CRD) is now in beta (#54647, @colemickens)
  • An example CRD controller has been added, at github.com/kubernetes/sample-controller. (#52753, @munnerz)
  • Custom resources served by CustomResourceDefinition objects now support field selectors for metadata.name and metadata.namespace. Also fixed an issue with watching a single object; earlier versions could watch only a collection, and so a watch on an instance would fail. (#53345, @ncdc)

Other

  • kube-apiserver now runs with the default value for service-cluster-ip-range (#52870, @jennybuckley)
  • Add --etcd-compaction-interval to apiserver for controlling request of compaction to etcd3 from apiserver. (#51765, @mitake)
  • The httpstream/spdy calls now support CIDR notation for NO_PROXY (#54413, @kad)
  • Code generation for CRD and User API server types is improved with the addition of two new scripts to k8s.io/code-generator: generate-groups.sh and generate-internal-groups.sh. (#52186, @sttts)
  • [beta] Flag --chunk-size={SIZE} is added to kubectl get to customize the number of results returned in large lists of resources. This reduces the perceived latency of managing large clusters because the server returns the first set of results to the client much more quickly. Pass 0 to disable this feature.(#53768, @smarterclayton)
  • [beta] API chunking via the limit and continue request parameters is promoted to beta in this release. Client libraries using the Informer or ListWatch types will automatically opt in to chunking. (#52949, @smarterclayton)
  • The --etcd-quorum-read flag now defaults to true to ensure correct operation with HA etcd clusters. This flag is deprecated and the flag will be removed in future versions, as well as the ability to turn off this functionality. (#53717, @liggitt)
  • Add events.k8s.io api group with v1beta1 API containing redesigned event type. (#49112, @gmarek)
  • Fixed a bug where API discovery failures were crashing the kube controller manager via the garbage collector. (#55259, @ironcladlou)
  • conversion-gen is now usable in a context without a vendored k8s.io/kubernetes. The Kubernetes core API is removed from default extra-peer-dirs. (#54394, @sttts)
  • Fixed a bug where the client-gen tag for code-generator required a newline between a comment block and a statement. tag shortcomings when newline is omitted (#53893) (#55233, @sttts)
  • The Apiserver proxy now rewrites the URL when a service returns an absolute path with the request's host. (#52556, @roycaihw)
  • The gRPC library is updated to pick up data race fix (#53124) (#53128, @dixudx)
  • Fixed server name verification of aggregated API servers and webhook admission endpoints (#56415, @liggitt)

Apps

  • The kubernetes.io/created-by annotation is no longer added to controller-created objects. Use the metadata.ownerReferences item with controller set to true to determine which controller, if any, owns an object. (#54445, @crimsonfaith91)
  • StatefulSet controller now creates a label for each Pod in a StatefulSet. The label is statefulset.kubernetes.io/pod-name, where pod-name = the name of the Pod. This allows users to create a Service per Pod to expose a connection to individual Pods. (#55329, @kow3ns)
  • DaemonSet status includes a new field named conditions, making it consistent with other workloads controllers. (#55272, @janetkuo)
  • StatefulSet status now supports conditions, making it consistent with other core controllers in v1 (#55268, @foxish)
  • The default garbage collection policy for Deployment, DaemonSet, StatefulSet, and ReplicaSet has changed from OrphanDependents to DeleteDependents when the deletion is requested through an apps/v1 endpoint. (#55148, @dixudx)
    • Clients using older endpoints will be unaffected. This change is only at the REST API level and is independent of the default behavior of particular clients (e.g. this does not affect the default for the kubectl --cascade flag).
    • If you upgrade your client-go libs and use the AppsV1() interface, please note that the default garbage collection behavior is changed.

Auth

Audit

RBAC

  • New permissions have been added to default RBAC roles (#52654, @liggitt):
    • The default admin and edit roles now include read/write permissions
    • The view role includes read permissions on poddisruptionbudget.policy resources.
  • RBAC rules can now match the same subresource on any resource using the form */(subresource). For example, */scale matches requests to replicationcontroller/scale. (#53722, @deads2k)
  • The RBAC bootstrapping policy now allows authenticated users to create selfsubjectrulesreviews. (#56095, @ericchiang)
  • RBAC ClusterRoles can now select other roles to aggregate. (#54005, @deads2k)
  • Fixed an issue with RBAC reconciliation that caused duplicated subjects in some bootstrapped RoleBinding objects on each restart of the API server. (#53239, @enj)

Other

  • Pod Security Policy can now manage access to specific FlexVolume drivers (#53179, @wanghaoran1988)
  • Audit policy files without apiVersion and kind are treated as invalid. (#54267, @ericchiang)
  • Fixed a bug that where forbidden errors were encountered when accessing ReplicaSet and DaemonSets objects via the apps API group. (#54309, @liggitt)
  • Improved PodSecurityPolicy admission latency. (#55643, @tallclair)
  • kube-apiserver: --oidc-username-prefix and --oidc-group-prefix flags are now correctly enabled. (#56175, @ericchiang)
  • If multiple PodSecurityPolicy objects allow a submitted pod, priority is given to policies that do not require default values for any fields in the pod spec. If default values are required, the first policy ordered by name that allows the pod is used. (#52849, @liggitt)
  • A new controller automatically cleans up Certificate Signing Requests that are Approved and Issued, or Denied. (#51840, @jcbsmpsn)
  • PodSecurityPolicies have been added for all in-tree cluster addons (#55509, @tallclair)

GCE

  • Added support for PodSecurityPolicy on GCE: ENABLE_POD_SECURITY_POLICY=true enables the admission controller, and installs policies for default addons. (#52367, @tallclair)

Autoscaling

  • HorizontalPodAutoscaler objects now properly functions on scalable resources in any API group. Fixed by adding a polymorphic scale client. (#53743, @DirectXMan12)
  • Fixed a set of minor issues with Cluster Autoscaler 1.0.1 (#54298, @mwielgus)
  • HPA tolerance is now configurable by setting the horizontal-pod-autoscaler-tolerance flag. (#52275, @mattjmcnaughton)
  • Fixed a bug that allowed the horizontal pod autoscaler to allocate more desiredReplica objects than maxReplica objects in certain instances. (#53690, @mattjmcnaughton)

AWS

  • Nodes can now use instance types (such as C5) that use NVMe. (#56607, @justinsb)
  • Nodes are now unreachable if volumes are stuck in the attaching state. Implemented by applying a taint to the node. (#55558, @gnufied)
  • Volumes are now checked for available state before attempting to attach or delete a volume in EBS. (#55008, @gnufied)
  • Fixed a bug where error log messages were breaking into two lines. (#49826, @dixudx)
  • Fixed a bug so that volumes are now detached from stopped nodes. (#55893, @gnufied)
  • You can now override the health check parameters for AWS ELBs by specifying annotations on the corresponding service. The new annotations are: healthy-threshold, unhealthy-threshold, timeout, interval. The prefix for all annotations is service.beta.kubernetes.io/aws-load-balancer-healthcheck-. (#56024, @dimpavloff)
  • Fixed a bug so that AWS ECR credentials are now supported in the China region. (#50108, @zzq889)
  • Added Amazon NLB support (#53400, @micahhausler)
  • Additional annotations are now properly set or updated for AWS load balancers (#55731, @georgebuckerfield)
  • AWS SDK is updated to version 1.12.7 (#53561, @justinsb)

Azure

  • Fixed several issues with properly provisioning Azure disk storage (#55927, @andyzhangx)
  • A new service annotation service.beta.kubernetes.io/azure-dns-label-name now sets the Azure DNS label for a public IP address. (#47849, @tomerf)
  • Support for GetMountRefs function added; warning messages no longer displayed. (#54670, #52401, @andyzhangx)
  • Fixed an issue where an Azure PersistentVolume object would crash because the value of volumeSource.ReadOnly was set to nil. (#54607, @andyzhangx)
  • Fixed an issue with Azure disk mount failures on CoreOS and some other distros (#54334, @andyzhangx)
  • GRS, RAGRS storage account types are now supported for Azure disks. (#55931, @andyzhangx)
  • Azure NSG rules are now restricted so that external access is allowed only to the load balancer IP. (#54177, @itowlson)
  • Azure NSG rules can be consolidated to reduce the likelihood of hitting Azure resource limits (available only in regions where the Augmented Security Groups preview is available). (#55740, @itowlson)
  • The Azure SDK is upgraded to v11.1.1. (#54971, @itowlson)
  • You can now create Windows mount paths (#51240, @andyzhangx)
  • Fixed a controller manager crash issue on a manually created k8s cluster. (#53694, @andyzhangx)
  • Azure-based clusters now support unlimited mount points. (#54668) (#53629, @andyzhangx)
  • Load balancer reconciliation now considers NSG rules based not only on Name, but also on Protocol, SourcePortRange, DestinationPortRange, SourceAddressPrefix, DestinationAddressPrefix, Access, and Direction. This change makes it possible to update NSG rules under more conditions. (#55752, @kevinkim9264)
  • Custom mountOptions for the azurefile StorageClass object are now respected. Specifically, dir_mode and file_mode can now be customized. (#54674, @andyzhangx)
  • Azure Load Balancer Auto Mode: Services can be annotated to allow auto selection of available load balancers and to provide specific availability sets that host the load balancers (for example, service.beta.kubernetes.io/azure-load-balancer-mode=auto|as1,as2...)

CLI

Kubectl

  • kubectl cp can now copy a remote file into a local directory. (#46762, @bruceauyeung)
  • kubectl cp now honors destination names for directories. A complete directory is now copied; in previous versions only the file contents were copied. (#51215, @juanvallejo)
  • You can now use kubectl get with a fieldSelector. (#50140, @dixudx)
  • Secret data containing Docker registry auth objects is now generated using the config.json format (#53916, @juanvallejo)
  • kubectl apply now calculates the diff between the current and new configurations based on the OpenAPI spec. If the OpenAPI spec is not available, it falls back to baked-in types. (#51321, @mengqiy)
  • kubectl explain now explains apiservices and customresourcedefinition. (Updated to use OpenAPI instead of Swagger 1.2.) (#53228, @apelisse)
  • kubectl get now uses OpenAPI schema extensions by default to select columns for custom types. (#53483, @apelisse)
  • kubectl top node now sorts by name and top pod sorts by namespace. Fixed a bug where results were inconsistently sorted. (#53560, @dixudx)
  • Added --dry-run option to kubectl drain. (#52440, @juanvallejo)
  • Kubectl now outputs <none> for columns specified by -o custom-columns but not found in object, rather than "xxx is not found" (#51750, @jianhuiz)
  • kubectl create pdb no longer sets the min-available field by default. (#53047, @yuexiao-wang)
  • The canonical pronunciation of kubectl is "cube control".
  • Added --raw to kubectl create to POST using the normal transport. (#54245, @deads2k)
  • Added kubectl create priorityclass subcommand (#54858, @wackxu)
  • Fixed an issue where kubectl set commands occasionally encountered conversion errors for ReplicaSet and DaemonSet objects (#53158, @liggitt)

Cluster Lifecycle

API Server

  • [alpha] Added an --endpoint-reconciler-type command-line argument to select the endpoint reconciler to use. The default is to use the 'master-count' reconciler which is the default for 1.9 and in use prior to 1.9. The 'lease' reconciler stores endpoints within the storage api for better cleanup of deleted (or removed) API servers. The 'none' reconciler is a no-op reconciler, which can be used in self-hosted environments. (#51698, @rphillips)

Cloud Provider Integration

  • Added cloud-controller-manager to hyperkube. This is useful as a number of deployment tools run all of the kubernetes components from the hyperkube image/binary. It also makes testing easier as a single binary/image can be built and pushed quickly. (#54197, @colemickens)
  • Added the concurrent service sync flag to the Cloud Controller Manager to allow changing the number of workers. (--concurrent-service-syncs) (#55561, @jhorwit2)
  • kubelet's --cloud-provider flag no longer defaults to "auto-detect". If you want cloud-provider support in kubelet, you must set a specific cloud-provider explicitly. (#53573, @dims)

Kubeadm

  • kubeadm health checks can now be skipped with --ignore-preflight-errors; the --skip-preflight-checks flag is now deprecated and will be removed in a future release. (#56130, @anguslees) (#56072, @kad)
  • You now have the option to use CoreDNS instead of KubeDNS. To install CoreDNS instead of kube-dns, set CLUSTER_DNS_CORE_DNS to 'true'. This support is experimental. (#52501, @rajansandeep) (#55728, @rajansandeep)
  • Added --print-join-command flag for kubeadm token create. (#56185, @mattmoyer)
  • Added a new --etcd-upgrade keyword to kubeadm upgrade apply. When this keyword is specified, etcd's static pod gets upgraded to the etcd version officially recommended for a target kubernetes release. (#55010, @sbezverk)
  • Kubeadm now supports Kubelet Dynamic Configuration on an alpha level. (#55803, @xiangpengzhao)
  • Added support for adding a Windows node (#53553, @bsteciuk)

Juju

  • Added support for SAN entries in the master node certificate. (#54234, @hyperbolic2346)
  • Add extra-args configs for scheduler and controller-manager to kubernetes-master charm (#55185, @Cynerva)
  • Add support for RBAC (#53820, @ktsakalozos)
  • Fixed iptables FORWARD policy for Docker 1.13 in kubernetes-worker charm (#54796, @Cynerva)
  • Upgrading the kubernetes-master units now results in staged upgrades just like the kubernetes-worker nodes. Use the upgrade action in order to continue the upgrade process on each unit such as juju run-action kubernetes-master/0 upgrade (#55990, @hyperbolic2346)
  • Added extra_sans config option to kubeapi-load-balancer charm. This allows the user to specify extra SAN entries on the certificate generated for the load balancer. (#54947, @hyperbolic2346)
  • Added extra-args configs to kubernetes-worker charm (#55334, @Cynerva)

Other

GCP

  • The service account made available on your nodes is now configurable. (#52868, @ihmccreery)
  • GCE nodes with NVIDIA GPUs attached now expose nvidia.com/gpu as a resource instead of alpha.kubernetes.io/nvidia-gpu. (#54826, @mindprince)
  • Docker's live-restore on COS/ubuntu can now be disabled (#55260, @yujuhong)
  • Metadata concealment is now controlled by the ENABLE_METADATA_CONCEALMENT env var. See cluster/gce/config-default.sh for more info. (#54150, @ihmccreery)
  • Masquerading rules are now added by default to GCE/GKE (#55178, @dnardo)
  • Fixed master startup issues with concurrent iptables invocations. (#55945, @x13n)
  • Fixed issue deleting internal load balancers when the firewall resource may not exist. (#53450, @nicksardo)

Instrumentation

Audit

  • Adjust batching audit webhook default parameters: increase queue size, batch size, and initial backoff. Add throttling to the batching audit webhook. Default rate limit is 10 QPS. (#53417, @crassirostris)

Other

  • Fix a typo in prometheus-to-sd configuration, that drops some stackdriver metrics. (#56473, @loburm)
  • [fluentd-elasticsearch addon] Elasticsearch and Kibana are updated to version 5.6.4 (#55400, @mrahbar)
  • fluentd now supports CRI log format. (#54777, @Random-Liu)
  • Bring all prom-to-sd container to the same image version (#54583)
    • Reduce log noise produced by prometheus-to-sd, by bumping it to version 0.2.2. (#54635, @loburm)
  • [fluentd-elasticsearch addon] Elasticsearch service name can be overridden via env variable ELASTICSEARCH_SERVICE_NAME (#54215, @mrahbar)

Multicluster

Federation

  • Kubefed init now supports --imagePullSecrets and --imagePullPolicy, making it possible to use private registries. (#50740, @dixudx)
  • Updated cluster printer to enable --show-labels (#53771, @dixudx)
  • Kubefed init now supports --nodeSelector, enabling you to determine on what node the controller will be installed. (#50749, @dixudx)

Network

IPv6

  • [alpha] IPv6 support has been added. Notable IPv6 support details include:
    • Support for IPv6-only Kubernetes cluster deployments. <span style="text-decoration:underline;">Note:</span> This feature does not provide dual-stack support.
    • Support for IPv6 Kubernetes control and data planes.
    • Support for Kubernetes IPv6 cluster deployments using kubeadm.
    • Support for the iptables kube-proxy backend using ip6tables.
    • Relies on CNI 0.6.0 binaries for IPv6 pod networking.
    • Adds IPv6 support for kube-dns using SRV records.
    • Caveats
      • Only the CNI bridge and local-ipam plugins have been tested for the alpha release, although other CNI plugins do support IPv6.
      • HostPorts are not supported.
  • An IPv6 network mask for pod or cluster cidr network must be /66 or longer. For example: 2001:db1::/66, 2001:dead:beef::/76, 2001:cafe::/118 are supported. 2001:db1::/64 is not supported
  • For details, see the complete list of merged pull requests for IPv6 support.

IPVS

  • You can now use the --cleanup-ipvs flag to tell kube-proxy whether to flush all existing ipvs rules in on startup (#56036, @m1093782566)
  • Graduate kube-proxy IPVS mode to beta. (#56623, @m1093782566)

Kube-Proxy

  • Added iptables rules to allow Pod traffic even when default iptables policy is to reject. (#52569, @tmjd)
  • You can once again use 0 values for conntrack min, max, max per core, tcp close wait timeout, and tcp established timeout; this functionality was broken in 1.8. (#55261, @ncdc)

CoreDNS

  • You now have the option to use CoreDNS instead of KubeDNS. To install CoreDNS instead of kube-dns, set CLUSTER_DNS_CORE_DNS to 'true'. This support is experimental. (#52501, @rajansandeep) (#55728, @rajansandeep)

Other

  • Pod addresses will now be removed from the list of endpoints when the pod is in graceful termination. (#54828, @freehan)
  • You can now use a new supported service annotation for AWS clusters, service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy, which lets you specify which predefined AWS SSL policy you would like to use. (#54507, @micahhausler)
  • Termination grace period for the calico/node add-on DaemonSet has been eliminated, reducing downtime during a rolling upgrade or deletion. (#55015, @fasaxc)
  • Fixed bad conversion in host port chain name generating func which led to some unreachable host ports. (#55153, @chenchun)
  • Fixed IPVS availability check (#51874, @vfreex)
  • The output for kubectl describe networkpolicy * has been enhanced to be more useful. (#46951, @aanm)
  • Kernel modules are now loaded automatically inside a kube-proxy pod (#52003, @vfreex)
  • Improve resilience by annotating kube-dns addon with podAntiAffinity to prefer scheduling on different nodes. (#52193, @StevenACoffman)
  • [alpha] Added DNSConfig field to PodSpec. "None" mode for DNSPolicy is now supported. (#55848, @MrHohn)
  • You can now add "options" to the host's /etc/resolv.conf (or --resolv-conf), and they will be copied into pod's resolv.conf when dnsPolicy is Default. Being able to customize options is important because it is common to leverage options to fine-tune the behavior of DNS client. (#54773, @phsiao)
  • Fixed a bug so that the service controller no longer retries if doNotRetry service update fails. (#54184, @MrHohn)
  • Added --no-negcache flag to kube-dns to prevent caching of NXDOMAIN responses. (#53604, @cblecker)

Node

Pod API

  • A single value in metadata.annotations/metadata.labels can now be passed into the containers via the Downward API. (#55902, @yguo0905)
  • Pods will no longer briefly transition to a "Pending" state during the deletion process. (#54593, @dashpole)
  • Added pod-level local ephemeral storage metric to the Summary API. Pod-level ephemeral storage reports the total filesystem usage for the containers and emptyDir volumes in the measured Pod. (#55447, @jingxu97)

Hardware Accelerators

  • Kubelet now exposes metrics for NVIDIA GPUs attached to the containers. (#55188, @mindprince)
  • The device plugin Alpha API no longer supports returning artifacts per device as part of AllocateResponse. (#53031, @vishh)
  • Fix to ignore extended resources that are not registered with kubelet during container resource allocation. (#53547, @jiayingz)

Container Runtime

  • [alpha] cri-tools: CLI and validation tools for CRI is now v1.0.0-alpha.0. This release mainly focuses on UX improvements. [@feiskyer]
    • Make crictl command more user friendly and add more subcommands.
    • Integrate with CRI verbose option to provide extra debug information.
    • Update CRI to kubernetes v1.9.
    • Bug fixes in validation test suites.
  • [beta] cri-containerd: CRI implementation for containerd is now v1.0.0-beta.0, [@Random-Liu]
  • [stable] cri-o: CRI implementation for OCI-based runtimes is now v1.9. [@mrunalp]
    • Pass all the Kubernetes 1.9 end-to-end test suites and now gating PRs as well
    • Pass all the CRI validation tests
    • Release has been focused on bug fixes, stability and performance with runc and Clear Containers
    • Minikube integration
  • [stable] frakti: CRI implementation for hypervisor-based runtimes is now v1.9. [@resouer]
    • Added ARM64 release. Upgraded to CNI 0.6.0, added block device as Pod volume mode. Fixed CNI plugin compatibility.
    • Passed all CRI validation conformance tests and node end-to-end conformance tests.
  • [alpha] rktlet: CRI implementation for the rkt runtime is now v0.1.0. [@iaguis]
    • This is the first release of rktlet and it implements support for the CRI including fetching images, running pods, CNI networking, logging and exec. This release passes 129/145 Kubernetes e2e conformance tests.
  • Container Runtime Interface API change. [@yujuhong]
    • A new field is added to CRI container log format to support splitting a long log line into multiple lines. (#55922, @Random-Liu)
    • CRI now supports debugging via a verbose option for status functions. (#53965, @Random-Liu)
    • Kubelet can now provide full summary api support for the CRI container runtime, with the exception of container log stats. (#55810, @abhi)
    • CRI now uses the correct localhost seccomp path when provided with input in the format of localhost//profileRoot/profileName. (#55450, @feiskyer)

Kubelet

  • The EvictionHard, EvictionSoft, EvictionSoftGracePeriod, EvictionMinimumReclaim, SystemReserved, and KubeReserved fields in the KubeletConfiguration object (kubeletconfig/v1alpha1) are now of type map[string]string, which facilitates writing JSON and YAML files. (#54823, @mtaufen)
  • Relative paths in the Kubelet's local config files (--init-config-dir) will now be resolved relative to the location of the containing files. (#55648, @mtaufen)
  • It is now possible to set multiple manifest URL headers with the kubelet's --manifest-url-header flag. Multiple headers for the same key will be added in the order provided. The ManifestURLHeader field in KubeletConfiguration object (kubeletconfig/v1alpha1) is now a map[string][]string, which facilitates writing JSON and YAML files. (#54643, @mtaufen)
  • The Kubelet's feature gates are now specified as a map when provided via a JSON or YAML KubeletConfiguration, rather than as a string of key-value pairs, making them less awkward for users. (#53025, @mtaufen)
Other
  • Fixed a performance issue (#51899) identified in large-scale clusters when deleting thousands of pods simultaneously across hundreds of nodes, by actively removing containers of deleted pods, rather than waiting for periodic garbage collection and batching resulting pod API deletion requests. (#53233, @dashpole)
  • Problems deleting local static pods have been resolved. (#48339, @dixudx)
  • CRI now only calls UpdateContainerResources when cpuset is set. (#53122, @resouer)
  • Containerd monitoring is now supported. (#56109, @dashpole)
  • deviceplugin has been extended to more gracefully handle the full device plugin lifecycle, including: (#55088, @jiayingz)
    • Kubelet now uses an explicit cm.GetDevicePluginResourceCapacity() function that makes it possible to more accurately determine what resources are inactive and return a more accurate view of available resources.
    • Extends the device plugin checkpoint data to record registered resources so that we can finish resource removing devices even upon kubelet restarts.
    • Passes sourcesReady from kubelet to the device plugin to avoid removing inactive pods during the grace period of kubelet restart.
    • Extends the gpu_device_plugin e2e_node test to verify that scheduled pods can continue to run even after a device plugin deletion and kubelet restart.
  • The NodeController no longer supports kubelet 1.2. (#48996, @k82cn)
  • Kubelet now provides more specific events via FailedSync when unable to sync a pod. (#53857, @derekwaynecarr)
  • You can now disable AppArmor by setting the AppArmor profile to unconfined. (#52395, @dixudx)
  • ImageGCManage now consumes ImageFS stats from StatsProvider rather than cadvisor. (#53094, @yguo0905)
  • Hyperkube now supports the support --experimental-dockershim kubelet flag. (#54508, @ivan4th)
  • Kubelet no longer removes default labels from Node API objects on startup (#54073, @liggitt)
  • The overlay2 container disk metrics for Docker and CRI-O now work properly. (#54827, @dashpole)
  • Removed docker dependency during kubelet start up. (#54405, @resouer)
  • Added Windows support to the system verification check. (#53730, @bsteciuk)
  • Kubelet no longer removes unregistered extended resource capacities from node status; cluster admins will have to manually remove extended resources exposed via device plugins when they the remove plugins themselves. (#53353, @jiayingz)
  • The stats summary network value now takes into account multiple network interfaces, and not just eth0. (#52144, @andyxning)
  • Base images have been bumped to Debian Stretch (9). (#52744, @rphillips)

OpenStack

  • OpenStack Cinder support has been improved:
  • Load balancing is now more flexible:
    • The OpenStack LBaaS v2 Provider is now configurable. (#54176, @gonzolino)
    • OpenStack Octavia v2 is now supported as a load balancer provider in addition to the existing support for the Neutron LBaaS V2 implementation. Neutron LBaaS V1 support has been removed. (#55393, @jamiehannaford)
  • OpenStack security group support has been beefed up (#50836, @FengyunPan):
    • Kubernetes will now automatically determine the security group for the node
    • Nodes can now belong to multiple security groups

Scheduling

Hardware Accelerators

  • Add ExtendedResourceToleration admission controller. This facilitates creation of dedicated nodes with extended resources. If operators want to create dedicated nodes with extended resources (such as GPUs, FPGAs, and so on), they are expected to taint the node with extended resource name as the key. This admission controller, if enabled, automatically adds tolerations for such taints to pods requesting extended resources, so users don't have to manually add these tolerations. (#55839, @mindprince)

Other

  • Scheduler cache ignores updates to an assumed pod if updates are limited to pod annotations. (#54008, @yguo0905)
  • Issues with namespace deletion have been resolved. (#53720, @shyamjvs) (#53793, @wojtek-t)
  • Pod preemption has been improved.
    • Now takes PodDisruptionBudget into account. (#56178, @bsalamat)
    • Nominated pods are taken into account during scheduling to avoid starvation of higher priority pods. (#55933, @bsalamat)
  • Fixed 'Schedulercache is corrupted' error in kube-scheduler (#55262, @liggitt)
  • The kube-scheduler command now supports a --config flag which is the location of a file containing a serialized scheduler configuration. Most other kube-scheduler flags are now deprecated. (#52562, @ironcladlou)
  • A new scheduling queue helps schedule the highest priority pending pod first. (#55109, @bsalamat)
  • A Pod can now listen to the same port on multiple IP addresses. (#52421, @WIZARD-CXY)
  • Object count quotas supported on all standard resources using count/<resource>.<group> syntax (#54320, @derekwaynecarr)
  • Apply algorithm in scheduler by feature gates. (#52723, @k82cn)
  • A new priority function ResourceLimitsPriorityMap (disabled by default and behind alpha feature gate and not part of the scheduler's default priority functions list) that assigns a lowest possible score of 1 to a node that satisfies one or both of input pod's cpu and memory limits, mainly to break ties between nodes with same scores. (#55906, @aveshagarwal)
  • Kubelet evictions now take pod priority into account (#53542, @dashpole)
  • PodTolerationRestriction admisson plugin: if namespace level tolerations are empty, now they override cluster level tolerations. (#54812, @aveshagarwal)

Storage

  • [stable] PersistentVolume and PersistentVolumeClaim objects must now have a capacity greater than zero.
  • [stable] Mutation of PersistentVolumeSource after creation is no longer allowed
  • [alpha] Deletion of PersistentVolumeClaim objects that are in use by a pod no longer permitted (if alpha feature is enabled).
  • [alpha] Container Storage Interface
    • New CSIVolumeSource enables Kubernetes to use external CSI drivers to provision, attach, and mount volumes.
  • [alpha] Raw block volumes
    • Support for surfacing volumes as raw block devices added to Kubernetes storage system.
    • Only Fibre Channel volume plugin supports exposes this functionality, in this release.
  • [alpha] Volume resizing
    • Added file system resizing for the following volume plugins: GCE PD, Ceph RBD, AWS EBS, OpenStack Cinder
  • [alpha] Topology Aware Volume Scheduling
    • Improved volume scheduling for Local PersistentVolumes, by allowing the scheduler to make PersistentVolume binding decisions while respecting the Pod's scheduling requirements.
    • Dynamic provisioning is not supported with this feature yet.
  • [alpha] Containerized mount utilities
    • Allow mount utilities, used to mount volumes, to run inside a container instead of on the host.
  • Bug Fixes
    • ScaleIO volume plugin is no longer dependent on the drv_cfg binary, so a Kubernetes cluster can easily run a containerized kubelet. (#54956, @vladimirvivien)
    • AWS EBS Volumes are detached from stopped AWS nodes. (#55893, @gnufied)
    • AWS EBS volumes are detached if attached to a different node than expected. (#55491, @gnufied)
    • PV Recycle now works in environments that use architectures other than x86. (#53958, @dixudx)
    • Pod Security Policy can now manage access to specific FlexVolume drivers.(#53179, @wanghaoran1988)
    • To prevent unauthorized access to CHAP Secrets, you can now set the secretNamespace storage class parameters for the following volume types:
    • In GCE multizonal clusters, PersistentVolume objects will no longer be dynamically provisioned in zones without nodes. (#52322, @davidz627)
    • Multi Attach PVC errors and events are now more useful and less noisy. (#53401, @gnufied)
    • The compute-rw scope has been removed from GCE nodes (#53266, @mikedanese)
    • Updated vSphere cloud provider to support k8s cluster spread across multiple vCenters (#55845, @rohitjogvmw)
    • vSphere: Fix disk is not getting detached when PV is provisioned on clustered datastore. (#54438, @pshahzeb)
    • If a non-absolute mountPath is passed to the kubelet, it must now be prefixed with the appropriate root path. (#55665, @brendandburns)

External Dependencies

v1.9.0-beta.2

Documentation & Examples

Downloads for v1.9.0-beta.2

filenamesha256 hash
kubernetes.tar.gze5c88addf6aca01635f283021a72e05be99daf3e87fd3cda92477d0ed63c2d11
kubernetes-src.tar.gz2419a0ef3681460b64eefc083d07377786b308f6cc62d0618a5c74dfb4729b03

Client Binaries

filenamesha256 hash
kubernetes-client-darwin-386.tar.gz68d971576c3e9a16fb736f06c07ce53b8371fc67c2f37fb60e9f3a366cd37a80
kubernetes-client-darwin-amd64.tar.gz36251b7b6043adb79706ac115181aa7ecf365ced9198a4c192f1fbc2817d030c
kubernetes-client-linux-386.tar.gz585a3dd6a3440988bce3f83ea14fb9a0a18011bc62e28959301861faa06d6da9
kubernetes-client-linux-amd64.tar.gz169769d6030d8c1d9d9bc01408b62ea3275d4632a7de85392fc95a48feeba522
kubernetes-client-linux-arm64.tar.gz7841c2af49be9ae04cda305165b172021c0e72d809c2271d05061330c220256b
kubernetes-client-linux-arm.tar.gz9ab32843cec68b036de83f54a68c2273a913be5180dc20b5cf1e084b314a9a2d
kubernetes-client-linux-ppc64le.tar.gz5a2bb39b78ef381382f9b8aac17d5dbcbef08a80ad3518ff2cf6c65bd7a6d07d
kubernetes-client-linux-s390x.tar.gzddf4b3780f5879b9fb9115353cc26234cfc3a6db63a3cd39122340189a4bf0ca
kubernetes-client-windows-386.tar.gz5960a0a50c92a788e90eca9d85a1d12ff1d41264816b55b3a1a28ffd3f6acf93
kubernetes-client-windows-amd64.tar.gzd85778ace9bf25f5d3626aef3a9419a2c4aaa3847d5e0c2bf34d4dd8ae6b5205

Server Binaries

filenamesha256 hash
kubernetes-server-linux-amd64.tar.gz43e16b3d79c2805d712fd61ed6fd110d9db09a60d39584ef78c24821eb32b77a
kubernetes-server-linux-arm64.tar.gz8580e454e6c467a30687ff5c85248919b3c0d2d0114e28cb3bf64d2e8998ff00
kubernetes-server-linux-arm.tar.gzd2e767be85ebf7c6c537c8e796e8fe0ce8a3f2ca526984490646acd30bf5e6fc
kubernetes-server-linux-ppc64le.tar.gz81dd9072e805c181b4db2dfd00fe2bdb43c00da9e07b50285bce703bfd0d75ba
kubernetes-server-linux-s390x.tar.gzf432c816c755d05e62cb5d5e8ac08dcb60d0df6d5121e1adaf42a32de65d6174

Node Binaries

filenamesha256 hash
kubernetes-node-linux-amd64.tar.gz2bf2268735ca4ecbdca1a692b25329d6d9d4805963cbe0cfcbb92fc725c42481
kubernetes-node-linux-arm64.tar.gz3bb4a695fd2e4fca1c77283c1ad6c2914d12b33d9c5f64ac9c630a42d5e30ab2
kubernetes-node-linux-arm.tar.gz331c1efadf99dcb634c8da301349e3be63d27a5c5f06cc124b59fcc8b8a91cb0
kubernetes-node-linux-ppc64le.tar.gzab036fdb64ed4702d7dbbadddf77af90de35f73aa13854bb5accf82acc95c7e6
kubernetes-node-linux-s390x.tar.gz8257af566f98325549de320d2167c1f56fd137b6225c70f6c1e34507ba124a1f
kubernetes-node-windows-amd64.tar.gz4146fcb5bb6bf3e04641b27e4aa8501649178716fa16bd9bcb7f1fe3449db7f2

Changelog since v1.9.0-beta.1

Other notable changes

v1.9.0-beta.1

Documentation & Examples

Downloads for v1.9.0-beta.1

filenamesha256 hash
kubernetes.tar.gzffdcf0f7cd972340bc666395d759fc18573a32775d38ed3f4fd99d4369e856e4
kubernetes-src.tar.gz09bee9a955987d53c7a65d2f1a3129854ca3a34f9fb38218f0c58f5bd603494a

Client Binaries

filenamesha256 hash
kubernetes-client-darwin-386.tar.gz9d54db976ca7a12e9208e5595b552b094e0cc532b49ba6e919d776e52e56f4a8
kubernetes-client-darwin-amd64.tar.gz0a22af2c6c84ff8b3022c0ecebf4ba3021048fceddf7375c87c13a83488ffe2c
kubernetes-client-linux-386.tar.gz84bb638c8e61d7a7b415d49d76d166f3924052338c454d1ae57ae36eb37445c6
kubernetes-client-linux-amd64.tar.gz08b56240288d17f147485e79c5f6594391c5b46e26450d64e7510f65db1f9a79
kubernetes-client-linux-arm64.tar.gz7206573b131a8915d3bc14aa660fb44890ed79fdbd498bc8f9951c221aa12ea5
kubernetes-client-linux-arm.tar.gz7ad21796b0e0a9d247beb41d6b3a3d0aaa822b85adae4c90533ba0ef94c05b2e
kubernetes-client-linux-ppc64le.tar.gz2076328ca0958a96c8f551b91a393aa2d6fc24bef92991a1a4d9fc8df52519a7
kubernetes-client-linux-s390x.tar.gz17ac0aba9a4e2003cb3d06bd631032b760d1a2d521c60a25dc26687aadb5ba14
kubernetes-client-windows-386.tar.gz3a2bebd4adb6e1bf2b30a8cedb7ec212fc43c4b02e26a0a60c3429e478a86073
kubernetes-client-windows-amd64.tar.gzfcc852e97f0e64d1025344aefd042ceff05227bfded80142bfa99927de1a5f0e

Server Binaries

filenamesha256 hash
kubernetes-server-linux-amd64.tar.gz7ed2a789b86f258f1739cb165276150512a171a715da9372aeff000e946548fd
kubernetes-server-linux-arm64.tar.gze4e04a33698ac665a3e61fd8d60d4010fec6b0e3b0627dee9a965c2c2a510e3a
kubernetes-server-linux-arm.tar.gzbefce41457fc15c8fadf37ee5bf80b83405279c60665cfb9ecfc9f61fcd549c7
kubernetes-server-linux-ppc64le.tar.gze59e4fb84d6b890e9c6cb216ebb20546212e6c14feb077d9d0761c88e2685f4c
kubernetes-server-linux-s390x.tar.gz0aa47d01907ea78b9a1a8001536d5091fca93409b81bac6eb3e90a4dff6c3faa

Node Binaries

filenamesha256 hash
kubernetes-node-linux-amd64.tar.gz107bfaf72b8b6d3b5c163e61ed169c89288958750636c16bc3d781cf94bf5f4c
kubernetes-node-linux-arm64.tar.gz6bc58e913a2467548664ece743617a1e595f6223100a1bad27e9a90bdf2e2927
kubernetes-node-linux-arm.tar.gzd4ff8f37d7c95f7ca3aca30fa3c191f2cc5e48f0159ac6a5395ec09092574baa
kubernetes-node-linux-ppc64le.tar.gza88d65343ccb515c4eaab11352e69afee4a19c7fa345b08aaffa854b225cf305
kubernetes-node-linux-s390x.tar.gz16d6a67d18273460cab4c293a5b130d4827f41ee4bf5b79b07c60ef517f580cd
kubernetes-node-windows-amd64.tar.gzf086659462b6dcdd78abdf13bed339dd67c1111931bae962044aa4ae2396921d

Changelog since v1.9.0-alpha.3

Action Required

  • Adds alpha support for volume scheduling, which allows the scheduler to make PersistentVolume binding decisions while respecting the Pod's scheduling requirements. Dynamic provisioning is not supported with this feature yet. (#55039, @msau42)
    • Action required for existing users of the LocalPersistentVolumes alpha feature:
      • The VolumeScheduling feature gate also has to be enabled on kube-scheduler and kube-controller-manager.
      • The NoVolumeNodeConflict predicate has been removed. For non-default schedulers, update your scheduler policy.
      • The CheckVolumeBinding predicate has to be enabled in non-default schedulers.
  • Action required: (#56004, @caesarxuchao)
    • The admission/v1alpha1 API has graduated to v1beta1. Please delete your existing webhooks before upgrading the cluster, and update your admission webhooks to use the latest API, because the API has backwards incompatible changes.
    • The webhook registration related part of the admissionregistration API has graduated to v1beta1. Please delete your existing configurations before upgrading the cluster, and update your configuration file to use the latest API.
  • [action required] kubeadm join: Error out if CA pinning isn't used or opted out of (#55468, @yuexiao-wang) * kubeadm now requires the user to specify either the --discovery-token-ca-cert-hash flag or the --discovery-token-unsafe-skip-ca-verification flag.

Other notable changes

  • A new priority function ResourceLimitsPriorityMap (disabled by default and behind alpha feature gate and not part of the scheduler's default priority functions list) that assigns a lowest possible score of 1 to a node that satisfies one or both of input pod's cpu and memory limits, mainly to break ties between nodes with same scores. (#55906, @aveshagarwal)
  • AWS: Fix detaching volume from stopped nodes. (#55893, @gnufied)
  • Fix stats summary network value when multiple network interfaces are available. (#52144, @andyxning)
  • Fix a typo in prometheus-to-sd configuration, that drops some stackdriver metrics. (#56473, @loburm)
  • Fixes server name verification of aggregated API servers and webhook admission endpoints (#56415, @liggitt)
  • OpenStack cloud provider supports Cinder v3 API. (#52910, @FengyunPan)
  • kube-up: Add optional addon CoreDNS. (#55728, @rajansandeep)
    • Install CoreDNS instead of kube-dns by setting CLUSTER_DNS_CORE_DNS value to 'true'.
  • kubeadm health checks can also be skipped with --ignore-checks-errors (#56130, @anguslees)
  • Adds kubeadm support for using ComponentConfig for the kube-proxy (#55972, @rpothier)
  • Pod Security Policy can now manage access to specific FlexVolume drivers (#53179, @wanghaoran1988)
  • PVC Finalizing Controller is introduced in order to prevent deletion of a PVC that is being used by a pod. (#55824, @pospispa)
  • Kubelet can provide full summary api support except container log stats for CRI container runtime now. (#55810, @abhi)
  • Add support for resizing EBS disks (#56118, @gnufied)
  • Add PodDisruptionBudget support during pod preemption (#56178, @bsalamat)
  • Fix CRI localhost seccomp path in format localhost//profileRoot/profileName. (#55450, @feiskyer)
  • kubeadm: Add CoreDNS support for kubeadm "upgrade" and "alpha phases addons". (#55952, @rajansandeep)
  • The default garbage collection policy for Deployment, DaemonSet, StatefulSet, and ReplicaSet has changed from OrphanDependents to DeleteDependents when the deletion is requested through an apps/v1 endpoint. Clients using older endpoints will be unaffected. This change is only at the REST API level and is independent of the default behavior of particular clients (e.g. this does not affect the default for the kubectl --cascade flag). (#55148, @dixudx)
    • If you upgrade your client-go libs and use the AppsV1() interface, please note that the default garbage collection behavior is changed.
  • Add resize support for ceph RBD (#52767, @NickrenREN)
  • Expose single annotation/label via downward API (#55902, @yguo0905)
  • kubeadm: added --print-join-command flag for kubeadm token create. (#56185, @mattmoyer)
  • Implement kubelet side file system resizing. Also implement GCE PD resizing (#55815, @gnufied)
  • Improved PodSecurityPolicy admission latency, but validation errors are no longer limited to only errors from authorized policies. (#55643, @tallclair)
  • Add containerd monitoring support (#56109, @dashpole)
  • Add pod-level CPU and memory stats from pod cgroup information (#55969, @jingxu97)
  • kubectl apply use openapi to calculate diff be default. It will fall back to use baked-in types when openapi is not available. (#51321, @mengqiy)
  • It is now possible to override the healthcheck parameters for AWS ELBs via annotations on the corresponding service. The new annotations are healthy-threshold, unhealthy-threshold, timeout, interval (all prefixed with service.beta.kubernetes.io/aws-load-balancer-healthcheck-) (#56024, @dimpavloff)
  • Adding etcd version display to kubeadm upgrade plan subcommand (#56156, @sbezverk)
  • [fluentd-gcp addon] Fixes fluentd deployment on GCP when custom resources are set. (#55950, @crassirostris)
  • [fluentd-elasticsearch addon] Elasticsearch and Kibana are updated to version 5.6.4 (#55400, @mrahbar)
  • install ipset in debian-iptables docker image (#56115, @m1093782566)
  • Add cleanup-ipvs flag for kube-proxy (#56036, @m1093782566)
  • Remove opaque integer resources (OIR) support (deprecated in v1.8.) (#55103, @ConnorDoyle)
  • Implement volume resize for cinder (#51498, @NickrenREN)
  • Block volumes Support: FC plugin update (#51493, @mtanino)
  • kube-apiserver: fixed --oidc-username-prefix and --oidc-group-prefix flags which previously weren't correctly enabled (#56175, @ericchiang)
  • New kubeadm flag --ignore-preflight-errors that enables to decrease severity of each individual error to warning. (#56072, @kad)
    • Old flag --skip-preflight-checks is marked as deprecated and acts as --ignore-preflight-errors=all
  • Block volumes Support: CRI, volumemanager and operationexecutor changes (#51494, @mtanino)
  • StatefulSet controller will create a label for each Pod in a StatefulSet. The label is named statefulset.kubernetes.io/pod-name and it is equal to the name of the Pod. This allows users to create a Service per Pod to expose a connection to individual Pods. (#55329, @kow3ns)
  • Initial basic bootstrap-checkpoint support (#50984, @timothysc)
  • Add DNSConfig field to PodSpec and support "None" mode for DNSPolicy (Alpha). (#55848, @MrHohn)
  • Add pod-level local ephemeral storage metric in Summary API. Pod-level ephemeral storage reports the total filesystem usage for the containers and emptyDir volumes in the measured Pod. (#55447, @jingxu97)
  • Kubernetes update Azure nsg rules based on not just difference in Name, but also in Protocol, SourcePortRange, DestinationPortRange, SourceAddressPrefix, DestinationAddressPrefix, Access, and Direction. (#55752, @kevinkim9264)
  • Add support to take nominated pods into account during scheduling to avoid starvation of higher priority pods. (#55933, @bsalamat)
  • Add Amazon NLB support - Fixes #52173 (#53400, @micahhausler)
  • Extends deviceplugin to gracefully handle full device plugin lifecycle. (#55088, @jiayingz)
  • A new field is added to CRI container log format to support splitting a long log line into multiple lines. (#55922, @Random-Liu)
  • [advanced audit]add a policy wide omitStage (#54634, @CaoShuFeng)
  • Fix a bug in GCE multizonal clusters where PersistentVolumes were sometimes created in zones without nodes. (#52322, @davidz627)
  • With this change (#55845, @rohitjogvmw)
      • User should be able to create k8s cluster which spans across multiple ESXi clusters, datacenters or even vCenters.
      • vSphere cloud provider (VCP) uses OS hostname and not vSphere Inventory VM Name.
    • That means, now VCP can handle cases where user changes VM inventory name.
      • VCP can handle cases where VM migrates to other ESXi cluster or datacenter or vCenter.
    • The only requirement is the shared storage. VCP needs shared storage on all Node VMs.
  • The RBAC bootstrapping policy now allows authenticated users to create selfsubjectrulesreviews. (#56095, @ericchiang)
  • Defaulting of controller-manager options for --cluster-signing-cert-file and --cluster-signing-key-file is deprecated and will be removed in a later release. (#54495, @mikedanese)
  • Add ExtendedResourceToleration admission controller. This facilitates creation of dedicated nodes with extended resources. If operators want to create dedicated nodes with extended resources (like GPUs, FPGAs etc.), they are expected to taint the node with extended resource name as the key. This admission controller, if enabled, automatically adds tolerations for such taints to pods requesting extended resources, so users don't have to manually add these tolerations. (#55839, @mindprince)
  • Move unreachable taint key out of alpha. (#54208, @resouer)
    • Please note the existing pods with the alpha toleration should be updated by user himself to tolerate the GA taint.
  • add GRS, RAGRS storage account type support for azure disk (#55931, @andyzhangx)
  • Upgrading the kubernetes-master units now results in staged upgrades just like the kubernetes-worker nodes. Use the upgrade action in order to continue the upgrade process on each unit such as juju run-action kubernetes-master/0 upgrade (#55990, @hyperbolic2346)
  • Using ipset doing SNAT and packet filtering in IPVS kube-proxy (#54219, @m1093782566)
  • Add a new scheduling queue that helps schedule the highest priority pending pod first. (#55109, @bsalamat)
  • Adds to kubeadm upgrade apply, a new --etcd-upgrade keyword. When this keyword is specified, etcd's static pod gets upgraded to the etcd version officially recommended for a target kubernetes release. (#55010, @sbezverk)
  • Adding vishh as an reviewer/approver for hack directory (#54007, @vishh)
  • The GenericAdmissionWebhook is renamed as ValidatingAdmissionWebhook. Please update you apiserver configuration file to use the new name to pass to the apiserver's --admission-control flag. (#55988, @caesarxuchao)
  • iSCSI Persistent Volume Sources can now reference CHAP Secrets in namespaces other than the namespace of the bound Persistent Volume Claim (#51530, @rootfs)
  • Bugfix: master startup script on GCP no longer fails randomly due to concurrent iptables invocations. (#55945, @x13n)
  • fix azure disk storage account init issue (#55927, @andyzhangx)
  • Allow code-generator tags in the 2nd closest comment block and directly above a statement. (#55233, @sttts)
  • Ensure additional resource tags are set/updated AWS load balancers (#55731, @georgebuckerfield)
  • kubectl get will now use OpenAPI schema extensions by default to select columns for custom types. (#53483, @apelisse)
  • AWS: Apply taint to a node if volumes being attached to it are stuck in attaching state (#55558, @gnufied)
  • Kubeadm now supports for Kubelet Dynamic Configuration. (#55803, @xiangpengzhao)
  • Added mutation supports to admission webhooks. (#54892, @caesarxuchao)
  • Upgrade to go1.9.2 (#55420, @cblecker)
  • If a non-absolute mountPath is passed to the kubelet, prefix it with the appropriate root path. (#55665, @brendandburns)
  • action-required: please update your admission webhook to use the latest Admission API. (#55829, @cheftako)
    • admission/v1alpha1#AdmissionReview now contains AdmissionRequest and AdmissionResponse. AdmissionResponse includes a Patch field to allow mutating webhooks to send json patch to the apiserver.
  • support mount options in azure file (#54674, @andyzhangx)
  • Support AWS ECR credentials in China (#50108, @zzq889)
  • The EvictionHard, EvictionSoft, EvictionSoftGracePeriod, EvictionMinimumReclaim, SystemReserved, and KubeReserved fields in the KubeletConfiguration object (kubeletconfig/v1alpha1) are now of type map[string]string, which facilitates writing JSON and YAML files. (#54823, @mtaufen)
  • Added service annotation for AWS ELB SSL policy (#54507, @micahhausler)
  • Implement correction mechanism for dangling volumes attached for deleted pods (#55491, @gnufied)
  • Promote validation for custom resources defined through CRD to beta (#54647, @colemickens)
  • Octavia v2 now supported as a LB provider (#55393, @jamiehannaford)
  • Kubelet now exposes metrics for NVIDIA GPUs attached to the containers. (#55188, @mindprince)
  • Addon manager supports HA masters. (#55782, @x13n)
  • Fix kubeadm reset crictl command (#55717, @runcom)
  • Fix code-generators to produce correct code when GroupName, PackageName and/or GoName differ. (#55614, @sttts)
  • Fixes bad conversion in host port chain name generating func which leads to some unreachable host ports. (#55153, @chenchun)
  • Relative paths in the Kubelet's local config files (--init-config-dir) will be resolved relative to the location of the containing files. (#55648, @mtaufen)
  • kubeadm: Fix a bug on some OSes where the kubelet tried to mount a volume path that is non-existent and on a read-only filesystem (#55320, @andrewrynhard)
  • add hostIP and protocol to the original hostport predicates procedure in scheduler. (#52421, @WIZARD-CXY)

v1.9.0-alpha.3

Documentation & Examples

Downloads for v1.9.0-alpha.3

filenamesha256 hash
kubernetes.tar.gzdce2a70ca51fb4f8979645330f36c346b9c02be0501708380ae50956485a53a4
kubernetes-src.tar.gz4a8c8eaf32c83968e18f75888ae0d432210262090893cad0a105eebab82b0302

Client Binaries

filenamesha256 hash
kubernetes-client-darwin-386.tar.gz354d6c8d65e4248c3393a3789e9394b8c31c63da4c42f3da60c7b8bc4713ad51
kubernetes-client-darwin-amd64.tar.gz98c53e4108276535218f4c89c58974121cc28308cecf5bca676f68fa083a62c5
kubernetes-client-linux-386.tar.gzc0dc219073dcae6fb654f33ca6d83faf5f37a2dcba3cc86b32ea5f9e18054faa
kubernetes-client-linux-amd64.tar.gzdf68fc512d173d1914f7863303cc0a4335439eb76000fa5a6134d5c454f4ef31
kubernetes-client-linux-arm64.tar.gzedbf086c5446a7b48bbf5ac0e65dacf472e7e2eb7ac434ffb4835b0c643363a4
kubernetes-client-linux-arm.tar.gz138b02e0e96e9e30772e814d2650b40594e9f190442c9b31af5dcf4bd3c29fb2
kubernetes-client-linux-ppc64le.tar.gz8edb568048f64052e9ab3e2f0d9d9fee3a5c90667d00669d815c07cc1986eb03
kubernetes-client-linux-s390x.tar.gz9f0f0464041e85221cb65ab5908f7295d7237acdb6a39abff062e40be0a53b4c
kubernetes-client-windows-386.tar.gza9d4b6014c2856b0602b7124dad41f2f932cccea7f48ba57583352f0fbf2710f
kubernetes-client-windows-amd64.tar.gz16827a05b0538ab8ef6e47b173dc5ad1c4398070324b0d2fc0510ad1efe66567

Server Binaries

filenamesha256 hash
kubernetes-server-linux-amd64.tar.gze2aad29fff3cc3a98c642d8bc021a6caa42b4143696ca9d42a1ae3f7e803e777
kubernetes-server-linux-arm64.tar.gza7e2370d29086dadcb59fc4c3f6e88610ef72ff168577cc1854b4e9c221cad8a
kubernetes-server-linux-arm.tar.gzb8da04e06946b221b2ac4f6ebc8e0900cf8e750f0ca5d2e213984644048d1903
kubernetes-server-linux-ppc64le.tar.gz539db8044dcacc154fff92029d7c18ac9a68de426477cabcd52e01053e8de6e6
kubernetes-server-linux-s390x.tar.gzd793be99d39f1f7b55d381f656b059e4cd78418a6c6bcc77c2c026db82e98769

Node Binaries

filenamesha256 hash
kubernetes-node-linux-amd64.tar.gz22dae55dd97026eae31562fde6d8459f1594b050313ef294e009144aa8c27a8e
kubernetes-node-linux-arm64.tar.gz8d9bd9307cd5463b2e13717c862e171e20a1ba29a91d86fa3918a460006c823b
kubernetes-node-linux-arm.tar.gzc696f882b4a95b13c8cf3c2e05695decb81407359911fba169a308165b06be55
kubernetes-node-linux-ppc64le.tar.gz611a0e04b1014263e66be91ef108a4a56291cae1438da562b157d04dfe84fd1a
kubernetes-node-linux-s390x.tar.gz61b619e3af7fcb836072c4b855978d7d76d6256aa99b9378488f063494518a0e
kubernetes-node-windows-amd64.tar.gzc274258e4379f0b50b2023f659bc982a82783c3de3ae07ef2759159300175a8a

Changelog since v1.9.0-alpha.2

Action Required

  • action required: The storage.k8s.io/v1beta1 API and volume.beta.kubernetes.io/storage-class annotation are deprecated. They will be removed in a future release. Please use v1 API and field v1.PersistentVolumeClaim.Spec.StorageClassName/v1.PersistentVolume.Spec.StorageClassName instead. (#53580, @xiangpengzhao)
  • action required: Deprecated flags --portal-net and service-node-ports of kube-apiserver are removed. (#52547, @xiangpengzhao)
  • The node.kubernetes.io/memory-pressure taint now respects the configured whitelist. If you need to use it, you'll have to add it to the whitelist. (#55251, @deads2k)

Other notable changes

  • hyperkube: add cloud-controller-manager (#54197, @colemickens)
  • Metrics have been added for monitoring admission plugins, including the new dynamic (webhook-based) ones. (#55183, @jpbetz)
  • Addon manager supports HA masters. (#55466, @x13n)
    • Add PodSecurityPolicies for cluster addons (#55509, @tallclair)
        • Remove SSL cert HostPath volumes from heapster addons
  • Add iptables rules to allow Pod traffic even when default iptables policy is to reject. (#52569, @tmjd)
  • Validate positive capacity for PVs and PVCs. (#55532, @ianchakeres)
  • Kubelet supports running mount utilities and final mount in a container instead running them on the host. (#53440, @jsafrane)
  • The PodSecurityPolicy annotation kubernetes.io/psp on pods is only set once on create. (#55486, @sttts)
  • The apiserver sends external versioned object to the admission webhooks now. Please update the webhooks to expect admissionReview.spec.object.raw to be serialized external versions of objects. (#55127, @caesarxuchao)
  • RBAC ClusterRoles can now select other roles to aggregate (#54005, @deads2k)
  • GCE nodes with NVIDIA GPUs attached now expose nvidia.com/gpu as a resource instead of alpha.kubernetes.io/nvidia-gpu. (#54826, @mindprince)
  • Remove docker dependency during kubelet start up (#54405, @resouer)
  • Fix session affinity issue with external load balancer traffic when ExternalTrafficPolicy=Local. (#55519, @MrHohn)
  • Add the concurrent service sync flag to the Cloud Controller Manager to allow changing the number of workers. (--concurrent-service-syncs) (#55561, @jhorwit2)
  • move IsMissingVersion comments (#55523, @chenpengdev)
  • The dynamic admission webhook now supports a URL in addition to a service reference, to accommodate out-of-cluster webhooks. (#54889, @lavalamp)
  • Correct wording of kubeadm upgrade response for missing ConfigMap. (#53337, @jmhardison)
  • add create priorityclass sub command (#54858, @wackxu)
  • Added namespaceSelector to externalAdmissionWebhook configuration to allow applying webhooks only to objects in the namespaces that have matching labels. (#54727, @caesarxuchao)
  • Base images bumped to Debian Stretch (9) (#52744, @rphillips)
  • [fluentd-elasticsearch addon] Elasticsearch service name can be overridden via env variable ELASTICSEARCH_SERVICE_NAME (#54215, @mrahbar)
  • Increase waiting time (120s) for docker startup in health-monitor.sh (#54099, @dchen1107)
  • not calculate new priority when user update other spec of a pod (#55221, @CaoShuFeng)
  • kubectl create pdb will no longer set the min-available field by default. (#53047, @yuexiao-wang)
  • StatefulSet status now has support for conditions, making it consistent with other core controllers in v1 (#55268, @foxish)
  • kubeadm: use the CRI for preflights checks (#55055, @runcom)
  • kubeadm now produces error during preflight checks if swap is enabled. Users, who can setup kubelet to run in unsupported environment with enabled swap, will be able to skip that preflight check. (#55399, @kad)
    • kubeadm will produce error if kubelet too new for control plane (#54868, @kad)
  • validate if default and defaultRequest match when creating LimitRange for GPU and hugepages. (#54919, @tianshapjq)
  • Add extra-args configs to kubernetes-worker charm (#55334, @Cynerva)
  • Restored kube-proxy's support for 0 values for conntrack min, max, max per core, tcp close wait timeout, and tcp established timeout. (#55261, @ncdc)
  • Audit policy files without apiVersion and kind are treated as invalid. (#54267, @ericchiang)
  • ReplicationController now shares its underlying controller implementation with ReplicaSet to reduce the maintenance burden going forward. However, they are still separate resources and there should be no externally visible effects from this change. (#49429, @enisoc)
  • Add limitrange/resourcequota/downward_api e2e tests for local ephemeral storage (#52523, @NickrenREN)
  • Support copying "options" in resolv.conf into pod sandbox when dnsPolicy is Default (#54773, @phsiao)
  • Fix support for configmap resource lock type in CCM (#55125, @jhorwit2)
  • The minimum supported go version bumps to 1.9.1. (#55301, @xiangpengzhao)
  • GCE: provide an option to disable docker's live-restore on COS/ubuntu (#55260, @yujuhong)
  • Azure NSG rules for services exposed via external load balancer (#54177, @itowlson)
    • now limit the destination IP address to the relevant front end load
    • balancer IP.
  • DaemonSet status now has a new field named "conditions", making it consistent with other workloads controllers. (#55272, @janetkuo)
  • kubeadm: Add an experimental mode to deploy CoreDNS instead of KubeDNS (#52501, @rajansandeep)
  • Allow HPA to read custom metrics. (#54854, @kawych)
  • Fixed 'Schedulercache is corrupted' error in kube-scheduler (#55262, @liggitt)
  • API discovery failures no longer crash the kube controller manager via the garbage collector. (#55259, @ironcladlou)
  • The kube-scheduler command now supports a --config flag which is the location of a file containing a serialized scheduler configuration. Most other kube-scheduler flags are now deprecated. (#52562, @ironcladlou)
  • add field selector for kubectl get (#50140, @dixudx)
  • Removes Priority Admission Controller from kubeadm since it's alpha. (#55237, @andrewsykim)
  • Add support for the webhook authorizer to make a Deny decision that short-circuits the union authorizer and immediately returns Deny. (#53273, @mikedanese)
  • kubeadm init: fix a bug that prevented the --token-ttl flag and tokenTTL configuration value from working as expected for infinite (0) values. (#54640, @mattmoyer)
  • Add CRI log parsing library at pkg/kubelet/apis/cri/logs (#55140, @feiskyer)
  • Add extra-args configs for scheduler and controller-manager to kubernetes-master charm (#55185, @Cynerva)
  • Add masquerading rules by default to GCE/GKE (#55178, @dnardo)
  • Upgraded Azure SDK to v11.1.1. (#54971, @itowlson)
  • Disable the termination grace period for the calico/node add-on DaemonSet to reduce downtime during a rolling upgrade or deletion. (#55015, @fasaxc)
  • Google KMS integration was removed from in-tree in favor of a out-of-process extension point that will be used for all KMS providers. (#54759, @sakshamsharma)
  • kubeadm: reset: use crictl to reset containers (#54721, @runcom)
  • Check for available volume before attach/delete operation in EBS (#55008, @gnufied)
  • DaemonSet, Deployment, ReplicaSet, and StatefulSet have been promoted to GA and are available in the apps/v1 group version. (#53679, @kow3ns)
  • In conversion-gen removed Kubernetes core API from default extra-peer-dirs. (#54394, @sttts)
  • Fix IPVS availability check (#51874, @vfreex)
  • ScaleIO driver completely removes dependency on drv_cfg binary so a Kubernetes cluster can easily run a containerized kubelet. (#54956, @vladimirvivien)
  • Avoid unnecessary spam in kube-controller-manager log if --cluster-cidr is not specified and --allocate-node-cidrs is false. (#54934, @akosiaris)
  • It is now possible to set multiple manifest url headers via the Kubelet's --manifest-url-header flag. Multiple headers for the same key will be added in the order provided. The ManifestURLHeader field in KubeletConfiguration object (kubeletconfig/v1alpha1) is now a map[string][]string, which facilitates writing JSON and YAML files. (#54643, @mtaufen)
  • Add support for PodSecurityPolicy on GCE: ENABLE_POD_SECURITY_POLICY=true enables the admission controller, and installs policies for default addons. (#52367, @tallclair)
  • In PodTolerationRestriction admisson plugin, if namespace level tolerations are empty, now they override cluster level tolerations. (#54812, @aveshagarwal)
    • Fix handling of IPv6 URLs in NO_PROXY. (#53898, @kad)
  • Added extra_sans config option to kubeapi-load-balancer charm. This allows the user to specify extra SAN entries on the certificate generated for the load balancer. (#54947, @hyperbolic2346)
  • set leveled logging (v=4) for 'updating container' message (#54865, @phsiao)
  • Fix a bug where pod address is not removed from endpoints object while pod is in graceful termination. (#54828, @freehan)
  • kubeadm: Add support for adding a Windows node (#53553, @bsteciuk)

v1.9.0-alpha.2

Documentation & Examples

Downloads for v1.9.0-alpha.2

filenamesha256 hash
kubernetes.tar.gz9d548271e8475171114b3b68323ab3c0e024cf54e25debe4702ffafe3f1d0952
kubernetes-src.tar.gz99901fa7f996ddf75ecab7fcd1d33a3faca38e9d1398daa2ae30c9b3ac6a71ce

Client Binaries

filenamesha256 hash
kubernetes-client-darwin-386.tar.gz5a5e1ce20db98d7f7f0c88957440ab6c7d4b4a4dfefcb31dcd1d6546e9db01d6
kubernetes-client-darwin-amd64.tar.gz094481f8f650321f39ba79cd6348de5052db2bb3820f55a74cf5ce33d5c98701
kubernetes-client-linux-386.tar.gz9a7d8e682a35772ba24bd3fa7a06fb153067b9387daa4db285e15dda75de757d
kubernetes-client-linux-amd64.tar.gz3bb742ffed1a6a51cac01c16614873fea2864c2a4432057a15db90a9d7e40aed
kubernetes-client-linux-arm64.tar.gz928936f06161e8a6f40196381d3e0dc215ca7e7dbc5f7fe6ebccd8d8268b8177
kubernetes-client-linux-arm.tar.gz0a0fa24107f490db0ad57f33638b1aa9ba2baccb5f250caa75405d6612a3e10a
kubernetes-client-linux-ppc64le.tar.gza92f790d1a480318ea206d84d24d2c1d7e43c3683e60f22e7735b63ee73ccbb4
kubernetes-client-linux-s390x.tar.gz1bfb7f056ad91fcbc50657fb9760310a0920c15a5770eaa74cf1a17b1725a232
kubernetes-client-windows-386.tar.gzd1b0abbc9cd0376fa0d56096e42094db8a40485082b301723d05c8e78d8f4717
kubernetes-client-windows-amd64.tar.gz69799ea8741caadac8949a120a455e08aba4d2babba6b63fba2ee9aaeb10c84b

Server Binaries

filenamesha256 hash
kubernetes-server-linux-amd64.tar.gzf3d9f67e94176aa65cffcc6557a7a251ec2384a3f89a81d3daedd8f8dd4c51a7
kubernetes-server-linux-arm64.tar.gz3747b7e26d8bfba59c53b3f20d547e7e90cbb9356e513183ac27f901d7317630
kubernetes-server-linux-arm.tar.gz397b7a49adf90735ceea54720dbf012c8566b34dadde911599bceefb507bc29a
kubernetes-server-linux-ppc64le.tar.gz56f76ebb0788c4e23fc3ede36b52eb34b50b456bb5ff0cf7d78c383c04837565
kubernetes-server-linux-s390x.tar.gz83d961657a50513db82bf421854c567206ccd34240eb8a017167cb98bdb6d38f

Node Binaries

filenamesha256 hash
kubernetes-node-linux-amd64.tar.gz1bb0f5ac920e27b4e51260a80fbfaa013ed7d446d58cd1f9d5f73af4d9517edf
kubernetes-node-linux-arm64.tar.gz47635b9097fc6e3d9b1f1f2c3bd1558d144b1a26d1bf03cfc2e97a3c6db4c439
kubernetes-node-linux-arm.tar.gz212117f1d027c79d50e7c7388951da40b440943748691ba82a3f9f6af75b3ed0
kubernetes-node-linux-ppc64le.tar.gzf2b1d086d07bf2f807dbf02e1f0cd7f6528e57c55be9dadfcecde73e73980068
kubernetes-node-linux-s390x.tar.gzba6803a5c065b06cf43d1db674319008f15d4bc45900299d0b90105002af245e
kubernetes-node-windows-amd64.tar.gz6d928e3bdba87db3b9198e02f84696f7345b4b78d07ff4ea048d47691c67b212

Changelog since v1.8.0

Action Required

  • PodSecurityPolicy: Fixes a compatibility issue that caused policies that previously allowed privileged pods to start forbidding them, due to an incorrect default value for allowPrivilegeEscalation. PodSecurityPolicy objects defined using a 1.8.0 client or server that intended to set allowPrivilegeEscalation to false must be reapplied after upgrading to 1.8.1. (#53443, @liggitt)
  • RBAC objects are now stored in etcd in v1 format. After completing an upgrade to 1.9, RBAC objects (Roles, RoleBindings, ClusterRoles, ClusterRoleBindings) should be migrated to ensure all persisted objects are written in v1 format, prior to v1alpha1 support being removed in a future release. (#52950, @liggitt)

Other notable changes

  • Log error of failed healthz check (#53048, @mrIncompetent)
  • fix azure file mount limit issue on windows due to using drive letter (#53629, @andyzhangx)
  • Update AWS SDK to 1.12.7 (#53561, @justinsb)
  • The kubernetes.io/created-by annotation is no longer added to controller-created objects. Use the metadata.ownerReferences item that has controller set to true to determine which controller, if any, owns an object. (#54445, @crimsonfaith91)
  • Fix overlay2 container disk metrics for Docker and CRI-O (#54827, @dashpole)
  • Fix iptables FORWARD policy for Docker 1.13 in kubernetes-worker charm (#54796, @Cynerva)
  • Fix kubeadm upgrade plan for offline operation: ignore errors when trying to fetch latest versions from dl.k8s.io (#54016, @praseodym)
  • fluentd now supports CRI log format. (#54777, @Random-Liu)
  • Validate that PersistentVolumeSource is not changed during PV Update (#54761, @ianchakeres)
  • If you are using the cloud provider API to determine the external host address of the apiserver, set --external-hostname explicitly instead. The cloud provider detection has been deprecated and will be removed in the future (#54516, @dims)
  • Fixes discovery information for scale subresources in the apps API group (#54683, @liggitt)
  • Optimize Repeated registration of AlgorithmProvider when ApplyFeatureGates (#54047, @kuramal)
  • kubectl get will by default fetch large lists of resources in chunks of up to 500 items rather than requesting all resources up front from the server. This reduces the perceived latency of managing large clusters since the server returns the first set of results to the client much more quickly. A new flag --chunk-size=SIZE may be used to alter the number of items or disable this feature when 0 is passed. This is a beta feature. (#53768, @smarterclayton)
  • Add a new feature gate for enabling an alpha annotation which, if present, excludes the annotated node from being added to a service load balancers. (#54644, @brendandburns)
  • Implement graceful shutdown of the kube-apiserver by waiting for open connections to finish before exiting. Moreover, the audit backend will stop dropping events on shutdown. (#53695, @hzxuzhonghu)
  • fix warning messages due to GetMountRefs func not implemented in windows (#52401, @andyzhangx)
  • Object count quotas supported on all standard resources using count/<resource>.<group> syntax (#54320, @derekwaynecarr)
  • Add openssh-client back into the hyperkube image. This allows the gitRepo volume plugin to work properly. (#54250, @ixdy)
  • Bump version of all prometheus-to-sd images to v0.2.2. (#54635, @loburm)
  • [fluentd-gcp addon] Fluentd now runs in its own network, not in the host one. (#54395, @crassirostris)
  • fix azure storage account num exhausting issue (#54459, @andyzhangx)
  • Add Windows support to the system verification check (#53730, @bsteciuk)
  • allow windows mount path (#51240, @andyzhangx)
  • Development of Kubernetes Federation has moved to github.com/kubernetes/federation. This move out of tree also means that Federation will begin releasing separately from Kubernetes. The impact of this is Federation-specific behavior will no longer be included in kubectl, kubefed will no longer be released as part of Kubernetes, and the Federation servers will no longer be included in the hyperkube binary and image. (#53816, @marun)
  • Metadata concealment on GCE is now controlled by the ENABLE_METADATA_CONCEALMENT env var. See cluster/gce/config-default.sh for more info. (#54150, @ihmccreery)
  • Fixed a bug which is causes kube-apiserver to not run without specifying service-cluster-ip-range (#52870, @jennybuckley)
  • the generic admission webhook is now available in the generic apiserver (#54513, @deads2k)
  • ScaleIO persistent volumes now support referencing a secret in a namespace other than the bound persistent volume claim's namespace; this is controlled during provisioning with the secretNamespace storage class parameter; StoragePool and ProtectionDomain attributes no longer defaults to the value default (#54013, @vladimirvivien)
  • Feature gates now check minimum versions (#54539, @jamiehannaford)
  • fix azure pv crash due to volumeSource.ReadOnly value nil (#54607, @andyzhangx)
  • Fix an issue where pods were briefly transitioned to a "Pending" state during the deletion process. (#54593, @dashpole)
  • move getMaxVols function to predicates.go and add some NewVolumeCountPredicate funcs (#51783, @jiulongzaitian)
  • Remove the LbaasV1 of OpenStack cloud provider, currently only support LbaasV2. (#52717, @FengyunPan)
  • generic webhook admission now takes a config file which describes how to authenticate to webhook servers (#54414, @deads2k)
  • The NodeController will not support kubelet 1.2. (#48996, @k82cn)
    • fluentd-gcp runs with a dedicated fluentd-gcp service account (#54175, @tallclair)
        • Stop mounting the host certificates into fluentd's prometheus-to-sd container
  • fix azure disk mount failure on coreos and some other distros (#54334, @andyzhangx)
  • Allow GCE users to configure the service account made available on their nodes (#52868, @ihmccreery)
  • Load kernel modules automatically inside a kube-proxy pod (#52003, @vfreex)
  • kube-apiserver: --ssh-user and --ssh-keyfile are now deprecated and will be removed in a future release. Users of SSH tunnel functionality used in Google Container Engine for the Master -> Cluster communication should plan to transition to alternate methods for bridging master and node networks. (#54433, @dims)
  • Fix hyperkube kubelet --experimental-dockershim (#54508, @ivan4th)
  • Fix clustered datastore name to be absolute. (#54438, @pshahzeb)
  • Fix for service controller so that it won't retry on doNotRetry service update failure. (#54184, @MrHohn)
  • Add support for RBAC support to Kubernetes via Juju (#53820, @ktsakalozos)
  • RBD Persistent Volume Sources can now reference User's Secret in namespaces other than the namespace of the bound Persistent Volume Claim (#54302, @sbezverk)
  • Apiserver proxy rewrites URL when service returns absolute path with request's host. (#52556, @roycaihw)
  • Logging cleanups (#54443, @bowei) * Updates kube-dns to use client-go 3 * Updates containers to use alpine as the base image on all platforms * Adds support for IPv6
  • add --raw to kubectl create to POST using the normal transport (#54245, @deads2k)
  • Remove the --network-plugin-dir flag. (#53564, @supereagle)
  • Introduces a polymorphic scale client, allowing HorizontalPodAutoscalers to properly function on scalable resources in any API group. (#53743, @DirectXMan12)
  • Add PodDisruptionBudget to scheduler cache. (#53914, @bsalamat)
    • API machinery's httpstream/spdy calls now support CIDR notation for NO_PROXY (#54413, @kad)
  • Added option lb-provider to OpenStack cloud provider config (#54176, @gonzolino)
  • Allow for configuring etcd hostname in the manifest (#54403, @wojtek-t)
    • kubeadm will warn users if access to IP ranges for Pods or Services will be done via HTTP proxy. (#52792, @kad)
  • Resolves forbidden error when accessing replicasets and daemonsets via the apps API group (#54309, @liggitt)
  • Cluster Autoscaler 1.0.1 (#54298, @mwielgus)
  • secret data containing Docker registry auth objects is now generated using the config.json format (#53916, @juanvallejo)
  • Added support for SAN entries in the master node certificate via juju kubernetes-master config. (#54234, @hyperbolic2346)
  • support imagePullSecrets and imagePullPolicy in kubefed init (#50740, @dixudx)
  • update gRPC to v1.6.0 to pick up data race fix grpc/grpc-go#1316 (#53128, @dixudx)
  • admission webhook registrations without a specific failure policy default to failing closed. (#54162, @deads2k)
  • Device plugin Alpha API no longer supports returning artifacts per device as part of AllocateResponse. (#53031, @vishh)
  • admission webhook registration now allows URL paths (#54145, @deads2k)
  • The Kubelet's --enable-custom-metrics flag is now marked deprecated. (#54154, @mtaufen)
  • Use multi-arch busybox image for e2e (#54034, @dixudx)
  • sample-controller: add example CRD controller (#52753, @munnerz)
  • RBAC PolicyRules now allow resource=*/<subresource> to cover any-resource/<subresource>. For example, */scale covers replicationcontroller/scale. (#53722, @deads2k)
  • Upgrade to go1.9 (#51375, @cblecker)
  • Webhook always retries connection reset error. (#53947, @crassirostris)
  • fix PV Recycle failed on non-amd64 platform (#53958, @dixudx)
  • Verbose option is added to each status function in CRI. Container runtime could return extra information in status response for debugging. (#53965, @Random-Liu)
  • Fixed log fallback termination messages when using docker with journald log driver (#52503, @joelsmith)
  • falls back to parse Docker runtime version as generic if not semver (#54040, @dixudx)
  • kubelet: prevent removal of default labels from Node API objects on startup (#54073, @liggitt)
  • Change scheduler to skip pod with updates only on pod annotations (#54008, @yguo0905)
  • PodSecurityPolicy: when multiple policies allow a submitted pod, priority is given to ones which do not require any fields in the pod spec to be defaulted. If the pod must be defaulted, the first policy (ordered by name) that allows the pod is used. (#52849, @liggitt)
  • Control HPA tolerance through the horizontal-pod-autoscaler-tolerance flag. (#52275, @mattjmcnaughton)
  • bump CNI to v0.6.0 (#51250, @dixudx)
  • Improve resilience by annotating kube-dns addon with podAntiAffinity to prefer scheduling on different nodes. (#52193, @StevenACoffman)
  • Azure cloudprovider: Fix controller manager crash issue on a manually created k8s cluster. (#53694, @andyzhangx)
  • Enable Priority admission control in kubeadm. (#53175, @andrewsykim)
  • Add --no-negcache flag to kube-dns to prevent caching of NXDOMAIN responses. (#53604, @cblecker)
  • kubelet provides more specific events when unable to sync pod (#53857, @derekwaynecarr)
  • Kubelet evictions take pod priority into account (#53542, @dashpole)
  • Adds a new controller which automatically cleans up Certificate Signing Requests that are (#51840, @jcbsmpsn)
    • Approved and Issued, or Denied.
  • Optimize random string generator to avoid multiple locks & use bit-masking (#53720, @shyamjvs)
  • update cluster printer to enable --show-labels (#53771, @dixudx)
  • add RequestReceivedTimestamp and StageTimestamp to audit event (#52981, @CaoShuFeng)
  • Deprecation: The flag etcd-quorum-read of kube-apiserver is deprecated and the ability to switch off quorum read will be removed in a future release. (#53795, @xiangpengzhao)
  • Use separate client for leader election in scheduler to avoid starving leader election by regular scheduler operations. (#53793, @wojtek-t)
  • Support autoprobing node-security-group for openstack cloud provider, Support multiple Security Groups for cluster's nodes. (#50836, @FengyunPan)
  • fix a bug where disk pressure could trigger prematurely when using overlay2 (#53684, @dashpole)
  • "kubectl cp" updated to honor destination names (#51215, @juanvallejo)
  • kubeadm: Strip bootstrap tokens from the kubeadm-config ConfigMap (#53559, @fabriziopandini)
  • Skip podpreset test if the alpha feature settings/v1alpha1 is disabled (#53080, @jennybuckley)
  • Log when node is successfully initialized by Cloud Controller Manager (#53517, @andrewsykim)
  • apiserver: --etcd-quorum-read now defaults to true, to ensure correct operation with HA etcd clusters (#53717, @liggitt)
  • The Kubelet's feature gates are now specified as a map when provided via a JSON or YAML KubeletConfiguration, rather than as a string of key-value pairs. (#53025, @mtaufen)
  • Address a bug which allowed the horizontal pod autoscaler to allocate desiredReplicas > maxReplicas in certain instances. (#53690, @mattjmcnaughton)
  • Horizontal pod autoscaler uses REST clients through the kube-aggregator instead of the legacy client through the API server proxy. (#53205, @kawych)
  • Fix to prevent downward api change break on older versions (#53673, @timothysc)
  • API chunking via the limit and continue request parameters is promoted to beta in this release. Client libraries using the Informer or ListWatch types will automatically opt in to chunking. (#52949, @smarterclayton)
  • GCE: Bump GLBC version to 0.9.7. (#53625, @nikhiljindal)
  • kubelet's --cloud-provider flag no longer defaults to "auto-detect". If you want cloud-provider support in kubelet, you must set a specific cloud-provider explicitly. (#53573, @dims)
  • Ignore extended resources that are not registered with kubelet during container resource allocation. (#53547, @jiayingz)
  • kubectl top pod and node should sort by namespace / name so that results don't jump around. (#53560, @dixudx)
  • Added --dry-run option to kubectl drain (#52440, @juanvallejo)
  • Fix a bug that prevents client-go metrics from being registered in prometheus in multiple components. (#53434, @crassirostris)
  • Adjust batching audit webhook default parameters: increase queue size, batch size, and initial backoff. Add throttling to the batching audit webhook. Default rate limit is 10 QPS. (#53417, @crassirostris)
  • Added integration test for TaintNodeByCondition. (#53184, @k82cn)
  • Add API version apps/v1, and bump DaemonSet to apps/v1 (#53278, @janetkuo)
  • Change kubeadm create token to default to the group that almost everyone will want to use. The group is system:bootstrappers:kubeadm:default-node-token and is the group that kubeadm sets up, via an RBAC binding, for auto-approval (system:certificates.k8s.io:certificatesigningrequests:nodeclient). (#53512, @jbeda)
  • Using OpenStack service catalog to do version detection (#53115, @FengyunPan)
  • Fix metrics API group name in audit configuration (#53493, @piosz)
  • GCE: Fixes ILB sync on legacy networks and auto networks with unique subnet names (#53410, @nicksardo)
  • outputs <none> for columns specified by -o custom-columns but not found in object (#51750, @jianhuiz)
  • Metrics were added to network plugin to report latency of CNI operations (#53446, @sjenning)
  • GCE: Fix issue deleting internal load balancers when the firewall resource may not exist. (#53450, @nicksardo)
  • Custom resources served through CustomResourceDefinition now support field selectors for metadata.name and metadata.namespace. (#53345, @ncdc)
  • Add generate-groups.sh and generate-internal-groups.sh to k8s.io/code-generator to easily run generators against CRD or User API Server types. (#52186, @sttts)
  • kubelet --cert-dir now defaults to /var/lib/kubelet/pki, in order to ensure bootstrapped and rotated certificates persist beyond a reboot. resolves an issue in kubeadm with false-positive /var/lib/kubelet is not empty message during pre-flight checks (#53317, @liggitt)
  • Fix multi-attach error spam in logs and events (#53401, @gnufied)
  • Use not-ready to replace notReady in node condition taint keys. (#51266, @resouer)
  • Support completion for --clusterrole of kubectl create clusterrolebinding (#48267, @superbrothers)
  • Don't remove extended resource capacities that are not registered with kubelet from node status. (#53353, @jiayingz)
  • Kubectl: Remove swagger 1.2 validation. Also removes options --use-openapi and --schema-cache-dir as these are no longer needed. (#53232, @apelisse)
  • kubectl explain now uses openapi rather than swagger 1.2. (#53228, @apelisse)
  • Fixes a performance issue (#51899) identified in large-scale clusters when deleting thousands of pods simultaneously across hundreds of nodes, by actively removing containers of deleted pods, rather than waiting for periodic garbage collection and batching resulting pod API deletion requests. (#53233, @dashpole)
  • Improve explanation of ReplicaSet (#53403, @rcorre)
  • avoid newline " " in the error to break log msg to 2 lines (#49826, @dixudx)
  • don't recreate a mirror pod for static pod when node gets deleted (#48339, @dixudx)
  • Fix permissions for Metrics Server. (#53330, @kawych)
  • default fail-swap-on to false for kubelet on kubernetes-worker charm (#53386, @wwwtyro)
  • Add --etcd-compaction-interval to apiserver for controlling request of compaction to etcd3 from apiserver. (#51765, @mitake)
  • Apply algorithm in scheduler by feature gates. (#52723, @k82cn)
  • etcd: update version to 3.1.10 (#49393, @hongchaodeng)
  • support nodeSelector in kubefed init (#50749, @dixudx)
  • Upgrade fluentd-elasticsearch addon to Elasticsearch/Kibana 5.6.2 (#53307, @aknuds1)
  • enable to specific unconfined AppArmor profile (#52395, @dixudx)
  • Update Influxdb image to latest version. (#53319, @kairen)
    • Update Grafana image to latest version.
    • Change influxdb-grafana-controller resource to Deployment.
  • Only do UpdateContainerResources when cpuset is set (#53122, @resouer)
  • Fixes an issue with RBAC reconciliation that could cause duplicated subjects in some bootstrapped rolebindings on each restart of the API server. (#53239, @enj)
  • gce: remove compute-rw, see what breaks (#53266, @mikedanese)
  • Fix the bug that query Kubelet's stats summary with CRI stats enabled results in error. (#53107, @Random-Liu)
  • kubeadm allows the kubelets in the cluster to automatically renew their client certificates (#53252, @kad)
  • Fixes an issue with kubectl set commands encountering conversion errors for ReplicaSet and DaemonSet objects (#53158, @liggitt)
  • RBAC: The default admin and edit roles now include read/write permissions and the view role includes read permissions on poddisruptionbudget.policy resources. (#52654, @liggitt)
  • Change ImageGCManage to consume ImageFS stats from StatsProvider (#53094, @yguo0905)
  • BugFix: Exited containers are not Garbage Collected by the kubelet while the pod is running (#53167, @dashpole)
    • Improved generation of deb and rpm packages in bazel build (#53163, @kad)
  • Add a label which prevents a node from being added to a cloud load balancer (#53146, @brendandburns)
  • Fixes an issue pulling pod specs referencing unqualified images from docker.io on centos/fedora/rhel (#53161, @dims)
  • Update kube-dns to 1.14.5 (#53153, @bowei)
    • kubeadm init can now deploy exact build from CI area by specifying ID with "ci/" prefix. Example: "ci/v1.9.0-alpha.1.123+01234567889" (#53043, @kad)
        • kubeadm upgrade apply supports all standard ways of specifying version via labels. Examples: stable-1.8, latest-1.8, ci/latest-1.9 and similar.
    • kubeadm 1.9 will detect and fail init or join pre-flight checks if kubelet is lower than 1.8.0-alpha (#52913, @kad)
  • s390x ingress controller support (#52663, @wwwtyro)
  • CRI: Add stdout/stderr fields to Exec and Attach requests. (#52686, @yujuhong)
  • Cluster Autoscaler 1.0.0 (#53005, @mwielgus)
  • Remove the --docker-exec-handler flag. Only native exec handler is supported. (#52287, @yujuhong)
  • The Rackspace cloud provider has been removed after a long deprecation period. It was deprecated because it duplicates a lot of the OpenStack logic and can no longer be maintained. Please use the OpenStack cloud provider instead. (#52855, @NickrenREN)
  • Fixes an initializer bug where update requests which had an empty pending initializers list were erroneously rejected. (#52558, @jennybuckley)
  • BulkVerifyVolumes() implementation for vSphere (#52131, @BaluDontu)
  • added --list option to the kubectl label command (#51971, @juanvallejo)
  • Removing --prom-push-gateway flag from e2e tests (#52485, @nielsole)
  • If a container does not create a file at the terminationMessagePath, no message should be output about being unable to find the file. (#52567, @smarterclayton)
  • Support German cloud for azure disk mount feature (#50673, @clement-buchart)
  • Add s390x to juju kubernetes (#52537, @ktsakalozos)
  • Fix kubernetes charms not restarting services properly after host reboot on LXD (#52445, @Cynerva)
  • Add monitoring of Windows Server containers metrics in the kubelet via the stats/summary endpoint. (#50396, @bobbypage)
  • Restores redirect behavior for proxy subresources (#52933, @liggitt)
  • A new service annotation has been added for services of type LoadBalancer on Azure, (#51757, @itowlson)
    • to specify the subnet on which the service's front end IP should be provisioned. The
    • annotation is service.beta.kubernetes.io/azure-load-balancer-internal-subnet and its
    • value is the subnet name (not the subnet ARM ID). If omitted, the default is the
    • master subnet. It is ignored if the service is not on Azure, if the type is not
    • LoadBalancer, or if the load balancer is not internal.
  • Adds a command-line argument to kube-apiserver called (#51698, @rphillips)
    • --alpha-endpoint-reconciler-type=(master-count, lease, none) (default
    • "master-count"). The original reconciler is 'master-count'. The 'lease'
    • reconciler uses the storageapi and a TTL to keep alive an endpoint within the
    • kube-apiserver-endpoint storage namespace. The 'none' reconciler is a noop
    • reconciler that does not do anything. This is useful for self-hosted
    • environments.
  • Improved Italian translation for kubectl (#51463, @lucab85)
  • Add a metric to the kubelet to monitor remaining lifetime of the certificate that (#51031, @jcbsmpsn)
    • authenticates the kubelet to the API server.
  • change AddEventHandlerWithResyncPeriod to AddEventHandler in factory.go (#51582, @jiulongzaitian)
  • Validate that cronjob names are 52 characters or less (#52733, @julia-stripe)
  • add readme file of ipvs (#51937, @Lion-Wei)

v1.9.0-alpha.1

Documentation & Examples

Downloads for v1.9.0-alpha.1

filenamesha256 hash
kubernetes.tar.gze2dc3eebf79368c783b64f5b6642a287cc2fd777547d99f240a35cce1f620ffc
kubernetes-src.tar.gzca8659187047f2d38a7c0ee313189c19ec35646c6ebaa8f59f2f098eca33dca0

Client Binaries

filenamesha256 hash
kubernetes-client-darwin-386.tar.gz51e0df7e6611ff4a9b3759b05e65c80555317bff03282ef39a9b53b27cdeff42
kubernetes-client-darwin-amd64.tar.gzc6c57cc92cc456a644c0965a6aa2bd260125807b450d69376e0edb6c98aaf4d7
kubernetes-client-linux-386.tar.gz399c8cb448d76accb71edcb00bee474f172d416c8c4f5253994e4e2d71e0dece
kubernetes-client-linux-amd64.tar.gzfde75d7267592b34609299a93ee7e54b26a948e6f9a1f64ced666c0aae4455aa
kubernetes-client-linux-arm64.tar.gzb38810cf87735efb0af027b7c77e4e8c8f5821f235cf33ae9eee346e6d1a0b84
kubernetes-client-linux-arm.tar.gza36427c2f2b81d42702a12392070f7dd3635b651bb04ae925d0bdf3ec50f83aa
kubernetes-client-linux-ppc64le.tar.gz9dee0f636eef09bfec557a50e4f8f4b69e0588bbd0b77f6da50cc155e1679880
kubernetes-client-linux-s390x.tar.gz4a6246d5de5c3957ed41b8943fa03e74fb646595346f7c72beaf7b030fe6872e
kubernetes-client-windows-386.tar.gz1ee384f4bb02e614c86bf84cdfdc42faffa659aaba4a1c759ec26f03eb438149
kubernetes-client-windows-amd64.tar.gze70d8935abefea0307780e899238bb10ec27c8f0d77702cf25de230b6abf7fb4

Server Binaries

filenamesha256 hash
kubernetes-server-linux-amd64.tar.gz7fff06370c4f37e1fe789cc160fce0c93535991f63d7fe7d001378f17027d9d8
kubernetes-server-linux-arm64.tar.gz65cd60512ea0bf508aa65f8d22a6f3094db394f00b3cd6bd63fe02b795514ab2
kubernetes-server-linux-arm.tar.gz0ecb341a047f1a9dface197f11f05f15853570cfb474c82538c7d61b40bd53ae
kubernetes-server-linux-ppc64le.tar.gzcea9eed4c24e7f29994ecc12674bff69d108692d3c9be3e8bd939b3c4f281892
kubernetes-server-linux-s390x.tar.gz4d50799e5989de6d9ec316d2051497a3617b635e89fa44e01e64fed544d96e07

Node Binaries

filenamesha256 hash
kubernetes-node-linux-amd64.tar.gze956b9c1e5b47f800953ad0f82fae23774a2f43079dc02d98a90d5bfdca0bad6
kubernetes-node-linux-arm64.tar.gzede6a85db555dd84e8d7180bdd58712933c38567ab6c97a80d0845be2974d968
kubernetes-node-linux-arm.tar.gz4ac6a1784fa1e20be8a4e7fa0ff8b4defc725e6c058ff97068bf7bfa6a11c77d
kubernetes-node-linux-ppc64le.tar.gz0d9c8c7e0892d7b678f3b4b7736087da91cb40c5f169e4302e9f4637c516207a
kubernetes-node-linux-s390x.tar.gz2fdde192a84410c784e5d1e813985e9a19ce62e3d9bb2215481cbce9286329da
kubernetes-node-windows-amd64.tar.gz543110cc69b57471f3824d96cbd16b003ac2cddaa19ca4bdefced0af61fd24f2

Changelog since v1.8.0-alpha.3

Action Required

  • New GCE or GKE clusters created with cluster/kube-up.sh will not enable the legacy ABAC authorizer by default. If you would like to enable the legacy ABAC authorizer, export ENABLE_LEGACY_ABAC=true before running cluster/kube-up.sh. (#51367, @cjcullen)
  • The OwnerReferencesPermissionEnforcement admission plugin now requires update permission on the finalizers subresource of the referenced owner in order to set blockOwnerDeletion on an owner reference. (#49133, @deads2k)
  • The deprecated alpha and beta initContainer annotations are no longer supported. Init containers must be specified using the initContainers field in the pod spec. (#51816, @liggitt)
  • Action required: validation rule on metadata.initializers.pending[x].name is tightened. The initializer name needs to contain at least three segments separated by dots. If you create objects with pending initializers, (i.e., not relying on apiserver adding pending initializers according to initializerconfiguration), you need to update the initializer name in existing objects and in configuration files to comply to the new validation rule. (#51283, @caesarxuchao)
  • Audit policy supports matching subresources and resource names, but the top level resource no longer matches the subresouce. For example "pods" no longer matches requests to the logs subresource of pods. Use "pods/logs" to match subresources. (#48836, @ericchiang)
  • Protobuf serialization does not distinguish between [] and null. (#45294, @liggitt)
    • API fields previously capable of storing and returning either [] and null via JSON API requests (for example, the Endpoints subsets field) can now store only null when created using the protobuf content-type or stored in etcd using protobuf serialization (the default in 1.6+). JSON API clients should tolerate null values for such fields, and treat null and [] as equivalent in meaning unless specifically documented otherwise for a particular field.

Other notable changes

  • PersistentVolumeLabel admission controller is now deprecated. (#52618, @dims)
  • Mark the LBaaS v1 of OpenStack cloud provider deprecated. (#52821, @FengyunPan)
  • Mark image as deliberately optional in v1 Container struct. Many objects in the Kubernetes API inherit the container struct and only Pods require the field to be set. (#48406, @gyliu513)
  • [fluentd-gcp addon] Update Stackdriver plugin to version 0.6.7 (#52565, @crassirostris)
  • Remove duplicate proto errors in kubelet. (#52132, @adityadani)
  • [fluentd-gcp addon] Remove audit logs from the fluentd configuration (#52777, @crassirostris)
  • Set defaults for successfulJobsHistoryLimit (3) and failedJobsHistoryLimit (1) in batch/v1beta1.CronJobs (#52533, @soltysh)
  • Fix: update system spec to support Docker 17.03 (#52666, @yguo0905)
  • Fix panic in ControllerManager on GCE when it has a problem with creating external loadbalancer healthcheck (#52646, @gmarek)
  • PSP: add support for using * as a value in allowedCapabilities to allow to request any capabilities (#51337, @php-coder)
  • [fluentd-gcp addon] By default ingest apiserver audit logs written to file in JSON format. (#52541, @crassirostris)
  • The autoscaling/v2beta1 API group is now enabled by default. (#52549, @DirectXMan12)
  • Add CLUSTER_SIGNING_DURATION environment variable to cluster (#52497, @jcbsmpsn)
    • configuration scripts to allow configuration of signing duration of
    • certificates issued via the Certificate Signing Request API.
  • Introduce policy to allow the HPA to consume the metrics.k8s.io and custom.metrics.k8s.io API groups. (#52572, @DirectXMan12)
  • kubelet to master communication when doing node status updates now has a timeout to prevent indefinite hangs (#52176, @liggitt)
  • Introduced Metrics Server in version v0.2.0. For more details see https://github.com/kubernetes-incubator/metrics-server/releases/tag/v0.2.0. (#52548, @piosz)
  • Adds ROTATE_CERTIFICATES environment variable to kube-up.sh script for GCE (#52115, @jcbsmpsn)
    • clusters. When that var is set to true, the command line flag enabling kubelet
    • client certificate rotation will be added to the kubelet command line.
  • Make sure that resources being updated are handled correctly by Quota system (#52452, @gnufied)
  • WATCHLIST calls are now reported as WATCH verbs in prometheus for the apiserver_request_* series. A new "scope" label is added to all apiserver_request_* values that is either 'cluster', 'resource', or 'namespace' depending on which level the query is performed at. (#52237, @smarterclayton)
  • Fixed the webhook admission plugin so that it works even if the apiserver and the nodes are in two networks (e.g., in GKE). (#50476, @caesarxuchao)
    • Fixed the webhook admission plugin so that webhook author could use the DNS name of the service as the CommonName when generating the server cert for the webhook.
    • Action required:
    • Anyone who generated server cert for admission webhooks need to regenerate the cert. Previously, when generating server cert for the admission webhook, the CN value doesn't matter. Now you must set it to the DNS name of the webhook service, i.e., <service.Name>.<service.Namespace>.svc.
  • Ignore pods marked for deletion that exceed their grace period in ResourceQuota (#46542, @derekwaynecarr)
  • custom resources that use unconventional pluralization now work properly with kubectl and garbage collection (#50012, @deads2k)
  • [fluentd-gcp addon] Fluentd will trim lines exceeding 100KB instead of dropping them. (#52289, @crassirostris)
  • dockershim: check the error when syncing the checkpoint. (#52125, @yujuhong)
  • By default, clusters on GCE no longer send RequestReceived audit event, if advanced audit is configured. (#52343, @crassirostris)
  • [BugFix] Soft Eviction timer works correctly (#52046, @dashpole)
  • Azuredisk mount on windows node (#51252, @andyzhangx)
  • [fluentd-gcp addon] Bug with event-exporter leaking memory on metrics in clusters with CA is fixed. (#52263, @crassirostris)
  • kubeadm: Enable kubelet client certificate rotation (#52196, @luxas)
  • Scheduler predicate developer should respect equivalence class cache (#52146, @resouer)
  • The kube-cloud-controller-manager flag --service-account-private-key-file was non-functional and is now deprecated. (#50289, @liggitt)
    • The kube-cloud-controller-manager flag --use-service-account-credentials is now honored consistently, regardless of whether --service-account-private-key-file was specified.
  • Fix credentials providers for docker sandbox image. (#51870, @feiskyer)
  • Fixed an issue looking up cronjobs when they existed in more than one API version (#52227, @liggitt)
  • Add priority-based preemption to the scheduler. (#50949, @bsalamat)
  • Add CLUSTER_SIGNING_DURATION environment variable to cluster configuration scripts (#51844, @jcbsmpsn)
    • to allow configuration of signing duration of certificates issued via the Certificate
    • Signing Request API.
  • Adding German translation for kubectl (#51867, @Steffen911)
  • The ScaleIO volume plugin can now read the SDC GUID value as node label scaleio.sdcGuid; if binary drv_cfg is not installed, the plugin will still work properly; if node label not found, it defaults to drv_cfg if installed. (#50780, @vladimirvivien)
  • A policy with 0 rules should return an error (#51782, @charrywanganthony)
  • Log a warning when --audit-policy-file not passed to apiserver (#52071, @CaoShuFeng)
  • Fixes an issue with upgrade requests made via pod/service/node proxy subresources sending a non-absolute HTTP request-uri to backends (#52065, @liggitt)
  • kubeadm: add kubeadm phase addons command (#51171, @andrewrynhard)
  • Fix for Nodes in vSphere lacking an InternalIP. (#48760) (#49202, @cbonte)
  • v2 of the autoscaling API group, including improvements to the HorizontalPodAutoscaler, has moved from alpha1 to beta1. (#50708, @DirectXMan12)
  • Fixed a bug where some alpha features were enabled by default. (#51839, @jennybuckley)
  • Implement StatsProvider interface using CRI stats (#51557, @yguo0905)
  • set AdvancedAuditing feature gate to true by default (#51943, @CaoShuFeng)
  • Migrate the metrics/v1alpha1 API to metrics/v1beta1. The HorizontalPodAutoscaler (#51653, @DirectXMan12)
    • controller REST client now uses that version. For v1beta1, the API is now known as
    • resource-metrics.metrics.k8s.io.
  • In GCE with COS, increase TasksMax for Docker service to raise cap on number of threads/processes used by containers. (#51986, @yujuhong)
  • Fixes an issue with APIService auto-registration affecting rolling HA apiserver restarts that add or remove API groups being served. (#51921, @liggitt)
  • Sharing a PID namespace between containers in a pod is disabled by default in 1.8. To enable for a node, use the --docker-disable-shared-pid=false kubelet flag. Note that PID namespace sharing requires docker >= 1.13.1. (#51634, @verb)
  • Build test targets for all server platforms (#51873, @luxas)
  • Add EgressRule to NetworkPolicy (#51351, @cmluciano)
  • Allow DNS resolution of service name for COS using containerized mounter. It fixed the issue with DNS resolution of NFS and Gluster services. (#51645, @jingxu97)
  • Fix journalctl leak on kubelet restart (#51751, @dashpole)
    • Fix container memory rss
    • Add hugepages monitoring support
    • Fix incorrect CPU usage metrics with 4.7 kernel
    • Add tmpfs monitoring support
  • Support for Huge pages in empty_dir volume plugin (#50072, @squall0gd)
    • Huge pages can now be used with empty dir volume plugin.
  • Alpha support for pre-allocated hugepages (#50859, @derekwaynecarr)
  • add support for client-side spam filtering of events (#47367, @derekwaynecarr)
  • Provide a way to omit Event stages in audit policy (#49280, @CaoShuFeng)
  • Introduced Metrics Server (#51792, @piosz)
  • Implement Controller for growing persistent volumes (#49727, @gnufied)
  • Kubernetes 1.8 supports docker version 1.11.x, 1.12.x and 1.13.x. And also supports overlay2. (#51845, @Random-Liu)
  • The Deployment, DaemonSet, and ReplicaSet kinds in the extensions/v1beta1 group version are now deprecated, as are the Deployment, StatefulSet, and ControllerRevision kinds in apps/v1beta1. As they will not be removed until after a GA version becomes available, you may continue to use these kinds in existing code. However, all new code should be developed against the apps/v1beta2 group version. (#51828, @kow3ns)
  • kubeadm: Detect kubelet readiness and error out if the kubelet is unhealthy (#51369, @luxas)
  • Fix providerID update validation (#51761, @karataliu)
  • Calico has been updated to v2.5, RBAC added, and is now automatically scaled when GCE clusters are resized. (#51237, @gunjan5)
  • Add backoff policy and failed pod limit for a job (#51153, @clamoriniere1A)
  • Adds a new alpha EventRateLimit admission control that is used to limit the number of event queries that are accepted by the API Server. (#50925, @staebler)
  • The OpenID Connect authenticator can now use a custom prefix, or omit the default prefix, for username and groups claims through the --oidc-username-prefix and --oidc-groups-prefix flags. For example, the authenticator can map a user with the username "jane" to "google:jane" by supplying the "google:" username prefix. (#50875, @ericchiang)
  • Implemented kubeadm upgrade plan for checking whether you can upgrade your cluster to a newer version (#48899, @luxas)
    • Implemented kubeadm upgrade apply for upgrading your cluster from one version to an other
  • Switch to audit.k8s.io/v1beta1 in audit. (#51719, @soltysh)
  • update QEMU version to v2.9.1 (#50597, @dixudx)
  • controllers backoff better in face of quota denial (#49142, @joelsmith)
  • The event table output under kubectl describe has been simplified to show only the most essential info. (#51748, @smarterclayton)
  • Use arm32v7|arm64v8 images instead of the deprecated armhf|aarch64 image organizations (#50602, @dixudx)
  • audit newest impersonated user info in the ResponseStarted, ResponseComplete audit stage (#48184, @CaoShuFeng)
  • Fixed bug in AWS provider to handle multiple IPs when using more than 1 network interface per ec2 instance. (#50112, @jlz27)
  • Add flag "--include-uninitialized" to kubectl annotate, apply, edit-last-applied, delete, describe, edit, get, label, set. "--include-uninitialized=true" makes kubectl commands apply to uninitialized objects, which by default are ignored if the names of the objects are not provided. "--all" also makes kubectl commands apply to uninitialized objects. Please see the initializer doc for more details. (#50497, @dixudx)
  • GCE: Service object now supports "Network Tiers" as an Alpha feature via annotations. (#51301, @yujuhong)
  • When using kube-up.sh on GCE, user could set env ENABLE_POD_PRIORITY=true to enable pod priority feature gate. (#51069, @MrHohn)
  • The names generated for ControllerRevision and ReplicaSet are consistent with the GenerateName functionality of the API Server and will not contain "bad words". (#51538, @kow3ns)
  • PersistentVolumeClaim metrics like "volume_stats_inodes" and "volume_stats_capacity_bytes" are now reported via kubelet prometheus (#51553, @wongma7)
  • When using IP aliases, use a secondary range rather than subnetwork to reserve cluster IPs. (#51690, @bowei)
  • IPAM controller unifies handling of node pod CIDR range allocation. (#51374, @bowei)
    • It is intended to supersede the logic that is currently in range_allocator
    • and cloud_cidr_allocator. (ALPHA FEATURE)
    • Note: for this change, the other allocators still exist and are the default.
    • It supports two modes:
      • CIDR range allocations done within the cluster that are then propagated out to the cloud provider.
      • Cloud provider managed IPAM that is then reflected into the cluster.
  • The Kubernetes API server now supports the ability to break large LIST calls into multiple smaller chunks. A client can specify a limit to the number of results to return, and if more results exist a token will be returned that allows the client to continue the previous list call repeatedly until all results are retrieved. The resulting list is identical to a list call that does not perform chunking thanks to capabilities provided by etcd3. This allows the server to use less memory and CPU responding with very large lists. This feature is gated as APIListChunking and is not enabled by default. The 1.9 release will begin using this by default from all informers. (#48921, @smarterclayton)
  • add reconcile command to kubectl auth (#51636, @deads2k)
  • Advanced audit allows logging failed login attempts (#51119, @soltysh)
  • kubeadm: Add support for using an external CA whose key is never stored in the cluster (#50832, @nckturner)
  • the custom metrics API (custom.metrics.k8s.io) has moved from v1alpha1 to v1beta1 (#50920, @DirectXMan12)
  • Add backoff policy and failed pod limit for a job (#48075, @clamoriniere1A)
  • Performs validation (when applying for example) against OpenAPI schema rather than Swagger 1.0. (#51364, @apelisse)
  • Make all e2e tests lookup image to use from a centralized place. In that centralized place, add support for multiple platforms. (#49457, @mkumatag)
  • kubelet has alpha support for mount propagation. It is disabled by default and it is there for testing only. This feature may be redesigned or even removed in a future release. (#46444, @jsafrane)
  • Add selfsubjectrulesreview API for allowing users to query which permissions they have in a given namespace. (#48051, @xilabao)
  • Kubelet re-binds /var/lib/kubelet directory with rshared mount propagation during startup if it is not shared yet. (#45724, @jsafrane)
  • Deviceplugin jiayingz (#51209, @jiayingz)
  • Make logdump support kubemark and support gke with 'use_custom_instance_list' (#51834, @shyamjvs)
  • add apps/v1beta2 conversion test (#49645, @dixudx)
  • Fixed an issue (#47800) where kubectl logs -f failed with unexpected stream type "". (#50381, @sczizzo)
  • GCE: Internal load balancer IPs are now reserved during service sync to prevent losing the address to another service. (#51055, @nicksardo)
  • Switch JSON marshal/unmarshal to json-iterator library. Performance should be close to previous with no generated code. (#48287, @thockin)
  • Adds optional group and version information to the discovery interface, so that if an endpoint uses non-default values, the proper value of "kind" can be determined. Scale is a common example. (#49971, @deads2k)
  • Fix security holes in GCE metadata proxy. (#51302, @ihmccreery)
  • #43077 introduced a condition where DaemonSet controller did not respect the TerminationGracePeriodSeconds of the Pods it created. This is now corrected. (#51279, @kow3ns)
  • Tainted nodes by conditions as following: (#49257, @k82cn) * 'node.kubernetes.io/network-unavailable=:NoSchedule' if NetworkUnavailable is true * 'node.kubernetes.io/disk-pressure=:NoSchedule' if DiskPressure is true * 'node.kubernetes.io/memory-pressure=:NoSchedule' if MemoryPressure is true * 'node.kubernetes.io/out-of-disk=:NoSchedule' if OutOfDisk is true
  • rbd: default image format to v2 instead of deprecated v1 (#51574, @dillaman)
  • Surface reasonable error when client detects connection closed. (#51381, @mengqiy)
  • Allow PSP's to specify a whitelist of allowed paths for host volume (#50212, @jhorwit2)
  • For Deployment, ReplicaSet, and DaemonSet, selectors are now immutable when updating via the new apps/v1beta2 API. For backward compatibility, selectors can still be changed when updating via apps/v1beta1 or extensions/v1beta1. (#50719, @crimsonfaith91)
  • Allows kubectl to use http caching mechanism for the OpenAPI schema. The cache directory can be configured through --cache-dir command line flag to kubectl. If set to empty string, caching will be disabled. (#50404, @apelisse)
  • Pod log attempts are now reported in apiserver prometheus metrics with verb CONNECT since they can run for very long periods of time. (#50123, @WIZARD-CXY)
  • The emptyDir.sizeLimit field is now correctly omitted from API requests and responses when unset. (#50163, @jingxu97)
  • Promote CronJobs to batch/v1beta1. (#51465, @soltysh)
  • Add local ephemeral storage support to LimitRange (#50757, @NickrenREN)
  • Add mount options field to StorageClass. The options listed there are automatically added to PVs provisioned using the class. (#51228, @wongma7)
  • Add 'kubectl set env' command for setting environment variables inside containers for resources embedding pod templates (#50998, @zjj2wry)
  • Implement IPVS-based in-cluster service load balancing (#46580, @dujun1990)
  • Release the kubelet client certificate rotation as beta. (#51045, @jcbsmpsn)
  • Adds --append-hash flag to kubectl create configmap/secret, which will append a short hash of the configmap/secret contents to the name during creation. (#49961, @mtaufen)
  • Add validation for CustomResources via JSON Schema. (#47263, @nikhita)
  • enqueue a sync task to wake up jobcontroller to check job ActiveDeadlineSeconds in time (#48454, @weiwei04)
  • Remove previous local ephemeral storage resource names: "ResourceStorageOverlay" and "ResourceStorageScratch" (#51425, @NickrenREN)
  • Add retainKeys to patchStrategy for v1 Volumes and extensions/v1beta1 DeploymentStrategy. (#50296, @mengqiy)
  • Add mount options field to PersistentVolume spec (#50919, @wongma7)
  • Use of the alpha initializers feature now requires enabling the Initializers feature gate. This feature gate is auto-enabled if the Initialzers admission plugin is enabled. (#51436, @liggitt)
  • Fix inconsistent Prometheus cAdvisor metrics (#51473, @bboreham)
  • Add local ephemeral storage to downward API (#50435, @NickrenREN)
  • kubectl zsh autocompletion will work with compinit (#50561, @cblecker)
  • [Experiment Only] When using kube-up.sh on GCE, user could set env KUBE_PROXY_DAEMONSET=true to run kube-proxy as a DaemonSet. kube-proxy is run as static pods by default. (#50705, @MrHohn)
  • Add --request-timeout to kube-apiserver to make global request timeout configurable. (#51415, @jpbetz)
  • Deprecate auto detecting cloud providers in kubelet. Auto detecting cloud providers go against the initiative for out-of-tree cloud providers as we'll now depend on cAdvisor integrations with cloud providers instead of the core repo. In the near future, --cloud-provider for kubelet will either be an empty string or external. (#51312, @andrewsykim)
  • Add local ephemeral storage support to Quota (#49610, @NickrenREN)
  • Kubelet updates default labels if those are deprecated (#47044, @mrIncompetent)
  • Add error count and time-taken metrics for storage operations such as mount and attach, per-volume-plugin. (#50036, @wongma7)
  • A new predicates, named 'CheckNodeCondition', was added to replace node condition filter. 'NetworkUnavailable', 'OutOfDisk' and 'NotReady' maybe reported as a reason when failed to schedule pods. (#51117, @k82cn)
  • Add support for configurable groups for bootstrap token authentication. (#50933, @mattmoyer)
  • Fix forbidden message format (#49006, @CaoShuFeng)
  • make volumesInUse sorted in node status updates (#49849, @dixudx)
  • Adds InstanceExists and InstanceExistsByProviderID to cloud provider interface for the cloud controller manager (#51087, @prydie)
  • Dynamic Flexvolume plugin discovery. Flexvolume plugins can now be discovered on the fly rather than only at system initialization time. (#50031, @verult)
  • add fieldSelector spec.schedulerName (#50582, @dixudx)
  • Change eviction manager to manage one single local ephemeral storage resource (#50889, @NickrenREN)
  • Cloud Controller Manager now sets Node.Spec.ProviderID (#50730, @andrewsykim)
  • Parameterize session affinity timeout seconds in service API for Client IP based session affinity. (#49850, @m1093782566)
  • Changing scheduling part of the alpha feature 'LocalStorageCapacityIsolation' to manage one single local ephemeral storage resource (#50819, @NickrenREN)
  • set --audit-log-format default to json (#50971, @CaoShuFeng)
  • kubeadm: Implement a --dry-run mode and flag for kubeadm (#51122, @luxas)
  • kubectl rollout history, status, and undo subcommands now support StatefulSets. (#49674, @crimsonfaith91)
  • Add IPBlock to Network Policy (#50033, @cmluciano)
  • Adding Italian translation for kubectl (#50155, @lucab85)
  • Simplify capabilities handling in FlexVolume. (#51169, @MikaelCluseau)
  • cloudprovider.Zones should support external cloud providers (#50858, @andrewsykim)
  • Finalizers are now honored on custom resources, and on other resources even when garbage collection is disabled via the apiserver flag --enable-garbage-collector=false (#51148, @ironcladlou)
  • Renamed the API server flag --experimental-bootstrap-token-auth to --enable-bootstrap-token-auth. The old value is accepted with a warning in 1.8 and will be removed in 1.9. (#51198, @mattmoyer)
  • Azure file persistent volumes can use a new secretNamespace field to reference a secret in a different namespace than the one containing their bound persistent volume claim. The azure file persistent volume provisioner honors a corresponding secretNamespace storage class parameter to determine where to place secrets containing the storage account key. (#47660, @rootfs)
  • Bumped gRPC version to 1.3.0 (#51154, @RenaudWasTaken)
  • Delete load balancers if the UIDs for services don't match. (#50539, @brendandburns)
  • Show events when describing service accounts (#51035, @mrogers950)
  • implement proposal 34058: hostPath volume type (#46597, @dixudx)
  • HostAlias is now supported for both non-HostNetwork Pods and HostNetwork Pods. (#50646, @rickypai)
  • CRDs support metadata.generation and implement spec/status split (#50764, @nikhita)
  • Allow attach of volumes to multiple nodes for vSphere (#51066, @BaluDontu)

Please see the Releases Page for older releases.

Release notes of older releases can be found in: