{"id":22933,"date":"2024-06-19T00:27:30","date_gmt":"2024-06-18T21:27:30","guid":{"rendered":"https:\/\/kifarunix.com\/?p=22933"},"modified":"2024-07-12T22:28:56","modified_gmt":"2024-07-12T19:28:56","slug":"safely-upgrade-kubeadm-kubernetes-cluster-a-step-by-step-guide","status":"publish","type":"post","link":"https:\/\/kifarunix.com\/safely-upgrade-kubeadm-kubernetes-cluster-a-step-by-step-guide\/","title":{"rendered":"Safely Upgrade Kubeadm Kubernetes Cluster: A Step-by-Step Guide"},"content":{"rendered":"\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1057\" height=\"589\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2024\/06\/upgrade-kubernetes-cluster.png?v=1718745697\" alt=\"Safely Upgrade Kubeadm Kubernetes Cluster: A Step-by-Step Guide\" class=\"wp-image-22948\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2024\/06\/upgrade-kubernetes-cluster.png?v=1718745697 1057w, https:\/\/kifarunix.com\/wp-content\/uploads\/2024\/06\/upgrade-kubernetes-cluster-768x428.png?v=1718745697 768w\" sizes=\"(max-width: 1057px) 100vw, 1057px\" \/><\/figure>\n\n\n\n<p>In this guide, we&#8217;ll walk you through the step-by-step process of how to safely upgrade kubeadm Kubernetes cluster. Upgrading a Kubernetes cluster that was originally deployed using kubeadm requires careful planning and execution to ensure minimal downtime and smooth operation.<\/p>\n\n\n\n<div class=\"wp-block-rank-math-toc-block\" id=\"rank-math-toc\"><h2>Table of Contents<\/h2><nav><ul><li><a href=\"#upgrading-kubeadm-kubernetes-cluster\">Upgrading Kubeadm Kubernetes Cluster<\/a><ul><li><a href=\"#important-cluster-upgrade-considerations\">Important Cluster Upgrade Considerations<\/a><\/li><li><a href=\"#safely-upgrade-kubeadm-kubernetes-cluster\">Safely Upgrade Kubeadm Kubernetes Cluster<\/a><ul><li><a href=\"#what-is-the-order-of-kubernetes-nodes-upgrade\">What is the Order of Kubernetes Nodes Upgrade?<\/a><\/li><li><a href=\"#determine-the-version-of-kubernetes-to-upgrade-to\">Determine the version of Kubernetes to upgrade to<\/a><\/li><li><a href=\"#change-the-kubernetes-package-repository\">Change the Kubernetes Package Repository<\/a><\/li><li><a href=\"#drain-the-nodes-before-upgrade\">Drain the Nodes Before Upgrade<\/a><\/li><li><a href=\"#upgrade-kubernetes-control-plane-nodes\">Upgrade Kubernetes Control Plane Nodes<\/a><\/li><li><a href=\"#upgrade-worker-nodes\">Upgrade Worker Nodes<\/a><\/li><\/ul><\/li><li><a href=\"#verify-cluster-upgrade\">Verify Cluster Upgrade<\/a><\/li><li><a href=\"#conclusion\">Conclusion<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"upgrading-kubeadm-kubernetes-cluster\">Upgrading Kubeadm Kubernetes Cluster<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"important-cluster-upgrade-considerations\">Important Cluster Upgrade Considerations<\/h3>\n\n\n\n<p>Before you even begin to think about the kubeadm Kubernetes cluster upgrade, there are important considerations you need to look into. Some of these are:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Cluster Backup<\/strong>: Backup is Crucial. Murphy is always around the corner and you never know what might go wrong with the upgrade. You need to be able to recover your Kubernetes cluster data in case of any unforeseen circumstances. This is where backups become crucial. You can check how to <a href=\"https:\/\/kifarunix.com\/disaster-recovery-in-kubernetes-etcd-backup-and-restore-with-etcdctl-and-etcdutl\/\" target=\"_blank\" rel=\"noreferrer noopener\">backup and restore etcd in a Kubernetes cluster<\/a>.<\/li>\n\n\n\n<li><strong>Review Kubernetes Release Notes:<\/strong> Always review the <a href=\"https:\/\/kubernetes.io\/releases\/\" target=\"_blank\" rel=\"noreferrer noopener\">official Kubernetes release notes<\/a> for the version you&#8217;re targeting. This can help identify potential upgrade issues or breaking changes specific to that release.<\/li>\n\n\n\n<li><strong>Static Control Plane Required:<\/strong> kubeadm upgrades only work with clusters using a static control plane and etcd pods or external etcd. If your setup differs, you might need to explore alternative upgrade methods.<\/li>\n\n\n\n<li><strong>Disable Swap:<\/strong> Ensure swap is disabled on all cluster nodes. Check how to disable swap on <a href=\"https:\/\/kifarunix.com\/install-and-setup-kubernetes-cluster-on-ubuntu-24-04\/#disable-swap-on-cluster-nodes\" target=\"_blank\" rel=\"noreferrer noopener\">Kubernetes nodes<\/a>.<\/li>\n\n\n\n<li><strong>Upgrade path<\/strong>: It is recommended to upgrade from a minor version to a next higher one (e.g., from <strong>1.29.0<\/strong> to <strong>1.30.0<\/strong>), or from a patch release version to a higher one (e.g., from <strong>1.30.1<\/strong> to <strong>1.30.2<\/strong>).<\/li>\n\n\n\n<li>It is recommended to match kubelet and kubeadm versions. However, in some cases, it is okay to use a kubelet that is three versions older than kubeadm.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"safely-upgrade-kubeadm-kubernetes-cluster\">Safely Upgrade Kubeadm Kubernetes Cluster<\/h3>\n\n\n\n<p>Kubernetes releases regular updates with new features, bug fixes, and security patches. Upgrading your cluster ensures you can leverage these improvements and maintain a secure environment for your applications.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"what-is-the-order-of-kubernetes-nodes-upgrade\">What is the Order of Kubernetes Nodes Upgrade?<\/h4>\n\n\n\n<p>You need to start upgrading the control plane node before upgrading the worker nodes. If you have multiple control plane nodes, you need to start with the primary one, basically the one that is running as etcd cluster leader, first, then proceed with the rest of the control planes and the worker nodes in any order. This is summarized as:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Upgrade primary control plane node<\/li>\n\n\n\n<li>Upgrade other control plane nodes<\/li>\n\n\n\n<li>Upgrade worker nodes<\/li>\n<\/ol>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"determine-the-version-of-kubernetes-to-upgrade-to\">Determine the version of Kubernetes to upgrade to<\/h4>\n\n\n\n<p>To begin with, you need to check your current version of the cluster packages and of course the version you are upgrading to.<\/p>\n\n\n\n<p>Current installed versions of <strong>kubeadm<\/strong>, <strong>kubectl<\/strong>, <strong>kubelet<\/strong>.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubeadm version -o json<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>{\n  \"clientVersion\": {\n   <strong> \"major\": \"1\",\n    \"minor\": \"30\",\n    \"gitVersion\": \"v1.30.1\",<\/strong>\n    \"gitCommit\": \"6911225c3f747e1cd9d109c305436d08b668f086\",\n    \"gitTreeState\": \"clean\",\n    \"buildDate\": \"2024-05-14T10:49:05Z\",\n    \"goVersion\": \"go1.22.2\",\n    \"compiler\": \"gc\",\n    \"platform\": \"linux\/amd64\"\n  }\n}\n<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl version<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>Client Version: <strong>v1.30.1<\/strong>\nKustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3\nServer Version: <strong>v1.30.1<\/strong>\n<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>kubelet --version<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>Kubernetes <strong>v1.30.1<\/strong><\/code><\/pre>\n\n\n\n<p>Cluster nodes;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl get nodes<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>NAME        STATUS   ROLES           AGE    VERSION\nmaster-01   Ready    control-plane   2d1h   v1.30.1\nmaster-02   Ready    control-plane   2d1h   v1.30.1\nmaster-03   Ready    control-plane   2d1h   v1.30.1\nworker-01   Ready    &lt;none&gt;          2d1h   v1.30.1\nworker-02   Ready    &lt;none&gt;          2d1h   v1.30.1\nworker-03   Ready    &lt;none&gt;          2d1h   v1.30.1\n<\/code><\/pre>\n\n\n\n<p>As you can see, we are on the release version <strong>v1.30.1<\/strong> of Kubernetes. As per the <a href=\"https:\/\/kubernetes.io\/releases\/\" target=\"_blank\" rel=\"noreferrer noopener\">release notes<\/a>, the current latest release as of this writing is <strong>v1.30.2<\/strong> which is one patch release version higher than our current running version.<\/p>\n\n\n\n<p>As such, we will be upgrading our cluster to a patch release version.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"change-the-kubernetes-package-repository\">Change the Kubernetes Package Repository<\/h4>\n\n\n\n<p>Chances are you are using the correct Kubernetes package repository and not the legacy ones (<code>apt.kubernetes.io<\/code>&nbsp;and&nbsp;<code>yum.kubernetes.io<\/code>) which have since deprecated. Instead, ensure you are using either of:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>pkgs.k8s.io<\/li>\n\n\n\n<li>pkgs.kubernetes.io<\/li>\n\n\n\n<li>packages.kubernetes.io<\/li>\n<\/ul>\n\n\n\n<p>You can check which repository you are using;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>grep -irlE \"kubernetes|k8s\" \/etc\/apt\/<\/code><\/pre>\n\n\n\n<p>This will print a file containing any of the <strong>kubernetes<\/strong> or <strong>k8s<\/strong> keywords.<\/p>\n\n\n\n<p>Sample output;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>\/etc\/apt\/sources.list.d\/kurbenetes.list\n\/etc\/apt\/trusted.gpg.d\/k8s.gpg<\/code><\/pre>\n\n\n\n<p>Therefore, our K8S package repository file is <strong>\/etc\/apt\/sources.list.d\/kurbenetes.list<\/strong>.<\/p>\n\n\n\n<p>Get the contents of the file to confirm the repository address;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat \/etc\/apt\/sources.list.d\/kurbenetes.list<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>deb https:\/\/pkgs.k8s.io\/core:\/stable:\/v1.30\/deb\/ \/<\/code><\/pre>\n\n\n\n<p>Based on the release version we are intending to upgrade to as seen in determining the version section above, the repository we have is enough to upgrade to the patch release v1.30.2.<\/p>\n\n\n\n<p>Also, if you are using legacy repositories, be sure to update them before you can proceed.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"drain-the-nodes-before-upgrade\">Drain the Nodes Before Upgrade<\/h4>\n\n\n\n<p>Draining the nodes is not really necessary. However, <em>If you are performing a&nbsp;<strong>minor<\/strong>&nbsp;version upgrade for any kubelet, you&nbsp;<strong>must<\/strong>&nbsp;first drain the node(s) that you are upgrading<\/em>.<\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/kubernetes-nodes-maintenance-drain-vs-cordon-demystified\/#kubectl-drain\" target=\"_blank\" rel=\"noreferrer noopener\">How to Drain Kubernetes Nodes<\/a><\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"upgrade-kubernetes-control-plane-nodes\">Upgrade Kubernetes Control Plane Nodes<\/h4>\n\n\n\n<p>As stated, you need to start the upgrade with control plane nodes. If you have multiple control plane nodes, start with the primary node. I would check which one is currently the etcd cluster leader to know the primary control plane.<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>sudo etcdctl endpoint status --cacert=\/etc\/kubernetes\/pki\/etcd\/ca.crt \\\n\t --cert=\/etc\/kubernetes\/pki\/etcd\/server.crt \\\n\t --key=\/etc\/kubernetes\/pki\/etcd\/server.key \\\n\t --cluster \\\n\t -w table\n<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>|          ENDPOINT           |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |\n+-----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+\n| https:\/\/192.168.122.60:2379 | 71adc069c6babb75 |  3.5.12 |   13 MB |     false |      false |         5 |      69810 |              69810 |        |\n| https:\/\/192.168.122.58:2379 | 7f56434149e2cc7f |  3.5.12 |   13 MB |     false |      false |         5 |      69810 |              69810 |        |\n| https:\/\/192.168.122.59:2379 | db4091f30b21595e |  3.5.12 |   13 MB |      true |      false |         5 |      69810 |              69810 |        |\n+-----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+\n<\/code><\/pre>\n\n\n\n<p>As you can see, the node with the IP 192.168.122.59, is the current etcd leader and hence, we will treat this as the primary control plane.<\/p>\n\n\n\n<p>Hence, login to your primary control plane. For example;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ssh kifarunix@192.168.122.59<\/code><\/pre>\n\n\n\n<p>Now that we have confirmed that the package repository is all good, let&#8217;s check the current versions of <strong>kubeadm<\/strong>, <strong>kubelet<\/strong> and <strong>kubectl<\/strong> available.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>apt-cache madison kubeadm kubelet kubectl<\/code><\/pre>\n\n\n\n<p>Sample output;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>   kubeadm | 1.30.1-1.1 | https:\/\/pkgs.k8s.io\/core:\/stable:\/v1.30\/deb  Packages\n   kubeadm | 1.30.0-1.1 | https:\/\/pkgs.k8s.io\/core:\/stable:\/v1.30\/deb  Packages\n   kubelet | 1.30.1-1.1 | https:\/\/pkgs.k8s.io\/core:\/stable:\/v1.30\/deb  Packages\n   kubelet | 1.30.0-1.1 | https:\/\/pkgs.k8s.io\/core:\/stable:\/v1.30\/deb  Packages\n   kubectl | 1.30.1-1.1 | https:\/\/pkgs.k8s.io\/core:\/stable:\/v1.30\/deb  Packages\n   kubectl | 1.30.0-1.1 | https:\/\/pkgs.k8s.io\/core:\/stable:\/v1.30\/deb  Packages\n<\/code><\/pre>\n\n\n\n<p>As you can see, the latest versions provided by the Kubernetes repositories for each of the package is <strong>v1.30.1-1.1<\/strong><\/p>\n\n\n\n<p>To get the latest release versions, run system package cache update.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo apt update<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>Hit:1 http:\/\/de.archive.ubuntu.com\/ubuntu noble InRelease\nGet:2 https:\/\/download.docker.com\/linux\/ubuntu noble InRelease [48.8 kB]     \nGet:3 http:\/\/de.archive.ubuntu.com\/ubuntu noble-updates InRelease [126 kB]   \nHit:4 http:\/\/de.archive.ubuntu.com\/ubuntu noble-backports InRelease                     \nGet:5 http:\/\/security.ubuntu.com\/ubuntu noble-security InRelease [126 kB]\nGet:6 http:\/\/de.archive.ubuntu.com\/ubuntu noble-updates\/main amd64 Packages [177 kB]\nGet:7 http:\/\/de.archive.ubuntu.com\/ubuntu noble-updates\/main Translation-en [49.1 kB]                 \nGet:8 http:\/\/de.archive.ubuntu.com\/ubuntu noble-updates\/restricted amd64 Packages [70.1 kB]     \nGet:9 http:\/\/de.archive.ubuntu.com\/ubuntu noble-updates\/restricted Translation-en [14.3 kB]                            \nGet:10 http:\/\/de.archive.ubuntu.com\/ubuntu noble-updates\/universe amd64 Packages [66.7 kB]                  \nGet:11 http:\/\/de.archive.ubuntu.com\/ubuntu noble-updates\/universe Translation-en [25.1 kB]                         \nGet:12 http:\/\/de.archive.ubuntu.com\/ubuntu noble-updates\/multiverse amd64 Packages [2,968 B]                             \nGet:13 http:\/\/de.archive.ubuntu.com\/ubuntu noble-updates\/multiverse Translation-en [968 B]                               \nGet:15 http:\/\/security.ubuntu.com\/ubuntu noble-security\/main amd64 Packages [158 kB]                                     \nGet:16 http:\/\/security.ubuntu.com\/ubuntu noble-security\/main Translation-en [41.5 kB]\nGet:17 http:\/\/security.ubuntu.com\/ubuntu noble-security\/restricted amd64 Packages [70.1 kB]\nGet:18 http:\/\/security.ubuntu.com\/ubuntu noble-security\/restricted Translation-en [14.3 kB]               \nGet:19 http:\/\/security.ubuntu.com\/ubuntu noble-security\/universe amd64 Packages [44.4 kB]                  \nGet:20 http:\/\/security.ubuntu.com\/ubuntu noble-security\/universe Translation-en [17.0 kB]            \nGet:14 https:\/\/prod-cdn.packages.k8s.io\/repositories\/isv:\/kubernetes:\/core:\/stable:\/v1.30\/deb  InRelease [1,186 B]\nGet:21 https:\/\/prod-cdn.packages.k8s.io\/repositories\/isv:\/kubernetes:\/core:\/stable:\/v1.30\/deb  Packages [5,226 B]\nFetched 1,059 kB in 1s (1,025 kB\/s)    \nReading package lists... Done\nBuilding dependency tree... Done\nReading state information... Done\n30 packages can be upgraded. Run 'apt list --upgradable' to see them.\n<\/code><\/pre>\n\n\n\n<p>Next, confirm the available versions;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>apt-cache madison kubeadm kubelet kubectl<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>   kubeadm | 1.30.2-1.1 | https:\/\/pkgs.k8s.io\/core:\/stable:\/v1.30\/deb  Packages\n   kubeadm | 1.30.1-1.1 | https:\/\/pkgs.k8s.io\/core:\/stable:\/v1.30\/deb  Packages\n   kubeadm | 1.30.0-1.1 | https:\/\/pkgs.k8s.io\/core:\/stable:\/v1.30\/deb  Packages\n   kubelet | 1.30.2-1.1 | https:\/\/pkgs.k8s.io\/core:\/stable:\/v1.30\/deb  Packages\n   kubelet | 1.30.1-1.1 | https:\/\/pkgs.k8s.io\/core:\/stable:\/v1.30\/deb  Packages\n   kubelet | 1.30.0-1.1 | https:\/\/pkgs.k8s.io\/core:\/stable:\/v1.30\/deb  Packages\n   kubectl | 1.30.2-1.1 | https:\/\/pkgs.k8s.io\/core:\/stable:\/v1.30\/deb  Packages\n   kubectl | 1.30.1-1.1 | https:\/\/pkgs.k8s.io\/core:\/stable:\/v1.30\/deb  Packages\n   kubectl | 1.30.0-1.1 | https:\/\/pkgs.k8s.io\/core:\/stable:\/v1.30\/deb  Packages\n<\/code><\/pre>\n\n\n\n<p>So, we now have the correct patch release version available and we ready to run the upgrades.<\/p>\n\n\n\n<p>It is usually recommended to mark hold the Kubernetes packages to avoid automatic updates and keep a controlled version of the cluster. Check if these packages are hold;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo apt-mark showhold<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>kubeadm\nkubectl\nkubelet\n<\/code><\/pre>\n\n\n\n<p>Thus, to upgrade any hold package, unhold it;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo apt-mark unhold kubeadm kubelet kubectl<\/code><\/pre>\n\n\n\n<p>You are now ready to upgrade.<\/p>\n\n\n\n<p>Upgrade <strong>kubeadm<\/strong> on primary control plane node;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo apt update<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo apt install kubeadm<\/code><\/pre>\n\n\n\n<p>The confirm;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubeadm version -o yaml<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>clientVersion:\n  buildDate: \"2024-06-11T20:27:59Z\"\n  compiler: gc\n  gitCommit: 39683505b630ff2121012f3c5b16215a1449d5ed\n  gitTreeState: clean\n  <strong>gitVersion: v1.30.2<\/strong>\n  goVersion: go1.22.4\n  major: \"1\"\n  minor: \"30\"\n  platform: linux\/amd64\n<\/code><\/pre>\n\n\n\n<p>Upgrade the package on the other control plane nodes.<\/p>\n\n\n\n<p>Once <strong>kubeadm<\/strong> package is upgraded across all control plane nodes, verify that the cluster can be upgraded and fetch the version to upgrade to on the primary control plane node;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo kubeadm upgrade plan<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>[preflight] Running pre-flight checks.\n[upgrade\/config] Reading configuration from the cluster...\n[upgrade\/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'\n[upgrade] Running cluster health checks\n[upgrade] Fetching available versions to upgrade to\n[upgrade\/versions] Cluster version: 1.30.1\n[upgrade\/versions] kubeadm version: v1.30.2\n...\nYou can now apply the upgrade by executing the following command:\nkubeadm upgrade apply v1.30.2\n...\n<\/code><\/pre>\n\n\n\n<p>If the cluster is upgrade-able, then you will get the command to use to apply the upgrade.<\/p>\n\n\n\n<p>For example;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo kubeadm upgrade apply v1.30.2<\/code><\/pre>\n\n\n\n<p>When prompted to confirm, do confirm and proceed. It may take a while to complete the upgrade.<\/p>\n\n\n\n<p>After upgrade;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>...\n[addons] Applied essential addon: CoreDNS\n[addons] Applied essential addon: kube-proxy\n\n[upgrade\/successful] SUCCESS! Your cluster was upgraded to \"v1.30.2\". Enjoy!\n\n[upgrade\/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven't already done so.\n<\/code><\/pre>\n\n\n\n<p>While it is not really required, but the for the sake of consistency, run the command on other control plane node just to ensure everything is upgraded.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo kubeadm upgrade node<\/code><\/pre>\n\n\n\n<p>Next, upgrade the kubelet and the kubectl tool to the same version on all nodes (you can start with the leader):<\/p>\n\n\n\n<p>As mentioned before, for minor release upgrade, you would have to drain other control plane nodes before proceeding with kubelet\/kubectl upgrade.<\/p>\n\n\n\n<p>Since we are doing a patch release, just proceed.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo apt install kubelet kubectl<\/code><\/pre>\n\n\n\n<p>The services, <strong>kubelet<\/strong>, will be restarted during upgrade.<\/p>\n\n\n\n<p>After you are done with upgrade, hold the packages on all control plane nodes.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo apt-mark hold kubeadm kubelet kubectl<\/code><\/pre>\n\n\n\n<p>(Again, if you had drained the other nodes, uncordon them)<\/p>\n\n\n\n<p>Confirm the nodes versions;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl get nodes<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>NAME        STATUS   ROLES           AGE    VERSION\n<strong>master-01   Ready    control-plane   2d3h   v1.30.2\nmaster-02   Ready    control-plane   2d2h   v1.30.2\nmaster-03   Ready    control-plane   2d2h   v1.30.2<\/strong>\nworker-01   Ready    &lt;none&gt;          2d2h   v1.30.1\nworker-02   Ready    &lt;none&gt;          2d2h   v1.30.1\nworker-03   Ready    &lt;none&gt;          2d2h   v1.30.1\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"upgrade-worker-nodes\">Upgrade Worker Nodes<\/h4>\n\n\n\n<p>Next, upgrade the worker nodes one by one.<\/p>\n\n\n\n<p>Unhold the packages on all worker nodes;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo apt-mark unhold kubeadm kubelet kubectl<\/code><\/pre>\n\n\n\n<p>Upgrade kubeadm;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo apt update;sudo apt install kubeadm<\/code><\/pre>\n\n\n\n<p>Upgrade the Kubelet configuration;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo kubeadm upgrade node<\/code><\/pre>\n\n\n\n<p>Sample output;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>[upgrade] Reading configuration from the cluster...\n[upgrade] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'\n[preflight] Running pre-flight checks\n[preflight] Skipping prepull. Not a control plane node.\n[upgrade] Skipping phase. Not a control plane node.\n[upgrade] Backing up kubelet config file to \/etc\/kubernetes\/tmp\/kubeadm-kubelet-config2070136212\/config.yaml\n[kubelet-start] Writing kubelet configuration to file \"\/var\/lib\/kubelet\/config.yaml\"\n[upgrade] The configuration for this node was successfully updated!\n[upgrade] Now you should go ahead and upgrade the kubelet package using your package manager.\n<\/code><\/pre>\n\n\n\n<p>Again the node drain condition applies here! So proceed accordingly.<\/p>\n\n\n\n<p>Upgrade kubelet and kubectl;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo apt update;sudo apt install kubectl kubelet<\/code><\/pre>\n\n\n\n<p>Kubelet service will be restarted during upgrade.<\/p>\n\n\n\n<p>If you had drained the node, it is now time to uncordon it.<\/p>\n\n\n\n<p>Upgrade the next worker nodes using the same approach.<\/p>\n\n\n\n<p>Mark and hold the packages;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo apt-mark hold kubeadm kubelet kubectl<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"verify-cluster-upgrade\">Verify Cluster Upgrade<\/h3>\n\n\n\n<p>You can now check the status of the cluster upgrade by listing the nodes;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl get nodes<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>NAME        STATUS   ROLES           AGE    VERSION\nmaster-01   Ready    control-plane   2d3h   v1.30.2\nmaster-02   Ready    control-plane   2d3h   v1.30.2\nmaster-03   Ready    control-plane   2d3h   v1.30.2\nworker-01   Ready    &lt;none&gt;          2d3h   v1.30.2\nworker-02   Ready    &lt;none&gt;          2d3h   v1.30.2\nworker-03   Ready    &lt;none&gt;          2d3h   v1.30.2\n<\/code><\/pre>\n\n\n\n<p>And all nodes are of the same version. Ensure that all nodes are in a ready state.<\/p>\n\n\n\n<p>After a successful upgrade, I would recommended that you review cluster logs for any errors or warnings. Additionally, consider testing your applications to ensure they function as expected on the upgraded cluster.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"conclusion\">Conclusion<\/h3>\n\n\n\n<p>Upgrading a Kubernetes cluster created with kubeadm involves several critical steps to maintain the reliability and security of your infrastructure. By following these steps, you can successfully upgrade your  cluster while minimizing downtime and ensuring a smooth transition. Remember, planning, attention to detail, and following best practices are key to a seamless upgrade experience.<\/p>\n\n\n\n<p>Read more on <a href=\"https:\/\/kubernetes.io\/docs\/tasks\/administer-cluster\/kubeadm\/kubeadm-upgrade\/\" target=\"_blank\" rel=\"noreferrer noopener\">upgrading the cluster documentation<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this guide, we&#8217;ll walk you through the step-by-step process of how to safely upgrade kubeadm Kubernetes cluster. Upgrading a Kubernetes cluster that was originally<\/p>\n","protected":false},"author":10,"featured_media":22948,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"rank_math_lock_modified_date":false,"footnotes":""},"categories":[1076,121,1668],"tags":[7538,7536,7537],"class_list":["post-22933","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-containers","category-howtos","category-kubernetes","tag-upgrade-kubernetes-cluster","tag-upgrade-kubernetes-control-plane","tag-upgrade-kubernetes-worker-node","generate-columns","tablet-grid-50","mobile-grid-100","grid-parent","grid-50","resize-featured-image"],"_links":{"self":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/22933"}],"collection":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/comments?post=22933"}],"version-history":[{"count":10,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/22933\/revisions"}],"predecessor-version":[{"id":23183,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/22933\/revisions\/23183"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media\/22948"}],"wp:attachment":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media?parent=22933"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/categories?post=22933"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/tags?post=22933"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}