Kubernetes Part 17: How-To Upgrade Kubernetes cluster to newer version

In this blog I will explain how-to upgrade your k8s kubernetes cluster to a newer version. In this example I will upgrade my kubernetes cluster from version 1.23.0 to version 1.23.1. The upgrade procedure is not that difficult. I have upgraded my server from my original version 1.19.2 up to the version 1.23.1 (and all versions in between) with no serious issues.

The upgrade the kubernetes cluster consists of these steps:

1. Upgrade the software on the master server and worker nodes to the latest version
2. Upgrade the deployment of the system containers to the latest version
3. Sometimes with major version upgrades you need to update yaml file format if new or updates API's are introduced or depricated.  This is usually announced in a long time in advance.  For  example the new ingress api with the introduction of kubernetes version 1.22 (see link )

Option 3 doesn't occur that often, but it is always recommended to read the kubernetes blog to get info about the new kubernetest version.  Also note that during the upgrade process you will get notified when there are major updates on the API's. 

It's always recommended to do a backup before you do an upgrade. You can use velero for kubernetes backups as described in of my previous blog.

So when everything is ready, let's start with the upgrade with step 1.

1. Upgrade the software on the master server and worker nodes to the latest version

Upgrading the software packages on the master and workers nodes is pretty straightforward. This just like upgrade regular linux packages via apt. So start an ssh session to your master and worker nodes and type the following command:

sudo apt update && sudo apt upgrade -y

This command will update the software repositories and upgrade the latest version of all installed packages (like the kubernetes package kubeadm, kubectl and kubelet) on Ubuntu linux we are using on our Raspberry Pi cluster.  (see screenshot below).

When we have updated all the software packages on your master and worker nodes, you can check the installed version of kubernetes on your master node via the command. 

kubectl get nodes

As you can see in the screenshot below version v1.23.1 is installed on the master and worker nodes.

2. Upgrade the deployment of the system containers to the latest version

Now the next bit maybe a bit confusing. Although the software itself has been updated on your master and worker nodes, the kubernetes cluster components are not.  

If you type in the following command on your master node.

kubectl version

 

You will see the following output (see screenshot below)

You see that your client version has been upgraded to v1.23.1, but the that the server version has not, that is still on v.1.23.0. 

If you now type the following command on your master node.

sudo kubeadm upgrade plan

This command will run a check and will show you what it plans to upgrade on your k8s cluster and provides you with additional info about the upgrade. (see screenshot below)

As you can see in this case,  it will upgrade the kubernetes components kube-apiserver, kube-controller-manager, kube-scheduler and kube-proxy from version 1.23.0 to 1.23.1. This basically means it will replace these kubernetes system containers with the newer version.

Also have a look at the recommendations of the API Group. This tells you which API versions your kubernetes cluster is using and if it requires a manual upgrade. In this example it doesn't.

So we are ready to go, and run the upgrade command as mentioned in the plan overview, which is in our case v1.23.1, this might be different in your case.

sudo kubeadm upgrade apply v1.23.1 

You should something similar like the screenshot below, and press "y" to start the upgrade.

Now the kubernetes upgrade will start. It will download the latest kubernetes container images and deploy them, just like the screens shots below.  

Tip: If you have a low bandwith internetes connection you can use the command "sudo kubeadm config images pull" to download the new kubernetes images in advance.

After the upgrade of a container is will wait (statis pod messages) until it's started before continuing with the next container.

When everything went ok. You will see the message "SUCCES! Your cluster was upgrade to "v.1.23.1". Enjoy!"

Now run the following command on your master server again:

kubectl version

And you shall see that the server part is now also upgraded to version v1.23.1.

We had no API updates during this upgrade, so step 3 wasn't required. When it does in the future I will write a blog about it.

I hope you find this blog useful. You can find yaml files en scripts used in my blogs on my github page here

If you have any questions, do not hesitate to leave a comment below.