Centos7升級K8S叢集
適用於使用kubeadm部署的k8s環境。
Kubernetes 版本以 x.y.z 表示，其中 x 是主要版本， y 是次要版本，z 是補丁版本。
升級時不可以跳過次版本。 例如，你只能從 1.y 升級到 1.y+1，而不能從 1.y 升級到 1.y+2。
版本偏差說明：
https://kubernetes.io/zh-cn/releases/version-skew-policy/
注意：
生產環境升級之前注意備份資料。為了方便恢復，如果是虛擬機器環境可以藉助kube快照進行恢復。
本次操作升級的為一個單節點的K8S環境，主要步驟如下：
1. 升級kubeadm
# 當前版本
[root@k8s ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
k8s Ready control-plane,master 3d18h v1.22.0
# 檢視可用的kubeadm版本。
[root@k8s ~]# yum list --showduplicates kubeadm --disableexcludes=kubernetes
# 升級kubeadm到1.23.17-0版本。
[root@k8s ~]# yum install -y kubeadm-1.23.17-0 --disableexcludes=kubernetes
# kubeadm upgrade plan驗證升級計劃，COMPONENT CURRENT TARGET ：告訴我們元件可以從當前版本升級到的版本。
[root@k8s ~]# kubeadm upgrade plan
[upgrade/config] Making sure the configuration is correct:
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[preflight] Running pre-flight checks.
[upgrade] Running cluster health checks
[upgrade] Fetching available versions to upgrade to
[upgrade/versions] Cluster version: v1.22.0
[upgrade/versions] kubeadm version: v1.23.17
I0615 16:48:13.655864 54942 version.go:256] remote version is much newer: v1.30.2; falling back to: stable-1.23
[upgrade/versions] Target version: v1.23.17
[upgrade/versions] Latest version in the v1.22 series: v1.22.17
Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':
COMPONENT CURRENT TARGET
kubelet 1 x v1.22.0 v1.22.17
Upgrade to the latest version in the v1.22 series:
COMPONENT CURRENT TARGET
kube-apiserver v1.22.0 v1.22.17
kube-controller-manager v1.22.0 v1.22.17
kube-scheduler v1.22.0 v1.22.17
kube-proxy v1.22.0 v1.22.17
CoreDNS v1.8.4 v1.8.6
etcd 3.5.0-0 3.5.6-0
You can now apply the upgrade by executing the following command:
kubeadm upgrade apply v1.22.17
_____________________________________________________________________
Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':
COMPONENT CURRENT TARGET
kubelet 1 x v1.22.0 v1.23.17
Upgrade to the latest stable version:
COMPONENT CURRENT TARGET
kube-apiserver v1.22.0 v1.23.17
kube-controller-manager v1.22.0 v1.23.17
kube-scheduler v1.22.0 v1.23.17
kube-proxy v1.22.0 v1.23.17
CoreDNS v1.8.4 v1.8.6
etcd 3.5.0-0 3.5.6-0
You can now apply the upgrade by executing the following command:
kubeadm upgrade apply v1.23.17
_____________________________________________________________________
The table below shows the current state of component configs as understood by this version of kubeadm.
Configs that have a "yes" mark in the "MANUAL UPGRADE REQUIRED" column require manual config upgrade or
resetting to kubeadm defaults before a successful upgrade can be performed. The version to manually
upgrade to is denoted in the "PREFERRED VERSION" column.
API GROUP CURRENT VERSION PREFERRED VERSION MANUAL UPGRADE REQUIRED
kubeproxy.config.k8s.io v1alpha1 v1alpha1 no
kubelet.config.k8s.io v1beta1 v1beta1 no
_____________________________________________________________________
2. 升級各個元件
接下來升級各個元件，包括kube-apiserver，kube-controller-manager等等
kubeadm upgrade apply v1.23.17 升級各個元件到1.23.17版本，如果etcd這個元件不想升級，可以加上選項:kubeadm upgrade apply v1.23.17 --etcd-upgrade=false。
# 透過將節點標記為不可排程並騰空節點為節點作升級準備：
[root@k8s ~]# kubectl drain k8s --ignore-daemonsets --delete-emptydir-data
# 升級各個元件
[root@k8s ~]# kubeadm upgrade apply v1.23.17 --etcd-upgrade=false
# 如果是worker節點，執行`kubeadm upgrade node`升級
# 解除節點保護
[root@k8s ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
k8s Ready,SchedulingDisabled control-plane,master 3d18h v1.22.0
[root@k8s ~]# kubectl uncordon k8s
node/k8s uncordoned
[root@k8s ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
k8s Ready control-plane,master 3d18h v1.22.0
# 此時查詢到的版本還是老版本
3. 升級kubelet和kubectl
升級 kubelet 和 kubectl到1.21.9版本：
[root@k8s ~]# yum install -y kubelet-1.23.17 kubectl-1.23.17 --disableexcludes=kubernetes
# 重啟kubelet
[root@k8s ~]# systemctl daemon-reload ;systemctl restart kubelet
# 檢視版本資訊
[root@k8s ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
k8s Ready control-plane,master 3d19h v1.23.17
本文只記錄了單節點環境的升級，如果是升級工作節點步驟與上面過程類似，只是worker節點不需要驗證升級計劃、無需升級管理元件，在上面的步驟中，將節點設定為不可排程後，執行kubeadm upgrade node升級worker節點即可。
升級worker節點步驟：
- 升級kubeadm版本；
- 騰空節點；
- 管理節點執行
kubeadm upgrade node升級worker節點； - uncordon節點；
- workder節點升級kubelet和kubectl並重啟；
- 管理節點驗證升級情況；