kubespray續簽k8s證書

落魄運維發表於2021-04-28
K8S

檢視證書過期時期

[root@node1 ~]# openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -text |grep ' Not '
            Not Before: Sep  4 08:29:00 2019 GMT
            Not After : Sep  3 08:29:02 2020 GMT

備份etcd

[root@node1 etcd-backup]# export ETCDCTL_API=3
[root@node1 etcd-backup]# etcdctl snapshot save "/root/$(date +%Y%m%d_%H%M%S)_snapshot.db" --cacert=/etc/ssl/etcd/ssl/ca.pem --cert=/etc/ssl/etcd/ssl/node-node1.pem --key=/etc/ssl/etcd/ssl/node-node1-key.pem --endpoints=https://192.168.10.132:2379
Snapshot saved at /root/20200814_142210_snapshot.db

master1備份證書

cp -ar /etc/kubernetes{,.bak}

Kubespray部署的k8s會生成以下證書

K8s元件之間認證需要的證書

ca.crt ca.key

apiserver.crt apiserver.key

apiserver-kubelet-client.crt apiserver-kubelet-client.key

front-proxy-ca.crt front-proxy-ca.key front-proxy-client.crt front-proxy-client.key

sa.key sa.pub

Etcd認證的證書

ca.pem ca-key.pem

admin-node*.pem admin-node*-key.pem

member-node*.pem member-node*-key.pem

node-node*.pem node-node*-key.pem

ca.crt預設是10年，apiserver.crt apiserver-kubelet-client.crt預設一年，front-proxy-ca.crt是獨立的ca證書，預設是10年， front-proxy-client.crt預設1年。Etcd的證書預設是100年
我們只需要更新apiserver.crt apiserver-kubelet-client.crt front-proxy-client即可

Master1節點重新生成證書，並同步證書至其他master節點

kubeadm alpha certs renew apiserver --config "/etc/kubernetes/kubeadm-config.yaml"
kubeadm alpha certs renew apiserver-kubelet-client --config "/etc/kubernetes/kubeadm-config.yaml"
kubeadm alpha certs renew front-proxy-client --config "/etc/kubernetes/kubeadm-config.yaml"

刪除所有主機元件之間認證的kubeconfig

Kubeconfig是k8s其他元件與apiserver通訊的認證金鑰，apiserver證書更新了，這些金鑰檔案都需要更新。

ansible -i /tools/kubespray/inventory/inventory.cfg all -m shell -a "cd /etc/kubernetes && rm -rf admin.conf scheduler.conf controller-manager.conf kubelet.conf bootstrap-kubelet.conf"

Master節點分別再次生成kubeconfig（所有master節點都需要執行）

kubeadm init phase kubeconfig all --config "/etc/kubernetes/kubeadm-config.yaml"

安裝有kubectl的節點覆蓋config

\cp /etc/kubernetes/admin.conf /root/.kube/config

master1節點重啟k8s相關元件

docker ps |grep apiserver|grep -v pause|awk '{print $1}'|xargs docker kill -s HUP

docker ps |grep kube-scheduler|grep -v pause|awk '{print $1}'|xargs docker kill -s HUP

docker ps |grep kube-controller|grep -v pause|awk '{print $1}'|xargs docker kill -s HUP

systemctl restart kubelet

拷貝master1的證書到master2上

scp -r /etc/kubernetes/ssl root@192.168.10.133:/etc/kubernetes/

master2上刪除舊配置

cd /etc/kubernetes && rm -rf admin.conf scheduler.conf controller-manager.conf kubelet.conf bootstrap-kubelet.conf

重新生成新配置

kubeadm init phase kubeconfig all --config "/etc/kubernetes/kubeadm-config.yaml"

master2節點重啟k8s相關元件

docker ps |grep apiserver|grep -v pause|awk '{print $1}'|xargs docker kill -s HUP

docker ps |grep kube-scheduler|grep -v pause|awk '{print $1}'|xargs docker kill -s HUP

docker ps |grep kube-controller|grep -v pause|awk '{print $1}'|xargs docker kill -s HUP

systemctl restart kubelet

檢視節點狀態

kubectl get node

worker node節點無需操作，證書會通過kubelet自動更新，各節點執行以下命令驗證續簽是否生效：

# Cert from api-server

echo -n | openssl s_client -connect localhost:6443 2>&1 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | openssl x509 -text -noout | grep Not

# Cert from controller manager

echo -n | openssl s_client -connect localhost:10257 2>&1 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | openssl x509 -text -noout | grep Not

# Cert from scheduler

echo -n | openssl s_client -connect localhost:10259 2>&1 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | openssl x509 -text -noout | grep Not

相關文章