本文分享自華為雲社群《openEuler部署Kubernetes 1.29.4版本叢集》，作者：江晚正愁餘。
一、Kubernetes叢集節點準備
1.1 主機作業系統說明
序號 作業系統及版本 備註
1 CentOS7u9或 OpenEuler2203
1.2 主機硬體配置說明
需求 CPU 記憶體 硬碟 角色 主機名
值 8C 8G 1024GB master k8s-master01
值 8C 16G 1024GB worker(node) k8s-worker01
值 8C 16G 1024GB worker(node) k8s-worker02
1.3 主機配置
1.3.1 主機名配置
由於本次使用3臺主機完成kubernetes叢集部署，其中1臺為master節點,名稱為k8s-master01;其中2臺為worker節點，名稱分別為：k8s-worker01及k8s-worker02
# master節點 hostnamectl set-hostname k8s-master01 #worker01節點 hostnamectl set-hostname k8s-worker01 #worker02節點 hostnamectl set-hostname k8s-worker02
1.3.2 IP地址,名稱解析與互信
#IP配置這裡不再講解 #下面是名稱解析配置 [root@k8s-master01 ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.0.11 k8s-master01 192.168.0.12 k8s-worker01 192.168.0.13 k8s-worker02 #主機互信配置 [root@k8s-master01 ~]# ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa Your public key has been saved in /root/.ssh/id_rsa.pub The key fingerprint is: SHA256:Rr6W4rdnY350fzMeszeWFR/jUJt0VOZ3yZECp5VJJQA root@k8s-master01 The key's randomart image is: +---[RSA 3072]----+ | E.o+=++*| | ++o*+| | . . +oB| | o . *o| | S o =| | . o . ..o| | . + . . +o| | . o. = . *B| | ...*.o oo*| +----[SHA256]-----+ [root@k8s-master01 ~]# for i in {11..13};do ssh-copy-id 192.168.0.${i};done /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub" The authenticity of host '192.168.0.11 (192.168.0.11)' can't be established. ED25519 key fingerprint is SHA256:s2R582xDIla4wyNozHa/HEmRR7LOU4WAciEcAw57U/Q. This key is not known by any other names Are you sure you want to continue connecting (yes/no/[fingerprint])? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys Authorized users only. All activities may be monitored and reported. root@192.168.0.11's password: Number of key(s) added: 1
1.3.4 防火牆配置
所有主機均需要操作。
關閉現有防火牆firewalld
# systemctl disable firewalld
# systemctl stop firewalld
或
systemctl disable --now firewalld
檢視firewalld狀態
# firewall-cmd --state
not running
參考執行命令：
[root@k8s-master01 ~]# for i in {11..13};do ssh 192.168.0.${i} 'systemctl disable --now firewalld' ;done Authorized users only. All activities may be monitored and reported. Authorized users only. All activities may be monitored and reported. Authorized users only. All activities may be monitored and reported. [root@k8s-master01 ~]# for i in {11..13};do ssh 192.168.0.${i} 'firewall-cmd --state' ;done Authorized users only. All activities may be monitored and reported. not running Authorized users only. All activities may be monitored and reported. not running Authorized users only. All activities may be monitored and reported. not running
1.3.5 SELINUX配置
所有主機均需要操作。修改SELinux配置需要重啟作業系統。
# sed -ri 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config # sestatus
參考執行命令：
[root@k8s-master01 ~]# for i in {11..13};do ssh 192.168.0.${i} 'sed -ri 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config' ;done Authorized users only. All activities may be monitored and reported. Authorized users only. All activities may be monitored and reported. Authorized users only. All activities may be monitored and reported. [root@k8s-master01 ~]# for i in {11..13};do ssh 192.168.0.${i} 'sestatus' ;done Authorized users only. All activities may be monitored and reported. SELinux status: disabled Authorized users only. All activities may be monitored and reported. SELinux status: disabled Authorized users only. All activities may be monitored and reported. SELinux status: disabled
1.3.6 時間同步配置
所有主機均需要操作。最小化安裝系統需要安裝ntpdate軟體。
# crontab -l 0 */1 * * * /usr/sbin/ntpdate time1.aliyun.com for i in {11..13};do ssh 192.168.0.${i} ' echo '0 */1 * * * /usr/sbin/ntpdate time1.aliyun.com' >> /etc/crontab' ;done #設定上海時區，東八區 timedatectl set-timezone Asia/Shanghai for i in {11..13};do ssh 192.168.0.${i} ' timedatectl set-timezone Asia/Shanghai' ;done
1.3.7 升級作業系統核心
centos系統需要升級內容，具體百度，OpenEuler2203不需要
1.3.8 配置核心路由轉發及網橋過濾
所有主機均需要操作。
新增網橋過濾及核心轉發配置檔案
sed -i 's/net.ipv4.ip_forward=0/net.ipv4.ip_forward=1/g' /etc/sysctl.conf # cat > /etc/sysctl.d/k8s.conf << EOF net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 vm.swappiness = 0 EOF # 配置載入br_netfilter模組 cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf overlay br_netfilter EOF #載入br_netfilter overlay模組 modprobe br_netfilter modprobe overlay #檢視是否載入 # lsmod | grep br_netfilter br_netfilter 22256 0 bridge 151336 1 br_netfilter # 使其生效 sysctl --system # 使用預設配置檔案生效 sysctl -p # 使用新新增配置檔案生效 sysctl -p /etc/sysctl.d/k8s.conf
1.3.9 安裝ipset及ipvsadm
所有主機均需要操作。
安裝ipset及ipvsadm # yum -y install ipset ipvsadm 配置ipvsadm模組載入方式 新增需要載入的模組 # cat > /etc/sysconfig/modules/ipvs.modules <<EOF #!/bin/bash modprobe -- ip_vs modprobe -- ip_vs_rr modprobe -- ip_vs_wrr modprobe -- ip_vs_sh modprobe -- nf_conntrack EOF 授權、執行、檢查是否載入 chmod 755 /etc/sysconfig/modules/ipvs.module && /etc/sysconfig/modules/ipvs.module 檢視對應的模組是否載入成功 # lsmod | grep -e ip_vs -e nf_conntrack_ipv4
1.3.10 關閉SWAP分割槽
修改完成後需要重啟作業系統，如不重啟，可臨時關閉，命令為swapoff -a
永遠關閉swap分割槽，需要重啟作業系統
# cat /etc/fstab ...... # /dev/mapper/centos-swap swap swap defaults 0 0 在上一行中行首新增#
二、containerd容器環境安裝
2.1 安裝containerd環境包
所有主機均需要操作。
# 打包的檔案 for i in {11..13};do ssh 192.168.0.${i} ' wget https://blog-source-mkt.oss-cn-chengdu.aliyuncs.com/resources/k8s/kubeadm%20init/k8s1.29.tar.gz'; done # 解壓containerd並安裝 for i in {11..13};do ssh 192.168.0.${i} ' tar -zxvf /root/k8s1.29.tar.gz'; done for i in {11..13};do ssh 192.168.0.${i} ' tar -zxvf /root/workdir/containerd-1.7.11-linux-amd64.tar.gz && mv /root/bin/* /usr/local/bin/ && rm -rf /root/bin'; done # 建立服務，所有主機都要操作 cat << EOF > /usr/lib/systemd/system/containerd.service [Unit] Description=containerd container runtime Documentation=https://containerd.io After=network.target local-fs.target [Service] ExecStartPre=-/sbin/modprobe overlay ExecStart=/usr/local/bin/containerd Type=notify Delegate=yes KillMode=process Restart=always RestartSec=5 # Having non-zero Limit*s causes performance problems due to accounting overhead # in the kernel. We recommend using cgroups to do container-local accounting. LimitNPROC=infinity LimitCORE=infinity # Comment TasksMax if your systemd version does not supports it. # Only systemd 226 and above support this version. TasksMax=infinity OOMScoreAdjust=-999 [Install] WantedBy=multi-user.target EOF # 啟動容器服務 for i in {11..13};do ssh 192.168.0.${i} 'systemctl daemon-reload && systemctl enable --now containerd '; done # 安裝runc for i in {11..13};do ssh 192.168.0.${i} 'install -m 755 /root/workdir/runc.amd64 /usr/local/sbin/runc '; done # 安裝cni外掛 for i in {11..13};do ssh 192.168.0.${i} 'mkdir -p /opt/cni/bin && tar -xzvf /root/workdir/cni-plugins-linux-amd64-v1.4.0.tgz -C /opt/cni/bin/ '; done # 生成容器配置檔案並修改 for i in {11..13};do ssh 192.168.0.${i} 'mkdir -p /etc/containerd && containerd config default | sudo tee /etc/containerd/config.toml '; done # 修改沙箱映象，所有主機都要操作 sed -i 's#sandbox_image = "registry.k8s.io/pause:.*"#sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.9"#' /etc/containerd/config.toml #重啟containerd systemctl restart containerd
2.2 master主機安裝k8s
# 配置k8s v2.19源，所有節點均要安裝 cat <<EOF | tee /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.29/rpm/ enabled=1 gpgcheck=1 gpgkey=https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.29/rpm/repodata/repomd.xml.key EOF # 安裝k8s工具，所有節點均要安裝 yum clean all && yum makecache yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes # 配置kubelet為了實現docker使用的cgroupdriver與kubelet使用的cgroup的一致性，建議修改如下檔案內容。所有節點均要安裝 # vim /etc/sysconfig/kubelet KUBELET_EXTRA_ARGS="--cgroup-driver=systemd" 或是下面命令 echo 'KUBELET_EXTRA_ARGS="--cgroup-driver=systemd"' > /etc/sysconfig/kubelet systemctl enable kubelet #注意，kubelet不要啟動，kubeadm會自動啟動，如果已啟動，安裝會報錯。 # 安裝k8s命令，主master節點執行，這裡只有1.29.4版本映象 kubeadm init --apiserver-advertise-address=192.168.0.11 --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.29.4 --service-cidr=10.96.0.0/12 --pod-network-cidr=10.224.0.0/16 # 最後執行以下命令 mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config export KUBECONFIG=/etc/kubernetes/admin.conf
2.3 安裝calico網路外掛
kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.26.1/manifests/calico.yaml # 最後檢視節點與pod支行情況 kubectl get nodes kubectl get pods -A
點選關注，第一時間瞭解華為雲新鮮技術~