Author: ACatSmiling
Since: 2024-10-17
配置要求
伺服器要求：
- k8s-master：192.168.1.120。
- k8s-node1：192.168.1.121。
- k8s-node1：192.168.1.122。
每臺伺服器最低配置：2 核、2G 記憶體、20G 硬碟。
使用 Hyper-V 時，注意配置動態記憶體的最小值：
作業系統
[root@k8s-master k8s]# cat /etc/centos-release
CentOS Linux release 7.9.2009 (Core)
[root@k8s-master k8s]# uname -r
3.10.0-1160.el7.x86_64
[root@k8s-master k8s]# uname -a
Linux k8s-master 3.10.0-1160.el7.x86_64 #1 SMP Mon Oct 19 16:18:59 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
設定 hostname
[root@k8s-master ~]# hostnamectl set-hostname k8s-master
- 修改 hostname 後，需要重啟虛擬機器。
設定 hosts
[root@k8s-master ~]# vim /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.120 k8s-master
192.168.1.121 k8s-node1
192.168.1.122 k8s-node2
關閉防火牆
# 檢視防火牆狀態
[root@k8s-master ~]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2024-08-27 23:20:58 CST; 21min ago
Docs: man:firewalld(1)
Main PID: 549 (firewalld)
CGroup: /system.slice/firewalld.service
└─549 /usr/bin/python2 -Es /usr/sbin/firewalld --nofork --nopid
Aug 27 23:20:58 master systemd[1]: Starting firewalld - dynamic firewall daemon...
Aug 27 23:20:58 master systemd[1]: Started firewalld - dynamic firewall daemon.
Aug 27 23:20:58 master firewalld[549]: WARNING: AllowZoneDrifting is enabled. This is considered an insecure configuration option. It will... it now.
Hint: Some lines were ellipsized, use -l to show in full.
# 關閉防火牆
[root@k8s-master ~]# systemctl stop firewalld
# 禁止防火牆開機自啟
[root@k8s-master ~]# systemctl disable firewalld
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@k8s-master ~]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
Active: inactive (dead)
Docs: man:firewalld(1)
Aug 27 23:20:58 master systemd[1]: Starting firewalld - dynamic firewall daemon...
Aug 27 23:20:58 master systemd[1]: Started firewalld - dynamic firewall daemon.
Aug 27 23:20:58 master firewalld[549]: WARNING: AllowZoneDrifting is enabled. This is considered an insecure configuration option. It will... it now.
Aug 27 23:42:27 master systemd[1]: Stopping firewalld - dynamic firewall daemon...
Aug 27 23:42:28 master systemd[1]: Stopped firewalld - dynamic firewall daemon.
Hint: Some lines were ellipsized, use -l to show in full.
$ systemctl stop firewalld $ systemctl disable firewalld
關閉 swap
[root@k8s-master ~]# free -m
total used free shared buff/cache available
Mem: 3950 2286 918 8 745 1437
Swap: 4095 0 4095
[root@k8s-master ~]# cat /etc/fstab
#
# /etc/fstab
# Created by anaconda on Thu Aug 22 23:53:48 2024
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=f724baab-c349-497d-ba1f-da8f619fdf89 / xfs defaults 0 0
UUID=F0B8-EAC5 /boot/efi vfat umask=0077,shortname=winnt 0 0
UUID=94ca6141-23b0-4eff-ab4f-ae74b1cfe406 swap swap defaults 0 0
# 永久關閉 swap
[root@k8s-master ~]# sed -ri 's/.*swap.*/#&/' /etc/fstab
[root@k8s-master ~]# cat /etc/fstab
#
# /etc/fstab
# Created by anaconda on Thu Aug 22 23:53:48 2024
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=f724baab-c349-497d-ba1f-da8f619fdf89 / xfs defaults 0 0
UUID=F0B8-EAC5 /boot/efi vfat umask=0077,shortname=winnt 0 0
#UUID=94ca6141-23b0-4eff-ab4f-ae74b1cfe406 swap swap defaults 0 0
# 關閉 swap 後，一定要重啟虛擬機器
[root@k8s-master ~]# reboot
[root@k8s-master ~]# free -m
total used free shared buff/cache available
Mem: 3950 198 3671 8 79 3593
Swap: 0 0 0
- 臨時關閉：
swapoff -a。
$ sed -ri 's/.*swap.*/#&/' /etc/fstab $ reboot
關閉 selinux
[root@k8s-master ~]# cat /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=enforcing
# SELINUXTYPE= can take one of three values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
# 永久關閉
[root@k8s-master ~]# sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config
[root@k8s-master ~]# cat /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
- 臨時關閉：
setenforce 0。
SELinux 是一個為 Linux 提供訪問控制安全策略的安全模組，它透過定義和執行安全策略來限制程序對系統資源的訪問，從而增強系統的安全性。然而，K8s 叢集的某些元件或配置可能與 SELinux 的預設策略不相容，導致安裝或執行過程中出現許可權問題。
- 臨時關閉
# 此命令將 SELinux 設定為寬容模式（permissive mode），在這種模式下，SELinux 會記錄違反策略的行為但不會阻止它們 $ setenforce 0
- 永久關閉
$ sed -i 's/SELINUX=permissive/SELINUX=disabled/' /etc/sysconfig/selinux $ sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config $ reboot需要注意的是，關閉 SELinux 會降低系統的安全性。因此，在關閉 SELinux 之前，應該仔細評估潛在的安全風險，並確保已經採取了其他適當的安全措施來保護系統。
在 K8s 叢集中，關閉 Swap 和 SELinux 是出於效能和穩定性方面的考慮。
為什麼要關閉 Swap？
效能問題：
Swap 記憶體通常比實體記憶體慢很多。當系統記憶體不足時，Linux 會將部分記憶體內容交換到磁碟上的 Swap 分割槽中，這會導致應用程式的效能顯著下降。在 K8s 叢集中，容器應用對效能有較高要求，頻繁的 Swap 操作會嚴重影響這些應用的執行效率。
關閉 Swap 可以避免這種效能損失，確保容器應用能夠直接從實體記憶體中獲取所需資源，從而提高整體效能。
穩定性問題：
K8s 對容器執行環境的要求比較高，關閉 Swap 可以減少因為記憶體兌換（即記憶體與 Swap 之間的資料交換）引起的異常現象。這有助於確保叢集的穩定性，避免因為記憶體問題導致的容器崩潰或系統不穩定。
資源管理：
K8s 本身對記憶體的管理非常嚴格，它透過資源配額（Resource Quotas）和限制（Limits）來確保容器不會超出其分配的資源範圍。如果啟用 Swap，這些管理機制可能無法有效發揮作用，因為 Swap 允許容器在實體記憶體不足時繼續執行，但效能會大幅下降。
為什麼要關閉 SELinux？
相容性問題：
SELinux 是一種安全增強型的 Linux 核心安全模組，它可以提供強大的訪問控制機制。然而，在某些情況下，SELinux 可能會與 K8s 的某些元件或特性不相容，導致叢集執行不穩定或出現許可權問題。
簡化部署和管理：
關閉 SELinux 可以簡化 K8s 的部署和管理過程。在沒有 SELinux 的情況下，管理員可以更容易地配置和除錯叢集，而無需擔心 SELinux 的安全策略可能會干擾叢集的正常執行。
安全風險：
雖然 SELinux 可以增強系統的安全性，但在某些情況下，它也可能成為安全漏洞的源頭。例如，如果 SELinux 的策略配置不當，可能會允許未授權的訪問或操作。因此，在某些情況下，關閉 SELinux 可能是一種更安全的選擇，特別是當管理員能夠確保透過其他方式（如網路隔離、身份驗證等）來保護系統時。
綜上所述，關閉 Swap 和 SELinux 是 K8s 叢集部署中的常見做法，旨在提高叢集的效能和穩定性，並簡化部署和管理過程。然而，這些決策也需要在安全性和效能之間做出權衡，並根據具體的業務需求和安全策略來確定是否適用。
將橋接的 IPv4 流量傳遞到 iptables 的鏈
[root@k8s-master ~]# cat > /etc/sysctl.d/k8s.conf << EOF
> net.ipv4.ip_forward = 1
> net.bridge.bridge-nf-call-ip6tables = 1
> net.bridge.bridge-nf-call-iptables = 1
> vm.swappiness = 0
> EOF
# 生效配置
[root@k8s-master ~]# sysctl --system
* Applying /usr/lib/sysctl.d/00-system.conf ...
* Applying /usr/lib/sysctl.d/10-default-yama-scope.conf ...
kernel.yama.ptrace_scope = 0
* Applying /usr/lib/sysctl.d/50-default.conf ...
kernel.sysrq = 16
kernel.core_uses_pid = 1
kernel.kptr_restrict = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.promote_secondaries = 1
net.ipv4.conf.all.promote_secondaries = 1
fs.protected_hardlinks = 1
fs.protected_symlinks = 1
* Applying /etc/sysctl.d/99-sysctl.conf ...
* Applying /etc/sysctl.d/k8s.conf ...
net.ipv4.ip_forward = 1
vm.swappiness = 0
* Applying /etc/sysctl.conf ...
$ cat > /etc/sysctl.d/k8s.conf << EOF net.ipv4.ip_forward = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 vm.swappiness = 0 EOF $ sysctl --system
時間同步
[root@k8s-master ~]# yum -y install ntpdate
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors.aliyun.com
* extras: mirrors.aliyun.com
* updates: mirrors.aliyun.com
Resolving Dependencies
--> Running transaction check
---> Package ntpdate.x86_64 0:4.2.6p5-29.el7.centos.2 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
=====================================================================================================================================================
Package Arch Version Repository Size
=====================================================================================================================================================
Installing:
ntpdate x86_64 4.2.6p5-29.el7.centos.2 base 87 k
Transaction Summary
=====================================================================================================================================================
Install 1 Package
Total download size: 87 k
Installed size: 121 k
Downloading packages:
ntpdate-4.2.6p5-29.el7.centos.2.x86_64.rpm | 87 kB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : ntpdate-4.2.6p5-29.el7.centos.2.x86_64 1/1
Verifying : ntpdate-4.2.6p5-29.el7.centos.2.x86_64 1/1
Installed:
ntpdate.x86_64 0:4.2.6p5-29.el7.centos.2
Complete!
[root@k8s-master ~]# ntpdate time.windows.com
28 Aug 00:08:27 ntpdate[1077]: adjust time server 52.231.114.183 offset 0.248979 sec
$ yum -y install ntpdate $ ntpdate time.windows.com
Docker 安裝
安裝過程略。
注意，Docker 在預設情況下使用 Vgroup Driver 為 cgroupfs，而 Kubernetes 推薦使用 systemd 來替代 cgroupfs，因此，Docker 安裝好後，需要修改配置檔案 daemon.json，新增以下配置：
"exec-opts": ["native.cgroupdriver=systemd"]
配置 Kubernetes 映象源
[root@k8s-master yum.repos.d]# cat > /etc/yum.repos.d/kubernetes.repo << EOF
> [kubernetes]
> name=Kubernetes
> baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
> enabled=1
> gpgcheck=0
> repo_gpgcheck=0
> gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
> EOF
$ cat > /etc/yum.repos.d/kubernetes.repo << EOF [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=0 repo_gpgcheck=0 gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg EOF
安裝 kubeadm、kubelet 和 kubectl
[root@k8s-master yum.repos.d]# yum install -y --setopt=obsoletes=0 kubeadm-1.23.6 kubelet-1.23.6 kubectl-1.23.6
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors.aliyun.com
* extras: mirrors.aliyun.com
* updates: mirrors.aliyun.com
base | 3.6 kB 00:00:00
docker-ce-stable | 3.5 kB 00:00:00
extras | 2.9 kB 00:00:00
kubernetes | 1.4 kB 00:00:00
updates | 2.9 kB 00:00:00
kubernetes/primary | 137 kB 00:00:00
kubernetes 1022/1022
Resolving Dependencies
--> Running transaction check
---> Package kubeadm.x86_64 0:1.23.6-0 will be installed
--> Processing Dependency: kubernetes-cni >= 0.8.6 for package: kubeadm-1.23.6-0.x86_64
--> Processing Dependency: cri-tools >= 1.19.0 for package: kubeadm-1.23.6-0.x86_64
---> Package kubectl.x86_64 0:1.23.6-0 will be installed
---> Package kubelet.x86_64 0:1.23.6-0 will be installed
--> Processing Dependency: socat for package: kubelet-1.23.6-0.x86_64
--> Processing Dependency: conntrack for package: kubelet-1.23.6-0.x86_64
--> Running transaction check
---> Package conntrack-tools.x86_64 0:1.4.4-7.el7 will be installed
--> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.1)(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
--> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.0)(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
--> Processing Dependency: libnetfilter_cthelper.so.0(LIBNETFILTER_CTHELPER_1.0)(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
--> Processing Dependency: libnetfilter_queue.so.1()(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
--> Processing Dependency: libnetfilter_cttimeout.so.1()(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
--> Processing Dependency: libnetfilter_cthelper.so.0()(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
---> Package cri-tools.x86_64 0:1.26.0-0 will be installed
---> Package kubernetes-cni.x86_64 0:1.2.0-0 will be installed
---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed
--> Running transaction check
---> Package libnetfilter_cthelper.x86_64 0:1.0.0-11.el7 will be installed
---> Package libnetfilter_cttimeout.x86_64 0:1.0.0-7.el7 will be installed
---> Package libnetfilter_queue.x86_64 0:1.0.2-2.el7_2 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
=====================================================================================================================================================
Package Arch Version Repository Size
=====================================================================================================================================================
Installing:
kubeadm x86_64 1.23.6-0 kubernetes 9.0 M
kubectl x86_64 1.23.6-0 kubernetes 9.5 M
kubelet x86_64 1.23.6-0 kubernetes 21 M
Installing for dependencies:
conntrack-tools x86_64 1.4.4-7.el7 base 187 k
cri-tools x86_64 1.26.0-0 kubernetes 8.6 M
kubernetes-cni x86_64 1.2.0-0 kubernetes 17 M
libnetfilter_cthelper x86_64 1.0.0-11.el7 base 18 k
libnetfilter_cttimeout x86_64 1.0.0-7.el7 base 18 k
libnetfilter_queue x86_64 1.0.2-2.el7_2 base 23 k
socat x86_64 1.7.3.2-2.el7 base 290 k
Transaction Summary
=====================================================================================================================================================
Install 3 Packages (+7 Dependent packages)
Total download size: 65 M
Installed size: 296 M
Downloading packages:
(1/10): conntrack-tools-1.4.4-7.el7.x86_64.rpm | 187 kB 00:00:00
(2/10): 89104c7beafab5f04d6789e5425963fc8f91ba9711c9603f1ad89003cdea4fe4-kubeadm-1.23.6-0.x86_64.rpm | 9.0 MB 00:00:02
(3/10): 3f5ba2b53701ac9102ea7c7ab2ca6616a8cd5966591a77577585fde1c434ef74-cri-tools-1.26.0-0.x86_64.rpm | 8.6 MB 00:00:02
(4/10): 868c4a6ee448d1e8488938812a19a991b5132c81de511cd737d93493b98451cc-kubectl-1.23.6-0.x86_64.rpm | 9.5 MB 00:00:02
(5/10): libnetfilter_cthelper-1.0.0-11.el7.x86_64.rpm | 18 kB 00:00:00
(6/10): libnetfilter_cttimeout-1.0.0-7.el7.x86_64.rpm | 18 kB 00:00:00
(7/10): libnetfilter_queue-1.0.2-2.el7_2.x86_64.rpm | 23 kB 00:00:00
(8/10): socat-1.7.3.2-2.el7.x86_64.rpm | 290 kB 00:00:00
(9/10): 68a98b2ae673eef4a5ddbf1f3c830db0df8fbb888e035aea6054677d88f8a8bc-kubelet-1.23.6-0.x86_64.rpm | 21 MB 00:00:05
(10/10): 0f2a2afd740d476ad77c508847bad1f559afc2425816c1f2ce4432a62dfe0b9d-kubernetes-cni-1.2.0-0.x86_64.rpm | 17 MB 00:00:04
-----------------------------------------------------------------------------------------------------------------------------------------------------
Total 7.3 MB/s | 65 MB 00:00:08
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : kubectl-1.23.6-0.x86_64 1/10
Installing : libnetfilter_cthelper-1.0.0-11.el7.x86_64 2/10
Installing : socat-1.7.3.2-2.el7.x86_64 3/10
Installing : libnetfilter_cttimeout-1.0.0-7.el7.x86_64 4/10
Installing : cri-tools-1.26.0-0.x86_64 5/10
Installing : libnetfilter_queue-1.0.2-2.el7_2.x86_64 6/10
Installing : conntrack-tools-1.4.4-7.el7.x86_64 7/10
Installing : kubernetes-cni-1.2.0-0.x86_64 8/10
Installing : kubelet-1.23.6-0.x86_64 9/10
Installing : kubeadm-1.23.6-0.x86_64 10/10
Verifying : kubeadm-1.23.6-0.x86_64 1/10
Verifying : conntrack-tools-1.4.4-7.el7.x86_64 2/10
Verifying : libnetfilter_queue-1.0.2-2.el7_2.x86_64 3/10
Verifying : cri-tools-1.26.0-0.x86_64 4/10
Verifying : kubernetes-cni-1.2.0-0.x86_64 5/10
Verifying : kubelet-1.23.6-0.x86_64 6/10
Verifying : libnetfilter_cttimeout-1.0.0-7.el7.x86_64 7/10
Verifying : socat-1.7.3.2-2.el7.x86_64 8/10
Verifying : libnetfilter_cthelper-1.0.0-11.el7.x86_64 9/10
Verifying : kubectl-1.23.6-0.x86_64 10/10
Installed:
kubeadm.x86_64 0:1.23.6-0 kubectl.x86_64 0:1.23.6-0 kubelet.x86_64 0:1.23.6-0
Dependency Installed:
conntrack-tools.x86_64 0:1.4.4-7.el7 cri-tools.x86_64 0:1.26.0-0 kubernetes-cni.x86_64 0:1.2.0-0
libnetfilter_cthelper.x86_64 0:1.0.0-11.el7 libnetfilter_cttimeout.x86_64 0:1.0.0-7.el7 libnetfilter_queue.x86_64 0:1.0.2-2.el7_2
socat.x86_64 0:1.7.3.2-2.el7
Complete!
$ yum install -y --setopt=obsoletes=0 kubeadm-1.23.6 kubelet-1.23.6 kubectl-1.23.6
設定 kubelet 開機自啟
[root@k8s-master yum.repos.d]# systemctl enable kubelet
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.
[root@k8s-master yum.repos.d]# systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: inactive (dead)
Docs: https://kubernetes.io/docs/
$ systemctl enable kubelet $ systemctl status kubelet
叢集 Master 節點初始化
# 此操作只在 Master 節點執行
[root@k8s-master yum.repos.d]# kubeadm init \
> --apiserver-advertise-address=192.168.1.120 \
> --image-repository registry.aliyuncs.com/google_containers \
> --kubernetes-version=v1.23.6 \
> --service-cidr=10.96.0.0/12 \
> --pod-network-cidr=10.244.0.0/16
[init] Using Kubernetes version: v1.23.6
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [k8s-master kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.1.120]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [k8s-master localhost] and IPs [192.168.1.120 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [k8s-master localhost] and IPs [192.168.1.120 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 6.504656 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.23" in namespace kube-system with the configuration for the kubelets in the cluster
NOTE: The "kubelet-config-1.23" naming of the kubelet ConfigMap is deprecated. Once the UnversionedKubeletConfigMap feature gate graduates to Beta the default name will become just "kubelet-config". Kubeadm upgrade will handle this transition transparently.
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node k8s-master as control-plane by adding the labels: [node-role.kubernetes.io/master(deprecated) node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node k8s-master as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: rczo3l.hgi643ox3vzw4ttr
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.1.120:6443 --token rczo3l.hgi643ox3vzw4ttr \
--discovery-token-ca-cert-hash sha256:714d757f758bbf794c88d48078fcceaca6993a71c32a2e0f131f17ded0099f75
注意結合自己的實際情況，修改對應的引數配置：
$ kubeadm init \ --apiserver-advertise-address=192.168.1.120 \ --image-repository registry.aliyuncs.com/google_containers \ --kubernetes-version=v1.23.6 \ --service-cidr=10.96.0.0/12 \ --pod-network-cidr=10.244.0.0/16
apiserver-advertise-address：指定 Api Server 地址。image-repository：映象倉庫地址。kubernetes-version：Kubernetes 版本。service-cidr：Service 的網路 IP 地址段。pod-network-cidr：Pod 的網路 IP 地址段。
當看到Your Kubernetes control-plane has initialized successfully!提示，說明叢集 Master 節點初始化成功，按照提示，依次執行以下命令：
[root@k8s-master ~]# mkdir -p $HOME/.kube
[root@k8s-master ~]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@k8s-master ~]# sudo chown $(id -u):$(id -g) $HOME/.kube/config
Master 節點初始化成功後，可以檢視啟動的 Docker 容器：
[root@k8s-master ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
516411b3e4f9 df7b72818ad2 "kube-controller-man…" About an hour ago Up About an hour k8s_kube-controller-manager_kube-controller-manager-k8s-master_kube-system_cf39bc9bfe4da000c1780f37274acf68_3
2ed3b58d0c11 595f327f224a "kube-scheduler --au…" About an hour ago Up About an hour k8s_kube-scheduler_kube-scheduler-k8s-master_kube-system_ad30b41672979e80f74f72181c1c9762_3
85de5232c506 4c0375452406 "/usr/local/bin/kube…" 2 hours ago Up 2 hours k8s_kube-proxy_kube-proxy-m2nbx_kube-system_199d9d71-dd85-4363-8c46-addd357aac3b_1
9394d7715a75 registry.aliyuncs.com/google_containers/pause:3.6 "/pause" 2 hours ago Up 2 hours k8s_POD_kube-proxy-m2nbx_kube-system_199d9d71-dd85-4363-8c46-addd357aac3b_1
a4c2a7a80eb7 8fa62c12256d "kube-apiserver --ad…" 2 hours ago Up 2 hours k8s_kube-apiserver_kube-apiserver-k8s-master_kube-system_f844c52da54beeb845dab75e338dce7a_1
82c2e3320e5f 25f8c7f3da61 "etcd --advertise-cl…" 2 hours ago Up 2 hours k8s_etcd_etcd-k8s-master_kube-system_2f6b828b40e9edd50c7cb676b0b4bacf_1
84f3a53a6139 registry.aliyuncs.com/google_containers/pause:3.6 "/pause" 2 hours ago Up 2 hours k8s_POD_kube-scheduler-k8s-master_kube-system_ad30b41672979e80f74f72181c1c9762_1
ca132ea49edc registry.aliyuncs.com/google_containers/pause:3.6 "/pause" 2 hours ago Up 2 hours k8s_POD_kube-apiserver-k8s-master_kube-system_f844c52da54beeb845dab75e338dce7a_1
848d64a5403d registry.aliyuncs.com/google_containers/pause:3.6 "/pause" 2 hours ago Up 2 hours k8s_POD_kube-controller-manager-k8s-master_kube-system_cf39bc9bfe4da000c1780f37274acf68_1
435f1caea216 registry.aliyuncs.com/google_containers/pause:3.6 "/pause" 2 hours ago Up 2 hours k8s_POD_etcd-k8s-master_kube-system_2f6b828b40e9edd50c7cb676b0b4bacf_1
叢集 Node 節點 join
在兩個 Node 節點，執行以下命令：
[root@k8s-node1 ~]# kubeadm join 192.168.1.120:6443 --token y4a0gj.5iaxcci7uqkdjcf0 --discovery-token-ca-cert-hash sha256:714d757f758bbf794c88d48078fcceaca6993a71c32a2e0f131f17ded0099f75
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
-
--toke：Master 節點初次 init 時，在 init 成功後，會在最後輸出 Node 節點 join 的命令。在後續操作時，如果沒有 token，可以透過以下方式獲取：# 檢視沒有過期的 token 列表 [root@k8s-master ~]# kubeadm token list # 重新申請 token [root@k8s-master ~]# kubeadm token create y4a0gj.5iaxcci7uqkdjcf0 [root@k8s-master ~]# kubeadm token list TOKEN TTL EXPIRES USAGES DESCRIPTION EXTRA GROUPS y4a0gj.5iaxcci7uqkdjcf0 23h 2024-09-05T15:40:56Z authentication,signing <none> system:bootstrappers:kubeadm:default-node-token -
--discovery-token-ca-cert-hash：SSL 證書對應的 Hash 值。同樣的，Master 節點初次 init 時，也會輸出，如果後續沒有了，透過以下命令獲取，再將獲得的值，拼接上 "sha256:"，即可獲得 Hash 值為 "sha256:714d757f758bbf794c88d48078fcceaca6993a71c32a2e0f131f17ded0099f75"。$ [root@k8s-master ~]# openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl sha256 -hex | sed 's/^.* //' 714d757f758bbf794c88d48078fcceaca6993a71c32a2e0f131f17ded0099f75openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt：從 Kubernetes 的 CA 證書（位於/etc/kubernetes/pki/ca.crt）中提取公鑰。| openssl rsa -pubin -outform der 2>/dev/null：將公鑰轉換為 DER 格式。2>/dev/null 用於重定向錯誤輸出，以保持命令列整潔。| openssl sha256 -hex：對 DER 格式的公鑰進行 SHA-256 雜湊計算，並以十六進位制形式輸出。| sed 's/^.* //'：使用 sed 工具刪除輸出中的任何前導文字，只留下雜湊值。
如果是 Ubuntu 系統，使用以下命令：
$ openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
此時，可以檢視 Node 節點上啟動的容器：
[root@k8s-node1 yum.repos.d]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6caa9ae8a696 registry.aliyuncs.com/google_containers/kube-proxy "/usr/local/bin/kube…" 11 minutes ago Up 11 minutes k8s_kube-proxy_kube-proxy-6xntx_kube-system_a0a43318-a317-46a8-9a01-a510218b93f7_0
58126cacdd00 registry.aliyuncs.com/google_containers/pause:3.6 "/pause" 11 minutes ago Up 11 minutes k8s_POD_kube-proxy-6xntx_kube-system_a0a43318-a317-46a8-9a01-a510218b93f7_0
同時，在 Master 節點上，檢視叢集的狀態，仍處於 NotReady 的狀態：
[root@k8s-master ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master NotReady control-plane,master 6d23h v1.23.6
k8s-node1 NotReady <none> 56s v1.23.6
k8s-node2 NotReady <none> 53s v1.23.6
叢集 Master 節點安裝網路外掛
在 Node 節點 join 之後，執行以下命令檢視 Component 和 Pod 的狀態：
[root@k8s-master ~]# kubectl get componentstatus
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME STATUS MESSAGE ERROR
controller-manager Healthy ok
scheduler Healthy ok
etcd-0 Healthy {"health":"true","reason":""}
[root@k8s-master ~]# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-6d8c4cb4d-7m22g 0/1 Pending 0 7d23h
coredns-6d8c4cb4d-q22dm 0/1 Pending 0 7d23h
etcd-k8s-master 1/1 Running 4 (23h ago) 7d23h
kube-apiserver-k8s-master 1/1 Running 4 (23h ago) 7d23h
kube-controller-manager-k8s-master 1/1 Running 7 (105m ago) 7d23h
kube-proxy-6xntx 1/1 Running 1 (23h ago) 23h
kube-proxy-m2nbx 1/1 Running 5 (105m ago) 7d23h
kube-proxy-vzsp7 1/1 Running 1 (23h ago) 23h
kube-scheduler-k8s-master 1/1 Running 6 (23h ago) 7d23h
可以看到，Component 的狀態都是 Healthy，說明叢集是正常的，但是 "coredns-6d8c4cb4d-7m22g" 和 "coredns-6d8c4cb4d-q22dm" 都是 Pending 狀態，造成這種現象的原因，就是因為網路。因此，下面在 Master 節點上安裝網路外掛。
$ wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
如果無法透過 wget 下載，可以直接訪問 https://github.com/flannel-io/flannel/tree/master/Documentation/kube-flannel.yml，複製檔案內容，然後在 /opt/k8s 路徑下，建立 kube-flannel.yml 檔案，再將複製的內容貼上。
[root@k8s-master k8s]# pwd
/opt/k8s
[root@k8s-master k8s]# ls
kube-flannel.yml
# 需要下載的映象
[root@k8s-master k8s]# grep image kube-flannel.yml
image: docker.io/flannel/flannel-cni-plugin:v1.5.1-flannel2
image: docker.io/flannel/flannel:v0.25.6
image: docker.io/flannel/flannel:v0.25.6
# 將 docker.io 替換為空，否則到官方下載映象，可能很慢
[root@k8s-master k8s]# sed -i 's#docker.io/##g' kube-flannel.yml
[root@k8s-master k8s]# grep image kube-flannel.yml
image: flannel/flannel-cni-plugin:v1.5.1-flannel2
image: flannel/flannel:v0.25.6
image: flannel/flannel:v0.25.6
# 應用網路外掛
[root@k8s-master k8s]# kubectl apply -f kube-flannel.yml
namespace/kube-flannel created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created
等待一段時間，再次檢視 Pod 的狀態，可以看到，"coredns-6d8c4cb4d-7m22g" 和 "coredns-6d8c4cb4d-q22dm" 都是 Running 的狀態：
[root@k8s-master ~]# kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-6d8c4cb4d-7m22g 1/1 Running 0 7d23h
coredns-6d8c4cb4d-q22dm 1/1 Running 0 7d23h
etcd-k8s-master 1/1 Running 4 (23h ago) 7d23h
kube-apiserver-k8s-master 1/1 Running 4 (23h ago) 7d23h
kube-controller-manager-k8s-master 1/1 Running 7 (129m ago) 7d23h
kube-proxy-6xntx 1/1 Running 1 (23h ago) 23h
kube-proxy-m2nbx 1/1 Running 5 (129m ago) 7d23h
kube-proxy-vzsp7 1/1 Running 1 (23h ago) 23h
kube-scheduler-k8s-master 1/1 Running 6 (23h ago) 7d23h
如果需要檢視某個 Pod 的詳細資訊，可以使用以下命令：
$ kubectl describe pod coredns-6d8c4cb4d-7m22g -n kube-system
此處使用的是 Flannel 網路外掛，也可以使用 CNI 網路外掛，訪問 https://calico-v3-25.netlify.app/archive/v3.25/manifests/calico.yaml，複製檔案內容，然後在 /opt/k8s 路徑下，建立 calico.yaml，再將複製的內容貼上。
修改 CALICO_IPV4POOL_CIDR 配置，該配置預設是註釋掉的，如果不是註釋掉的，將其修改為與 Master 節點 init 時的 "--pod-network-cidr=10.244.0.0/16" 保持一致。
將檔案中需要下載的映象前面的 docker.io 替換為空。
應用 CNI 外掛，使用命令
kubectl apply -f calico.yaml。
檢視叢集狀態
使用以下命令，檢視叢集狀態，當 STATUS 全部為 Ready 時，才可繼續後面的操作（此過程可能需耗費較長時間）：
[root@k8s-master ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master Ready control-plane,master 7d23h v1.23.6
k8s-node1 Ready <none> 24h v1.23.6
k8s-node2 Ready <none> 24h v1.23.6
如果 Flannel 需檢查網路情況，重新進行如下操作：
kubectl delete -f kube-flannel.yml---> 再次下載檔案 --->kubectl apply -f kube-flannel.yml。
測試 Kubenetes 叢集
# 建立部署一個 Nginx 服務
[root@k8s-master ~]# kubectl create deployment nginx --image=nginx
deployment.apps/nginx created
# 暴露容器內的埠
[root@k8s-master ~]# kubectl expose deployment nginx --port=80 --type=NodePort
service/nginx exposed
# 檢視 Pod 以及服務資訊，容器內埠 80，對應宿主機的埠 31173
[root@k8s-master ~]# kubectl get pod,svc
NAME READY STATUS RESTARTS AGE
pod/nginx-85b98978db-j4sjq 1/1 Running 0 10m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 8d
service/nginx NodePort 10.98.189.130 <none> 80:31173/TCP 10m
檢視 Nginx 服務：
[root@k8s-master ~]# curl 192.168.1.120:31173
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
- "curl 192.168.1.121:31173" 和 "curl 192.168.1.122:31173" 具有相同的效果。
頁面訪問 192.168.1.120:31173：
- 訪問 192.168.1.121:31173 和 192.168.1.122:31173 具有相同的效果。
命令列工具 kubectl
kubectl 是使用 Kubernetes API 與 Kubernetes 叢集進行通訊的命令列工具。
kubectl 工具，預設只在 Master 節點上有效，在 Node 節點上執行無效：
[root@k8s-node1 ~]# kubectl get nodes
The connection to the server localhost:8080 was refused - did you specify the right host or port?
其原因是，在 Master 節點上，有使用者認證相關的設定，並且 kubectl 工具呼叫時，知道呼叫的 Api 的伺服器地址（https://192.168.1.120:6443）：
[root@k8s-master .kube]# pwd
/root/.kube
[root@k8s-master .kube]# cat config
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMvakNDQWVhZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJME1EZ3lPREUyTVRReE5Gb1hEVE0wTURneU5qRTJNVFF4TkZvd0ZURVRNQkVHQTFVRQpBeE1LY99886776SnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBS01rCjlmMWRYY2k2TUc5ZkpBUVN2Y1JqM2QwaWk1b0NockcyNGxUSmJha3c3dzV4TVhmUU9qM21FdUFROVkvYkpacnYKaytSUGVvSVlsdUYzbytPbHdWSktRLzBvbnZVbXM2YTFCMi9MRmVLNTRmUGllcXVmSVZDMElGalhvQVBiRTZOZQphc1lRbjV0SDQ2MDV3TjA3OFNHVzk4L1JuVjlkYTlDenEvb1BTYmNpUTRnbU1UYm0rSzU2eTJ0MWdOM0krUW91ClVHaFlucWZJa0VZTzNsTm1VbWtpOFUzaWNHaXhHWkVRYjA0UVowQ0JEck14MGV6anRWS1hrbVFMWW9zVjlhOFcKM1Bkb3AySXlqdkVCVTZxR001aE9BVlA3QUcrQ1hDSTVCVDBxVm1aeXVzYXVuekVnaTVOcWhKZ1hQZGdMWXNpUQprY2Rzc1lyWkFsRnhoenFTdGhzQ0F3RUFBYU5aTUZjd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZQNlQ4aE5aTnMrcjUzWm1Cd3Q0UURvOVJIa0ZNQlVHQTFVZEVRUU8KTUF5Q0NtdDFZbVZ5Ym1WMFpYTXdEUVlKS29aSWh2Y05BUUVMQlFBRGdnRUJBQWg0QWZKTnU1YWxLbVJrSmJCOAppZ2wzZm53YVNkTXZCVUFFaE5JYWt0NytTWGZKdGNnLy9zanpPYlhIQVVDcHgrbnNualhVUW40Uzl1OW9IQlBOCnFUbGxSelBDZ0U4MXRRMnBVekU0T3BGSDQwblRucjYzRWtsbGNoZFU0QzlyMXcwbDJLS296UmJaV3ZCajBZa20KNmh3SlpzVzZyMW9iUjI2VzlFcXpLUXFweWRVcHZFR3lUcDk4YXBsTXFKazNlUS9hYkVOS1k3TW0vNll2K3FicwpCc1dSTVJuMEhzbTZ0dWNlZ21GOFNOZUxvSU5SdExSc1Y4VWZTU0F6WmJTSTVjY1VSTDFycGp2c0NBZUFkWGdECkh4a2l1Ny9IRW5wNmpoM05mRkZKZ0FIL2hvdW52RS9hODk3VmduTHBpcE8xZjBFKzR4YUFMNC9tVWozVm9DRTQKUVRBPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
server: https://192.168.1.120:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: kubernetes-admin
name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
user:
client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURJVENDQWdtZ0F3SUJBZ0lJZWpxb29NcVhGS3d3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TkRBNE1qZ3hOakUwTVRSYUZ3MHlOVEE0TWpneE5qRTBNVFZhTURReApGekFWQmdOVkJBb1REbk41YzNSbGJUcHRZWE4wWlhKek1Sa3dGd1lEVlFRREV4QnJkV0psY201bGRHVnpMV0ZrCmJXbHVNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQXVVbys0dkhnZmNLeGxmdWMKeHkhj56hfh0ZHd1JCU2F6MVRaMmdKV1dUeEtrTE5XemtET3BjWk5oUkdlVGozaVlvdjAvdWxkZGJzdml2LwpSQ2tjUTZrWGdVaXdGSytNUkdKR3dKbmFCY0QwSFdzc0tqMUh6RjVYMm5kbUFvNXhsTThQcms0TWYyREtnbjBHClRiaG5uc1ZERk9tWG5JSTRucFJFMXA4SFIrQUQvTkdDOXhIbVRkV0szSVZOWERvYXhIZXY3VmoxVHk1TkNUWEwKWmQraEJEa0h1NHRwTzM0bGpuWlU4elBua0ljcHJMeU5wdjNCb2UvUjR1WXZIdkFrUG1qYXRUWHMvaGc5QXVSegpKcDcrODFiR3Voc3A2cVZBbzlwOUJsNHovck1XUk9qR3JxdDNEWnFKU2tVTVpqVWdaZHFyY0kzNHJiTWJySlMwCmtqdUpHd0lEQVFBQm8xWXdWREFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUgKQXdJd0RBWURWUjBUQVFIL0JBSXdBREFmQmdOVkhTTUVHREFXZ0JUK2svSVRXVGJQcStkMlpnY0xlRUE2UFVSNQpCVEFOQmdrcWhraUc5dzBCQVFzRkFBT0NBUUVBbnNMVzNpaU9XMTBFbEtVVXlaM3NIRkpzeW1oSUxpOVpJdmE2ClRDVmh5S3huZVEyY1daKzRFTWZJT3RJTi92WitMak96WmVYbzMzRDlybUdBOUtSMi9udTJ6b2pmbzZEdnJLL2YKMVVPOGFWNnlHMFBoeFlCT1dXeUhNczFzd1dzNU8yY01PRGR5TU9JcUIzZHJ3eVEzWkZrblgxa3UwZ1NqeVdheApWR2VQN1grSFowZ1RpSUtGWFdaVjdHWCtlSEdQZHBIZVBkR0doMXdGbUJCcTRVOUdoZERoTVBtenNZMkFpMjJrClpZOTFxYmtucGF4blEvNlRBN0lwdkcya0wwT2N6ZFpoQXh0NWZFRWVmTDZ2M1VVM1c4c3lYTnd0M3ROamJlQk4KZTdRSitaQjcwYjdQQi9lSFZQTmlxRThRdVYySzlIQjRUS3ZkdlNJZTZ1VHN6VXJDZ0E9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcFFJQkFBS0NBUUVBdVVvKzR2SGdmY0t4bGZ1Y3hySUhIOEM0SEtGR3dSQlNhejFUWjJnSldXVHhLa0xOCld6a0RPcGNaTmhSR2VUajNpWW92MC91bGRkYnN2aXYvUkNrY1E2a1hnVWl3RksrTVJHSkd3Sm5hQmNEMEhXc3MKS2oxSHpGNVgybmRtQW81eGxNOFByazRNZjJES2duMEdUYmhubnNWREZPbVhuSUk0bnBSRTFwOEhSK0FEL05HQwo5eEhtVGRXSzNJVk5YRG9heEhldjdWajFUeTVOQ1RYTFpkK2hCRGtIdTR0cE8zNGxqblpVOHpQbmtJY3ByTHlOCnB2M0JvZS9SNHVZdkh2QWtQbWphdFRYcy9oZzlBdVJ6SnA3KzgxYkd1aHNwNnFWQW85cDlCbDR6L3JNV1JPakcKcnF0M0RacUpTa1VNWmpVZ1pkcXJjSTM0cmJNYnJKUzBranVKR3dJREFRQUJBb0lCQVFDTkdONitqeFkyYmpZeApVa05XZzJjdFpPSk8ydms0TjZlcmhpMm5CdkJucEppSmFBbGRPQk1mWU1TUUMreUdqenpnL2R2aC96VkdnUDRTCjZ3b2Q2M2hjaGIwaWRDbXg5dVJIaHRiOS82cW95d0NhRG15NVZhVUJHYTZvN0ZkQUJ4eXpCdUtZQjFNNUJJbngKeUNjdXRBZ2tQVzhSMDdmaU5ML004bmRoUUFTWlU4NlpNTUJyUjB6cS9DNy9PS2lITDJZdjI5eng3ek5TSGUxUQozNkwrWmhzVXlsclYvdnMwWFY4cHBQaCsvVEFVREZkeXVXNlJsQ1lNQXF2U2RxVEp4M2dOK3Rva3R3VWF5NVk1ClVFV0srODMwQjYyNE5lRVJqSHBmOFhVYmJ1UFovcXBnbmxLSjBiRWM4RENvb2szQVBBUEtSMkU0eTBlRHBMWFQKV0xhR05wY2hBb0dCQU96RldFWG5OYWVMY2JtZzFuaXFkQms4anBBOWowUVVkR0lBZkpMMnA5cE1RVnFrVHlMdApCaE9MVEY4RUVHS1dsN0ticDhZZXVyM2gycnlmNldVa2svTTRIMWhFbE9FdGp3aFpWa1hGMENPUzFoSjJRSUlmCm10SFFSUENuZFg0TTdmekIvUmptSE9KWVdWM3BOVjZ3ZE5KOS9PSjghfsdfs3sFJ1VE1xMmpYWTRKQW9HQkFNaFcKa3FSOFZVM0J4Mm9oWHFBQTUxb1ZKbVg3L2p2akhhUXJTRHhQOFNJcVVnVjJaQ3EyWDBRTlRLV0lSVDhhcmsrYQpvaGd2ZytnVWlIcjZDQUdYWmRESTluWEFYbG40aGVyaUYzNVJSMkRVbzhndHpIaDFyNUVad2pEOG1DTlFhMkZaCndidWgreGxPVzRSeU1IY2Jkb1QzUDlFTGdrWHRSU1pRa0pyYlk2Y0RBb0dCQU5VUVZYbzZNTWMvcmF4TXR4TkkKMkVicGZxVUFNSThrRlFNbnl2SjVNZDA0cDhzSWR3cEgzeUx4UkYxd2k4b2NHQkNyRDlReVRQdVlaYjA5N2NxTgptdkhRdkN3ek13SmJmQTRZVHBGbERBTW5IS3JxYk94cndtY3lrd2M0dW5zZTZYNTlsdU8wRjZQN3V4Zk9SNitZCi9OZDZkbm1CdXdWeFIzUk1CdHZJV2VUNUFvR0FMMWhHVDVrU2o4MjcwdGtRQThBeTdKY1MvQWNSamhXZWE2M08KNUhJQUNwTDF6MVNyVjJ6Q0Z0TU55aERxVEgrQnNrNVpBRjQ2VGg2TUlvUDBZR3ZuSS9CYVRubW4wcHRwQ3Bsago4L1pCYUNEWWsvWSszRGp6eE5iUmpjSWtNalJQTERLS0ZrMnhpY2w2MTFJbElnRGJnWkR0QS9vMFQxSkRoVXFFCjRoUDIrUUVDZ1lFQWd2U1owOFRrS2VKM1Z1dkdIeU4wWUk3Umx1Q284ZVVVY3IrOEFvR0MrNjRHbmQ1WFc1eGcKZFViQ1lxN3h2OERjN2MzTkt1RVNKSmg4czYyRllXVjRmQkxGckwrYzRreVpxYnRxdDExN3ZnZklvZU1MNkxTdwpINmc2R3Ria01Fand1eldPQUFlSllsNEptMUUzSDB3eGVBZUlWeldoZ2I5NWF4aFZkWlZlT2VZPQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=
如果需要在 Node 節點使用 kubectl 工具，需要進行如下配置：
-
複製 Master 節點中 "/etc/kubernetes/admin.conf" 檔案到對應的 Node 節點伺服器的 "/etc/kubernetes" 目錄中：
[root@k8s-master .kube]# scp /etc/kubernetes/admin.conf root@k8s-node1:/etc/kubernetes/ The authenticity of host 'k8s-node1 (192.168.1.121)' can't be established. ECDSA key fingerprint is SHA256:1wcmZ7fQ/AJ9ak1Hu/qURBJMFWuqvu66TbMV8OUD9us. ECDSA key fingerprint is MD5:ce:5e:b8:01:2d:f6:ee:77:87:33:b4:7d:4b:64:a5:d9. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'k8s-node1,192.168.1.121' (ECDSA) to the list of known hosts. root@k8s-node1's password: admin.conf 100% 5641 4.7MB/s 00:00 -
在對應的 Node 伺服器上配置環境變數：
[root@k8s-node1 ~]# echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> ~/.bash_profile [root@k8s-node1 ~]# source ~/.bash_profile [root@k8s-node1 ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master Ready control-plane,master 11d v1.23.6 k8s-node1 Ready <none> 4d23h v1.23.6 k8s-node2 Ready <none> 4d23h v1.23.6
kubectl 工具的常用的命令和功能如下：
-
更多命令檢視官網：https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands
-
資源型別與別名：
資源型別 縮寫別名 clusterscomponentstatusescsconfigmapscmdaemonsetsdsdeploymentsdeployendpointsepeventevhorizontalpodautoscalershpaingressesingjobslimitrangeslimitsnamespacesnsnetworkpoliciesnodesnostatefulsetspersistentvolumeclaimspvcpersistentvolumespvpodspopodsecuritypoliciespsppodtemplatesreplicasetsrsreplicationcontrollersrcresourcequotasquotacronjobsecretsserviceaccountsaservicessvcstorageclassesthirdpartyresources -
格式化輸出，例如：
$ kubectl get deploy nginx -o yaml
API 概述
官網：https://kubernetes.io/zh-cn/docs/reference/using-api/（https://kubernetes.io/docs/reference/using-api/deprecation-guide/）
REST API 是 Kubernetes 系統的重要部分，元件之間的所有操作和通訊均由 API Server 處理的 REST API 呼叫，大多數情況下，API 定義和實現都符合標準的 HTTP REST 格式，可以透過 kubectl 命令管理工具或其他命令列工具來執行。
不同的 API 版本代表著不同的穩定性和支援級別，下面是每個級別的摘要：
-
Alpha：
- 版本名稱包含
alpha，例如：v1alpha1。 - 內建的 Alpha API 版本預設被禁用，且必須在 kube-apiserver 配置中顯式啟用才能使用。
- 軟體可能會有 Bug，啟用某個特性可能會暴露出 Bug。
- 對某個 Alpha API 特性的支援可能會隨時被刪除，恕不另行通知。
- API 可能在以後的軟體版本中以不相容的方式更改，恕不另行通知。
- 由於缺陷風險增加和缺乏長期支援，建議該軟體僅用於短期測試叢集。
- 版本名稱包含
-
Beta：
-
版本名稱包含
beta，例如：v2beta3。 -
內建的 Beta API 版本預設被禁用，且必須在 kube-apiserver 配置中顯式啟用才能使用（例外情況是 Kubernetes 1.22 之前引入的 Beta 版本的 API，這些 API 預設被啟用）。
-
內建 Beta API 版本從引入到棄用的最長生命週期為 9 個月或 3 個次要版本（以較長者為準），從棄用到移除的最長生命週期為 9 個月或 3 個次要版本（以較長者為準）。
-
軟體被很好的測試過，啟用某個特性被認為是安全的。
-
儘管一些特性會發生細節上的變化，但它們將會被長期支援。
-
在隨後的 Beta 版或 Stable 版中，物件的模式和（或）語義可能以不相容的方式改變。當這種情況發生時，將提供遷移說明。適配後續的 Beta 或 Stable API 版本可能需要編輯或重新建立 API 物件，這可能並不簡單。對於依賴此功能的應用程式，可能需要停機遷移。
-
該版本的軟體不建議生產使用，後續釋出版本可能會有不相容的變動，一旦 Beta API 版本被棄用且不再提供服務，則使用 Beta API 版本的使用者需要轉為使用後續的 Beta 或 Stable API 版本。
-
請嘗試 Beta 版時特性時並提供反饋。特性完成 Beta 階段測試後，就可能不會有太多的變更了。
-
-
Stable：
- 版本名稱如
vX，其中 X 為整數。 - 特性的 Stable 版本會出現在後續很多版本的釋出軟體中。Stable API 版本仍然適用於 Kubernetes 主要版本範圍內的所有後續釋出，並且 Kubernetes 的主要版本當前沒有移除 Stable API 的修訂計劃。
- 版本名稱如
已棄用 API 的遷移指南：https://kubernetes.io/zh-cn/docs/reference/using-api/deprecation-guide/
原文連結
https://github.com/ACatSmiling/zero-to-zero/blob/main/Operation/kubernetes.md