centos k3s部署

呂林光發表於2021-12-12

一、k3s介紹

1、k3s是一個輕量級的 Kubernetes 發行版，它針對邊緣計算、物聯網等場景進行了高度優化。我們希望安裝的 Kubernetes 在記憶體佔用方面只是一半的大小。Kubernetes 是一個 10 個字母的單詞，簡寫為 k8s。所以，有 Kubernetes 一半大的東西就是一個 5 個字母的單詞，簡寫為 k3s。k3s 沒有全稱，也沒有官方的發音。

2、文件/下載地址

官網下載地址：https://github.com/k3s-io/k3s/releases
國內下載地址：http://mirror.cnrancher.com
k3s文件：http://docs.rancher.cn/docs/k3s/_index
k8s文件：https://www.kubernetes.org.cn/docs

3、環境規劃

本例使用的是指定版本和docker容器方式安裝
docker的安裝及使用：https://www.cnblogs.com/lvlinguang/p/15183571.html
伺服器列表如下

叢集角色	ip地址
server節點	192.168.3.17
agent節點	192.168.3.18

關閉防火牆

# 關閉防火牆
systemctl stop firewalld

# 開機禁用
systemctl disable firewalld

# 檢查狀態
firewall-cmd --state

4、注意：每臺計算機必須具有唯一的主機名。如果您的計算機沒有唯一的主機名，請傳遞K3S_NODE_NAME環境變數，併為每個節點提供一個有效且唯一的主機名。

二、線上安裝

1、server節點安裝

執行指令碼

export INSTALL_K3S_VERSION=v1.21.5+k3s2
export K3S_NODE_NAME=k3s-master
export INSTALL_K3S_EXEC="--docker --write-kubeconfig ~/.kube/config --cluster-cidr 10.72.0.0/16 --service-cidr 10.73.0.0/16 --write-kubeconfig-mode 666"

# 高可用（配置外部sql，第二個server節點需要配置K3S_TOKEN）
# export K3S_TOKEN=server1的token
# export INSTALL_K3S_EXEC="--docker --datastore-endpoint=mysql://root:123456@tcp(192.168.3.13:6008)/k3s --write-kubeconfig ~/.kube/config --cluster-cidr 10.72.0.0/16 --service-cidr 10.73.0.0/16 --write-kubeconfig-mode 666"

curl -sfL http://rancher-mirror.cnrancher.com/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn sh -

server引數說明：
- --docker：k3s server元件以containerd作為容器執行時。可以順便在k3s server節點上啟動一個Agent節點，Agent節點可以使用Docker作為容器執行時，這樣k3s server節點也可以當做工作節點用。當然也可以不在server節點上啟動Agent節點（新增引數--disable-agent即可）。
- --bind-address：k3s監聽的IP地址，非必選，預設是localhost。
- --cluster-cidr：與Kubernetes一樣，也就是Pod所在網路平面，非必選，預設是10.42.0.0/16。
- --service-cidr：與Kubernetes一樣，服務所在的網路平面，非必選，預設是10.43.0.0/16。
- --kube-apiserver-arg：額外的API Server配置引數，具體可以參考Kuberntes官方網站了解支援的配置選項，非必選。
- --write-kubeconfig：安裝時順便寫一個Kubeconfig檔案，方便使用kubectl工具直接訪問。如果不加此引數，則預設的配置檔案路徑為/etc/rancher/k3s/k3s.yaml，預設只有root使用者能讀。
- --write-kubeconfig-mode：與--write-kubeconfig一起使用，指定Kubeconfig檔案的許可權。
- --node-label：順便給節點打上一個asrole=worker的label，非必選。

2、agent節點安裝

從server節點獲取token

cat /var/lib/rancher/k3s/server/node-token

agent節點執行指令碼

export INSTALL_K3S_VERSION=v1.21.5+k3s2
export K3S_NODE_NAME=k3s-node1
export INSTALL_K3S_EXEC="--docker --write-kubeconfig ~/.kube/config --cluster-cidr 10.72.0.0/16 --service-cidr 10.73.0.0/16 --write-kubeconfig-mode 666"
curl -sfL http://rancher-mirror.cnrancher.com/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn K3S_URL=https://192.168.3.17:6443 K3S_TOKEN=server1的token sh -

三、離線安裝

本例以安裝v1.21.5-k3s2版本
百度網盤離線包：https://pan.baidu.com/s/1V7CvTRuaxuchI3zIEeX2Bw 提取碼：zp9r

1、server節點安裝

下載檔案

cd /usr/local/bin

# 下載k3s-airgap-images-amd64.tar
wget http://rancher-mirror.cnrancher.com/k3s/v1.21.5-k3s2/k3s-airgap-images-amd64.tar

# 下載k3s可執行檔案
wget http://rancher-mirror.cnrancher.com/k3s/v1.21.5-k3s2/k3s

# 下載安裝指令碼
wget http://rancher-mirror.cnrancher.com/k3s/k3s-install.sh

# 設定許可權
chmod +x k3s-install.sh k3s

設定環境變數並安裝

# 跳過二進位制檔案下載
export INSTALL_K3S_SKIP_DOWNLOAD=true

# 修改節點名稱
export K3S_NODE_NAME=k3s-master

# 配置額外引數
export INSTALL_K3S_EXEC="--docker --write-kubeconfig ~/.kube/config --cluster-cidr 10.72.0.0/16 --service-cidr 10.73.0.0/16 --write-kubeconfig-mode 666"

# 高可用（配置外部sql，第二個server節點需要配置K3S_TOKEN）
# export K3S_TOKEN=server1的token
# export INSTALL_K3S_EXEC="--docker --datastore-endpoint=mysql://root:123456@tcp(192.168.3.13:6008)/k3s --write-kubeconfig ~/.kube/config --cluster-cidr 10.72.0.0/16 --service-cidr 10.73.0.0/16 --write-kubeconfig-mode 666"

# 執行安裝指令碼
./k3s-install.sh

2、agent節點安裝

從server節點獲取token

cat /var/lib/rancher/k3s/server/node-token

下載檔案

cd /usr/local/bin

# 下載k3s-airgap-images-amd64.tar
wget http://rancher-mirror.cnrancher.com/k3s/v1.21.5-k3s2/k3s-airgap-images-amd64.tar

# 下載k3s可執行檔案
wget http://rancher-mirror.cnrancher.com/k3s/v1.21.5-k3s2/k3s

# 下載安裝指令碼
wget http://rancher-mirror.cnrancher.com/k3s/k3s-install.sh

# 設定許可權
chmod +x k3s-install.sh k3s

設定環境變數並安裝

# 跳過二進位制檔案下載
export INSTALL_K3S_SKIP_DOWNLOAD=true

# 修改節點名稱
export K3S_NODE_NAME=k3s-node1

# 配置server節點資訊，token來自server節點
export K3S_URL=https://192.168.3.17:6443
export K3S_TOKEN=server1的token

# 配置額外引數
export INSTALL_K3S_EXEC="--docker --write-kubeconfig ~/.kube/config --cluster-cidr 10.72.0.0/16 --service-cidr 10.73.0.0/16 --write-kubeconfig-mode 666"

# 執行安裝指令碼
./k3s-install.sh

四、高可用安裝

1、一個 K3s HA 叢集由以下幾個部分組成：

兩個或多個server 節點，將為 Kubernetes API 提供服務並執行其他 control-plane 服務
零個或多個agent 節點，用於執行您的應用和服務
外部資料儲存 (與單個 k3s server 設定中使用的嵌入式 SQLite 資料儲存相反)
固定的註冊地址，位於 server 節點的前面，以允許 agent 節點向叢集註冊

2、指定mysql資料庫

格式：mysql://username:password@tcp(hostname:3306)/database-name
如果指定的資料庫名稱不存在，k3s server 將嘗試建立它

3、server節點修改

增加引數：--datastore-endpoint

# 配置資料庫資訊
export INSTALL_K3S_EXEC="--docker --datastore-endpoint=mysql://root:123456@tcp(192.168.3.13:6008)/k3s --write-kubeconfig ~/.kube/config --write-kubeconfig-mode 666"

第二個server節點需要指定K3S_TOKEN

export K3S_TOKEN=server1的token

4、agent節點不做任何修改

五、配置k3s映象倉庫

在Docker中可以通過配置registry-mirrors來實現從其他映象倉庫中獲取K3s映象。這樣配置之後，會先從registry-mirrors配置的地址拉取映象，如果獲取不到才會從預設的docker.io獲取映象，從而滿足了我們的需求

修改daemon.json

vim /etc/docker/daemon.json

新增以下配置

{
    "insecure-registries":["192.168.3.12:6007"]
}

重啟docker

systemctl daemon-reload
systemctl restart docker

六、Kubernetes 儀表盤

1、kuboard儀表盤（推薦）

官網地址：https://www.kuboard.cn/
安裝

kubectl apply -f https://kuboard.cn/install-script/kuboard.yaml

獲取token

kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep kuboard-user | awk '{print $1}')

解除安裝

kubectl delete -f https://kuboard.cn/install-script/kuboard.yaml

訪問：ip:32567

2、Kubernetes Dashboard儀表盤

新建 kubernetes-dashboard.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: kubernetes-dashboard

---

apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard

---

apiVersion: v1
kind: ServiceAccount
metadata:
  name: dashboard-admin
  namespace: kubernetes-dashboard

---

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  type: NodePort
  ports:
    - port: 443
      targetPort: 8443
      nodePort: 30002
  selector:
    k8s-app: kubernetes-dashboard

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-certs
  namespace: kubernetes-dashboard
type: Opaque

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-csrf
  namespace: kubernetes-dashboard
type: Opaque
data:
  csrf: ""

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-key-holder
  namespace: kubernetes-dashboard
type: Opaque

---

kind: ConfigMap
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-settings
  namespace: kubernetes-dashboard

---

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs", "kubernetes-dashboard-csrf"]
    verbs: ["get", "update", "delete"]
  - apiGroups: [""]
    resources: ["configmaps"]
    resourceNames: ["kubernetes-dashboard-settings"]
    verbs: ["get", "update"]
  - apiGroups: [""]
    resources: ["services"]
    resourceNames: ["heapster", "dashboard-metrics-scraper"]
    verbs: ["proxy"]
  - apiGroups: [""]
    resources: ["services/proxy"]
    resourceNames: ["heapster", "http:heapster:", "https:heapster:", "dashboard-metrics-scraper", "http:dashboard-metrics-scraper"]
    verbs: ["get"]

---

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
rules:
  - apiGroups: ["metrics.k8s.io"]
    resources: ["pods", "nodes"]
    verbs: ["get", "list", "watch"]

---

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: kubernetes-dashboard
subjects:
  - kind: ServiceAccount
    name: kubernetes-dashboard
    namespace: kubernetes-dashboard

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kubernetes-dashboard
subjects:
  - kind: ServiceAccount
    name: kubernetes-dashboard
    namespace: kubernetes-dashboard

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: dashboard-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: dashboard-admin
    namespace: kubernetes-dashboard

---

kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: kubernetes-dashboard
  template:
    metadata:
      labels:
        k8s-app: kubernetes-dashboard
    spec:
      containers:
        - name: kubernetes-dashboard
          image: kubernetesui/dashboard:v2.4.0
          imagePullPolicy: Always
          ports:
            - containerPort: 8443
              protocol: TCP
          args:
            - --auto-generate-certificates
            - --namespace=kubernetes-dashboard
          volumeMounts:
            - name: kubernetes-dashboard-certs
              mountPath: /certs
            - mountPath: /tmp
              name: tmp-volume
          livenessProbe:
            httpGet:
              scheme: HTTPS
              path: /
              port: 8443
            initialDelaySeconds: 30
            timeoutSeconds: 30
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            runAsUser: 1001
            runAsGroup: 2001
      volumes:
        - name: kubernetes-dashboard-certs
          secret:
            secretName: kubernetes-dashboard-certs
        - name: tmp-volume
          emptyDir: {}
      serviceAccountName: kubernetes-dashboard
      nodeSelector:
        "kubernetes.io/os": linux
      tolerations:
        - key: node-role.kubernetes.io/master
          effect: NoSchedule

---

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: dashboard-metrics-scraper
  name: dashboard-metrics-scraper
  namespace: kubernetes-dashboard
spec:
  ports:
    - port: 8000
      targetPort: 8000
  selector:
    k8s-app: dashboard-metrics-scraper

---

kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    k8s-app: dashboard-metrics-scraper
  name: dashboard-metrics-scraper
  namespace: kubernetes-dashboard
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: dashboard-metrics-scraper
  template:
    metadata:
      labels:
        k8s-app: dashboard-metrics-scraper
    spec:
      securityContext:
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: dashboard-metrics-scraper
          image: kubernetesui/metrics-scraper:v1.0.7
          ports:
            - containerPort: 8000
              protocol: TCP
          livenessProbe:
            httpGet:
              scheme: HTTP
              path: /
              port: 8000
            initialDelaySeconds: 30
            timeoutSeconds: 30
          volumeMounts:
          - mountPath: /tmp
            name: tmp-volume
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            runAsUser: 1001
            runAsGroup: 2001
      serviceAccountName: kubernetes-dashboard
      nodeSelector:
        "kubernetes.io/os": linux
      tolerations:
        - key: node-role.kubernetes.io/master
          effect: NoSchedule
      volumes:
        - name: tmp-volume
          emptyDir: {}

安裝

kubectl apply -f kubernetes-dashboard.yaml

獲取token

kubectl -n kubernetes-dashboard describe secret dashboard-admin-token | grep '^token'

解除安裝

kubectl delete -f kubernetes-dashboard.yaml

訪問：ip:30002

七、常用命令

1、node節點操作

得到節點資訊

kubectl get node
或
kubectl get node -o wide

刪除node節點

kubectl delete node node1

修改節點ROLES

kubectl label node ${node} node-role.kubernetes.io/worker=worker

2、名稱空間操作

檢視所有名稱空間

kubectl get namespace
或
kubectl get ns

建立名稱空間

kubectl create namespace test

刪除名稱空間

kubectl delete namespaces test

3、pod/deployment/service共有（get）

不加引數預設default名稱空間下的內容
指定名稱空間：--namespace default / -n default
所有名稱空間下的：--all-namespaces
詳細資訊：-o wide

4、pod操作

檢視pod

kubectl get pod
或
kubectl get pods

詳情

kubectl describe pod springboot-demo-dept-6568749656-cztl4 --namespace default

刪除pod

kubectl delete pod pod1

5、controller操作

檢視Deployment控制器型別

kubectl get deployment
或
kubectl get deployment.apps
或
kubectl get deploy

檢視ReplicaSet控制器型別

kubectl get replicaset
或
kubectl get replicaset.apps

詳情

kubectl describe deployment springboot-demo-dept --namespace default

刪除Deployment控制器型別

kubectl delete deployment nginx-app

6、service操作

檢視service

kubectl get service
或
kubectl get svc

詳情

kubectl describe service springboot-demo-svc --namespace default

刪除service

kubectl delete service nginx-app1

7、顯示top資源

檢視節點top資源

kubectl top node

檢視pod top資源

kubectl top pod --all-namespaces

8、檢視pod日誌

# 實時日誌
kubectl logs -f pod-name -n pod-namespace
kubectl logs --tail 200 -f pod-name -n pod-namespace

# 前200條日誌
kubectl logs --tail 200 pod-name -n pod-namespace

9、其它操作

檢視k3s資訊或版本號

k3s

檢視kubectl幫助文件

kubectl --help

部署資源清單

kubectl apply -f xxx.yaml

刪除部署清單

kubectl delete -f xxx.yaml

檢視k3s服務狀態

systemctl status k3s

重啟k3s服務

systemctl restart k3s

解除安裝 K3s

# server節點解除安裝
/usr/local/bin/k3s-uninstall.sh

# agent節點解除安裝
/usr/local/bin/k3s-agent-uninstall.sh

八、參考

Apline部署K3s的Agent
2024-06-04
如何使用 K3s 部署 Wordpress
2022-08-19
Ubuntu 22.04單機部署K3s
2024-05-31
Ubuntu
跨雲廠商部署 k3s 叢集
2020-06-19
CentOS 部署 NodeBB
2018-08-06
CentOS
CentOS部署pyspider
2018-08-06
CentOSIDE
從架構到部署，全面瞭解K3s
2020-09-10
架構
centos 部署jar包
2020-10-20
CentOSJAR
vsftpd部署(centos7.9)
2024-09-21
FTPCentOS
centos7 部署 memcache
2020-10-08
CentOS
CentOS中部署NET Core
2020-10-15
CentOS
CentOS7.4 部署 Docker
2018-12-10
CentOSDocker
CentOS 部署 flask專案
2019-03-03
CentOSFlask
Django在centos上部署
2018-04-15
DjangoCentOS
CentOS部署Harbor映象倉庫
2022-10-21
CentOS
CentOS部署L2TP
2024-05-30
CentOS
CentOS 7/8上部署Ceph
2020-08-25
CentOS
sonarqube centos 部署到掃描
2020-10-16
CentOS
Docker方式部署TiDB(CentOS 7.5)
2018-09-07
DockerTiDBCentOS
CentOS7上部署https
2018-08-31
CentOSHTTP
k3s單機版安裝部署附一鍵安裝指令碼
2021-11-10
指令碼
入門實踐丨如何在K3s上部署Web應用程式
2021-06-22
Web
centos7部署django
2024-09-09
CentOSDjango
centos 7 部署k8s
2024-08-21
CentOSK8S
CentOS7離線部署JDK
2024-05-12
CentOSJDK
centos 7安裝部署elasticsearch 7.5.1
2021-05-21
CentOSElasticsearch
CentOS 部署 Laravel 應用記錄
2019-12-26
CentOSLaravel
CentOS7部署ceph
2021-11-30
CentOS
Centos系統部署nginx1.18.0
2022-04-06
CentOSNginx
CentOS7部署Bind
2021-12-15
CentOS
CentOS部署ElasticSearch7.6.1叢集
2020-03-27
CentOSElasticsearch
Laravel部署到阿里雲CentOS/Debian
2019-02-16
Laravel阿里CentOS
瞄準邊緣計算，K3S讓Kubernetes部署變得更加輕量化
2019-02-27
一鍵部署openGauss2.0.1 CentOS 7.6
2024-04-10
CentOS
CentOS7 docker安裝部署教程
2019-11-08
CentOSDocker
雲伺服器centOS如何部署php
2020-05-21
伺服器CentOSPHP
Centos部署Loki日誌聚合系統
2022-03-07
CentOSLoki
CentOS 7 Docker安裝部署Go Web
2020-07-05
CentOSDockerGoWeb