k8s資源物件

1874發表於2023-05-11

  什麼是資源物件?

  所謂資源物件是指在k8s上建立的資源例項;即透過apiserver提供的各資源api介面(可以理解為各種資源模板),使用yaml檔案或者命令列的方式向對應資源api介面傳遞引數賦值例項化的結果;比如我們在k8s上建立一個pod,那麼我們就需要透過給apiserver互動,傳遞建立pod的相關引數,讓apiserver拿著這些引數去例項化一個pod的相關資訊存放在etcd中,然後再由排程器進行排程,由node節點的kubelet執行建立pod;簡單講資源物件就是把k8s之上的api介面進行例項化的結果;

  k8s邏輯執行環境

  提示:k8s執行環境如上,k8s能夠將多個node節點的底層提供的資源(如記憶體,cpu,儲存,網路等)邏輯的整合成一個大的資源池,統一由k8s進行排程編排;使用者只管在k8s上建立各種資源即可,建立完成的資源是透過k8s統一編排排程,使用者無需關注具體資源在那個node上執行,也無需關注node節點資源情況;

  k8s的設計理念——分層架構

  k8s的設計理念——API設計原則

  1、所有API應該是宣告式的;

  2、API物件是彼此互補而且可組合的,即“高內聚,松耦合”;

  3、高層API以操作意圖為基礎設計;

  4、低層API根據高層API的控制需要設計;

  5、儘量避免簡單封裝,不要有在外部API無法顯式知道的內部隱藏的機制;

  6、API操作複雜度與物件數量成正比;

  7、API物件狀態不能依賴於網路連線狀態;

  8、儘量避免讓操作機制依賴於全域性狀態,因為在分散式系統中要保證全域性狀態的同步是非常困難的;

  kubernetes API簡介

  提示:在k8s上api分內建api和自定義api;所謂內建api是指部署好k8s叢集后自帶的api介面;自定義api也稱自定義資源(CRD,Custom Resource Definition),部署好k8s之後,透過安裝其他元件等方式擴充套件出來的api;

  apiserver資源組織邏輯

  提示:apiserver對於不同資源是透過分類,分組,分版本的方式邏輯組織的,如上圖所示;

  k8s內建資源物件簡介

  k8s資源物件操作命令

  資源配置清單必需欄位

  1、apiVersion - 建立該物件所使用的Kubernetes API的版本;

  2、kind - 想要建立的物件的型別;

  3、metadata - 定義識別物件唯一性的資料,包括一個name名稱 、可選的namespace,預設不寫就是default名稱空間;

  4、spec:定義資源物件的詳細規範資訊(統一的label標籤、容器名稱、映象、埠對映等),即使用者期望對應資源處於什麼狀態;

  5、status(Pod建立完成後k8s自動生成status狀態),該欄位資訊由k8s自動維護,使用者無需定義,即對應資源的實際狀態;

  Pod資源物件

  提示:pod是k8s中最小控制單元,一個pod中可以執行一個或多個容器;一個pod的中的容器是一起排程,即排程的最小單位是pod;pod的生命週期是短暫的,不會自愈,是用完就銷燬的實體;一般我們透過Controller來建立和管理pod;使用控制器建立的pod具有自動恢復功能,即pod狀態不滿足使用者期望狀態,對應控制器會透過重啟或重建的方式,讓對應pod狀態和數量始終和使用者定義的期望狀態一致;

  示例:自主式pod配置清單

apiVersion: v1
kind: Pod
metadata:
  name: "pod-demo"
  namespace: default
  labels:
    app: "pod-demo"
spec:
  containers:
  - name: pod-demo
    image: "harbor.ik8s.cc/baseimages/nginx:v1"
    ports:
    - containerPort:  80
      name:  http
    volumeMounts:
    - name: localtime
      mountPath: /etc/localtime
  volumes:
    - name: localtime
      hostPath:
        path: /usr/share/zoneinfo/Asia/Shanghai

  應用配置清單

root@k8s-deploy:/yaml# kubectl get pods 
NAME        READY   STATUS    RESTARTS        AGE
net-test1   1/1     Running   2 (4m35s ago)   7d7h
test        1/1     Running   4 (4m34s ago)   13d
test1       1/1     Running   4 (4m35s ago)   13d
test2       1/1     Running   4 (4m35s ago)   13d
root@k8s-deploy:/yaml# kubectl apply -f pod-demo.yaml
pod/pod-demo created
root@k8s-deploy:/yaml# kubectl get pods 
NAME        READY   STATUS              RESTARTS        AGE
net-test1   1/1     Running             2 (4m47s ago)   7d7h
pod-demo    0/1     ContainerCreating   0               4s
test        1/1     Running             4 (4m46s ago)   13d
test1       1/1     Running             4 (4m47s ago)   13d
test2       1/1     Running             4 (4m47s ago)   13d
root@k8s-deploy:/yaml# kubectl get pods 
NAME        READY   STATUS    RESTARTS        AGE
net-test1   1/1     Running   2 (4m57s ago)   7d7h
pod-demo    1/1     Running   0               14s
test        1/1     Running   4 (4m56s ago)   13d
test1       1/1     Running   4 (4m57s ago)   13d
test2       1/1     Running   4 (4m57s ago)   13d
root@k8s-deploy:/yaml# 

  提示:此pod只是在k8s上執行起來,它沒有控制器的監視,對應pod刪除,故障都不會自動恢復;

  Job控制器,詳細說明請參考https://www.cnblogs.com/qiuhom-1874/p/14157306.html

  job控制器配置清單示例

apiVersion: batch/v1
kind: Job
metadata:
  name: job-demo
  namespace: default
  labels:
    app: job-demo
spec:
  template:
    metadata:
      name: job-demo
      labels:
        app: job-demo
    spec:    
      containers:
      - name: job-demo-container
        image: harbor.ik8s.cc/baseimages/centos7:2023
        command: ["/bin/sh"]
        args: ["-c", "echo data init job at `date +%Y-%m-%d_%H-%M-%S` >> /cache/data.log"]
        volumeMounts:
        - mountPath: /cache
          name: cache-volume
        - name: localtime
          mountPath: /etc/localtime
      volumes:
      - name: cache-volume
        hostPath:
          path: /tmp/jobdata
      - name: localtime
        hostPath:
          path: /usr/share/zoneinfo/Asia/Shanghai
      restartPolicy: Never

  提示:定義job資源必須定義restartPolicy;

  應用清單

root@k8s-deploy:/yaml# kubectl get pods 
NAME        READY   STATUS    RESTARTS      AGE
net-test1   1/1     Running   3 (48m ago)   7d10h
pod-demo    1/1     Running   1 (48m ago)   3h32m
test        1/1     Running   5 (48m ago)   14d
test1       1/1     Running   5 (48m ago)   14d
test2       1/1     Running   5 (48m ago)   14d
root@k8s-deploy:/yaml# kubectl apply -f job-demo.yaml 
job.batch/job-demo created
root@k8s-deploy:/yaml# kubectl get pods -o wide
NAME             READY   STATUS      RESTARTS      AGE     IP               NODE           NOMINATED NODE   READINESS GATES
job-demo-z8gmb   0/1     Completed   0             26s     10.200.211.130   192.168.0.34   <none>           <none>
net-test1        1/1     Running     3 (49m ago)   7d10h   10.200.211.191   192.168.0.34   <none>           <none>
pod-demo         1/1     Running     1 (49m ago)   3h32m   10.200.155.138   192.168.0.36   <none>           <none>
test             1/1     Running     5 (49m ago)   14d     10.200.209.6     192.168.0.35   <none>           <none>
test1            1/1     Running     5 (49m ago)   14d     10.200.209.8     192.168.0.35   <none>           <none>
test2            1/1     Running     5 (49m ago)   14d     10.200.211.177   192.168.0.34   <none>           <none>
root@k8s-deploy:/yaml# 

  驗證:檢視192.168.0.34的/tmp/jobdata目錄下是否有job執行的任務資料?

root@k8s-deploy:/yaml# ssh 192.168.0.34 "ls /tmp/jobdata"
data.log
root@k8s-deploy:/yaml# ssh 192.168.0.34 "cat /tmp/jobdata/data.log"
data init job at 2023-05-06_23-31-32
root@k8s-deploy:/yaml# 

  提示:可以看到對應job所在宿主機的/tmp/jobdata/目錄下有job執行過後的資料,這說明我們定義的job任務順利完成;

  定義並行job

apiVersion: batch/v1
kind: Job
metadata:
  name: job-multi-demo
  namespace: default
  labels:
    app: job-multi-demo
spec:
  completions: 5   
  template:
    metadata:
      name: job-multi-demo
      labels:
        app: job-multi-demo
    spec: 
      containers:
      - name: job-multi-demo-container
        image: harbor.ik8s.cc/baseimages/centos7:2023
        command: ["/bin/sh"]
        args: ["-c", "echo data init job at `date +%Y-%m-%d_%H-%M-%S` >> /cache/data.log"]
        volumeMounts:
        - mountPath: /cache
          name: cache-volume
        - name: localtime
          mountPath: /etc/localtime
      volumes:
      - name: cache-volume
        hostPath:
          path: /tmp/jobdata
      - name: localtime
        hostPath:
          path: /usr/share/zoneinfo/Asia/Shanghai
      restartPolicy: Never

  提示:spec欄位下使用completions來指定執行任務需要的對應pod的數量;

  應用清單

root@k8s-deploy:/yaml# kubectl get pods 
NAME             READY   STATUS      RESTARTS      AGE
job-demo-z8gmb   0/1     Completed   0             24m
net-test1        1/1     Running     3 (73m ago)   7d11h
pod-demo         1/1     Running     1 (73m ago)   3h56m
test             1/1     Running     5 (73m ago)   14d
test1            1/1     Running     5 (73m ago)   14d
test2            1/1     Running     5 (73m ago)   14d
root@k8s-deploy:/yaml# kubectl apply -f job-multi-demo.yaml 
job.batch/job-multi-demo created
root@k8s-deploy:/yaml# kubectl get job
NAME             COMPLETIONS   DURATION   AGE
job-demo         1/1           5s         24m
job-multi-demo   1/5           10s        10s
root@k8s-deploy:/yaml# kubectl get pods -o wide
NAME                   READY   STATUS              RESTARTS      AGE     IP               NODE           NOMINATED NODE   READINESS GATES
job-demo-z8gmb         0/1     Completed           0             24m     10.200.211.130   192.168.0.34   <none>           <none>
job-multi-demo-5vp9w   0/1     Completed           0             12s     10.200.211.144   192.168.0.34   <none>           <none>
job-multi-demo-frstg   0/1     Completed           0             22s     10.200.211.186   192.168.0.34   <none>           <none>
job-multi-demo-gd44s   0/1     Completed           0             17s     10.200.211.184   192.168.0.34   <none>           <none>
job-multi-demo-kfm79   0/1     ContainerCreating   0             2s      <none>           192.168.0.34   <none>           <none>
job-multi-demo-nsmpg   0/1     Completed           0             7s      10.200.211.135   192.168.0.34   <none>           <none>
net-test1              1/1     Running             3 (73m ago)   7d11h   10.200.211.191   192.168.0.34   <none>           <none>
pod-demo               1/1     Running             1 (73m ago)   3h56m   10.200.155.138   192.168.0.36   <none>           <none>
test                   1/1     Running             5 (73m ago)   14d     10.200.209.6     192.168.0.35   <none>           <none>
test1                  1/1     Running             5 (73m ago)   14d     10.200.209.8     192.168.0.35   <none>           <none>
test2                  1/1     Running             5 (73m ago)   14d     10.200.211.177   192.168.0.34   <none>           <none>
root@k8s-deploy:/yaml# kubectl get pods -o wide
NAME                   READY   STATUS      RESTARTS      AGE     IP               NODE           NOMINATED NODE   READINESS GATES
job-demo-z8gmb         0/1     Completed   0             24m     10.200.211.130   192.168.0.34   <none>           <none>
job-multi-demo-5vp9w   0/1     Completed   0             33s     10.200.211.144   192.168.0.34   <none>           <none>
job-multi-demo-frstg   0/1     Completed   0             43s     10.200.211.186   192.168.0.34   <none>           <none>
job-multi-demo-gd44s   0/1     Completed   0             38s     10.200.211.184   192.168.0.34   <none>           <none>
job-multi-demo-kfm79   0/1     Completed   0             23s     10.200.211.140   192.168.0.34   <none>           <none>
job-multi-demo-nsmpg   0/1     Completed   0             28s     10.200.211.135   192.168.0.34   <none>           <none>
net-test1              1/1     Running     3 (73m ago)   7d11h   10.200.211.191   192.168.0.34   <none>           <none>
pod-demo               1/1     Running     1 (73m ago)   3h57m   10.200.155.138   192.168.0.36   <none>           <none>
test                   1/1     Running     5 (73m ago)   14d     10.200.209.6     192.168.0.35   <none>           <none>
test1                  1/1     Running     5 (73m ago)   14d     10.200.209.8     192.168.0.35   <none>           <none>
test2                  1/1     Running     5 (73m ago)   14d     10.200.211.177   192.168.0.34   <none>           <none>
root@k8s-deploy:/yaml# 

  驗證:檢視192.168.0.34的/tmp/jobdata/目錄下是否有job資料產生?

root@k8s-deploy:/yaml# ssh 192.168.0.34 "ls /tmp/jobdata" 
data.log
root@k8s-deploy:/yaml# ssh 192.168.0.34 "cat /tmp/jobdata/data.log"
data init job at 2023-05-06_23-31-32
data init job at 2023-05-06_23-55-44
data init job at 2023-05-06_23-55-49
data init job at 2023-05-06_23-55-54
data init job at 2023-05-06_23-55-59
data init job at 2023-05-06_23-56-04
root@k8s-deploy:/yaml# 

  定義並行度

apiVersion: batch/v1
kind: Job
metadata:
  name: job-multi-demo2
  namespace: default
  labels:
    app: job-multi-demo2
spec:
  completions: 6
  parallelism: 2   
  template:
    metadata:
      name: job-multi-demo2
      labels:
        app: job-multi-demo2
    spec: 
      containers:
      - name: job-multi-demo2-container
        image: harbor.ik8s.cc/baseimages/centos7:2023
        command: ["/bin/sh"]
        args: ["-c", "echo data init job at `date +%Y-%m-%d_%H-%M-%S` >> /cache/data.log"]
        volumeMounts:
        - mountPath: /cache
          name: cache-volume
        - name: localtime
          mountPath: /etc/localtime
      volumes:
      - name: cache-volume
        hostPath:
          path: /tmp/jobdata
      - name: localtime
        hostPath:
          path: /usr/share/zoneinfo/Asia/Shanghai
      restartPolicy: Never

  提示:在spec欄位下使用parallelism欄位來指定並行度,即一次幾個pod同時執行;上述清單表示,一次2個pod同時執行,總共需要6個pod;

  應用清單

root@k8s-deploy:/yaml# kubectl get jobs
NAME             COMPLETIONS   DURATION   AGE
job-demo         1/1           5s         34m
job-multi-demo   5/5           25s        9m56s
root@k8s-deploy:/yaml# kubectl apply -f job-multi-demo2.yaml 
job.batch/job-multi-demo2 created
root@k8s-deploy:/yaml# kubectl get jobs
NAME              COMPLETIONS   DURATION   AGE
job-demo          1/1           5s         34m
job-multi-demo    5/5           25s        10m
job-multi-demo2   0/6           2s         3s
root@k8s-deploy:/yaml# kubectl get pods 
NAME                    READY   STATUS      RESTARTS      AGE
job-demo-z8gmb          0/1     Completed   0             34m
job-multi-demo-5vp9w    0/1     Completed   0             10m
job-multi-demo-frstg    0/1     Completed   0             10m
job-multi-demo-gd44s    0/1     Completed   0             10m
job-multi-demo-kfm79    0/1     Completed   0             9m59s
job-multi-demo-nsmpg    0/1     Completed   0             10m
job-multi-demo2-7ppxc   0/1     Completed   0             10s
job-multi-demo2-mxbtq   0/1     Completed   0             5s
job-multi-demo2-rhgh7   0/1     Completed   0             4s
job-multi-demo2-th6ff   0/1     Completed   0             11s
net-test1               1/1     Running     3 (83m ago)   7d11h
pod-demo                1/1     Running     1 (83m ago)   4h6m
test                    1/1     Running     5 (83m ago)   14d
test1                   1/1     Running     5 (83m ago)   14d
test2                   1/1     Running     5 (83m ago)   14d
root@k8s-deploy:/yaml# kubectl get pods 
NAME                    READY   STATUS      RESTARTS      AGE
job-demo-z8gmb          0/1     Completed   0             34m
job-multi-demo-5vp9w    0/1     Completed   0             10m
job-multi-demo-frstg    0/1     Completed   0             10m
job-multi-demo-gd44s    0/1     Completed   0             10m
job-multi-demo-kfm79    0/1     Completed   0             10m
job-multi-demo-nsmpg    0/1     Completed   0             10m
job-multi-demo2-7ppxc   0/1     Completed   0             16s
job-multi-demo2-8bh22   0/1     Completed   0             6s
job-multi-demo2-dbjqw   0/1     Completed   0             6s
job-multi-demo2-mxbtq   0/1     Completed   0             11s
job-multi-demo2-rhgh7   0/1     Completed   0             10s
job-multi-demo2-th6ff   0/1     Completed   0             17s
net-test1               1/1     Running     3 (83m ago)   7d11h
pod-demo                1/1     Running     1 (83m ago)   4h6m
test                    1/1     Running     5 (83m ago)   14d
test1                   1/1     Running     5 (83m ago)   14d
test2                   1/1     Running     5 (83m ago)   14d
root@k8s-deploy:/yaml# kubectl get pods -o wide
NAME                    READY   STATUS      RESTARTS      AGE     IP               NODE           NOMINATED NODE   READINESS GATES
job-demo-z8gmb          0/1     Completed   0             35m     10.200.211.130   192.168.0.34   <none>           <none>
job-multi-demo-5vp9w    0/1     Completed   0             10m     10.200.211.144   192.168.0.34   <none>           <none>
job-multi-demo-frstg    0/1     Completed   0             11m     10.200.211.186   192.168.0.34   <none>           <none>
job-multi-demo-gd44s    0/1     Completed   0             11m     10.200.211.184   192.168.0.34   <none>           <none>
job-multi-demo-kfm79    0/1     Completed   0             10m     10.200.211.140   192.168.0.34   <none>           <none>
job-multi-demo-nsmpg    0/1     Completed   0             10m     10.200.211.135   192.168.0.34   <none>           <none>
job-multi-demo2-7ppxc   0/1     Completed   0             57s     10.200.211.145   192.168.0.34   <none>           <none>
job-multi-demo2-8bh22   0/1     Completed   0             47s     10.200.211.148   192.168.0.34   <none>           <none>
job-multi-demo2-dbjqw   0/1     Completed   0             47s     10.200.211.141   192.168.0.34   <none>           <none>
job-multi-demo2-mxbtq   0/1     Completed   0             52s     10.200.211.152   192.168.0.34   <none>           <none>
job-multi-demo2-rhgh7   0/1     Completed   0             51s     10.200.211.143   192.168.0.34   <none>           <none>
job-multi-demo2-th6ff   0/1     Completed   0             58s     10.200.211.136   192.168.0.34   <none>           <none>
net-test1               1/1     Running     3 (84m ago)   7d11h   10.200.211.191   192.168.0.34   <none>           <none>
pod-demo                1/1     Running     1 (84m ago)   4h7m    10.200.155.138   192.168.0.36   <none>           <none>
test                    1/1     Running     5 (84m ago)   14d     10.200.209.6     192.168.0.35   <none>           <none>
test1                   1/1     Running     5 (84m ago)   14d     10.200.209.8     192.168.0.35   <none>           <none>
test2                   1/1     Running     5 (84m ago)   14d     10.200.211.177   192.168.0.34   <none>           <none>
root@k8s-deploy:/yaml# 

  驗證job資料

  提示:可以看到後面job追加的時間幾乎都是兩個重複的,這說明兩個pod同時執行了job裡的任務;

  Cronjob控制器,詳細說明請參考https://www.cnblogs.com/qiuhom-1874/p/14157306.html

  示例:定義cronjob

apiVersion: batch/v1
kind: CronJob
metadata:
  name: job-cronjob
  namespace: default
spec:
  schedule: "*/1 * * * *"
  jobTemplate:
    spec:
      parallelism: 2
      template:
        spec:
          containers:
          - name: job-cronjob-container
            image: harbor.ik8s.cc/baseimages/centos7:2023
            command: ["/bin/sh"]
            args: ["-c", "echo data init job at `date +%Y-%m-%d_%H-%M-%S` >> /cache/cronjob-data.log"]
            volumeMounts:
            - mountPath: /cache
              name: cache-volume
            - name: localtime
              mountPath: /etc/localtime
          volumes:
          - name: cache-volume
            hostPath:
              path: /tmp/jobdata
          - name: localtime
            hostPath:
              path: /usr/share/zoneinfo/Asia/Shanghai
          restartPolicy: OnFailure

  應用清單

root@k8s-deploy:/yaml# kubectl apply -f cronjob-demo.yaml
cronjob.batch/job-cronjob created
root@k8s-deploy:/yaml# kubectl get cronjob
NAME          SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE
job-cronjob   */1 * * * *   False     0        <none>          6s
root@k8s-deploy:/yaml# kubectl get pods
NAME                         READY   STATUS      RESTARTS       AGE
job-cronjob-28056516-njddz   0/1     Completed   0              12s
job-cronjob-28056516-wgbns   0/1     Completed   0              12s
job-demo-z8gmb               0/1     Completed   0              64m
job-multi-demo-5vp9w         0/1     Completed   0              40m
job-multi-demo-frstg         0/1     Completed   0              40m
job-multi-demo-gd44s         0/1     Completed   0              40m
job-multi-demo-kfm79         0/1     Completed   0              40m
job-multi-demo-nsmpg         0/1     Completed   0              40m
job-multi-demo2-7ppxc        0/1     Completed   0              30m
job-multi-demo2-8bh22        0/1     Completed   0              30m
job-multi-demo2-dbjqw        0/1     Completed   0              30m
job-multi-demo2-mxbtq        0/1     Completed   0              30m
job-multi-demo2-rhgh7        0/1     Completed   0              30m
job-multi-demo2-th6ff        0/1     Completed   0              30m
net-test1                    1/1     Running     3 (113m ago)   7d11h
pod-demo                     1/1     Running     1 (113m ago)   4h36m
test                         1/1     Running     5 (113m ago)   14d
test1                        1/1     Running     5 (113m ago)   14d
test2                        1/1     Running     5 (113m ago)   14d
root@k8s-deploy:/yaml# kubectl get cronjob
NAME          SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE
job-cronjob   */1 * * * *   False     0        12s             108s
root@k8s-deploy:/yaml# kubectl get pods   
NAME                         READY   STATUS      RESTARTS       AGE
job-cronjob-28056516-njddz   0/1     Completed   0              77s
job-cronjob-28056516-wgbns   0/1     Completed   0              77s
job-cronjob-28056517-d6n9h   0/1     Completed   0              17s
job-cronjob-28056517-krsvb   0/1     Completed   0              17s
job-demo-z8gmb               0/1     Completed   0              65m
job-multi-demo-5vp9w         0/1     Completed   0              41m
job-multi-demo-frstg         0/1     Completed   0              41m
job-multi-demo-gd44s         0/1     Completed   0              41m
job-multi-demo-kfm79         0/1     Completed   0              41m
job-multi-demo-nsmpg         0/1     Completed   0              41m
job-multi-demo2-7ppxc        0/1     Completed   0              31m
job-multi-demo2-8bh22        0/1     Completed   0              31m
job-multi-demo2-dbjqw        0/1     Completed   0              31m
job-multi-demo2-mxbtq        0/1     Completed   0              31m
job-multi-demo2-rhgh7        0/1     Completed   0              31m
job-multi-demo2-th6ff        0/1     Completed   0              31m
net-test1                    1/1     Running     3 (114m ago)   7d11h
pod-demo                     1/1     Running     1 (114m ago)   4h38m
test                         1/1     Running     5 (114m ago)   14d
test1                        1/1     Running     5 (114m ago)   14d
test2                        1/1     Running     5 (114m ago)   14d
root@k8s-deploy:/yaml# 

  提示:cronjob 預設保留最近3個歷史記錄;

  驗證:檢視週期執行任務的資料

   提示:從上面的時間就可以看到每過一分鐘就有兩個pod執行一次任務;

  RC/RS 副本控制器

  RC(Replication Controller),副本控制器,該控制器主要負責控制pod副本數量始終滿足使用者期望的副本數量,該副本控制器是第一代pod副本控制器,僅支援selector = !=;

  rc控制器示例

apiVersion: v1  
kind: ReplicationController  
metadata:  
  name: ng-rc
spec:  
  replicas: 2
  selector:  
    app: ng-rc-80 
  template:   
    metadata:  
      labels:  
        app: ng-rc-80
    spec:  
      containers:
      - name: pod-demo
        image: "harbor.ik8s.cc/baseimages/nginx:v1"
        ports:
        - containerPort:  80
          name:  http
        volumeMounts:
        - name: localtime
          mountPath: /etc/localtime
      volumes:
        - name: localtime
          hostPath:
            path: /usr/share/zoneinfo/Asia/Shanghai

  應用配置清單

root@k8s-deploy:/yaml# kubectl get pods 
NAME    READY   STATUS    RESTARTS      AGE
test    1/1     Running   6 (11m ago)   16d
test1   1/1     Running   6 (11m ago)   16d
test2   1/1     Running   6 (11m ago)   16d
root@k8s-deploy:/yaml# kubectl apply -f rc-demo.yaml
replicationcontroller/ng-rc created
root@k8s-deploy:/yaml# kubectl get pods -o wide
NAME          READY   STATUS    RESTARTS      AGE   IP               NODE           NOMINATED NODE   READINESS GATES
ng-rc-l7xmp   1/1     Running   0             10s   10.200.211.136   192.168.0.34   <none>           <none>
ng-rc-wl5d6   1/1     Running   0             9s    10.200.155.185   192.168.0.36   <none>           <none>
test          1/1     Running   6 (11m ago)   16d   10.200.209.24    192.168.0.35   <none>           <none>
test1         1/1     Running   6 (11m ago)   16d   10.200.209.31    192.168.0.35   <none>           <none>
test2         1/1     Running   6 (11m ago)   16d   10.200.211.186   192.168.0.34   <none>           <none>
root@k8s-deploy:/yaml# kubectl get rc
NAME    DESIRED   CURRENT   READY   AGE
ng-rc   2         2         2       25s
root@k8s-deploy:/yaml# 

  驗證:修改pod標籤,看看對應pod是否會重新建立?

root@k8s-deploy:/yaml# kubectl get pods --show-labels
NAME          READY   STATUS    RESTARTS      AGE     LABELS
ng-rc-l7xmp   1/1     Running   0             2m32s   app=ng-rc-80
ng-rc-wl5d6   1/1     Running   0             2m31s   app=ng-rc-80
test          1/1     Running   6 (13m ago)   16d     run=test
test1         1/1     Running   6 (13m ago)   16d     run=test1
test2         1/1     Running   6 (13m ago)   16d     run=test2
root@k8s-deploy:/yaml# kubectl label pod/ng-rc-l7xmp app=nginx-demo --overwrite
pod/ng-rc-l7xmp labeled
root@k8s-deploy:/yaml# kubectl get pods --show-labels                          
NAME          READY   STATUS              RESTARTS      AGE     LABELS
ng-rc-l7xmp   1/1     Running             0             4m42s   app=nginx-demo
ng-rc-rxvd4   0/1     ContainerCreating   0             3s      app=ng-rc-80
ng-rc-wl5d6   1/1     Running             0             4m41s   app=ng-rc-80
test          1/1     Running             6 (15m ago)   16d     run=test
test1         1/1     Running             6 (15m ago)   16d     run=test1
test2         1/1     Running             6 (15m ago)   16d     run=test2
root@k8s-deploy:/yaml# kubectl get pods --show-labels
NAME          READY   STATUS    RESTARTS      AGE     LABELS
ng-rc-l7xmp   1/1     Running   0             4m52s   app=nginx-demo
ng-rc-rxvd4   1/1     Running   0             13s     app=ng-rc-80
ng-rc-wl5d6   1/1     Running   0             4m51s   app=ng-rc-80
test          1/1     Running   6 (16m ago)   16d     run=test
test1         1/1     Running   6 (16m ago)   16d     run=test1
test2         1/1     Running   6 (16m ago)   16d     run=test2
root@k8s-deploy:/yaml# kubectl label pod/ng-rc-l7xmp app=ng-rc-80 --overwrite  
pod/ng-rc-l7xmp labeled
root@k8s-deploy:/yaml# kubectl get pods --show-labels                        
NAME          READY   STATUS    RESTARTS      AGE     LABELS
ng-rc-l7xmp   1/1     Running   0             5m27s   app=ng-rc-80
ng-rc-wl5d6   1/1     Running   0             5m26s   app=ng-rc-80
test          1/1     Running   6 (16m ago)   16d     run=test
test1         1/1     Running   6 (16m ago)   16d     run=test1
test2         1/1     Running   6 (16m ago)   16d     run=test2
root@k8s-deploy:/yaml# 

  提示:rc控制器是透過標籤選擇器來識別對應pod是否歸屬對應rc控制器管控,如果發現對應pod的標籤發生改變,那麼rc控制器會透過新建或刪除的方法將對應pod數量始終和使用者定義的數量保持一致;

  RS(ReplicaSet),副本控制器,該副本控制器和rc類似,都是透過標籤選擇器來匹配歸屬自己管控的pod數量,如果標籤或對應pod數量少於或多餘使用者期望的數量,該控制器會透過新建或刪除pod的方式將對應pod數量始終和使用者期望的pod數量保持一致;rs控制器和rc控制器唯一區別就是rs控制器支援selector = !=精確匹配外,還支援模糊匹配in notin;是k8s之上的第二代pod副本控制器;

  rs控制器示例

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: rs-demo
  labels:
    app: rs-demo
spec:
  replicas: 3
  selector:
    matchLabels:
      app: rs-demo
  template:
    metadata:
      labels:
        app: rs-demo
    spec:
      containers:
      - name: rs-demo
        image: "harbor.ik8s.cc/baseimages/nginx:v1"
        ports:
        - name: web
          containerPort:  80
          protocol: TCP
        env:
        - name: NGX_VERSION
          value: 1.16.1
        volumeMounts:
        - name: localtime
          mountPath: /etc/localtime
      volumes:
        - name: localtime
          hostPath:
            path: /usr/share/zoneinfo/Asia/Shanghai

  應用配置清單

  驗證:修改pod標籤,看看對應pod是否會發生變化?

root@k8s-deploy:/yaml# kubectl get pods --show-labels
NAME            READY   STATUS    RESTARTS      AGE   LABELS
ng-rc-l7xmp     1/1     Running   0             18m   app=ng-rc-80
ng-rc-wl5d6     1/1     Running   0             18m   app=ng-rc-80
rs-demo-nzmqs   1/1     Running   0             71s   app=rs-demo
rs-demo-v2vb6   1/1     Running   0             71s   app=rs-demo
rs-demo-x27fv   1/1     Running   0             71s   app=rs-demo
test            1/1     Running   6 (29m ago)   16d   run=test
test1           1/1     Running   6 (29m ago)   16d   run=test1
test2           1/1     Running   6 (29m ago)   16d   run=test2
root@k8s-deploy:/yaml# kubectl label pod/rs-demo-nzmqs app=nginx --overwrite 
pod/rs-demo-nzmqs labeled
root@k8s-deploy:/yaml# kubectl get pods --show-labels                       
NAME            READY   STATUS    RESTARTS      AGE    LABELS
ng-rc-l7xmp     1/1     Running   0             19m    app=ng-rc-80
ng-rc-wl5d6     1/1     Running   0             19m    app=ng-rc-80
rs-demo-bdfdd   1/1     Running   0             4s     app=rs-demo
rs-demo-nzmqs   1/1     Running   0             103s   app=nginx
rs-demo-v2vb6   1/1     Running   0             103s   app=rs-demo
rs-demo-x27fv   1/1     Running   0             103s   app=rs-demo
test            1/1     Running   6 (30m ago)   16d    run=test
test1           1/1     Running   6 (30m ago)   16d    run=test1
test2           1/1     Running   6 (30m ago)   16d    run=test2
root@k8s-deploy:/yaml# kubectl label pod/rs-demo-nzmqs app=rs-demo --overwrite
pod/rs-demo-nzmqs labeled
root@k8s-deploy:/yaml# kubectl get pods --show-labels                         
NAME            READY   STATUS    RESTARTS      AGE    LABELS
ng-rc-l7xmp     1/1     Running   0             19m    app=ng-rc-80
ng-rc-wl5d6     1/1     Running   0             19m    app=ng-rc-80
rs-demo-nzmqs   1/1     Running   0             119s   app=rs-demo
rs-demo-v2vb6   1/1     Running   0             119s   app=rs-demo
rs-demo-x27fv   1/1     Running   0             119s   app=rs-demo
test            1/1     Running   6 (30m ago)   16d    run=test
test1           1/1     Running   6 (30m ago)   16d    run=test1
test2           1/1     Running   6 (30m ago)   16d    run=test2
root@k8s-deploy:/yaml# 

  提示:可以看到當我們修改pod標籤為其他標籤以後,對應rs控制器會新建一個pod,其標籤為app=rs-demo,這是因為當我們修改pod標籤以後,rs控制器發現標籤選擇器匹配的pod數量少於使用者定義的數量,所以rs控制器會新建一個標籤為app=rs-demo的pod;當我們把pod標籤修改為rs-demo時,rs控制器發現對應標籤選擇器匹配pod數量多餘使用者期望的pod數量,此時rs控制器會透過刪除pod方法,讓app=rs-demo標籤的pod和使用者期望的pod數量保持一致;

  Deployment 副本控制器,詳細說明請參考https://www.cnblogs.com/qiuhom-1874/p/14149042.html

  Deployment副本控制器時k8s第三代pod副本控制器,該控制器比rs控制器更高階,除了有rs的功能之外,還有很多高階功能,,比如說最重要的滾動升級、回滾等;

  deploy控制器示例

apiVersion: apps/v1
kind: Deployment
metadata:
  name:  deploy-demo
  namespace: default
  labels:
    app:  deploy-demo
spec:
  selector:
    matchLabels:
      app: deploy-demo
  replicas: 2
  template:
    metadata:
      labels:
        app:  deploy-demo
    spec:
      containers:
      - name:  deploy-demo
        image:  "harbor.ik8s.cc/baseimages/nginx:v1"
        ports:
        - containerPort:  80
          name:  http
        volumeMounts:
        - name: localtime
          mountPath: /etc/localtime
      volumes:
        - name: localtime
          hostPath:
            path: /usr/share/zoneinfo/Asia/Shanghai

  應用配置清單

  提示:deploy控制器是透過建立rs控制器來實現管控對應pod數量;

   透過修改映象版本來更新pod版本

  應用配置清單

  使用命令更新pod版本

   檢視rs更新歷史版本

   檢視更新歷史記錄

  提示:這裡歷史記錄中沒有記錄版本資訊,原因是預設不記錄,需要記錄歷史版本,可以手動使用--record選項來記錄版本資訊;如下所示

  檢視某個歷史版本的詳細資訊

  提示:檢視某個歷史版本的詳細資訊,加上--revision=對應歷史版本的編號即可;

  回滾到上一個版本

  提示:使用kubectl rollout undo 命令可以將對應deploy回滾到上一個版本;

  回滾指定編號的歷史版本

  提示:使用--to-revision選項來指定對應歷史版本編號,即可回滾到對應編號的歷史版本;

  Service資源,詳細說明請參考https://www.cnblogs.com/qiuhom-1874/p/14161950.html

  nodeport型別的service訪問流程

  nodeport型別service主要解決了k8s叢集外部客戶端訪問pod,其流程是外部客戶端訪問k8s叢集任意node節點的對應暴露的埠,被訪問的node或透過本機的iptables或ipvs規則將外部客戶端流量轉發給對應pod之上,從而實現外部客戶端訪問k8s叢集pod的目的;通常使用nodeport型別service為了方便外部客戶端訪問,都會在叢集外部部署一個負載均衡器,即外部客戶端訪問對應負載均衡器的對應埠,透過負載均衡器將外部客戶端流量引入k8s叢集,從而完成對pod的訪問;

  ClusterIP型別svc示例

apiVersion: v1
kind: Service
metadata:
  name: ngx-svc
  namespace: default
spec:
  selector:
    app: deploy-demo
  type: ClusterIP
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 80

  應用配置清單

  提示:可以看到建立clusterip型別service以後,對應svc會有一個clusterip,後端endpoints會透過標籤選擇器去關聯對應pod,即我們訪問對應svc的clusterip,對應流量會被轉發至後端endpoint pod之上進行響應;不過這種clusterip型別svc只能在k8s叢集內部客戶端訪問,叢集外部客戶端是訪問不到的,原因是這個clusterip是k8s內部網路IP地址;

  驗證,訪問10.100.100.23的80埠,看看對應後端nginxpod是否可以正常被訪問呢?

root@k8s-node01:~# curl 10.100.100.23
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
root@k8s-node01:~# 

  nodeport型別service示例

apiVersion: v1
kind: Service
metadata:
  name: ngx-nodeport-svc
  namespace: default
spec:
  selector:
    app: deploy-demo
  type: NodePort
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 80
    nodePort: 30012

  提示:nodeport型別service只需要在clusterip型別的svc之上修改type為NodePort,然後再ports欄位下用nodePort指定對應node埠即可;

  應用配置清單

root@k8s-deploy:/yaml# kubectl apply -f nodeport-svc-demo.yaml
service/ngx-nodeport-svc created
root@k8s-deploy:/yaml# kubectl get svc
NAME               TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
kubernetes         ClusterIP   10.100.0.1       <none>        443/TCP        16d
ngx-nodeport-svc   NodePort    10.100.209.225   <none>        80:30012/TCP   11s
root@k8s-deploy:/yaml# kubectl describe svc ngx-nodeport-svc
Name:                     ngx-nodeport-svc
Namespace:                default
Labels:                   <none>
Annotations:              <none>
Selector:                 app=deploy-demo
Type:                     NodePort
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.100.209.225
IPs:                      10.100.209.225
Port:                     http  80/TCP
TargetPort:               80/TCP
NodePort:                 http  30012/TCP
Endpoints:                10.200.155.178:80,10.200.211.138:80
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>
root@k8s-deploy:/yaml# 

  驗證:訪問k8s叢集任意node的30012埠,看看對應nginxpod是否能夠被訪問到?

root@k8s-deploy:/yaml# curl 192.168.0.34:30012  
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
root@k8s-deploy:/yaml# 

  提示:可以看到k8s外部客戶端訪問k8snode節點的30012埠是能夠正常訪問到nginxpod;當然叢集內部的客戶端是可以透過對應生成的clusterip進行訪問的;

root@k8s-node01:~# curl 10.100.209.225:30012
curl: (7) Failed to connect to 10.100.209.225 port 30012 after 0 ms: Connection refused
root@k8s-node01:~# curl 127.0.0.1:30012     
curl: (7) Failed to connect to 127.0.0.1 port 30012 after 0 ms: Connection refused
root@k8s-node01:~# curl 192.168.0.34:30012
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
root@k8s-node01:~# 

  提示:叢集內部客戶端只能訪問clusterip的80埠,或者訪問node的對外IP的30012埠;

  Volume資源,詳細說明請參考https://www.cnblogs.com/qiuhom-1874/p/14180752.html

  pod掛載nfs的使用

  在nfs伺服器上準備資料目錄

root@harbor:~# cat /etc/exports
# /etc/exports: the access control list for filesystems which may be exported
#               to NFS clients.  See exports(5).
#
# Example for NFSv2 and NFSv3:
# /srv/homes       hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
#
# Example for NFSv4:
# /srv/nfs4        gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homes  gss/krb5i(rw,sync,no_subtree_check)
#
/data/k8sdata/kuboard *(rw,no_root_squash)
/data/volumes *(rw,no_root_squash)
/pod-vol *(rw,no_root_squash)
root@harbor:~# mkdir -p /pod-vol
root@harbor:~# ls /pod-vol -d
/pod-vol
root@harbor:~# exportfs -av
exportfs: /etc/exports [1]: Neither 'subtree_check' or 'no_subtree_check' specified for export "*:/data/k8sdata/kuboard".
  Assuming default behaviour ('no_subtree_check').
  NOTE: this default has changed since nfs-utils version 1.0.x

exportfs: /etc/exports [2]: Neither 'subtree_check' or 'no_subtree_check' specified for export "*:/data/volumes".
  Assuming default behaviour ('no_subtree_check').
  NOTE: this default has changed since nfs-utils version 1.0.x

exportfs: /etc/exports [3]: Neither 'subtree_check' or 'no_subtree_check' specified for export "*:/pod-vol".
  Assuming default behaviour ('no_subtree_check').
  NOTE: this default has changed since nfs-utils version 1.0.x

exporting *:/pod-vol
exporting *:/data/volumes
exporting *:/data/k8sdata/kuboard
root@harbor:~# 

  在pod上掛載nfs目錄

apiVersion: apps/v1
kind: Deployment
metadata:
  name:  ngx-nfs-80
  namespace: default
  labels:
    app:  ngx-nfs-80
spec:
  selector:
    matchLabels:
      app: ngx-nfs-80
  replicas: 1
  template:
    metadata:
      labels:
        app:  ngx-nfs-80
    spec:
      containers:
      - name:  ngx-nfs-80
        image:  "harbor.ik8s.cc/baseimages/nginx:v1"
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
          limits:
            cpu: 100m
            memory: 100Mi
        ports:
        - containerPort:  80
          name:  ngx-nfs-80
        volumeMounts:
        - name: localtime
          mountPath: /etc/localtime
        - name: nfs-vol
          mountPath: /usr/share/nginx/html/
      volumes:
        - name: localtime
          hostPath:
            path: /usr/share/zoneinfo/Asia/Shanghai
        - name: nfs-vol
          nfs:
            server: 192.168.0.42
            path: /pod-vol
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: ngx-nfs-svc
  namespace: default
spec:
  selector:
    app: ngx-nfs-80
  type: NodePort
  ports:
  - name: ngx-nfs-svc
    protocol: TCP
    port: 80
    targetPort: 80
    nodePort: 30013

  應用配置清單

root@k8s-deploy:/yaml# kubectl apply -f nfs-vol.yaml 
deployment.apps/ngx-nfs-80 created
service/ngx-nfs-svc created
root@k8s-deploy:/yaml# kubectl get pods 
NAME                           READY   STATUS    RESTARTS      AGE
deploy-demo-6849bdf444-pvsc9   1/1     Running   1 (57m ago)   46h
deploy-demo-6849bdf444-sg8fz   1/1     Running   1 (57m ago)   46h
ng-rc-l7xmp                    1/1     Running   1 (57m ago)   47h
ng-rc-wl5d6                    1/1     Running   1 (57m ago)   47h
ngx-nfs-80-66c9697cf4-8pm9k    1/1     Running   0             7s
rs-demo-nzmqs                  1/1     Running   1 (57m ago)   47h
rs-demo-v2vb6                  1/1     Running   1 (57m ago)   47h
rs-demo-x27fv                  1/1     Running   1 (57m ago)   47h
test                           1/1     Running   7 (57m ago)   17d
test1                          1/1     Running   7 (57m ago)   17d
test2                          1/1     Running   7 (57m ago)   17d
root@k8s-deploy:/yaml# kubectl get svc
NAME               TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
kubernetes         ClusterIP   10.100.0.1       <none>        443/TCP        18d
ngx-nfs-svc        NodePort    10.100.16.14     <none>        80:30013/TCP   15s
ngx-nodeport-svc   NodePort    10.100.209.225   <none>        80:30012/TCP   45h
root@k8s-deploy:/yaml# 

  在nfs伺服器上/pod-vol目錄下提供index.html檔案

root@harbor:~# echo "this page from nfs server.." >> /pod-vol/index.html
root@harbor:~# cat /pod-vol/index.html
this page from nfs server..
root@harbor:~# 

  訪問pod,看看nfs伺服器上的inde.html是否能夠正常訪問到?

root@k8s-deploy:/yaml# curl 192.168.0.35:30013
this page from nfs server..
root@k8s-deploy:/yaml# 

  提示:能夠看到訪問pod對應返回的頁面就是剛才在nfs伺服器上建立的頁面,說明pod正常掛載了nfs提供的目錄;

  PV、PVC資源,詳細說明請參考https://www.cnblogs.com/qiuhom-1874/p/14188621.html

  nfs實現靜態pvc的使用

  在nfs伺服器上準備目錄

root@harbor:~# cat /etc/exports 
# /etc/exports: the access control list for filesystems which may be exported
#               to NFS clients.  See exports(5).
#
# Example for NFSv2 and NFSv3:
# /srv/homes       hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
#
# Example for NFSv4:
# /srv/nfs4        gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homes  gss/krb5i(rw,sync,no_subtree_check)
#
/data/k8sdata/kuboard *(rw,no_root_squash)
/data/volumes *(rw,no_root_squash)
/pod-vol *(rw,no_root_squash)
/data/k8sdata/myserver/myappdata *(rw,no_root_squash)
root@harbor:~# mkdir -p /data/k8sdata/myserver/myappdata
root@harbor:~# exportfs -av
exportfs: /etc/exports [1]: Neither 'subtree_check' or 'no_subtree_check' specified for export "*:/data/k8sdata/kuboard".
  Assuming default behaviour ('no_subtree_check').
  NOTE: this default has changed since nfs-utils version 1.0.x

exportfs: /etc/exports [2]: Neither 'subtree_check' or 'no_subtree_check' specified for export "*:/data/volumes".
  Assuming default behaviour ('no_subtree_check').
  NOTE: this default has changed since nfs-utils version 1.0.x

exportfs: /etc/exports [3]: Neither 'subtree_check' or 'no_subtree_check' specified for export "*:/pod-vol".
  Assuming default behaviour ('no_subtree_check').
  NOTE: this default has changed since nfs-utils version 1.0.x

exportfs: /etc/exports [4]: Neither 'subtree_check' or 'no_subtree_check' specified for export "*:/data/k8sdata/myserver/myappdata".
  Assuming default behaviour ('no_subtree_check').
  NOTE: this default has changed since nfs-utils version 1.0.x

exporting *:/data/k8sdata/myserver/myappdata
exporting *:/pod-vol
exporting *:/data/volumes
exporting *:/data/k8sdata/kuboard
root@harbor:~# 

  建立pv

apiVersion: v1
kind: PersistentVolume
metadata:
  name: myapp-static-pv
  namespace: default
spec: capacity: storage: 2Gi accessModes: - ReadWriteOnce nfs: path: /data/k8sdata/myserver/myappdata server: 192.168.0.42

  建立pvc關聯pv

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: myapp-static-pvc
  namespace: default
spec:
  volumeName: myapp-static-pv
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi

  建立pod使用pvc

apiVersion: apps/v1
kind: Deployment
metadata:
  name:  ngx-nfs-pvc-80
  namespace: default
  labels:
    app:  ngx-pvc-80
spec:
  selector:
    matchLabels:
      app: ngx-pvc-80
  replicas: 1
  template:
    metadata:
      labels:
        app:  ngx-pvc-80
    spec:
      containers:
      - name:  ngx-pvc-80
        image:  "harbor.ik8s.cc/baseimages/nginx:v1"
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
          limits:
            cpu: 100m
            memory: 100Mi
        ports:
        - containerPort:  80
          name:  ngx-pvc-80
        volumeMounts:
        - name: localtime
          mountPath: /etc/localtime
        - name: data-pvc
          mountPath: /usr/share/nginx/html/
      volumes:
        - name: localtime
          hostPath:
            path: /usr/share/zoneinfo/Asia/Shanghai
        - name: data-pvc
          persistentVolumeClaim:
            claimName: myapp-static-pvc 

---
apiVersion: v1
kind: Service
metadata:
  name: ngx-pvc-svc
  namespace: default
spec:
  selector:
    app: ngx-pvc-80
  type: NodePort
  ports:
  - name: ngx-nfs-svc
    protocol: TCP
    port: 80
    targetPort: 80
    nodePort: 30014

  應用上述配置清單

root@k8s-deploy:/yaml# kubectl apply -f nfs-static-pvc-demo.yaml 
persistentvolume/myapp-static-pv created
persistentvolumeclaim/myapp-static-pvc created
deployment.apps/ngx-nfs-pvc-80 created
service/ngx-pvc-svc created
root@k8s-deploy:/yaml# kubectl get pv
NAME              CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                      STORAGECLASS   REASON   AGE
myapp-static-pv   2Gi        RWO            Retain           Bound    default/myapp-static-pvc                           4s
root@k8s-deploy:/yaml# kubectl get pvc
NAME               STATUS    VOLUME            CAPACITY   ACCESS MODES   STORAGECLASS   AGE
myapp-static-pvc   Pending   myapp-static-pv   0                                        7s
root@k8s-deploy:/yaml# kubectl get pods 
NAME                            READY   STATUS    RESTARTS       AGE
deploy-demo-6849bdf444-pvsc9    1/1     Running   1 (151m ago)   47h
deploy-demo-6849bdf444-sg8fz    1/1     Running   1 (151m ago)   47h
ng-rc-l7xmp                     1/1     Running   1 (151m ago)   2d1h
ng-rc-wl5d6                     1/1     Running   1 (151m ago)   2d1h
ngx-nfs-pvc-80-f776bb6d-nwwwq   0/1     Pending   0              10s
rs-demo-nzmqs                   1/1     Running   1 (151m ago)   2d
rs-demo-v2vb6                   1/1     Running   1 (151m ago)   2d
rs-demo-x27fv                   1/1     Running   1 (151m ago)   2d
test                            1/1     Running   7 (151m ago)   18d
test1                           1/1     Running   7 (151m ago)   18d
test2                           1/1     Running   7 (151m ago)   18d
root@k8s-deploy:/yaml# 

  在nfs伺服器上/data/k8sdata/myserver/myappdata建立index.html,看看對應主頁是否能夠被訪問?

root@harbor:~# echo "this page from nfs-server /data/k8sdata/myserver/myappdata/index.html" >> /data/k8sdata/myserver/myappdata/index.html
root@harbor:~# cat /data/k8sdata/myserver/myappdata/index.html
this page from nfs-server /data/k8sdata/myserver/myappdata/index.html
root@harbor:~# 

  訪問pod

root@harbor:~# curl 192.168.0.36:30014
this page from nfs-server /data/k8sdata/myserver/myappdata/index.html
root@harbor:~# 

  nfs實現動態pvc的使用

   建立名稱空間、服務賬號、clusterrole、clusterrolebindding、role、rolebinding

apiVersion: v1
kind: Namespace
metadata:
  name: nfs
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: nfs
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: nfs
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: nfs
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: nfs
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: nfs
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

  建立sc

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner # or choose another name, must match deployment's env PROVISIONER_NAME'
reclaimPolicy: Retain #PV的刪除策略,預設為delete,刪除PV後立即刪除NFS server的資料
mountOptions:
  #- vers=4.1 #containerd有部分引數異常
  #- noresvport #告知NFS客戶端在重新建立網路連線時,使用新的傳輸控制協議源埠
  - noatime #訪問檔案時不更新檔案inode中的時間戳,高併發環境可提高效能
parameters:
  #mountOptions: "vers=4.1,noresvport,noatime"
  archiveOnDelete: "true"  #刪除pod時保留pod資料,預設為false時為不保留資料 

  建立provision

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
  labels:
    app: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: nfs
spec:
  replicas: 1
  strategy: #部署策略
    type: Recreate
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          #image: k8s.gcr.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2 
          image: registry.cn-qingdao.aliyuncs.com/zhangshijie/nfs-subdir-external-provisioner:v4.0.2 
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: k8s-sigs.io/nfs-subdir-external-provisioner
            - name: NFS_SERVER
              value: 192.168.0.42
            - name: NFS_PATH
              value: /data/volumes
      volumes:
        - name: nfs-client-root
          nfs:
            server: 192.168.0.42
            path: /data/volumes

  呼叫sc建立pvc

apiVersion: v1
kind: Namespace
metadata:
  name: myserver
---   
# Test PVC
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: myserver-myapp-dynamic-pvc
  namespace: myserver
spec:
  storageClassName: managed-nfs-storage #呼叫的storageclass 名稱
  accessModes:
    - ReadWriteMany #訪問許可權
  resources:
    requests:
      storage: 500Mi #空間大小

  建立app使用pvc

kind: Deployment
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
metadata:
  labels:
    app: myserver-myapp 
  name: myserver-myapp-deployment-name
  namespace: myserver
spec:
  replicas: 1 
  selector:
    matchLabels:
      app: myserver-myapp-frontend
  template:
    metadata:
      labels:
        app: myserver-myapp-frontend
    spec:
      containers:
        - name: myserver-myapp-container
          image: nginx:1.20.0 
          #imagePullPolicy: Always
          volumeMounts:
          - mountPath: "/usr/share/nginx/html/statics"
            name: statics-datadir
      volumes:
        - name: statics-datadir
          persistentVolumeClaim:
            claimName: myserver-myapp-dynamic-pvc 

---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: myserver-myapp-service
  name: myserver-myapp-service-name
  namespace: myserver
spec:
  type: NodePort
  ports:
  - name: http
    port: 80
    targetPort: 80
    nodePort: 30015
  selector:
    app: myserver-myapp-frontend

  應用上述配置清單

root@k8s-deploy:/yaml/myapp# kubectl apply -f .
namespace/nfs created
serviceaccount/nfs-client-provisioner created
clusterrole.rbac.authorization.k8s.io/nfs-client-provisioner-runner created
clusterrolebinding.rbac.authorization.k8s.io/run-nfs-client-provisioner created
role.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner created
rolebinding.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner created
storageclass.storage.k8s.io/managed-nfs-storage created
deployment.apps/nfs-client-provisioner created
namespace/myserver created
persistentvolumeclaim/myserver-myapp-dynamic-pvc created
deployment.apps/myserver-myapp-deployment-name created
service/myserver-myapp-service-name created
root@k8s-deploy:

  驗證:檢視sc、pv、pvc是否建立?pod是否正常執行?

root@k8s-deploy:/yaml/myapp# kubectl get sc
NAME                  PROVISIONER                                   RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
managed-nfs-storage   k8s-sigs.io/nfs-subdir-external-provisioner   Retain          Immediate           false                  105s
root@k8s-deploy:/yaml/myapp# kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                 STORAGECLASS          REASON   AGE
pvc-01709c7f-0cf9-4554-9ae9-72db89e7308c   500Mi      RWX            Retain           Bound    myserver/myserver-myapp-dynamic-pvc   managed-nfs-storage            107s
root@k8s-deploy:/yaml/myapp# kubectl get pvc -n myserver
NAME                         STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS          AGE
myserver-myapp-dynamic-pvc   Bound    pvc-01709c7f-0cf9-4554-9ae9-72db89e7308c   500Mi      RWX            managed-nfs-storage   117s
root@k8s-deploy:/yaml/myapp# kubectl get pods -n myserver
NAME                                              READY   STATUS    RESTARTS   AGE
myserver-myapp-deployment-name-65ff65446f-xpd5p   1/1     Running   0          2m8s
root@k8s-deploy:/yaml/myapp# 

  提示:可以看到pv自動由sc建立,pvc自動和pv關聯;

  驗證:在nfs伺服器上的/data/volumes/下建立index.html檔案,訪問pod service,看看對應檔案是否能夠正常被訪問到?

root@harbor:/data/volumes# ls
myserver-myserver-myapp-dynamic-pvc-pvc-01709c7f-0cf9-4554-9ae9-72db89e7308c
root@harbor:/data/volumes# cd myserver-myserver-myapp-dynamic-pvc-pvc-01709c7f-0cf9-4554-9ae9-72db89e7308c/
root@harbor:/data/volumes/myserver-myserver-myapp-dynamic-pvc-pvc-01709c7f-0cf9-4554-9ae9-72db89e7308c# ls
root@harbor:/data/volumes/myserver-myserver-myapp-dynamic-pvc-pvc-01709c7f-0cf9-4554-9ae9-72db89e7308c# echo "this page from nfs-server /data/volumes" >> index.html
root@harbor:/data/volumes/myserver-myserver-myapp-dynamic-pvc-pvc-01709c7f-0cf9-4554-9ae9-72db89e7308c# cat index.html 
this page from nfs-server /data/volumes
root@harbor:/data/volumes/myserver-myserver-myapp-dynamic-pvc-pvc-01709c7f-0cf9-4554-9ae9-72db89e7308c# 

  提示:在nfs伺服器上的/data/volumes目錄下會自動生成一個使用pvcpod所在名稱空間+pvc名字+pv名字的一個目錄,這個目錄就是由provision建立;

  訪問pod

root@harbor:~# curl 192.168.0.36:30015/statics/index.html
this page from nfs-server /data/volumes
root@harbor:~# 

  提示:能夠訪問到我們剛才建立的檔案,說明pod正常掛載nfs伺服器對應目錄;

  PV/PVC總結

  PV是對底層網路儲存的抽象,即將網路儲存定義為一種儲存資源,將一個整體的儲存資源拆分成多份後給不同的業務使用。

  PVC是對PV資源的申請呼叫,pod是透過PVC將資料儲存至PV,PV再把資料儲存至真正的硬體儲存。

  PersistentVolume引數

  Capacity: #當前PV空間大小,kubectl explain PersistentVolume.spec.capacity

  accessModes :訪問模式,#kubectl explain PersistentVolume.spec.accessModes

    ReadWriteOnce – PV只能被單個節點以讀寫許可權掛載,RWO

    ReadOnlyMany – PV以可以被多個節點掛載但是許可權是隻讀的,ROX
    ReadWriteMany – PV可以被多個節點是讀寫方式掛載使用,RWX

  persistentVolumeReclaimPolicy #刪除機制即刪除儲存卷卷時候,已經建立好的儲存卷由以下刪除操作:

    Retain – 刪除PV後保持原裝,最後需要管理員手動刪除

    Recycle – 空間回收,及刪除儲存捲上的所有資料(包括目錄和隱藏檔案),目前僅支援NFS和hostPath

    Delete – 自動刪除儲存卷

  volumeMode #卷型別,kubectl explain PersistentVolume.spec.volumeMode;定義儲存卷使用的檔案系統是塊裝置還是檔案系統,預設為檔案系統

  mountOptions #附加的掛載選項列表,實現更精細的許可權控制;

  官方文件:持久卷 | Kubernetes

  PersistentVolumeClaim建立引數

  accessModes :PVC 訪問模式,#kubectl explain PersistentVolumeClaim.spec.volumeMode

    ReadWriteOnce – PVC只能被單個節點以讀寫許可權掛載,RWO

    ReadOnlyMany – PVC以可以被多個節點掛載但是許可權是隻讀的,ROX

    ReadWriteMany – PVC可以被多個節點是讀寫方式掛載使用,RWX

  resources: #定義PVC建立儲存卷的空間大小

  selector: #標籤選擇器,選擇要繫結的PV

    matchLabels #匹配標籤名稱

    matchExpressions #基於正規表示式匹配

  volumeName #要繫結的PV名稱

  volumeMode #卷型別,定義PVC使用的檔案系統是塊裝置還是檔案系統,預設為檔案系統

  Volume- 儲存卷型別

  static:靜態儲存卷 ,需要在使用前手動建立PV、然後建立PVC並繫結到PV然後掛載至pod使用,適用於PV和PVC相對比較固定的業務場景。

  dynamin:動態儲存卷,先建立一個儲存類storageclass,後期pod在使用PVC的時候可以透過儲存類動態建立PVC,適用於有狀態服務叢集如MySQL一主多從、zookeeper叢集等。

  儲存類官方文件:儲存類 | Kubernetes

相關文章