Kubernetes(k8s)控制器(一):deployment

人生的哲理發表於2023-02-08

一.系統環境

伺服器版本 docker軟體版本 Kubernetes(k8s)叢集版本 CPU架構
CentOS Linux release 7.4.1708 (Core) Docker version 20.10.12 v1.21.9 x86_64

Kubernetes叢集架構:k8scloude1作為master節點,k8scloude2,k8scloude3作為worker節點

伺服器 作業系統版本 CPU架構 程式 功能描述
k8scloude1/192.168.110.130 CentOS Linux release 7.4.1708 (Core) x86_64 docker,kube-apiserver,etcd,kube-scheduler,kube-controller-manager,kubelet,kube-proxy,coredns,calico k8s master節點
k8scloude2/192.168.110.129 CentOS Linux release 7.4.1708 (Core) x86_64 docker,kubelet,kube-proxy,calico k8s worker節點
k8scloude3/192.168.110.128 CentOS Linux release 7.4.1708 (Core) x86_64 docker,kubelet,kube-proxy,calico k8s worker節點

二.前言

使用deployment的前提是已經有一套可以正常執行的Kubernetes叢集,關於Kubernetes(k8s)叢集的安裝部署,可以檢視部落格《Centos7 安裝部署Kubernetes(k8s)叢集》https://www.cnblogs.com/renshengdezheli/p/16686769.html。

三.Kubernetes 控制器

在 Kubernetes 中,控制器透過監控叢集 的公共狀態,並致力於將當前狀態轉變為期望的狀態。

一個控制器至少追蹤一種型別的 Kubernetes 資源。這些 物件 有一個代表期望狀態的 spec 欄位該資源的控制器負責確保其當前狀態接近期望狀態。

控制器可能會自行執行操作;在 Kubernetes 中更常見的是一個控制器會傳送資訊給 API 伺服器。

Kubernetes 內建一組控制器,執行在 kube-controller-manager 內。 這些內建的控制器提供了重要的核心功能。

Deployment 控制器和 Job 控制器是 Kubernetes 內建控制器的典型例子。 Kubernetes 允許你執行一個穩定的控制平面,這樣即使某些內建控制器失敗了, 控制平面的其他部分會接替它們的工作。

四.Deployment概覽

一個 Deployment 為 Pod 和 ReplicaSet 提供宣告式的更新能力

你負責描述 Deployment 中的 目標狀態,而 Deployment 控制器(Controller) 以受控速率更改實際狀態, 使其變為期望狀態。你可以定義 Deployment 以建立新的 ReplicaSet,或刪除現有 Deployment, 並透過新的 Deployment 收養其資源。

注意:不要管理 Deployment 所擁有的 ReplicaSet 。

以下是 Deployments 的典型用例:

  • 建立 Deployment 以將 ReplicaSet 上線。ReplicaSet 在後臺建立 Pod。 檢查 ReplicaSet 的上線狀態,檢視其是否成功。
  • 透過更新 Deployment 的 PodTemplateSpec,宣告 Pod 的新狀態 。 新的 ReplicaSet 會被建立,Deployment 以受控速率將 Pod 從舊 ReplicaSet 遷移到新 ReplicaSet。 每個新的 ReplicaSet 都會更新 Deployment 的修訂版本。
  • 如果 Deployment 的當前狀態不穩定,回滾到較早的 Deployment 版本。 每次回滾都會更新 Deployment 的修訂版本。
  • 擴大 Deployment 規模以承擔更多負載。
  • 暫停 Deployment 的上線 以應用對 PodTemplateSpec 所作的多項修改, 然後恢復其執行以啟動新的上線版本。
  • 使用 Deployment 狀態來判定上線過程是否出現停滯。
  • 清理較舊的不再需要的 ReplicaSet 。

五.建立deployment

建立存放deployment配置檔案的目錄和namespace

[root@k8scloude1 ~]# mkdir deployment

[root@k8scloude1 ~]# cd deployment/

[root@k8scloude1 deployment]# pwd
/root/deployment

#建立namespace
[root@k8scloude1 deployment]# kubectl create ns deployment
namespace/deployment created

#切換名稱空間為deployment
[root@k8scloude1 deployment]# kubens deployment
Context "kubernetes-admin@kubernetes" modified.
Active namespace is "deployment".

[root@k8scloude1 deployment]# kubectl get pod 
No resources found in deployment namespace.

Kubernetes裡有很多簡寫,比如deployment簡稱deploy,pod簡稱po

檢視pod

[root@k8scloude1 deployment]# kubectl get po
No resources found in deployment namespace.

檢視對資源的操作有哪些

[root@k8scloude1 deployment]# kubectl --help | grep resource
  create        Create a resource from a file or from stdin.
  get           顯示一個或更多 resources
  delete        Delete resources by filenames, stdin, resources and names, or by resources and label selector
  rollout       Manage the rollout of a resource
  describe      顯示一個指定 resource 或者 group 的 resources 詳情
  patch         Update field(s) of a resource
  wait          Experimental: Wait for a specific condition on one or many resources.
  api-resources Print the supported API resources on the server

可以檢視k8s支援的簡寫

[root@k8scloude1 deployment]# kubectl api-resources
NAME                              SHORTNAMES   APIVERSION                             NAMESPACED   KIND
bindings                                       v1                                     true         Binding
componentstatuses                 cs           v1                                     false        ComponentStatus
configmaps                        cm           v1                                     true         ConfigMap
endpoints                         ep           v1                                     true         Endpoints
events                            ev           v1                                     true         Event
limitranges                       limits       v1                                     true         LimitRange
namespaces                        ns           v1                                     false        Namespace
nodes                             no           v1                                     false        Node
persistentvolumeclaims            pvc          v1                                     true         PersistentVolumeClaim
persistentvolumes                 pv           v1                                     false        PersistentVolume
pods                              po           v1                                     true         Pod
podtemplates                                   v1                                     true         PodTemplate
replicationcontrollers            rc           v1                                     true         ReplicationController
resourcequotas                    quota        v1                                     true         ResourceQuota
secrets                                        v1                                     true         Secret
serviceaccounts                   sa           v1                                     true         ServiceAccount
services                          svc          v1                                     true         Service
mutatingwebhookconfigurations                  admissionregistration.k8s.io/v1        false        MutatingWebhookConfiguration
validatingwebhookconfigurations                admissionregistration.k8s.io/v1        false        ValidatingWebhookConfiguration
customresourcedefinitions         crd,crds     apiextensions.k8s.io/v1                false        CustomResourceDefinition
apiservices                                    apiregistration.k8s.io/v1              false        APIService
controllerrevisions                            apps/v1                                true         ControllerRevision
daemonsets                        ds           apps/v1                                true         DaemonSet
deployments                       deploy       apps/v1                                true         Deployment
replicasets                       rs           apps/v1                                true         ReplicaSet
statefulsets                      sts          apps/v1                                true         StatefulSet
tokenreviews                                   authentication.k8s.io/v1               false        TokenReview
localsubjectaccessreviews                      authorization.k8s.io/v1                true         LocalSubjectAccessReview
selfsubjectaccessreviews                       authorization.k8s.io/v1                false        SelfSubjectAccessReview
selfsubjectrulesreviews                        authorization.k8s.io/v1                false        SelfSubjectRulesReview
subjectaccessreviews                           authorization.k8s.io/v1                false        SubjectAccessReview
horizontalpodautoscalers          hpa          autoscaling/v1                         true         HorizontalPodAutoscaler
cronjobs                          cj           batch/v1                               true         CronJob
jobs                                           batch/v1                               true         Job
certificatesigningrequests        csr          certificates.k8s.io/v1                 false        CertificateSigningRequest
leases                                         coordination.k8s.io/v1                 true         Lease
bgpconfigurations                              crd.projectcalico.org/v1               false        BGPConfiguration
bgppeers                                       crd.projectcalico.org/v1               false        BGPPeer
blockaffinities                                crd.projectcalico.org/v1               false        BlockAffinity
caliconodestatuses                             crd.projectcalico.org/v1               false        CalicoNodeStatus
clusterinformations                            crd.projectcalico.org/v1               false        ClusterInformation
felixconfigurations                            crd.projectcalico.org/v1               false        FelixConfiguration
globalnetworkpolicies                          crd.projectcalico.org/v1               false        GlobalNetworkPolicy
globalnetworksets                              crd.projectcalico.org/v1               false        GlobalNetworkSet
hostendpoints                                  crd.projectcalico.org/v1               false        HostEndpoint
ipamblocks                                     crd.projectcalico.org/v1               false        IPAMBlock
ipamconfigs                                    crd.projectcalico.org/v1               false        IPAMConfig
ipamhandles                                    crd.projectcalico.org/v1               false        IPAMHandle
ippools                                        crd.projectcalico.org/v1               false        IPPool
ipreservations                                 crd.projectcalico.org/v1               false        IPReservation
kubecontrollersconfigurations                  crd.projectcalico.org/v1               false        KubeControllersConfiguration
networkpolicies                                crd.projectcalico.org/v1               true         NetworkPolicy
networksets                                    crd.projectcalico.org/v1               true         NetworkSet
endpointslices                                 discovery.k8s.io/v1                    true         EndpointSlice
events                            ev           events.k8s.io/v1                       true         Event
ingresses                         ing          extensions/v1beta1                     true         Ingress
flowschemas                                    flowcontrol.apiserver.k8s.io/v1beta1   false        FlowSchema
prioritylevelconfigurations                    flowcontrol.apiserver.k8s.io/v1beta1   false        PriorityLevelConfiguration
nodes                                          metrics.k8s.io/v1beta1                 false        NodeMetrics
pods                                           metrics.k8s.io/v1beta1                 true         PodMetrics
ingressclasses                                 networking.k8s.io/v1                   false        IngressClass
ingresses                         ing          networking.k8s.io/v1                   true         Ingress
networkpolicies                   netpol       networking.k8s.io/v1                   true         NetworkPolicy
runtimeclasses                                 node.k8s.io/v1                         false        RuntimeClass
poddisruptionbudgets              pdb          policy/v1                              true         PodDisruptionBudget
podsecuritypolicies               psp          policy/v1beta1                         false        PodSecurityPolicy
clusterrolebindings                            rbac.authorization.k8s.io/v1           false        ClusterRoleBinding
clusterroles                                   rbac.authorization.k8s.io/v1           false        ClusterRole
rolebindings                                   rbac.authorization.k8s.io/v1           true         RoleBinding
roles                                          rbac.authorization.k8s.io/v1           true         Role
priorityclasses                   pc           scheduling.k8s.io/v1                   false        PriorityClass
csidrivers                                     storage.k8s.io/v1                      false        CSIDriver
csinodes                                       storage.k8s.io/v1                      false        CSINode
csistoragecapacities                           storage.k8s.io/v1beta1                 true         CSIStorageCapacity
storageclasses                    sc           storage.k8s.io/v1                      false        StorageClass
volumeattachments                              storage.k8s.io/v1                      false        VolumeAttachment

檢視deployment控制器

[root@k8scloude1 deployment]# kubectl get deploy
No resources found in deployment namespace.

建議使用yaml檔案建立deploy,命令列的方式,支援的選項比較少

生成deployment的yaml檔案

#nginx為deploy的名字    --image=nginx使用Nginx映象
[root@k8scloude1 deployment]# kubectl create deploy nginx --image=nginx --dry-run=client -o yaml >nginx.yaml

[root@k8scloude1 deployment]# ll -h nginx.yaml 
-rw-r--r-- 1 root root 384 1月  25 15:39 nginx.yaml

檢視生成的deploy yaml配置檔案

[root@k8scloude1 deployment]# cat nginx.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: nginx
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        name: nginx
        resources: {}
status: {}

pod模板可以有多個標籤,只要其中一個標籤和matchLabels裡的標籤一致即可

[root@k8scloude1 deployment]# vim nginx.yaml 

[root@k8scloude1 deployment]# cat nginx.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  #這個labels是deployment的標籤
  labels:
    app: nginx
  #deployment名字
  name: nginx
spec:
  #replicas: 1 表示副本數為1,只生成一個pod
  replicas: 1
  #selector標籤選擇器
  selector:
    matchLabels:
      appx: xyz
  strategy: {}
  #template下面是pod的模板
  template:
    metadata:
      creationTimestamp: null
      #這個labels表示pod標籤
      labels:
         #標籤
        app: nginx
        appx: xyz
    spec:
      containers:
      - image: nginx
        name: nginx
        resources: {}
status: {}

建立deploy

[root@k8scloude1 deployment]# kubectl apply -f nginx.yaml 
deployment.apps/nginx created

[root@k8scloude1 deployment]# kubectl get deploy
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   0/1     1            0           11s

直接下載nginx映象太慢,重新修改下yaml檔案

[root@k8scloude1 deployment]# kubectl delete deploy nginx 
deployment.apps "nginx" deleted


[root@k8scloude1 deployment]# vim nginx.yaml 

[root@k8scloude1 deployment]# cat nginx.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: nginx
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      appx: xyz
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
        appx: xyz
    spec:
      #優雅刪除pod的期限時間
      terminationGracePeriodSeconds: 0
      containers:
      - image: nginx
        name: nginx
        #修改映象下載策略:IfNotPresent表示本地有映象就不下載
        imagePullPolicy: IfNotPresent
        resources: {}
status: {}

建立deploy

[root@k8scloude1 deployment]# kubectl apply -f nginx.yaml 
deployment.apps/nginx created

檢視deployment

[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES   SELECTOR
nginx   1/1     1            1           11s   nginx        nginx    appx=xyz

#--show-labels 顯示標籤
[root@k8scloude1 deployment]# kubectl get deploy -o wide --show-labels
NAME    READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES   SELECTOR   LABELS
nginx   1/1     1            1           60s   nginx        nginx    appx=xyz   app=nginx

可以看到pod也被建立出來了

[root@k8scloude1 deployment]# kubectl get pod -o wide --show-labels
NAME                     READY   STATUS    RESTARTS   AGE     IP               NODE         NOMINATED NODE   READINESS GATES   LABELS
nginx-6fcb7995f5-c8lpc   1/1     Running   0          2m25s   10.244.112.146   k8scloude2   <none>           <none>            app=nginx,appx=xyz,pod-template-hash=6fcb7995f5

注意:刪除pod之後,deploy會根據副本數replicas,建立缺失的pod,所以要想真正刪除pod,需要刪除deployment控制器

[root@k8scloude1 deployment]# kubectl delete pod nginx-6fcb7995f5-c8lpc --force
warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "nginx-6fcb7995f5-c8lpc" force deleted

#pod又重新建立了
[root@k8scloude1 deployment]# kubectl get pod -o wide --show-labels
NAME                     READY   STATUS    RESTARTS   AGE   IP               NODE         NOMINATED NODE   READINESS GATES   LABELS
nginx-6fcb7995f5-s6smg   1/1     Running   0          5s    10.244.112.147   k8scloude2   <none>           <none>            app=nginx,appx=xyz,pod-template-hash=6fcb7995f5

六.修改deploy副本數

修改deployment副本數有三種方法:

  1. kubectl edit deploy deployname ,直接修改replicas的數目即可,實時生效,注意:可以線上修改deploy但是不能線上修改pod
  2. 可以直接修改yaml檔案: 比如:replicas: 6
  3. kubectl scale deployment deployname 線上伸縮副本數

6.1 kubectl edit deploy 修改副本數

檢視deploy

[root@k8scloude1 deployment]# kubectl get deploy
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   1/1     1            1           10m

第一種:kubectl edit deploy deployname ,直接修改replicas的數目即可,實時生效,注意可以線上修改deploy但是不能線上修改pod

[root@k8scloude1 deployment]# kubectl edit deploy nginx 
deployment.apps/nginx edited

檢視deploy,可以發現副本數變了,READY為3/3

[root@k8scloude1 deployment]# kubectl get deploy
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   3/3     3            3           11m

[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES   SELECTOR
nginx   3/3     3            3           11m   nginx        nginx    appx=xyz

6.2 修改yaml檔案更改副本數

第二種:可以直接修改yaml檔案: replicas: 6,修改為6個副本

[root@k8scloude1 deployment]# vim nginx.yaml 

[root@k8scloude1 deployment]# cat nginx.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: nginx
  name: nginx
spec:
  replicas: 6
  selector:
    matchLabels:
      appx: xyz
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
        appx: xyz
    spec:
      terminationGracePeriodSeconds: 0
      containers:
      - image: nginx
        name: nginx
        imagePullPolicy: IfNotPresent
        resources: {}
status: {}

kubectl edit可以立即生效,但是修改yaml檔案需要apply才能生效

[root@k8scloude1 deployment]# kubectl get deploy
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   3/3     3            3           14m

[root@k8scloude1 deployment]# kubectl apply -f nginx.yaml 
deployment.apps/nginx configured

[root@k8scloude1 deployment]# kubectl get deploy
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   6/6     6            6           15m

可以看到deploy生成了6個pod

[root@k8scloude1 deployment]# kubectl get pod -o wide
NAME                     READY   STATUS    RESTARTS   AGE     IP               NODE         NOMINATED NODE   READINESS GATES
nginx-6fcb7995f5-64gm2   1/1     Running   0          22s     10.244.251.204   k8scloude3   <none>           <none>
nginx-6fcb7995f5-75jfd   1/1     Running   0          22s     10.244.251.206   k8scloude3   <none>           <none>
nginx-6fcb7995f5-8c75n   1/1     Running   0          4m23s   10.244.251.205   k8scloude3   <none>           <none>
nginx-6fcb7995f5-cs6w4   1/1     Running   0          4m23s   10.244.112.143   k8scloude2   <none>           <none>
nginx-6fcb7995f5-s6smg   1/1     Running   0          11m     10.244.112.147   k8scloude2   <none>           <none>
nginx-6fcb7995f5-tpwr4   1/1     Running   0          22s     10.244.112.148   k8scloude2   <none>           <none>

6.3 kubectl scale修改副本數

kubectl scale線上伸縮deploy,--replicas=10修改副本數為10個

[root@k8scloude1 deployment]# kubectl scale deployment nginx --replicas=10
deployment.apps/nginx scaled

[root@k8scloude1 deployment]# kubectl get pod -o wide
NAME                     READY   STATUS    RESTARTS   AGE     IP               NODE         NOMINATED NODE   READINESS GATES
nginx-6fcb7995f5-5sv29   1/1     Running   0          4s      10.244.112.152   k8scloude2   <none>           <none>
nginx-6fcb7995f5-64gm2   1/1     Running   0          2m12s   10.244.251.204   k8scloude3   <none>           <none>
nginx-6fcb7995f5-75jfd   1/1     Running   0          2m12s   10.244.251.206   k8scloude3   <none>           <none>
nginx-6fcb7995f5-8c75n   1/1     Running   0          6m13s   10.244.251.205   k8scloude3   <none>           <none>
nginx-6fcb7995f5-cs6w4   1/1     Running   0          6m13s   10.244.112.143   k8scloude2   <none>           <none>
nginx-6fcb7995f5-f6nz9   1/1     Running   0          4s      10.244.112.150   k8scloude2   <none>           <none>
nginx-6fcb7995f5-hx224   1/1     Running   0          4s      10.244.112.149   k8scloude2   <none>           <none>
nginx-6fcb7995f5-s6smg   1/1     Running   0          13m     10.244.112.147   k8scloude2   <none>           <none>
nginx-6fcb7995f5-tpwr4   1/1     Running   0          2m12s   10.244.112.148   k8scloude2   <none>           <none>
nginx-6fcb7995f5-zmsfx   1/1     Running   0          4s      10.244.251.208   k8scloude3   <none>           <none>

修改pod副本為1

[root@k8scloude1 deployment]# kubectl scale deployment nginx --replicas=1
deployment.apps/nginx scaled

[root@k8scloude1 deployment]# kubectl get pod -o wide
NAME                     READY   STATUS    RESTARTS   AGE     IP               NODE         NOMINATED NODE   READINESS GATES
nginx-6fcb7995f5-8c75n   1/1     Running   0          6m33s   10.244.251.205   k8scloude3   <none>           <none>

七.HorizontalPodAutoscaler(HPA)

7.1 Pod 水平自動擴縮HorizontalPodAutoscaler概覽

在 Kubernetes 中,HorizontalPodAutoscaler 自動更新工作負載資源 (例如 Deployment 或者 StatefulSet), 目的是自動擴縮工作負載以滿足需求

水平擴縮意味著對增加的負載的響應是部署更多的 Pod。 這與 “垂直(Vertical)” 擴縮不同,對於 Kubernetes, 垂直擴縮意味著將更多資源(例如:記憶體或 CPU)分配給已經為工作負載執行的 Pod

如果負載減少,並且 Pod 的數量高於配置的最小值, HorizontalPodAutoscaler 會指示工作負載資源(Deployment、StatefulSet 或其他類似資源)縮減。

水平 Pod 自動擴縮不適用於無法擴縮的物件(例如:DaemonSet。)

HorizontalPodAutoscaler 被實現為 Kubernetes API 資源和控制器。

資源決定了控制器的行為。 在 Kubernetes 控制平面內執行的水平 Pod 自動擴縮控制器會定期調整其目標(例如:Deployment)的所需規模,以匹配觀察到的指標, 例如,平均 CPU 利用率、平均記憶體利用率或你指定的任何其他自定義指標。

image-20230203164833817

7.2 HPA自動伸縮pod副本數實戰

HPA(horizontal pod autoscalers)水平自動伸縮 透過檢測pod CPU的負載,解決deployment裡某pod負 載太重,動態伸縮pod的數量來負載均衡。

現在deploy只生成了一個pod

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-6fcb7995f5-8c75n   1/1     Running   0          45m

檢視HPA

[root@k8scloude1 deployment]# kubectl get hpa
No resources found in deployment namespace.

設定HPA:--min=2 最小pod副本數為2,--max=5 最大pod副本數為5

[root@k8scloude1 deployment]# kubectl autoscale deployment nginx --max=5 --min=2
horizontalpodautoscaler.autoscaling/nginx autoscaled

觀察pod副本數,發現pod副本數自動變為2了,因為HPA設定了最小pod副本數為2

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-6fcb7995f5-8c75n   1/1     Running   0          49m

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-6fcb7995f5-8c75n   1/1     Running   0          49m

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-6fcb7995f5-8c75n   1/1     Running   0          49m
nginx-6fcb7995f5-zrdpj   1/1     Running   0          22s

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-6fcb7995f5-8c75n   1/1     Running   0          49m
nginx-6fcb7995f5-zrdpj   1/1     Running   0          39s

使用kubectl scale 設定pod副本數為1

[root@k8scloude1 deployment]# kubectl scale deployment nginx --replicas=1
deployment.apps/nginx scaled

因為autoscale是實時擴充套件的,所以就算scale修改為1,autoscale也會自動把副本數修改為2

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS              RESTARTS   AGE
nginx-6fcb7995f5-5jl6h   0/1     ContainerCreating   0          0s
nginx-6fcb7995f5-8c75n   1/1     Running             0          50m

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-6fcb7995f5-5jl6h   1/1     Running   0          4s
nginx-6fcb7995f5-8c75n   1/1     Running   0          50m

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-6fcb7995f5-5jl6h   1/1     Running   0          16s
nginx-6fcb7995f5-8c75n   1/1     Running   0          50m

檢視hpa

[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS         MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   <unknown>/80%   2         5         2          4m18s

刪除HPA

[root@k8scloude1 deployment]# kubectl delete hpa nginx 
horizontalpodautoscaler.autoscaling "nginx" deleted

[root@k8scloude1 deployment]# kubectl get hpa
No resources found in deployment namespace.

刪除nginx deployment

[root@k8scloude1 deployment]# kubectl delete -f nginx.yaml 
deployment.apps "nginx" deleted

[root@k8scloude1 deployment]# kubectl get deploy
No resources found in deployment namespace.

接下來給pod設定資源限制,對pod的資源限制可以透過pod裡的resource欄位來限制,resources裡的requests欄位表示容器所在節點資源的最小值,最低要求,滿足不了這個要求pod建立不成功。

requests:cpu: 400m 表示pod需要400個微核心才能建立成功注意:m代表微核心,1個核等於1000個微核心。

[root@k8scloude1 deployment]# vim nginxcpulimit.yaml 

[root@k8scloude1 deployment]# cat nginxcpulimit.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: nginx
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      appx: xyz
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
        appx: xyz
    spec:
      terminationGracePeriodSeconds: 0
      containers:
      - image: nginx
        name: nginx
        imagePullPolicy: IfNotPresent
        resources: 
          #設定資源限制:CPU:400m    #m代表微核心,1個核等於1000個微核心
          requests:
            cpu: 400m
status: {}

建立deployment

[root@k8scloude1 deployment]# kubectl apply -f nginxcpulimit.yaml 
deployment.apps/nginx created

[root@k8scloude1 deployment]# kubectl get deployments -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES   SELECTOR
nginx   1/1     1            1           13s   nginx        nginx    appx=xyz

檢視node節點的負載

[root@k8scloude1 deployment]# kubectl top node 
W0125 17:06:20.377474   15650 top_node.go:119] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag
NAME         CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%   
k8scloude1   230m         11%    1375Mi          42%       
k8scloude2   100m         5%     669Mi           34%       
k8scloude3   113m         5%     737Mi           38%   

設定HPA:最小pod副本數為1,最大pod副本數為15,--cpu-percent=80表示當deployment資源物件的CPU使用率高達80%時,就會進行擴容,最多擴容到15個

[root@k8scloude1 deployment]# kubectl autoscale deployment nginx --min=1 --max=15 --cpu-percent=80
horizontalpodautoscaler.autoscaling/nginx autoscaled

檢視hpa

[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   0%/80%    1         15        1          69s

安裝一個壓力測試的包:ab工具用於壓力測試

[root@k8scloude1 deployment]# yum -y install httpd-tools

建立一個service服務,服務埠為80,service型別為NodePort

[root@k8scloude1 deployment]# kubectl expose --name=nginxsvc deployment nginx --port=80 --type=NodePort
service/nginxsvc exposed

[root@k8scloude1 deployment]# kubectl get svc -o wide
NAME       TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE   SELECTOR
nginxsvc   NodePort   10.96.231.54   <none>        80:31085/TCP   10s   appx=xyz

檢視pod

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-79b8d956bd-wbcq5   1/1     Running   0          12m

使用ab進行壓力測試,一直給nginx服務發請求,Nginx服務的網址為:http://k8s節點ip:31085/,也就是http://192.168.110.130:31085/index.html。

ab壓力測試命令引數解釋:

  • -t:測試所進行的最大秒數。其內部隱含值是-n 50000,它可以使對伺服器的測試限制在一個固定的總時間以內。預設時,沒有時間限制。
  • -n:在測試會話中所執行的請求個數。預設時,僅執行一個請求。
  • -c:一次產生的請求個數。預設是一次一個。

下面一直給Nginx服務發請求

[root@k8scloude1 ~]# ab -t 600 -n 1000000 -c 1000 http://192.168.110.130:31085/index.html
This is ApacheBench, Version 2.3 <$Revision: 1430300 %sSourceCode%gt;
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.110.130 (be patient)
Completed 100000 requests
^C

Server Software:        nginx/1.21.5
Server Hostname:        192.168.110.130
Server Port:            31085

Document Path:          /index.html
Document Length:        615 bytes

Concurrency Level:      1000
Time taken for tests:   27.096 seconds
Complete requests:      112145
Failed requests:        2029
   (Connect: 0, Receive: 0, Length: 0, Exceptions: 2029)
Write errors:           0
Total transferred:      95814989 bytes
HTML transferred:       69487620 bytes
Requests per second:    4138.87 [#/sec] (mean)
Time per request:       241.612 [ms] (mean)
Time per request:       0.242 [ms] (mean, across all concurrent requests)
Transfer rate:          3453.31 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0  121 219.7     73    2693
Processing:     1  113  84.3     96    5111
Waiting:        0   95  78.9     80    5063
Total:          2  235 241.9    178    7568

Percentage of the requests served within a certain time (ms)
  50%    178
  66%    208
  75%    230
  80%    247
  90%    319
  95%    469
  98%   1188
  99%   1233
 100%   7568 (longest request)

ab壓力測試之後,檢視pod的變化,pod數自動變為了3個

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-79b8d956bd-622fw   1/1     Running   0          14s
nginx-79b8d956bd-8z5h7   1/1     Running   0          14s
nginx-79b8d956bd-wbcq5   1/1     Running   0          14m

檢視hpa的變化

[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   31%/80%   1         15        3          11m

[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   61%/80%   1         15        3          12m

[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   61%/80%   1         15        3          12m
[root@k8scloude1 deployment]# kubectl get hpa

NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   61%/80%   1         15        3          12m

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-79b8d956bd-622fw   1/1     Running   0          101s
nginx-79b8d956bd-8z5h7   1/1     Running   0          101s
nginx-79b8d956bd-wbcq5   1/1     Running   0          15m

刪除服務並重新建立一個service服務,這次的service服務型別為ClusterIP。

[root@k8scloude1 deployment]# kubectl delete svc nginxsvc

[root@k8scloude1 deployment]# kubectl get svc
No resources found in deployment namespace.

[root@k8scloude1 deployment]# kubectl expose --name=nginxsvc deployment nginx --port=80
service/nginxsvc exposed

[root@k8scloude1 deployment]# kubectl get svc
NAME       TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
nginxsvc   ClusterIP   10.109.240.56   <none>        80/TCP    3s

Nginx服務的網址為:http://10.109.240.56:80/index.html,繼續進行ab壓力測試,給Nginx服務發請求。

#繼續ab壓力測試
[root@k8scloude1 ~]# ab -t 600 -n 1000000 -c 1000 http://10.109.240.56:80/index.html
This is ApacheBench, Version 2.3 <$Revision: 1430300 %sSourceCode%gt;
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 10.109.240.56 (be patient)
Completed 100000 requests
Completed 200000 requests
Completed 300000 requests
Completed 400000 requests
Completed 500000 requests
Completed 600000 requests
Completed 700000 requests
Completed 800000 requests
Completed 900000 requests
apr_pollset_poll: The timeout specified has expired (70007)
Total of 985106 requests completed

執行ab壓力測試的同時觀察pod的變化

檢視pod的負載

[root@k8scloude1 deployment]# kubectl top pods
W0125 17:25:03.753728   30784 top_pod.go:140] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag
NAME                     CPU(cores)   MEMORY(bytes)   
nginx-79b8d956bd-5hq5j   179m         4Mi             
nginx-79b8d956bd-622fw   366m         5Mi             
nginx-79b8d956bd-6wjsl   248m         5Mi             
nginx-79b8d956bd-8z5h7   360m         5Mi             
nginx-79b8d956bd-wbcq5   254m         5Mi      

檢視hpa,現在已經有5個pod了

[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   71%/80%   1         15        5          16m

繼續觀察pod

[root@k8scloude1 deployment]# kubectl top pods
W0125 17:25:49.908634   31380 top_pod.go:140] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag
NAME                     CPU(cores)   MEMORY(bytes)   
nginx-79b8d956bd-5hq5j   254m         5Mi             
nginx-79b8d956bd-622fw   326m         5Mi             
nginx-79b8d956bd-6wjsl   322m         5Mi             
nginx-79b8d956bd-8z5h7   255m         5Mi             
nginx-79b8d956bd-wbcq5   264m         5Mi             

[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   70%/80%   1         15        5          18m

[root@k8scloude1 deployment]# kubectl top pods
W0125 17:27:46.476364   32773 top_pod.go:140] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag
NAME                     CPU(cores)   MEMORY(bytes)   
nginx-79b8d956bd-5hq5j   285m         5Mi             
nginx-79b8d956bd-622fw   331m         5Mi             
nginx-79b8d956bd-6wjsl   351m         5Mi             
nginx-79b8d956bd-8z5h7   277m         5Mi             
nginx-79b8d956bd-wbcq5   264m         6Mi    
         
[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   75%/80%   1         15        5          19m

停止給nginx傳送請求之後hpa的TARGETS降下來了

[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   0%/80%    1         15        5          20m

但是pod數沒有降下來

[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   0%/80%    1         15        5          20m

[root@k8scloude1 deployment]# kubectl top pods
W0125 17:29:32.004502   34116 top_pod.go:140] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag
NAME                     CPU(cores)   MEMORY(bytes)   
nginx-79b8d956bd-5hq5j   0m           5Mi             
nginx-79b8d956bd-622fw   0m           5Mi             
nginx-79b8d956bd-6wjsl   0m           5Mi             
nginx-79b8d956bd-8z5h7   0m           5Mi             
nginx-79b8d956bd-wbcq5   0m           6Mi             

[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   0%/80%    1         15        5          21m

注意:pod負載降下來之後,pod副本數不會立即減少,過一段時間pod副本數才會減少,預設時間間隔為5分鐘,這樣的目的是為了防止pod副本數的抖動。

10分鐘之後觀察pod,pod副本降下來了。

[root@k8scloude1 deployment]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   0%/80%    1         15        1          46m

[root@k8scloude1 deployment]# kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx-79b8d956bd-wbcq5   1/1     Running   0          50m

[root@k8scloude1 deployment]# kubectl delete hpa nginx 
horizontalpodautoscaler.autoscaling "nginx" deleted

八.k8s升級映象

8.1 kubectl edit deployment升級映象

檢視deploy,可以看到映象為nginx

[root@k8scloude1 deployment]# kubectl get deploy
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   1/1     1            1           179m

[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE    CONTAINERS   IMAGES   SELECTOR
nginx   1/1     1            1           179m   nginx        nginx    appx=xyz

第一種方法:使用kubectl edit deployment deployname升級映象。把deploy的映象由nginx升級為hub.c.163.com/library/nginx:1.12.0,修改 - image: nginx 為 - image: hub.c.163.com/library/nginx:1.12.0

[root@k8scloude1 deployment]# kubectl edit deployment nginx 
deployment.apps/nginx edited

修改之後檢視deploy,IMAGES變了。

修改deploy的映象的時候,本質上是刪除現有的pod,然後用新映象建立新pod

[root@k8scloude1 deployment]# kubectl get deployments -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE    CONTAINERS   IMAGES                               SELECTOR
nginx   1/1     1            1           3h3m   nginx        hub.c.163.com/library/nginx:1.12.0   appx=xyz

[root@k8scloude1 deployment]# kubectl get pods -o wide
NAME                     READY   STATUS    RESTARTS   AGE   IP               NODE         NOMINATED NODE   READINESS GATES
nginx-66bcf4489b-ztvnc   1/1     Running   0          77s   10.244.251.211   k8scloude3   <none>           <none>

8.2 kubectl set image deploy升級映象

檢視deploy

[root@k8scloude1 deployment]# kubectl get deployment -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS   IMAGES                               SELECTOR
nginx   1/1     1            1           6h41m   nginx        hub.c.163.com/library/nginx:1.12.0   appx=xyz

第二種方式:使用kubectl set image deploy deployname 容器名=映象名 升級映象。

[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.13.0
deployment.apps/nginx image updated

檢視deploy,可以發現映象升級為hub.c.163.com/library/nginx:1.13.0了

[root@k8scloude1 deployment]# kubectl get deployment -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS   IMAGES                               SELECTOR
nginx   1/1     1            1           6h42m   nginx        hub.c.163.com/library/nginx:1.13.0   appx=xyz

繼續升級映象

[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:latest
deployment.apps/nginx image updated

[root@k8scloude1 deployment]# kubectl get deployment -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS   IMAGES                               SELECTOR
nginx   1/1     1            1           6h43m   nginx        hub.c.163.com/library/nginx:latest   appx=xyz

[root@k8scloude1 deployment]# kubectl get pod -o wide
NAME                     READY   STATUS    RESTARTS   AGE   IP               NODE         NOMINATED NODE   READINESS GATES
nginx-75698d6db5-bzv9d   1/1     Running   0          17s   10.244.251.212   k8scloude3   <none>           <none>

8.3 映象升級歷史記錄

檢視映象替換的歷史記錄 kubectl rollout history deployment nginx,現在映象替換的歷史記錄是不清楚的,是因為升級映象的時候沒有record記錄。

[root@k8scloude1 deployment]# kubectl rollout history deployment nginx 
deployment.apps/nginx 
REVISION  CHANGE-CAUSE
1         <none>
2         <none>
3         <none>
4         <none>

替換映象並記錄

[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.12.0 --record
deployment.apps/nginx image updated

可以看到映象替換成功

[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS   IMAGES                               SELECTOR
nginx   1/1     1            1           6h47m   nginx        hub.c.163.com/library/nginx:1.12.0   appx=xyz

多進行幾次映象替換並記錄

[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.13.0 --record
deployment.apps/nginx image updated

[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:latest --record
deployment.apps/nginx image updated

[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=nginx --record
deployment.apps/nginx image updated

檢視映象替換的歷史記錄

[root@k8scloude1 deployment]# kubectl rollout history deployment nginx
deployment.apps/nginx 
REVISION  CHANGE-CAUSE
5         kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.12.0 --record=true
6         kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.13.0 --record=true
7         kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:latest --record=true
8         kubectl set image deploy nginx nginx=nginx --record=true

檢視deploy和pod

[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS   IMAGES   SELECTOR
nginx   1/1     1            1           6h48m   nginx        nginx    appx=xyz

[root@k8scloude1 deployment]# kubectl get pod -o wide
NAME                     READY   STATUS    RESTARTS   AGE   IP               NODE         NOMINATED NODE   READINESS GATES
nginx-79b8d956bd-xblj9   1/1     Running   0          61s   10.244.251.214   k8scloude3   <none>           <none>

kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.13.0 --record=true是第6條修改記錄,我們可以進行回滾,回滾到映象是hub.c.163.com/library/nginx:1.13.0的時候。

[root@k8scloude1 deployment]# kubectl rollout undo deployment nginx --to-revision=6
deployment.apps/nginx rolled back

[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS   IMAGES                               SELECTOR
nginx   1/1     1            1           6h51m   nginx        hub.c.163.com/library/nginx:1.13.0   appx=xyz

變更映象的本質是刪除舊版本的pod,然後用新映象建立pod。

修改deploy副本數為10個

[root@k8scloude1 deployment]# kubectl scale deployment nginx --replicas=10
deployment.apps/nginx scaled

編輯deploy,修改映象

[root@k8scloude1 deployment]# kubectl edit deployment nginx 
deployment.apps/nginx edited

[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE    CONTAINERS   IMAGES                               SELECTOR
nginx   5/5     5            5           7h1m   nginx        hub.c.163.com/library/nginx:1.13.0   appx=xyz

 #滾動更新配置說明:
strategy:
    rollingUpdate:
          #maxSurge:在升級過程中一次升級幾個,可以是數字也可以是百分比
      maxSurge: 25%
          #maxUnavailable 在升級過程中,只能有幾個不可用, 一次性刪除多少個pod,可以是數字也可以是百分比
      maxUnavailable: 25%
    type: RollingUpdate

相關文章