一.系統環境
伺服器版本 | docker軟體版本 | Kubernetes(k8s)叢集版本 | CPU架構 |
---|---|---|---|
CentOS Linux release 7.4.1708 (Core) | Docker version 20.10.12 | v1.21.9 | x86_64 |
Kubernetes叢集架構:k8scloude1作為master節點,k8scloude2,k8scloude3作為worker節點
伺服器 | 作業系統版本 | CPU架構 | 程式 | 功能描述 |
---|---|---|---|---|
k8scloude1/192.168.110.130 | CentOS Linux release 7.4.1708 (Core) | x86_64 | docker,kube-apiserver,etcd,kube-scheduler,kube-controller-manager,kubelet,kube-proxy,coredns,calico | k8s master節點 |
k8scloude2/192.168.110.129 | CentOS Linux release 7.4.1708 (Core) | x86_64 | docker,kubelet,kube-proxy,calico | k8s worker節點 |
k8scloude3/192.168.110.128 | CentOS Linux release 7.4.1708 (Core) | x86_64 | docker,kubelet,kube-proxy,calico | k8s worker節點 |
二.前言
使用deployment的前提是已經有一套可以正常執行的Kubernetes叢集,關於Kubernetes(k8s)叢集的安裝部署,可以檢視部落格《Centos7 安裝部署Kubernetes(k8s)叢集》https://www.cnblogs.com/renshengdezheli/p/16686769.html。
三.Kubernetes 控制器
在 Kubernetes 中,控制器透過監控叢集 的公共狀態,並致力於將當前狀態轉變為期望的狀態。
一個控制器至少追蹤一種型別的 Kubernetes 資源。這些 物件 有一個代表期望狀態的 spec 欄位。 該資源的控制器負責確保其當前狀態接近期望狀態。
控制器可能會自行執行操作;在 Kubernetes 中更常見的是一個控制器會傳送資訊給 API 伺服器。
Kubernetes 內建一組控制器,執行在 kube-controller-manager 內。 這些內建的控制器提供了重要的核心功能。
Deployment 控制器和 Job 控制器是 Kubernetes 內建控制器的典型例子。 Kubernetes 允許你執行一個穩定的控制平面,這樣即使某些內建控制器失敗了, 控制平面的其他部分會接替它們的工作。
四.Deployment概覽
一個 Deployment 為 Pod 和 ReplicaSet 提供宣告式的更新能力。
你負責描述 Deployment 中的 目標狀態,而 Deployment 控制器(Controller) 以受控速率更改實際狀態, 使其變為期望狀態。你可以定義 Deployment 以建立新的 ReplicaSet,或刪除現有 Deployment, 並透過新的 Deployment 收養其資源。
注意
:不要管理 Deployment 所擁有的 ReplicaSet 。
以下是 Deployments 的典型用例:
- 建立 Deployment 以將 ReplicaSet 上線。ReplicaSet 在後臺建立 Pod。 檢查 ReplicaSet 的上線狀態,檢視其是否成功。
- 透過更新 Deployment 的 PodTemplateSpec,宣告 Pod 的新狀態 。 新的 ReplicaSet 會被建立,Deployment 以受控速率將 Pod 從舊 ReplicaSet 遷移到新 ReplicaSet。 每個新的 ReplicaSet 都會更新 Deployment 的修訂版本。
- 如果 Deployment 的當前狀態不穩定,回滾到較早的 Deployment 版本。 每次回滾都會更新 Deployment 的修訂版本。
- 擴大 Deployment 規模以承擔更多負載。
- 暫停 Deployment 的上線 以應用對 PodTemplateSpec 所作的多項修改, 然後恢復其執行以啟動新的上線版本。
- 使用 Deployment 狀態來判定上線過程是否出現停滯。
- 清理較舊的不再需要的 ReplicaSet 。
五.建立deployment
建立存放deployment配置檔案的目錄和namespace
[root@k8scloude1 ~]# mkdir deployment
[root@k8scloude1 ~]# cd deployment/
[root@k8scloude1 deployment]# pwd
/root/deployment
#建立namespace
[root@k8scloude1 deployment]# kubectl create ns deployment
namespace/deployment created
#切換名稱空間為deployment
[root@k8scloude1 deployment]# kubens deployment
Context "kubernetes-admin@kubernetes" modified.
Active namespace is "deployment".
[root@k8scloude1 deployment]# kubectl get pod
No resources found in deployment namespace.
Kubernetes裡有很多簡寫,比如deployment簡稱deploy,pod簡稱po
檢視pod
[root@k8scloude1 deployment]# kubectl get po
No resources found in deployment namespace.
檢視對資源的操作有哪些
[root@k8scloude1 deployment]# kubectl --help | grep resource
create Create a resource from a file or from stdin.
get 顯示一個或更多 resources
delete Delete resources by filenames, stdin, resources and names, or by resources and label selector
rollout Manage the rollout of a resource
describe 顯示一個指定 resource 或者 group 的 resources 詳情
patch Update field(s) of a resource
wait Experimental: Wait for a specific condition on one or many resources.
api-resources Print the supported API resources on the server
可以檢視k8s支援的簡寫
[root@k8scloude1 deployment]# kubectl api-resources
NAME SHORTNAMES APIVERSION NAMESPACED KIND
bindings v1 true Binding
componentstatuses cs v1 false ComponentStatus
configmaps cm v1 true ConfigMap
endpoints ep v1 true Endpoints
events ev v1 true Event
limitranges limits v1 true LimitRange
namespaces ns v1 false Namespace
nodes no v1 false Node
persistentvolumeclaims pvc v1 true PersistentVolumeClaim
persistentvolumes pv v1 false PersistentVolume
pods po v1 true Pod
podtemplates v1 true PodTemplate
replicationcontrollers rc v1 true ReplicationController
resourcequotas quota v1 true ResourceQuota
secrets v1 true Secret
serviceaccounts sa v1 true ServiceAccount
services svc v1 true Service
mutatingwebhookconfigurations admissionregistration.k8s.io/v1 false MutatingWebhookConfiguration
validatingwebhookconfigurations admissionregistration.k8s.io/v1 false ValidatingWebhookConfiguration
customresourcedefinitions crd,crds apiextensions.k8s.io/v1 false CustomResourceDefinition
apiservices apiregistration.k8s.io/v1 false APIService
controllerrevisions apps/v1 true ControllerRevision
daemonsets ds apps/v1 true DaemonSet
deployments deploy apps/v1 true Deployment
replicasets rs apps/v1 true ReplicaSet
statefulsets sts apps/v1 true StatefulSet
tokenreviews authentication.k8s.io/v1 false TokenReview
localsubjectaccessreviews authorization.k8s.io/v1 true LocalSubjectAccessReview
selfsubjectaccessreviews authorization.k8s.io/v1 false SelfSubjectAccessReview
selfsubjectrulesreviews authorization.k8s.io/v1 false SelfSubjectRulesReview
subjectaccessreviews authorization.k8s.io/v1 false SubjectAccessReview
horizontalpodautoscalers hpa autoscaling/v1 true HorizontalPodAutoscaler
cronjobs cj batch/v1 true CronJob
jobs batch/v1 true Job
certificatesigningrequests csr certificates.k8s.io/v1 false CertificateSigningRequest
leases coordination.k8s.io/v1 true Lease
bgpconfigurations crd.projectcalico.org/v1 false BGPConfiguration
bgppeers crd.projectcalico.org/v1 false BGPPeer
blockaffinities crd.projectcalico.org/v1 false BlockAffinity
caliconodestatuses crd.projectcalico.org/v1 false CalicoNodeStatus
clusterinformations crd.projectcalico.org/v1 false ClusterInformation
felixconfigurations crd.projectcalico.org/v1 false FelixConfiguration
globalnetworkpolicies crd.projectcalico.org/v1 false GlobalNetworkPolicy
globalnetworksets crd.projectcalico.org/v1 false GlobalNetworkSet
hostendpoints crd.projectcalico.org/v1 false HostEndpoint
ipamblocks crd.projectcalico.org/v1 false IPAMBlock
ipamconfigs crd.projectcalico.org/v1 false IPAMConfig
ipamhandles crd.projectcalico.org/v1 false IPAMHandle
ippools crd.projectcalico.org/v1 false IPPool
ipreservations crd.projectcalico.org/v1 false IPReservation
kubecontrollersconfigurations crd.projectcalico.org/v1 false KubeControllersConfiguration
networkpolicies crd.projectcalico.org/v1 true NetworkPolicy
networksets crd.projectcalico.org/v1 true NetworkSet
endpointslices discovery.k8s.io/v1 true EndpointSlice
events ev events.k8s.io/v1 true Event
ingresses ing extensions/v1beta1 true Ingress
flowschemas flowcontrol.apiserver.k8s.io/v1beta1 false FlowSchema
prioritylevelconfigurations flowcontrol.apiserver.k8s.io/v1beta1 false PriorityLevelConfiguration
nodes metrics.k8s.io/v1beta1 false NodeMetrics
pods metrics.k8s.io/v1beta1 true PodMetrics
ingressclasses networking.k8s.io/v1 false IngressClass
ingresses ing networking.k8s.io/v1 true Ingress
networkpolicies netpol networking.k8s.io/v1 true NetworkPolicy
runtimeclasses node.k8s.io/v1 false RuntimeClass
poddisruptionbudgets pdb policy/v1 true PodDisruptionBudget
podsecuritypolicies psp policy/v1beta1 false PodSecurityPolicy
clusterrolebindings rbac.authorization.k8s.io/v1 false ClusterRoleBinding
clusterroles rbac.authorization.k8s.io/v1 false ClusterRole
rolebindings rbac.authorization.k8s.io/v1 true RoleBinding
roles rbac.authorization.k8s.io/v1 true Role
priorityclasses pc scheduling.k8s.io/v1 false PriorityClass
csidrivers storage.k8s.io/v1 false CSIDriver
csinodes storage.k8s.io/v1 false CSINode
csistoragecapacities storage.k8s.io/v1beta1 true CSIStorageCapacity
storageclasses sc storage.k8s.io/v1 false StorageClass
volumeattachments storage.k8s.io/v1 false VolumeAttachment
檢視deployment控制器
[root@k8scloude1 deployment]# kubectl get deploy
No resources found in deployment namespace.
建議使用yaml檔案建立deploy,命令列的方式,支援的選項比較少
,
生成deployment的yaml檔案
#nginx為deploy的名字 --image=nginx使用Nginx映象
[root@k8scloude1 deployment]# kubectl create deploy nginx --image=nginx --dry-run=client -o yaml >nginx.yaml
[root@k8scloude1 deployment]# ll -h nginx.yaml
-rw-r--r-- 1 root root 384 1月 25 15:39 nginx.yaml
檢視生成的deploy yaml配置檔案
[root@k8scloude1 deployment]# cat nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: nginx
name: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: nginx
spec:
containers:
- image: nginx
name: nginx
resources: {}
status: {}
pod模板可以有多個標籤,只要其中一個標籤和matchLabels裡的標籤一致即可
[root@k8scloude1 deployment]# vim nginx.yaml
[root@k8scloude1 deployment]# cat nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
#這個labels是deployment的標籤
labels:
app: nginx
#deployment名字
name: nginx
spec:
#replicas: 1 表示副本數為1,只生成一個pod
replicas: 1
#selector標籤選擇器
selector:
matchLabels:
appx: xyz
strategy: {}
#template下面是pod的模板
template:
metadata:
creationTimestamp: null
#這個labels表示pod標籤
labels:
#標籤
app: nginx
appx: xyz
spec:
containers:
- image: nginx
name: nginx
resources: {}
status: {}
建立deploy
[root@k8scloude1 deployment]# kubectl apply -f nginx.yaml
deployment.apps/nginx created
[root@k8scloude1 deployment]# kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 0/1 1 0 11s
直接下載nginx映象太慢,重新修改下yaml檔案
[root@k8scloude1 deployment]# kubectl delete deploy nginx
deployment.apps "nginx" deleted
[root@k8scloude1 deployment]# vim nginx.yaml
[root@k8scloude1 deployment]# cat nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: nginx
name: nginx
spec:
replicas: 1
selector:
matchLabels:
appx: xyz
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: nginx
appx: xyz
spec:
#優雅刪除pod的期限時間
terminationGracePeriodSeconds: 0
containers:
- image: nginx
name: nginx
#修改映象下載策略:IfNotPresent表示本地有映象就不下載
imagePullPolicy: IfNotPresent
resources: {}
status: {}
建立deploy
[root@k8scloude1 deployment]# kubectl apply -f nginx.yaml
deployment.apps/nginx created
檢視deployment
[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 1/1 1 1 11s nginx nginx appx=xyz
#--show-labels 顯示標籤
[root@k8scloude1 deployment]# kubectl get deploy -o wide --show-labels
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR LABELS
nginx 1/1 1 1 60s nginx nginx appx=xyz app=nginx
可以看到pod也被建立出來了
[root@k8scloude1 deployment]# kubectl get pod -o wide --show-labels
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES LABELS
nginx-6fcb7995f5-c8lpc 1/1 Running 0 2m25s 10.244.112.146 k8scloude2 <none> <none> app=nginx,appx=xyz,pod-template-hash=6fcb7995f5
注意:刪除pod之後,deploy會根據副本數replicas,建立缺失的pod,所以要想真正刪除pod,需要刪除deployment控制器
[root@k8scloude1 deployment]# kubectl delete pod nginx-6fcb7995f5-c8lpc --force
warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "nginx-6fcb7995f5-c8lpc" force deleted
#pod又重新建立了
[root@k8scloude1 deployment]# kubectl get pod -o wide --show-labels
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES LABELS
nginx-6fcb7995f5-s6smg 1/1 Running 0 5s 10.244.112.147 k8scloude2 <none> <none> app=nginx,appx=xyz,pod-template-hash=6fcb7995f5
六.修改deploy副本數
修改deployment副本數有三種方法:
- kubectl edit deploy deployname ,直接修改replicas的數目即可,實時生效,注意:可以線上修改deploy但是不能線上修改pod
- 可以直接修改yaml檔案: 比如:replicas: 6
- kubectl scale deployment deployname 線上伸縮副本數
6.1 kubectl edit deploy 修改副本數
檢視deploy
[root@k8scloude1 deployment]# kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 1/1 1 1 10m
第一種:kubectl edit deploy deployname ,直接修改replicas的數目即可,實時生效,注意:可以線上修改deploy但是不能線上修改pod
[root@k8scloude1 deployment]# kubectl edit deploy nginx
deployment.apps/nginx edited
檢視deploy,可以發現副本數變了,READY為3/3
[root@k8scloude1 deployment]# kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 3/3 3 3 11m
[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 3/3 3 3 11m nginx nginx appx=xyz
6.2 修改yaml檔案更改副本數
第二種:可以直接修改yaml檔案: replicas: 6,修改為6個副本
[root@k8scloude1 deployment]# vim nginx.yaml
[root@k8scloude1 deployment]# cat nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: nginx
name: nginx
spec:
replicas: 6
selector:
matchLabels:
appx: xyz
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: nginx
appx: xyz
spec:
terminationGracePeriodSeconds: 0
containers:
- image: nginx
name: nginx
imagePullPolicy: IfNotPresent
resources: {}
status: {}
kubectl edit可以立即生效,但是修改yaml檔案需要apply才能生效
[root@k8scloude1 deployment]# kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 3/3 3 3 14m
[root@k8scloude1 deployment]# kubectl apply -f nginx.yaml
deployment.apps/nginx configured
[root@k8scloude1 deployment]# kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 6/6 6 6 15m
可以看到deploy生成了6個pod
[root@k8scloude1 deployment]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-6fcb7995f5-64gm2 1/1 Running 0 22s 10.244.251.204 k8scloude3 <none> <none>
nginx-6fcb7995f5-75jfd 1/1 Running 0 22s 10.244.251.206 k8scloude3 <none> <none>
nginx-6fcb7995f5-8c75n 1/1 Running 0 4m23s 10.244.251.205 k8scloude3 <none> <none>
nginx-6fcb7995f5-cs6w4 1/1 Running 0 4m23s 10.244.112.143 k8scloude2 <none> <none>
nginx-6fcb7995f5-s6smg 1/1 Running 0 11m 10.244.112.147 k8scloude2 <none> <none>
nginx-6fcb7995f5-tpwr4 1/1 Running 0 22s 10.244.112.148 k8scloude2 <none> <none>
6.3 kubectl scale修改副本數
kubectl scale線上伸縮deploy,--replicas=10修改副本數為10個
[root@k8scloude1 deployment]# kubectl scale deployment nginx --replicas=10
deployment.apps/nginx scaled
[root@k8scloude1 deployment]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-6fcb7995f5-5sv29 1/1 Running 0 4s 10.244.112.152 k8scloude2 <none> <none>
nginx-6fcb7995f5-64gm2 1/1 Running 0 2m12s 10.244.251.204 k8scloude3 <none> <none>
nginx-6fcb7995f5-75jfd 1/1 Running 0 2m12s 10.244.251.206 k8scloude3 <none> <none>
nginx-6fcb7995f5-8c75n 1/1 Running 0 6m13s 10.244.251.205 k8scloude3 <none> <none>
nginx-6fcb7995f5-cs6w4 1/1 Running 0 6m13s 10.244.112.143 k8scloude2 <none> <none>
nginx-6fcb7995f5-f6nz9 1/1 Running 0 4s 10.244.112.150 k8scloude2 <none> <none>
nginx-6fcb7995f5-hx224 1/1 Running 0 4s 10.244.112.149 k8scloude2 <none> <none>
nginx-6fcb7995f5-s6smg 1/1 Running 0 13m 10.244.112.147 k8scloude2 <none> <none>
nginx-6fcb7995f5-tpwr4 1/1 Running 0 2m12s 10.244.112.148 k8scloude2 <none> <none>
nginx-6fcb7995f5-zmsfx 1/1 Running 0 4s 10.244.251.208 k8scloude3 <none> <none>
修改pod副本為1
[root@k8scloude1 deployment]# kubectl scale deployment nginx --replicas=1
deployment.apps/nginx scaled
[root@k8scloude1 deployment]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-6fcb7995f5-8c75n 1/1 Running 0 6m33s 10.244.251.205 k8scloude3 <none> <none>
七.HorizontalPodAutoscaler(HPA)
7.1 Pod 水平自動擴縮HorizontalPodAutoscaler概覽
在 Kubernetes 中,HorizontalPodAutoscaler 自動更新工作負載資源 (例如 Deployment 或者 StatefulSet), 目的是自動擴縮工作負載以滿足需求。
水平擴縮意味著對增加的負載的響應是部署更多的 Pod。 這與 “垂直(Vertical)” 擴縮不同,對於 Kubernetes, 垂直擴縮意味著將更多資源(例如:記憶體或 CPU)分配給已經為工作負載執行的 Pod。
如果負載減少,並且 Pod 的數量高於配置的最小值, HorizontalPodAutoscaler 會指示工作負載資源(Deployment、StatefulSet 或其他類似資源)縮減。
水平 Pod 自動擴縮不適用於無法擴縮的物件(例如:DaemonSet。)
HorizontalPodAutoscaler 被實現為 Kubernetes API 資源和控制器。
資源決定了控制器的行為。 在 Kubernetes 控制平面內執行的水平 Pod 自動擴縮控制器會定期調整其目標(例如:Deployment)的所需規模,以匹配觀察到的指標, 例如,平均 CPU 利用率、平均記憶體利用率或你指定的任何其他自定義指標。
7.2 HPA自動伸縮pod副本數實戰
HPA(horizontal pod autoscalers)水平自動伸縮 透過檢測pod CPU的負載,解決deployment裡某pod負 載太重,動態伸縮pod的數量來負載均衡。
現在deploy只生成了一個pod
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-6fcb7995f5-8c75n 1/1 Running 0 45m
檢視HPA
[root@k8scloude1 deployment]# kubectl get hpa
No resources found in deployment namespace.
設定HPA:--min=2 最小pod副本數為2,--max=5 最大pod副本數為5
[root@k8scloude1 deployment]# kubectl autoscale deployment nginx --max=5 --min=2
horizontalpodautoscaler.autoscaling/nginx autoscaled
觀察pod副本數,發現pod副本數自動變為2了,因為HPA設定了最小pod副本數為2
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-6fcb7995f5-8c75n 1/1 Running 0 49m
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-6fcb7995f5-8c75n 1/1 Running 0 49m
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-6fcb7995f5-8c75n 1/1 Running 0 49m
nginx-6fcb7995f5-zrdpj 1/1 Running 0 22s
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-6fcb7995f5-8c75n 1/1 Running 0 49m
nginx-6fcb7995f5-zrdpj 1/1 Running 0 39s
使用kubectl scale 設定pod副本數為1
[root@k8scloude1 deployment]# kubectl scale deployment nginx --replicas=1
deployment.apps/nginx scaled
因為autoscale是實時擴充套件的,所以就算scale修改為1,autoscale也會自動把副本數修改為2
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-6fcb7995f5-5jl6h 0/1 ContainerCreating 0 0s
nginx-6fcb7995f5-8c75n 1/1 Running 0 50m
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-6fcb7995f5-5jl6h 1/1 Running 0 4s
nginx-6fcb7995f5-8c75n 1/1 Running 0 50m
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-6fcb7995f5-5jl6h 1/1 Running 0 16s
nginx-6fcb7995f5-8c75n 1/1 Running 0 50m
檢視hpa
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx <unknown>/80% 2 5 2 4m18s
刪除HPA
[root@k8scloude1 deployment]# kubectl delete hpa nginx
horizontalpodautoscaler.autoscaling "nginx" deleted
[root@k8scloude1 deployment]# kubectl get hpa
No resources found in deployment namespace.
刪除nginx deployment
[root@k8scloude1 deployment]# kubectl delete -f nginx.yaml
deployment.apps "nginx" deleted
[root@k8scloude1 deployment]# kubectl get deploy
No resources found in deployment namespace.
接下來給pod設定資源限制,對pod的資源限制可以透過pod裡的resource欄位來限制,resources裡的requests欄位表示容器所在節點資源的最小值,最低要求,滿足不了這個要求pod建立不成功。
requests:cpu: 400m 表示pod需要400個微核心才能建立成功
,注意:m代表微核心,1個核等於1000個微核心。
[root@k8scloude1 deployment]# vim nginxcpulimit.yaml
[root@k8scloude1 deployment]# cat nginxcpulimit.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: nginx
name: nginx
spec:
replicas: 1
selector:
matchLabels:
appx: xyz
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: nginx
appx: xyz
spec:
terminationGracePeriodSeconds: 0
containers:
- image: nginx
name: nginx
imagePullPolicy: IfNotPresent
resources:
#設定資源限制:CPU:400m #m代表微核心,1個核等於1000個微核心
requests:
cpu: 400m
status: {}
建立deployment
[root@k8scloude1 deployment]# kubectl apply -f nginxcpulimit.yaml
deployment.apps/nginx created
[root@k8scloude1 deployment]# kubectl get deployments -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 1/1 1 1 13s nginx nginx appx=xyz
檢視node節點的負載
[root@k8scloude1 deployment]# kubectl top node
W0125 17:06:20.377474 15650 top_node.go:119] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
k8scloude1 230m 11% 1375Mi 42%
k8scloude2 100m 5% 669Mi 34%
k8scloude3 113m 5% 737Mi 38%
設定HPA:最小pod副本數為1,最大pod副本數為15,--cpu-percent=80表示當deployment資源物件的CPU使用率高達80%時,就會進行擴容,最多擴容到15個
[root@k8scloude1 deployment]# kubectl autoscale deployment nginx --min=1 --max=15 --cpu-percent=80
horizontalpodautoscaler.autoscaling/nginx autoscaled
檢視hpa
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 0%/80% 1 15 1 69s
安裝一個壓力測試的包:ab工具用於壓力測試
[root@k8scloude1 deployment]# yum -y install httpd-tools
建立一個service服務,服務埠為80,service型別為NodePort
[root@k8scloude1 deployment]# kubectl expose --name=nginxsvc deployment nginx --port=80 --type=NodePort
service/nginxsvc exposed
[root@k8scloude1 deployment]# kubectl get svc -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
nginxsvc NodePort 10.96.231.54 <none> 80:31085/TCP 10s appx=xyz
檢視pod
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-79b8d956bd-wbcq5 1/1 Running 0 12m
使用ab進行壓力測試,一直給nginx服務發請求,Nginx服務的網址為:http://k8s節點ip:31085/,也就是http://192.168.110.130:31085/index.html。
ab壓力測試命令引數解釋:
- -t:測試所進行的最大秒數。其內部隱含值是-n 50000,它可以使對伺服器的測試限制在一個固定的總時間以內。預設時,沒有時間限制。
- -n:在測試會話中所執行的請求個數。預設時,僅執行一個請求。
- -c:一次產生的請求個數。預設是一次一個。
下面一直給Nginx服務發請求
[root@k8scloude1 ~]# ab -t 600 -n 1000000 -c 1000 http://192.168.110.130:31085/index.html
This is ApacheBench, Version 2.3 <$Revision: 1430300 %sSourceCode%gt;
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 192.168.110.130 (be patient)
Completed 100000 requests
^C
Server Software: nginx/1.21.5
Server Hostname: 192.168.110.130
Server Port: 31085
Document Path: /index.html
Document Length: 615 bytes
Concurrency Level: 1000
Time taken for tests: 27.096 seconds
Complete requests: 112145
Failed requests: 2029
(Connect: 0, Receive: 0, Length: 0, Exceptions: 2029)
Write errors: 0
Total transferred: 95814989 bytes
HTML transferred: 69487620 bytes
Requests per second: 4138.87 [#/sec] (mean)
Time per request: 241.612 [ms] (mean)
Time per request: 0.242 [ms] (mean, across all concurrent requests)
Transfer rate: 3453.31 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 121 219.7 73 2693
Processing: 1 113 84.3 96 5111
Waiting: 0 95 78.9 80 5063
Total: 2 235 241.9 178 7568
Percentage of the requests served within a certain time (ms)
50% 178
66% 208
75% 230
80% 247
90% 319
95% 469
98% 1188
99% 1233
100% 7568 (longest request)
ab壓力測試之後,檢視pod的變化,pod數自動變為了3個
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-79b8d956bd-622fw 1/1 Running 0 14s
nginx-79b8d956bd-8z5h7 1/1 Running 0 14s
nginx-79b8d956bd-wbcq5 1/1 Running 0 14m
檢視hpa的變化
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 31%/80% 1 15 3 11m
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 61%/80% 1 15 3 12m
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 61%/80% 1 15 3 12m
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 61%/80% 1 15 3 12m
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-79b8d956bd-622fw 1/1 Running 0 101s
nginx-79b8d956bd-8z5h7 1/1 Running 0 101s
nginx-79b8d956bd-wbcq5 1/1 Running 0 15m
刪除服務並重新建立一個service服務,這次的service服務型別為ClusterIP。
[root@k8scloude1 deployment]# kubectl delete svc nginxsvc
[root@k8scloude1 deployment]# kubectl get svc
No resources found in deployment namespace.
[root@k8scloude1 deployment]# kubectl expose --name=nginxsvc deployment nginx --port=80
service/nginxsvc exposed
[root@k8scloude1 deployment]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginxsvc ClusterIP 10.109.240.56 <none> 80/TCP 3s
Nginx服務的網址為:http://10.109.240.56:80/index.html,繼續進行ab壓力測試,給Nginx服務發請求。
#繼續ab壓力測試
[root@k8scloude1 ~]# ab -t 600 -n 1000000 -c 1000 http://10.109.240.56:80/index.html
This is ApacheBench, Version 2.3 <$Revision: 1430300 %sSourceCode%gt;
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 10.109.240.56 (be patient)
Completed 100000 requests
Completed 200000 requests
Completed 300000 requests
Completed 400000 requests
Completed 500000 requests
Completed 600000 requests
Completed 700000 requests
Completed 800000 requests
Completed 900000 requests
apr_pollset_poll: The timeout specified has expired (70007)
Total of 985106 requests completed
執行ab壓力測試的同時觀察pod的變化
檢視pod的負載
[root@k8scloude1 deployment]# kubectl top pods
W0125 17:25:03.753728 30784 top_pod.go:140] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag
NAME CPU(cores) MEMORY(bytes)
nginx-79b8d956bd-5hq5j 179m 4Mi
nginx-79b8d956bd-622fw 366m 5Mi
nginx-79b8d956bd-6wjsl 248m 5Mi
nginx-79b8d956bd-8z5h7 360m 5Mi
nginx-79b8d956bd-wbcq5 254m 5Mi
檢視hpa,現在已經有5個pod了
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 71%/80% 1 15 5 16m
繼續觀察pod
[root@k8scloude1 deployment]# kubectl top pods
W0125 17:25:49.908634 31380 top_pod.go:140] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag
NAME CPU(cores) MEMORY(bytes)
nginx-79b8d956bd-5hq5j 254m 5Mi
nginx-79b8d956bd-622fw 326m 5Mi
nginx-79b8d956bd-6wjsl 322m 5Mi
nginx-79b8d956bd-8z5h7 255m 5Mi
nginx-79b8d956bd-wbcq5 264m 5Mi
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 70%/80% 1 15 5 18m
[root@k8scloude1 deployment]# kubectl top pods
W0125 17:27:46.476364 32773 top_pod.go:140] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag
NAME CPU(cores) MEMORY(bytes)
nginx-79b8d956bd-5hq5j 285m 5Mi
nginx-79b8d956bd-622fw 331m 5Mi
nginx-79b8d956bd-6wjsl 351m 5Mi
nginx-79b8d956bd-8z5h7 277m 5Mi
nginx-79b8d956bd-wbcq5 264m 6Mi
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 75%/80% 1 15 5 19m
停止給nginx傳送請求之後hpa的TARGETS降下來了
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 0%/80% 1 15 5 20m
但是pod數沒有降下來
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 0%/80% 1 15 5 20m
[root@k8scloude1 deployment]# kubectl top pods
W0125 17:29:32.004502 34116 top_pod.go:140] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag
NAME CPU(cores) MEMORY(bytes)
nginx-79b8d956bd-5hq5j 0m 5Mi
nginx-79b8d956bd-622fw 0m 5Mi
nginx-79b8d956bd-6wjsl 0m 5Mi
nginx-79b8d956bd-8z5h7 0m 5Mi
nginx-79b8d956bd-wbcq5 0m 6Mi
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 0%/80% 1 15 5 21m
注意
:pod負載降下來之後,pod副本數不會立即減少,過一段時間pod副本數才會減少,預設時間間隔為5分鐘,這樣的目的是為了防止pod副本數的抖動。
10分鐘之後觀察pod,pod副本降下來了。
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 0%/80% 1 15 1 46m
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-79b8d956bd-wbcq5 1/1 Running 0 50m
[root@k8scloude1 deployment]# kubectl delete hpa nginx
horizontalpodautoscaler.autoscaling "nginx" deleted
八.k8s升級映象
8.1 kubectl edit deployment升級映象
檢視deploy,可以看到映象為nginx
[root@k8scloude1 deployment]# kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 1/1 1 1 179m
[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 1/1 1 1 179m nginx nginx appx=xyz
第一種方法:使用kubectl edit deployment deployname升級映象。把deploy的映象由nginx升級為hub.c.163.com/library/nginx:1.12.0,修改 - image: nginx 為 - image: hub.c.163.com/library/nginx:1.12.0
[root@k8scloude1 deployment]# kubectl edit deployment nginx
deployment.apps/nginx edited
修改之後檢視deploy,IMAGES變了。
修改deploy的映象的時候,本質上是刪除現有的pod,然後用新映象建立新pod
。
[root@k8scloude1 deployment]# kubectl get deployments -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 1/1 1 1 3h3m nginx hub.c.163.com/library/nginx:1.12.0 appx=xyz
[root@k8scloude1 deployment]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-66bcf4489b-ztvnc 1/1 Running 0 77s 10.244.251.211 k8scloude3 <none> <none>
8.2 kubectl set image deploy升級映象
檢視deploy
[root@k8scloude1 deployment]# kubectl get deployment -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 1/1 1 1 6h41m nginx hub.c.163.com/library/nginx:1.12.0 appx=xyz
第二種方式:使用kubectl set image deploy deployname 容器名=映象名
升級映象。
[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.13.0
deployment.apps/nginx image updated
檢視deploy,可以發現映象升級為hub.c.163.com/library/nginx:1.13.0了
[root@k8scloude1 deployment]# kubectl get deployment -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 1/1 1 1 6h42m nginx hub.c.163.com/library/nginx:1.13.0 appx=xyz
繼續升級映象
[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:latest
deployment.apps/nginx image updated
[root@k8scloude1 deployment]# kubectl get deployment -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 1/1 1 1 6h43m nginx hub.c.163.com/library/nginx:latest appx=xyz
[root@k8scloude1 deployment]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-75698d6db5-bzv9d 1/1 Running 0 17s 10.244.251.212 k8scloude3 <none> <none>
8.3 映象升級歷史記錄
檢視映象替換的歷史記錄 kubectl rollout history deployment nginx,現在映象替換的歷史記錄是不清楚的,是因為升級映象的時候沒有record記錄。
[root@k8scloude1 deployment]# kubectl rollout history deployment nginx
deployment.apps/nginx
REVISION CHANGE-CAUSE
1 <none>
2 <none>
3 <none>
4 <none>
替換映象並記錄
[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.12.0 --record
deployment.apps/nginx image updated
可以看到映象替換成功
[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 1/1 1 1 6h47m nginx hub.c.163.com/library/nginx:1.12.0 appx=xyz
多進行幾次映象替換並記錄
[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.13.0 --record
deployment.apps/nginx image updated
[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:latest --record
deployment.apps/nginx image updated
[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=nginx --record
deployment.apps/nginx image updated
檢視映象替換的歷史記錄
[root@k8scloude1 deployment]# kubectl rollout history deployment nginx
deployment.apps/nginx
REVISION CHANGE-CAUSE
5 kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.12.0 --record=true
6 kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.13.0 --record=true
7 kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:latest --record=true
8 kubectl set image deploy nginx nginx=nginx --record=true
檢視deploy和pod
[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 1/1 1 1 6h48m nginx nginx appx=xyz
[root@k8scloude1 deployment]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-79b8d956bd-xblj9 1/1 Running 0 61s 10.244.251.214 k8scloude3 <none> <none>
kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.13.0 --record=true是第6條修改記錄,我們可以進行回滾,回滾到映象是hub.c.163.com/library/nginx:1.13.0的時候。
[root@k8scloude1 deployment]# kubectl rollout undo deployment nginx --to-revision=6
deployment.apps/nginx rolled back
[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 1/1 1 1 6h51m nginx hub.c.163.com/library/nginx:1.13.0 appx=xyz
變更映象的本質是刪除舊版本的pod,然後用新映象建立pod。
修改deploy副本數為10個
[root@k8scloude1 deployment]# kubectl scale deployment nginx --replicas=10
deployment.apps/nginx scaled
編輯deploy,修改映象
[root@k8scloude1 deployment]# kubectl edit deployment nginx
deployment.apps/nginx edited
[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 5/5 5 5 7h1m nginx hub.c.163.com/library/nginx:1.13.0 appx=xyz
#滾動更新配置說明:
strategy:
rollingUpdate:
#maxSurge:在升級過程中一次升級幾個,可以是數字也可以是百分比
maxSurge: 25%
#maxUnavailable 在升級過程中,只能有幾個不可用, 一次性刪除多少個pod,可以是數字也可以是百分比
maxUnavailable: 25%
type: RollingUpdate