目錄
- 一.系統環境
- 二.前言
- 三.PodSecurityPolicy 簡介
- 四.為什麼需要 PodSecurityPolicy
- 五.給客戶端授權
- 六.啟用PodSecurityPolicy（PSP）
- 七.PSP規則之禁止建立特權使用者pod
- 八.PSP規則之資料卷
- 8.1 限定資料卷使用某一個目錄
- 8.2 限定資料卷的型別為emptyDir
- 九.PSP規則之指定使用宿主機網路
- 十.PSP規則之pod使用特定的UID執行
- 十一.PSP規則之指定hostport埠
- 十二.PSP規則實戰1
- 十三.PSP規則實戰2:建立deployment
- 十四.總結
一.系統環境
本文主要基於Kubernetes1.22.2和Linux作業系統Ubuntu 18.04。
| 伺服器版本 | docker軟體版本 | Kubernetes(k8s)叢集版本 | CPU架構 |
|---|---|---|---|
| Ubuntu 18.04.5 LTS | Docker version 20.10.14 | v1.22.2 | x86_64 |
Kubernetes叢集架構：k8scludes1作為master節點，k8scludes2，k8scludes3作為worker節點。
| 伺服器 | 作業系統版本 | CPU架構 | 程序 | 功能描述 |
|---|---|---|---|---|
| k8scludes1/192.168.110.128 | Ubuntu 18.04.5 LTS | x86_64 | docker，kube-apiserver，etcd，kube-scheduler，kube-controller-manager，kubelet，kube-proxy，coredns，calico | k8s master節點 |
| k8scludes2/192.168.110.129 | Ubuntu 18.04.5 LTS | x86_64 | docker，kubelet，kube-proxy，calico | k8s worker節點 |
| k8scludes3/192.168.110.130 | Ubuntu 18.04.5 LTS | x86_64 | docker，kubelet，kube-proxy，calico | k8s worker節點 |
二.前言
隨著容器化和微服務架構的普及，Kubernetes 已經成為企業中編排和管理容器的主要平臺。然而，隨著叢集中執行的容器數量和種類的增加，確保容器的安全性變得越來越重要。PodSecurityPolicy（PSP）是 Kubernetes 用於限制 Pod 建立的一組規則，以確保叢集的安全性。在本文中，我們將深入瞭解 PSP 的概念、重要性以及如何在 Kubernetes 叢集中實施。
使用PodSecurityPolicy（PSP）的前提是已經有一套可以正常執行的Kubernetes叢集，關於Kubernetes(k8s)叢集的安裝部署，可以檢視部落格《Ubuntu 安裝部署Kubernetes(k8s)叢集》https://www.cnblogs.com/renshengdezheli/p/17632858.html。
三.PodSecurityPolicy 簡介
PodSecurityPolicy 是一個內建的准入控制器， 允許叢集管理員控制 Pod 規約中涉及安全的敏感內容。
首先，在叢集中建立一個或多個 PodSecurityPolicy 資源來定義 Pod 必須滿足的要求。 然後，建立 RBAC 規則來決定為特定的 Pod 應用哪個 PodSecurityPolicy。 如果 Pod 滿足其 PSP 的要求，則照常被允許進入叢集。 在某些情況下，PSP 還可以修改 Pod 欄位，有效地為這些欄位建立新的預設值。 如果 Pod 不符合 PSP 要求，則被拒絕進入叢集，並且無法執行。
簡單說來，就是PodSecurityPolicy 可以設定pod裡的程序只能以某個UID執行，或者設定pod只能使用某一型別儲存，不能使用其他，比如只能使用nfs儲存。
四.為什麼需要 PodSecurityPolicy
在 Kubernetes 中，我們定義了 Deployment、StatefulSet 和 Service 等資源。 這些資源代表軟體應用程式的構建塊。Kubernetes 叢集中的各種控制器根據這些資源做出反應， 建立更多的 Kubernetes 資源或配置一些軟體或硬體來實現我們的目標。
在大多數 Kubernetes 叢集中，由 RBAC（基於角色的訪問控制）規則 控制對這些資源的訪問。 list、get、create、edit 和 delete 是 RBAC 關心的 API 操作型別， 但 RBAC 不考慮其所控制的資源中加入了哪些設定。例如， Pod 幾乎可以是任何東西，例如簡單的網路伺服器，或者是特權命令提示（提供對底層伺服器節點和所有資料的完全訪問許可權）。 這對 RBAC 來說都是一樣的：Pod 就是 Pod 而已。
要控制叢集中定義的資源允許哪些型別的設定，除了 RBAC 之外，還需要准入控制。 從 Kubernetes 1.3 開始，內建 PodSecurityPolicy 一直被作為 Pod 安全相關欄位的准入控制機制。 使用 PodSecurityPolicy，可以防止“建立 Pod”這個能力自動變成“每個叢集節點上的 root 使用者”， 並且無需部署額外的外部准入控制器。
五.給客戶端授權
建立minsvcbug目錄存放配置檔案。
root@k8scludes1:~# mkdir minsvcbug
root@k8scludes1:~# cd minsvcbug
root@k8scludes1:~/minsvcbug# pwd
/root/minsvcbug
建立minsvcbug名稱空間。
root@k8scludes1:~/minsvcbug# kubectl create ns minsvcbug
namespace/minsvcbug created
切換名稱空間。
root@k8scludes1:~/minsvcbug# kubens minsvcbug
Context "kubernetes-admin@kubernetes" modified.
Active namespace is "minsvcbug".
root@k8scludes1:~/minsvcbug# kubectl get pod
No resources found in minsvcbug namespace.
本次使用etcd2機器作為k8s客戶端，kctom為kubeconfig檔案，裡面的使用者為tom,需要給tom授權，客戶端才能具有相關訪問許可權。
關於kubeconfig檔案的詳細操作，請檢視部落格《Kubernetes(k8s)訪問控制：身份認證》。
關於授權的詳細操作，請檢視部落格《Kubernetes(k8s)訪問控制：許可權管理之RBAC鑑權》。
[root@etcd2 ~]# ls
kctom
檢視minsvcbug名稱空間下的pod，現在tom使用者還沒有minsvcbug名稱空間的pod訪問許可權。
[root@etcd2 ~]# kubectl get pod --kubeconfig=kctom -n minsvcbug
Error from server (Forbidden): pods is forbidden: User "tom" cannot list resource "pods" in API group "" in the namespace "minsvcbug"
給客戶端tom使用者授予建立/刪除/檢視pod的許可權，編輯角色role配置檔案。
root@k8scludes1:~/minsvcbug# vim role1.yaml
#role1角色對pod,svc,secret具有get,list,watch,create,delete許可權
root@k8scludes1:~/minsvcbug# cat role1.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
creationTimestamp: null
name: role1
rules:
- apiGroups:
- ""
resources:
- pods
- services
- secrets
verbs:
- get
- list
- watch
- create
- delete
建立角色role1。
root@k8scludes1:~/minsvcbug# kubectl apply -f role1.yaml
role.rbac.authorization.k8s.io/role1 created
檢視角色。
root@k8scludes1:~/minsvcbug# kubectl get role -o wide
NAME CREATED AT
role1 2022-05-20T03:42:48Z
檢視role1的詳細描述。
root@k8scludes1:~/minsvcbug# kubectl describe role role1
Name: role1
Labels: <none>
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
pods [] [] [get list watch create delete]
secrets [] [] [get list watch create delete]
services [] [] [get list watch create delete]
把角色role1繫結給tom使用者。
root@k8scludes1:~/minsvcbug# kubectl create rolebinding rolebindingtotom --role=role1 --user=tom
rolebinding.rbac.authorization.k8s.io/rolebindingtotom created
root@k8scludes1:~/minsvcbug# kubectl get rolebinding
NAME ROLE AGE
rolebindingtotom Role/role1 11s
給tom使用者授予建立/刪除/檢視pod的許可權之後，在客戶端檢視。可以看到tom使用者對minsvcbug名稱空間的pod有訪問許可權了。
[root@etcd2 ~]# kubectl get pod --kubeconfig=kctom -n minsvcbug
No resources found in minsvcbug namespace.
可以看到tom使用者對default名稱空間的pod沒有訪問許可權。
[root@etcd2 ~]# kubectl get pod --kubeconfig=kctom
Error from server (Forbidden): pods is forbidden: User "tom" cannot list resource "pods" in API group "" in the namespace "default"
在客戶端etcd2機器上建立目錄存放檔案。
[root@etcd2 ~]# mkdir minsvcbug
[root@etcd2 ~]# cd minsvcbug/
[root@etcd2 minsvcbug]# pwd
/root/minsvcbug
編輯pod配置檔案，使用nginx建立pod。
[root@etcd2 minsvcbug]# cat pod.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
#當需要關閉容器時，立即殺死容器而不等待預設的30秒優雅停機時長。
terminationGracePeriodSeconds: 0
containers:
- image: hub.c.163.com/library/nginx:latest
#imagePullPolicy: IfNotPresent：表示如果本地已經存在該映象，則不重新下載；否則從遠端 Docker Hub 下載該映象
imagePullPolicy: IfNotPresent
name: podtest
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
在客戶端使用nginx映象建立pod。
[root@etcd2 minsvcbug]# kubectl apply -f pod.yaml --kubeconfig=kctom -n minsvcbug
pod/podtest created
pod建立成功，說明成功授權了。
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 36s 10.244.218.132 k8scludes2 <none> <none>
[root@etcd2 minsvcbug]# kubectl delete -f pod.yaml --kubeconfig=kctom -n minsvcbug
pod "podtest" deleted
現在tom使用者就具備了minsvcbug名稱空間的建立/刪除/檢視pod的許可權了。
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
No resources found in minsvcbug namespace.
六.啟用PodSecurityPolicy（PSP）
接下來啟用PSP，編輯kube-apiserver.yaml檔案。
root@k8scludes1:~/minsvcbug# ls /etc/kubernetes/manifests/kube-apiserver.yaml
/etc/kubernetes/manifests/kube-apiserver.yaml
啟用PSP需要新增- --enable-admission-plugins=PodSecurityPolicy。
root@k8scludes1:~/minsvcbug# vim /etc/kubernetes/manifests/kube-apiserver.yaml
root@k8scludes1:~/minsvcbug# grep enable-admission-plugins /etc/kubernetes/manifests/kube-apiserver.yaml
- --enable-admission-plugins=NodeRestriction,PodSecurityPolicy
重啟kubelet使配置生效。
root@k8scludes1:~/minsvcbug# systemctl restart kubelet
root@k8scludes1:~/minsvcbug# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8scludes1 Ready control-plane,master 33d v1.22.2
k8scludes2 Ready <none> 33d v1.22.2
k8scludes3 Ready <none> 33d v1.22.2
PSP(pod security policy)預設沒有開啟，如果開啟PSP之後，沒有寫PSP規則的話，任何使用者都建立不了POD，包括管理員。
注意：一定要等k8s正常工作之後再啟動PSP，如果etcd,api都沒啟動好就啟動PSP，則相關pod都啟動不了了 ，等PSP啟動之後再去寫PSP規則。
啟用PSP之後，建立pod必須先訪問PSP規則，透過了PSP規則才能建立POD，否則建立pod失敗。
預設情況下，普通使用者沒有訪問PSP規則的許可權，管理員使用者可以直接訪問PSP規則，建立好PSP規則之後，對普通使用者要做一個RBAC的許可權設定，使其可以訪問某個PSP規則，再決定是不是能夠把pod建立出 來(1.是否有足夠的RBAC許可權 2.建立的POD是否符合PSP規則)。
七.PSP規則之禁止建立特權使用者pod
檢視PSP規則。
root@k8scludes1:~/minsvcbug# kubectl get psp
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
如果沒有任何PSP規則，則任何使用者都不能建立pod。
編輯PSP規則，我們的目的是拒絕特權pod執行，privileged: false 表示不允許特權pod執行。
root@k8scludes1:~/minsvcbug# vim psp-deny-privileges-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-deny-privileges-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-deny-privileges-pod
spec:
# Don't allow privileged pods!
privileged: false
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
volumes:
- '*'
建立psp。
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-deny-privileges-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-deny-privileges-pod created
psp-deny-privileges-pod規則就建立好了。
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-deny-privileges-pod false RunAsAny RunAsAny RunAsAny RunAsAny false *
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
建立psp-deny-privileges-pod的PSP規則之後,在客戶端使用普通使用者建立pod。
編輯pod配置檔案。
[root@etcd2 minsvcbug]# cat pod.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
#當需要關閉容器時，立即殺死容器而不等待預設的30秒優雅停機時長。
terminationGracePeriodSeconds: 0
containers:
- image: hub.c.163.com/library/nginx:latest
#imagePullPolicy: IfNotPresent：表示如果本地已經存在該映象，則不重新下載；否則從遠端 Docker Hub 下載該映象
imagePullPolicy: IfNotPresent
name: podtest
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
建立pod失敗，就算普通使用者tom建立沒有特權使用者的pod，因為tom沒有訪問PSP規則的許可權，所以建立pod失敗。
[root@etcd2 minsvcbug]# kubectl apply -f pod.yaml --kubeconfig=kctom -n minsvcbug
Error from server (Forbidden): error when creating "pod.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: []
[root@etcd2 minsvcbug]# kubectl get pod --kubeconfig=kctom -n minsvcbug
No resources found in minsvcbug namespace.
下面要給tom使用者授予PSP的訪問許可權，對PSP的授權可以是clusterrole，也可以是role，PSP規則是全域性生效的，在其他名稱空間也可以看到，本次使用clusterrole授權。
生成clusterrole-access-psp-rule的yaml檔案，表示：角色clusterrole-access-psp-rule對名為psp-deny-privileges-pod有use許可權。
root@k8scludes1:~# kubectl create clusterrole clusterrole-access-psp-rule --verb=use --resource=psp --resource-name=psp-deny-privileges-pod --dry-run=client -o yaml >clusterrole-access-psp-rule.yaml
生成的yaml檔案如下：
root@k8scludes1:~# vim clusterrole-access-psp-rule.yaml
root@k8scludes1:~# cat clusterrole-access-psp-rule.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
creationTimestamp: null
name: clusterrole-access-psp-rule
rules:
- apiGroups:
- policy
resourceNames:
- psp-deny-privileges-pod
resources:
- podsecuritypolicies
verbs:
- use
建立clusterrole-access-psp-rule。
root@k8scludes1:~# kubectl apply -f clusterrole-access-psp-rule.yaml
clusterrole.rbac.authorization.k8s.io/clusterrole-access-psp-rule created
root@k8scludes1:~# kubectl get clusterrole | grep clusterrole-access-psp-rule
clusterrole-access-psp-rule 2022-05-20T10:21:15Z
檢視角色的詳細描述。
root@k8scludes1:~# kubectl describe clusterrole clusterrole-access-psp-rule
Name: clusterrole-access-psp-rule
Labels: <none>
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
podsecuritypolicies.policy [] [psp-deny-privileges-pod] [use]
把角色clusterrole-access-psp-rule繫結給tom使用者。
root@k8scludes1:~# kubectl create rolebinding rolebinding-clusterrole-access-psp --clusterrole=clusterrole-access-psp-rule --user=tom
rolebinding.rbac.authorization.k8s.io/rolebinding-clusterrole-access-psp created
root@k8scludes1:~# kubectl get rolebinding -o wide
NAME ROLE AGE USERS GROUPS SERVICEACCOUNTS
rolebinding-clusterrole-access-psp ClusterRole/clusterrole-access-psp-rule 15s tom
rolebindingtotom Role/role1 6h40m tom
給tom使用者授予PSP的訪問許可權之後，到客戶端建立無特權使用者的pod。
[root@etcd2 minsvcbug]# kubectl apply -f pod.yaml --kubeconfig=kctom -n minsvcbug
pod/podtest created
pod建立成功，現在tom使用者就可以建立無特權使用者的pod了。
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 21s 10.244.218.145 k8scludes2 <none> <none>
刪除pod。
[root@etcd2 minsvcbug]# kubectl delete pod podtest --kubeconfig=kctom -n minsvcbug
pod "podtest" deleted
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
No resources found in minsvcbug namespace.
編輯pod檔案，securityContext:privileged: true 表示建立特權使用者的pod。
[root@etcd2 minsvcbug]# cat podprivileged.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
#當需要關閉容器時，立即殺死容器而不等待預設的30秒優雅停機時長。
terminationGracePeriodSeconds: 0
containers:
- image: hub.c.163.com/library/nginx:latest
#imagePullPolicy: IfNotPresent：表示如果本地已經存在該映象，則不重新下載；否則從遠端 Docker Hub 下載該映象
imagePullPolicy: IfNotPresent
name: podtest
#建立特權pod
securityContext:
privileged: true
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
建立pod失敗，Privileged containers are not allowed 建立特權容器是不允許的。
[root@etcd2 minsvcbug]# kubectl apply -f podprivileged.yaml --kubeconfig=kctom -n minsvcbug
Error from server (Forbidden): error when creating "podprivileged.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
No resources found in minsvcbug namespace.
刪除psp-deny-privileges-pod規則。
root@k8scludes1:~/minsvcbug# kubectl delete psp psp-deny-privileges-pod
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy "psp-deny-privileges-pod" deleted
root@k8scludes1:~/minsvcbug# kubectl get psp
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
刪除clusterrole-access-psp-rule(不刪除也可以)。
root@k8scludes1:~/minsvcbug# kubectl get clusterrole | grep clusterrole-access-psp-rule
clusterrole-access-psp-rule 2022-05-20T10:21:15Z
root@k8scludes1:~/minsvcbug# kubectl delete clusterrole clusterrole-access-psp-rule
clusterrole.rbac.authorization.k8s.io "clusterrole-access-psp-rule" deleted
刪除rolebinding-clusterrole-access-psp(不刪除也可以)。
root@k8scludes1:~/minsvcbug# kubectl delete rolebinding rolebinding-clusterrole-access-psp
rolebinding.rbac.authorization.k8s.io "rolebinding-clusterrole-access-psp" deleted
總結：
- 在啟用了PSP之後，沒有任何PSP規則的情況下，任何使用者(包括管理員)都不能建立pod；
- 在啟用了PSP之後，且建立了PSP規則，要確保使用者能訪問到此PSP規則，建立clusterrole，讓使用者能use某條PSP規則；
- 能不能成功建立pod，決定權在於是否具備建立pod的RBAC許可權，有對應的RBAC許可權之後，建立的pod是否滿足PSP規則。
八.PSP規則之資料卷
8.1 限定資料卷使用某一個目錄
編輯psp配置檔案，privileged: true表示可以建立特權使用者的pod，volumes:- '*' 表示可以使用所有型別的資料卷，allowedHostPaths:- pathPrefix: "/tmp" 表示資料卷只能掛載/tmp目錄，psp-mount-pod這條PSP規則規定hostPath只能使用/tmp目錄。
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
volumes:
- '*'
allowedHostPaths:
- pathPrefix: "/tmp"
建立PSP。
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod created
root@k8scludes1:~/minsvcbug# kubectl get psp
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false *
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
建立了新的PSP規則之後，需要重新給客戶端的tom使用者授權。
建立clusterrole--psp-mount-rule，角色clusterrole--psp-mount-rule對名為psp-mount-pod的psp規則有use許可權。
root@k8scludes1:~/minsvcbug# kubectl create clusterrole clusterrole--psp-mount-rule --verb=use --resource=psp --resource-name=psp-mount-pod
clusterrole.rbac.authorization.k8s.io/clusterrole--psp-mount-rule created
把角色clusterrole--psp-mount-rule繫結給tom使用者。
root@k8scludes1:~/minsvcbug# kubectl create rolebinding rolebinding-clusterrole-mount-psp --clusterrole=clusterrole--psp-mount-rule --user=tom
rolebinding.rbac.authorization.k8s.io/rolebinding-clusterrole-mount-psp created
給客戶端tom使用者授權之後，編輯pod配置檔案，volumes:- name: v1 hostPath: path: /opt 表示建立名叫v1的資料卷，目錄為/opt ,把v1這個資料卷掛載到容器的/data目錄。securityContext:privileged: true表示建立特權pod。
關於資料卷的詳細操作，請檢視部落格《Kubernetes(k8s)儲存管理之資料卷volumes(一)：volumes的引入和emptyDir資料卷》。
[root@etcd2 minsvcbug]# vim podmount.yaml
[root@etcd2 minsvcbug]# cat podmount.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
#當需要關閉容器時，立即殺死容器而不等待預設的30秒優雅停機時長。
terminationGracePeriodSeconds: 0
#資料卷v1
volumes:
- name: v1
hostPath:
path: /opt
containers:
- image: hub.c.163.com/library/nginx:latest
#imagePullPolicy: IfNotPresent：表示如果本地已經存在該映象，則不重新下載；否則從遠端 Docker Hub 下載該映象
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
#把資料卷v1掛載到/data目錄
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
建立pod失敗，"/opt": is not allowed to be used] ，/opt目錄不被允許使用。
[root@etcd2 minsvcbug]# kubectl apply -f podmount.yaml --kubeconfig=kctom -n minsvcbug
Error from server (Forbidden): error when creating "podmount.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.volumes[0].hostPath.pathPrefix: Invalid value: "/opt": is not allowed to be used]
修改pod配置檔案，把v1資料卷的/opt目錄換成/tmp目錄。
[root@etcd2 minsvcbug]# vim podmount.yaml
[root@etcd2 minsvcbug]# cat podmount.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
#當需要關閉容器時，立即殺死容器而不等待預設的30秒優雅停機時長。
terminationGracePeriodSeconds: 0
volumes:
- name: v1
hostPath:
path: /tmp
containers:
- image: hub.c.163.com/library/nginx:latest
#imagePullPolicy: IfNotPresent：表示如果本地已經存在該映象，則不重新下載；否則從遠端 Docker Hub 下載該映象
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
建立pod。
[root@etcd2 minsvcbug]# kubectl apply -f podmount.yaml --kubeconfig=kctom -n minsvcbug
pod/podtest created
把v1資料卷的/opt目錄換成/tmp目錄之後，pod就建立成功了。
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 14s 10.244.218.134 k8scludes2 <none> <none>
[root@etcd2 minsvcbug]# kubectl delete pod podtest --kubeconfig=kctom -n minsvcbug
pod "podtest" deleted
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
No resources found in minsvcbug namespace.
8.2 限定資料卷的型別為emptyDir
編輯PSP配置檔案，volumes:- 'emptyDir' 限定資料卷的型別為emptyDir。
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
volumes:
- 'emptyDir'
#allowedHostPaths:
#- pathPrefix: "/tmp"
建立PSP。
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod configured
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false emptyDir
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
編輯pod配置檔案，hostPath:path: /tmp 指定hostPath目錄，這裡使用的是hostPath型別的資料卷。
root@k8scludes1:~/minsvcbug# cat podmount.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
hostPath:
path: /tmp
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
建立pod失敗，Invalid value: "hostPath": hostPath volumes are not allowed to be used 表示不能使用hostPath資料卷。
root@k8scludes1:~/minsvcbug# kubectl apply -f podmount.yaml
Error from server (Forbidden): error when creating "podmount.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.volumes[0]: Invalid value: "hostPath": hostPath volumes are not allowed to be used spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed spec.volumes[0]: Invalid value: "hostPath": hostPath volumes are not allowed to be used spec.volumes[1]: Invalid value: "projected": projected volumes are not allowed to be used spec.volumes[0]: Invalid value: "hostPath": hostPath volumes are not allowed to be used]
這次在客戶端建立pod。
修改pod配置檔案，emptyDir: {}表示使用emptyDir資料卷。
[root@etcd2 minsvcbug]# cat podmount.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
建立pod失敗，pod的資料卷型別修改為emptyDir之後，還是報錯：Invalid value: "projected": projected volumes are not allowed to be used]。
[root@etcd2 minsvcbug]# kubectl apply -f podmount.yaml --kubeconfig=kctom -n minsvcbug
Error from server (Forbidden): error when creating "podmount.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.volumes[1]: Invalid value: "projected": projected volumes are not allowed to be used]
先修改PSP規則，資料卷型別還是設定為所有型別volumes:- "*" 。
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
volumes:
- "*"
#- 'emptyDir'
#allowedHostPaths:
#- pathPrefix: "/tmp"
應用PSP。
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod configured
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false *
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
PSP規則的資料卷型別修改為所有型別之後，建立pod成功。
root@k8scludes1:~/minsvcbug# cat podmount.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
root@k8scludes1:~/minsvcbug# kubectl apply -f podmount.yaml
pod/podtest created
root@k8scludes1:~/minsvcbug# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 9s 10.244.218.155 k8scludes2 <none> <none>
使用yaml檔案的格式檢視podtest，可以發現有兩個資料卷，一個是emptyDir型別的，一個是projected型別的。
root@k8scludes1:~/minsvcbug# kubectl get pod podtest -o yaml
apiVersion: v1
kind: Pod
metadata:
......
volumes:
- emptyDir: {}
name: v1
- name: kube-api-access-8xjp8
projected:
defaultMode: 420
sources:
- serviceAccountToken:
expirationSeconds: 3607
path: token
- configMap:
items:
- key: ca.crt
path: ca.crt
name: kube-root-ca.crt
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: namespace
......
更新軟體源。
root@k8scludes1:~/minsvcbug# apt-get update
安裝jq,jq可以使JSON格式更好看。
root@k8scludes1:~/minsvcbug# apt-get install jq -y
root@k8scludes1:~/minsvcbug# which jq
/usr/bin/jq
以json格式的方式檢視podtest，jsonpath='{.spec.volumes}'表示只檢視spec欄位下的volumes，這樣顯示不直觀，使用jq使JSON格式化。
root@k8scludes1:~/minsvcbug# kubectl get pod podtest -o jsonpath='{.spec.volumes}'
[{"emptyDir":{},"name":"v1"},{"name":"kube-api-access-8xjp8","projected":{"defaultMode":420,"sources":[{"serviceAccountToken":{"expirationSeconds":3607,"path":"token"}},{"configMap":{"items":[{"key":"ca.crt","path":"ca.crt"}],"name":"kube-root-ca.crt"}},{"downwardAPI":{"items":[{"fieldRef":{"apiVersion":"v1","fieldPath":"metadata.namespace"},"path":"namespace"}]}}]}}]root@k8scludes1:~/minsvcbug#
可以發現不僅有emptyDir資料卷，還有projected型別的資料卷。
root@k8scludes1:~/minsvcbug# kubectl get pod podtest -o jsonpath='{.spec.volumes}' | jq
[
{
"emptyDir": {},
"name": "v1"
},
{
"name": "kube-api-access-8xjp8",
"projected": {
"defaultMode": 420,
"sources": [
{
"serviceAccountToken": {
"expirationSeconds": 3607,
"path": "token"
}
},
{
"configMap": {
"items": [
{
"key": "ca.crt",
"path": "ca.crt"
}
],
"name": "kube-root-ca.crt"
}
},
{
"downwardAPI": {
"items": [
{
"fieldRef": {
"apiVersion": "v1",
"fieldPath": "metadata.namespace"
},
"path": "namespace"
}
]
}
}
]
}
}
]
所以建立PSP規則的時候不僅要指定emptyDir型別的資料卷，還要指定projected型別的資料卷，修改PSP規則。
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
volumes:
- 'emptyDir'
- 'projected'
#allowedHostPaths:
#- pathPrefix: "/tmp"
應用PSP規則。
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod configured
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false emptyDir,projected
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
刪除pod。
root@k8scludes1:~/minsvcbug# kubectl get pod
NAME READY STATUS RESTARTS AGE
podtest 1/1 Running 0 13m
root@k8scludes1:~/minsvcbug# kubectl delete pod podtest
pod "podtest" deleted
修改pod配置檔案，資料卷型別為emptyDir。
[root@etcd2 minsvcbug]# cat podmount.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
客戶端繼續建立pod，在客戶端成功建立emptyDir型別資料卷的pod。
[root@etcd2 minsvcbug]# kubectl apply -f podmount.yaml --kubeconfig=kctom -n minsvcbug
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 6s 10.244.218.136 k8scludes2 <none> <none>
刪除pod。
[root@etcd2 minsvcbug]# kubectl delete pod podtest --kubeconfig=kctom -n minsvcbug
pod "podtest" deleted
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
No resources found in minsvcbug namespace.
九.PSP規則之指定使用宿主機網路
編輯pod配置檔案，hostNetwork: true 表示是否使用宿主機網路，true表示使用宿主機網路，預設是false，不使用宿主機網路。
hostNetwork: true相當於docker run -dit --name=nginx --network=net nginx 共享宿主機網路空間。
[root@etcd2 minsvcbug]# vim podnetwork.yaml
[root@etcd2 minsvcbug]# cat podnetwork.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
#hostNetwork: true 表示pod使用宿主機網路
hostNetwork: true
restartPolicy: Always
status: {}
建立pod失敗，Invalid value: true: Host network is not allowed to be used]表示 pod不能使用宿主機網路。
[root@etcd2 minsvcbug]# kubectl apply -f podnetwork.yaml --kubeconfig=kctom -n minsvcbug
Error from server (Forbidden): error when creating "podnetwork.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.securityContext.hostNetwork: Invalid value: true: Host network is not allowed to be used]
修改PSP規則，使pod可以使用宿主機網路，hostNetwork: true 表示可以使用宿主機網路。
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
hostNetwork: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
volumes:
- 'emptyDir'
- 'projected'
#allowedHostPaths:
#- pathPrefix: "/tmp"
應用PSP規則。
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod configured
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false emptyDir,projected
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
客戶端再次建立使用宿主機網路的pod，這次pod建立成功。
[root@etcd2 minsvcbug]# kubectl apply -f podnetwork.yaml --kubeconfig=kctom -n minsvcbug
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 17s 192.168.110.129 k8scludes2 <none> <none>
刪除pod。
[root@etcd2 minsvcbug]# kubectl delete pod podtest --kubeconfig=kctom -n minsvcbug
pod "podtest" deleted
在default名稱空間，建立宿主機網路的pod。
root@k8scludes1:~/minsvcbug# kubectl apply -f podnetwork.yaml
pod/podtest created
root@k8scludes1:~/minsvcbug# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 7s 192.168.110.129 k8scludes2 <none> <none>
進入pod。
#可以看到pod裡使用的是宿主機裡的網路
root@k8scludes1:~/minsvcbug# kubectl exec -it podtest -- bash
root@k8scludes2:/# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:0c:29:71:45:c1 brd ff:ff:ff:ff:ff:ff
inet 192.168.110.129/24 brd 192.168.110.255 scope global ens32
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe71:45c1/64 scope link
valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:fd:bc:7b:4b brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
6: tunl0@NONE: <NOARP,UP,LOWER_UP> mtu 1480 qdisc noqueue state UNKNOWN group default qlen 1000
link/ipip 0.0.0.0 brd 0.0.0.0
inet 10.244.218.128/32 scope global tunl0
valid_lft forever preferred_lft forever
#退出pod
root@k8scludes2:/# exit
exit
刪除pod。
root@k8scludes1:~/minsvcbug# kubectl delete pod podtest
pod "podtest" deleted
root@k8scludes1:~/minsvcbug# kubectl get pod
No resources found in minsvcbug namespace.
十.PSP規則之pod使用特定的UID執行
編輯pod配置檔案，hostNetwork: true 表示pod使用宿主機網路 。
root@k8scludes1:~/minsvcbug# cat podnetwork.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
hostNetwork: true
restartPolicy: Always
status: {}
建立pod。
root@k8scludes1:~/minsvcbug# kubectl apply -f podnetwork.yaml
pod/podtest created
root@k8scludes1:~/minsvcbug# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 12s 192.168.110.129 k8scludes2 <none> <none>
進入pod，可以發現pod預設是以root身份執行的，可不可以設定pod以其他身份執行呢？
root@k8scludes1:~/minsvcbug# kubectl exec -it podtest -- bash
#檢視網路，pod使用的是宿主機的網路
root@k8scludes2:/# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:0c:29:71:45:c1 brd ff:ff:ff:ff:ff:ff
inet 192.168.110.129/24 brd 192.168.110.255 scope global ens32
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe71:45c1/64 scope link
valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:4c:05:bc:65 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
6: tunl0@NONE: <NOARP,UP,LOWER_UP> mtu 1480 qdisc noqueue state UNKNOWN group default qlen 1000
link/ipip 0.0.0.0 brd 0.0.0.0
inet 10.244.218.128/32 scope global tunl0
valid_lft forever preferred_lft forever
#pod是以root使用者執行的
root@k8scludes2:/# id
uid=0(root) gid=0(root) groups=0(root)
#退出pod
root@k8scludes2:/# exit
exit
檢視PSP規則。
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false emptyDir,projected
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
編輯PSP規則，runAsUser:rule: RunAsAny 中 runAsUser表示以某種身份執行，RunAsAny表示可以以任何身份執行，rule: MustRunAs ranges:- min: 1000 max: 2000 表示pod必須以1000-2000的UID身份執行。
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
hostNetwork: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
#rule: RunAsAny
rule: MustRunAs
#MustRunAs 表示pod必須以1000-2000的UID身份執行
ranges:
- min: 1000
max: 2000
fsGroup:
rule: RunAsAny
volumes:
- 'emptyDir'
- 'projected'
#allowedHostPaths:
#- pathPrefix: "/tmp"
使PSP規則生效。
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod configured
root@k8scludes1:~/minsvcbug# kubectl get psp
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny MustRunAs RunAsAny RunAsAny false emptyDir,projected
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
因為客戶端etcd2需要進入到pod裡，以及檢視pod的日誌，再給客戶端新增pods/exec， pods/log許可權，直接在Role配置檔案中加入即可。
root@k8scludes1:~/minsvcbug# vim role1.yaml
root@k8scludes1:~/minsvcbug# cat role1.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
creationTimestamp: null
name: role1
rules:
- apiGroups:
- ""
resources:
- pods
#pods/exec， pods/log是新加的許可權
- pods/exec
- pods/log
- services
- secrets
verbs:
- get
- list
- watch
- create
- delete
應用Role。
root@k8scludes1:~/minsvcbug# kubectl apply -f role1.yaml
role.rbac.authorization.k8s.io/role1 configured
檢視role1的許可權資訊。
root@k8scludes1:~/minsvcbug# kubectl describe role role1
Name: role1
Labels: <none>
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
pods/exec [] [] [get list watch create delete]
pods/log [] [] [get list watch create delete]
pods [] [] [get list watch create delete]
secrets [] [] [get list watch create delete]
services [] [] [get list watch create delete]
接下來在客戶端etcd2機器進行測試。
把kubeconfig檔案kctom複製到~/.kube/config，這樣執行命令列就不用每次都指定--kubeconfig了。
[root@etcd2 minsvcbug]# cp kctom ~/.kube/config
[root@etcd2 minsvcbug]# ls ~/.kube/
cache config
[root@etcd2 minsvcbug]# kubectl get pod -n minsvcbug
No resources found in minsvcbug namespace.
編輯pod檔案。
[root@etcd2 minsvcbug]# vim poduid.yaml
[root@etcd2 minsvcbug]# cat poduid.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
#當需要關閉容器時，立即殺死容器而不等待預設的30秒優雅停機時長。
terminationGracePeriodSeconds: 0
#emptyDir型別的資料卷
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
#imagePullPolicy: IfNotPresent：表示如果本地已經存在該映象，則不重新下載；否則從遠端 Docker Hub 下載該映象
imagePullPolicy: IfNotPresent
name: podtest
#建立特權pod
securityContext:
privileged: true
resources: {}
#資料卷掛載
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
#hostNetwork: true 表示pod使用宿主機網路
hostNetwork: true
restartPolicy: Always
status: {}
建立pod。
[root@etcd2 minsvcbug]# kubectl apply -f poduid.yaml -n minsvcbug
pod/podtest created
pod建立失敗。
[root@etcd2 minsvcbug]# kubectl get pod -n minsvcbug
NAME READY STATUS RESTARTS AGE
podtest 0/1 CrashLoopBackOff 1 (5s ago) 6s
檢視pod日誌：podtest裡不允許root來執行，podtest預設執行的是nginx程序，nginx程序需要使用root身份來執行。
[root@etcd2 minsvcbug]# kubectl logs podtest -n minsvcbug
2022/05/24 07:53:04 [warn] 1#1: the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:2
nginx: [warn] the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:2
2022/05/24 07:53:04 [emerg] 1#1: mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)
nginx: [emerg] mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)
刪除pod。
[root@etcd2 minsvcbug]# kubectl delete pod podtest -n minsvcbug
pod "podtest" deleted
編輯pod配置檔案，這次pod不執行nginx的預設程序，因為nginx程序需要root才能正常執行，我們執行sleep 100000命令，sleep 100000命令什麼使用者都可以執行。
[root@etcd2 minsvcbug]# vim poduid.yaml
[root@etcd2 minsvcbug]# cat poduid.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
#執行sleep 100000命令
command: ["sh","-c","sleep 100000"]
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
hostNetwork: true
restartPolicy: Always
status: {}
建立pod。
[root@etcd2 minsvcbug]# kubectl apply -f poduid.yaml -n minsvcbug
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -o wide -n minsvcbug
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 12s 192.168.110.129 k8scludes2 <none> <none>
進入pod。
[root@etcd2 minsvcbug]# kubectl exec -it podtest -n minsvcbug -- bash
#現在預設的使用者UID為1000
I have no name!@k8scludes2:/$ id
uid=1000 gid=0(root) groups=0(root)
#退出pod
I have no name!@k8scludes2:/$ exit
exit
刪除pod。
[root@etcd2 minsvcbug]# kubectl delete pod podtest -n minsvcbug
pod "podtest" deleted
[root@etcd2 minsvcbug]# kubectl get pod -n minsvcbug
No resources found in minsvcbug namespace.
編輯pod配置檔案，securityContext:runAsUser: 3000 表示pod以UID為3000的身份執行。
[root@etcd2 minsvcbug]# vim poduid.yaml
[root@etcd2 minsvcbug]# cat poduid.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
securityContext:
runAsUser: 3000
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
command: ["sh","-c","sleep 100000"]
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
建立pod失敗，UID 3000不在PSP規則規定的{1000 2000}之間，所以建立pod失敗。
[root@etcd2 minsvcbug]# kubectl apply -f poduid.yaml -n minsvcbug
Error from server (Forbidden): error when creating "poduid.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].securityContext.runAsUser: Invalid value: 3000: must be in the ranges: [{1000 2000}]]
編輯pod配置檔案，securityContext:runAsUser: 1500 表示pod以UID為1500的身份執行。
[root@etcd2 minsvcbug]# vim poduid.yaml
[root@etcd2 minsvcbug]# cat poduid.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
securityContext:
runAsUser: 1500
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
command: ["sh","-c","sleep 100000"]
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
建立pod成功。
[root@etcd2 minsvcbug]# kubectl apply -f poduid.yaml -n minsvcbug
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -n minsvcbug
NAME READY STATUS RESTARTS AGE
podtest 1/1 Running 0 12s
進入pod。
[root@etcd2 minsvcbug]# kubectl exec -it podtest -n minsvcbug -- bash
#UID 1500在{1000 2000}範圍，所以pod執行成功，pod執行程序UID也為1500
I have no name!@podtest:/$ id
uid=1500 gid=0(root) groups=0(root)
#退出pod
I have no name!@podtest:/$ exit
exit
十一.PSP規則之指定hostport埠
修改一下vim設定，設定為貼上模式之後，貼上格式看著更規範。
root@k8scludes1:~/minsvcbug# vim /etc/vim/vimrc
root@k8scludes1:~/minsvcbug# tail -1 /etc/vim/vimrc
set paste
編輯PSP規則，因為nginx程序需要root身份才能執行，RUNASUSER 變為RunAsAny，RunAsAny表示可以以任何身份執行。
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
#hostNetwork: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
#rule: MustRunAs
#ranges:
#- min: 1000
# max: 2000
fsGroup:
rule: RunAsAny
volumes:
- '*'
#- 'emptyDir'
#- 'projected'
#allowedHostPaths:
#- pathPrefix: "/tmp"
應用PSP規則。
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod configured
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false *
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
去客戶端建立pod。
編輯pod配置檔案，containerPort: 80 表示容器埠80，hostPort: 8080表示宿主機埠8080 ，表示把nginx容器的80埠對映為宿主機的8080埠。
這樣建立的pod類似於docker埠對映：docker run -p 80 nginx 只寫一個埠80的話表示容器埠，docker run -p 80:80 nginx 80:80 表示宿主機埠:容器埠。
[root@etcd2 minsvcbug]# vim podport.yaml
[root@etcd2 minsvcbug]# cat podport.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
resources: {}
volumeMounts:
- name: v1
mountPath: /data
ports:
- name: http
containerPort: 80
hostPort: 8080
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
建立pod的時候發現tom使用者沒有pods的patch許可權，我們去授權一下。
[root@etcd2 minsvcbug]# kubectl apply -f podport.yaml -n minsvcbug
for: "podport.yaml": pods "podtest" is forbidden: User "tom" cannot patch resource "pods" in API group "" in the namespace "minsvcbug"
修改Role配置檔案，新增patch許可權。
root@k8scludes1:~/minsvcbug# vim role1.yaml
root@k8scludes1:~/minsvcbug# cat role1.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
creationTimestamp: null
name: role1
rules:
- apiGroups:
- ""
resources:
- pods
- pods/exec
- pods/log
- services
- secrets
verbs:
- get
- list
- watch
- create
- delete
#新增patch許可權
- patch
應用角色。
root@k8scludes1:~/minsvcbug# kubectl apply -f role1.yaml
role.rbac.authorization.k8s.io/role1 configured
客戶端繼續建立pod。
建立pod失敗，Invalid value: 8080: Host port 8080 is not allowed to be used. Allowed ports: [] 表示PSP規則規定的Host port為空，不能使用hostport埠，我們需要修改PSP規則。
[root@etcd2 minsvcbug]# kubectl get pod -n minsvcbug
No resources found in minsvcbug namespace.
[root@etcd2 minsvcbug]# kubectl apply -f podport.yaml -n minsvcbug
Error from server (Forbidden): error when creating "podport.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].hostPort: Invalid value: 8080: Host port 8080 is not allowed to be used. Allowed ports: []]
修改PSP規則，hostPorts:- min: 1000 max: 8090表示hostport的埠範圍為1000-8090。
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
#hostNetwork: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
#rule: MustRunAs
#ranges:
#- min: 1000
# max: 2000
fsGroup:
rule: RunAsAny
volumes:
- '*'
#- 'emptyDir'
#- 'projected'
#allowedHostPaths:
#- pathPrefix: "/tmp"
#指定hostport的埠範圍為1000-8090
hostPorts:
- min: 1000
max: 8090
應用PSP規則。
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod configured
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false *
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
現在客戶端建立pod就成功了。
[root@etcd2 minsvcbug]# kubectl apply -f podport.yaml -n minsvcbug
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -o wide -n minsvcbug
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 53s 10.244.218.150 k8scludes2 <none> <none>
訪問nginx的方法為：curl k8scludes2的ip:8080，nginx訪問成功。
[root@etcd2 minsvcbug]# curl 192.168.110.129:8080
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
刪除pod。
[root@etcd2 minsvcbug]# kubectl delete pod podtest -n minsvcbug
pod "podtest" deleted
[root@etcd2 minsvcbug]# kubectl get pod -o wide -n minsvcbug
No resources found in minsvcbug namespace.
刪除rolebinding。
root@k8scludes1:~# cd minsvcbug/
root@k8scludes1:~/minsvcbug# kubectl get rolebinding
NAME ROLE AGE
rolebinding-clusterrole-mount-psp ClusterRole/clusterrole--psp-mount-rule 3d21h
rolebindingtotom Role/role1 5d3h
root@k8scludes1:~/minsvcbug# kubectl delete rolebinding rolebinding-clusterrole-mount-psp
rolebinding.rbac.authorization.k8s.io "rolebinding-clusterrole-mount-psp" deleted
root@k8scludes1:~/minsvcbug# kubectl get rolebinding
NAME ROLE AGE
rolebindingtotom Role/role1 5d3h
root@k8scludes1:~/minsvcbug# kubectl delete rolebinding rolebindingtotom
rolebinding.rbac.authorization.k8s.io "rolebindingtotom" deleted
root@k8scludes1:~/minsvcbug# kubectl get rolebinding
No resources found in minsvcbug namespace.
刪除role。
root@k8scludes1:~/minsvcbug# kubectl get role
NAME CREATED AT
role1 2022-05-20T03:42:48Z
root@k8scludes1:~/minsvcbug# kubectl delete role role1
role.rbac.authorization.k8s.io "role1" deleted
root@k8scludes1:~/minsvcbug# kubectl get role
No resources found in minsvcbug namespace.
刪除PSP規則。
root@k8scludes1:~/minsvcbug# kubectl get psp
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false *
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
#刪除psp-mount-pod規則
root@k8scludes1:~/minsvcbug# kubectl delete psp psp-mount-pod
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy "psp-mount-pod" deleted
十二.PSP規則實戰1
我們現在要解決的問題是：有三個名稱空間ns1,ns2,ns3，tom使用者只有在ns1名稱空間裡可以建立特權使用者pod，其他名稱空間都不能建立特權使用者pod，需要怎麼做？
建立3個名稱空間。
root@k8scludes1:~/minsvcbug# kubectl create ns ns1
namespace/ns1 created
root@k8scludes1:~/minsvcbug# kubectl create ns ns2
namespace/ns2 created
root@k8scludes1:~/minsvcbug# kubectl create ns ns3
namespace/ns3 created
建立一個普通的RBAC許可權，讓客戶端tom使用者可以在所有的名稱空間裡建立和刪除pod,使用clusterrole和clusterrolebinding。
編輯ClusterRole配置檔案，clusterole1具有對pod的查詢/建立/刪除許可權，因為需要看日誌和進入到pod裡，新增了pods/exec和pods/log資源。
root@k8scludes1:~/minsvcbug# vim clusterrole1.yaml
root@k8scludes1:~/minsvcbug# cat clusterrole1.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
creationTimestamp: null
name: clusterole1
rules:
- apiGroups:
- ""
resources:
- pods
- pods/exec
- pods/log
verbs:
- get
- list
- watch
- create
- delete
- patch
應用ClusterRole。
root@k8scludes1:~/minsvcbug# kubectl apply -f clusterrole1.yaml
clusterrole.rbac.authorization.k8s.io/clusterole1 created
root@k8scludes1:~/minsvcbug# kubectl get clusterrole | grep clusterole1
clusterole1 2022-05-25T07:31:28Z
檢視clusterole1許可權資訊。
root@k8scludes1:~/minsvcbug# kubectl describe clusterrole clusterole1
Name: clusterole1
Labels: <none>
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
pods/exec [] [] [get list watch create delete patch]
pods/log [] [] [get list watch create delete patch]
pods [] [] [get list watch create delete patch]
把clusterole1繫結給tom使用者，clusterole1和clusterrolebinding1的目的是讓tom使用者可以在所有名稱空間裡建立和刪除pod。
現在客戶端的tom使用者在所有名稱空間都可以建立和刪除pod了。
root@k8scludes1:~/minsvcbug# kubectl create clusterrolebinding clusterrolebinding1 --clusterrole=clusterole1 --user=tom
clusterrolebinding.rbac.authorization.k8s.io/clusterrolebinding1 created
root@k8scludes1:~/minsvcbug# kubectl get clusterrolebinding | grep clusterrolebinding1
clusterrolebinding1 ClusterRole/clusterole1 21s
建立兩條psp規則，一條PSP規則禁止建立特權使用者pod，另外一條PSP規則允許建立特權使用者pod。
編輯psp-allowpri-pod規則，psp-allowpri-pod規則允許建立特權使用者pod，privileged: true。
root@k8scludes1:~/minsvcbug# vim psp-allowpri-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-allowpri-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-allowpri-pod
spec:
#allow privileged pods!
privileged: true
#hostNetwork: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
#rule: MustRunAs
#ranges:
#- min: 1000
# max: 2000
fsGroup:
rule: RunAsAny
volumes:
- '*'
#- 'emptyDir'
#- 'projected'
#allowedHostPaths:
#- pathPrefix: "/tmp"
#hostPorts:
#- min: 1000
# max: 8090
編輯psp-denypri-pod規則，psp-allowpri-pod規則禁止建立特權使用者pod，privileged: false。
root@k8scludes1:~/minsvcbug# vim psp-denypri-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-denypri-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-denypri-pod
spec:
#allow privileged pods!
privileged: false
#hostNetwork: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
#rule: MustRunAs
#ranges:
#- min: 1000
# max: 2000
fsGroup:
rule: RunAsAny
volumes:
- '*'
#- 'emptyDir'
#- 'projected'
#allowedHostPaths:
#- pathPrefix: "/tmp"
#hostPorts:
#- min: 1000
# max: 8090
建立PSP規則。
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-allowpri-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-allowpri-pod created
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-denypri-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-denypri-pod created
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-allowpri-pod true RunAsAny RunAsAny RunAsAny RunAsAny false *
psp-denypri-pod false RunAsAny RunAsAny RunAsAny RunAsAny false *
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
為了達到tom使用者在所有的namespace裡都不能建立特權使用者pod的效果：需要透過clusterrole和clusterrolebinding把psp-denypri-pod規則授權給tom使用者。
現在的clusterrole，clusterrolebinding如下：
root@k8scludes1:~/minsvcbug# kubectl get clusterrole | grep cluste
cert-manager-controller-clusterissuers 2022-04-24T17:07:47Z
cluster-admin 2022-04-16T14:57:55Z
clusterole1 2022-05-25T07:31:28Z
clusterrole--psp-mount-rule 2022-05-21T09:42:10Z
system:controller:clusterrole-aggregation-controller 2022-04-16T14:57:55Z
root@k8scludes1:~/minsvcbug# kubectl get clusterrolebinding | grep cluster
cert-manager-controller-clusterissuers ClusterRole/cert-manager-controller-clusterissuers 30d
cluster-admin ClusterRole/cluster-admin 38d
clusterrolebinding1 ClusterRole/clusterole1 30m
system:controller:clusterrole-aggregation-controller ClusterRole/system:controller:clusterrole-aggregation-controller 38d
建立clusterrole，角色clusterrole--psp-denypri-rule對名為psp-denypri-pod的psp規則有use許可權。
root@k8scludes1:~/minsvcbug# kubectl create clusterrole clusterrole--psp-denypri-rule --verb=use --resource=psp --resource-name=psp-denypri-pod
clusterrole.rbac.authorization.k8s.io/clusterrole--psp-denypri-rule created
檢視clusterrole--psp-denypri-rule的描述資訊。
root@k8scludes1:~/minsvcbug# kubectl describe clusterrole clusterrole--psp-denypri-rule
Name: clusterrole--psp-denypri-rule
Labels: <none>
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
podsecuritypolicies.policy [] [psp-denypri-pod] [use]
把角色clusterrole--psp-denypri-rule繫結給tom使用者，clusterrole--psp-denypri-rule和clusterrolebinding-clusterrole-denypri-psp的目的是讓tom具有psp-denypri-pod規則的use許可權，這樣tom在所有的名稱空間就不能建立特權使用者pod了。
root@k8scludes1:~/minsvcbug# kubectl create clusterrolebinding clusterrolebinding-clusterrole-denypri-psp --clusterrole=clusterrole--psp-denypri-rule --user=tom
clusterrolebinding.rbac.authorization.k8s.io/clusterrolebinding-clusterrole-denypri-psp created
在客戶端進行驗證，首先測試在多個namespace建立和刪除pod。
編輯pod配置檔案，這是非特權使用者pod。
[root@etcd2 minsvcbug]# vim podnotpri.yaml
[root@etcd2 minsvcbug]# cat podnotpri.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
resources: {}
volumeMounts:
- name: v1
mountPath: /data
#ports:
#- name: http
# containerPort: 80
# hostPort: 8080
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
tom使用者在ns1，ns2，ns3都能建立非特權pod。
[root@etcd2 minsvcbug]# kubectl apply -f podnotpri.yaml -n ns1
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -n ns1
NAME READY STATUS RESTARTS AGE
podtest 1/1 Running 0 12s
[root@etcd2 minsvcbug]# kubectl apply -f podnotpri.yaml -n ns2
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -n ns2
NAME READY STATUS RESTARTS AGE
podtest 1/1 Running 0 7s
[root@etcd2 minsvcbug]# kubectl apply -f podnotpri.yaml -n ns3
pod/podtest created
tom使用者在ns1，ns2，ns3都能刪除pod。
[root@etcd2 minsvcbug]# kubectl delete -f podnotpri.yaml -n ns1
pod "podtest" deleted
[root@etcd2 minsvcbug]# kubectl delete -f podnotpri.yaml -n ns2
pod "podtest" deleted
#這樣tom使用者就可以在ns1，ns2，ns3建立和刪除pod了
[root@etcd2 minsvcbug]# kubectl delete -f podnotpri.yaml -n ns3
pod "podtest" deleted
編輯pod配置檔案，securityContext:privileged: true 表示建立特權使用者pod。
[root@etcd2 minsvcbug]# vim podpri.yaml
[root@etcd2 minsvcbug]# cat podpri.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
#securityContext:privileged: true 表示建立特權使用者pod
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
#ports:
#- name: http
# containerPort: 80
# hostPort: 8080
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
可以發現，tom使用者可以在ns1，ns2，ns3名稱空間建立pod，但是不能建立特權pod。
[root@etcd2 minsvcbug]# kubectl apply -f podpri.yaml -n ns1
Error from server (Forbidden): error when creating "podpri.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]
[root@etcd2 minsvcbug]# kubectl apply -f podpri.yaml -n ns2
Error from server (Forbidden): error when creating "podpri.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]
[root@etcd2 minsvcbug]# kubectl apply -f podpri.yaml -n ns3
Error from server (Forbidden): error when creating "podpri.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]
建立clusterrole，角色clusterrole--psp-allowpri-rule對名為psp-allowpri-pod的psp規則有use許可權。
root@k8scludes1:~/minsvcbug# kubectl create clusterrole clusterrole--psp-allowpri-rule --verb=use --resource=psp --resource-name=psp-allowpri-pod
clusterrole.rbac.authorization.k8s.io/clusterrole--psp-allowpri-rule created
把角色clusterrole--psp-allowpri-rule繫結給tom使用者，這樣tom使用者既可以訪問psp-allowpri-pod規則，也可以訪問psp-denypri-pod規則。
root@k8scludes1:~/minsvcbug# kubectl create rolebinding rolebinding-clusterrole-allowpri-psp --clusterrole=clusterrole--psp-allowpri-rule --user=tom
rolebinding.rbac.authorization.k8s.io/rolebinding-clusterrole-allowpri-psp created
刪除rolebinding-clusterrole-allowpri-psp，因為沒有指定是哪個名稱空間，不指定的話預設是當前名稱空間。
root@k8scludes1:~/minsvcbug# kubectl delete rolebinding rolebinding-clusterrole-allowpri-psp
rolebinding.rbac.authorization.k8s.io "rolebinding-clusterrole-allowpri-psp" deleted
重新進行角色繫結，這樣tom使用者在ns1名稱空間既可以訪問psp-allowpri-pod規則，也可以訪問psp-denypri-pod規則。
注意：對於PSP規則，拒絕和允許同時出現的話，允許優先順序比較高。
root@k8scludes1:~/minsvcbug# kubectl create rolebinding rolebinding-clusterrole-allowpri-psp --clusterrole=clusterrole--psp-allowpri-rule --user=tom -n ns1
rolebinding.rbac.authorization.k8s.io/rolebinding-clusterrole-allowpri-psp created
下面在客戶端進行測試。
編輯pod配置檔案，下面的配置檔案是為了建立特權使用者pod。
[root@etcd2 minsvcbug]# cat podpri.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
#建立特權pod
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
#ports:
#- name: http
# containerPort: 80
# hostPort: 8080
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
在ns1名稱空間建立特權pod成功。
[root@etcd2 minsvcbug]# kubectl apply -f podpri.yaml -n ns1
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -o wide -n ns1
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 12s 10.244.218.148 k8scludes2 <none> <none>
在ns2，ns3名稱空間建立特權pod失敗。
[root@etcd2 minsvcbug]# kubectl apply -f podpri.yaml -n ns2
Error from server (Forbidden): error when creating "podpri.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]
[root@etcd2 minsvcbug]# kubectl apply -f podpri.yaml -n ns3
Error from server (Forbidden): error when creating "podpri.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]
這樣就實現了tom使用者可以在ns1名稱空間建立特權使用者pod，在其他名稱空間都不可以建立特權pod的需求。
清理一下pod。
[root@etcd2 minsvcbug]# kubectl delete -f podpri.yaml -n ns1
pod "podtest" deleted
[root@etcd2 minsvcbug]# kubectl get pod -o wide -n ns1
No resources found in ns1 namespace.
[root@etcd2 minsvcbug]# kubectl get pod -o wide -n ns2
No resources found in ns2 namespace.
[root@etcd2 minsvcbug]# kubectl get pod -o wide -n ns3
No resources found in ns3 namespace.
十三.PSP規則實戰2:建立deployment
現在沒有deploy，使用hub.c.163.com/library/nginx:latest映象建立一個deploy，先生成yaml檔案。關於deploy的詳細操作，請檢視部落格《Kubernetes(k8s)控制器(一)：deployment》。
root@k8scludes1:~/minsvcbug# kubectl get deploy
No resources found in minsvcbug namespace.
root@k8scludes1:~/minsvcbug# kubectl create deployment nginxdeploy --image=hub.c.163.com/library/nginx:latest --dry-run=client -o yaml >nginxdeploy.yaml
生成的deploy配置檔案如下：
root@k8scludes1:~/minsvcbug# vim nginxdeploy.yaml
root@k8scludes1:~/minsvcbug# cat nginxdeploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
#這個labels是deployment的標籤
labels:
app: nginxdeploy
#deployment名字
name: nginxdeploy
spec:
#replicas: 1 表示副本數為1，只生成一個pod
replicas: 1
#selector標籤選擇器
selector:
matchLabels:
app: nginxdeploy
strategy: {}
#template下面是pod的模板
template:
metadata:
creationTimestamp: null
#這個labels表示pod標籤
labels:
app: nginxdeploy
spec:
#當需要關閉容器時，立即殺死容器而不等待預設的30秒優雅停機時長。
terminationGracePeriodSeconds: 0
containers:
- image: hub.c.163.com/library/nginx:latest
#imagePullPolicy: IfNotPresent：表示如果本地已經存在該映象，則不重新下載；否則從遠端 Docker Hub 下載該映象
imagePullPolicy: IfNotPresent
name: nginx
resources: {}
status: {}
建立deploy。
root@k8scludes1:~/minsvcbug# kubectl apply -f nginxdeploy.yaml
deployment.apps/nginxdeploy created
發現deploy不正常。
root@k8scludes1:~/minsvcbug# kubectl get deploy -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginxdeploy 0/1 0 0 18s nginx hub.c.163.com/library/nginx:latest app=nginxdeploy
pod也沒有生成。
root@k8scludes1:~/minsvcbug# kubectl get pod -o wide
No resources found in minsvcbug namespace.
注意：當我們建立一個deploy，比如nginxdeploy，則系統會為我們建立一個replicaset，名字為nginxdeploy-XXX，我們在deploy裡指定的副本，本質上是由replicaset建立出來的，pod名字為nginxdeploy-XXX-yyyy。
可以發現，deployment和replicaset都沒有ready。
root@k8scludes1:~/minsvcbug# kubectl get all -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
deployment.apps/nginxdeploy 0/1 0 0 7m17s nginx hub.c.163.com/library/nginx:latest app=nginxdeploy
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
replicaset.apps/nginxdeploy-5d458fcbfc 1 0 0 7m17s nginx hub.c.163.com/library/nginx:latest app=nginxdeploy,pod-template-hash=5d458fcbfc
在kube-system名稱空間裡有很多個給控制器使用的sa。
root@k8scludes1:~/minsvcbug# kubectl get sa -n kube-system
NAME SECRETS AGE
attachdetach-controller 1 38d
bootstrap-signer 1 38d
calico-kube-controllers 1 38d
calico-node 1 38d
certificate-controller 1 38d
......
persistent-volume-binder 1 38d
pod-garbage-collector 1 38d
pv-protection-controller 1 38d
pvc-protection-controller 1 38d
replicaset-controller 1 38d
replication-controller 1 38d
resourcequota-controller 1 38d
root-ca-cert-publisher 1 38d
service-account-controller 1 38d
service-controller 1 38d
statefulset-controller 1 38d
token-cleaner 1 38d
ttl-after-finished-controller 1 38d
ttl-controller 1 38d
只檢視和controller有關的SA。
root@k8scludes1:~/minsvcbug# kubectl get sa -n kube-system | egrep '*controller'
attachdetach-controller 1 38d
calico-kube-controllers 1 38d
certificate-controller 1 38d
clusterrole-aggregation-controller 1 38d
cronjob-controller 1 38d
daemon-set-controller 1 38d
deployment-controller 1 38d
disruption-controller 1 38d
endpoint-controller 1 38d
endpointslice-controller 1 38d
endpointslicemirroring-controller 1 38d
ephemeral-volume-controller 1 38d
expand-controller 1 38d
job-controller 1 38d
namespace-controller 1 38d
node-controller 1 38d
pv-protection-controller 1 38d
pvc-protection-controller 1 38d
replicaset-controller 1 38d
replication-controller 1 38d
resourcequota-controller 1 38d
service-account-controller 1 38d
service-controller 1 38d
statefulset-controller 1 38d
ttl-after-finished-controller 1 38d
ttl-controller 1 38d
控制器建立pod的過程：在kube-system裡有很多個給控制器使用的sa,啟用PSP之後，現在控制器的SA都訪問不了PSP規則，就建立不了pod,只有控制器的sa能訪問PSP規則，才能正常建立pod。
現在需要給控制器SA進行授權，使控制器能訪問PSP規則。
進行角色繫結，表示kube-system名稱空間裡的所有serviceaccounts都可以訪問psp-allowpri-pod規則。
root@k8scludes1:~/minsvcbug# kubectl get clusterrole | grep cluster
cert-manager-controller-clusterissuers 2022-04-24T17:07:47Z
cluster-admin 2022-04-16T14:57:55Z
clusterole1 2022-05-25T07:31:28Z
clusterrole--psp-allowpri-rule 2022-05-25T08:57:00Z
clusterrole--psp-denypri-rule 2022-05-25T08:09:01Z
system:controller:clusterrole-aggregation-controller 2022-04-16T14:57:55Z
root@k8scludes1:~/minsvcbug# kubectl create rolebinding rolebind-allowpri-psp-kube-system --clusterrole=clusterrole--psp-allowpri-rule --group=system:serviceaccounts -n kube-system --dry-run=client -o yaml > rolebind-allowpri-kube-system.yaml
root@k8scludes1:~/minsvcbug# kubectl apply -f rolebind-allowpri-kube-system.yaml
rolebinding.rbac.authorization.k8s.io/rolebind-allowpri-psp-kube-system created
刪除建立失敗的nginxdeploy。
root@k8scludes1:~/minsvcbug# kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
nginxdeploy 0/1 0 0 3h13m
root@k8scludes1:~/minsvcbug# kubectl delete deploy nginxdeploy
deployment.apps "nginxdeploy" deleted
root@k8scludes1:~/minsvcbug# kubectl get deployments
No resources found in minsvcbug namespace.
再次建立deploy。
root@k8scludes1:~/minsvcbug# kubectl apply -f nginxdeploy.yaml
deployment.apps/nginxdeploy created
deploy還是沒有建立成功。
root@k8scludes1:~/minsvcbug# kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
nginxdeploy 0/1 0 0 8s
root@k8scludes1:~/minsvcbug# kubectl get pod
No resources found in minsvcbug namespace.
檢視nginxdeploy描述資訊：replicaset-controller Error creating: pods "nginxdeploy-5d458fcbfc-" is forbidden: PodSecurityPolicy: unable to admit pod: []，可以發現許可權還是不夠。
root@k8scludes1:~/minsvcbug# kubectl describe deployment nginxdeploy
Name: nginxdeploy
Namespace: minsvcbug
......
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
ReplicaFailure True FailedCreate
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedCreate 34s (x14 over 75s) replicaset-controller Error creating: pods "nginxdeploy-5d458fcbfc-" is forbidden: PodSecurityPolicy: unable to admit pod: []
kubectl get ev檢視事件。
root@k8scludes1:~/minsvcbug# kubectl get ev
LAST SEEN TYPE REASON OBJECT MESSAGE
11m Warning FailedCreate replicaset/nginxdeploy-5d458fcbfc Error creating: pods "nginxdeploy-5d458fcbfc-" is forbidden: PodSecurityPolicy: unable to admit pod: []
101s Warning FailedCreate replicaset/nginxdeploy-5d458fcbfc Error creating: pods "nginxdeploy-5d458fcbfc-" is forbidden: PodSecurityPolicy: unable to admit pod: []
4m25s Normal ScalingReplicaSet deployment/nginxdeploy Scaled up replica set nginxdeploy-5d458fcbfc to 1
刪除nginxdeploy。
root@k8scludes1:~/minsvcbug# kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
nginxdeploy 0/1 0 0 5m18s
root@k8scludes1:~/minsvcbug# kubectl delete deploy nginxdeploy
deployment.apps "nginxdeploy" deleted
root@k8scludes1:~/minsvcbug# kubectl get deployments
No resources found in minsvcbug namespace.
root@k8scludes1:~/minsvcbug# kubectl get all
No resources found in minsvcbug namespace.
編輯ClusterRoleBinding配置檔案，把RoleBinding變為ClusterRoleBinding，使其全域性生效，剛才的角色繫結rolebind-allowpri-psp-kube-system是錯誤的。
root@k8scludes1:~/minsvcbug# vim rolebind-allowpri-kube-system.yaml
root@k8scludes1:~/minsvcbug# cat rolebind-allowpri-kube-system.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
creationTimestamp: null
name: clusterrolebind-allowpri-psp-kube-system
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: clusterrole--psp-allowpri-rule
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:serviceaccounts
刪除rolebind-allowpri-psp-kube-system。
root@k8scludes1:~/minsvcbug# kubectl get rolebinding -n kube-system | grep rolebind-allowpri-psp-kube-system
rolebind-allowpri-psp-kube-system ClusterRole/clusterrole--psp-allowpri-rule 12m
root@k8scludes1:~/minsvcbug# kubectl delete rolebinding rolebind-allowpri-psp-kube-system -n kube-system
rolebinding.rbac.authorization.k8s.io "rolebind-allowpri-psp-kube-system" deleted
root@k8scludes1:~/minsvcbug# kubectl get rolebinding -n kube-system | grep rolebind-allowpri-psp-kube-system
建立clusterrolebinding。
root@k8scludes1:~/minsvcbug# kubectl apply -f rolebind-allowpri-kube-system.yaml
clusterrolebinding.rbac.authorization.k8s.io/clusterrolebind-allowpri-psp-kube-system created
root@k8scludes1:~/minsvcbug# kubectl get clusterrolebinding -n kube-system | egrep clusterrolebind-allowpri-psp-kube-system
clusterrolebind-allowpri-psp-kube-system ClusterRole/clusterrole--psp-allowpri-rule 45s
建立deployment。
root@k8scludes1:~/minsvcbug# kubectl apply -f nginxdeploy.yaml
deployment.apps/nginxdeploy created
現在deploy就建立成功了。
root@k8scludes1:~/minsvcbug# kubectl get all -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/nginxdeploy-5d458fcbfc-rs42b 1/1 Running 0 12s 10.244.218.160 k8scludes2 <none> <none>
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
deployment.apps/nginxdeploy 1/1 1 1 12s nginx hub.c.163.com/library/nginx:latest app=nginxdeploy
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
replicaset.apps/nginxdeploy-5d458fcbfc 1 1 1 12s nginx hub.c.163.com/library/nginx:latest app=nginxdeploy,pod-template-hash=5d458fcbfc
root@k8scludes1:~/minsvcbug# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginxdeploy-5d458fcbfc-rs42b 1/1 Running 0 71s
clusterrolebinding全域性生效，所有名稱空間都能看到。
root@k8scludes1:~/minsvcbug# kubectl get clusterrolebinding | egrep clusterrolebind-allowpri-psp-kube-system
clusterrolebind-allowpri-psp-kube-system ClusterRole/clusterrole--psp-allowpri-rule 4m43s
root@k8scludes1:~/minsvcbug# kubectl get clusterrolebinding -n default | egrep clusterrolebind-allowpri-psp-kube-system
clusterrolebind-allowpri-psp-kube-system ClusterRole/clusterrole--psp-allowpri-rule 4m53s
刪除nginxdeploy。
root@k8scludes1:~/minsvcbug# kubectl delete deploy nginxdeploy
deployment.apps "nginxdeploy" deleted
root@k8scludes1:~/minsvcbug# kubectl get all
No resources found in minsvcbug namespace.
十四.總結
PodSecurityPolicy 是 Kubernetes 叢集安全的重要組成部分。透過定義 Pod 的安全策略，管理員可以確保叢集中的 Pod 遵循特定的安全標準，從而降低潛在的安全風險。本文介紹了 PSP 的概念、重要性以及如何在 Kubernetes 叢集中實施。希望這些資訊能夠幫助您更好地保護您的 Kubernetes 叢集。