Kubernetes雲原生儲存解決方案openebs部署實踐-3.10.0版本（helm部署）
記錄在k8s 1.19.0叢集環境下安裝openebs 3.10.0。
環境資訊如下：
[root@k8s-master ~]# cat /etc/centos-release
CentOS Linux release 7.9.2009 (Core)
[root@k8s-master ~]# uname -a
Linux k8s-master 3.10.0-1160.71.1.el7.x86_64 #1 SMP Tue Jun 28 15:37:28 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
[root@k8s-master ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
k8s-master Ready master 16d v1.19.0
k8s-node1 Ready worker 16d v1.19.0
k8s-node2 Ready worker 16d v1.19.0
[root@k8s-master ~]# helm version
version.BuildInfo{Version:"v3.6.1", GitCommit:"61d8e8c4a6f95540c15c6a65f36a6dd0a45e7a2f", GitTreeState:"clean", GoVersion:"go1.16.5"}
1. 安裝openebs
openebs支援kubectl基於yaml安裝，也可以使用helm進行安裝。本文基於helm方式。
- 配置helm倉庫
[root@k8s-master ~]# helm repo add openebs https://openebs.github.io/charts
[root@k8s-master ~]# helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "grafana" chart repository
...Successfully got an update from the "ingress-nginx" chart repository
...Successfully got an update from the "bitnami" chart repository
...Successfully got an update from the "prometheus-community" chart repository
...Successfully got an update from the "openebs" chart repository
Update Complete. ⎈Happy Helming!⎈
- 安裝 openebs chart
[root@k8s-master ~]# helm install openebs --namespace openebs openebs/openebs --create-namespace
NAME: openebs
LAST DEPLOYED: Fri Jun 28 15:14:46 2024
NAMESPACE: openebs
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Successfully installed OpenEBS.
Check the status by running: kubectl get pods -n openebs
The default values will install NDM and enable OpenEBS hostpath and device
storage engines along with their default StorageClasses. Use `kubectl get sc`
to see the list of installed OpenEBS StorageClasses.
**Note**: If you are upgrading from the older helm chart that was using cStor
and Jiva (non-csi) volumes, you will have to run the following command to include
the older provisioners:
helm upgrade openebs openebs/openebs \
--namespace openebs \
--set legacy.enabled=true \
--reuse-values
For other engines, you will need to perform a few more additional steps to
enable the engine, configure the engines (e.g. creating pools) and create
StorageClasses.
For example, cStor can be enabled using commands like:
helm upgrade openebs openebs/openebs \
--namespace openebs \
--set cstor.enabled=true \
--reuse-values
For more information,
- view the online documentation at https://openebs.io/docs or
- connect with an active community on Kubernetes slack #openebs channel.
- 檢查部署的資源：
# 部署的chart
[root@k8s-master ~]# helm ls -n openebs
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
openebs openebs 1 2024-06-28 15:14:46.982427602 +0800 CST deployed openebs-3.10.0 3.10.0
# pods
[root@k8s-master ~]# kubectl get pod -n openebs -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
openebs-localpv-provisioner-685b678c88-lq57l 1/1 Running 0 9m49s 10.244.0.232 k8s-master <none> <none>
openebs-ndm-bxzwv 1/1 Running 0 9m49s 192.168.0.51 k8s-master <none> <none>
openebs-ndm-d54bt 1/1 Running 1 9m49s 192.168.0.53 k8s-node2 <none> <none>
openebs-ndm-hjnpx 1/1 Running 2 9m49s 192.168.0.52 k8s-node1 <none> <none>
openebs-ndm-operator-c959d9d77-dscd2 1/1 Running 0 9m49s 10.244.0.233 k8s-master <none> <none>
# 儲存類
[root@k8s-master ~]# kubectl get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
openebs-device openebs.io/local Delete WaitForFirstConsumer false 16d
openebs-hostpath openebs.io/local Delete WaitForFirstConsumer false 16d
本文安裝的openebs版本較低，3.10.0版本，較新版本的chart倉庫已經變更，詳情參考官方文件：
官方文件：https://openebs.netlify.app/docs/quickstart-guide/installation
官方倉庫：https://github.com/openebs/openebs
2. demo測試
openebs部署完成後會自動建立儲存類，我們使用openebs-hostpath這個StorageClass來建立PVC。
建立一個示例應用掛載openebs提供的儲存進行測試。資源清單檔案openebs-test.yaml，包括pvc和pod：
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: local-hostpath-pvc
spec:
storageClassName: openebs-hostpath
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
---
apiVersion: v1
kind: Pod
metadata:
name: hello-local-hostpath-pod
spec:
volumes:
- name: local-storage
persistentVolumeClaim:
claimName: local-hostpath-pvc
containers:
- name: hello-container
image: busybox
command:
- sh
- -c
- 'while true; do echo "`date` [`hostname`] Hello from OpenEBS Local PV." >> /mnt/store/greet.txt; sleep $(($RANDOM % 5 + 300)); done'
volumeMounts:
- mountPath: /mnt/store
name: local-storage
建立資源：kubectl create -f openebs-test.yaml，等待建立完成。
[root@k8s-master openebs]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
hello-local-hostpath-pod 1/1 Running 0 15m 10.244.1.65 k8s-node1 <none> <none>
[root@k8s-master openebs]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
local-hostpath-pvc Bound pvc-b240b152-718e-44e2-938f-0255e457ec8f 5Gi RWO openebs-hostpath 11m
[root@k8s-master openebs]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-b240b152-718e-44e2-938f-0255e457ec8f 5Gi RWO Delete Bound default/local-hostpath-pvc openebs-hostpath 8m49s
# 檢查node1節點的本地路徑
[root@k8s-node1 ~]# cat /var/openebs/local/pvc-b240b152-718e-44e2-938f-0255e457ec8f/greet.txt
Fri Jun 28 08:11:00 UTC 2024 [hello-local-hostpath-pod] Hello from OpenEBS Local PV.
Fri Jun 28 08:16:04 UTC 2024 [hello-local-hostpath-pod] Hello from OpenEBS Local PV.
Fri Jun 28 08:21:08 UTC 2024 [hello-local-hostpath-pod] Hello from OpenEBS Local PV.