> 本文首發於 [Nebula Graph Community 公眾號](https://mp.weixin.qq.com/s/EoY1G6hYTa14gqd2ibK5vA)

![Nebula Operator](https://www-cdn.nebula-graph.com.cn/nebula-blog/nebula-operator-practice.png)

嗨，大家好！[Nebula Operator](https://github.com/vesoft-inc/nebula-operator) 開源也有一段時間了，之前也有一篇相關的[部落格](https://nebula-graph.com.cn/posts/nebula-operator-overview/)介紹，但是實踐相關的部落格卻還沒有，現在：

它來了！來了！它帶著**實踐**走來了！

## [Nebula Operator](https://github.com/vesoft-inc/nebula-operator) 介紹

關於 [Nebula Operator](https://github.com/vesoft-inc/nebula-operator) 的介紹，大家可以參考之前那篇部落格：[一文詳解雲上自動化部署叢集管理工具 Nebula Operator](https://nebula-graph.com.cn/posts/nebula-operator-overview/)。

本文將主要側重於實踐方向，讓你能很快地入手 [Nebula Operator](https://github.com/vesoft-inc/nebula-operator)，體驗圖資料庫的樂趣！

## [Nebula Operator](https://github.com/vesoft-inc/nebula-operator)雲上實踐

到這裡，開始進入正題，本文將使用阿里雲進行 [Nebula Operator](https://github.com/vesoft-inc/nebula-operator) 實踐，其他雲廠商類似。

### 安裝工具

本次實踐需要在操作電腦上安裝如下基礎工具：

* [Kubectl](https://kubernetes.io/zh/docs/tasks/tools/)
* [Helm](https://helm.sh/zh/docs/intro/install/)
* [Docker](https://docs.docker.com/get-docker/)

以上基礎工具相關的安裝方法請參考對應連結。

### 建立雲上 Kubernetes

因為 Operator 是依託於 Kubernetes 的，所以在進行 [Nebula Operator](https://github.com/vesoft-inc/nebula-operator) 實踐之前，需要先準備好 Kubernetes 環境。

首先進入到阿里雲的控制檯，然後進入到**容器服務 Kubernetes 版**，再建立一個叢集。此實踐選擇的是 ACK 託管版，相關的建立引數請按需選擇。

*注意： 為了方便外網訪問 Kubernetes API Server ，本次實踐勾選了**使用 EIP 暴露 API Server**，你可以根據自身情況選擇是否啟用，如果不開啟，你需要打通操作電腦與 Kubernetes 的之間網路。其他引數請按需選擇*。

等待 Kubernetes 叢集啟動後，將叢集的**連線資訊**中**公網訪問**中的內容複製到計算機`$HOME/.kube/config`檔案中。

然後你可以使用如下命令驗證下 Kubernetes 叢集：

```plain
$ kubectl get nodes
NAME                         STATUS   ROLES    AGE   VERSION
cn-beijing.192.168.250.13    Ready    <none>   51m   v1.20.4-aliyun.1
cn-beijing.192.168.250.185   Ready    <none>   51m   v1.20.4-aliyun.1
cn-beijing.192.168.250.89    Ready    <none>   51m   v1.20.4-aliyun.1
```

### 安裝 [Nebula Operator](https://github.com/vesoft-inc/nebula-operator) 依賴

在安裝 [Nebula Operator](https://github.com/vesoft-inc/nebula-operator) 之前，還需要先安裝一些依賴。

#### 安裝 [CertManager](https://cert-manager.io/)

```plain
# 安裝 CertManager
$ helm install cert-manager cert-manager --repo https://charts.jetstack.io \
    --namespace cert-manager --create-namespace --version v1.3.1 \
    --set installCRDs=true
# 稍等一會兒，檢測 CertManager 是否啟動正常
$ kubectl -n cert-manager get pod
NAME                                       READY   STATUS    RESTARTS   AGE
cert-manager-7998c69865-jfw9x              1/1     Running   0          93s
cert-manager-cainjector-7b744d56fb-846w9   1/1     Running   0          93s
cert-manager-webhook-7d6d4c78bc-ssk4w      1/1     Running   0          93s
```

#### 安裝 [OpenKruise](https://openkruise.io/)

```plain
# 安裝 OpenKruise
$ helm install kruise \
    https://github.com/openkruise/kruise/releases/download/v0.8.1/kruise-chart.tgz
# 稍等一會兒，檢測 OpenKruise 是否啟動正常
$ kubectl -n kruise-system get pod
NAME                                         READY   STATUS    RESTARTS   AGE
kruise-controller-manager-6797f89d9b-ppv65   1/1     Running   0          49s
kruise-controller-manager-6797f89d9b-wlkbd   1/1     Running   0          49s
kruise-daemon-7rljq                          1/1     Running   0          49s
kruise-daemon-8kd8d                          1/1     Running   0          49s
kruise-daemon-n6tdw                          1/1     Running   0          49s
```

### 新增 [Nebula Operator](https://github.com/vesoft-inc/nebula-operator) Charts

```plain
# 新增 Nebula Operator Charts Repo
$ helm repo add nebula-operator https://vesoft-inc.github.io/nebula-operator/charts
# 更新 repo
$ helm repo update
```

### 安裝 [Nebula Operator](https://github.com/vesoft-inc/nebula-operator)

由於阿里雲上無法拉取`gcr.io`和`k8s.gcr.io`映象，因此需要指定國內映象，這裡進行了如下替換：

|原映象|替換後映象|
|:----|:----|
|[gcr.io/kubebuilder/kube-rbac-proxy:v0.8.0](http://gcr.io/kubebuilder/kube-rbac-proxy:v0.8.0)|kubesphere/kube-rbac-proxy:v0.8.0|
|[k8s.gcr.io/kube-scheduler:v1.18.8](http://k8s.gcr.io/kube-scheduler:v1.18.8)|kubesphere/kube-scheduler:v1.18.8|

你可以透過如下命令檢視所有可以設定的引數：

```plain
$ helm show values nebula-operator/nebula-operator
```
此次實踐中的安裝命令如下：
```plain
# 安裝 Nebula Operator
$ helm install nebula-operator nebula-operator/nebula-operator \
    --namespace nebula-operator-system --create-namespace --version 0.1.0 \
    --set image.kubeRBACProxy.image=kubesphere/kube-rbac-proxy:v0.8.0 \
    --set image.kubeScheduler.image=kubesphere/kube-scheduler:v1.18.8
# 稍等一會兒，檢測 Nebula Operator 是否啟動正常
$ kubectl -n nebula-operator-system get pod
NAME                                                             READY   STATUS    RESTARTS   AGE
nebula-operator-controller-manager-deployment-6968547fff-k62b4   2/2     Running   0          19s
nebula-operator-controller-manager-deployment-6968547fff-lhpdx   2/2     Running   0          19s
nebula-operator-scheduler-deployment-7c5fc7945-hbkv8             2/2     Running   0          19s
nebula-operator-scheduler-deployment-7c5fc7945-sxc7w             2/2     Running   0          19s
```
如果你自定義了 Kubernetes 的 Cluster Domain ，則需要修改安裝命令，增加設定`kubernetesClusterDomain`，如下：
```plain
# 安裝 Nebula Operator ，請修改 <<yourcustomclusterdomain>>
$ helm install nebula-operator nebula-operator/nebula-operator \
    --namespace nebula-operator-system --create-namespace --version 0.1.0 \
    --set image.kubeRBACProxy.image=kubesphere/kube-rbac-proxy:v0.8.0 \
    --set image.kubeScheduler.image=kubesphere/kube-scheduler:v1.18.8 \
    --set kubernetesClusterDomain=<<yourcustomclusterdomain>>
```

### 部署 Nebula Cluster

至此，[Nebula Operator](https://github.com/vesoft-inc/nebula-operator) 已經就緒，接下來安裝`Nebula Cluster`來體驗圖資料吧！

首先，需要獲取`StorageClass`，這個將會用來設定`Nebula Cluster`所使用的儲存。

```plain
$ kubectl get sc
NAME                       PROVISIONER                       RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
alicloud-disk-available    diskplugin.csi.alibabacloud.com   Delete          Immediate              true                   100m
alicloud-disk-efficiency   diskplugin.csi.alibabacloud.com   Delete          Immediate              true                   100m
alicloud-disk-essd         diskplugin.csi.alibabacloud.com   Delete          Immediate              true                   100m
alicloud-disk-ssd          diskplugin.csi.alibabacloud.com   Delete          Immediate              true                   100m
alicloud-disk-topology     diskplugin.csi.alibabacloud.com   Delete          WaitForFirstConsumer   true                   100m
```
從上面得知，阿里雲目前支援`5`種`StorageClass`。此次實踐將會使用`alicloud-disk-ssd`。其他雲廠商會有對應的`StorageClass`，請按照實際情況進行選擇。
*注意：每個雲廠商可能對於申請儲存的大小有範圍限制，比如[阿里雲](https://partners-intl.aliyun.com/help/doc-detail/25513.htm)對於 SSD 限制在`20 Gi`到`32,768 Gi`，在建立`Nebula Cluster`的時候需要注意下。

你可以透過如下命令檢視所有可以設定的引數：

```plain
$ helm show values nebula-operator/nebula-cluster
```
此次實踐中的安裝命令如下：
```plain
# 建立 Nebula Cluster 的名稱
$ export NEBULA_CLUSTER_NAME=nebula
# 建立 Nebula Cluster 的 namespace
$ export NEBULA_CLUSTER_NAMESPACE=nebula
# 建立 Nebula Cluster 的 StorageClass 名稱，這裡設定為之前查詢到的 alicloud-disk-ssd
$ export STORAGE_CLASS_NAME=alicloud-disk-ssd
# 建立 Nebula Cluster 中每個組建所使用儲存的大小
$ export STORAGE_SIZE_GRAPHD=20Gi
$ export STORAGE_SIZE_METAD=20Gi
$ export STORAGE_SIZE_STORAGED=20Gi
# 建立 Nebula Cluster
$ helm install ${NEBULA_CLUSTER_NAME} nebula-operator/nebula-cluster \
    --namespace ${NEBULA_CLUSTER_NAMESPACE} --create-namespace --version 0.1.0 \
    --set nameOverride=${NEBULA_CLUSTER_NAME} \
    --set nebula.storageClassName="${STORAGE_CLASS_NAME}" \
    --set nebula.graphd.storage="${STORAGE_SIZE_GRAPHD}" \
    --set nebula.metad.storage="${STORAGE_SIZE_METAD}" \
    --set nebula.storaged.storage="${STORAGE_SIZE_STORAGED}"
# 稍等一會兒，檢測 Nebula Cluster 是否啟動正常
$ kubectl -n ${NEBULA_CLUSTER_NAMESPACE} get nebulacluster
NAME     GRAPHD-DESIRED   GRAPHD-READY   METAD-DESIRED   METAD-READY   STORAGED-DESIRED   STORAGED-READY   AGE
nebula   2                2              3               3             3                  3                4m10s
$ kubectl -n ${NEBULA_CLUSTER_NAMESPACE} get pod
NAME                READY   STATUS    RESTARTS   AGE
nebula-graphd-0     1/1     Running   0          96s
nebula-graphd-1     1/1     Running   0          96s
nebula-metad-0      1/1     Running   0          97s
nebula-metad-1      1/1     Running   0          97s
nebula-metad-2      1/1     Running   0          97s
nebula-storaged-0   1/1     Running   0          97s
nebula-storaged-1   1/1     Running   0          97s
nebula-storaged-2   1/1     Running   0          97s
```
當然，也可以將`Storaged`例項升級到`5`個，執行命令如下：
```plain
# 升級 Nebula Cluster
$ helm upgrade ${NEBULA_CLUSTER_NAME} nebula-operator/nebula-cluster \
    --namespace ${NEBULA_CLUSTER_NAMESPACE} --create-namespace --version 0.1.0 \
    --set nameOverride=${NEBULA_CLUSTER_NAME} \
    --set nebula.storageClassName="${STORAGE_CLASS_NAME}" \
    --set nebula.graphd.storage="${STORAGE_SIZE_GRAPHD}" \
    --set nebula.metad.storage="${STORAGE_SIZE_METAD}" \
    --set nebula.storaged.storage="${STORAGE_SIZE_STORAGED}" \
    --set nebula.storaged.replicas=5
# 稍等一會兒，檢測 Nebula Cluster 是否啟動正常
$ kubectl -n ${NEBULA_CLUSTER_NAMESPACE} get nebulacluster
NAME     GRAPHD-DESIRED   GRAPHD-READY   METAD-DESIRED   METAD-READY   STORAGED-DESIRED   STORAGED-READY   AGE
nebula   2                2              3               3             5                  5                6m12s
$ kubectl -n ${NEBULA_CLUSTER_NAMESPACE} get pod
NAME                READY   STATUS    RESTARTS   AGE
nebula-graphd-0     1/1     Running   0          2m30s
nebula-graphd-1     1/1     Running   0          2m30s
nebula-metad-0      1/1     Running   0          2m30s
nebula-metad-1      1/1     Running   0          2m30s
nebula-metad-2      1/1     Running   0          2m30s
nebula-storaged-0   1/1     Running   0          2m30s
nebula-storaged-1   1/1     Running   0          2m30s
nebula-storaged-2   1/1     Running   0          2m30s
nebula-storaged-3   1/1     Running   0          52s
nebula-storaged-4   1/1     Running   0          52s
```
詳細的安裝說明請見：[使用 Helm 安裝 Nebula Operator](https://github.com/vesoft-inc/nebula-operator/blob/master/doc/user/nebula_cluster_helm_guide.md)。
### 訪問 Nebula Cluster

終於，Nebula Cluster 啟動成功了，接下來開始訪問叢集吧！

#### Kubernetes 內部訪問

首先，在 Kubernetes 中啟動一個 [Nebula Graph Console](https://github.com/vesoft-inc/nebula-console)，執行命令如下：

```plain
$ cat <<eof kubectl="" apply="" apiversion:="" v1="" kind:="" pod="" metadata:="" name:="" nebula-console="" spec:="" containers:="" image:="" vesoft="" command:="" sleep="" eof="" graph="" console="" exec="" u="" p="" connection="" pool="" is="" initialized="" successfully="" welcome="" to="" nebula=""> show hosts
+-----------------------------------------------------------------------+------+----------+--------------+----------------------+------------------------+
| Host                                                                  | Port | Status   | Leader count | Leader distribution  | Partition distribution |
+-----------------------------------------------------------------------+------+----------+--------------+----------------------+------------------------+
| "nebula-storaged-0.nebula-storaged-headless.nebula.svc.cluster.local" | 9779 | "ONLINE" | 0            | "No valid partition" | "No valid partition"   |
+-----------------------------------------------------------------------+------+----------+--------------+----------------------+------------------------+
| "nebula-storaged-1.nebula-storaged-headless.nebula.svc.cluster.local" | 9779 | "ONLINE" | 0            | "No valid partition" | "No valid partition"   |
+-----------------------------------------------------------------------+------+----------+--------------+----------------------+------------------------+
| "nebula-storaged-2.nebula-storaged-headless.nebula.svc.cluster.local" | 9779 | "ONLINE" | 0            | "No valid partition" | "No valid partition"   |
+-----------------------------------------------------------------------+------+----------+--------------+----------------------+------------------------+
| "nebula-storaged-3.nebula-storaged-headless.nebula.svc.cluster.local" | 9779 | "ONLINE" | 0            | "No valid partition" | "No valid partition"   |
+-----------------------------------------------------------------------+------+----------+--------------+----------------------+------------------------+
| "nebula-storaged-4.nebula-storaged-headless.nebula.svc.cluster.local" | 9779 | "ONLINE" | 0            | "No valid partition" | "No valid partition"   |
+-----------------------------------------------------------------------+------+----------+--------------+----------------------+------------------------+
| "Total"                                                               |      |          | 0            |                      |                        |
+-----------------------------------------------------------------------+------+----------+--------------+----------------------+------------------------+
Got 4 rows (time spent 7669/9367 us)
Wed, 23 Jun 2021 06:21:26 UTC
```

#### Kubernetes 外部訪問

Kubernetes 內部的服務在叢集外部要想訪問，可以使用`hostPort`、`hostNetwork`、`Ingress`、`LoadBalancer`等。這裡利用雲廠商的便利性，直接使用`LoadBalancer`來訪問叢集。

***注意：此方法會暴露你的 Nebula 叢集，請勿在生產環境使用。***

首先，將`Graphd Service`的`type`改成`LoadBalancer`，然後再檢視`EXTERNAL-IP`。

```plain
# 將 service 的 type 改成 LoadBalancer
$ kubectl patch -n ${NEBULA_CLUSTER_NAMESPACE} svc ${NEBULA_CLUSTER_NAME}-graphd-svc \
    -p '{"spec": {"type": "LoadBalancer"}}'
# 獲取 EXTERNAL-IP ，如果為 pending ，請稍等一會兒再重試
$ kubectl -n ${NEBULA_CLUSTER_NAMESPACE} get svc nebula-graphd-svc
NAME                TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)                                          AGE
nebula-graphd-svc   LoadBalancer   172.16.85.222   x.x.x.x         9669:31460/TCP,19669:32579/TCP,19670:31481/TCP   27m
```
現在，可以根據`EXTERNAL-IP`來訪問叢集了，比如此處為`x.x.x.x`。
```plain
$ export EXTERNAL_IP=x.x.x.x
$ docker run -it --rm vesoft/nebula-console:v2-nightly -u u -p p --addr ${EXTERNAL_IP} --port 9669
2021/06/23 06:42:17 [INFO] connection pool is initialized successfully
Welcome to Nebula Graph!
(u@nebula) [(none)]> show hosts
+-----------------------------------------------------------------------+------+----------+--------------+----------------------+------------------------+
| Host                                                                  | Port | Status   | Leader count | Leader distribution  | Partition distribution |
+-----------------------------------------------------------------------+------+----------+--------------+----------------------+------------------------+
| "nebula-storaged-0.nebula-storaged-headless.nebula.svc.cluster.local" | 9779 | "ONLINE" | 0            | "No valid partition" | "No valid partition"   |
+-----------------------------------------------------------------------+------+----------+--------------+----------------------+------------------------+
| "nebula-storaged-1.nebula-storaged-headless.nebula.svc.cluster.local" | 9779 | "ONLINE" | 0            | "No valid partition" | "No valid partition"   |
+-----------------------------------------------------------------------+------+----------+--------------+----------------------+------------------------+
| "nebula-storaged-2.nebula-storaged-headless.nebula.svc.cluster.local" | 9779 | "ONLINE" | 0            | "No valid partition" | "No valid partition"   |
+-----------------------------------------------------------------------+------+----------+--------------+----------------------+------------------------+
| "nebula-storaged-3.nebula-storaged-headless.nebula.svc.cluster.local" | 9779 | "ONLINE" | 0            | "No valid partition" | "No valid partition"   |
+-----------------------------------------------------------------------+------+----------+--------------+----------------------+------------------------+
| "nebula-storaged-4.nebula-storaged-headless.nebula.svc.cluster.local" | 9779 | "ONLINE" | 0            | "No valid partition" | "No valid partition"   |
+-----------------------------------------------------------------------+------+----------+--------------+----------------------+------------------------+
| "Total"                                                               |      |          | 0            |                      |                        |
+-----------------------------------------------------------------------+------+----------+--------------+----------------------+------------------------+
Got 4 rows (time spent 3747/60433 us)
Wed, 23 Jun 2021 06:42:21 UTC
```

## 享用時間

**大功告成！**

**盡情地在 Nebula Graph 中馳騁吧！**</eof></yourcustomclusterdomain></yourcustomclusterdomain></none></none></none>

Nebula Operator 雲上實踐

相關文章