今天我們將深入探討：：如何在 K8s 叢集上部署 Compute storage coupled（存算耦合） 模式的 Doris 高可用叢集？
本文，我將為您提供一份全面的實戰指南，逐步引導您完成以下關鍵任務：
- 配置 Doris ConfigMap：實現自定義配置檔案
- 配置 Doris Secret：管理特殊密碼
- 配置 Doris Service：使用 NodePort 對外發布 Doris 服務
- 部署 Doris FE
- 部署 Doris BE
- Doris 使用者初始化
- Doris 圖形化管理概覽
透過本文的指導，您將掌握在 K8s 上部署 Compute storage coupled 模式下的 Doris 叢集的必備技能。
實戰伺服器配置(架構1:1復刻小規模生產環境，配置略有不同)
| 主機名 | IP | CPU | 記憶體 | 系統盤 | 資料盤 | 用途 |
|---|---|---|---|---|---|---|
| ksp-registry | 192.168.9.90 | 4 | 8 | 40 | 200 | Harbor 映象倉庫 |
| ksp-control-1 | 192.168.9.91 | 4 | 8 | 40 | 100 | KubeSphere/k8s-control-plane |
| ksp-control-2 | 192.168.9.92 | 4 | 8 | 40 | 100 | KubeSphere/k8s-control-plane |
| ksp-control-3 | 192.168.9.93 | 4 | 8 | 40 | 100 | KubeSphere/k8s-control-plane |
| ksp-worker-1 | 192.168.9.94 | 8 | 16 | 40 | 100 | k8s-worker/CI |
| ksp-worker-2 | 192.168.9.95 | 8 | 16 | 40 | 100 | k8s-worker |
| ksp-worker-3 | 192.168.9.96 | 8 | 16 | 40 | 100 | k8s-worker |
| ksp-storage-1 | 192.168.9.97 | 4 | 8 | 40 | 400+ | ElasticSearch/Longhorn/Ceph/NFS |
| ksp-storage-2 | 192.168.9.98 | 4 | 8 | 40 | 300+ | ElasticSearch/Longhorn/Ceph |
| ksp-storage-3 | 192.168.9.99 | 4 | 8 | 40 | 300+ | ElasticSearch/Longhorn/Ceph |
| ksp-gpu-worker-1 | 192.168.9.101 | 4 | 16 | 40 | 100 | k8s-worker(GPU NVIDIA Tesla M40 24G) |
| ksp-gpu-worker-2 | 192.168.9.102 | 4 | 16 | 40 | 100 | k8s-worker(GPU NVIDIA Tesla P100 16G) |
| ksp-gateway-1 | 192.168.9.103 | 2 | 4 | 40 | 自建應用服務代理閘道器/VIP：192.168.9.100 | |
| ksp-gateway-2 | 192.168.9.104 | 2 | 4 | 40 | 自建應用服務代理閘道器/VIP：192.168.9.100 | |
| ksp-mid | 192.168.9.105 | 4 | 8 | 40 | 100 | 部署在 k8s 叢集之外的服務節點（Gitlab 等） |
| 合計 | 15 | 68 | 152 | 600 | 2100+ |
實戰環境涉及軟體版本資訊
- 作業系統：openEuler 22.03 LTS SP3 x86_64
- KubeSphere：v3.4.1
- Kubernetes：v1.28.8
- KubeKey: v3.1.1
- Doris： 3.0.2
1. 部署方案規劃
Doris 官方提供了 Doris Operator 工具，用於在 K8s 叢集中部署和管理 Doris 叢集，詳細資訊可以查閱Doris 快速部署文件 。
一些特殊環境或是運維人員自身能力、喜好等原因，不適合使用 Doris Operator。這就需要我們使用 Kubectl 利用自定義資源清單，部署 Doris 叢集。
本文基於資源清單實現了 Doris 叢集的部署，生產環境建議使用 Doris Operator，部署方法也更加簡單。
在 K8s 上部署 Doris 叢集有兩種架構方案：
- Compute storage coupled（存算耦合）
- Compute storage decoupled（存算分離）
本文選擇了適用於中小規模場景的 Compute storage coupled 方案。Compute storage decoupled 的部署和維護更加複雜，更多細節請查閱官方文件。
1.1 部署架構圖
1.2 準備持久化儲存
本實戰環境使用 NFS 作為 K8s 叢集的持久化儲存，新叢集可以參考探索 K8s 持久化儲存之 NFS 終極實戰指南 部署 NFS 儲存。
1.3 名稱空間
Doris 叢集所有資源部署在名稱空間 opsxlab內。
1.4 前提準備
- 準備 root 密碼
編寫 Python 指令碼 hm.py，生成 2 階段 SHA-1 加密的密碼。
#!/bin/python
import hashlib
# 原始密碼
original_password = "PleaseChangeMe"
# 第一次SHA-1雜湊運算
first_hash = hashlib.sha1(original_password.encode('utf-8')).hexdigest()
# 第二次SHA-1雜湊運算
first_hash_bytes = bytes.fromhex(first_hash)
second_hash = hashlib.sha1(first_hash_bytes).hexdigest()
# 輸出兩階段加密後的密碼
print("*" + second_hash)生成密碼，記錄備用。
$ python3 hm.py
*aa7530f7c48740e92a4c0d2138324611e314d3972. 部署 Doris 叢集
2.1 建立 ConfigMap
- 建立 Doris FE 配置檔案
請使用 vi 編輯器，建立資源清單檔案 doris-cluster-fe-conf.yaml，並輸入以下內容：
apiVersion: v1
kind: ConfigMap
metadata:
name: doris-cluster-fe-conf
labels:
app.kubernetes.io/component: fe
data:
fe.conf: |
#####################################################################
## The uppercase properties are read and exported by bin/start_fe.sh.
## To see all Frontend configurations,
## see fe/src/org/apache/doris/common/Config.java
#####################################################################
CUR_DATE=`date +%Y%m%d-%H%M%S`
# Log dir
LOG_DIR = ${DORIS_HOME}/log
# For jdk 8
JAVA_OPTS="-Dfile.encoding=UTF-8 -Djavax.security.auth.useSubjectCredsOnly=false -Xss4m -Xmx8192m -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -Xloggc:$LOG_DIR/log/fe.gc.log.$CUR_DATE -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=50M -Dlog4j2.formatMsgNoLookups=true"
# For jdk 17, this JAVA_OPTS will be used as default JVM options
JAVA_OPTS_FOR_JDK_17="-Dfile.encoding=UTF-8 -Djavax.security.auth.useSubjectCredsOnly=false -Xmx8192m -Xms8192m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=$LOG_DIR -Xlog:gc*:$LOG_DIR/fe.gc.log.$CUR_DATE:time,uptime:filecount=10,filesize=50M --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens java.base/jdk.internal.ref=ALL-UNNAMED"
# Set your own JAVA_HOME
# JAVA_HOME=/path/to/jdk/
##
## the lowercase properties are read by main program.
##
# store metadata, must be created before start FE.
# Default value is ${DORIS_HOME}/doris-meta
# meta_dir = ${DORIS_HOME}/doris-meta
# Default dirs to put jdbc drivers,default value is ${DORIS_HOME}/jdbc_drivers
# jdbc_drivers_dir = ${DORIS_HOME}/jdbc_drivers
http_port = 8030
rpc_port = 9020
query_port = 9030
edit_log_port = 9010
arrow_flight_sql_port = -1
# Choose one if there are more than one ip except loopback address.
# Note that there should at most one ip match this list.
# If no ip match this rule, will choose one randomly.
# use CIDR format, e.g. 10.10.10.0/24 or IP format, e.g. 10.10.10.1
# Default value is empty.
# priority_networks = 10.10.10.0/24;192.168.0.0/16
# Advanced configurations
# log_roll_size_mb = 1024
# INFO, WARN, ERROR, FATAL
sys_log_level = INFO
# NORMAL, BRIEF, ASYNC
sys_log_mode = ASYNC
# sys_log_roll_num = 10
# sys_log_verbose_modules = org.apache.doris
# audit_log_dir = $LOG_DIR
# audit_log_modules = slow_query, query
# audit_log_roll_num = 10
# meta_delay_toleration_second = 10
# qe_max_connection = 1024
# qe_query_timeout_second = 300
# qe_slow_log_ms = 5000
enable_fqdn_mode = true
initial_root_password = *aa7530f7c48740e92a4c0d2138324611e314d397說明： 配置檔案在 FE 預設配置基礎上，增加了 initial_root_password 配置項，值是前面用 Python 生成的2段加密的密碼，生產環境請根據需要調整。
- 建立 Doris BE 配置檔案
請使用 vi 編輯器，建立資源清單檔案 doris-cluster-be-conf.yaml，並輸入以下內容：
kind: ConfigMap
apiVersion: v1
metadata:
name: doris-cluster-be-conf
labels:
app.kubernetes.io/component: be
data:
be.conf: >
CUR_DATE=`date +%Y%m%d-%H%M%S`
# Log dir
LOG_DIR="${DORIS_HOME}/log/"
# For jdk 8
JAVA_OPTS="-Dfile.encoding=UTF-8 -Xmx2048m -DlogPath=$LOG_DIR/jni.log -Xloggc:$LOG_DIR/be.gc.log.$CUR_DATE -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=50M -Djavax.security.auth.useSubjectCredsOnly=false -Dsun.security.krb5.debug=true -Dsun.java.command=DorisBE -XX:-CriticalJNINatives"
# For jdk 17, this JAVA_OPTS will be used as default JVM options
JAVA_OPTS_FOR_JDK_17="-Dfile.encoding=UTF-8 -Xmx2048m -DlogPath=$LOG_DIR/jni.log -Xlog:gc*:$LOG_DIR/be.gc.log.$CUR_DATE:time,uptime:filecount=10,filesize=50M -Djavax.security.auth.useSubjectCredsOnly=false -Dsun.security.krb5.debug=true -Dsun.java.command=DorisBE -XX:-CriticalJNINatives -XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-opens=java.management/sun.management=ALL-UNNAMED"
# Set your own JAVA_HOME
# JAVA_HOME=/path/to/jdk/
# https://github.com/apache/doris/blob/master/docs/zh-CN/community/developer-guide/debug-tool.md#jemalloc-heap-profile
# https://jemalloc.net/jemalloc.3.html
JEMALLOC_CONF="percpu_arena:percpu,background_thread:true,metadata_thp:auto,muzzy_decay_ms:15000,dirty_decay_ms:15000,oversize_threshold:0,prof:false,lg_prof_interval:32,lg_prof_sample:19,prof_gdump:false,prof_accum:false,prof_leak:false,prof_final:false"
JEMALLOC_PROF_PRFIX=""
# ports for admin, web, heartbeat service
be_port = 9060
webserver_port = 8040
heartbeat_service_port = 9050
brpc_port = 8060
arrow_flight_sql_port = -1
# HTTPS configures
enable_https = false
# path of certificate in PEM format.
ssl_certificate_path = "$DORIS_HOME/conf/cert.pem"
# path of private key in PEM format.
ssl_private_key_path = "$DORIS_HOME/conf/key.pem"
# Choose one if there are more than one ip except loopback address.
# Note that there should at most one ip match this list.
# If no ip match this rule, will choose one randomly.
# use CIDR format, e.g. 10.10.10.0/24 or IP format, e.g. 10.10.10.1
# Default value is empty.
# priority_networks = 10.10.10.0/24;192.168.0.0/16
# data root path, separate by ';'
# You can specify the storage type for each root path, HDD (cold data) or SSD (hot data)
# eg:
# storage_root_path = /home/disk1/doris;/home/disk2/doris;/home/disk2/doris
# storage_root_path = /home/disk1/doris,medium:SSD;/home/disk2/doris,medium:SSD;/home/disk2/doris,medium:HDD
# /home/disk2/doris,medium:HDD(default)
#
# you also can specify the properties by setting '<property>:<value>', separate by ','
# property 'medium' has a higher priority than the extension of path
#
# Default value is ${DORIS_HOME}/storage, you should create it by hand.
# storage_root_path = ${DORIS_HOME}/storage
# Default dirs to put jdbc drivers,default value is ${DORIS_HOME}/jdbc_drivers
# jdbc_drivers_dir = ${DORIS_HOME}/jdbc_drivers
# Advanced configurations
# INFO, WARNING, ERROR, FATAL
sys_log_level = INFO
# sys_log_roll_mode = SIZE-MB-1024
# sys_log_roll_num = 10
# sys_log_verbose_modules = *
# log_buffer_level = -1
# aws sdk log level
# Off = 0,
# Fatal = 1,
# Error = 2,
# Warn = 3,
# Info = 4,
# Debug = 5,
# Trace = 6
# Default to turn off aws sdk log, because aws sdk errors that need to be cared will be output through Doris logs
aws_log_level=0
## If you are not running in aws cloud, you can disable EC2 metadata
AWS_EC2_METADATA_DISABLED=true說明： 配置檔案使用了 BE 的預設配置，生產環境請根據需要調整。
- 建立資源
執行下面的命令，建立資源。
kubectl apply -f doris-cluster-fe-conf.yaml -n opsxlab
kubectl apply -f doris-cluster-be-conf.yaml -n opsxlab- 驗證資源
執行下面的命令，檢視建立結果。
$ kubectl get cm -n opsxlab
NAME DATA AGE
doris-cluster-be-conf 1 34s
doris-cluster-fe-conf 1 20s2.2 建立 Secret
- 建立管理 Doris 叢集節點所需的使用者名稱、密碼的保密字典
請使用 vi 編輯器，建立資源清單檔案 doris-cluster-secret.yaml，並輸入以下內容：
kind: Secret
apiVersion: v1
metadata:
name: doris-cluster-secret
stringData:
username: root
password: PleaseChangeMe
type: kubernetes.io/basic-auth提示： 密碼使用明文 PleaseChangeMe，生產環境請務必替換。
- 建立資源
執行下面的命令，建立資源。
kubectl apply -f doris-cluster-secret.yaml -n opsxlab- 驗證資源
執行下面的命令，檢視建立結果。
$ kubectl get secret -n opsxlab
NAME TYPE DATA AGE
doris-cluster-secret kubernetes.io/basic-auth 2 22s2.3 建立服務
我們採用 NodePort 方式在 K8s 叢集外發布 Doris 服務。
- FE 服務
請使用 vi 編輯器，建立資源清單檔案 doris-cluster-fe-service.yaml，並輸入以下內容：
kind: Service
apiVersion: v1
metadata:
name: doris-cluster-fe-service
labels:
app.kubernetes.io/component: doris-cluster-fe
spec:
ports:
- name: http-port
protocol: TCP
port: 8030
targetPort: 8030
nodePort: 31620
- name: rpc-port
protocol: TCP
port: 9020
targetPort: 9020
nodePort: 31621
- name: query-port
protocol: TCP
port: 9030
targetPort: 9030
nodePort: 31622
- name: edit-log-port
protocol: TCP
port: 9010
targetPort: 9010
nodePort: 31623
selector:
app.kubernetes.io/component: doris-cluster-fe
type: NodePort- BE 服務
請使用 vi 編輯器，建立資源清單檔案 doris-cluster-be-service.yaml，並輸入以下內容：
kind: Service
apiVersion: v1
metadata:
name: doris-cluster-be-service
labels:
app.kubernetes.io/component: doris-cluster-be
spec:
ports:
- name: be-port
protocol: TCP
port: 9060
targetPort: 9060
nodePort: 32189
- name: webserver-port
protocol: TCP
port: 8040
targetPort: 8040
nodePort: 31624
- name: heartbeat-port
protocol: TCP
port: 9050
targetPort: 9050
nodePort: 31625
- name: brpc-port
protocol: TCP
port: 8060
targetPort: 8060
nodePort: 31627
selector:
app.kubernetes.io/component: doris-cluster-be
type: NodePort- 建立資源
執行下面的命令，建立資源。
kubectl apply -f doris-cluster-fe-service.yaml -n opsxlab
kubectl apply -f doris-cluster-be-service.yaml -n opsxlab- 驗證資源
執行下面的命令，檢視建立結果。
$ kubectl get svc -o wide -n opsxlab
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
doris-cluster-be-service NodePort 10.233.11.255 <none> 9060:32189/TCP,8040:31624/TCP,9050:31625/TCP,8060:31627/TCP 12s app.kubernetes.io/component=doris-cluster-be
doris-cluster-fe-service NodePort 10.233.60.147 <none> 8030:31620/TCP,9020:31621/TCP,9030:31622/TCP,9010:31623/TCP 12s app.kubernetes.io/component=doris-cluster-fe2.4 建立 Doris FE
使用 StatefulSet 部署 Doris FE 服務，需要建立 StatefulSet 和 HeadLess 兩種資源。
- 建立資源清單
請使用 vi 編輯器，建立資源清單檔案 doris-cluster-fe-sts.yaml，並輸入以下內容：
kind: StatefulSet
apiVersion: apps/v1
metadata:
name: doris-cluster-fe
labels:
app.kubernetes.io/component: doris-cluster-fe
spec:
replicas: 3
selector:
matchLabels:
app.kubernetes.io/component: doris-cluster-fe
template:
metadata:
name: doris-cluster-fe
labels:
app.kubernetes.io/component: doris-cluster-fe
spec:
volumes:
- name: meta
persistentVolumeClaim:
claimName: meta
- name: podinfo
downwardAPI:
items:
- path: labels
fieldRef:
apiVersion: v1
fieldPath: metadata.labels
- path: annotations
fieldRef:
apiVersion: v1
fieldPath: metadata.annotations
defaultMode: 420
- name: basic-auth
secret:
secretName: doris-cluster-secret
defaultMode: 420
- name: doris-cluster-fe-conf
configMap:
name: doris-cluster-fe-conf
defaultMode: 420
containers:
- name: doris-cluster-fe
image: 'selectdb/doris.fe-ubuntu:3.0.2'
command:
- /opt/apache-doris/fe_entrypoint.sh
args:
- $(ENV_FE_ADDR)
ports:
- name: http-port
containerPort: 8030
protocol: TCP
- name: rpc-port
containerPort: 9020
protocol: TCP
- name: query-port
containerPort: 9030
protocol: TCP
- name: edit-log-port
containerPort: 9010
protocol: TCP
env:
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: POD_IP
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: status.podIP
- name: HOST_IP
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: status.hostIP
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: CONFIGMAP_MOUNT_PATH
value: /etc/doris
- name: USER
value: root
- name: DORIS_ROOT
value: /opt/apache-doris
- name: ENV_FE_ADDR
value: doris-cluster-fe-service
- name: FE_QUERY_PORT
value: '9030'
- name: ELECT_NUMBER
value: '3'
resources:
limits:
cpu: '8'
memory: 16Gi
requests:
cpu: '1'
memory: 1Gi
volumeMounts:
- name: podinfo
mountPath: /etc/podinfo
- name: log
mountPath: /opt/apache-doris/fe/log
- name: meta
mountPath: /opt/apache-doris/fe/doris-meta
- name: doris-cluster-fe-conf
mountPath: /etc/doris
- name: basic-auth
mountPath: /etc/basic_auth
livenessProbe:
tcpSocket:
port: 9030
initialDelaySeconds: 80
timeoutSeconds: 180
periodSeconds: 5
successThreshold: 1
failureThreshold: 3
readinessProbe:
httpGet:
path: /api/health
port: 8030
scheme: HTTP
timeoutSeconds: 1
periodSeconds: 5
successThreshold: 1
failureThreshold: 3
startupProbe:
tcpSocket:
port: 9030
timeoutSeconds: 1
periodSeconds: 5
successThreshold: 1
failureThreshold: 60
lifecycle:
preStop:
exec:
command:
- /opt/apache-doris/fe_prestop.sh
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: IfNotPresent
restartPolicy: Always
terminationGracePeriodSeconds: 30
dnsPolicy: ClusterFirst
securityContext: {}
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app.kubernetes.io/component
operator: In
values:
- doris-cluster-fe
topologyKey: kubernetes.io/hostname
schedulerName: default-scheduler
volumeClaimTemplates:
- kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: meta
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10G
storageClassName: nfs-sc
volumeMode: Filesystem
- kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: log
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: '10'
storageClassName: nfs-sc
volumeMode: Filesystem
serviceName: doris-cluster-fe-internal
podManagementPolicy: Parallel
---
kind: Service
apiVersion: v1
metadata:
name: doris-cluster-fe-internal
labels:
app.kubernetes.io/component: doris-cluster-fe
spec:
ports:
- name: query-port
protocol: TCP
port: 9030
targetPort: 9030
selector:
app.kubernetes.io/component: doris-cluster-fe
clusterIP: None
type: ClusterIP- 建立資源
執行下面的命令，建立資源。
kubectl apply -f doris-cluster-fe-sts.yaml -n opsxlab- 驗證資源
執行下面的命令，檢視建立結果（初次建立比較慢）。
$ kubectl get sts,pod -n opsxlab
NAME READY AGE
statefulset.apps/doris-cluster-fe 3/3 117s
NAME READY STATUS RESTARTS AGE
pod/doris-cluster-fe-0 1/1 Running 0 117s
pod/doris-cluster-fe-1 1/1 Running 0 96s
pod/doris-cluster-fe-2 1/1 Running 0 80s2.5 建立 Doris BE
使用 StatefulSet 部署 Doris BE 服務，需要建立 StatefulSet 和 HeadLess 兩種資源。
- 建立資源清單
請使用 vi 編輯器，建立資源清單檔案 doris-cluster-be-sts.yaml，並輸入以下內容：
kind: StatefulSet
apiVersion: apps/v1
metadata:
name: doris-cluster-be
labels:
app.kubernetes.io/component: doris-cluster-be
spec:
replicas: 3
selector:
matchLabels:
app.kubernetes.io/component: doris-cluster-be
template:
metadata:
name: doris-cluster-be
labels:
app.kubernetes.io/component: doris-cluster-be
spec:
volumes:
- name: podinfo
downwardAPI:
items:
- path: labels
fieldRef:
apiVersion: v1
fieldPath: metadata.labels
- path: annotations
fieldRef:
apiVersion: v1
fieldPath: metadata.annotations
defaultMode: 420
- name: basic-auth
secret:
secretName: doris-cluster-secret
defaultMode: 420
- name: doris-cluster-be-conf
configMap:
name: doris-cluster-be-conf
defaultMode: 420
initContainers:
- name: default-init
image: 'selectdb/alpine:latest'
command:
- /bin/sh
args:
- '-c'
- sysctl -w vm.max_map_count=2000000 && swapoff -a
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: IfNotPresent
securityContext:
privileged: true
containers:
- name: be
image: 'selectdb/doris.be-ubuntu:3.0.2'
command:
- /opt/apache-doris/be_entrypoint.sh
args:
- $(ENV_FE_ADDR)
ports:
- name: be-port
containerPort: 9060
protocol: TCP
- name: webserver-port
containerPort: 8040
protocol: TCP
- name: heartbeat-port
containerPort: 9050
protocol: TCP
- name: brpc-port
containerPort: 8060
protocol: TCP
env:
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: POD_IP
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: status.podIP
- name: HOST_IP
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: status.hostIP
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: CONFIGMAP_MOUNT_PATH
value: /etc/doris
- name: USER
value: root
- name: DORIS_ROOT
value: /opt/apache-doris
- name: ENV_FE_ADDR
value: doris-cluster-fe-service
- name: FE_QUERY_PORT
value: '9030'
resources:
limits:
cpu: '8'
memory: 16Gi
requests:
cpu: '1'
memory: 1Gi
volumeMounts:
- name: podinfo
mountPath: /etc/podinfo
- name: be-storage
mountPath: /opt/apache-doris/be/storage
- name: be-log
mountPath: /opt/apache-doris/be/log
- name: doris-cluster-be-conf
mountPath: /etc/doris
- name: basic-auth
mountPath: /etc/basic_auth
livenessProbe:
tcpSocket:
port: 9050
initialDelaySeconds: 80
timeoutSeconds: 180
periodSeconds: 5
successThreshold: 1
failureThreshold: 3
readinessProbe:
httpGet:
path: /api/health
port: 8040
scheme: HTTP
timeoutSeconds: 1
periodSeconds: 5
successThreshold: 1
failureThreshold: 3
startupProbe:
tcpSocket:
port: 9050
timeoutSeconds: 1
periodSeconds: 5
successThreshold: 1
failureThreshold: 60
lifecycle:
preStop:
exec:
command:
- /opt/apache-doris/be_prestop.sh
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: IfNotPresent
restartPolicy: Always
terminationGracePeriodSeconds: 30
dnsPolicy: ClusterFirst
securityContext: {}
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app.kubernetes.io/component
operator: In
values:
- doris-cluster-be
topologyKey: kubernetes.io/hostname
schedulerName: default-scheduler
volumeClaimTemplates:
- kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: be-storage
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: '10'
storageClassName: nfs-sc
volumeMode: Filesystem
- kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: be-log
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: '10'
storageClassName: nfs-sc
volumeMode: Filesystem
serviceName: doris-cluster-be-internal
podManagementPolicy: Parallel
---
kind: Service
apiVersion: v1
metadata:
name: doris-cluster-be-internal
labels:
app.kubernetes.io/component: doris-cluster-be-internal
spec:
ports:
- name: heartbeat-port
protocol: TCP
port: 9050
targetPort: 9050
selector:
app.kubernetes.io/component: doris-cluster-be
clusterIP: None
type: ClusterIP- 建立資源
執行下面的命令，建立資源。
kubectl apply -f doris-cluster-be-sts.yaml -n opsxlab- 驗證資源
執行下面的命令，檢視建立結果。
$ kubectl get sts,pod -n opsxlab
NAME READY AGE
statefulset.apps/doris-cluster-be 3/3 37s
statefulset.apps/doris-cluster-fe 3/3 12m
NAME READY STATUS RESTARTS AGE
pod/doris-cluster-be-0 1/1 Running 0 37s
pod/doris-cluster-be-1 1/1 Running 0 37s
pod/doris-cluster-be-2 1/1 Running 0 37s
pod/doris-cluster-fe-0 1/1 Running 0 13m
pod/doris-cluster-fe-1 1/1 Running 0 13m
pod/doris-cluster-fe-2 1/1 Running 0 12m3. 認證管理
登入 Doris 檢視使用者資訊並設定密碼。
- 執行下面的命令，進入 Pod
doris-fe-0的終端，連線 Doris 服務。
kubectl exec -n opsxlab -it doris-cluster-fe-0 -- /bin/bash- 在
doris-fe-0內，執行 mysql 命令，使用 root 使用者訪問 Doris 服務對應的 NodePort 埠（需要密碼），並檢視使用者及許可權。
root@doris-cluster-fe-0:/opt/apache-doris# mysql -uroot -P31622 -h192.168.9.91 -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 654
Server version: 5.7.99
Copyright (c) 2000, 2024, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> SHOW ALL GRANTS\G;
*************************** 1. row ***************************
UserIdentity: 'root'@'%'
Comment: ROOT
Password: Yes
Roles: operator
GlobalPrivs: Node_priv,Admin_priv
CatalogPrivs: NULL
DatabasePrivs: internal.information_schema: Select_priv; internal.mysql: Select_priv
TablePrivs: NULL
ColPrivs: NULL
ResourcePrivs: NULL
CloudClusterPrivs: NULL
CloudStagePrivs: NULL
StorageVaultPrivs: NULL
WorkloadGroupPrivs: normal: Usage_priv
ComputeGroupPrivs: NULL
*************************** 2. row ***************************
UserIdentity: 'admin'@'%'
Comment: ADMIN
Password: No
Roles: admin
GlobalPrivs: Admin_priv
CatalogPrivs: NULL
DatabasePrivs: internal.information_schema: Select_priv; internal.mysql: Select_priv
TablePrivs: NULL
ColPrivs: NULL
ResourcePrivs: NULL
CloudClusterPrivs: NULL
CloudStagePrivs: NULL
StorageVaultPrivs: NULL
WorkloadGroupPrivs: normal: Usage_priv
ComputeGroupPrivs: NULL
2 rows in set (0.05 sec)
ERROR:
No query specified提示：輸出結果顯示 root 使用者設定了密碼，admin 使用者沒有設定密碼。
- 執行 mysql 命令，使用 admin 使用者登入 Doris（無需密碼）
root@doris-cluster-fe-0:/opt/apache-doris# mysql -uadmin -P31622 -h192.168.9.91
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 680
Server version: 5.7.99 Doris version doris-3.0.2-rc03-c21b9f5bce
Copyright (c) 2000, 2024, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>- 請立即為管理員（admin）賬戶設定密碼。
SET PASSWORD FOR 'admin'@'%' = PASSWORD('PleaseChangeMe');- 再次檢視許可權，admin 使用者的 Password 項的值已經變成 Yes。
mysql> SHOW ALL GRANTS\G;
*************************** 2. row ***************************
UserIdentity: 'admin'@'%'
Comment: ADMIN
Password: Yes
Roles: admin
GlobalPrivs: Admin_priv
CatalogPrivs: NULL
DatabasePrivs: internal.information_schema: Select_priv; internal.mysql: Select_priv
TablePrivs: NULL
ColPrivs: NULL
ResourcePrivs: NULL
CloudClusterPrivs: NULL
CloudStagePrivs: NULL
StorageVaultPrivs: NULL
WorkloadGroupPrivs: normal: Usage_priv
ComputeGroupPrivs: NULL
2 rows in set (0.01 sec)退出 MySQL 控制檯，再次使用 admin 使用者登入，不輸入密碼，提示許可權拒絕。
root@doris-cluster-fe-0:/opt/apache-doris# mysql -uadmin -P31622 -h192.168.9.91
ERROR 1045 (28000): Access denied for user 'admin@10.233.93.0' (using password: NO)再次使用 admin 使用者登入，輸入密碼，可以登入 Doris 系統。
root@doris-cluster-fe-0:/opt/apache-doris# mysql -uadmin -P31622 -h192.168.9.91 -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 719
Server version: 5.7.99 Doris version doris-3.0.2-rc03-c21b9f5bce
Copyright (c) 2000, 2024, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>4. Doris 圖形化功能概覽
Doris FE 內建 Web UI。使用者無須安裝 MySQL 客戶端，即可透過內建的 Web UI 進行 SQL 查詢和其它相關資訊的檢視。
使用瀏覽器，開啟 FE 服務 http-port 埠 8030 對應的 NodePort 31620，例如 http://192.168.9.91:31620，開啟 Doris 內建的 Web 控制檯。
- 登入頁面
輸入使用者名稱 admin 及密碼，點選「Login」登入系統。
- Home
- Playground
- System
- Log
- QueryProfile
- Session
- Configuration
以上，就是我今天分享的全部內容。
免責宣告：
- 筆者水平有限，儘管經過多次驗證和檢查，盡力確保內容的準確性，但仍可能存在疏漏之處。敬請業界專家大佬不吝指教。
- 本文所述內容僅透過實戰環境驗證測試，讀者可學習、借鑑，但嚴禁直接用於生產環境。由此引發的任何問題，作者概不負責！
本文由部落格一文多發平臺 OpenWrite 釋出！