Apache Dolphin Scheduler - Docker Compose 詳解

AaronLin發表於2021-09-22
ApacheDocker

Apache DolphinScheduler 是一個分散式去中心化，易擴充套件的視覺化 DAG 工作流任務排程系統。簡稱 DS，包括 Web 及若干服務，它依賴 PostgreSQL 和 Zookeeper，自身的服務模組包括：api, alert, master, worker（有一個 logger 服務，執行在 worker 中）等。詳細部署可以參考：Docker 部署 Dolphin Scheduler

官方提供了 docker-compose.yml，位於專案的 docker/docker-swarm/ 目錄下，本文以 v1.3.8 版本為例，講解 docker-compose.yml 內的具體內容，該版本的 Compose 基於 apache/dolphinscheduler:1.3.8 的 Docker 映象，DS Docker 構建可以參考之前寫的這篇部落格：Apache Dolphin Scheduler - Dockerfile 詳解，主要的配置修改、流程啟動都封裝在 Dockerfile 中

Docker Compose

version: "3.1"

services:
  
  # PostgreSQL
  dolphinscheduler-postgresql:
    image: postgres:11.12
    environment:
      # 設定時區
      TZ: Asia/Shanghai
      # PostgreSQL 相關的配置
      POSTGRES_USER: root
      POSTGRES_PASSWORD: root
      POSTGRES_DB: dolphinscheduler
    # 資料卷
    volumes:
    - dolphinscheduler-postgresql:/var/lib/postgresql/data
    # 重啟策：在容器退出時總是重啟容器
    restart: unless-stopped
    # 配置網路
    networks:
    - dolphinscheduler

  # Zookeeper
  dolphinscheduler-zookeeper:
    image: zookeeper:3.6.3
    environment:
      TZ: Asia/Shanghai
      # Zookeeper 相關配置
      ZOO_DATA_LOG_DIR: /data
      ZOO_4LW_COMMANDS_WHITELIST: srvr,ruok,wchs,cons
    volumes:
    - dolphinscheduler-zookeeper:/data
    restart: unless-stopped
    networks:
    - dolphinscheduler

  # DS 服務模組
  dolphinscheduler-api:
    image: apache/dolphinscheduler:1.3.8
    command: api-server
    ports:
    - 12345:12345
    environment:
      TZ: Asia/Shanghai
    # 引入外部環境變數
    env_file: config.env.sh
    # 健康檢查
    healthcheck:
      test: ["CMD", "/root/checkpoint.sh", "ApiApplicationServer"]
      interval: 30s
      timeout: 5s
      retries: 3
    # 依賴 PostgreSQL 和 Zookeeper
    depends_on:
    - dolphinscheduler-postgresql
    - dolphinscheduler-zookeeper
    volumes:
    - dolphinscheduler-logs:/opt/dolphinscheduler/logs
    - dolphinscheduler-shared-local:/opt/soft
    - dolphinscheduler-resource-local:/dolphinscheduler
    restart: unless-stopped
    networks:
    - dolphinscheduler

  dolphinscheduler-alert:
    image: apache/dolphinscheduler:1.3.8
    command: alert-server
    environment:
      TZ: Asia/Shanghai
    env_file: config.env.sh
    healthcheck:
      test: ["CMD", "/root/checkpoint.sh", "AlertServer"]
      interval: 30s
      timeout: 5s
      retries: 3
    depends_on:
    - dolphinscheduler-postgresql
    volumes:
    - dolphinscheduler-logs:/opt/dolphinscheduler/logs
    restart: unless-stopped
    networks:
    - dolphinscheduler

  dolphinscheduler-master:
    image: apache/dolphinscheduler:1.3.8
    command: master-server
    environment:
      TZ: Asia/Shanghai
    env_file: config.env.sh
    healthcheck:
      test: ["CMD", "/root/checkpoint.sh", "MasterServer"]
      interval: 30s
      timeout: 5s
      retries: 3
    depends_on:
    - dolphinscheduler-postgresql
    - dolphinscheduler-zookeeper
    volumes:
    - dolphinscheduler-logs:/opt/dolphinscheduler/logs
    - dolphinscheduler-shared-local:/opt/soft
    restart: unless-stopped
    networks:
    - dolphinscheduler

  dolphinscheduler-worker:
    image: apache/dolphinscheduler:1.3.8
    command: worker-server
    environment:
      TZ: Asia/Shanghai
    env_file: config.env.sh
    healthcheck:
      test: ["CMD", "/root/checkpoint.sh", "WorkerServer"]
      interval: 30s
      timeout: 5s
      retries: 3
    depends_on:
    - dolphinscheduler-postgresql
    - dolphinscheduler-zookeeper
    volumes:
    - dolphinscheduler-worker-data:/tmp/dolphinscheduler
    - dolphinscheduler-logs:/opt/dolphinscheduler/logs
    - dolphinscheduler-shared-local:/opt/soft
    - dolphinscheduler-resource-local:/dolphinscheduler
    restart: unless-stopped
    networks:
    - dolphinscheduler
# 宣告使用到的網路
networks:
  dolphinscheduler:
    driver: bridge

# 宣告使用到的資料卷
volumes:
  dolphinscheduler-postgresql:
  dolphinscheduler-zookeeper:
  dolphinscheduler-worker-data:
  dolphinscheduler-logs:
  dolphinscheduler-shared-local:
  dolphinscheduler-resource-local:

每一個 service 都定義了 TZ 的環境變數，設定容器的時區為亞洲上海，restart 重啟策略都設定為：unless-stopped，即：在容器退出時總是重啟容器

在 yml 的最後定義了 Compose 使用到的 networks 和 volumes

所有 service 使用同一個網路：dolphinscheduler，driver 定義為：bridge，預設就是 bridge，bridge 用於應用部署在不同容器，它們之間需要通訊的情況

DS 的每個服務模組都通過 env_file 匯入獨立的環境變數檔案 config.env.sh

healthcheck 是健康檢查，呼叫容器內的 checkpoint.sh，並傳入服務名稱，檢查該 Java 程式是否存在。兩次健康檢查的間隔 30s，超時時間為 5s，如果超過這個時間，本次健康檢查就被視為失敗，retries 重試次數設定為 3，當連續失敗指定次數後，則將容器狀態視為 unhealthy

PostgreSQL：POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_DB 分別定義了 PostgreSQL 的使用者名稱、密碼、一個名為：dolphinscheduler 的資料庫

Zookeeper：新增環境變數 ZOO_4LW_COMMANDS_WHITELIST: srvr,ruok,wchs,cons，把這四個命令加入白名單，避免在使用這四個四字命令時提示：stat is not executed because it is not in the whitelist

conf 配置資訊

#============================================================================
# Database
#============================================================================
# postgresql
DATABASE_TYPE=postgresql
DATABASE_DRIVER=org.postgresql.Driver
DATABASE_HOST=dolphinscheduler-postgresql
DATABASE_PORT=5432
DATABASE_USERNAME=root
DATABASE_PASSWORD=root
DATABASE_DATABASE=dolphinscheduler
DATABASE_PARAMS=characterEncoding=utf8
# mysql
# DATABASE_TYPE=mysql
# DATABASE_DRIVER=com.mysql.jdbc.Driver
# DATABASE_HOST=dolphinscheduler-mysql
# DATABASE_PORT=3306
# DATABASE_USERNAME=root
# DATABASE_PASSWORD=root
# DATABASE_DATABASE=dolphinscheduler
# DATABASE_PARAMS=useUnicode=true&characterEncoding=UTF-8

#============================================================================
# ZooKeeper
#============================================================================
ZOOKEEPER_QUORUM=dolphinscheduler-zookeeper:2181
ZOOKEEPER_ROOT=/dolphinscheduler

#============================================================================
# Common
#============================================================================
# common opts
DOLPHINSCHEDULER_OPTS=
# common env
DATA_BASEDIR_PATH=/tmp/dolphinscheduler
RESOURCE_STORAGE_TYPE=HDFS
RESOURCE_UPLOAD_PATH=/dolphinscheduler
FS_DEFAULT_FS=file:///
FS_S3A_ENDPOINT=s3.xxx.amazonaws.com
FS_S3A_ACCESS_KEY=xxxxxxx
FS_S3A_SECRET_KEY=xxxxxxx
HADOOP_SECURITY_AUTHENTICATION_STARTUP_STATE=false
JAVA_SECURITY_KRB5_CONF_PATH=/opt/krb5.conf
LOGIN_USER_KEYTAB_USERNAME=hdfs@HADOOP.COM
LOGIN_USER_KEYTAB_PATH=/opt/hdfs.keytab
KERBEROS_EXPIRE_TIME=2
HDFS_ROOT_USER=hdfs
RESOURCE_MANAGER_HTTPADDRESS_PORT=8088
YARN_RESOURCEMANAGER_HA_RM_IDS=
YARN_APPLICATION_STATUS_ADDRESS=http://ds1:8088/ws/v1/cluster/apps/%s
# skywalking
SKYWALKING_ENABLE=false
SW_AGENT_COLLECTOR_BACKEND_SERVICES=127.0.0.1:11800
SW_GRPC_LOG_SERVER_HOST=127.0.0.1
SW_GRPC_LOG_SERVER_PORT=11800
# dolphinscheduler env
HADOOP_HOME=/opt/soft/hadoop
HADOOP_CONF_DIR=/opt/soft/hadoop/etc/hadoop
SPARK_HOME1=/opt/soft/spark1
SPARK_HOME2=/opt/soft/spark2
PYTHON_HOME=/usr/bin/python
JAVA_HOME=/usr/local/openjdk-8
HIVE_HOME=/opt/soft/hive
FLINK_HOME=/opt/soft/flink
DATAX_HOME=/opt/soft/datax

#============================================================================
# Master Server
#============================================================================
MASTER_SERVER_OPTS=-Xms1g -Xmx1g -Xmn512m
MASTER_EXEC_THREADS=100
MASTER_EXEC_TASK_NUM=20
MASTER_DISPATCH_TASK_NUM=3
MASTER_HOST_SELECTOR=LowerWeight
MASTER_HEARTBEAT_INTERVAL=10
MASTER_TASK_COMMIT_RETRYTIMES=5
MASTER_TASK_COMMIT_INTERVAL=1000
MASTER_MAX_CPULOAD_AVG=-1
MASTER_RESERVED_MEMORY=0.3

#============================================================================
# Worker Server
#============================================================================
WORKER_SERVER_OPTS=-Xms1g -Xmx1g -Xmn512m
WORKER_EXEC_THREADS=100
WORKER_HEARTBEAT_INTERVAL=10
WORKER_MAX_CPULOAD_AVG=-1
WORKER_RESERVED_MEMORY=0.3
WORKER_GROUPS=default

#============================================================================
# Alert Server
#============================================================================
ALERT_SERVER_OPTS=-Xms512m -Xmx512m -Xmn256m
# xls file
XLS_FILE_PATH=/tmp/xls
# mail
MAIL_SERVER_HOST=
MAIL_SERVER_PORT=
MAIL_SENDER=
MAIL_USER=
MAIL_PASSWD=
MAIL_SMTP_STARTTLS_ENABLE=true
MAIL_SMTP_SSL_ENABLE=false
MAIL_SMTP_SSL_TRUST=
# wechat
ENTERPRISE_WECHAT_ENABLE=false
ENTERPRISE_WECHAT_CORP_ID=
ENTERPRISE_WECHAT_SECRET=
ENTERPRISE_WECHAT_AGENT_ID=
ENTERPRISE_WECHAT_USERS=

#============================================================================
# Api Server
#============================================================================
API_SERVER_OPTS=-Xms512m -Xmx512m -Xmn256m

#============================================================================
# Logger Server
#============================================================================
LOGGER_SERVER_OPTS=-Xms512m -Xmx512m -Xmn256m

config.env.sh 定義了用到的配置，通過 env_file 的方式傳入容器，它會覆蓋容器內的預設配置

參考資料

Networking overview
Zookeeper 四字命令
zookeeper四字命令提示命令不在白名單中
The “env_file” configuration option

相關文章