k8s部署canal-1.1.6版本實現MySQL資料庫資料同步

星空流年發表於2023-02-08

1、版本說明

軟體&映象	版本&映象資訊	說明
Kubernetes	v1.23.7	k8s伺服器
Kuboard	v3.5.2.0	k8s連線管理工具
Canal	v1.1.6	資料同步
Canal-deployer	canal/canal-server:latest	canal-deplyer映象版本資訊
Canal-adapter	funnyzak/canal-adapter:1.1.6	canal-adapter映象版本資訊

注意：

1> canal deployer由於官方釋出的1.1.6版本不相容jdk1.8（具體原因可參看相關issue，https://github.com/alibaba/canal/issues/4358），後續官方釋出了latest版本相容jdk1.8，所以這裡deployer的映象版本採用latest版本（官方canal deployer映象地址：https://hub.docker.com/r/canal/canal-server/tags）；當然，官方1.1.6版本是相容jdk11及以上版本的，如果不想採用latest版本，也可以重新採用jdk11及以上版本自己構建docker映象即可（這裡提供一個非官方的基於jdk11的canal deployer 1.1.6版本映象，僅供參考：funnyzak/canal-server:1.1.6）

2> 關於canal-adapter映象說明，由於官方並未提供canal-adapter映象，這部分檢視issue也沒有具體說明，鑑於canal 1.1.6版本存在不相容jdk1.8版本的問題，所以當前docker hub上釋出的基於jdk1.8版本的canal-adapter 1.1.6版本映象，使用時均會發生部署失敗的問題，因此這裡使用了基於jdk11部署的canal-adapter映象，關於此映象說明可參看如下說明：https://github.com/funnyzak/canal-docker

3> 關於Kuboard說明，僅是一個k8s連線管理工具，如果介意，當然也可以採用原生方式部署，這裡不做要求

2、MySQL資料庫說明

MySQL資料庫連線	MySQL資料庫版本	說明
192.168.91.131:3306	5.7.34	源庫連線地址
192.168.91.135:3306	5.7.34	目標庫連結地址

3、k8s部署yml檔案說明

1> 當前yml檔案為兩個MySQL資料庫之間的同步，如果是MySQL和ES或者其他資料庫之間的資料同步，此文件僅供參考；如果僅同步某個庫下的某幾張表資料，可以參考官方文件（https://github.com/alibaba/canal/wiki/Sync-RDB），修改對應配置，這裡不再多做說明

2> 此yml檔案部署方式為tcp方式，如使用kafka等MQ方式，請注意修改對應配置，此文件不再進行說明

3> 當前方式部署為增量資料同步，如需全量同步，可參考官方文件：https://github.com/alibaba/canal/wiki/ClientAdapter#324-%E6%89%8B%E5%8A%A8etl

4> 當前部署資料同步方式採用映象同步，即源庫和目標庫之間的schema一致，從而實現兩個MySQL資料庫下的整個資料庫中所有的資料表同步；若是單庫單表資料同步，相關配置可參考官方文件（https://github.com/alibaba/canal/wiki/Sync-RDB），這裡不再贅述

5> Canal相關配置透過configMap方式掛載，關於掛載到對應伺服器檔案的方式，感興趣的小夥伴可以自行配置，這裡不再贅述

6> 使用Kuboard工具可以直接複製如下部署檔案，透過yml方式建立；當然透過其他k8s連線工具建立也是可以的，例如：kubesphere等

7> yml檔案部署時，可以參考如下部署順序，以減少部署失敗的次數

（1）請先部署canal deployer、canal adapter對應的configMap檔案

（2）之後再部署Zookeeper服務

（3）然後部署canal deployer服務，確保deployer服務部署啟動成功

（4）最後再部署canal adapter服務

8> 部署過程中如果遇到部署問題，可檢視issue尋找解決方案，官方issue地址：https://github.com/alibaba/canal/issues

3.1、Canal deployer部署檔案

---
kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    app: canal-deployer
  name: canal-deployer
  namespace: canal-mysql
spec:
  replicas: 1
  minReadySeconds: 10
  selector:
    matchLabels:
      app: canal-deployer
  template:
    metadata:
      name: canal-deployer
      labels:
        app: canal-deployer
    spec:
      volumes:
        - name: conf
          configMap:
            defaultMode: 493
            name: canal-deployer-configmap
      containers:
      - name: canal-deployer
        image: canal/canal-server:latest
        imagePullPolicy: IfNotPresent
        volumeMounts:
        - mountPath: /home/admin/canal-server/conf/canal.properties
          name: conf
          subPath: canal.properties
        - mountPath: /home/admin/canal-server/conf/example/instance.properties
          name: conf
          subPath: instance.properties
        - mountPath: /home/admin/app.sh
          name: conf
          subPath: app.sh
        ports:
        - containerPort: 11111
          protocol: TCP
        - containerPort: 11112
          protocol: TCP

---
kind: Service
apiVersion: v1
metadata:
  name: canal-deployer
  namespace: canal-mysql
  labels:
    app: canal-deployer
spec:
  ports:
  - port: 11111
    name: deployer
  - port: 11112
    name: metrics
  clusterIP: None
  selector:
    app: canal-deployer

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: canal-deployer-configmap
  namespace: canal-mysql
data:
  app.sh: |-
    #!/bin/bash
    set -e

    source /etc/profile
    export JAVA_HOME=/usr/java/latest
    export PATH=$JAVA_HOME/bin:$PATH
    touch /tmp/start.log
    host=`hostname -i`

    # waitterm
    #   wait TERM/INT signal.
    #   see: http://veithen.github.io/2014/11/16/sigterm-propagation.html
    waitterm() {
            local PID
            # any process to block
            tail -f /dev/null &
            PID="$!"
            # setup trap, could do nothing, or just kill the blocker
            trap "kill -TERM ${PID}" TERM INT
            # wait for signal, ignore wait exit code
            wait "${PID}" || true
            # clear trap
            trap - TERM INT
            # wait blocker, ignore blocker exit code
            wait "${PID}" 2>/dev/null || true
    }

    # waittermpid "${PIDFILE}".
    #   monitor process by pidfile && wait TERM/INT signal.
    #   if the process disappeared, return 1, means exit with ERROR.
    #   if TERM or INT signal received, return 0, means OK to exit.
    waittermpid() {
            local PIDFILE PID do_run error
            PIDFILE="${1?}"
            do_run=true
            error=0
            trap "do_run=false" TERM INT
            while "${do_run}" ; do
                    PID="$(cat "${PIDFILE}")"
                    if ! ps -p "${PID}" >/dev/null 2>&1 ; then
                            do_run=false
                            error=1
                    else
                            sleep 1
                    fi
            done
            trap - TERM INT
            return "${error}"
    }

    function checkStart() {
        local name=$1
        local cmd=$2
        local timeout=$3
        cost=5
        while [ $timeout -gt 0 ]; do
            ST=`eval $cmd`
            if [ "$ST" == "0" ]; then
                sleep 1
                let timeout=timeout-1
                let cost=cost+1
            elif [ "$ST" == "" ]; then
                sleep 1
                let timeout=timeout-1
                let cost=cost+1
            else
                break
            fi
        done
        echo "start $name successful"
    }

    function start_canal() {
        echo "start canal ..."
        managerAddress=`perl -le 'print $ENV{"canal.admin.manager"}'`
        if [ ! -z "$managerAddress" ] ; then
            # canal_local.properties mode
            adminPort=`perl -le 'print $ENV{"canal.admin.port"}'`
            if [ -z "$adminPort" ] ; then
                adminPort=11110
            fi

            su root -c 'cd /home/admin/canal-server/bin/ && sh restart.sh local 1>>/tmp/start.log 2>&1'
            sleep 5
            #check start
            checkStart "canal" "nc 127.0.0.1 $adminPort -w 1 -z | wc -l" 30
        else
            metricsPort=`perl -le 'print $ENV{"canal.metrics.pull.port"}'`
            if [ -z "$metricsPort" ] ; then
                metricsPort=11112
            fi

            destination=`perl -le 'print $ENV{"canal.destinations"}'`
            if [[ "$destination" =~ ',' ]]; then
                echo "multi destination:$destination is not support"
                exit 1;
            else
                if [ "$destination" != "" ] && [ "$destination" != "example" ] ; then
                    if [ -d /home/admin/canal-server/conf/example ]; then
                        mv /home/admin/canal-server/conf/example /home/admin/canal-server/conf/$destination
                    fi
                fi 
            fi

            su root -c 'cd /home/admin/canal-server/bin/ && sh restart.sh 1>>/tmp/start.log 2>&1'
            sleep 5
            #check start
            checkStart "canal" "nc 127.0.0.1 $metricsPort -w 1 -z | wc -l" 30
        fi  
    }

    function stop_canal() {
        echo "stop canal"
        su root -c 'cd /home/admin/canal-server/bin/ && sh stop.sh 1>>/tmp/start.log 2>&1'
        echo "stop canal successful ..."
    }

    function start_exporter() {
        su root -c 'cd /home/admin/node_exporter && ./node_exporter 1>>/tmp/start.log 2>&1 &'
    }

    function stop_exporter() {
        su root -c 'killall node_exporter'
    }

    echo "==> START ..."

    start_exporter
    start_canal

    echo "==> START SUCCESSFUL ..."

    tail -f /dev/null &
    # wait TERM signal
    waitterm

    echo "==> STOP"

    stop_canal
    stop_exporter

    echo "==> STOP SUCCESSFUL ..."
  canal.properties: >-
    canal.ip =
    canal.register.ip =
    canal.port = 11111
    canal.metrics.pull.port = 11112
    # 修改點: 此處值修改為部署的zookeeper服務的地址
    canal.zkServers = zookeeper:2181
    canal.withoutNetty = false
    canal.serverMode = tcp
    canal.destinations = example
    canal.auto.scan = true
    
    canal.instance.detecting.enable = false
    canal.instance.detecting.sql = select 1
    canal.instance.detecting.interval.time = 3
    canal.instance.detecting.retry.threshold = 3
    canal.instance.detecting.heartbeatHaEnable = false
    
    canal.instance.transaction.size =  1024
    canal.instance.fallbackIntervalInSeconds = 60
    
    canal.instance.network.receiveBufferSize = 16384
    canal.instance.network.sendBufferSize = 16384
    canal.instance.network.soTimeout = 30
    
    canal.instance.filter.druid.ddl = true
    canal.instance.filter.query.dcl = false
    canal.instance.filter.query.dml = false
    canal.instance.filter.query.ddl = false
    canal.instance.filter.table.error = false
    canal.instance.filter.rows = false
    canal.instance.filter.transaction.entry = false
    canal.instance.filter.dml.insert = false
    canal.instance.filter.dml.update = false
    canal.instance.filter.dml.delete = false
    
    canal.conf.dir = ../conf
    canal.instance.binlog.format = ROW,STATEMENT,MIXED
    canal.instance.binlog.image = FULL,MINIMAL,NOBLOB
    canal.instance.get.ddl.isolation = false
    canal.instance.parser.parallel = true
    canal.instance.parser.parallelThreadSize = 16
    
    canal.instance.tsdb.enable = false
    canal.instance.tsdb.dir = ${canal.conf.dir}/tsdb
    canal.instance.tsdb.url = jdbc:h2:${canal.instance.tsdb.dir}/h2;CACHE_SIZE=1000;MODE=MYSQL;
    canal.instance.tsdb.dbUsername = canal
    canal.instance.tsdb.dbPassword = canal
    canal.instance.tsdb.snapshot.interval = 24
    canal.instance.tsdb.snapshot.expire = 360
    canal.instance.tsdb.spring.xml = classpath:spring/tsdb/h2-tsdb.xml
    
    canal.instance.global.mode = spring
    canal.instance.global.lazy = false
    canal.instance.global.spring.xml = classpath:spring/default-instance.xml
  instance.properties: |-
    canal.instance.tsdb.enable = true
    canal.instance.gtidon = false
    # 修改點：將此處值修改為源庫的地址
    canal.instance.master.address = 192.168.91.131:3306
    canal.instance.master.journal.name =
    canal.instance.master.position =
    canal.instance.master.timestamp=
    canal.instance.master.gtid=
    # 修改點: 將此處的使用者名稱、密碼修改為源庫對應的使用者名稱、密碼
    canal.instance.dbUsername = root
    canal.instance.dbPassword = root@123456
    canal.instance.connectionCharset = UTF-8
    canal.instance.enableDruid = false
    canal.instance.filter.regex=.*\\..*
    # 過濾控制檯列印與配置表無關的日誌資訊
    canal.instance.filter.black.regex=mysql\\.slave_.*
    canal.mq.topic = example
    canal.mq.partition = 0

說明：

1> configMap中的app.sh檔案，主要是覆寫了官方映象中的app.sh檔案，原因是因為我這裡系統的使用者名稱為 root，而官方映象中的使用者名稱為 admin，如果不覆寫app.sh檔案，會導致部署失敗，具體失敗原因可參看我的另一篇文章：https://www.cnblogs.com/cndarren/p/16746300.html

2> canal deployer掛載配置的路徑與映象中對應資料夾路徑保持一致，映象資料夾路徑可參看：https://hub.docker.com/layers/canal/canal-server/latest/images/sha256-0d1018759efd92ad331c7cc379afa766c8d943ef48ef8d208ade646f54bf1565?context=explore

3.2、Canal adapter部署檔案

---
kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    app: canal-adapter
  name: canal-adapter
  namespace: canal-mysql
spec:
  replicas: 1
  minReadySeconds: 10
  selector:
    matchLabels:
      app: canal-adapter
  template:
    metadata:
      name: canal-adapter
      labels:
        app: canal-adapter
    spec:
      volumes:
      - name: conf
        defaultMode: 420
        configMap:
          name: canal-adapter-configmap
      containers:
      - name: canal-adapter
        image: funnyzak/canal-adapter:1.1.6
        imagePullPolicy: IfNotPresent
        volumeMounts:
          - mountPath: /opt/canal/canal-adapter/conf/application.yml
            name: conf
            subPath: application.yml
          - mountPath: /opt/canal/canal-adapter/conf/bootstrap.yml
            name: conf
            subPath: bootstrap.yml
          - mountPath: /opt/canal/canal-adapter/conf/rdb/monitoralter.yml
            name: conf
            subPath: monitoralter.yml
        ports:
        - containerPort: 8081
          protocol: TCP
 
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: canal-adapter-configmap
  namespace: canal-mysql
data:
  application.yml: |-
    server:
      port: 8081
    logging:
      level:
        org.springframework: WARN
        com.alibaba.otter.canal.client.adapter.rdb: WARN
    spring:
      jackson:
        date-format: yyyy-MM-dd HH:mm:ss
        time-zone: GMT+8
        default-property-inclusion: non_null
    canal.conf:
      mode: tcp
      flatMessage: true
      # 修改點：zookeeper服務對應的地址
      zookeeperHosts: zookeeper:2181
      syncBatchSize: 1000
      retries: 0
      timeout:
      accessKey:
      secretKey:
      consumerProperties:
        # 修改點：由於採用tcp方式，所以這裡配置就是tcp方式下canal deployer對應的地址
        canal.tcp.server.host: canal-deployer:11111
        canal.tcp.zookeeper.hosts:
        canal.tcp.batch.size: 500
        canal.tcp.username:
        canal.tcp.password:
      # 修改點：源庫資料來源配置資訊
      srcDataSources:
        monitorAlterDS:
          url: jdbc:mysql://192.168.91.131:3306/monitor_alert?useUnicode=true&characterEncoding=utf8&autoReconnect=true&useSSL=false
          username: root
          password: root@123456
      # 修改點：目標庫資料來源配置資訊
      canalAdapters:
      - instance: example
        groups:
        - groupId: g1
          outerAdapters:
          - name: logger
          - name: rdb
            key: mysql1
            properties:
              jdbc.driverClassName: com.mysql.jdbc.Driver
              jdbc.url: jdbc:mysql://192.168.91.135:3306/monitor_alert?useUnicode=true&characterEncoding=utf8&autoReconnect=true&useSSL=false
              jdbc.username: root
              jdbc.password: admin@4321
  bootstrap.yml:
  # 修改點：映象方式同步配置檔案
  monitoralter.yml: |-
    dataSourceKey: monitorAlterDS
    destination: example
    groupId: g1
    outerAdapterKey: mysql1
    concurrent: true
    dbMapping:
      mirrorDb: true
      database: monitor_alert

說明：

1> canal adapter掛載配置的路徑與映象中對應資料夾路徑保持一致，映象資料夾路徑可參看：https://hub.docker.com/layers/funnyzak/canal-adapter/1.1.6/images/sha256-1765db135c5e3610601f5680fa7d35f6b02f63218d8fdfe8b92d6371b1831190?context=explore

2> 如果有多個資料庫下的資料表資料需要同步，可以在canal adapter的Deployment檔案下新增對應的掛載的配置檔案路徑，然後再修改configMap檔案即可，具體操作可參考如下所示

## 例如：需要再同步product資料庫下的表資料
# Deployment檔案修改如下：
# 1、canal adapter的Deployment檔案新增掛載的product資料庫資訊
containers:
- name: canal-adapter
  image: funnyzak/canal-adapter:1.1.6
  imagePullPolicy: IfNotPresent
  volumeMounts:
    - mountPath: /opt/canal/canal-adapter/conf/application.yml
      name: conf
      subPath: application.yml
    - mountPath: /opt/canal/canal-adapter/conf/bootstrap.yml
      name: conf
      subPath: bootstrap.yml
    - mountPath: /opt/canal/canal-adapter/conf/rdb/monitoralter.yml
      name: conf
      subPath: monitoralter.yml
    - mountPath: /opt/canal/canal-adapter/conf/rdb/product.yml
      name: conf
      subPath: product.yml

# configMap修改如下：
# 1、application.yml檔案新增源庫、目標庫的地址資訊
# 源庫資訊
srcDataSources:
  monitorAlterDS:
    url: jdbc:mysql://192.168.91.131:3306/monitor_alert?useUnicode=true&characterEncoding=utf8&autoReconnect=true&useSSL=false
    username: root
    password: root@123456
  productDS:
    url: jdbc:mysql://192.168.91.131:3306/product?useUnicode=true&characterEncoding=utf8&autoReconnect=true&useSSL=false
    username: root
    password: root@123456
# 目標庫資訊
canalAdapters:
- instance: example
  groups:
  - groupId: g1
    outerAdapters:
    - name: logger
    - name: rdb
      key: mysql1
      properties:
        jdbc.driverClassName: com.mysql.jdbc.Driver
        jdbc.url: jdbc:mysql://192.168.91.135:3306/monitor_alert?useUnicode=true&characterEncoding=utf8&autoReconnect=true&useSSL=false
        jdbc.username: root
        jdbc.password: admin@4321
     - name: rdb
       key: mysql2
       properties:
         jdbc.driverClassName: com.mysql.jdbc.Driver
         jdbc.url: jdbc:mysql://192.168.91.135:3306/product?useUnicode=true&characterEncoding=utf8&autoReconnect=true&useSSL=false
         jdbc.username: root
         jdbc.password: admin@4321
# 2、新增對應的掛載配置檔案product.yml
product.yml: |-
  dataSourceKey: productDS
  destination: example
  groupId: g1
  outerAdapterKey: mysql2
  concurrent: true
  dbMapping:
    mirrorDb: true
    database: product

3> 關於bootstrap.yml檔案，這裡並沒有具體內容，原因僅是將canal adapter映象中的bootstrap.yml內容進行註釋，主要原因是canal adapter部署時並沒有用到對應的canal_manager資料庫配置，如果不覆寫，會導致canal adapter部署啟動時報連線不上canal_manager資料庫的異常，具體問題可參看相關issue（https://github.com/alibaba/canal/issues/4197）

3.3、Zookeeper部署檔案

---
kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    app: zk-deployment
  name: zookeeper
  namespace: canal-mysql
spec:
  replicas: 2
  selector:
    matchLabels:
      app: zk
  template:
    metadata:
      name: zk
      labels:
        app: zk
    spec:
      volumes:
      - name: localtime
        hostPath:
          path: /usr/share/zoneinfo/Asia/Shanghai
      containers:
      - name: zookeeper
        image: zookeeper:3.6.2
        imagePullPolicy: IfNotPresent
        volumeMounts:
        - name: localtime
          mountPath: /etc/localtime
          readOnly: true

---
kind: Service
apiVersion: v1
metadata:
  name: zookeeper
  namespace: canal-mysql
  labels:
    app: zk
spec:
  ports:
  - port: 2181
    name: client
  clusterIP: None
  selector:
    app: zk