Docker Redis 5.0 叢集(cluster)搭建

一角錢發表於2019-03-28

一、準備工具

安裝docker(來自官網

(1) 安裝所需的軟體包

$ sudo yum install -y yum-utils \ device-mapper-persistent-data \ lvm2
複製程式碼

(2) 使用以下命令來設定穩定的儲存庫

$ sudo yum-config-manager \ --add-repo \ https://download.docker.com/linux/centos/docker-ce.repo
複製程式碼

(3) 安裝docker ce(docker社群版)

$ sudo yum install docker-ce
複製程式碼

(4) 啟動docker

$ sudo systemctl start docker
複製程式碼

在docker庫獲取redis映象(截至2019-03-27,最新版為5.0.4)

$ docker pull redis
複製程式碼

至此,docker上redis cluster所有工具準備完畢,我們在命令列上輸入docker images,就可以檢視到已經安裝的映象

Docker Redis 5.0 叢集(cluster)搭建

二、 叢集搭建

建立Redis 容器

#####(1)建立redis配置檔案(redis-cluster.tmpl)

我在路徑/home下建立一個資料夾redis-cluster,在路徑/home/redis-cluster下建立一個檔案redis-cluster.tmpl,並把以下內容複製過去。(注:路徑可自定義,我用的是/home/redis-cluster

port ${PORT}                                       ##節點埠
protected-mode no                                  ##開啟叢集模式
cluster-enabled yes                                ##cluster叢集模式
cluster-config-file nodes.conf                     ##叢集配置名
cluster-node-timeout 5000                          ##超時時間
cluster-announce-ip 192.168.XX.XX                  ##實際為各節點網路卡分配ip  先用上閘道器ip代替
cluster-announce-port ${PORT}                      ##節點對映埠
cluster-announce-bus-port 1${PORT}                 ##節點匯流排埠
appendonly yes                                     ##持久化模式
複製程式碼

通過命令,可檢視

[root@node-01 redis-cluster]# cat /home/redis-cluster/redis-cluster.tmpl
port ${PORT}
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
cluster-announce-ip 192.168.1.221
cluster-announce-port ${PORT}
cluster-announce-bus-port 1${PORT}
appendonly yes
複製程式碼

備註:此模版檔案為叢集節點通用檔案  其中${PORT} 將讀取命令列變數  ip則根據網路卡分配ip進行替換  以保證節點配置檔案除埠以及ip 全部一致。

(2) 建立自定義network
$ docker network create redis-net
複製程式碼

docker network ls ##檢視網路卡資訊

[root@node-01 conf]# docker network ls 
NETWORK ID NAME DRIVER SCOPE 7f804fa5fbd2 redis-net bridge local
複製程式碼

備註:建立redis-net虛擬網路卡 目的是讓docker容器能與宿主(centos7)橋接網路 並間接與外界連線

(3) 檢視redis-net虛擬網路卡閘道器ip
[root@node-01 conf]# docker network inspect redis-net | grep "Gateway" | grep --color=auto -P '(\d{1,3}.){3}\d{1,3}' -o 172.21.0.1
複製程式碼

備註:docker network inspect network-name 顯示 network-name對應配置資訊         (gerp 過濾閘道器配置行 並篩選ip ) 可人工

(4) 在/home/redis-cluster下生成conf和data目標,並生成配置資訊
 $ for port in `seq 7000 7005`; do \
  mkdir -p ./${port}/conf \
  && PORT=${port} envsubst < ./redis-cluster.tmpl > ./${port}/conf/redis.conf \
  && mkdir -p ./${port}/data; \
  done
複製程式碼

共生成6個資料夾,從7000到7005,每個資料夾下包含data和conf資料夾,同時conf裡面有redis.conf配置檔案

(5) 建立6個redis容器
$ for port in `seq 7000 7005`; do \
docker run -d -ti -p ${port}:${port} -p 1${port}:1${port} \
-v /home/redis-cluster/${port}/conf/redis.conf:/usr/local/etc/redis/redis.conf \
-v /home/redis-cluster/${port}/data:/data \
--restart always --name redis-${port} --net redis-net \
--sysctl net.core.somaxconn=1024 redis redis-server /usr/local/etc/redis/redis.conf; \
done
複製程式碼
備註:命令譯為  迴圈7010 - 7015  執行redis 容器
docker  run            執行
-d                          守護程式模式
--restart always     保持容器啟動
--name redis-710* 容器起名
--net redis-net    容器使用虛擬網路卡
-p                        指定宿主機器與容器埠對映 701*:701*
-P                        指定宿主機與容器redis匯流排埠對映 1701*:1701*
--privileged=true -v /home/redis-cluster/701*/conf/redis.conf:/usr/local/etc/redis/redis.conf
付權將宿主701*節點檔案掛載到容器/usr/local/etc/redis/redis.conf 檔案中
--privileged=true -v /home/redis-cluster/${port}/data:/data \
付權將宿主701*/data目錄掛載到容器/data目錄中
--sysctl net.core.somaxconn=1024 redis redis-server /usr/local/etc/redis/redis.conf;
容器根據掛載的配置檔案啟動 redis服務端
複製程式碼
6、通過命令docker ps可檢視剛剛生成的6個容器資訊
[root@node-01 redis-cluster]# docker ps
CONTAINER ID        IMAGE                                         COMMAND                  CREATED             STATUS                PORTS                                                              NAMES
15c479074b87        redis                                         "docker-entrypoint..."   10 seconds ago      Up 9 seconds          0.0.0.0:7005->7005/tcp, 6379/tcp, 0.0.0.0:17005->17005/tcp         redis-7005
45bec33c2c35        redis                                         "docker-entrypoint..."   11 seconds ago      Up 10 seconds         0.0.0.0:7004->7004/tcp, 6379/tcp, 0.0.0.0:17004->17004/tcp         redis-7004
45482d9e5bb8        redis                                         "docker-entrypoint..."   11 seconds ago      Up 10 seconds         0.0.0.0:7003->7003/tcp, 6379/tcp, 0.0.0.0:17003->17003/tcp         redis-7003
f633d5c767c9        redis                                         "docker-entrypoint..."   11 seconds ago      Up 10 seconds         0.0.0.0:7002->7002/tcp, 6379/tcp, 0.0.0.0:17002->17002/tcp         redis-7002
eefc0d49fedf        redis                                         "docker-entrypoint..."   11 seconds ago      Up 10 seconds         0.0.0.0:7001->7001/tcp, 6379/tcp, 0.0.0.0:17001->17001/tcp         redis-7001
58b311e5dbcb        redis                                         "docker-entrypoint..."   12 seconds ago      Up 11 seconds         0.0.0.0:7000->7000/tcp, 6379/tcp, 0.0.0.0:17000->17000/tcp         redis-7000  
複製程式碼
7、檢視容器分配ip
[root@node-01 conf]# docker network inspect redis-net
[
{
    "Name": "redis-net",
    "Id": "7f804fa5fbd2473f1e6e9783ac52aa980c355e3253b15a28e10f8793c46fc39b",
    "Created": "2019-03-27T12:59:14.45569686+08:00",
    "Scope": "local",
    "Driver": "bridge",
    "EnableIPv6": false,
    "IPAM": {
        "Driver": "default",
        "Options": {},
        "Config": [
            {
                "Subnet": "172.21.0.0/16",
                "Gateway": "172.21.0.1"
            }
        ]
    },
    "Internal": false,
    "Attachable": false,
    "Containers": {
        "449c38916b4ef919d1b1828da3f69e0f5256eb2023204f25f69fbb8c449f2163": {
            "Name": "redis-7005",
            "EndpointID": "5c89e6fdd0e983d6d2b950c9bc41616920265dda93f2852650ca5e9da1da22b1",
            "MacAddress": "02:42:ac:15:00:07",
            "IPv4Address": "172.21.0.7/16",
            "IPv6Address": ""
        },
        "600aa2f84dfbe72a19caf5dc684cb7691e89a43753f70d74355468b7659caa23": {
            "Name": "redis-7000",
            "EndpointID": "75873081222c7a2eedc5f3ee639df5cc7645790012dd72e34c97fd97dcb5cf47",
            "MacAddress": "02:42:ac:15:00:02",
            "IPv4Address": "172.21.0.2/16",
            "IPv6Address": ""
        },
        "78aeae8b2d381d3789848f45f766dd01d7c0d5443505920decf41fce751f765d": {
            "Name": "redis-7002",
            "EndpointID": "a70d816cd9e3650b1d10507a85fa85c0c2e143ba99c0974fb000ec87ad5b173d",
            "MacAddress": "02:42:ac:15:00:04",
            "IPv4Address": "172.21.0.4/16",
            "IPv6Address": ""
        },
        "7ecd2e1372a2bc15cb89fad856ab410757b49936f662ce170990c04e1861037d": {
            "Name": "redis-7004",
            "EndpointID": "f8cdb522601e9667ec06dc0dc0d39b46331b3d24cbea1155b00b73e600eca8c7",
            "MacAddress": "02:42:ac:15:00:06",
            "IPv4Address": "172.21.0.6/16",
            "IPv6Address": ""
        },
        "bd8254045c66bc6569c9a585b58cb028e864b6c7f16d836009debd3a3350913c": {
            "Name": "redis-7003",
            "EndpointID": "6a25ff0046e6d6575579cdf08bccbb2a62b29460e3932de3f8e476bb908fdc9c",
            "MacAddress": "02:42:ac:15:00:05",
            "IPv4Address": "172.21.0.5/16",
            "IPv6Address": ""
        },
        "cfef6ca3aeea9d3b4fa527538fbe2a2f79aac25d52921b4edc31839b6f330887": {
            "Name": "redis-7001",
            "EndpointID": "98a36b61bc6f50f78444a6cefa5e9999f5366e72425602511bc95f602ef0dc93",
            "MacAddress": "02:42:ac:15:00:03",
            "IPv4Address": "172.21.0.3/16",
            "IPv6Address": ""
        }
    },
    "Options": {},
    "Labels": {}
}
]
複製程式碼

備註:每個容器對應分配了ip 需要修改節點配置檔案redis.conf與其對應並重啟容器。

(8) 修改宿主掛載目錄配置檔案
vim /home/redis-cluster/700*/conf/redis.conf ##編輯ip為查詢分配的ip 並esc+:wq退出

port 7000
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
cluster-announce-ip 172.21.0.2
cluster-announce-port 7000
cluster-announce-bus-port 17000
appendonly yes
複製程式碼

啟動叢集

進入一個節點
docker exec -it redis-7000 bash
複製程式碼
使用Redis 5建立叢集,redis-cli只需鍵入:
redis-cli --cluster create 172.21.0.2:7000 172.21.0.3:7001 172.21.0.4:7002 172.21.0.5:7003 172.21.0.6:7004 172.21.0.7:7005 --cluster-replicas 1
複製程式碼
至此,叢集已經建立好了。
[root@node-01 data]# docker exec -it redis-7000 bash
root@600aa2f84dfb:/data# redis-cli --cluster create 172.21.0.2:7000  172.21.0.3:7001  172.21.0.4:7002  172.21.0.5:7003  172.21.0.6:7004  172.21.0.7:7005 --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 172.21.0.6:7004 to 172.21.0.2:7000
Adding replica 172.21.0.7:7005 to 172.21.0.3:7001
Adding replica 172.21.0.5:7003 to 172.21.0.4:7002
M: 0d403d866c4bdfa90f3a1e21b2a33356474014ee 172.21.0.2:7000
   slots:[0-5460] (5461 slots) master
M: ab4bf4833926dd7ffca5f2d34ef754466730e7d7 172.21.0.3:7001
   slots:[5461-10922] (5462 slots) master
M: 846f087ee5885f17d5a5c05d2eb2ba470441f700 172.21.0.4:7002
   slots:[10923-16383] (5461 slots) master
S: e44c2b96f1446b1860fe582547634b12a461407d 172.21.0.5:7003
   replicates 846f087ee5885f17d5a5c05d2eb2ba470441f700
S: 4efa0a9b7b4b159807900efa5f86410e55c1d2b3 172.21.0.6:7004
   replicates 0d403d866c4bdfa90f3a1e21b2a33356474014ee
S: 8a616ee30a7104bcd341d57af825c6b5b30176ff 172.21.0.7:7005
   replicates ab4bf4833926dd7ffca5f2d34ef754466730e7d7
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
....
>>> Performing Cluster Check (using node 172.21.0.2:7000)
M: 0d403d866c4bdfa90f3a1e21b2a33356474014ee 172.21.0.2:7000
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: e44c2b96f1446b1860fe582547634b12a461407d 172.21.0.5:7003
   slots: (0 slots) slave
   replicates 846f087ee5885f17d5a5c05d2eb2ba470441f700
M: ab4bf4833926dd7ffca5f2d34ef754466730e7d7 172.21.0.3:7001
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 8a616ee30a7104bcd341d57af825c6b5b30176ff 172.21.0.7:7005
   slots: (0 slots) slave
   replicates ab4bf4833926dd7ffca5f2d34ef754466730e7d7
M: 846f087ee5885f17d5a5c05d2eb2ba470441f700 172.21.0.4:7002
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 4efa0a9b7b4b159807900efa5f86410e55c1d2b3 172.21.0.6:7004
   slots: (0 slots) slave
   replicates 0d403d866c4bdfa90f3a1e21b2a33356474014ee
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
複製程式碼
連線客戶端,檢視主從資訊
root@600aa2f84dfb:/data# redis-cli -c -p 7000     
127.0.0.1:7000> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=172.21.0.6,port=7004,state=online,offset=3556,lag=1
master_replid:ef97265d34769b68096786c3e802fb3a13968fe9
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:3556
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:3556
複製程式碼

其它注意事項

宿主主機對外開放ip
for port in `seq 7000 7005`; do \  
firewall-cmd --zone=public --add-port=${port}/tcp --permanent  
done
#重新載入
firewall-cmd --reload
複製程式碼

#####需要暫停容器並刪除容器 以便重複6步驟

for port in `seq 7000 7005`; do \
  docker stop redis-${port};
  docker rm redis-${port};
done
複製程式碼

相關文章