redis資料庫叢集三種模式

環境規劃:

172.16.8.23    node1   172.16.8.24    node2

172.16.8.178  node3   172.16.8.179  node4

一.主從複製

(1).節點模式分為master節點和slave節點，master節點負責寫入(也可讀)，slave節點必須是隻讀的，master和slave有任意節點宕不能自己切換或恢復

(2).一個master後面可以有多個slave，一個slave前端只能有一個master節點

(3). 當slave啟動後，主動向master傳送SYNC命令

1.下載安裝redis包

[root@node1 ~]# wget 

[root@node1 ~]# tar xvf redis-5.0.4.tar.gz -C /opt/

[root@node1 ~]# yum install make gcc gcc-c++ -y

[root@node1 ~]# cd /opt/redis-5.0.4/

[root@node1 redis-5.0.4]# make && make install

2.master修改redis配置檔案

[root@node1 redis-5.0.4]# vim redis.conf 

bind 172.16.8.23

daemonize yes

requirepass system123

[root@node1 redis-5.0.4]# src/redis-server redis.conf 

3.slave修改配置檔案(每個從庫都是這樣)

[root@node2 redis-5.0.4]# vim redis.conf 

bind 172.16.8.24

daemonize yes

replicaof 172.16.8.23 6379

masterauth system123

[root@node2 redis-5.0.4]# src/redis-server redis.conf 

4.在master節點檢查主從狀態

[root@node1 redis-5.0.4]# src/redis-cli -h 172.16.8.23

172.16.8.23:6379> AUTH system123

OK

172.16.8.23:6379> info replication

# Replication

role:master

connected_slaves:3

slave0:ip=172.16.8.24,port=6379,state=online,offset=378,lag=0

slave1:ip=172.16.8.178,port=6379,state=online,offset=378,lag=1

slave2:ip=172.16.8.179,port=6379,state=online,offset=378,lag=1

master_replid:7dd8e0a27bddfe2c275b4c9d83f6f2bac27ea1d9

master_replid2:0000000000000000000000000000000000000000

master_repl_offset:378

second_repl_offset:-1

repl_backlog_active:1

repl_backlog_size:1048576

repl_backlog_first_byte_offset:1

repl_backlog_histlen:378

172.16.8.23:6379> 

二.哨兵模式

(1).監控主從資料庫是否正常執行，master出現故障時，自動將slave轉化為master

(2).多哨兵配置的時候，哨兵之間也會自動監控，多個哨兵可以監控同一個redis

1.在master節點配置

[root@node1 redis-5.0.4]# vim sentinel.conf 

protected-mode no

daemonize yes             --服務後臺啟動

logfile "/tmp/1.log"      --生成日誌

sentinel monitor mymaster 172.16.8.23 6379 1    --mymaster自己定義,IP地址,埠,哨兵個數

sentinel auth-pass mymaster system123              --redis密碼

[root@node1 redis-5.0.4]# src/redis-sentinel sentinel.conf

[root@node1 redis-5.0.4]# src/redis-cli -h 172.16.8.23

172.16.8.23:6379> AUTH system123

OK

172.16.8.23:6379> info replication

# Replication

role:master

connected_slaves:3

slave0:ip=172.16.8.24,port=6379,state=online,offset=4765,lag=1

slave1:ip=172.16.8.179,port=6379,state=online,offset=4765,lag=1

slave2:ip=172.16.8.178,port=6379,state=online,offset=4765,lag=1

master_replid:a7978e6db27163acc91cbc673114855f98dbc7a8

master_replid2:0000000000000000000000000000000000000000

master_repl_offset:4765

second_repl_offset:-1

repl_backlog_active:1

repl_backlog_size:1048576

repl_backlog_first_byte_offset:1

repl_backlog_histlen:4765

172.16.8.23:6379> SHUTDOWN          --將master的redis服務關閉

not connected> exit

[root@node1 redis-5.0.4]# tailf /tmp/1.log       --檢視重新選master角色的日誌

30006:X 28 Mar 2019 19:09:25.753 # +sdown master mymaster 172.16.8.23 6379

30006:X 28 Mar 2019 19:09:25.754 # +odown master mymaster 172.16.8.23 6379 #quorum 1/1

30006:X 28 Mar 2019 19:09:33.124 # +new-epoch 2

30006:X 28 Mar 2019 19:09:33.124 # +try-failover master mymaster 172.16.8.23 6379

30006:X 28 Mar 2019 19:09:33.129 # +vote-for-leader 2d798dd3a84c9f80b5f1a6c73fe6f2092b65faaa 2

30006:X 28 Mar 2019 19:09:33.129 # +elected-leader master mymaster 172.16.8.23 6379

30006:X 28 Mar 2019 19:09:33.129 # +failover-state-select-slave master mymaster 172.16.8.23 6379

30006:X 28 Mar 2019 19:09:33.184 # +selected-slave slave 172.16.8.178:6379 172.16.8.178 6379 @ mymaster 172.16.8.23 6379

30006:X 28 Mar 2019 19:09:33.184 * +failover-state-send-slaveof-noone slave 172.16.8.178:6379 172.16.8.178 6379 @ mymaster 172.16.8.23 6379

30006:X 28 Mar 2019 19:09:33.268 * +failover-state-wait-promotion slave 172.16.8.178:6379 172.16.8.178 6379 @ mymaster 172.16.8.23 6379

30006:X 28 Mar 2019 19:09:34.182 # +promoted-slave slave 172.16.8.178:6379 172.16.8.178 6379 @ mymaster 172.16.8.23 6379

30006:X 28 Mar 2019 19:09:34.182 # +failover-state-reconf-slaves master mymaster 172.16.8.23 6379

30006:X 28 Mar 2019 19:09:34.234 * +slave-reconf-sent slave 172.16.8.179:6379 172.16.8.179 6379 @ mymaster 172.16.8.23 6379

30006:X 28 Mar 2019 19:09:35.170 * +slave-reconf-inprog slave 172.16.8.179:6379 172.16.8.179 6379 @ mymaster 172.16.8.23 6379

30006:X 28 Mar 2019 19:12:34.200 # +failover-end-for-timeout master mymaster 172.16.8.23 6379

30006:X 28 Mar 2019 19:12:34.201 # +failover-end master mymaster 172.16.8.23 6379

30006:X 28 Mar 2019 19:12:34.201 * +slave-reconf-sent-be slave 172.16.8.179:6379 172.16.8.179 6379 @ mymaster 172.16.8.23 6379

30006:X 28 Mar 2019 19:12:34.201 * +slave-reconf-sent-be slave 172.16.8.24:6379 172.16.8.24 6379 @ mymaster 172.16.8.23 6379

30006:X 28 Mar 2019 19:12:34.201 * +slave-reconf-sent-be slave 172.16.8.178:6379 172.16.8.178 6379 @ mymaster 172.16.8.23 6379

30006:X 28 Mar 2019 19:12:34.201 # +switch-master mymaster 172.16.8.23 6379 172.16.8.178 6379

30006:X 28 Mar 2019 19:12:34.201 * +slave slave 172.16.8.179:6379 172.16.8.179 6379 @ mymaster 172.16.8.178 6379

30006:X 28 Mar 2019 19:12:34.202 * +slave slave 172.16.8.24:6379 172.16.8.24 6379 @ mymaster 172.16.8.178 6379

30006:X 28 Mar 2019 19:12:34.202 * +slave slave 172.16.8.23:6379 172.16.8.23 6379 @ mymaster 172.16.8.178 6379

30006:X 28 Mar 2019 19:13:04.272 # +sdown slave 172.16.8.24:6379 172.16.8.24 6379 @ mymaster 172.16.8.178 6379

30006:X 28 Mar 2019 19:13:04.272 # +sdown slave 172.16.8.23:6379 172.16.8.23 6379 @ mymaster 172.16.8.178 6379

2.在 172.16.8.178節點檢視master節點

[root@node3 redis-5.0.4]# src/redis-cli -h 172.16.8.178       --node3節點被選為master角色

172.16.8.178:6379> info replication

# Replication

role:master

connected_slaves:0

master_replid:ebd18241e560c1ccea06bc4f528071bdc189405a

master_replid2:a7978e6db27163acc91cbc673114855f98dbc7a8

master_repl_offset:12131

second_repl_offset:6452

repl_backlog_active:1

repl_backlog_size:1048576

repl_backlog_first_byte_offset:1

repl_backlog_histlen:12131

172.16.8.178:6379> 

三.叢集模式(必須要有6臺redis伺服器,node1,2,3分別安裝三個redis,node4安裝三個redis)

1.修改配置檔案,啟動服務

[root@node1 redis-5.0.4]# vim redis.conf      --6臺redis配置檔案一樣,都啟動服務

bind 本機IP

daemonize yes 

appendonly yes

cluster-enabled yes

cluster-config-file nodes_node1.conf

cluster-node-timeout 15000

[root@node1 redis-5.0.4]# src/redis-cli -h 172.16.8.23      --必須清空redis中的資料

172.16.8.23:6379> FLUSHALL

OK

172.16.8.23:6379> SHUTDOWN

not connected> exit

[root@node1 redis-5.0.4]# src/redis-server  redis.conf 

13536:C 01 Apr 2019 15:26:15.629 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo

13536:C 01 Apr 2019 15:26:15.629 # Redis version=5.0.4, bits=64, commit=00000000, modified=0, pid=13536, just started

13536:C 01 Apr 2019 15:26:15.629 # Configuration loaded

9785:C 01 Apr 2019 14:23:34.529 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo

9785:C 01 Apr 2019 14:23:34.529 # Redis version=5.0.4, bits=64, commit=00000000, modified=0, pid=9785, just started

9785:C 01 Apr 2019 14:23:34.529 # Configuration loaded

[root@node1 redis-5.0.4]#

2.建立叢集,使用叢集

[root@node1 redis-5.0.4]# ./src/redis-cli --cluster create 172.16.8.23:6379 172.16.8.24:6379 172.16.8.178:6379 172.16.8.179:6379 

172.16.8.179:6380 172.16.8.179:6381 --cluster-replicas 1    --建立叢集成功,6個redis節點(如果建立叢集時出錯,必須

刪除flushall資料和刪除nodes_6379.conf檔案後啟 動redis服務)

>>> Performing Cluster Check (using node 172.16.8.23:6379)

M: 06026556e78db8599f13095038c15682fda37ab4 172.16.8.23:6379

   slots:[0-5460] (5461 slots) master

   1 additional replica(s)

S: 479ee18ba7b8f94870b9c7eaf474a5039024a53f 172.16.8.179:6381

   slots: (0 slots) slave

   replicates 12d0de4285f5801cf8de7683da7a50e354cd5d21

S: 9bd04e9109fad35b05e68028d3eb6c11886afe0e 172.16.8.179:6379

   slots: (0 slots) slave

   replicates 454cc89b5a6644199ae7895672c29078208495b9

M: 12d0de4285f5801cf8de7683da7a50e354cd5d21 172.16.8.24:6379

   slots:[5461-10922] (5462 slots) master

   1 additional replica(s)

M: 454cc89b5a6644199ae7895672c29078208495b9 172.16.8.178:6379

   slots:[10923-16383] (5461 slots) master

   1 additional replica(s)

S: ae36c01a1ec426f6443b01857b50dae868dc9a11 172.16.8.179:6380

   slots: (0 slots) slave

   replicates 06026556e78db8599f13095038c15682fda37ab4

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

[root@node1 redis-5.0.4]# src/redis-cli --cluster help    --幫助資訊

[root@node1 redis-5.0.4]# src/redis-cli --cluster info 172.16.8.23:6379      --檢視叢集狀態資訊

172.16.8.23:6379 (06026556...) -> 0 keys | 5461 slots | 1 slaves.

172.16.8.24:6379 (12d0de42...) -> 0 keys | 5462 slots | 1 slaves.

172.16.8.178:6379 (454cc89b...) -> 0 keys | 5461 slots | 1 slaves.

[OK] 0 keys in 3 masters.

0.00 keys per slot on average.

[root@ node1 redis-5.0.4]# src/redis-cli -h 172.16.8.23 -c -p 6379     --在node1節點寫入資料

172.16.8.23:6379> set k1 v1

-> Redirected to slot [12706] located at 172.16.8.178:6379

OK

172.16.8.178:6379> KEYS *

1) "k1"

172.16.8.178:6379> get k1

"v1"

172.16.8.178:6379> 

3.在其它節點檢視資料

[root@ node2 redis-5.0.4]# src/redis-cli -h 172.16.8.24 -c -p 6379     --在node2節點檢視資料

172.16.8.24:6379> get k1

-> Redirected to slot [12706] located at 172.16.8.178:6379

"v1"

172.16.8.178:6379> 

錯誤處理:

(1).redis.conf配置檔案有開啟auth認證,需要關閉認證

[root@node1 redis-5.0.4]# redis-cli --cluster create 172.16.8.23:6379 172.16.8.24:6379 172.16.8.178:6379 172.16.8.179:6379 --cluster-replicas 

1

[ERR] Node 172.16.8.23:6379 NOAUTH Authentication required.

[root@node1 redis-5.0.4]# 

解決方法:

[root@node1 redis-5.0.4]# vim redis.conf

#masterauth system123

[root@node1 redis-5.0.4]#

(2).每個redis必須為空,沒有資料

[root@node1 redis-5.0.4]# redis-cli --cluster create 172.16.8.23:6379 172.16.8.24:6379 172.16.8.178:6379 172.16.8.179:6379 --cluster-replicas 

1

[ERR] Node 172.16.8.23:6379 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in 

database 0.

[root@node1 redis-5.0.4]# 

解決方法:

[root@node1 redis-5.0.4]#src/redis-cli -h 172.16.8.24

172.16.8.24:6379> FLUSHALL

OK

172.16.8.24:6379> 

(3).redis節點資料必須要6個,6個以下不行

[root@node1 redis-5.0.4]# ./src/redis-cli --cluster create 172.16.8.23:6379 172.16.8.24:6379 172.16.8.178:6379 --cluster-replicas 1

*** ERROR: Invalid configuration for cluster creation.

*** Redis Cluster requires at least 3 master nodes.

*** This is not possible with 3 nodes and 1 replicas per node.

*** At least 6 nodes are required.

[root@node1 redis-5.0.4]#

(4).叢集槽位重新分配(重新啟動服務時必須清空redis資料和刪除nodes_6379.conf檔案)

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[ERR] Not all 16384 slots are covered by nodes.

[root@node1 redis-5.0.4]# 

解決方法:

[root@node1 redis-5.0.4]# src/redis-cli --cluster help                        --幫助資訊

[root@node1 redis-5.0.4]# src/redis-cli --cluster check 172.16.8.23:6379       --檢查各節點的狀態

[root@node1 redis-5.0.4]# src/redis-cli --cluster fix 172.16.8.23:6379            --修復節點

[root@node1 redis-5.0.4]# src/redis-cli --cluster reshard 172.16.8.23:6379    --重新分配節點