redis高可用之redis-Sentinel

hotdog04發表於2015-05-27

配置:
先搭建好redis主從環境:
主: 100.5.120.118  6379
從1  100.5.120.118  6380
從2  100.5.120.118  6381
從3  100.5.120.118  6394

sentinel機器:
100.5.120.119

1、獲取安裝包(2.8以後sentinel整合到了redis中,注意此處是版本2.8
之前測使用最新的3.0 切換存在問題,無法提升slave為主)
wget http://download.redis.io/releases/redis-2.8.11.tar.gz

2、安裝:
tar -xvf  redis-2.8.11.tar.gz
cd redis-2.8.11
make
make test
make install

安裝完成後,
進入到src目錄
把 redis-sentinel命令拷貝到/usr/local/bin下

3、引數檔案配置:
[root@localhost ~]# cat /etc/sentinel.conf_new
port 26379
dir "/tmp"
sentinel monitor mymaster 100.5.120.118 6379 1
sentinel down-after-milliseconds mymaster 5000
#sentinel can-failover mymaster yes
sentinel failover-timeout mymaster 910000
sentinel parallel-syncs mymaster 2


4、啟動:
redis-sentinel  /etc/sentinel.conf_new
或者
redis-server    /etc/sentinel.conf_new --sentinel 

提示輸出:
[root@localhost ~]# redis-sentinel  /etc/sentinel.conf_new
                _._                                                 
           _.-``__ ''-._                                            
      _.-``    `.  `_.  ''-._           Redis 2.8.11 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                  
 (    '      ,       .-`  | `,    )     Running in sentinel mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 26379
 |    `-._   `._    /     _.-'    |     PID: 3311
  `-._    `-._  `-./  _.-'    _.-'                                  
 |`-._`-._    `-.__.-'    _.-'_.-'|                                 
 |    `-._`-._        _.-'_.-'    |           http://redis.io       
  `-._    `-._`-.__.-'_.-'    _.-'                                  
 |`-._`-._    `-.__.-'    _.-'_.-'|                                 
 |    `-._`-._        _.-'_.-'    |                                 
  `-._    `-._`-.__.-'_.-'    _.-'                                  
      `-._    `-.__.-'    _.-'                                      
          `-._        _.-'                                          
              `-.__.-'                                              

[3311] 27 May 15:50:13.340 # Sentinel runid is 00f8715dbff384753eb96ba969d963ed87de89bc
[3311] 27 May 15:50:13.340 # +monitor master mymaster 100.5.120.118 6379 quorum 1
[3311] 27 May 15:50:13.341 * +slave slave 100.5.120.118:6381 100.5.120.118 6381 @ mymaster 100.5.120.118 6379
[3311] 27 May 15:50:13.341 * +slave slave 100.5.120.118:6380 100.5.120.118 6380 @ mymaster 100.5.120.118 6379
[3311] 27 May 15:50:13.341 * +slave slave 100.5.120.118:6394 100.5.120.118 6394 @ mymaster 100.5.120.118 6379

 

5、測試failover,關閉6379的redis:
[3311] 27 May 15:50:23.939 # +sdown master mymaster 100.5.120.118 6379
[3311] 27 May 15:50:23.939 # +odown master mymaster 100.5.120.118 6379 #quorum 1/1
[3311] 27 May 15:50:23.939 # +new-epoch 1
[3311] 27 May 15:50:23.939 # +try-failover master mymaster 100.5.120.118 6379
[3311] 27 May 15:50:23.948 # +vote-for-leader 00f8715dbff384753eb96ba969d963ed87de89bc 1
[3311] 27 May 15:50:23.948 # +elected-leader master mymaster 100.5.120.118 6379
[3311] 27 May 15:50:23.948 # +failover-state-select-slave master mymaster 100.5.120.118 6379
[3311] 27 May 15:50:24.048 # +selected-slave slave 100.5.120.118:6380 100.5.120.118 6380 @ mymaster 100.5.120.118 6379
[3311] 27 May 15:50:24.048 * +failover-state-send-slaveof-noone slave 100.5.120.118:6380 100.5.120.118 6380 @ mymaster 100.5.120.118 6379
[3311] 27 May 15:50:24.125 * +failover-state-wait-promotion slave 100.5.120.118:6380 100.5.120.118 6380 @ mymaster 100.5.120.118 6379
[3311] 27 May 15:50:24.468 # +promoted-slave slave 100.5.120.118:6380 100.5.120.118 6380 @ mymaster 100.5.120.118 6379
[3311] 27 May 15:50:24.468 # +failover-state-reconf-slaves master mymaster 100.5.120.118 6379
[3311] 27 May 15:50:24.526 * +slave-reconf-sent slave 100.5.120.118:6394 100.5.120.118 6394 @ mymaster 100.5.120.118 6379
[3311] 27 May 15:50:24.526 * +slave-reconf-sent slave 100.5.120.118:6381 100.5.120.118 6381 @ mymaster 100.5.120.118 6379
[3311] 27 May 15:50:25.489 * +slave-reconf-inprog slave 100.5.120.118:6381 100.5.120.118 6381 @ mymaster 100.5.120.118 6379
[3311] 27 May 15:50:25.489 * +slave-reconf-done slave 100.5.120.118:6381 100.5.120.118 6381 @ mymaster 100.5.120.118 6379
[3311] 27 May 15:50:25.489 * +slave-reconf-inprog slave 100.5.120.118:6394 100.5.120.118 6394 @ mymaster 100.5.120.118 6379
[3311] 27 May 15:50:25.489 * +slave-reconf-done slave 100.5.120.118:6394 100.5.120.118 6394 @ mymaster 100.5.120.118 6379
[3311] 27 May 15:50:25.588 # +failover-end master mymaster 100.5.120.118 6379
[3311] 27 May 15:50:25.588 # +switch-master mymaster 100.5.120.118 6379 100.5.120.118 6380
[3311] 27 May 15:50:25.588 * +slave slave 100.5.120.118:6394 100.5.120.118 6394 @ mymaster 100.5.120.118 6380
[3311] 27 May 15:50:25.589 * +slave slave 100.5.120.118:6381 100.5.120.118 6381 @ mymaster 100.5.120.118 6380
[3311] 27 May 15:50:25.590 * +slave slave 100.5.120.118:6379 100.5.120.118 6379 @ mymaster 100.5.120.118 6380
[3311] 27 May 15:50:30.672 # +sdown slave 100.5.120.118:6379 100.5.120.118 6379 @ mymaster 100.5.120.118 6380
[3311] 27 May 15:50:50.954 # -sdown slave 100.5.120.118:6379 100.5.120.118 6379 @ mymaster 100.5.120.118 6380
日誌記錄了完整的切換過程,6380升級為新主節點了。
登陸到6380上檢視確認:

127.0.0.1:6380> info replication
# Replication
role:master
connected_slaves:3
slave0:ip=100.5.120.118,port=6381,state=online,offset=53624,lag=0
slave1:ip=100.5.120.118,port=6394,state=online,offset=53624,lag=1
master_repl_offset:53624
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:53623

6、啟動6379埠的redis:
[3311] 27 May 15:51:00.912 * +convert-to-slave slave 100.5.120.118:6379 100.5.120.118 6379 @ mymaster 100.5.120.118 6380
可以看到6379自動轉換成了6380的一個slave了。


整個過程包括配置還是很簡單的,遇到的問題就是版本問題:之前使用sentinel使用redis_version:3.0.1版本
發現無法完成正常切換,至於網上有人說必須使用aof關掉save才能完成切換,本人親測跟是否開啟aof無關,都能
正常切換

7、
sentinel可以通過命令配置:

新增叢集:
sentinel monitor mymaster 100.5.120.118  6380 1
日誌:
[3311] 27 May 16:52:01.853 # +monitor master mymaster 100.5.120.118 6380 quorum 1
[3311] 27 May 16:52:01.901 * +slave slave 100.5.120.118:6381 100.5.120.118 6381 @ mymaster 100.5.120.118 6380
[3311] 27 May 16:52:01.901 * +slave slave 100.5.120.118:6394 100.5.120.118 6394 @ mymaster 100.5.120.118 6380
[3311] 27 May 16:52:01.902 * +slave slave 100.5.120.118:6379 100.5.120.118 6379 @ mymaster 100.5.120.118 6380

刪除叢集:
sentinel remove mymaster
日誌:[3311] 27 May 16:50:35.098 # -monitor master mymaster 100.5.120.118 6380

列出主節點資訊:
sentinel masters
列出從節點資訊:
sentinel slaves mymaster
強制叢集failover:
sentinel failover mymaster
日誌:
[3311] 27 May 16:56:21.927 # Executing user requested FAILOVER of 'mymaster'
[3311] 27 May 16:56:21.927 # +new-epoch 2
[3311] 27 May 16:56:21.927 # +try-failover master mymaster 100.5.120.118 6380
[3311] 27 May 16:56:21.934 # +vote-for-leader 00f8715dbff384753eb96ba969d963ed87de89bc 2
[3311] 27 May 16:56:21.934 # +elected-leader master mymaster 100.5.120.118 6380
[3311] 27 May 16:56:21.934 # +failover-state-select-slave master mymaster 100.5.120.118 6380
[3311] 27 May 16:56:21.993 # +selected-slave slave 100.5.120.118:6379 100.5.120.118 6379 @ mymaster 100.5.120.118 6380
[3311] 27 May 16:56:21.993 * +failover-state-send-slaveof-noone slave 100.5.120.118:6379 100.5.120.118 6379 @ mymaster 100.5.120.118 6380
[3311] 27 May 16:56:22.059 * +failover-state-wait-promotion slave 100.5.120.118:6379 100.5.120.118 6379 @ mymaster 100.5.120.118 6380
[3311] 27 May 16:56:22.956 # +promoted-slave slave 100.5.120.118:6379 100.5.120.118 6379 @ mymaster 100.5.120.118 6380
[3311] 27 May 16:56:22.956 # +failover-state-reconf-slaves master mymaster 100.5.120.118 6380
[3311] 27 May 16:56:23.009 * +slave-reconf-sent slave 100.5.120.118:6394 100.5.120.118 6394 @ mymaster 100.5.120.118 6380
[3311] 27 May 16:56:23.995 * +slave-reconf-inprog slave 100.5.120.118:6394 100.5.120.118 6394 @ mymaster 100.5.120.118 6380
[3311] 27 May 16:56:23.995 * +slave-reconf-done slave 100.5.120.118:6394 100.5.120.118 6394 @ mymaster 100.5.120.118 6380
[3311] 27 May 16:56:24.060 * +slave-reconf-sent slave 100.5.120.118:6381 100.5.120.118 6381 @ mymaster 100.5.120.118 6380
[3311] 27 May 16:56:25.026 * +slave-reconf-inprog slave 100.5.120.118:6381 100.5.120.118 6381 @ mymaster 100.5.120.118 6380
[3311] 27 May 16:56:25.026 * +slave-reconf-done slave 100.5.120.118:6381 100.5.120.118 6381 @ mymaster 100.5.120.118 6380
[3311] 27 May 16:56:25.126 # +failover-end master mymaster 100.5.120.118 6380
[3311] 27 May 16:56:25.126 # +switch-master mymaster 100.5.120.118 6380 100.5.120.118 6379
[3311] 27 May 16:56:25.126 * +slave slave 100.5.120.118:6394 100.5.120.118 6394 @ mymaster 100.5.120.118 6379
[3311] 27 May 16:56:25.127 * +slave slave 100.5.120.118:6381 100.5.120.118 6381 @ mymaster 100.5.120.118 6379
[3311] 27 May 16:56:25.127 * +slave slave 100.5.120.118:6380 100.5.120.118 6380 @ mymaster 100.5.120.118 6379
[3311] 27 May 16:56:35.244 * +convert-to-slave slave 100.5.120.118:6380 100.5.120.118 6380 @ mymaster 100.5.120.118 6379


8、sentinel可以做叢集:
複製26379埠的配置檔案修改成自己的埠:26380,26381啟動就ok了:

26379:
Sentinel runid is 649f4731eb4eef48834b278712d6a79d08e19fe0
26380:
Sentinel runid is 9f57020ff3e36582b8d272efb09641a875534329
26381:
Sentinel runid is 308e61fca8a4bd4355b12a6f87ad9dd8788a0e25

啟動日誌:
[root@localhost ~]# redis-sentinel /etc/sentinel.conf_new2
                _._                                                 
           _.-``__ ''-._                                            
      _.-``    `.  `_.  ''-._           Redis 2.8.11 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                  
 (    '      ,       .-`  | `,    )     Running in sentinel mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 26380
 |    `-._   `._    /     _.-'    |     PID: 28024
  `-._    `-._  `-./  _.-'    _.-'                                  
 |`-._`-._    `-.__.-'    _.-'_.-'|                                 
 |    `-._`-._        _.-'_.-'    |           http://redis.io       
  `-._    `-._`-.__.-'_.-'    _.-'                                  
 |`-._`-._    `-.__.-'    _.-'_.-'|                                 
 |    `-._`-._        _.-'_.-'    |                                 
  `-._    `-._`-.__.-'_.-'    _.-'                                  
      `-._    `-.__.-'    _.-'                                      
          `-._        _.-'                                          
              `-.__.-'                                              

[28024] 27 May 17:19:53.392 # Sentinel runid is 9f57020ff3e36582b8d272efb09641a875534329
[28024] 27 May 17:19:53.392 # +monitor master mymaster 100.5.120.118 6379 quorum 2
[28024] 27 May 17:19:53.393 * +slave slave 100.5.120.118:6394 100.5.120.118 6394 @ mymaster 100.5.120.118 6379
[28024] 27 May 17:19:53.393 * +slave slave 100.5.120.118:6381 100.5.120.118 6381 @ mymaster 100.5.120.118 6379
[28024] 27 May 17:19:53.393 * +slave slave 100.5.120.118:6380 100.5.120.118 6380 @ mymaster 100.5.120.118 6379
[28024] 27 May 17:19:53.773 * +sentinel sentinel 100.5.120.119:26379 100.5.120.119 26379 @ mymaster 100.5.120.118 6379
[28024] 27 May 17:19:59.971 * +sentinel sentinel 100.5.120.119:26381 100.5.120.119 26381 @ mymaster 100.5.120.118 6379

檢視sentinel狀態:
info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
master0:name=mymaster,status=ok,address=100.5.120.118:6380,slaves=3,sentinels=3


關閉主節點的redis,模擬failover:
觀察日誌:
26379:
[28012] 27 May 17:21:46.058 # +sdown master mymaster 100.5.120.118 6379
[28012] 27 May 17:21:46.102 # +new-epoch 1
[28012] 27 May 17:21:46.102 # +vote-for-leader 9f57020ff3e36582b8d272efb09641a875534329 1
[28012] 27 May 17:21:46.159 # +odown master mymaster 100.5.120.118 6379 #quorum 3/2
[28012] 27 May 17:21:46.159 # Next failover delay: I will not start a failover before Wed May 27 17:52:06 2015
[28012] 27 May 17:21:48.119 # +config-update-from sentinel 100.5.120.119:26380 100.5.120.119 26380 @ mymaster 100.5.120.118 6379
[28012] 27 May 17:21:48.119 # +switch-master mymaster 100.5.120.118 6379 100.5.120.118 6380
[28012] 27 May 17:21:48.119 * +slave slave 100.5.120.118:6381 100.5.120.118 6381 @ mymaster 100.5.120.118 6380
[28012] 27 May 17:21:48.120 * +slave slave 100.5.120.118:6394 100.5.120.118 6394 @ mymaster 100.5.120.118 6380
[28012] 27 May 17:21:48.121 * +slave slave 100.5.120.118:6379 100.5.120.118 6379 @ mymaster 100.5.120.118 6380
[28012] 27 May 17:21:53.145 # +sdown slave 100.5.120.118:6379 100.5.120.118 6379 @ mymaster 100.5.120.118 6380
[28012] 27 May 17:24:40.046 # -sdown slave 100.5.120.118:6379 100.5.120.118 6379 @ mymaster 100.5.120.118 6380

26380:
[28024] 27 May 17:21:46.045 # +sdown master mymaster 100.5.120.118 6379
[28024] 27 May 17:21:46.100 # +odown master mymaster 100.5.120.118 6379 #quorum 2/2
[28024] 27 May 17:21:46.100 # +new-epoch 1
[28024] 27 May 17:21:46.100 # +try-failover master mymaster 100.5.120.118 6379
[28024] 27 May 17:21:46.101 # +vote-for-leader 9f57020ff3e36582b8d272efb09641a875534329 1
[28024] 27 May 17:21:46.102 # 100.5.120.119:26379 voted for 9f57020ff3e36582b8d272efb09641a875534329 1
[28024] 27 May 17:21:46.103 # 100.5.120.119:26381 voted for 9f57020ff3e36582b8d272efb09641a875534329 1
[28024] 27 May 17:21:46.163 # +elected-leader master mymaster 100.5.120.118 6379
[28024] 27 May 17:21:46.163 # +failover-state-select-slave master mymaster 100.5.120.118 6379
[28024] 27 May 17:21:46.264 # +selected-slave slave 100.5.120.118:6380 100.5.120.118 6380 @ mymaster 100.5.120.118 6379
[28024] 27 May 17:21:46.264 * +failover-state-send-slaveof-noone slave 100.5.120.118:6380 100.5.120.118 6380 @ mymaster 100.5.120.118 6379
[28024] 27 May 17:21:46.322 * +failover-state-wait-promotion slave 100.5.120.118:6380 100.5.120.118 6380 @ mymaster 100.5.120.118 6379
[28024] 27 May 17:21:47.136 # +promoted-slave slave 100.5.120.118:6380 100.5.120.118 6380 @ mymaster 100.5.120.118 6379
[28024] 27 May 17:21:47.136 # +failover-state-reconf-slaves master mymaster 100.5.120.118 6379
[28024] 27 May 17:21:47.217 * +slave-reconf-sent slave 100.5.120.118:6394 100.5.120.118 6394 @ mymaster 100.5.120.118 6379
[28024] 27 May 17:21:47.217 * +slave-reconf-sent slave 100.5.120.118:6381 100.5.120.118 6381 @ mymaster 100.5.120.118 6379
[28024] 27 May 17:21:48.186 * +slave-reconf-inprog slave 100.5.120.118:6381 100.5.120.118 6381 @ mymaster 100.5.120.118 6379
[28024] 27 May 17:21:48.186 * +slave-reconf-done slave 100.5.120.118:6381 100.5.120.118 6381 @ mymaster 100.5.120.118 6379
[28024] 27 May 17:21:48.186 * +slave-reconf-inprog slave 100.5.120.118:6394 100.5.120.118 6394 @ mymaster 100.5.120.118 6379
[28024] 27 May 17:21:48.186 * +slave-reconf-done slave 100.5.120.118:6394 100.5.120.118 6394 @ mymaster 100.5.120.118 6379
[28024] 27 May 17:21:48.251 # -odown master mymaster 100.5.120.118 6379
[28024] 27 May 17:21:48.251 # +failover-end master mymaster 100.5.120.118 6379
[28024] 27 May 17:21:48.251 # +switch-master mymaster 100.5.120.118 6379 100.5.120.118 6380
[28024] 27 May 17:21:48.252 * +slave slave 100.5.120.118:6394 100.5.120.118 6394 @ mymaster 100.5.120.118 6380
[28024] 27 May 17:21:48.252 * +slave slave 100.5.120.118:6381 100.5.120.118 6381 @ mymaster 100.5.120.118 6380
[28024] 27 May 17:21:48.253 * +slave slave 100.5.120.118:6379 100.5.120.118 6379 @ mymaster 100.5.120.118 6380
[28024] 27 May 17:21:53.302 # +sdown slave 100.5.120.118:6379 100.5.120.118 6379 @ mymaster 100.5.120.118 6380
[28024] 27 May 17:24:40.012 # -sdown slave 100.5.120.118:6379 100.5.120.118 6379 @ mymaster 100.5.120.118 6380

26381:
[28036] 27 May 17:21:45.978 # +sdown master mymaster 100.5.120.118 6379
[28036] 27 May 17:21:46.102 # +new-epoch 1
[28036] 27 May 17:21:46.103 # +vote-for-leader 9f57020ff3e36582b8d272efb09641a875534329 1
[28036] 27 May 17:21:47.092 # +odown master mymaster 100.5.120.118 6379 #quorum 3/2
[28036] 27 May 17:21:47.092 # Next failover delay: I will not start a failover before Wed May 27 17:52:06 2015
[28036] 27 May 17:21:48.119 # +config-update-from sentinel 100.5.120.119:26380 100.5.120.119 26380 @ mymaster 100.5.120.118 6379
[28036] 27 May 17:21:48.119 # +switch-master mymaster 100.5.120.118 6379 100.5.120.118 6380
[28036] 27 May 17:21:48.119 * +slave slave 100.5.120.118:6381 100.5.120.118 6381 @ mymaster 100.5.120.118 6380
[28036] 27 May 17:21:48.120 * +slave slave 100.5.120.118:6394 100.5.120.118 6394 @ mymaster 100.5.120.118 6380
[28036] 27 May 17:21:48.121 * +slave slave 100.5.120.118:6379 100.5.120.118 6379 @ mymaster 100.5.120.118 6380
[28036] 27 May 17:21:53.153 # +sdown slave 100.5.120.118:6379 100.5.120.118 6379 @ mymaster 100.5.120.118 6380
[28036] 27 May 17:24:40.013 # -sdown slave 100.5.120.118:6379 100.5.120.118 6379 @ mymaster 100.5.120.118 6380


可以看到三個節點的投票過程最後 26380節點的sentinel(獲得3票)完成failover.

關閉26379的sentinel:
26380和26381輸出:

[28024] 27 May 17:35:12.689 # +sdown sentinel 100.5.120.119:26379 100.5.120.119 26379 @ mymaster 100.5.120.118 6380

觀察info輸出:
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
master0:name=mymaster,status=ok,address=100.5.120.118:6380,slaves=3,sentinels=3
沒有自動剔除節點(bug還是功能缺失?)

測試2節點狀態下的的failover,正常完成。

修改26379的埠為26382,然後啟動sentinel:
正常加入叢集,檢視叢集狀態:
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
master0:name=mymaster,status=ok,address=100.5.120.118:6379,slaves=3,sentinels=4
節點變成4個了..
改回到26379啟動,節點還是4.
結論就是不同埠號啟動的sentinel會累計到sentinels中。


總結:
redis-sentinel提供了 redis主從的自動failover切換,節點的重啟的自動加入,並且提供了自身的叢集配置,
配置起來也很簡單方便。結合vip的話,可以在簡單的生產環境中推廣使用.

 

來自 “ ITPUB部落格 ” ,連結:http://blog.itpub.net/20625855/viewspace-1674181/,如需轉載,請註明出處,否則將追究法律責任。

相關文章