部署Redis Cluster叢集並開啟密碼認證
如果只想簡單的搭建Redis Cluster，不需要設定密碼和公網訪問，可以參考官方文件。
節點介紹
Cluster模式推薦最少有6個節點，本次實驗搭建了6個節點，使用的埠為7000-7005。Cluster模式是資料分開存放在不同的節點上，如果有6個節點，通常設定3個主節點，每個主節點有一個從節點作為備份。正常情況下讀寫操作的是主節點，從節點會同步主節點的更變。當某個主節點掛了之後，其對應的從節點會變成主節點。如果一段時間後之前掛掉的主節點恢復了，它將變成從節點。如果某個主節點掛了之後，其對應的從節點也掛了，叢集將不可訪問。即每個主節點相互獨立，從節點作為主節點的備份。
下載和編譯
下載地址：https://redis.io/download
下載、解壓並編譯：
wget http://download.redis.io/releases/redis-6.0.5.tar.gz
tar -zxvf redis-6.0.5.tar.gz
cd cd redis-6.0.5
make
坑：6.x版本的Redis需要新版的gcc才能編譯，centOS 7 自帶的gcc版本為4.8.5，無法完成編譯，需要升級版本。
解決方案：
yum install centos-release-scl
yum install devtoolset-9-gcc*
# 執行如下僅對當前bash生效
scl enable devtoolset-9 bash
# 以上操作僅對當bash生效，若想永久生效，按以下操作：
echo "source /opt/rh/devtoolset-9/enable" >>/etc/profile
source /etc/profile
gcc升級版本之後，需要重新make編譯，編譯完成後src目錄下會生成幾個可執行檔案redis-benchmark redis-cli redis-sentinel redis-server等。
叢集部署
新建6個目錄，並將redis.conf和src目錄中的redis-server複製到每個目錄，目錄名稱可隨意，這裡採用redis執行時的埠號作為目錄的名稱。
cd /data/soft/redis/
mkdir -p /data/soft/redis/conf
cp redis.conf /data/soft/redis/conf/7001_redis.conf
cp redis.conf /data/soft/redis/conf/7002_redis.conf
cp redis.conf /data/soft/redis/conf/7003_redis.conf
cp redis.conf /data/soft/redis/conf/7004_redis.conf
cp redis.conf /data/soft/redis/conf/7005_redis.conf
cp redis.conf /data/soft/redis/conf/7006_redis.conf
cp src/redis-server /data/bin/redis-server
cp src/redis-cli /data/bin/redis-cli
修改配置
修改conf目錄中的*.conf，需要修改配置項如下：
# Redis configuration file.
# ./redis-server /path/to/redis.conf
################################## MODULES #####################################
# include /path/to/local.conf
# loadmodule /path/to/my_module.so
################################## NETWORK #####################################
# By default, if no "bind" configuration directive is specified, Redis listens
# for connections from all the network interfaces available on the server.
bind 192.168.0.239
protected-mode no
port 7005
tcp-backlog 2000
# unixsocket /tmp/redis.sock
# unixsocketperm 700
timeout 0
tcp-keepalive 300
################################# GENERAL #####################################
daemonize yes
supervised no
pidfile /log/redis/redis-7005.pid
# debug, verbose, notice, warning
loglevel notice
logfile /log/redis/redis-7005.log
# syslog-enabled no
# syslog-ident redis
# syslog-facility local0
databases 16
always-show-logo yes
################################ SNAPSHOTTING ################################
# save <seconds> <changes>
save 600 1
save 300 10
save 250 50
save 100 1000
save 80 5000
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump-7005.rdb
dir /log/redis/dump/
################################# REPLICATION #################################
# slaveof 192.168.53.5 7005
masterauth 123456
# slave-serve-stale-data yes
# slave-read-only yes
# repl-diskless-sync no
# repl-ping-slave-period 10
# repl-timeout 60
# repl-disable-tcp-nodelay no
# repl-backlog-size 5mb
# repl-backlog-ttl 3000
# slave-priority 100
# min-slaves-to-write 1
# min-slaves-max-lag 10
# slave-announce-ip 5.5.5.5
# slave-announce-port 1234
################################## SECURITY ###################################
requirepass 123456
# rename-command CONFIG XCONFIG
################################### CLIENTS ####################################
maxclients 10000
############################## MEMORY MANAGEMENT ################################
maxmemory 2048mb
# volatile-lru -> Evict using approximated LRU among the keys with an expire set.
# allkeys-lru -> Evict any key using approximated LRU.
# volatile-lfu -> Evict using approximated LFU among the keys with an expire set.
# allkeys-lfu -> Evict any key using approximated LFU.
# volatile-random -> Remove a random key among the ones with an expire set.
# allkeys-random -> Remove a random key, any key.
# volatile-ttl -> Remove the key with the nearest expire time (minor TTL)
# noeviction -> Don't evict anything, just return an error on write operations.
maxmemory-policy allkeys-lru
maxmemory-samples 5
############################# LAZY FREEING ####################################
# release memory in a non-blocking. DEL, UNLINK and ASYNC
# option of FLUSHALL and FLUSHDB are user-controlled.
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
slave-lazy-flush no
############################## APPEND ONLY MODE ###############################
appendonly yes
appendfilename appendonly-7005.aof
# always|everysec|no
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 3mb
aof-load-truncated yes
aof-use-rdb-preamble no
################################ LUA SCRIPTING ###############################
lua-time-limit 5000
################################ REDIS CLUSTER ###############################
cluster-enabled yes
cluster-config-file /log/redis/nodes-7005.conf
cluster-node-timeout 15000
cluster-slave-validity-factor 10
cluster-migration-barrier 1
cluster-require-full-coverage yes
################################## SLOW LOG ###################################
slowlog-log-slower-than 5000
slowlog-max-len 516
################################ LATENCY MONITOR ##############################
latency-monitor-threshold 0
############################# EVENT NOTIFICATION ##############################
notify-keyspace-events ""
############################### ADVANCED CONFIG ###############################
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
# -5: max size: 64 Kb <-- not recommended for normal workloads
# -4: max size: 32 Kb <-- not recommended
# -3: max size: 16 Kb <-- probably not recommended
# -2: max size: 8 Kb <-- good
# -1: max size: 4 Kb <-- good
list-max-ziplist-size -2
# 0|1|2|3
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
# normal -> normal clients including MONITOR clients
# slave -> slave clients
# pubsub -> clients subscribed to at least one pubsub channel or pattern
# client-output-buffer-limit <class> <hard limit> <soft limit> <soft seconds>
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
# client-query-buffer-limit 1gb
# proto-max-bulk-len 512mb
hz 10
aof-rewrite-incremental-fsync yes
# lfu-log-factor 10
# lfu-decay-time 1
########################### ACTIVE DEFRAGMENTATION #######################
# Enabled active defragmentation
# activedefrag yes
# Minimum amount of fragmentation waste to start active defrag
# active-defrag-ignore-bytes 100mb
# Minimum percentage of fragmentation to start active defrag
# active-defrag-threshold-lower 10
# Maximum percentage of fragmentation at which we use maximum effort
# active-defrag-threshold-upper 100
# Minimal effort for defrag in CPU percentage
# active-defrag-cycle-min 25
# Maximal effort for defrag in CPU percentage
# active-defrag-cycle-max 75
上面的配置檔案已經列出，當前為7005的埠，其他埠的配置檔案都類似，改下埠既可。
建立叢集
啟動每個節點：
/data/soft/redis/src/redis-check-aof --fix /log/redis/dump/appendonly-7001.aof
/data/soft/redis/src/redis-check-aof --fix /log/redis/dump/appendonly-7002.aof
/data/soft/redis/src/redis-check-aof --fix /log/redis/dump/appendonly-7003.aof
/data/soft/redis/src/redis-check-aof --fix /log/redis/dump/appendonly-7004.aof
/data/soft/redis/src/redis-check-aof --fix /log/redis/dump/appendonly-7005.aof
/data/soft/redis/src/redis-check-aof --fix /log/redis/dump/appendonly-7006.aof
/data/bin/redis-server /data/soft/redis/conf/redis-7001.conf >/dev/null 2>&1 &
/data/bin/redis-server /data/soft/redis/conf/redis-7002.conf >/dev/null 2>&1 &
/data/bin/redis-server /data/soft/redis/conf/redis-7003.conf >/dev/null 2>&1 &
/data/bin/redis-server /data/soft/redis/conf/redis-7004.conf >/dev/null 2>&1 &
/data/bin/redis-server /data/soft/redis/conf/redis-7005.conf >/dev/null 2>&1 &
/data/bin/redis-server /data/soft/redis/conf/redis-7006.conf >/dev/null 2>&1 &
啟動叢集：
/data/bin/redis-cli --cluster create ip:7000 ip:7001 ip:7002 ip:7003 ip:7004 ip:7005 --cluster-replicas 1 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 192.168.0.239:7006 to 192.168.0.239:7001
Adding replica 192.168.0.239:7003 to 192.168.0.239:7004
Adding replica 192.168.0.239:7005 to 192.168.0.239:7002
M: 4c8fd6640bac7463a517e10fda942568177f1175 192.168.0.239:7001
slots:[0-5460] (5461 slots) master
M: 8a924d45c5094bca66c736661f267de3ff348134 192.168.0.239:7002
slots:[10923-16383] (5461 slots) master
S: 9e6a1995558a142002579a1bf2659b0fff6a986b 192.168.0.239:7003
replicates b8985dcd19735b50a9b13ba8ef2449504b1da755
M: b8985dcd19735b50a9b13ba8ef2449504b1da755 192.168.0.239:7004
slots:[5461-10922] (5462 slots) master
S: baf6fb8fa06e58e7ca541d4f9c76ee63255e797f 192.168.0.239:7005
replicates 8a924d45c5094bca66c736661f267de3ff348134
S: 10993fd7b1cdf5d0293373bec821bdb7763426ed 192.168.0.239:7006
replicates 4c8fd6640bac7463a517e10fda942568177f1175
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
...
>>> Performing Cluster Check (using node 192.168.0.239:7001)
M: 4c8fd6640bac7463a517e10fda942568177f1175 192.168.0.239:7001
slots:[0-5460] (5461 slots) master
1 additional replica(s)
S: 9e6a1995558a142002579a1bf2659b0fff6a986b 192.168.0.239:7003
slots: (0 slots) slave
replicates b8985dcd19735b50a9b13ba8ef2449504b1da755
S: 10993fd7b1cdf5d0293373bec821bdb7763426ed 192.168.0.239:7006
slots: (0 slots) slave
replicates 4c8fd6640bac7463a517e10fda942568177f1175
M: 8a924d45c5094bca66c736661f267de3ff348134 192.168.0.239:7002
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: baf6fb8fa06e58e7ca541d4f9c76ee63255e797f 192.168.0.239:7005
slots: (0 slots) slave
replicates 8a924d45c5094bca66c736661f267de3ff348134
M: b8985dcd19735b50a9b13ba8ef2449504b1da755 192.168.0.239:7004
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
這裡的IP可以是公網IP也可以是內網IP，-a後面的引數是設定的訪問密碼，–cluster-replicas後面的引數是每個主節點對應的從節點的數量。
這時會自動分配主從節點，如果確認分配無誤，輸入yes即可建立cluster。
坑：在cluster模式中，每個節點除了要使用設定的訪問埠以外，還需要使用訪問埠加10000的埠號進行資料傳輸，所以在防火牆開放埠時也需要開啟對應的埠，否則會出現wait…的提示。
在本例中，需要開放的埠號為7001-7006和17001-17006。
測試叢集：
可以使用redis-cli來連線叢集，-c代表使用cluster模式。
/data/soft/redis-cli -c -h ip -p 7000 -a 密碼
檢視叢集資訊：
192.168.0.239:7001> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:107
cluster_stats_messages_pong_sent:111
cluster_stats_messages_sent:218
cluster_stats_messages_ping_received:106
cluster_stats_messages_pong_received:107
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:218
測試節點讀寫資訊：
192.168.0.239:7001> set foo bar
-> Redirected to slot [12182] located at 192.168.0.239:7002
OK
192.168.0.239:7002> get foo
"bar"
設定公網訪問
現在通過公網訪問會出現unknown cluster ip:7000的報錯資訊，這是因為在nodes.conf檔案中某些節點的IP是內網IP。解決方法為如下：
- 停止每個節點的程式。
- 編輯每個節點的nodes.conf檔案，將其中的內網IP換成公網IP。
- 重新啟動每個節點。
注意：設定公網訪問Redis叢集屬於危險行為，一般生產環境進位制使用公網redis叢集
配置Redis-cluster-proxy
以下資訊來自於官方的說明：
redis-cluster-proxy是Redis叢集的代理。Redis能夠在基於自動故障轉移和分片的叢集模式下執行。
這種特殊模式（指Redis叢集模式）需要使用特殊的客戶端來理解叢集協議:通過代理，叢集被抽象了出來，可以實現像單例項一樣實現redis叢集的訪問。
Redis叢集代理是多執行緒的，預設情況下，它目前使用多路複用通訊模型，這樣每個執行緒都有自己的叢集連線，所有屬於執行緒本身的客戶端都可以共享該連線。
無論如何，在某些特殊情況下(多事務或阻塞命令)，多路複用被禁用，客戶端將擁有自己的叢集連線。
通過這種方式，只傳送簡單命令(比如get和set)的客戶端將不需要一組到Redis叢集的私有連線。
注：依賴 gcc 4.9 以上版本，所以需要升級gcc版本，前面已經寫過了。
下載proxy工具
https://github.com/RedisLabs/redis-cluster-proxy/releases
wget https://github.com/RedisLabs/redis-cluster-proxy/releases/redis-cluster-proxy-1.0-beta2.tar.gz
tar -zxvf redis-cluster-proxy-1.0-beta2.tar.gz
mv redis-cluster-proxy-1.0-beta2 /data/soft/redis-cluster-proxy
cd /data/soft/redis-cluster-proxy/ && make install
修改配置檔案
修改/data/soft/redis-cluster-proxy目錄下proxy.conf檔案，該配置檔案和redis配置檔案相似，具體引數還未整理，後續更新文件
重點修改以下配置：
-
配置監聽的叢集節點資訊
-
可修改代理埠，預設7777
-
可修改執行緒數，因6.0後是多執行緒
-
如果後端redis叢集配置了密碼，需要在auth引數增加後端redis密碼
配置檔案如下：
# Redis Cluster Proxy configuration file example.
#
# Note that in order to read the configuration file, the proxy must be
# started with the file path passed to the -c option:
#
# ./redis-cluster-proxy -c /path/to/proxy.conf
################################## INCLUDES ###################################
# Include one or more other config files here. Include files can include
# other files.
#
# If instead you are interested in using includes to override configuration
# options, it is better to use include as the last line.
#
# include /path/to/local.conf
# include /path/to/other.conf
######################## CLUSTER ENTRY POINT ADDRESS ##########################
# Indicate the entry point address in the same way it can be indicated in the
# redis-cluster-proxy command line arguments.
# Note that it can be overridden by the command line argument itself.
# You can also specify multiple entry-points, by adding more lines, ie:
# cluster 127.0.0.1:7000
# cluster 127.0.0.1:7001
# You can also use the "entry-point" alias instead of cluster, ie:
# entry-point 127.0.0.1:7000
#
cluster 192.168.0.239:7001
cluster 192.168.0.239:7002
cluster 192.168.0.239:7003
cluster 192.168.0.239:7004
cluster 192.168.0.239:7005
cluster 192.168.0.239:7006
################################### MAIN ######################################
# Set the port used by Redis Cluster Proxy to listen to incoming connections
# from clients (default 7777)
port 7000
# If you want you can bind a single interface, if the bind option is not
# specified all the interfaces will listen for incoming connections.
# You can also bind on multiple interfaces by declaring bind on multiple lines
#
# bind 127.0.0.1
# Specify the path for the Unix socket that will be used to listen for
# incoming connections. There is no default, so Redis Cluster Proxy won't
# listen on a Unix socket when not specified.
#
# unixsocket /path/to/proxy.socket
# Set the Unix socket file permissions (default 0)
#
# unixsocketperm 760
# Set the number of threads.
threads 8
# Set the TCP keep-alive value on the Redis Cluster Proxy's socket
#
# tcpkeepalive 300
# Set the TCP backlog on the Redis Cluster Proxy's socket
#
# tcp-backlog 511
# Size of the connections pool used to provide ready-to-use sockets to
# private connections. The number (size) indicates the number of starting
# connections in the pool.
# Use 0 to disable connections pool at all.
# Every thread will have its pool of ready-to-use connections.
# When the proxy starts, every thread will populate a pool containing
# connections to all the nodes of the cluster.
# Whenever a client needs a private connection, it can take a connection
# from the pool, if available. This will speed-up the client transition from
# the thread's shared connection to its own private connection, since the
# connection from the thread's pool should be already connected and
# ready-to-use. Otherwise, clients with priovate connections must re-connect
# the the nodes of the cluster (this re-connection will act in a 'lazy' way).
#
# connections-pool-size 10
# Minimum number of connections in the the pool. Below this value, the
# thread will start re-spawning connections at the defined rate until
# the pool will be full again.
#
# connections-pool-min-size 10
# Interval in milliseconds used to re-spawn connections in the pool.
# Whenever the number of connections in the pool drops below the minimum
# (see 'connections-pool-min-size' above), the thread will start
# re-spawing connections in the pool, until the pool will be full again.
# New connections will be added at this specified interval.
#
# connections-pool-spawn-every 50
# Number of connections to re-spawn in the pool at every cycle that will
# happen with an interval defined by 'connections-pool-spawn-every' (see above).
#
# connections-pool-spawn-rate 50
# Run Redis Cluster Proxy as a daemon.
daemonize no
# If a pid file is specified, the proxy writes it where specified at startup
# and removes it at exit.
#
# When the proxy runs non daemonized, no pid file is created if none is
# specified in the configuration. When the proxy is daemonized, the pid file
# is used even if not specified, defaulting to
# "/var/run/redis-cluster-proxy.pid".
#
# Creating a pid file is best effort: if the proxy is not able to create it
# nothing bad happens, the server will start and run normally.
#
pidfile /log/redis/redis-cluster-proxy.pid
# Specify the log file name. Also the empty string can be used to force
# Redis Cluster Porxy to log on the standard output. Note that if you use
# standard output for logging but daemonize, logs will be sent to /dev/null
#
logfile "/log/redis/redis-cluster-proxy.log"
# Enable cross-slot queries that can use multiple keys belonging to different
# slots or even different nodes.
# WARN: these queries will break the the atomicity deisgn of many Redis
# commands.
# NOTE: cross-slots queries are not supported by all the commands, even if
# this feature is enabled
#
# enable-cross-slot no
# Maximum number of clients allowed
#
# max-clients 10000
# Authentication password used to authenticate on the cluster in case its nodes
# are password-protected. The password will be used both for fetching cluster's
# configuration and to automatically authenticate proxy's internal connections
# to the cluster itself (both multiplexing shared connections and clients'
# private connections. So, clients connected to the proxy won't need to issue
# the Redis AUTH command in order to be authenticated.
#
auth nfJNhCDwv0hOlyFdbT9XvlkdZq
# Authentication username (supported by Redis >= 6.0)
#
# auth-user myuser
################################# LOGGING #####################################
# Log level: can be debug, info, success, warning o error.
log-level error
# Dump queries received from clients in the log (log-level debug required)
#
# dump-queries no
# Dump buffer in the log (log-level debug required)
#
# dump-buffer no
# Dump requests' queues (requests to send to cluster, request pending, ...)
# in the log (log-level debug required)
#
# dump-queues no
啟動redis-cluster-proxy
第一次先當前控制檯啟動,可以看下錯誤輸出
/data/bin/redis-cluster-proxy -c /usr/local/redis-cluster-proxy/proxy.conf
第二次用後臺啟動
/data/bin/redis-cluster-proxy --daemonize -c /usr/local/redis-cluster-proxy/proxy.conf
啟動成功後，可以通過ps -ef 檢視是否啟動
ps -ef|grep redis
然後通過redis客戶端連線7000埠（proxy代理埠）
/data/bin/redis-cli -h 127.0.0.1 -p 7000 -a 123456
127.0.0.1:7000> set foo bar1
OK
127.0.0.1:7000> get foo
"bar1"
redis-cluster-proxy是完美的解決方案？
因為剛推出不久，生產環境基本上不會有太多實際的應用，裡面肯定有不少坑，但不妨害對其有更多的期待。
初次嘗試可以感受的到，redis-cluster-proxy是一個非常輕量級，清爽簡單的proxy代理層，它解決了一些redis cluster存在的一些實際問題，對應於程式來說也帶來了一些方便性。
如果沒有原始碼開發能力，相比其他第三方proxy中介軟體，必須要承認官方可靠性和權威性。
那麼，redis-cluster-proxy是一個完美的解決方案麼，留下兩個問題
-
如何解決redis-cluster-proxy單點故障？
-
proxy節點的如何面對網路流量風暴？