之前的文章已對RocketMQ做了詳細介紹,這裡就不再贅述了，下面是本人在測試和生產環境下RocketMQ3.4.6高可用叢集的部署手冊，在此分享下:
1) 基礎環境
ip地址 主機名 角色 192.168.10.200 mq-master01 broker-a 192.168.10.201 mq-master02 broker-b 192.168.10.202 mq-slave01 broker-a-s 192.168.10.239 mq-slave02 broker-b-s 192.168.10.205 mq-console-nameserver nameserver、console 生產環境準備5臺虛擬機器,如上,其中4臺伺服器為broker節點,2主2從;1臺伺服器作為nameserver和console節點. 測試環境的話,可以只用兩臺,分別為broker的一主一從節點,nameserver和console服務一同部署到master節點上. 關閉5臺機器的iptables和selinux（所有節點機器上都要操作） [root@mq-master01 ~]# /etc/init.d/iptables stop [root@mq-master01 ~]# vim /etc/sysconfig/selinux ...... SELINUX=disabled [root@mq-master01 ~]# setenforce 0 [root@mq-master01 ~]# getenforce Permissive 5臺機器都要安裝統一的java環境 將jdk-8u131-linux-x64.rpm下載到/usr/local/src目錄下 下載地址：https://pan.baidu.com/s/1pLaAjPp 提取密碼：x27s [root@mq-master01 ~]# java -version -bash: java: command not found 如果自帶了java環境,就解除安裝掉 yum list installed |grep java yum -y remove java-1.7.0-openjdk* yum -y remove tzdata-java.noarch [root@mq-master01 ~]# cd /usr/local/src/ [root@mq-master01 src]# ll jdk-8u131-linux-x64_.rpm -rw-r--r-- 1 root root 169983496 Nov 19 2017 jdk-8u131-linux-x64_.rpm [root@mq-master01 src]# rpm -ivh jdk-8u131-linux-x64_.rpm --force [root@mq-master01 src]# vim /etc/profile ...... JAVA_HOME=/usr/java/jdk1.8.0_131 JAVA_BIN=/usr/java/jdk1.8.0_131/bin PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin:/bin:/sbin/ CLASSPATH=.:/lib/dt.jar:/lib/tools.jar export JAVA_HOME JAVA_BIN PATH CLASSPATH [root@mq-master01 src]# source /etc/profile [root@mq-master01 src]# java -version java version "1.8.0_131" Java(TM) SE Runtime Environment (build 1.8.0_131-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode) 5臺機器最好前做hosts繫結，否則後續啟動broker的時候，會報錯諸如： java.net.UnknownHostException: mq-master01: mq-master01: Name or service not known [root@mq-master01 ~]# cat /etc/hosts ...... 192.168.10.200 mq-master01 192.168.10.201 mq-master02 192.168.10.202 mq-slave01 192.168.10.239 mq-slave02 192.168.10.205 mq-console-nameserver 5臺機器的系統時間統一下 [root@mq-master01 ~]# ntpdate ntp1.aliyun.com 6 Sep 12:59:51 ntpdate[1203]: adjust time server 120.25.115.20 offset -0.000831 sec 本案例rocketmq3.4.6的相關軟體包下載地址: https://pan.baidu.com/s/1VOYA60vIobtSxo9-9rTI6w 提取密碼: bxhr 軟體包分別是:rocketmq_admin.zip, rocketmq_namesrv.zip, rocketmq.zip 下載後放到/usr/local/src目錄下 服務安裝目錄都在/data下
2) RocektMQ Broker雙主雙從環境部署(mq-master01,mq-master02,mq-slave01,mq-slave02四節點上操作)
1）安裝rocketmq（四臺broker節點機器操作一樣） [root@mq-master01 ~]# cd /usr/local/src/ [root@mq-master01 src]# ll rocketmq.zip -rw-rw-r--. 1 1000 1000 16048060 May 22 10:33 rocketmq.zip [root@mq-master01 src]# unzip rocketmq.zip [root@mq-master01 src]# mv rocketmq /data/ 2) 配置rocektmq broker （四個節點機器都要操作）. [root@mq-master01 src]# cd /data/rocketmq/conf/2m-2s-sync/ [root@mq-master01 2m-2s-sync]# vim broker-a.properties brokerClusterName=Kevin-MQ-Cluster brokerName=broker-a brokerId=0 namesrvAddr=192.168.10.205:20901 autoCreateTopicEnable=false clusterTopicEnable=false brokerTopicEnable=false autoCreateSubscriptionGroup=false sendMessageThreadPoolNums=128 pullMessageThreadPoolNums=128 fetchNamesrvAddrByAddressServer=true filterServerNums=1 serverWorkerThreads=32 deleteWhen=04 fileReservedTime=168 brokerRole=SYNC_MASTER flushDiskType=SYNC_FLUSH #netty監聽埠 listenPort=20911 ==================================================================== mq-master02節點配置 [root@mq-master02 src]# cat /data/rocketmq/conf/2m-2s-sync/broker-b.properties brokerClusterName=Kevin-MQ-Cluster brokerName=broker-b brokerId=0 namesrvAddr=192.168.10.205:20901 autoCreateTopicEnable=false clusterTopicEnable=false brokerTopicEnable=false autoCreateSubscriptionGroup=false sendMessageThreadPoolNums=128 pullMessageThreadPoolNums=128 fetchNamesrvAddrByAddressServer=true filterServerNums=1 serverWorkerThreads=32 deleteWhen=04 fileReservedTime=168 brokerRole=SYNC_MASTER flushDiskType=SYNC_FLUSH #netty監聽埠 listenPort=20911 ==================================================================== mq-slave01節點配置 [root@mq-slave01 src]# cat /data/rocketmq/conf/2m-2s-sync/broker-a-s.properties brokerClusterName=Kevin-MQ-Cluster brokerName=broker-a brokerId=1 namesrvAddr=192.168.10.205:20901 autoCreateTopicEnable=false clusterTopicEnable=false brokerTopicEnable=false autoCreateSubscriptionGroup=false sendMessageThreadPoolNums=128 pullMessageThreadPoolNums=128 fetchNamesrvAddrByAddressServer=true filterServerNums=1 serverWorkerThreads=32 deleteWhen=04 fileReservedTime=168 brokerRole=SLAVE flushDiskType=SYNC_FLUSH #netty監聽埠 listenPort=20911 ==================================================================== mq-slave02節點配置 [root@mq-slave02 src]# cat /data/rocketmq/conf/2m-2s-sync/broker-b-s.properties brokerClusterName=Kevin-MQ-Cluster brokerName=broker-b brokerId=1 namesrvAddr=192.168.10.205:20901 autoCreateTopicEnable=false clusterTopicEnable=false brokerTopicEnable=false autoCreateSubscriptionGroup=false sendMessageThreadPoolNums=128 pullMessageThreadPoolNums=128 fetchNamesrvAddrByAddressServer=true filterServerNums=1 serverWorkerThreads=32 deleteWhen=04 fileReservedTime=168 brokerRole=SLAVE flushDiskType=SYNC_FLUSH #netty監聽埠 listenPort=20911 3) 啟動NameServer （四臺broker節點機器操作一樣） 只需要將/data/rocketmq/startBrokerAM.sh中的JAVA_HOME改成自己服務的java路徑就行了. 本案例的java家目錄路徑是/usr/java/jdk1.8.0_131 [root@mq-master01 ~]# cat /data/rocketmq/startBrokerAM.sh |grep JAVA_HOME export JAVA_HOME=/data/software/jdk1.7.0_80 [root@mq-master01 ~]# sed -i 's#/data/software/jdk1.7.0_80#/usr/java/jdk1.8.0_131#g' /data/rocketmq/startBrokerAM.sh [root@mq-master01 ~]# cat /data/rocketmq/startBrokerAM.sh |grep JAVA_HOME export JAVA_HOME=/usr/java/jdk1.8.0_131 溫馨提示： 啟動順序：最好是先啟動NameServer，再啟動BrokerServer（啟動BrokerServer之前關閉防火牆） 停止順序：最好先停止BrokerServer，再停止NameServer 四臺節點的啟動指令碼分別為: [root@mq-master01 rocketmq]# cat startBrokerAM.sh export JAVA_HOME=/usr/java/jdk1.8.0_131 nohup sh ./alibaba-rocketmq/bin/mqbroker -c ./conf/2m-2s-sync/broker-a.properties > ./nohup.out 2>&1 & [root@mq-master02 rocketmq]# cat startBrokerAM.sh export JAVA_HOME=/usr/java/jdk1.8.0_131 nohup sh ./alibaba-rocketmq/bin/mqbroker -c ./conf/2m-2s-sync/broker-b.properties > ./nohup.out 2>&1 & [root@mq-slave01 rocketmq]# cat startBrokerAM.sh export JAVA_HOME=/usr/java/jdk1.8.0_131 nohup sh ./alibaba-rocketmq/bin/mqbroker -c ./conf/2m-2s-sync/broker-a-s.properties > ./nohup.out 2>&1 & [root@mq-slave02 rocketmq]# cat startBrokerAM.sh export JAVA_HOME=/usr/java/jdk1.8.0_131 nohup sh ./alibaba-rocketmq/bin/mqbroker -c ./conf/2m-2s-sync/broker-b-s.properties > ./nohup.out 2>&1 & 啟動NameServer（四臺broker節點機器操作一樣） [root@mq-master01 ~]# cd /data/rocketmq/ #一定要先切換到該路徑下 [root@mq-master01 rocketmq]# /bin/bash /data/rocketmq/startBrokerAM.sh [root@mq-master01 rocketmq]# ps -ef|grep rocketmq root 16783 1 0 09:13 pts/1 00:00:00 sh ./alibaba-rocketmq/bin/mqbroker -c ./conf/2m-2s-sync/broker-a.properties root 16787 16783 0 09:13 pts/1 00:00:00 sh /data/rocketmq/alibaba-rocketmq/bin/runbroker.sh com.alibaba.rocketmq.broker.BrokerStartup -c ./conf/2m-2s-sync/broker-a.properties root 16790 16787 48 09:13 pts/1 00:00:01 /usr/java/jdk1.8.0_131/bin/java -Drocketmq.home.dir=/data/rocketmq/alibaba-rocketmq -Duser.home=/data/rocketmq -Xms2048M -Xmx2048M -XX:PermSize=512M -XX:MaxPermSize=512M -XX:NewSize=768M -XX:MaxNewSize=768M -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+CMSClassUnloadingEnabled -XX:SurvivorRatio=8 -XX:+DisableExplicitGC -verbose:gc -Xloggc:/data/rocketmq/logs/gclog/rmq_bk_gc.16787.log -XX:+PrintGCDetails -XX:-OmitStackTraceInFastThrow -Djava.ext.dirs=/data/rocketmq/alibaba-rocketmq/bin/../lib -cp .:/data/rocketmq/alibaba-rocketmq/bin/../conf: -Drocketmq.namesrv.domain=172.16.50.144 -Drocketmq.namesrv.domain.port=48080 -Drocketmq.namesrv.domain.subgroup=namesrv/getNSAddr.do com.alibaba.rocketmq.broker.BrokerStartup -c ./conf/2m-2s-sync/broker-a.properties root 16835 16088 0 09:13 pts/1 00:00:00 grep rocketmq [root@mq-master01 rocketmq]# lsof -i:20911 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME java 29934 root 79u IPv6 20211949 0t0 TCP *:20911 (LISTEN) java 29934 root 96u IPv6 20212542 0t0 TCP mq-master01:20911->mq-master02:53222 (ESTABLISHED) [root@mq-master01 rocketmq]# lsof -i:20901 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME java 29934 root 88u IPv6 20212051 0t0 TCP mq-master01:41510->mq-console-nameserver:20901 (ESTABLISHED) [root@mq-master01 rocketmq]# jps 30235 Jps 29934 BrokerStartup 檢視rockermq的版本: [root@mq-master01 rocketmq]# ll /data/rocketmq/alibaba-rocketmq/lib/ total 11836 -rw-rw-r--. 1 root root 16088 Nov 28 2016 annotation-indexer-1.4.jar -rw-rw-r--. 1 root root 3159 Nov 28 2016 bridge-method-annotation-1.12.jar -rw-rw-r--. 1 root root 41123 Nov 28 2016 commons-cli-1.2.jar -rw-rw-r--. 1 root root 259600 Nov 28 2016 commons-codec-1.7.jar -rw-rw-r--. 1 root root 185140 Nov 28 2016 commons-io-2.4.jar -rw-rw-r--. 1 root root 284220 Nov 28 2016 commons-lang-2.6.jar -rw-rw-r--. 1 root root 2838580 Nov 28 2016 derby-10.10.2.0.jar -rw-rw-r--. 1 root root 409203 Nov 28 2016 fastjson-1.2.3.jar -rw-rw-r--. 1 root root 154710 Nov 28 2016 github-api-1.59.jar -rw-rw-r--. 1 root root 33483 Nov 28 2016 jackson-annotations-2.2.3.jar -rw-rw-r--. 1 root root 192699 Nov 28 2016 jackson-core-2.2.3.jar -rw-rw-r--. 1 root root 865838 Nov 28 2016 jackson-databind-2.2.3.jar -rw-rw-r--. 1 root root 544021 Nov 28 2016 javassist-3.7.ga.jar -rw-rw-r--. 1 root root 264600 Nov 28 2016 logback-classic-1.0.13.jar -rw-rw-r--. 1 root root 418870 Nov 28 2016 logback-core-1.0.13.jar -rw-rw-r--. 1 root root 1581066 Nov 28 2016 mockito-all-1.9.5.jar -rw-rw-r--. 1 root root 964879 Nov 28 2016 mysql-connector-java-5.1.31.jar -rw-rw-r--. 1 root root 1887979 Nov 28 2016 netty-all-4.0.25.Final.jar -rw-rw-r--. 1 root root 160098 Nov 28 2016 rocketmq-broker-3.4.6-lo-20161026.115524-467.jar -rw-rw-r--. 1 root root 241439 Nov 28 2016 rocketmq-client-3.4.6-lo-20160927.070341-319.jar -rw-rw-r--. 1 root root 213219 Nov 28 2016 rocketmq-common-3.4.6-lo-20160927.070334-319.jar -rw-rw-r--. 1 root root 38123 Nov 28 2016 rocketmq-filtersrv-3.4.6-lo-20160927.070417-318.jar -rw-rw-r--. 1 root root 33525 Nov 28 2016 rocketmq-namesrv-3.4.6-lo-20160927.070407-319.jar -rw-rw-r--. 1 root root 93612 Nov 28 2016 rocketmq-remoting-3.4.6-lo-20160927.070325-319.jar -rw-rw-r--. 1 root root 4088 Nov 28 2016 rocketmq-srvutil-3.4.6-lo-20160927.070351-319.jar -rw-rw-r--. 1 root root 157893 Nov 28 2016 rocketmq-store-3.4.6-lo-20160927.070348-319.jar -rw-rw-r--. 1 root root 147117 Nov 28 2016 rocketmq-tools-3.4.6-lo-20160927.070403-319.jar -rw-rw-r--. 1 root root 26084 Nov 28 2016 slf4j-api-1.7.5.jar
3) NameServer和Console部署(mq-console-nameserver節點上操作)
安裝配置NameServer
[root@mq-console-nameserver src]# cd
[root@mq-console-nameserver ~]# cd /usr/local/src/
[root@mq-console-nameserver src]# ll rocketmq_namesrv.zip
-rw-rw-r-- 1 1000 1000 11802959 May 22 11:03 rocketmq_namesrv.zip
[root@mq-console-nameserver src]# unzip rocketmq_namesrv.zip
[root@mq-console-nameserver src]# mv rocketmq_namesrv /data/
[root@mq-console-nameserver src]# vim /data/rocketmq_namesrv/conf/namesrv.properties
listenPort=20901
修改JAVA_HOME
[root@mq-console-nameserver src]# sed -i 's#/data/software/jdk1.7.0_80#/usr/java/jdk1.8.0_131#g' /data/rocketmq_namesrv/startNameServer.sh
[root@mq-console-nameserver src]# cat /data/rocketmq_namesrv/startNameServer.sh
export JAVA_HOME=/usr/java/jdk1.8.0_131
nohup sh ./alibaba-rocketmq/bin/mqnamesrv -c ./conf/namesrv.properties > ./nohup.out 2>&1 &
啟動NameServer
[root@mq-console-nameserver src]# cd /data/rocketmq_namesrv/
[root@mq-console-nameserver rocketmq_namesrv]# sh startNameServer.sh
[root@mq-console-nameserver rocketmq_namesrv]# ps -ef|grep mqnamesrv
root 17455 1 0 09:23 pts/1 00:00:00 sh ./alibaba-rocketmq/bin/mqnamesrv -c ./conf/namesrv.properties
root 17555 24306 0 09:23 pts/1 00:00:00 grep mqnamesrv
[root@mq-console-nameserver rocketmq_namesrv]# lsof -i:20901
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
java 17461 root 47u IPv6 40870224 0t0 TCP *:20901 (LISTEN)
==================================================
安裝部署Console監控平臺
[root@mq-console-nameserver ~]# cd /usr/local/src/
[root@mq-console-nameserver src]# ll rocketmq_admin.zip
-rw-rw-r-- 1 1000 1000 41574127 May 22 11:03 rocketmq_admin.zip
[root@mq-console-nameserver src]# unzip rocketmq_admin.zip
服務埠(如下改成48080)
[root@mq-console-nameserver src]# vim /data/rocketmq_admin/apache-tomcat-8.0.32-rocketmq-console/conf/server.xml
......
<Connector port="48080" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="28443" />
[root@mq-console-nameserver src]# mv rocketmq_admin /data/
[root@mq-console-nameserver src]# cd /data/rocketmq_admin/
[root@mq-console-nameserver rocketmq_admin]# cd apache-tomcat-8.0.32-rocketmq-console/webapps/rocketmq/WEB-INF/classes/
[root@mq-console-nameserver classes]# vim config.properties
rocketmq.namesrv.addr=192.168.10.205:20901
throwDone=true
啟動console服務
[root@mq-console-nameserver classes]# /data/rocketmq_admin/apache-tomcat-8.0.32-rocketmq-console/bin/startup.sh
[root@mq-console-nameserver classes]# lsof -i:48080
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
java 19338 root 46u IPv6 40883769 0t0 TCP *:48080 (LISTEN)
訪問RockerMQ Console監控平臺http://192.168.10.205:48080/rocketmq/cluster/list.do,如下截圖:
