Mongodb是一種非關聯式資料庫(NoSQL),非關係型資料庫的產生就是為了解決大資料量、高擴充套件性、高效能、靈活資料模型、高可用性。MongoDB官方已經不建議使用主從模式了,替代方案是採用副本集的模式。主從模式其實就是一個單副本的應用,沒有很好的擴充套件性和容錯性,而Mongodb副本集具有多個副本保證了容錯性,就算一個副本掛掉了還有很多副本存在,主節點掛掉後,整個叢集內會實現自動切換。
Mongodb副本集的工作原理
客戶端連線到整個Mongodb副本集,不關心具體哪一臺節點機是否掛掉。主節點機負責整個副本集的讀寫,副本集定期同步資料備份,一但主節點掛掉,副本節點就會選舉一個新的主伺服器,這一切對於應用伺服器不需要關心。副本集中的副本節點在主節點掛掉後通過心跳機制檢測到後,就會在叢集內發起主節點的選舉機制,自動選舉一位新的主伺服器。
看起來Mongodb副本集很牛X的樣子,下面就演示下副本集環境部署過程,官方推薦的Mongodb副本集機器數量為至少3個節點,這裡我就選擇三個節點,一個主節點,兩個從節點,暫不使用仲裁節點。
一、環境準備
ip地址 主機名 角色 172.16.60.205 mongodb-master01 副本集主節點 172.16.60.206 mongodb-slave01 副本集副本節點 172.16.60.207 mongodb-slave02 副本集副本節點 三個節點機均設定好各自的主機名,並如下設定好hosts繫結 [root@mongodb-master01 ~]# cat /etc/hosts ............ 172.16.60.205 mongodb-master01 172.16.60.206 mongodb-slave01 172.16.60.207 mongodb-slave02 三個節點機均關閉selinux,為了測試方便,將iptables也關閉 [root@mongodb-master01 ~]# setenforce 0 [root@mongodb-master01 ~]# cat /etc/sysconfig/selinux ........... SELINUX=disabled [root@mongodb-master01 ~]# iptables -F [root@mongodb-master01 ~]# /etc/init.d/iptables stop [root@mongodb-master01 ~]# /etc/init.d/iptables stop iptables: Setting chains to policy ACCEPT: filter [ OK ] iptables: Flushing firewall rules: [ OK ] iptables: Unloading modules: [ OK ]
二、Mongodb安裝、副本集配置
1) 在三個節點機上建立mongodb副本集測試資料夾,用於存放整個副本集檔案 [root@mongodb-master01 ~]# mkdir -p /data/mongodb/data/replset/ 2)在三個節點機上安裝mongodb 下載地址:https://www.mongodb.org/dl/linux/x86_64-rhel62 [root@mongodb-master01 ~]# wget http://downloads.mongodb.org/linux/mongodb-linux-x86_64-rhel62-v3.6-latest.tgz [root@mongodb-master01 ~]# tar -zvxf mongodb-linux-x86_64-rhel62-v3.6-latest.tgz 3)分別在每個節點機上啟動mongodb(啟動時指明--bind_ip地址,預設是127.0.0.1,需要改成本機ip,否則遠端連線時失敗) [root@mongodb-master01 ~]# mv mongodb-linux-x86_64-rhel62-3.6.11-rc0-2-g2151d1d219 /usr/local/mongodb [root@mongodb-master01 ~]# nohup /usr/local/mongodb/bin/mongod -dbpath /data/mongodb/data/replset -replSet repset --bind_ip=172.16.60.205 --port=27017 & [root@mongodb-master01 ~]# ps -ef|grep mongodb root 7729 6977 1 15:10 pts/1 00:00:01 /usr/local/mongodb/bin/mongod -dbpath /data/mongodb/data/replset -replSet repset root 7780 6977 0 15:11 pts/1 00:00:00 grep mongodb [root@mongodb-master01 ~]# lsof -i:27017 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME mongod 7729 root 10u IPv4 6554476 0t0 TCP localhost:27017 (LISTEN) 4)初始化副本集 在三個節點中的任意一個節點機上操作(比如在172.16.60.205節點機) 登陸mongodb [root@mongodb-master01 ~]# /usr/local/mongodb/bin/mongo 172.16.60.205:27017 ......... #使用admin資料庫 > use admin switched to db admin #定義副本集配置變數,這裡的 _id:”repset” 和上面命令引數“ –replSet repset” 要保持一樣。 > config = { _id:"repset", members:[{_id:0,host:"172.16.60.205:27017"},{_id:1,host:"172.16.60.206:27017"},{_id:2,host:"172.16.60.207:27017"}]} { "_id" : "repset", "members" : [ { "_id" : 0, "host" : "172.16.60.205:27017" }, { "_id" : 1, "host" : "172.16.60.206:27017" }, { "_id" : 2, "host" : "172.16.60.207:27017" } ] } #初始化副本集配置 > rs.initiate(config); { "ok" : 1, "operationTime" : Timestamp(1551166191, 1), "$clusterTime" : { "clusterTime" : Timestamp(1551166191, 1), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } } } #檢視叢集節點的狀態 repset:SECONDARY> rs.status(); { "set" : "repset", "date" : ISODate("2019-02-26T07:31:07.766Z"), "myState" : 1, "term" : NumberLong(1), "syncingTo" : "", "syncSourceHost" : "", "syncSourceId" : -1, "heartbeatIntervalMillis" : NumberLong(2000), "optimes" : { "lastCommittedOpTime" : { "ts" : Timestamp(1551166263, 1), "t" : NumberLong(1) }, "readConcernMajorityOpTime" : { "ts" : Timestamp(1551166263, 1), "t" : NumberLong(1) }, "appliedOpTime" : { "ts" : Timestamp(1551166263, 1), "t" : NumberLong(1) }, "durableOpTime" : { "ts" : Timestamp(1551166263, 1), "t" : NumberLong(1) } }, "members" : [ { "_id" : 0, "name" : "172.16.60.205:27017", "health" : 1, "state" : 1, "stateStr" : "PRIMARY", "uptime" : 270, "optime" : { "ts" : Timestamp(1551166263, 1), "t" : NumberLong(1) }, "optimeDate" : ISODate("2019-02-26T07:31:03Z"), "syncingTo" : "", "syncSourceHost" : "", "syncSourceId" : -1, "infoMessage" : "could not find member to sync from", "electionTime" : Timestamp(1551166202, 1), "electionDate" : ISODate("2019-02-26T07:30:02Z"), "configVersion" : 1, "self" : true, "lastHeartbeatMessage" : "" }, { "_id" : 1, "name" : "172.16.60.206:27017", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 76, "optime" : { "ts" : Timestamp(1551166263, 1), "t" : NumberLong(1) }, "optimeDurable" : { "ts" : Timestamp(1551166263, 1), "t" : NumberLong(1) }, "optimeDate" : ISODate("2019-02-26T07:31:03Z"), "optimeDurableDate" : ISODate("2019-02-26T07:31:03Z"), "lastHeartbeat" : ISODate("2019-02-26T07:31:06.590Z"), "lastHeartbeatRecv" : ISODate("2019-02-26T07:31:06.852Z"), "pingMs" : NumberLong(0), "lastHeartbeatMessage" : "", "syncingTo" : "172.16.60.205:27017", "syncSourceHost" : "172.16.60.205:27017", "syncSourceId" : 0, "infoMessage" : "", "configVersion" : 1 }, { "_id" : 2, "name" : "172.16.60.207:27017", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 76, "optime" : { "ts" : Timestamp(1551166263, 1), "t" : NumberLong(1) }, "optimeDurable" : { "ts" : Timestamp(1551166263, 1), "t" : NumberLong(1) }, "optimeDate" : ISODate("2019-02-26T07:31:03Z"), "optimeDurableDate" : ISODate("2019-02-26T07:31:03Z"), "lastHeartbeat" : ISODate("2019-02-26T07:31:06.589Z"), "lastHeartbeatRecv" : ISODate("2019-02-26T07:31:06.958Z"), "pingMs" : NumberLong(0), "lastHeartbeatMessage" : "", "syncingTo" : "172.16.60.205:27017", "syncSourceHost" : "172.16.60.205:27017", "syncSourceId" : 0, "infoMessage" : "", "configVersion" : 1 } ], "ok" : 1, "operationTime" : Timestamp(1551166263, 1), "$clusterTime" : { "clusterTime" : Timestamp(1551166263, 1), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } } } 如上資訊表明: 副本集配置成功後,172.16.60.205為主節點PRIMARY,172.16.60.206/207為副本節點SECONDARY。 health:1 1表明狀態是正常,0表明異常 state:1 值小的是primary節點、值大的是secondary節點
三、測試Mongodb副本集資料複製功能 <mongodb預設是從主節點讀寫資料的,副本節點上不允許讀,需要設定副本節點可以讀>
1)在主節點172.16.60.205上連線到終端 [root@mongodb-master01 ~]# /usr/local/mongodb/bin/mongo 172.16.60.205:27017 ................ #建立test 資料庫 repset:PRIMARY> use test; switched to db test #往testdb表插入測試資料 repset:PRIMARY> db.testdb.insert({"test1":"testval1"}) WriteResult({ "nInserted" : 1 }) 2)在副本節點172.16.60.206、172.16.60.207上連線到mongodb檢視資料是否複製過來。 這裡在172.16.60.206副本節點上進行檢視 [root@mongodb-slave01 ~]# /usr/local/mongodb/bin/mongo 172.16.60.206:27017 ................ repset:SECONDARY> use test; switched to db test repset:SECONDARY> show tables; 2019-02-26T15:37:46.446+0800 E QUERY [thread1] Error: listCollections failed: { "operationTime" : Timestamp(1551166663, 1), "ok" : 0, "errmsg" : "not master and slaveOk=false", "code" : 13435, "codeName" : "NotMasterNoSlaveOk", "$clusterTime" : { "clusterTime" : Timestamp(1551166663, 1), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } } } : _getErrorWithCode@src/mongo/shell/utils.js:25:13 DB.prototype._getCollectionInfosCommand@src/mongo/shell/db.js:941:1 DB.prototype.getCollectionInfos@src/mongo/shell/db.js:953:19 DB.prototype.getCollectionNames@src/mongo/shell/db.js:964:16 shellHelper.show@src/mongo/shell/utils.js:853:9 shellHelper@src/mongo/shell/utils.js:750:15 @(shellhelp2):1:1 上面出現了報錯! 這是因為mongodb預設是從主節點讀寫資料的,副本節點上不允許讀,需要設定副本節點可以讀 repset:SECONDARY> db.getMongo().setSlaveOk(); repset:SECONDARY> db.testdb.find(); { "_id" : ObjectId("5c74ec9267d8c3d06506449b"), "test1" : "testval1" } repset:SECONDARY> show tables; testdb 如上發現已經在副本節點上發現了測試資料,即已經從主節點複製過來了。 (在另一個副本節點172.16.60.207也如上操作即可)
四、測試副本集故障轉移功能
先停掉主節點172.16.60.205,檢視mongodb副本集狀態,可以看到經過一系列的投票選擇操作,172.16.60.206當選主節點,172.16.60.207從172.16.60.206同步資料過來。
1)停掉原來的主節點172.16.60.205的mongodb,模擬故障 [root@mongodb-master01 ~]# ps -ef|grep mongodb|grep -v grep|awk '{print $2}'|xargs kill -9 [root@mongodb-master01 ~]# lsof -i:27017 [root@mongodb-master01 ~]# 2)接著登入到另外兩個正常的從節點172.16.60.206、172.16.60.207中的任意一個節點的mongodb,檢視副本集狀態 [root@mongodb-slave01 ~]# /usr/local/mongodb/bin/mongo 172.16.60.206:27017 ................. repset:PRIMARY> rs.status(); { "set" : "repset", "date" : ISODate("2019-02-26T08:06:02.996Z"), "myState" : 1, "term" : NumberLong(2), "syncingTo" : "", "syncSourceHost" : "", "syncSourceId" : -1, "heartbeatIntervalMillis" : NumberLong(2000), "optimes" : { "lastCommittedOpTime" : { "ts" : Timestamp(1551168359, 1), "t" : NumberLong(2) }, "readConcernMajorityOpTime" : { "ts" : Timestamp(1551168359, 1), "t" : NumberLong(2) }, "appliedOpTime" : { "ts" : Timestamp(1551168359, 1), "t" : NumberLong(2) }, "durableOpTime" : { "ts" : Timestamp(1551168359, 1), "t" : NumberLong(2) } }, "members" : [ { "_id" : 0, "name" : "172.16.60.205:27017", "health" : 0, "state" : 8, "stateStr" : "(not reachable/healthy)", "uptime" : 0, "optime" : { "ts" : Timestamp(0, 0), "t" : NumberLong(-1) }, "optimeDurable" : { "ts" : Timestamp(0, 0), "t" : NumberLong(-1) }, "optimeDate" : ISODate("1970-01-01T00:00:00Z"), "optimeDurableDate" : ISODate("1970-01-01T00:00:00Z"), "lastHeartbeat" : ISODate("2019-02-26T08:06:02.917Z"), "lastHeartbeatRecv" : ISODate("2019-02-26T08:03:37.492Z"), "pingMs" : NumberLong(0), "lastHeartbeatMessage" : "Connection refused", "syncingTo" : "", "syncSourceHost" : "", "syncSourceId" : -1, "infoMessage" : "", "configVersion" : -1 }, { "_id" : 1, "name" : "172.16.60.206:27017", "health" : 1, "state" : 1, "stateStr" : "PRIMARY", "uptime" : 2246, "optime" : { "ts" : Timestamp(1551168359, 1), "t" : NumberLong(2) }, "optimeDate" : ISODate("2019-02-26T08:05:59Z"), "syncingTo" : "", "syncSourceHost" : "", "syncSourceId" : -1, "infoMessage" : "", "electionTime" : Timestamp(1551168228, 1), "electionDate" : ISODate("2019-02-26T08:03:48Z"), "configVersion" : 1, "self" : true, "lastHeartbeatMessage" : "" }, { "_id" : 2, "name" : "172.16.60.207:27017", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 2169, "optime" : { "ts" : Timestamp(1551168359, 1), "t" : NumberLong(2) }, "optimeDurable" : { "ts" : Timestamp(1551168359, 1), "t" : NumberLong(2) }, "optimeDate" : ISODate("2019-02-26T08:05:59Z"), "optimeDurableDate" : ISODate("2019-02-26T08:05:59Z"), "lastHeartbeat" : ISODate("2019-02-26T08:06:02.861Z"), "lastHeartbeatRecv" : ISODate("2019-02-26T08:06:02.991Z"), "pingMs" : NumberLong(0), "lastHeartbeatMessage" : "", "syncingTo" : "172.16.60.206:27017", "syncSourceHost" : "172.16.60.206:27017", "syncSourceId" : 1, "infoMessage" : "", "configVersion" : 1 } ], "ok" : 1, "operationTime" : Timestamp(1551168359, 1), "$clusterTime" : { "clusterTime" : Timestamp(1551168359, 1), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } } } 發現當原來的主節點172.16.60.205宕掉後,經過選舉,原來的從節點172.16.60.206被推舉為新的主節點。 3)現在在172.16.60.206新主節點上建立測試資料 repset:PRIMARY> use kevin; switched to db kevin repset:PRIMARY> db.kevin.insert({"shibo":"hahaha"}) WriteResult({ "nInserted" : 1 }) 4)另一個從節點172.16.60.207上登入mongodb檢視 [root@mongodb-slave02 ~]# /usr/local/mongodb/bin/mongo 172.16.60.207:27017 ................ repset:SECONDARY> use kevin; switched to db kevin repset:SECONDARY> db.getMongo().setSlaveOk(); repset:SECONDARY> show tables; kevin repset:SECONDARY> db.kevin.find(); { "_id" : ObjectId("5c74f42bb0b339ed6eb68e9c"), "shibo" : "hahaha" } 發現從節點172.16.60.207可以同步新的主節點172.16.60.206的資料 5)再重新啟動原來的主節點172.16.60.205的mongodb [root@mongodb-master01 ~]# nohup /usr/local/mongodb/bin/mongod -dbpath /data/mongodb/data/replset -replSet repset --bind_ip=172.16.60.205 --port=27017 & mongod 9162 root 49u IPv4 6561201 0t0 TCP mongodb-master01:55236->mongodb-slave01:27017 (ESTABLISHED) [root@mongodb-master01 ~]# ps -ef|grep mongodb root 9162 6977 4 16:14 pts/1 00:00:01 /usr/local/mongodb/bin/mongod -dbpath /data/mongodb/data/replset -replSet repset --bind_ip=172.16.60.205 --port=27017 root 9244 6977 0 16:14 pts/1 00:00:00 grep mongodb 再次登入到三個節點中的任意一個的mongodb,檢視副本集狀態 [root@mongodb-master01 ~]# /usr/local/mongodb/bin/mongo 172.16.60.205:27017 .................... repset:SECONDARY> rs.status(); { "set" : "repset", "date" : ISODate("2019-02-26T08:16:11.741Z"), "myState" : 2, "term" : NumberLong(2), "syncingTo" : "172.16.60.206:27017", "syncSourceHost" : "172.16.60.206:27017", "syncSourceId" : 1, "heartbeatIntervalMillis" : NumberLong(2000), "optimes" : { "lastCommittedOpTime" : { "ts" : Timestamp(1551168969, 1), "t" : NumberLong(2) }, "readConcernMajorityOpTime" : { "ts" : Timestamp(1551168969, 1), "t" : NumberLong(2) }, "appliedOpTime" : { "ts" : Timestamp(1551168969, 1), "t" : NumberLong(2) }, "durableOpTime" : { "ts" : Timestamp(1551168969, 1), "t" : NumberLong(2) } }, "members" : [ { "_id" : 0, "name" : "172.16.60.205:27017", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 129, "optime" : { "ts" : Timestamp(1551168969, 1), "t" : NumberLong(2) }, "optimeDate" : ISODate("2019-02-26T08:16:09Z"), "syncingTo" : "172.16.60.206:27017", "syncSourceHost" : "172.16.60.206:27017", "syncSourceId" : 1, "infoMessage" : "", "configVersion" : 1, "self" : true, "lastHeartbeatMessage" : "" }, { "_id" : 1, "name" : "172.16.60.206:27017", "health" : 1, "state" : 1, "stateStr" : "PRIMARY", "uptime" : 127, "optime" : { "ts" : Timestamp(1551168969, 1), "t" : NumberLong(2) }, "optimeDurable" : { "ts" : Timestamp(1551168969, 1), "t" : NumberLong(2) }, "optimeDate" : ISODate("2019-02-26T08:16:09Z"), "optimeDurableDate" : ISODate("2019-02-26T08:16:09Z"), "lastHeartbeat" : ISODate("2019-02-26T08:16:10.990Z"), "lastHeartbeatRecv" : ISODate("2019-02-26T08:16:11.518Z"), "pingMs" : NumberLong(0), "lastHeartbeatMessage" : "", "syncingTo" : "", "syncSourceHost" : "", "syncSourceId" : -1, "infoMessage" : "", "electionTime" : Timestamp(1551168228, 1), "electionDate" : ISODate("2019-02-26T08:03:48Z"), "configVersion" : 1 }, { "_id" : 2, "name" : "172.16.60.207:27017", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 127, "optime" : { "ts" : Timestamp(1551168969, 1), "t" : NumberLong(2) }, "optimeDurable" : { "ts" : Timestamp(1551168969, 1), "t" : NumberLong(2) }, "optimeDate" : ISODate("2019-02-26T08:16:09Z"), "optimeDurableDate" : ISODate("2019-02-26T08:16:09Z"), "lastHeartbeat" : ISODate("2019-02-26T08:16:10.990Z"), "lastHeartbeatRecv" : ISODate("2019-02-26T08:16:11.655Z"), "pingMs" : NumberLong(0), "lastHeartbeatMessage" : "", "syncingTo" : "172.16.60.206:27017", "syncSourceHost" : "172.16.60.206:27017", "syncSourceId" : 1, "infoMessage" : "", "configVersion" : 1 } ], "ok" : 1, "operationTime" : Timestamp(1551168969, 1), "$clusterTime" : { "clusterTime" : Timestamp(1551168969, 1), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } } } 發現原來的主節點172.16.60.205在故障恢復後,變成了新的主節點172.16.60.206的從節點
五、Mongodb讀寫分離
目前來看。Mongodb副本集可以完美支援故障轉移。至於主節點的讀寫壓力過大如何解決?常見的解決方案是讀寫分離。
一般情況下,常規寫操作來說並沒有讀操作多,所以在Mongodb副本集中,一臺主節點負責寫操作,兩臺副本節點負責讀操作。 1)設定讀寫分離需要先在副本節點SECONDARY 設定 setSlaveOk。 2)在程式中設定副本節點負責讀操作,如下程式碼:
public class TestMongoDBReplSetReadSplit { public static void main(String[] args) { try { List<ServerAddress> addresses = new ArrayList<ServerAddress>(); ServerAddress address1 = new ServerAddress("172.16.60.205" , 27017); ServerAddress address2 = new ServerAddress("172.16.60.206" , 27017); ServerAddress address3 = new ServerAddress("172.16.60.207" , 27017); addresses.add(address1); addresses.add(address2); addresses.add(address3); MongoClient client = new MongoClient(addresses); DB db = client.getDB( "test" ); DBCollection coll = db.getCollection( "testdb" ); BasicDBObject object = new BasicDBObject(); object.append( "test2" , "testval2" ); //讀操作從副本節點讀取 ReadPreference preference = ReadPreference. secondary(); DBObject dbObject = coll.findOne(object, null , preference); System. out .println(dbObject); } catch (Exception e) { e.printStackTrace(); } } }
讀引數除了secondary一共還有五個引數:primary、primaryPreferred、secondary、secondaryPreferred、nearest。
primary:預設引數,只從主節點上進行讀取操作;
primaryPreferred:大部分從主節點上讀取資料,只有主節點不可用時從secondary節點讀取資料。
secondary:只從secondary節點上進行讀取操作,存在的問題是secondary節點的資料會比primary節點資料“舊”。
secondaryPreferred:優先從secondary節點進行讀取操作,secondary節點不可用時從主節點讀取資料;
nearest:不管是主節點、secondary節點,從網路延遲最低的節點上讀取資料。
讀寫分離做好後,就可以進行資料分流,減輕壓力,解決了"主節點的讀寫壓力過大如何解決?"這個問題。不過當副本節點增多時,主節點的複製壓力會加大有什麼辦法解決嗎?基於這個問題,Mongodb已有了相應的解決方案 - 引用仲裁節點:
在Mongodb副本集中,仲裁節點不儲存資料,只是負責故障轉移的群體投票,這樣就少了資料複製的壓力。看起來想的很周到啊,其實不只是主節點、副本節點、仲裁節點,還有Secondary-Only、Hidden、Delayed、Non-Voting,其中:
Secondary-Only:不能成為primary節點,只能作為secondary副本節點,防止一些效能不高的節點成為主節點。
Hidden:這類節點是不能夠被客戶端制定IP引用,也不能被設定為主節點,但是可以投票,一般用於備份資料。
Delayed:可以指定一個時間延遲從primary節點同步資料。主要用於備份資料,如果實時同步,誤刪除資料馬上同步到從節點,恢復又恢復不了。
Non-Voting:沒有選舉權的secondary節點,純粹的備份資料節點。