## 1、準備工作 ##
之前已經介紹過了 MongoDB 在[滴滴雲](https://blog.didiyun.com)上的搭建。參考：[<>](http://blog.itpub.net/31559758/viewspace-2284946/)

但是生產環境中，通常需要更高的穩定性要求，單機版的 MongoDB 伺服器並不能滿足，因此我們需要高可用的 MongoDB 叢集。

### 硬體準備 ###
本次只是模擬叢集的搭建。因此直接採用上一次的搭建環境，透過不同的埠模擬叢集。

## 2、簡介 ##
MongoDB 的叢集搭建方式主要有三種方式：Master-Slave，Replica Sets，Sharding。

其中 Master-Slave 官方已不再支援，故本文不再介紹。 本文主要介紹 Replica Sets 方式搭建高可用叢集。

### 2.1 Replica Sets ###

#### 2.1.1 介紹 ####

副本集是一組維護相同資料集的 Mongod 例項。副本集包含多個資料承載節點和一個仲裁節點（可選），在資料承載節點中，一個且只有一個成員被視為主節點，而其他節點則被視為輔助節點。

主節點接收所有寫入操作，一個副本集只能有一個主例項能夠寫入，主節點記錄所有變更到它的 oplog。

![](https://blog.didiyun.com/wp-content/uploads/2019/02/1-1.png)
 
輔助節點複製主節點的 oplog 並將操作應用於資料集。

![](https://blog.didiyun.com/wp-content/uploads/2019/02/2-1.png)

也可以將一個額外的例項作為仲裁者新增到副本集。仲裁員不維護資料集，仲裁器的目的是透過響應其他副本整合員的心跳和選擇請求來維護副本集中的仲裁。因為它們不儲存資料集，所以仲裁器是提供副本集仲裁功能的一種好方法。

與具有資料集的完全功能副本整合員相比，仲裁器的資源成本更低，如果副本集的成員數為偶數，則新增一個仲裁器以在初選中獲得多數票。

![](https://blog.didiyun.com/wp-content/uploads/2019/02/3-1.png)

當一個主伺服器在超過配置的週期（預設為 10 秒）內未與該組的其他成員通訊時，符合條件的輔助伺服器將要求選擇將其自身指定為新的主伺服器。叢集試圖完成新的初選並恢復正常操作。

![](https://blog.didiyun.com/wp-content/uploads/2019/02/4-1.png)

#### 2.1.2 搭建過程 ####

登入之前搭建單機服務的機器，並切換到 root 賬戶：

```
ssh dc2-user@ip
sudo -i
```

建立配置檔案：

```
vi /etc/mongod_27017.conf
```

檔案內容如下：

```
# mongod_27017.conf

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  # log的路徑與上邊建立的路徑一一致
  path: /data/log/mongodb/mongod_27017.log

# Where and how to store data.
storage:
	# db儲存的路徑與上邊建立的路徑一一致
  dbPath: /data/db/mongo_27017
  journal:
    enabled: true
#  engine:
#  mmapv1:
#  wiredTiger:

# how the process runs
processManagement:
  fork: true  # fork and run in background
  pidFilePath: /var/run/mongodb/mongod_27017.pid  # location of pidfile
  timeZoneInfo: /usr/share/zoneinfo

# network interfaces
net:
  port: 27017
  bindIp: 127.0.0.1  # Enter 0.0.0.0,:: to bind to all IPv4 and IPv6 addresses or, alternatively, use the net.bindIpAll setting.

#security:

#operationProfiling:

replication:
   oplogSizeMB: 1024
   replSetName: test_mongo_replica_set
   secondaryIndexPrefetch: all
   enableMajorityReadConcern: true
#sharding:

## Enterprise-Only Options

#auditLog:

#snmp:

```

```
vi /etc/mongod_27018.conf
```

檔案內容如下：

```
# mongod_27018.conf

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  # log的路徑與上邊建立的路徑一一致
  path: /data/log/mongodb/mongod_27018.log

# Where and how to store data.
storage:
	# db儲存的路徑與上邊建立的路徑一一致
  dbPath: /data/db/mongo_27018
  journal:
    enabled: true
#  engine:
#  mmapv1:
#  wiredTiger:

# how the process runs
processManagement:
  fork: true  # fork and run in background
  pidFilePath: /var/run/mongodb/mongod_27018.pid  # location of pidfile
  timeZoneInfo: /usr/share/zoneinfo

# network interfaces
net:
  port: 27018
  bindIp: 127.0.0.1  # Enter 0.0.0.0,:: to bind to all IPv4 and IPv6 addresses or, alternatively, use the net.bindIpAll setting.

#security:

#operationProfiling:

replication:
   oplogSizeMB: 1024
   replSetName: test_mongo_replica_set
   secondaryIndexPrefetch: all
   enableMajorityReadConcern: true
#sharding:

## Enterprise-Only Options

#auditLog:

#snmp:
```

```
vi /etc/mongod_27019.conf
```

檔案內容如下：

```
# mongod_27019.conf

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  # log的路徑與上邊建立的路徑一一致
  path: /data/log/mongodb/mongod_27019.log

# Where and how to store data.
storage:
	# db儲存的路徑與上邊建立的路徑一一致
  dbPath: /data/db/mongo_27019
  journal:
    enabled: true
#  engine:
#  mmapv1:
#  wiredTiger:

# how the process runs
processManagement:
  fork: true  # fork and run in background
  pidFilePath: /var/run/mongodb/mongod_27019.pid  # location of pidfile
  timeZoneInfo: /usr/share/zoneinfo

# network interfaces
net:
  port: 27019
  bindIp: 127.0.0.1  # Enter 0.0.0.0,:: to bind to all IPv4 and IPv6 addresses or, alternatively, use the net.bindIpAll setting.

#security:

#operationProfiling:

replication:
   oplogSizeMB: 1024
   replSetName: test_mongo_replica_set
   secondaryIndexPrefetch: all
   enableMajorityReadConcern: true
#sharding:

## Enterprise-Only Options

#auditLog:

#snmp:

```

根據配置檔案建立對應的目錄：

```
mkdir /data/db/mongo_27017 && mkdir /data/db/mongo_27018 && mkdir /data/db/mongo_27019
```

啟動對應的服務：

```
mongod -f /etc/mongod_27017.conf
mongod -f /etc/mongod_27018.conf
mongod -f /etc/mongod_27019.conf

```
見到如下輸出即為啟動成功

```
2019-01-28T22:23:39.965+0800 I CONTROL  [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
about to fork child process, waiting until server is ready for connections.
forked process: 11004
child process started successfully, parent exiting
```

使用 MongoDB shell 連線服務：

```
mongo
```

檢視副本集的配置情況：

```
rs.conf()
```

此時我們還未配置副本集應顯示如下：

```
2019-01-28T22:25:51.406+0800 E QUERY    [js] Error: Could not retrieve replica set config: {
	"operationTime" : Timestamp(0, 0),
	"ok" : 0,
	"errmsg" : "no replset config has been received",
	"code" : 94,
	"codeName" : "NotYetInitialized",
	"$clusterTime" : {
		"clusterTime" : Timestamp(0, 0),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
} :
rs.conf@src/mongo/shell/utils.js:1448:11
@(shell):1:1
```

使用命令rs.initiate()來啟動一個新的副本集：

```
rs.initiate()
```

顯示如下：

```
{
	"info2" : "no configuration specified. Using a default configuration for the set",
	"me" : "127.0.0.1:27017",
	"ok" : 1,
	"operationTime" : Timestamp(1548685575, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1548685575, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
```

使用命令rs.add("HOST_NAME:PORT")新增副本整合員：

```
rs.add("127.0.0.1:27018")

```

顯示如下：

```
{
	"ok" : 1,
	"operationTime" : Timestamp(1548685681, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1548685681, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
```

```
rs.add("127.0.0.1:27019")

```

顯示如下：

```
{
	"ok" : 1,
	"operationTime" : Timestamp(1548685690, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1548685690, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
```

再次執行：

```
rs.conf()
```

此時我們還未配置副本集應顯示如下：

```
{
	"_id" : "test_mongo_replica_set",
	"version" : 3,
	"protocolVersion" : NumberLong(1),
	"writeConcernMajorityJournalDefault" : true,
	"members" : [
		{
			"_id" : 0,
			"host" : "127.0.0.1:27017",
			"arbiterOnly" : false,
			"buildIndexes" : true,
			"hidden" : false,
			"priority" : 1,
			"tags" : {

},
			"slaveDelay" : NumberLong(0),
			"votes" : 1
		},
		{
			"_id" : 1,
			"host" : "127.0.0.1:27018",
			"arbiterOnly" : false,
			"buildIndexes" : true,
			"hidden" : false,
			"priority" : 1,
			"tags" : {

},
			"slaveDelay" : NumberLong(0),
			"votes" : 1
		},
		{
			"_id" : 2,
			"host" : "127.0.0.1:27019",
			"arbiterOnly" : false,
			"buildIndexes" : true,
			"hidden" : false,
			"priority" : 1,
			"tags" : {

},
			"slaveDelay" : NumberLong(0),
			"votes" : 1
		}
	],
	"settings" : {
		"chainingAllowed" : true,
		"heartbeatIntervalMillis" : 2000,
		"heartbeatTimeoutSecs" : 10,
		"electionTimeoutMillis" : 10000,
		"catchUpTimeoutMillis" : -1,
		"catchUpTakeoverDelayMillis" : 30000,
		"getLastErrorModes" : {

},
		"getLastErrorDefaults" : {
			"w" : 1,
			"wtimeout" : 0
		},
		"replicaSetId" : ObjectId("5c4f11068679c0236588f2bd")
	}
}
```

執行 db.isMaster() 檢視是否為主節點：

```
db.isMaster() 
```

發現本身 27017 埠已經設定為了主節點：

```
{
	"hosts" : [
		"127.0.0.1:27017",
		"127.0.0.1:27018",
		"127.0.0.1:27019"
	],
	"setName" : "test_mongo_replica_set",
	"setVersion" : 3,
	"ismaster" : true,
	"secondary" : false,
	"primary" : "127.0.0.1:27017",
	"me" : "127.0.0.1:27017",
	"electionId" : ObjectId("7fffffff0000000000000001"),
	"lastWrite" : {
		"opTime" : {
			"ts" : Timestamp(1548685717, 1),
			"t" : NumberLong(1)
		},
		"lastWriteDate" : ISODate("2019-01-28T14:28:37Z"),
		"majorityOpTime" : {
			"ts" : Timestamp(1548685717, 1),
			"t" : NumberLong(1)
		},
		"majorityWriteDate" : ISODate("2019-01-28T14:28:37Z")
	},
	"maxBsonObjectSize" : 16777216,
	"maxMessageSizeBytes" : 48000000,
	"maxWriteBatchSize" : 100000,
	"localTime" : ISODate("2019-01-28T14:28:45.684Z"),
	"logicalSessionTimeoutMinutes" : 30,
	"minWireVersion" : 0,
	"maxWireVersion" : 7,
	"readOnly" : false,
	"ok" : 1,
	"operationTime" : Timestamp(1548685717, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1548685717, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
```

#### 2.1.3 驗證效果 ####

連線主例項：

```
mongo  --port 27017
```

檢視主狀態：

```
 db.isMaster()
```

顯示如下：
```
 {
	"hosts" : [
		"127.0.0.1:27017",
		"127.0.0.1:27018",
		"127.0.0.1:27019"
	],
	"setName" : "test_mongo_replica_set",
	"setVersion" : 3,
	"ismaster" : true,
	"secondary" : false,
	"primary" : "127.0.0.1:27017",
	"me" : "127.0.0.1:27017",
	"electionId" : ObjectId("7fffffff0000000000000001"),
	"lastWrite" : {
		"opTime" : {
			"ts" : Timestamp(1548685717, 1),
			"t" : NumberLong(1)
		},
		"lastWriteDate" : ISODate("2019-01-28T14:28:37Z"),
		"majorityOpTime" : {
			"ts" : Timestamp(1548685717, 1),
			"t" : NumberLong(1)
		},
		"majorityWriteDate" : ISODate("2019-01-28T14:28:37Z")
	},
	"maxBsonObjectSize" : 16777216,
	"maxMessageSizeBytes" : 48000000,
	"maxWriteBatchSize" : 100000,
	"localTime" : ISODate("2019-01-28T14:28:45.684Z"),
	"logicalSessionTimeoutMinutes" : 30,
	"minWireVersion" : 0,
	"maxWireVersion" : 7,
	"readOnly" : false,
	"ok" : 1,
	"operationTime" : Timestamp(1548685717, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1548685717, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
```

寫入資料：

```
use mongo_test
db.mongo_test.insert({"name":"滴滴雲MongoDBTest1","id":1})
```

讀取資料：

```
db.mongo_test.find().pretty()
```

顯示如下：

```
{
	"_id" : ObjectId("5c4f156792a30dbd4d11afdc"),
	"name" : "滴滴雲MongoDBTest1",
	"id" : 1
}

```

再開一個視窗連線副本例項：

```
mongo  --port 27018
```

檢視主狀態：

```
db.isMaster()
```

顯示如下：

```
{
	"hosts" : [
		"127.0.0.1:27017",
		"127.0.0.1:27018",
		"127.0.0.1:27019"
	],
	"setName" : "test_mongo_replica_set",
	"setVersion" : 3,
	"ismaster" : false,
	"secondary" : true,
	"primary" : "127.0.0.1:27017",
	"me" : "127.0.0.1:27018",
	"lastWrite" : {
		"opTime" : {
			"ts" : Timestamp(1548686587, 1),
			"t" : NumberLong(1)
		},
		"lastWriteDate" : ISODate("2019-01-28T14:43:07Z"),
		"majorityOpTime" : {
			"ts" : Timestamp(1548686587, 1),
			"t" : NumberLong(1)
		},
		"majorityWriteDate" : ISODate("2019-01-28T14:43:07Z")
	},
	"maxBsonObjectSize" : 16777216,
	"maxMessageSizeBytes" : 48000000,
	"maxWriteBatchSize" : 100000,
	"localTime" : ISODate("2019-01-28T14:43:12.576Z"),
	"logicalSessionTimeoutMinutes" : 30,
	"minWireVersion" : 0,
	"maxWireVersion" : 7,
	"readOnly" : false,
	"ok" : 1,
	"operationTime" : Timestamp(1548686587, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1548686587, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
```

模擬主例項故障 ( kill 程式) ：

```
ps aux|grep mongo

kill 對應的程式id

```

檢視副本例項的狀態（此處會有幾秒的延遲，多執行幾次檢視效果)：

```
db.isMaster()
```

如發現如下顯示，再執行幾次，直到 ismaster 變為 true：

```
{
	"hosts" : [
		"127.0.0.1:27017",
		"127.0.0.1:27018",
		"127.0.0.1:27019"
	],
	"setName" : "test_mongo_replica_set",
	"setVersion" : 3,
	"ismaster" : false,
	"secondary" : true,
	"primary" : "127.0.0.1:27018",
	"me" : "127.0.0.1:27018",
	"electionId" : ObjectId("7fffffff0000000000000002"),
	"lastWrite" : {
		"opTime" : {
			"ts" : Timestamp(1548686967, 1),
			"t" : NumberLong(1)
		},
		"lastWriteDate" : ISODate("2019-01-28T14:49:27Z"),
		"majorityOpTime" : {
			"ts" : Timestamp(1548686967, 1),
			"t" : NumberLong(1)
		},
		"majorityWriteDate" : ISODate("2019-01-28T14:49:27Z")
	},
	"maxBsonObjectSize" : 16777216,
	"maxMessageSizeBytes" : 48000000,
	"maxWriteBatchSize" : 100000,
	"localTime" : ISODate("2019-01-28T14:49:40.069Z"),
	"logicalSessionTimeoutMinutes" : 30,
	"minWireVersion" : 0,
	"maxWireVersion" : 7,
	"readOnly" : false,
	"ok" : 1,
	"operationTime" : Timestamp(1548686967, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1548686979, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
```

即：

```
{
	"hosts" : [
		"127.0.0.1:27017",
		"127.0.0.1:27018",
		"127.0.0.1:27019"
	],
	"setName" : "test_mongo_replica_set",
	"setVersion" : 3,
	"ismaster" : true,
	"secondary" : false,
	"primary" : "127.0.0.1:27018",
	"me" : "127.0.0.1:27018",
	"electionId" : ObjectId("7fffffff0000000000000002"),
	"lastWrite" : {
		"opTime" : {
			"ts" : Timestamp(1548686981, 1),
			"t" : NumberLong(2)
		},
		"lastWriteDate" : ISODate("2019-01-28T14:49:41Z"),
		"majorityOpTime" : {
			"ts" : Timestamp(1548686981, 1),
			"t" : NumberLong(2)
		},
		"majorityWriteDate" : ISODate("2019-01-28T14:49:41Z")
	},
	"maxBsonObjectSize" : 16777216,
	"maxMessageSizeBytes" : 48000000,
	"maxWriteBatchSize" : 100000,
	"localTime" : ISODate("2019-01-28T14:49:51.062Z"),
	"logicalSessionTimeoutMinutes" : 30,
	"minWireVersion" : 0,
	"maxWireVersion" : 7,
	"readOnly" : false,
	"ok" : 1,
	"operationTime" : Timestamp(1548686981, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1548686981, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
```

讀取剛才寫入的資料：

```
use mongo_test
db.mongo_test.find().pretty()
```

顯示如下：

```
{
	"_id" : ObjectId("5c4f156792a30dbd4d11afdc"),
	"name" : "滴滴雲MongoDBTest1",
	"id" : 1
}
```

驗證成功。

在滴滴雲上搭建 MongoDB 叢集（一）：

相關文章

在滴滴雲上搭建 MongoDB 叢集 （一）：

相關文章

在滴滴雲上搭建 MongoDB 叢集（一）：