實際環境架構
分別在3臺機器執行一個mongod例項（稱為mongod shard11，mongod shard12，mongod shard13）組織replica set1，作為cluster的shard1
分別在3臺機器執行一個mongod例項（稱為mongod shard21，mongod shard22，mongod shard23）組織replica set2，作為cluster的shard2
每臺機器執行一個mongod例項，作為3個config server
每臺機器執行一個mongs程式，用於客戶端連線
-------------------------------------------------------------------------
Server1 10.3.0.100
Mongod shard11:27017
Mongod shard21:17017
Mongod config1:20000
Mongos1:20001
-------------------------------------------------------------------------
Server1 10.3.0.101
Mongod shard12:27017
Mongod shard22:17017
Mongod config1:20000
Mongos1:20001
--------------------------------------------------------------------------
Server3 10.3.0.102
Mongod shard13:27017
Mongod shard23:17017
Mongod config1:20000
Mongos1:20001
-------------------------------------------------------------------------
1. 建立使用者 mongodb (如果用root啟用mongodb,可略此步)
useradd -u 1001 mongodb
2. 下載 mongodb-linux-x86_64-2.2.0
tar zxvf mongodb-linux-x86_64-2.2.0.tgz
mv mongodb-linux-x86_64-2.2.0 /opt/local/mongodb
3. 建立 資料目錄 日誌目錄 配置目錄 (server 1)
mkdir -p /opt/local/mongodb/data/shard
mkdir -p /opt/local/mongodb/data/logs
mkdir -p /opt/local/mongodb/data/config
4. 建立 資料目錄 日誌目錄 配置目錄 (server 2)
mkdir -p /opt/local/mongodb/data/shard12
mkdir -p /opt/local/mongodb/data/shard22
mkdir -p /opt/local/mongodb/data/logs
mkdir -p /opt/local/mongodb/data/config
5. 建立 資料目錄 日誌目錄 配置目錄 (server 3)
mkdir -p /opt/local/mongodb/data/shard13
mkdir -p /opt/local/mongodb/data/shard23
mkdir -p /opt/local/mongodb/data/logs
mkdir -p /opt/local/mongodb/data/config
6. 修改 目錄 所有者
chown -R mongodb:mongodb /opt/local/mongodb
7. 配置relica sets
7.1.1 配置shard1所用到的replica sets:
--------------------------------------------------------------------------------------
port=27017 #埠號
fork=true #以守護程式的方式執行，建立伺服器程式
logpath=/opt/local/mongodb/data/logs/shard.log #日誌輸出檔案路徑
logappend=true #日誌輸出方式
dbpath=/opt/local/mongodb/data/shard/ #資料庫路徑
shardsvr=true #設定是否分片
maxConns=10000 #資料庫的最大連線數
replSet=shard1 #設定副本集名稱
oplogSize=5000 #設定oplog的大小(MB)
----------------------------------------------------------------------------------------
server 1
/opt/local/mongodb/bin/mongod --shardsvr --maxConns 10000 --replSet shard1 --port 27017 --dbpath /opt/local/mongodb/data/shard11/ --oplogSize 500 --logpath=/opt/local/mongodb/data/logs/shard11.log --logappend --fork
server 2
/opt/local/mongodb/bin/mongod --shardsvr --maxConns 10000 --replSet shard1 --port 27017 --dbpath /opt/local/mongodb/data/shard12/ --oplogSize 500 --logpath=/opt/local/mongodb/data/logs/shard12.log --logappend --fork
server 3
/opt/local/mongodb/bin/mongod --shardsvr --maxConns 10000 --replSet shard1 --port 27017 --dbpath /opt/local/mongodb/data/shard13/ --oplogSize 500 --logpath=/opt/local/mongodb/data/logs/shard13.log --logappend --fork
7.1.2 初始化replica set
用mongo連線其中一個mongod，執行:
/opt/local/mongodb/bin/mongo --host 10.3.0.100:27017
----------------------------------------------------------------------
> config= {_id: 'shard1', members: [ {_id:0,host:'10.3.0.100:27017'},
... {_id:1,host:'10.3.0.101:27017'},
... {_id:2,host:'10.3.0.102:27017'},]
... }
{
"_id" : "shard1",
"members" : [
{
"_id" : 0,
"host" : "10.3.0.100:27017"
},
{
"_id" : 1,
"host" : "10.3.0.101:27017"
},
{
"_id" : 2,
"host" : "10.3.0.102:27017"
}
]
}
> rs.initiate(config);
{
"info" : "Config now saved locally. Should come online in about a minute.",
"ok" : 1
}
-------------------------------------------------------------------------
7.2.1 配置shard2所用到的replica sets:
/opt/local/mongodb/bin/mongod --shardsvr --maxConns 10000 --replSet shard2 --port 17017 --dbpath /opt/local/mongodb/data/shard21/ --oplogSize 100 --logpath=/opt/local/mongodb/data/logs/shard21.log --logappend --fork
server 2
/opt/local/mongodb/bin/mongod --shardsvr --maxConns 10000 --replSet shard2 --port 17017 --dbpath /opt/local/mongodb/data/shard22/ --oplogSize 100 --logpath=/opt/local/mongodb/data/logs/shard22.log --logappend --fork
server 3
/opt/local/mongodb/bin/mongod --shardsvr --maxConns 10000 --replSet shard2 --port 17017 --dbpath /opt/local/mongodb/data/shard23/ --oplogSize 100 --logpath=/opt/local/mongodb/data/logs/shard23.log --logappend --fork
7.2.2 初始化replica set
用mongo連線其中一個mongod，執行:
/opt/local/mongodb/bin/mongo --host 10.3.0.100:17017
-------------------------------------------------------------------------
> config= {_id: 'shard2', members: [ {_id:0,host:'10.3.0.100:17017'},
... {_id:1,host:'10.3.0.101:17017'},
... {_id:2,host:'10.3.0.102:17017'},]
... }
{
"_id" : "shard2",
"members" : [
{
"_id" : 0,
"host" : "10.3.0.100:17017"
},
{
"_id" : 1,
"host" : "10.3.0.101:17017"
},
{
"_id" : 2,
"host" : "10.3.0.102:17017"
}
]
}
> rs.initiate(config);
{
"errmsg" : "couldn't initiate : set name does not match the set name host 10.3.0.101:17017 expects",
"ok" : 0
}
>
-------------------------------------------------------------------------
8. 配置三臺config server
server 1
/opt/local/mongodb/bin/mongod --configsvr --dbpath /opt/local/mongodb/data/config/ --port 20000 --logpath /opt/local/mongodb/data/logs/config.log --logappend --fork
server 2
/opt/local/mongodb/bin/mongod --configsvr --dbpath /opt/local/mongodb/data/config/ --port 20000 --logpath /opt/local/mongodb/data/logs/config.log --logappend --fork
server 3
/opt/local/mongodb/bin/mongod --configsvr --dbpath /opt/local/mongodb/data/config/ --port 20000 --logpath /opt/local/mongodb/data/logs/config.log --logappend --fork
9. 配置 mongs
在server1,server2,server3 上分別執行：
/opt/local/mongodb/bin/mongos --configdb 10.3.0.100:20000,10.3.0.101:20000,10.3.0.102:20000 --port 20001 --chunkSize 5 --logpath /opt/local/mongodb/data/logs/mongos.log --logappend --fork
#mongs不需要dbpath
10 Configuring the Shard Cluster
連線到其中一個mongos程式，並切換到admin資料庫做以下配置
10.1 連線到mongs，並切換到admin
/opt/local/mongodb/bin/mongo 10.3.0.100:20001/admin
MongoDB shell version: 2.2.0
connecting to: 10.3.0.100:20001/admin
mongos> db
admin
10.2 加入shards
如裡shard是單臺伺服器，
用>db.runCommand( { addshard : “<serverhostname>[:<port>]” } )加入
如果shard是replica sets，用replicaSetName/<serverhostname>[:port]
----------------------------------------------------------------------
shard 1
db.runCommand( { addshard : "shard1/10.3.0.100:27017,10.3.0.101:27017,10.3.0.102:27017", maxsize:204800});
{ "shardAdded" : "shard1", "ok" : 1 }
--------------------------------------------------------------------
shard 2
mongos> db.runCommand( { addshard : "shard2/10.3.0.100:17017,10.3.0.101:17017,10.3.0.102:17017", maxsize:204800});
{ "shardAdded" : "shard2", "ok" : 1 }
----------------------------------------------------------------------
檢視是否新增成功
mongos> db.runCommand( { listshards : 1 } )
{
"shards" : [
{
"_id" : "shard1",
"host" : "shard1/10.3.0.100:27017,10.3.0.101:27017,10.3.0.102:27017"
},
{
"_id" : "shard2",
"host" : "shard2/10.3.0.100:17017,10.3.0.101:17017,10.3.0.102:17017"
}
],
"ok" : 1
}
刪除 shards
db.runCommand( { removeshard : "shard1/10.3.0.100:27017,10.3.0.101:27017"} );
11. 啟用分片
命令：
> db.runCommand( { enablesharding : “<dbname>” } );
通過執行以上命令，可以讓資料庫跨shard，如果不執行這步，資料庫只會存放在一個shard，一旦啟用資料庫分片，資料庫中不同的collection將被存放在不同的shard上，但一個collection仍舊存放在同一個shard上，要使單個collection也分片，
還需單獨對collection作些操作
PS: 其他命令.....操作...
#增添節點:
以一個新建的mongodb服務為例,
連線 mongo
/opt/local/mongodb/bin/mongo --host 10.3.0.100:27017
MongoDB shell version: 2.2.0
connecting to: 10.3.0.100:27017/test
shard1:PRIMARY>rs.add(“IP:port”);
#刪除節點：
必須在主節點上操作：
PRIMARY>rs.remove(“127.0.0.1:27020”)
檢視同步狀態
rs.status()
讓從有讀的許可權：
在主控上執行:PRIMARY> db.getMongo().setSlaveOk();
SECONDARY> rs.slaveOk();
群集增加驗證 auth keyFile
群集如果加了 auth 驗證，群集之前互相取不到 主 就無法驗證...必須要增加 keyFile 驗證才行...
先建立auth 驗證
-----------------------------------------------------------------
> use admin
switched to db admin
> db.addUser('sa','sa')
{
"_id" : ObjectId("4e2914a585178da4e03a16c3"),
"user" : "sa",
"readOnly" : false,
"pwd" : "75692b1d11c072c6c79332e248c4f699"
}
>
----------------------------------------------------------------------------------
然後在每個server裡建立 key 檔案....
vi /opt/local/mongodb/data/config/key
輸入隨意字元.... 1234567890111111111
授權 chown mongodb:mongodb key
chmod 600 key
-------------------------------------------------------------------------
然後分別啟動
server 1
/opt/local/mongodb/bin/mongod --shardsvr --replSet shard1 --port 27017 --dbpath /opt/local/mongodb/data/shard11/ --oplogSize 100 --logpath=/opt/local/mongodb/data/logs/shard11.log --logappend --fork --keyFile /opt/local/mongodb/data/config/key
server 2
/opt/local/mongodb/bin/mongod --shardsvr --replSet shard1 --port 27017 --dbpath /opt/local/mongodb/data/shard12/ --oplogSize 100 --logpath=/opt/local/mongodb/data/logs/shard12.log --logappend --fork --keyFile /opt/local/mongodb/data/config/key
server 3
/opt/local/mongodb/bin/mongod --shardsvr --replSet shard1 --port 27017 --dbpath /opt/local/mongodb/data/shard13/ --oplogSize 100 --logpath=/opt/local/mongodb/data/logs/shard13.log --logappend --fork --keyFile /opt/local/mongodb/data/config/key
-------------------------------------------------------------------------
在啟動config
server 1
/opt/local/mongodb/bin/mongod --configsvr --dbpath /opt/local/mongodb/data/config/ --port 20000 --logpath /opt/local/mongodb/data/logs/config.log --logappend --fork --keyFile /opt/local/mongodb/data/config/key
server 2
/opt/local/mongodb/bin/mongod --configsvr --dbpath /opt/local/mongodb/data/config/ --port 20000 --logpath /opt/local/mongodb/data/logs/config.log --logappend --fork --keyFile /opt/local/mongodb/data/config/key
server 3
/opt/local/mongodb/bin/mongod --configsvr --dbpath /opt/local/mongodb/data/config/ --port 20000 --logpath /opt/local/mongodb/data/logs/config.log --logappend --fork --keyFile /opt/local/mongodb/data/config/key
-------------------------------------------------------------------------
最後分別啟動mongos
/opt/local/mongodb/bin/mongos --configdb 10.3.0.100:20000,10.3.0.101:20000,10.3.0.102:20000 --port 20001 --chunkSize 5 --logpath /opt/local/mongodb/data/logs/mongos.log --logappend --fork --keyFile /opt/local/mongodb/data/config/key
-------------------------------------------------------------------------
然後在檢視是否能讀取主....
./mongo
MongoDB shell version: 2.2.0
connecting to: test
> show dbs
Wed Sep 5 01:51:44 uncaught exception: listDatabases failed:{ "errmsg" : "need to login", "ok" : 0 }
> use admin
switched to db admin
> db.auth('sa','sa')
1
shard1:SECONDARY> db.serverStatus()
就可以檢視....
注意:
新增auth 以後...匯入資料 等一系列操作...都必須驗證使用者...
如：
/opt/local/mongodb/bin/mongorestore -u sa -p sa --drop /opt/1
其他都需要進行操作！否則！報錯！！！
強制切換主
cfg = rs.conf()
cfg.members[0].priority = 0.5
cfg.members[1].priority = 0.5
cfg.members[2].priority = 1
rs.reconfig(cfg)
升級MongoDB分片叢集到2.2版本
MongoDB 2.2版本與2.0版本在分片叢集中共存存在問題，所以升級需要按照官方文件的步驟進行：
禁用Balancer
mongos> sh.setBalancerState(false)
首先升級所有的mongos
升級Config伺服器,一次升級一臺
升級一個分片的複製集
依次關閉並升級一個複製集的secondary
在Primary庫上執行rs.stepDown()，強制下線。關閉Primary庫並升級二進位制程式。
當都升級完成後，連線mongos執行：sh.setBalancerState(true)恢復平衡器。
----------------------------------------------------------------------------------
WARNING: You are running on a NUMA machine.
We suggest launching mongod like this to avoid performance problems:
numactl --interleave=all mongod [other options]
sudo -u mongodb numactl --interleave=all /opt/local/mongodb/bin/mongod --shardsvr --maxConns 10000 --replSet shard1 --port 27017 --dbpath /opt/local/mongodb/data/shard16/ --oplogSize 1000 --logpath=/opt/local/mongodb/data/logs/shard16.log --logappend --fork
編輯核心 echo 0 > /proc/sys/vm/zone_reclaim_mode
----------------------------------------------------------------------------------
限制 mongodb 佔用系統所有記憶體
ulimit -s 4096 && ulimit -m 31457280 && sudo -u mongodb numactl --interleave=all /opt/local/mongodb/bin/mongod --shardsvr --maxConns 10000 --replSet shard1 --port 27017 --dbpath /opt/local/mongodb/data/shard16/ --oplogSize 1000 --logpath=/opt/local/mongodb/data/logs/shard16.log --logappend --fork