MongoDB主從複製

jx_yu發表於2015-01-14
MongoDB

1.    MongoDB主從複製

主從複製是Mongodb最常用的複製方式，可用於備份、故障恢復、讀擴充套件等

規劃

這裡資源有限,在一臺機器上不同dbpath和port啟動多個mongodb例項來實現主從

　

conf_file

dbpath

logpath

port

主

/etc/mongodb/master.conf

/data/mongodb/master

/data/mongodb/logs/master.log

10000

從

/etc/mongodb/slave.conf

/data/mongodb/slave

/data/mongodb/logs/slave.log

20000

建立[db|log]path

~]# mkdir /data/mongodb/master

~]# mkdir /data/mongodb/slave

~]# mkdir /data/mongodb/logs

建立配置檔案

~]# mkdir /etc/mongodb/

#Master配置檔案

~]# vim /etc/mongodb/master.conf

# mongodb config file

dbpath = /data/mongodb/master

logpath = /data/mongodb/logs/master.log

logappend = true

fork = true

oplogSize = 100

port = 10000

master = true

#Slave配置檔案

~]# vim /etc/mongodb/slave.conf

# mongodb config file

dbpath = /data/mongodb/slave

logpath = /data/mongodb/logs/slave.log

logappend = true

fork = true

port = 20000

slave = true

source = 192.168.211.217:10000

啟動master

~]# /usr/local/mongodb/bin/mongod -f /etc/mongodb/master.conf

about to fork child process, waiting until server is ready for connections.

forked process: 30160

child process started successfully, parent exiting

~]# netstat -ntpl|grep mong

tcp        0      0 0.0.0.0:10000               0.0.0.0:*                   LISTEN      30160/mongod   

~]# ps -ef |grep mong

root     30160     1  0 10:06 ?        00:00:00 /usr/local/mongodb/bin/mongod -f /etc/mongodb/master.conf

#監控master的log

~]# tail –f /data/mongodb/logs/master.log

2015-01-14T10:06:45.147+0800 [initandlisten] MongoDB starting : pid=30160 port=10000 dbpath=/data/mongodb/master master=1 64-bit host=test-whole-db

2015-01-14T10:06:45.147+0800 [initandlisten] db version v2.6.6

2015-01-14T10:06:45.147+0800 [initandlisten] git version: 608e8bc319627693b04cc7da29ecc300a5f45a1f

2015-01-14T10:06:45.147+0800 [initandlisten] build info: Linux build10.nj1.10gen.cc 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Jan 3 21:39:27 UTC 2014 x86_64 BOOST_LIB_VERSION=1_49

2015-01-14T10:06:45.147+0800 [initandlisten] allocator: tcmalloc

2015-01-14T10:06:45.147+0800 [initandlisten] options: { config: "/etc/mongodb/master.conf", master: true, net: { port: 10000 }, processManagement: { fork: true }, replication: { oplogSizeMB: 100 }, storage: { dbPath: "/data/mongodb/master" }, systemLog: { destination: "file", logAppend: true, path: "/data/mongodb/logs/master.log" } }

2015-01-14T10:06:45.214+0800 [initandlisten] journal dir=/data/mongodb/master/journal

2015-01-14T10:06:45.214+0800 [initandlisten] recover : no journal files present, no recovery needed

2015-01-14T10:06:45.476+0800 [initandlisten] ******

2015-01-14T10:06:45.476+0800 [initandlisten] creating replication oplog of size: 100MB...

2015-01-14T10:06:45.476+0800 [FileAllocator] allocating new datafile /data/mongodb/master/local.1, filling with zeroes...

2015-01-14T10:06:45.476+0800 [FileAllocator] creating directory /data/mongodb/master/_tmp

2015-01-14T10:06:45.525+0800 [FileAllocator] done allocating datafile /data/mongodb/master/local.1, size: 128MB,  took 0.024 secs

2015-01-14T10:06:45.526+0800 [initandlisten] ******

2015-01-14T10:06:45.526+0800 [initandlisten] waiting for connections on port 10000

2015-01-14T10:07:24.089+0800 [initandlisten] connection accepted from 127.0.0.1:61489 #1 (1 connection now open)

#確認是否為master

~]# /usr/local/mongodb/bin/mongo --port 10000

MongoDB shell version: 2.6.6

connecting to: 127.0.0.1:10000/test

>  db.isMaster()

{

        "ismaster" : true,

        "maxBsonObjectSize" : 16777216,

        "maxMessageSizeBytes" : 48000000,

        "maxWriteBatchSize" : 1000,

        "localTime" : ISODate("2015-01-14T02:50:13.810Z"),

        "maxWireVersion" : 2,

        "minWireVersion" : 0,

        "ok" : 1

}

啟動slave

~]# /usr/local/mongodb/bin/mongod -f /etc/mongodb/slave.conf

about to fork child process, waiting until server is ready for connections.

forked process: 31056

child process started successfully, parent exiting

~]# tail -f /data/mongodb/logs/slave.log

2015-01-14T10:55:26.719+0800 [initandlisten] MongoDB starting : pid=31056 port=20000 dbpath=/data/mongodb/slave slave=1 64-bit host=test-whole-db

2015-01-14T10:55:26.719+0800 [initandlisten] db version v2.6.6

2015-01-14T10:55:26.719+0800 [initandlisten] git version: 608e8bc319627693b04cc7da29ecc300a5f45a1f

2015-01-14T10:55:26.719+0800 [initandlisten] build info: Linux build10.nj1.10gen.cc 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Jan 3 21:39:27 UTC 2014 x86_64 BOOST_LIB_VERSION=1_49

2015-01-14T10:55:26.719+0800 [initandlisten] allocator: tcmalloc

2015-01-14T10:55:26.719+0800 [initandlisten] options: { config: "/etc/mongodb/slave.conf", net: { port: 20000 }, processManagement: { fork: true }, slave: true, source: "192.168.211.217:10000", storage: { dbPath: "/data/mongodb/slave" }, systemLog: { destination: "file", logAppend: true, path: "/data/mongodb/logs/slave.log" } }

2015-01-14T10:55:26.774+0800 [initandlisten] journal dir=/data/mongodb/slave/journal

2015-01-14T10:55:26.774+0800 [initandlisten] recover : no journal files present, no recovery needed

2015-01-14T10:55:28.453+0800 [initandlisten] preallocateIsFaster=true 20.96

2015-01-14T10:55:30.548+0800 [initandlisten] preallocateIsFaster=true 13.2

2015-01-14T10:55:33.139+0800 [initandlisten] preallocateIsFaster=true 19.04

2015-01-14T10:55:33.139+0800 [initandlisten] preallocateIsFaster check took 6.364 secs

2015-01-14T10:55:33.139+0800 [initandlisten] preallocating a journal file /data/mongodb/slave/journal/prealloc.0

2015-01-14T10:56:08.323+0800 [initandlisten] preallocating a journal file /data/mongodb/slave/journal/prealloc.1

2015-01-14T10:56:36.587+0800 [initandlisten] preallocating a journal file /data/mongodb/slave/journal/prealloc.2

2015-01-14T10:57:17.838+0800 [initandlisten] allocating new ns file /data/mongodb/slave/local.ns, filling with zeroes...

2015-01-14T10:57:18.417+0800 [FileAllocator] allocating new datafile /data/mongodb/slave/local.0, filling with zeroes...

2015-01-14T10:57:18.417+0800 [FileAllocator] creating directory /data/mongodb/slave/_tmp

2015-01-14T10:57:18.484+0800 [FileAllocator] done allocating datafile /data/mongodb/slave/local.0, size: 64MB,  took 0.024 secs

2015-01-14T10:57:18.505+0800 [initandlisten] build index on: local.startup_log properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "local.startup_log" }

2015-01-14T10:57:18.505+0800 [initandlisten]     added index to empty collection

2015-01-14T10:57:18.509+0800 [initandlisten] command local.$cmd command: create { create: "startup_log", size: 10485760, capped: true } ntoreturn:1 keyUpdates:0 numYields:0  reslen:37 671ms

2015-01-14T10:57:18.519+0800 [initandlisten] waiting for connections on port 20000

2015-01-14T10:57:19.521+0800 [replslave] build index on: local.me properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "local.me" }

2015-01-14T10:57:19.521+0800 [replslave]         added index to empty collection

2015-01-14T10:57:19.523+0800 [replslave] build index on: local.sources properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "local.sources" }

2015-01-14T10:57:19.523+0800 [replslave]         added index to empty collection

2015-01-14T10:57:19.523+0800 [replslave] repl: syncing from host:192.168.211.217:10000

2015-01-14T10:57:19.530+0800 [replslave] resync: dropping database admin

2015-01-14T10:57:19.690+0800 [replslave] removeJournalFiles

2015-01-14T10:57:19.750+0800 [replslave] resync: cloning database admin to get an initial copy

2015-01-14T10:57:19.760+0800 [replslave] allocating new ns file /data/mongodb/slave/admin.ns, filling with zeroes...

2015-01-14T10:57:20.182+0800 [FileAllocator] allocating new datafile /data/mongodb/slave/admin.0, filling with zeroes...

2015-01-14T10:57:20.223+0800 [FileAllocator] done allocating datafile /data/mongodb/slave/admin.0, size: 64MB,  took 0.041 secs

2015-01-14T10:57:20.320+0800 [replslave] build index on: admin.system.version properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "admin.system.version" }

2015-01-14T10:57:20.320+0800 [replslave]         building index using bulk method

2015-01-14T10:57:20.320+0800 [replslave] build index done.  scanned 1 total records. 0 secs

2015-01-14T10:57:20.335+0800 [replslave] build index on: admin.system.users properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "admin.system.users" }

2015-01-14T10:57:20.335+0800 [replslave]         building index using bulk method

2015-01-14T10:57:20.335+0800 [replslave] build index done.  scanned 3 total records. 0 secs

2015-01-14T10:57:20.336+0800 [replslave] build index on: admin.system.users properties: { v: 1, unique: true, key: { user: 1, db: 1 }, name: "user_1_db_1", ns: "admin.system.users" }

2015-01-14T10:57:20.336+0800 [replslave]         building index using bulk method

2015-01-14T10:57:20.336+0800 [replslave] build index done.  scanned 3 total records. 0 secs

2015-01-14T10:57:20.336+0800 [replslave] resync: done with initial clone for db: admin

2015-01-14T10:57:28.379+0800 [replslave] repl:   applied 1 operations

2015-01-14T10:57:28.379+0800 [replslave] repl:  end sync_pullOpLog syncedTo: Jan 14 10:57:28 54b5db18:1

2015-01-14T10:57:28.379+0800 [replslave] repl: syncing from host:192.168.211.217:10000

2015-01-14T10:57:28.379+0800 [replslave] resync: dropping database test

2015-01-14T10:57:29.073+0800 [replslave] removeJournalFiles

2015-01-14T10:57:29.171+0800 [replslave] resync: cloning database test to get an initial copy

2015-01-14T10:57:29.172+0800 [replslave] allocating new ns file /data/mongodb/slave/test.ns, filling with zeroes...

2015-01-14T10:57:29.663+0800 [FileAllocator] allocating new datafile /data/mongodb/slave/test.0, filling with zeroes...

2015-01-14T10:57:29.689+0800 [FileAllocator] done allocating datafile /data/mongodb/slave/test.0, size: 64MB,  took 0.026 secs

2015-01-14T10:57:29.776+0800 [replslave] build index on: test.emp properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "test.emp" }

2015-01-14T10:57:29.776+0800 [replslave]         building index using bulk method

2015-01-14T10:57:29.776+0800 [replslave] build index done.  scanned 15 total records. 0 secs

2015-01-14T10:57:29.792+0800 [replslave] build index on: test.my_collection properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "test.my_collection" }

2015-01-14T10:57:29.792+0800 [replslave]         added index to empty collection

2015-01-14T10:57:29.793+0800 [replslave] build index on: test.yu_collection properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "test.yu_collection" }

2015-01-14T10:57:29.793+0800 [replslave]         added index to empty collection

2015-01-14T10:57:29.794+0800 [replslave] resync: done with initial clone for db: test

2015-01-14T10:57:34.806+0800 [replslave] repl: syncing from host:192.168.211.217:10000

測試主從

#slave上當前沒有masterslave集合

#master建立masterslave，並且插入文件

#再次檢視slave上，也同步了master的操作

local資料庫

       Local（本地）資料庫用來存放所有內部複製狀態，主從節點都有。Local資料庫不會被複制，每個節點獨自擁有自己的local資料庫

#master上檢視master的資訊

> use local

switched to db local

> db.slaves.find()

{ "_id" : ObjectId("54b5db0fddeb3d1dd59a5aa1"), "config" : { "host" : "192.168.211.217:46756", "upgradeNeeded" : true }, "ns" : "local.oplog.$main", "syncedTo" : Timestamp(1421216379, 1) }

#slave上檢視master的資訊

> use local

switched to db local

> db.sources.find()

{ "_id" : ObjectId("54b5db0fddeb3d1dd59a5aa2"), "host" : "192.168.211.217:10000", "source" : "main", "syncedTo" : Timestamp(1421213869, 1) }

主從複製原理

在主從結構中，主節點的操作記錄成為oplog（operation log）。

oplog儲存在一個系統資料庫local的集合oplog.$main中，這個集合的每個文件都代表主節點上執行的一個操作。

#master上檢視oplog內容

> db

local

> db.oplog.$main.find()

{ "ts" : Timestamp(1421201205, 11), "op" : "n", "ns" : "", "o" : {  } }

{ "ts" : Timestamp(1421201219, 1), "op" : "n", "ns" : "", "o" : {  } }

{ "ts" : Timestamp(1421201229, 1), "op" : "n", "ns" : "", "o" : {  } }

{ "ts" : Timestamp(1421201239, 1), "op" : "n", "ns" : "", "o" : {  } }

{ "ts" : Timestamp(1421201249, 1), "op" : "n", "ns" : "", "o" : {  } }

。。。。。。

鍵“ts”表示時間戳，鍵“op”表示操作型別，“ns”表示操作的集合名，“o”表示具體操作的文件

注意：儲存在oplog中的操作有時可能和主節點操作並不一致，因為儲存操作前要進行等冪變換，也就是這些操作必須可以再從伺服器上多次執行，只要順序一致，結果就一致。  

因此，比如更新器$inc的操作，最後會被變換為$set操作！並且這個集合中只會儲存影響資料的操作，查詢這類不影響資料的操作不會以oplog的形式進行儲存！從伺服器會定期從主伺服器中獲取oplog記錄，然後在本機上執行！

對於儲存oplog的集合，MongoDB採用的是固定集合，也就是說隨著操作過多，新的操作會覆蓋舊的操作！這樣做也是有道理的，不然，這個集合佔用的空間就無法估算了！我們在啟動服務時，可以透過選項--oplogSize來指定這個集合的大小，單位是MB，在Windows平臺下，預設MongoDB會使用資料庫安裝分割槽可用空間的5%作為這個集合的大小！

常用選項

--only

在從節點上指定只複製特定的某個資料庫(預設是複製所有資料庫)

--slavedelay

用在從節點上,當應用主節點的操作時,從節點增加延時複製(單位秒).這樣就能輕鬆設定延時從節點,這種節點對使用者無意中刪除重要文件或者插入垃圾資料等有防護作用,這些不良操作都會被複制到所有的從節點上,透過延時執行操作,可以有個恢復的時間差.

--fastsync

以主節點的資料快照為基礎啟動從節點.如果資料目錄一開始是主節點的資料快照,從節點用這個選項啟動要比做完整的同步快的多.

--autoresync

如果從節點與主節點不同步了,則自動重新同步

--oplogsize

主節點oplog的大小(單位MB)

新增刪除源

來源於《mongodb權威指南》



來自 “ ITPUB部落格 ” ，連結：http://blog.itpub.net/27000195/viewspace-1401941/，如需轉載，請註明出處，否則將追究法律責任。

相關文章