zookeeper 叢集安裝(單點與分散式成功安裝)
ZooKeeper是一個分散式開源框架,提供了協調分散式應用的基本服務,它向外部應用暴露一組通用服務——分散式同步(Distributed Synchronization)、命名服務(Naming Service)、叢集維護(Group Maintenance)等,簡化分散式應用協調及其管理的難度,提供高效能的分散式服務。ZooKeeper本身可以以Standalone模式安裝執行,不過它的長處在於透過分散式ZooKeeper叢集(一個Leader,多個Follower),基於一定的策略來保證ZooKeeper叢集的穩定性和可用性,從而實現分散式應用的可靠性。
有關ZooKeeper的介紹,網上很多,也可以參考文章後面,我整理的一些相關連結。
下面,我們簡單說明一下ZooKeeper的配置。
ZooKeeper Standalone模式
從Apache網站上(zookeeper.apache.org)下載ZooKeeper軟體包,我選擇了3.3.4版本的(zookeeper-3.3.4.tar.gz),在一臺Linux機器上安裝非常容易,只需要解壓縮後,簡單配置一下即可以啟動ZooKeeper伺服器程式。
將zookeeper-3.3.4/conf目錄下面的 zoo_sample.cfg修改為zoo.cfg,配置檔案內容如下所示:
- tickTime=2000
- dataDir=/home/hadoop/storage/zookeeper
- clientPort=2181
- initLimit=5
- syncLimit=2
- tickTime —— the basic time unit in milliseconds used by ZooKeeper. It is used to do heartbeats and the minimum session timeout will be twice the tickTime.
- dataDir —— the location to store the in-memory database snapshots and, unless specified otherwise, the transaction log of updates to the database.
- clientPort —— the port to listen for client connections
- cd zookeeper-3.3.4/
- bin/zkServer.sh start
在客戶端連線ZooKeeper伺服器,執行如下命令:
- bin/zkCli.sh -server dynamic:2181
- bin/zkCli.sh
ZooKeeper Distributed模式
ZooKeeper分散式模式安裝(ZooKeeper叢集)也比較容易,這裡說明一下基本要點。
首先要明確的是,ZooKeeper叢集是一個獨立的分散式協調服務叢集,“獨立”的含義就是說,如果想使用ZooKeeper實現分散式應用的協調與管理,簡化協調與管理,任何分散式應用都可以使用,這就要歸功於Zookeeper的資料模型(Data Model)和層次名稱空間(Hierarchical Namespace)結構,詳細可以參考。在設計你的分散式應用協調服務時,首要的就是考慮如何組織層次名稱空間。
下面說明分散式模式的安裝配置,過程如下所示:
第一步:主機名稱到IP地址對映配置
ZooKeeper叢集中具有兩個關鍵的角色:Leader和Follower。叢集中所有的結點作為一個整體對分散式應用提供服務,叢集中每個結點之間都互相連線,所以,在配置的ZooKeeper叢集的時候,每一個結點的host到IP地址的對映都要配置上叢集中其它結點的對映資訊。
例如,我的ZooKeeper叢集中每個結點的配置,以slave-01為例,/etc/hosts內容如下所示:
- 192.168.0.179 slave-01
- 192.168.0.178 slave-02
- 192.168.0.177 slave-03
ZooKeeper採用一種稱為Leader election的選舉演算法。在整個叢集執行過程中,只有一個Leader,其他的都是Follower,如果ZooKeeper叢集在執行過程中Leader出了問題,系統會採用該演算法重新選出一個Leader。因此,各個結點之間要能夠保證互相連線,必須配置上述對映。
ZooKeeper叢集啟動的時候,會首先選出一個Leader,在Leader election過程中,某一個滿足選舉算的結點就能成為Leader。整個叢集的架構可以參考。
第二步:修改ZooKeeper配置檔案
在其中一臺機器(slave-01)上,解壓縮zookeeper-3.3.4.tar.gz,修改配置檔案conf/zoo.cfg,內容如下所示:
- tickTime=2000
- dataDir=/home/hadoop/storage/zookeeper
- clientPort=2181
- initLimit=5
- syncLimit=2
- server.1=slave-01:2888:3888
- server.2=slave-02:2888:3888
- server.3=slave-03:2888:3888
第三步:遠端複製分發安裝檔案
上面已經在一臺機器slave-01上配置完成ZooKeeper,現在可以將該配置好的安裝檔案遠端複製到叢集中的各個結點對應的目錄下:
- cd /home/hadoop/installation/
- scp -r zookeeper-3.3.4/ hadoop@slave-02:/home/hadoop/installation/
- scp -r zookeeper-3.3.4/ hadoop@slave-03:/home/hadoop/installation/
在我們配置的dataDir指定的目錄下面,建立一個myid檔案,裡面內容為一個數字,用來標識當前主機,conf/zoo.cfg檔案中配置的server.X中X為什麼數字,則myid檔案中就輸入這個數字,例如:
- hadoop@slave-01:~/installation/zookeeper-3.3.4$ echo "1" > /home/hadoop/storage/zookeeper/myid
- hadoop@slave-02:~/installation/zookeeper-3.3.4$ echo "2" > /home/hadoop/storage/zookeeper/myid
- hadoop@slave-03:~/installation/zookeeper-3.3.4$ echo "3" > /home/hadoop/storage/zookeeper/myid
第五步:啟動ZooKeeper叢集
在ZooKeeper叢集的每個結點上,執行啟動ZooKeeper服務的指令碼,如下所示:
- hadoop@slave-01:~/installation/zookeeper-3.3.4$ bin/zkServer.sh start
- hadoop@slave-02:~/installation/zookeeper-3.3.4$ bin/zkServer.sh start
- hadoop@slave-03:~/installation/zookeeper-3.3.4$ bin/zkServer.sh start
- hadoop@slave-01:~/installation/zookeeper-3.3.4$ tail -500f zookeeper.out
- 2012-01-08 06:51:19,117 - INFO [main:QuorumPeerConfig@90] - Reading configuration from: /home/hadoop/installation/zookeeper-3.3.4/bin/../conf/zoo.cfg
- 2012-01-08 06:51:19,133 - INFO [main:QuorumPeerConfig@310] - Defaulting to majority quorums
- 2012-01-08 06:51:19,167 - INFO [main:QuorumPeerMain@119] - Starting quorum peer
- 2012-01-08 06:51:19,227 - INFO [main:NIOServerCnxn$Factory@143] - binding to port 0.0.0.0/0.0.0.0:2181
- 2012-01-08 06:51:19,277 - INFO [main:QuorumPeer@819] - tickTime set to 2000
- 2012-01-08 06:51:19,278 - INFO [main:QuorumPeer@830] - minSessionTimeout set to -1
- 2012-01-08 06:51:19,279 - INFO [main:QuorumPeer@841] - maxSessionTimeout set to -1
- 2012-01-08 06:51:19,281 - INFO [main:QuorumPeer@856] - initLimit set to 5
- 2012-01-08 06:51:19,347 - INFO [Thread-1:QuorumCnxManager$Listener@473] - My election bind port: 3888
- 2012-01-08 06:51:19,393 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumPeer@621] - LOOKING
- 2012-01-08 06:51:19,396 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@663] - New election. My id = 1, Proposed zxid = 0
- 2012-01-08 06:51:19,400 - INFO [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 1 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 1 (n.sid), LOOKING (my state)
- 2012-01-08 06:51:19,416 - WARN [WorkerSender Thread:QuorumCnxManager@384] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
- java.net.ConnectException: Connection refused
- at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
- at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
- at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:340)
- at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:360)
- at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:333)
- at java.lang.Thread.run(Thread.java:662)
- 2012-01-08 06:51:19,420 - WARN [WorkerSender Thread:QuorumCnxManager@384] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
- java.net.ConnectException: Connection refused
- at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
- at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
- at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:340)
- at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:360)
- at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:333)
- at java.lang.Thread.run(Thread.java:662)
- 2012-01-08 06:51:19,612 - WARN [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
- java.net.ConnectException: Connection refused
- at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
- at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
- at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
- at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
- at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
- 2012-01-08 06:51:19,615 - WARN [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
- java.net.ConnectException: Connection refused
- at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
- at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
- at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
- at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
- at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
- 2012-01-08 06:51:19,616 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@697] - Notification time out: 400
- 2012-01-08 06:51:20,019 - WARN [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
- java.net.ConnectException: Connection refused
- at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
- at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
- at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
- at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
- at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
- 2012-01-08 06:51:20,021 - WARN [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
- java.net.ConnectException: Connection refused
- at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
- at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
- at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
- at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
- at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
- 2012-01-08 06:51:20,022 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@697] - Notification time out: 800
- 2012-01-08 06:51:20,825 - WARN [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
- java.net.ConnectException: Connection refused
- at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
- at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
- at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
- at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
- at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
- 2012-01-08 06:51:20,827 - WARN [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
- java.net.ConnectException: Connection refused
- at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
- at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
- at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
- at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
- at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
- 2012-01-08 06:51:20,828 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@697] - Notification time out: 1600
- 2012-01-08 06:51:22,435 - WARN [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
- java.net.ConnectException: Connection refused
- at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
- at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
- at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
- at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
- at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
- 2012-01-08 06:51:22,439 - WARN [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
- java.net.ConnectException: Connection refused
- at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
- at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
- at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
- at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
- at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
- 2012-01-08 06:51:22,441 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@697] - Notification time out: 3200
- 2012-01-08 06:51:22,945 - INFO [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 2 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 2 (n.sid), LOOKING (my state)
- 2012-01-08 06:51:22,946 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@721] - Updating proposal
- 2012-01-08 06:51:22,949 - WARN [WorkerSender Thread:QuorumCnxManager@384] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
- java.net.ConnectException: Connection refused
- at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
- at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
- at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:340)
- at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:360)
- at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:333)
- at java.lang.Thread.run(Thread.java:662)
- 2012-01-08 06:51:22,951 - INFO [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 2 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 1 (n.sid), LOOKING (my state)
- 2012-01-08 06:51:23,156 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumPeer@643] - FOLLOWING
- 2012-01-08 06:51:23,170 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Learner@80] - TCP NoDelay set to: true
- 2012-01-08 06:51:23,206 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
- 2012-01-08 06:51:23,207 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:host.name=slave-01
- 2012-01-08 06:51:23,207 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.version=1.6.0_30
- 2012-01-08 06:51:23,208 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.vendor=Sun Microsystems Inc.
- 2012-01-08 06:51:23,208 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.home=/home/hadoop/installation/jdk1.6.0_30/jre
- 2012-01-08 06:51:23,209 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.class.path=/home/hadoop/installation/zookeeper-3.3.4/bin/../build/classes:/home/hadoop/installation/zookeeper-3.3.4/bin/../build/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../zookeeper-3.3.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/log4j-1.2.15.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/jline-0.9.94.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-lang-2.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-collections-3.2.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-cli-1.1.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-tasks-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-core-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../src/java/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../conf:/home/hadoop/installation/jdk1.6.0_30/lib/*.jar:/home/hadoop/installation/jdk1.6.0_30/jre/lib/*.jar
- 2012-01-08 06:51:23,210 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.library.path=/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
- 2012-01-08 06:51:23,210 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.io.tmpdir=/tmp
-
2012-01-08 06:51:23,212 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.compiler=
- 2012-01-08 06:51:23,212 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:os.name=Linux
- 2012-01-08 06:51:23,212 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:os.arch=i386
- 2012-01-08 06:51:23,213 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:os.version=3.0.0-14-generic
- 2012-01-08 06:51:23,213 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:user.name=hadoop
- 2012-01-08 06:51:23,214 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:user.home=/home/hadoop
- 2012-01-08 06:51:23,214 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:user.dir=/home/hadoop/installation/zookeeper-3.3.4
- 2012-01-08 06:51:23,223 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:ZooKeeperServer@151] - Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000 datadir /home/hadoop/storage/zookeeper/version-2 snapdir /home/hadoop/storage/zookeeper/version-2
- 2012-01-08 06:51:23,339 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Learner@294] - Getting a snapshot from leader
- 2012-01-08 06:51:23,358 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Learner@325] - Setting leader epoch 1
- 2012-01-08 06:51:23,358 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FileTxnSnapLog@254] - Snapshotting: 0
- 2012-01-08 06:51:25,511 - INFO [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 3 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 3 (n.sid), FOLLOWING (my state)
- 2012-01-08 06:51:42,584 - INFO [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 3 (n.leader), 0 (n.zxid), 2 (n.round), LOOKING (n.state), 3 (n.sid), FOLLOWING (my state)
其他結點可能也出現類似問題,屬於正常。
第六步:安裝驗證
可以透過ZooKeeper的指令碼來檢視啟動狀態,包括叢集中各個結點的角色(或是Leader,或是Follower),如下所示,是在ZooKeeper叢集中的每個結點上查詢的結果:
- hadoop@slave-01:~/installation/zookeeper-3.3.4$ bin/zkServer.sh status
- JMX enabled by default
- Using config: /home/hadoop/installation/zookeeper-3.3.4/bin/../conf/zoo.cfg
- Mode: follower
- hadoop@slave-02:~/installation/zookeeper-3.3.4$ bin/zkServer.sh status
- JMX enabled by default
- Using config: /home/hadoop/installation/zookeeper-3.3.4/bin/../conf/zoo.cfg
- Mode: leader
- hadoop@slave-03:~/installation/zookeeper-3.3.4$ bin/zkServer.sh status
- JMX enabled by default
- Using config: /home/hadoop/installation/zookeeper-3.3.4/bin/../conf/zoo.cfg
- Mode: follower
另外,可以透過客戶端指令碼,連線到ZooKeeper叢集上。對於客戶端來說,ZooKeeper是一個整體(ensemble),連線到ZooKeeper叢集實際上感覺在獨享整個叢集的服務,所以,你可以在任何一個結點上建立到服務叢集的連線,例如:
- hadoop@slave-03:~/installation/zookeeper-3.3.4$ bin/zkCli.sh -server slave-01:2181
- Connecting to slave-01:2181
- 2012-01-08 07:14:21,068 - INFO [main:Environment@97] - Client environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
- 2012-01-08 07:14:21,080 - INFO [main:Environment@97] - Client environment:host.name=slave-03
- 2012-01-08 07:14:21,085 - INFO [main:Environment@97] - Client environment:java.version=1.6.0_30
- 2012-01-08 07:14:21,089 - INFO [main:Environment@97] - Client environment:java.vendor=Sun Microsystems Inc.
- 2012-01-08 07:14:21,095 - INFO [main:Environment@97] - Client environment:java.home=/home/hadoop/installation/jdk1.6.0_30/jre
- 2012-01-08 07:14:21,104 - INFO [main:Environment@97] - Client environment:java.class.path=/home/hadoop/installation/zookeeper-3.3.4/bin/../build/classes:/home/hadoop/installation/zookeeper-3.3.4/bin/../build/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../zookeeper-3.3.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/log4j-1.2.15.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/jline-0.9.94.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-lang-2.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-collections-3.2.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-cli-1.1.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-tasks-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-core-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../src/java/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../conf:/home/hadoop/installation/jdk1.6.0_30/lib/*.jar:/home/hadoop/installation/jdk1.6.0_30/jre/lib/*.jar
- 2012-01-08 07:14:21,111 - INFO [main:Environment@97] - Client environment:java.library.path=/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
- 2012-01-08 07:14:21,116 - INFO [main:Environment@97] - Client environment:java.io.tmpdir=/tmp
-
2012-01-08 07:14:21,124 - INFO [main:Environment@97] - Client environment:java.compiler=
- 2012-01-08 07:14:21,169 - INFO [main:Environment@97] - Client environment:os.name=Linux
- 2012-01-08 07:14:21,175 - INFO [main:Environment@97] - Client environment:os.arch=i386
- 2012-01-08 07:14:21,177 - INFO [main:Environment@97] - Client environment:os.version=3.0.0-14-generic
- 2012-01-08 07:14:21,185 - INFO [main:Environment@97] - Client environment:user.name=hadoop
- 2012-01-08 07:14:21,188 - INFO [main:Environment@97] - Client environment:user.home=/home/hadoop
- 2012-01-08 07:14:21,190 - INFO [main:Environment@97] - Client environment:user.dir=/home/hadoop/installation/zookeeper-3.3.4
- 2012-01-08 07:14:21,197 - INFO [main:ZooKeeper@379] - Initiating client connection, connectString=slave-01:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@bf32c
- 2012-01-08 07:14:21,305 - INFO [main-SendThread():ClientCnxn$SendThread@1061] - Opening socket connection to server slave-01/192.168.0.179:2181
- Welcome to ZooKeeper!
- 2012-01-08 07:14:21,376 - INFO [main-SendThread(slave-01:2181):ClientCnxn$SendThread@950] - Socket connection established to slave-01/192.168.0.179:2181, initiating session
- JLine support is enabled
- [zk: slave-01:2181(CONNECTING) 0] 2012-01-08 07:14:21,872 - INFO [main-SendThread(slave-01:2181):ClientCnxn$SendThread@739] - Session establishment complete on server slave-01/192.168.0.179:2181, sessionid = 0x134bdcd6b730000, negotiated timeout = 30000
- WATCHER::
- WatchedEvent state:SyncConnected type:None path:null
- [zk: slave-01:2181(CONNECTED) 0] ls /
- [zookeeper]
總結說明
主機名與IP地址對映配置問題
啟動ZooKeeper叢集時,如果ZooKeeper叢集中slave-01結點的日誌出現如下錯誤:
- java.net.SocketTimeoutException
- at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:109)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
- at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
- at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
- 2012-01-08 06:37:46,026 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@697] - Notification time out: 6400
- 2012-01-08 06:37:57,431 - WARN [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 2 at election address slave-02/202.106.199.35:3888
- java.net.SocketTimeoutException
- at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:109)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
- at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
- at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
- at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
- 2012-01-08 06:38:02,442 - WARN [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 3 at election address slave-03/202.106.199.35:3888
很顯然,slave-01在啟動時連線叢集中其他結點(slave-02、slave-03)時,主機名對映的IP與我們實際配置的不一致,所以叢集中各個結點之間無法建立鏈路,整個ZooKeeper叢集啟動是失敗的。
上面錯誤日誌中slave-02/202.106.199.35:3888實際應該是slave-02/202.192.168.0.178:3888就對了,但是在進行域名解析的時候對映有問題,修改每個結點的/etc/hosts檔案,將ZooKeeper叢集中所有結點主機名到IP地址的對映配置上。
原文地址:http://www.blogjava.net/hello-yun/archive/2012/05/03/377250.html
來自 “ ITPUB部落格 ” ,連結:http://blog.itpub.net/29754888/viewspace-1247258/,如需轉載,請註明出處,否則將追究法律責任。
相關文章
- 【Zookeeper】zookeeper叢集安裝
- zookeeper 叢集安裝
- Zookeeper介紹與叢集安裝
- ZooKeeper叢集安裝和部署
- linux ZooKeeper叢集安裝Linux
- 完全分散式模式hadoop叢集安裝與配置分散式模式Hadoop
- 【Hadoop】 分散式Hadoop叢集安裝配置Hadoop分散式
- hadoop+hbase+zookeeper叢集安裝方法Hadoop
- centos7下zookeeper叢集安裝部署CentOS
- Hadoop3.0完全分散式叢集安裝部署Hadoop分散式
- Storm叢集安裝與部署ORM
- 大資料規整-10-zookeeper單機叢集安裝和使用大資料
- hadoop學習之hadoop完全分散式叢集安裝Hadoop分散式
- centOS安裝zookeeper3.4.6(分散式模式)CentOS分散式模式
- Hadoop 叢集安裝與配置Hadoop
- ZooKeeper分散式專題(一) -- zookeeper安裝以及介紹分散式
- Redis概述和單機、叢集安裝Redis
- 安裝配置 zookeeper (單機非叢集模式)模式
- 【redis叢集安裝】Redis
- kafka叢集安裝Kafka
- hbase叢集安裝
- StarRocks 叢集安裝
- cdh 叢集安裝
- elasticsearch叢集安裝+安全驗證+kibana安裝Elasticsearch
- mysql cluster 7.38 叢集安裝 rpm安裝方式MySql
- k8s叢集安裝-kubeadm安裝K8S
- 安裝Zookeeper和Kafka叢集Kafka
- Cassandra安裝及分散式叢集搭建分散式
- ElasticSearch 6.6.0叢集安裝Elasticsearch
- TiDB叢集安裝TiDB
- Elasticsearch 叢集安裝部署Elasticsearch
- 記HBase叢集安裝
- Hadoop叢集安裝Hadoop
- Oracle 11G RAC叢集安裝(3)——安裝OracleOracle
- Zookeeper單機安裝
- dubbo系列(一):linux安裝dubbo與zookeeper(單節點)Linux
- hadoop偽分散式叢集的安裝(不是單機版)Hadoop分散式
- Oracle 11G RAC叢集安裝(2)——安裝gridOracle