Hadoop2.7實戰v1.0之HDFS HA

HDFS HA實戰v1.0

當前環境:hadoop+zookeeper(namenode,resourcemanager HA)

namenode	serviceId	Init status
sht-sgmhadoopnn-01	nn1	active
sht-sgmhadoopnn-02	nn2	standby

參考: http://blog.csdn.net/u011414200/article/details/50336735

一.檢視namenode是active還是standby

1.開啟網頁

2.檢視zkfc日誌 

 3. 透過命令hdfs haadmin –getServiceState    

###$HADOOP_HOME/etc/hadoop/hdfs-site.xml, dfs.ha.namenodes.[dfs.nameservices]

               

                                <!--設定NameNode IDs 此版本最大隻支援兩個NameNode -->

                                dfs.ha.namenodes.mycluster

                                nn1,nn2

               

 

 

二.基本命令

點選(此處)摺疊或開啟

1. [root@sht-sgmhadoopnn-02 logs]# hdfs --help
   
2. 
   
3. Usage: hdfs [--config confdir] [--loglevel loglevel] COMMAND
   
4. 
   
5.        where COMMAND is one of:
   
6. 
   
7.   dfs run a filesystem command on the file systems supported in Hadoop.
   
8. 
   
9.   classpath prints the classpath
   
10. 
    
11.   namenode -format format the DFS filesystem
    
12. 
    
13.   secondarynamenode run the DFS secondary namenode
    
14. 
    
15.   namenode run the DFS namenode
    
16. 
    
17.   journalnode run the DFS journalnode
    
18. 
    
19.   zkfc run the ZK Failover Controller daemon
    
20. 
    
21.   datanode run a DFS datanode
    
22. 
    
23.   dfsadmin run a DFS admin client
    
24. 
    
25.   haadmin run a DFS HA admin client
    
26. 
    
27.   fsck run a DFS filesystem checking utility
    
28. 
    
29.   balancer run a cluster balancing utility
    
30. 
    
31.   jmxget get JMX exported values from NameNode or DataNode.
    
32. 
    
33.   mover run a utility to move block replicas across
    
34. 
    
35.                                storage types
    
36. 
    
37.   oiv apply the offline fsimage viewer to an fsimage
    
38. 
    
39.   oiv_legacy apply the offline fsimage viewer to an legacy fsimage
    
40. 
    
41.   oev apply the offline edits viewer to an edits file
    
42. 
    
43.   fetchdt fetch a delegation token from the NameNode
    
44. 
    
45.   getconf get config values from configuration
    
46. 
    
47.   groups get the groups which users belong to
    
48. 
    
49.   snapshotDiff diff two snapshots of a directory or diff the
    
50. 
    
51.                        current directory contents with a snapshot
    
52. 
    
53.   lsSnapshottableDir list all snapshottable dirs owned by the current user
    
54. 
    
55.                                                 Use -help to see options
    
56. 
    
57.   portmap run a portmap service
    
58. 
    
59.   nfs3 run an NFS version 3 gateway
    
60. 
    
61.   cacheadmin configure the HDFS cache
    
62. 
    
63.   crypto configure HDFS encryption zones
    
64. 
    
65.   storagepolicies list/get/set block storage policies
    
66. 
    
67.   version print the version
    
68. 
    
69. ###########################################################################
    
70. 
    
71. [root@sht-sgmhadoopnn-02 logs]# hdfs namenode --help
    
72. 
    
73. Usage: java NameNode [-backup] |
    
74. 
    
75.         [-checkpoint] |
    
76. 
    
77.         [-format [-clusterid cid ] [-force] [-nonInteractive] ] |
    
78. 
    
79.         [-upgrade [-clusterid cid] [-renameReserved<k-v pairs>] ] |
    
80. 
    
81.         [-upgradeOnly [-clusterid cid] [-renameReserved<k-v pairs>] ] |
    
82. 
    
83.         [-rollback] |
    
84. 
    
85.         [-rollingUpgrade <rollback|downgrade|started> ] |
    
86. 
    
87.         [-finalize] |
    
88. 
    
89.         [-importCheckpoint] |
    
90. 
    
91.         [-initializeSharedEdits] |
    
92. 
    
93.         [-bootstrapStandby] |
    
94. 
    
95.         [-recover [ -force] ] |
    
96. 
    
97.         [-metadataVersion ] ]
    
98. 
    
99. 
    
100. 
     
101. 
     
102. ###########################################################################
     
103. 
     
104. [root@sht-sgmhadoopnn-02 logs]# hdfs haadmin --help
     
105. 
     
106. -help: Unknown command
     
107. 
     
108. Usage: haadmin
     
109. 
     
110.     [-transitionToActive [--forceactive] <serviceId>]
     
111. 
     
112.     [-transitionToStandby <serviceId>]
     
113. 
     
114.     [-failover [--forcefence] [--forceactive] <serviceId> <serviceId>]
     
115. 
     
116.     [-getServiceState <serviceId>]
     
117. 
     
118.     [-checkHealth <serviceId>]
     
119. 
     
120.     [-help <command>]

transitionToActive和transitionToStandby是用於在不同狀態之間切換的。 

failover 初始化一個故障恢復。該命令會從一個失效的NameNode切換到另一個上面(不支援在自動切換的環境操作)。

getServiceState 獲取當前NameNode的狀態。

checkHealth 檢查NameNode的狀態。正常就返回0，否則返回非0值。  

 

三.實驗

1.測試HDFS的手工切換功能(失敗)

#hdfs-site.xml 中設定dfs.ha.automatic-failover.enabled為 true,故提示不能手動切換

 

2.測試HDFS的自動切換功能(成功)

   a.在active namenode機器上透過jps命令查詢到namenode的程式號，然後透過kill -9的方式殺掉程式，觀察另一個namenode節點是否會從狀態standby變成active狀態

##也可以透過觀察zkfc日誌或者 來檢視是否切換成功
 

   b.手動啟動sht-sgmhadoopnn-01的namenode 

#### sht-sgmhadoopnn-01 機器上namenode 程式已經起來,且狀態為standby

   c. 再次切換