在HDFS中,提供了fsck命令,用於檢查HDFS上檔案和目錄的健康狀態、獲取檔案的block塊資訊和位置資訊等。
具體命令介紹:
-move: 移動損壞的檔案到/lost+found目錄下
-delete: 刪除損壞的檔案
-openforwrite: 輸出檢測中的正在被寫的檔案
-list-corruptfileblocks: 輸出損壞的塊及其所屬的檔案
-files: 輸出正在被檢測的檔案
-blocks: 輸出block的詳細報告 (需要和-files引數一起使用)
-locations: 輸出block的位置資訊 (需要和-files引數一起使用)
-racks: 輸出檔案塊位置所在的機架資訊(需要和-files引數一起使用)
例如要檢視HDFS中某個檔案的block塊的具體分佈,可以這樣寫:
hadoop fsck /your_file_path -files -blocks -locations -racks
具體例項:
#hdfs fsck /tmp/test/input/word.log -files -blocks -locations -racks
Connecting to namenode via http://c12-138:50070
FSCK started by root (auth:SIMPLE) from /10.254.13.141 for path /tmp/test/input/word.log at Wed Aug 10 17:17:45 CST 2016
/tmp/test/input/word.log 70 bytes, 1 block(s): OK
0. BP-1390896613-10.254.12.138-1467366342106:blk_1074295976_555178 len=70 Live_repl=3 [/default/10.254.13.141:50010, /default/10.254.12.139:50010, /default/10.254.12.138:50010]
Status: HEALTHY
Total size: 70 B
Total dirs: 0
Total files: 1
Total symlinks: 0
Total blocks (validated): 1 (avg. block size 70 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 3
Average block replication: 3.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 10
Number of racks: 1
FSCK ended at Wed Aug 10 17:17:45 CST 2016 in 1 milliseconds
The filesystem under path '/tmp/test/input/word.log' is HEALTHY
其中:
0. 表示block個數;
BP-1390896613-10.254.12.138-1467366342106:blk_1074295976_555178表示block id;
len=
70 表示該檔案塊大小;
repl=3 表示該block塊的副本數;