閱讀目錄
本文版權歸mephisto和部落格園共有,歡迎轉載,但須保留此段宣告,並給出原文連結,謝謝合作。
文章是哥(mephisto)寫的,SourceLink
序
上一篇,我們對Hadoop命令進行了簡略的列舉,但是Hadoop命令特多,還有一部分沒有列舉完,官網基本都是英文的,所以只能拙略的翻譯下,妄大家見諒。
下面,我們就開始對Hadoop命令中進行講解。
HDFS Commands
一:介紹
所有的HDFS命令通過bin/ HDFS指令碼呼叫。指定引數執行HDFS指令碼會列印所有命令的描述。
用法: hdfs [SHELL_OPTIONS] COMMAND [GENERIC_OPTIONS] [COMMAND_OPTIONS]
Hadoop有一個選項解析框架用於解析一般的選項和執行類。
COMMAND_OPTIONS Description --config
--loglevelThe common set of shell options. These are documented on the Commands Manual page. GENERIC_OPTIONS The common set of options supported by multiple commands. See the Hadoop Commands Manual for more information. COMMAND COMMAND_OPTIONS Various commands with their options are described in the following sections. The commands have been grouped into User Commands and Administration Commands.
User Commands
一:介紹
用於Hadoop叢集使用者命令。
二:classPath
列印Hadoop jar和需要的庫類路徑。
用法: hdfs classpath
三:dfs
執行在支援Hadoop檔案系統檔案系統的命令.The various COMMAND_OPTIONS can be found at File System Shell Guide.
用法: hdfs dfs [COMMAND [COMMAND_OPTIONS]]
四:fetchdt
從NameNode獲取授權令牌.See fetchdt for more info.
用法: hdfs fetchdt [--webservice <namenode_http_addr>] <path>
五:fsck
在HDFS檔案系統檢查工具.See fsck for more info.
用法:hdfs fsck <path>
[-list-corruptfileblocks |
[-move | -delete | -openforwrite]
[-files [-blocks [-locations | -racks]]]
[-includeSnapshots]六:getconf
從配置目錄中獲取配置資訊然後處理。
用法:hdfs getconf -namenodes
hdfs getconf -secondaryNameNodes
hdfs getconf -backupNodes
hdfs getconf -includeFile
hdfs getconf -excludeFile
hdfs getconf -nnRpcAddresses
hdfs getconf -confKey [key]七:groups
返回給定的一個或多個使用者組資訊。
用法: hdfs groups [username ...]
八:lsSnapshottableDir
獲得snapshottable目錄列表。當是超級使用者執行時,它返回所有的snapshottable目錄。否則它返回當前使用者所擁有的目錄。
用法: hdfs lsSnapshottableDir [-help]
九:jmxget
把一個服務的JMX資訊丟棄
用法: hdfs jmxget [-localVM ConnectorURL | -port port | -server mbeanserver | -service service]
十:oev
Hadoop離線編輯檢視器。
用法: hdfs oev [OPTIONS] -i INPUT_FILE -o OUTPUT_FILE
十一:oiv
用於檢視較新的映象檔案的Hadoop離線映象檢視器
用法: hdfs oiv [OPTIONS] -i INPUT_FILE
十二:oiv_legacy
老版本的Hadoop Hadoop離線映象檢視器。
用法: hdfs oiv_legacy [OPTIONS] -i INPUT_FILE -o OUTPUT_FILE
十三:snapshotDiff
確定HDFS的快照之間的差異。See the HDFS Snapshot Documentation for more information.
用法: hdfs snapshotDiff <path> <fromSnapshot> <toSnapshot>
十四:version
列印版本。
用法: hdfs version
Administration Commands
一:介紹
用於Hadoop叢集使用者命令。
二:balancer
執行群集平衡實用程式。管理員可以按Ctrl-C停止再平衡過程。See Balancer for more details.
用法:hdfs balancer
[-threshold <threshold>]
[-policy <policy>]
[-exclude [-f <hosts-file> | <comma-separated list of hosts>]]
[-include [-f <hosts-file> | <comma-separated list of hosts>]]
[-idleiterations <idleiterations>]三:crypto
See the HDFS Transparent Encryption Documentation for more information.
用法:hdfs crypto -createZone -keyName <keyName> -path <path>
hdfs crypto -help <command-name>
hdfs crypto -listZones四:datanode
執行一個HDFS datanode
用法: hdfs datanode [-regular | -rollback | -rollingupgrace rollback]
五:dfsadmin
執行一個HDFS dfsadmin客戶端
用法:hdfs dfsadmin [GENERIC_OPTIONS]
[-report [-live] [-dead] [-decommissioning]]
[-safemode enter | leave | get | wait]
[-saveNamespace]
[-rollEdits]
[-restoreFailedStorage true |false |check]
[-refreshNodes]
[-setQuota <quota> <dirname>...<dirname>]
[-clrQuota <dirname>...<dirname>]
[-setSpaceQuota <quota> <dirname>...<dirname>]
[-clrSpaceQuota <dirname>...<dirname>]
[-setStoragePolicy <path> <policyName>]
[-getStoragePolicy <path>]
[-finalizeUpgrade]
[-rollingUpgrade [<query> |<prepare> |<finalize>]]
[-metasave filename]
[-refreshServiceAcl]
[-refreshUserToGroupsMappings]
[-refreshSuperUserGroupsConfiguration]
[-refreshCallQueue]
[-refresh <host:ipc_port> <key> [arg1..argn]]
[-reconfig <datanode |...> <host:ipc_port> <start |status>]
[-printTopology]
[-refreshNamenodes datanodehost:port]
[-deleteBlockPool datanode-host:port blockpoolId [force]]
[-setBalancerBandwidth <bandwidth in bytes per second>]
[-allowSnapshot <snapshotDir>]
[-disallowSnapshot <snapshotDir>]
[-fetchImage <local directory>]
[-shutdownDatanode <datanode_host:ipc_port> [upgrade]]
[-getDatanodeInfo <datanode_host:ipc_port>]
[-triggerBlockReport [-incremental] <datanode_host:ipc_port>]
[-help [cmd]]六:haadmin
See HDFS HA with NFS or HDFS HA with QJM for more information on this command.
用法:hdfs haadmin -checkHealth <serviceId>
hdfs haadmin -failover [--forcefence] [--forceactive] <serviceId> <serviceId>
hdfs haadmin -getServiceState <serviceId>
hdfs haadmin -help <command>
hdfs haadmin -transitionToActive <serviceId> [--forceactive]
hdfs haadmin -transitionToStandby <serviceId>七:journalnode
This comamnd starts a journalnode for use with HDFS HA with QJM.
用法: hdfs journalnode
八:mover
執行資料遷移實用程式. See Mover for more details.
用法: hdfs mover [-p <files/dirs> | -f <local file name>]
九:namenode
執行namenode. More info about the upgrade, rollback and finalize is at Upgrade Rollback.
用法:hdfs namenode [-backup] |
[-checkpoint] |
[-format [-clusterid cid ] [-force] [-nonInteractive] ] |
[-upgrade [-clusterid cid] [-renameReserved<k-v pairs>] ] |
[-upgradeOnly [-clusterid cid] [-renameReserved<k-v pairs>] ] |
[-rollback] |
[-rollingUpgrade <downgrade |rollback> ] |
[-finalize] |
[-importCheckpoint] |
[-initializeSharedEdits] |
[-bootstrapStandby] |
[-recover [-force] ] |
[-metadataVersion ]十:nfs3
該指令從HDFS nfs3服務使用nfs3閘道器。
用法: hdfs nfs3
十一:portmap
該指令從HDFS nfs3服務使用RPC portmap。
用法:Usage: hdfs portmap
十二:secondarynamenode
執行 second namenode.See Secondary Namenode for more info.
用法: hdfs secondarynamenode [-checkpoint [force]] | [-format] | [-geteditsize]
十三:storagepolicies
列出所有儲存策。See the HDFS Storage Policy Documentation for more information.
十四:zkfc
這個指令開始一個管理員切換控制器的過程使用HDFS HA QJM。
用法: hdfs zkfc [-formatZK [-force] [-nonInteractive]]
Debug Commands
一:介紹
為了幫助管理員除錯HDFS問題有用的命令,如驗證塊檔案和呼叫recoverlease。
二:verify
HDFS的後設資料和檔案塊的驗證。如果一個塊指定檔案,我們將驗證在後設資料檔案的校驗和匹配塊檔案。
用法: hdfs debug verify [-meta <metadata-file>] [-block <block-file>]
三:recoverLease
恢復指定路徑上的租約。路徑必須駐留在一個HDFS檔案系統。重試的預設號碼是1。
用法: hdfs debug recoverLease [-path <path>] [-retries <num-retries>]
--------------------------------------------------------------------
到此,本章節的內容講述完畢。
引用
Apache hadoop commands:http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html
Apache 1.04 中文:http://hadoop.apache.org/docs/r1.0.4/cn/commands_manual.html
系列索引
本文版權歸mephisto和部落格園共有,歡迎轉載,但須保留此段宣告,並給出原文連結,謝謝合作。
文章是哥(mephisto)寫的,SourceLink