hadoop0.20.2配置 in linux(ubuntu)

配置ssh

建立金鑰，這裡p後面是空密碼，不推薦使用空密碼

ssh-keygen -t rsa -P ''

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

配置完後，執行一下 ssh localhost, 確認你的機器可以用 SSH 連線，並且連線時不需要手工輸入密碼

下載hadoop

wget http://mirror.bjtu.edu.cn/apache/hadoop/common/hadoop-0.20.2/hadoop-0.20.2.tar.gz

tar -xvf hadoop-0.20.2.tar.gz

增加環境變數

在/etc/environment 

HADOOP_HOME=目錄路徑

JAVA_HOME=jdk路徑

在/etc/profile中增加

export HADOOP_HOME=目錄路徑

export JAVA_HOME=jdk路徑

修改hadoop配置檔案

修改$HADOOP_HOME/conf/hadoop-env.sh

#export JAVA_HOME=

改為export JAVA_HOME=jdk路徑

設定HDFS目錄

mkdir /$HOME/tmp

chmod -R 777 /$HOME/tmp

修改/conf/core-site.xml檔案

增加

<property>
  <name>hadoop.tmp.dir</name>
  <value>/home/du/tmp</value>
  <description>A base for other temporary directories.</description>
</property>
<property>
  <name>fs.default.name</name>
  <value>hdfs://localhost:54310</value>
</property>

修改conf/mapred-site.xml

<property>
  <name>mapred.job.tracker</name>
  <value>localhost:54311</value>
  <description>The host and port that the MapReduce job tracker runs
  at.  If "local", then jobs are run in-process as a single map
  and reduce task.
  </description>
</property>

修改 conf/hdfs-site.xml

增加

<property>
  <name>dfs.replication</name>
  <value>1</value>
  <description>Default block replication.
  The actual number of replications can be specified when the file is created.
  The default is used if replication is not specified in create time.
  </description>
</property>

格式化檔案系統

hadoop namenode -format

啟動偽分散式

$HADOOP_HOME/bin/start-all.sh

檢查是否啟動成功

jps

18160 SecondaryNameNode
17777 NameNode
17970 DataNode
18477 Jps
18409 TaskTracker
18231 JobTracker

停止

$HADOOP_HOME/bin/stop-all.sh

hadoop web訪問介面

http://localhost:50030/ job tracker訪問

http://localhost:50060/  task tracker訪問

http://localhost:50070/ name node訪問