hadoop2.2.0 搭建初體驗

下載 hadoop-2.2.0.tar.gz

tar -xzvf hadoop-2.2.0.tar.gz

cd hadoop-2.2.0

vi etc/hadoop/hadoop-env.sh

修改：

export JAVA_HOME=/usr/local/jdk1.7.0_15

vi etc/hadoop/slaves

修改localhost為hostmaster

vi etc/hadoop/core-site.xml

增加：

<property>

       <name>fs.defaultFS</name>

       <value>hdfs://hostmaster:9000</value>

</property>

cp etc/hadoop/mapred-site.xml.template etc/hadoop/mapred-site.xml

vi etc/hadoop/mapred-site.xml

增加：

<property>

       <name>mapreduce.framework.name</name>

       <value>yarn</value>

</property>

<property>

       <name>mapreduce.cluster.local.dir</name>

       <value>/home/ysc/mapreduce/local</value>

</property>

vi etc/hadoop/yarn-site.xml

增加：

<property>   

       <name>yarn.resourcemanager.hostname</name>  

       <value>host001</value>

</property>

<property>

       <name>yarn.nodemanager.aux-services</name> 

       <value>mapreduce_shuffle</value> 

</property>

vi etc/hadoop/hdfs-site.xml

增加：

<property>

       <name>dfs.name.dir</name>

       <value>你的目錄/dfs/filesystem/name</value>

</property>

<property>

       <name>dfs.data.dir</name>

       <value>你的目錄/dfs/filesystem/data</value>

</property>

<property>

       <name>dfs.replication</name>

       <value>3</value> <!--為分散式改為1-->

</property>

sudo vi /etc/profile

增加：

export HADOOP_PREFIX=/home/ysc/hadoop-2.2.0

export HADOOP_COMMON_HOME=/home/ysc/hadoop-2.2.0

export HADOOP_MAPRED_HOME=/home/ysc/hadoop-2.2.0

export HADOOP_CONF_DIR=/home/ysc/hadoop-2.2.0/etc/hadoop

export HADOOP_HDFS_HOME=/home/ysc/hadoop-2.2.0

export HADOOP_YARN_HOME=/home/ysc/hadoop-2.2.0

source /etc/profile

格式化：

bin/hdfs namenode -format

啟動dfs：

sbin/start-dfs.sh

啟動yarn：

sbin/start-yarn.sh

執行wordcount:

echo "Hello this is first hadoop2.2.0" > text1.txt

echo "start stop start stop" > text2.txt

bin/hadoop fs -mkdir input

bin/hadoop fs -put text1.txt input

bin/hadoop fs -put text2.txt input

bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar wordcount input output

訪問管理頁面

http://hostmaster:8088

         http://hostmaster:50070

停止dfs：

sbin/stop-dfs.sh

停止yarn：

sbin/stop-yarn.sh