關於64位Linux編譯hadoop2

Apache官方提供hadoop2的安裝包是在32位機器下編譯的，生產環境一般使用64的Linux，那麼需要在64位機器下重新編譯

可以檢視hadoop-2.2.0-src下的BUILDING.txt

Build instructions for Hadoop

———————————————————————————-

Requirements:

* Unix System

* JDK 1.6+

* Maven 3.0 or later

* Findbugs 1.3.9 (if running findbugs)

* ProtocolBuffer 2.5.0

* CMake 2.6 or newer (if compiling native code)

* Internet connection for first build (to fetch all Maven and Hadoop dependencies)

———————————————————————————-

…

步驟：

1.安裝JDK 1.6+ (驗證:java -version) 

2.安裝Maven 3.0 or later (驗證:mvn -version)

3.ProtocolBuffer 2.5.0 (驗證:protoc –version)下載地址：https://protobuf.googlecode.com/files/protobuf-2.5.0.tar.gz

#為了編譯安裝protobuf，Linux需要上網，使用YUM線上安裝依賴。如不能上網，就比較麻煩，需要一一下載每個依賴包再安裝。

sudo yum install gcc 

sudo yum install gcc-c++ 

sudo yum install make

#解壓protobuf

sudo tar -zxvf protobuf-2.5.0.tar.gz

#進入到protobuf-2.5.0

cd protobuf-2.5.0

#編譯安裝

sudo ./configure

sudo make

sudo make install

4.安裝CMake 2.6 or newer

sudo yum install cmake

sudo yum install openssl-devel

sudo yum install ncurses-devel

5.編譯hadoop-2.2.0

#解壓hadoop-2.2.0-src.tar.gz

tar -zxvf hadoop-2.2.0-src.tar.gz

#進入到hadoop-2.2.0-src

cd hadoop-2.2.0-src


#hadoop-2.2.0-src/hadoop-common-project/hadoop-auth/pom.xml有個bug，這裡需要修改一下

vim hadoop-common-project/hadoop-auth/pom.xml

#在<dependencies>標籤內新增如下內容

<dependency>

<groupId>org.mortbay.jetty</groupId>

<artifactId>jetty-util</artifactId>

<scope>test</scope>

</dependency>


#編譯

mvn package -DskipTests -Pdist,native


#編譯好的hadoop-2.2.0再hadoop-2.2.0-src/hadoop-dist/target目錄下


[INFO] ————————————————————————

[INFO] Reactor Summary:

[INFO] 

[INFO] Apache Hadoop Main ………………………….. SUCCESS [ 1.228 s]

[INFO] Apache Hadoop Project POM ……………………. SUCCESS [ 0.894 s]

[INFO] Apache Hadoop Annotations ……………………. SUCCESS [ 1.809 s]

[INFO] Apache Hadoop Assemblies …………………….. SUCCESS [ 0.222 s]

[INFO] Apache Hadoop Project Dist POM ……………….. SUCCESS [ 1.198 s]

[INFO] Apache Hadoop Maven Plugins ………………….. SUCCESS [ 2.205 s]

[INFO] Apache Hadoop Auth ………………………….. SUCCESS [ 2.169 s]

[INFO] Apache Hadoop Auth Examples ………………….. SUCCESS [ 1.583 s]

[INFO] Apache Hadoop Common ………………………… SUCCESS [01:02 min]

[INFO] Apache Hadoop NFS …………………………… SUCCESS [ 5.132 s]

[INFO] Apache Hadoop Common Project …………………. SUCCESS [ 0.038 s]

[INFO] Apache Hadoop HDFS ………………………….. SUCCESS [01:02 min]

[INFO] Apache Hadoop HttpFS ………………………… SUCCESS [ 9.002 s]

[INFO] Apache Hadoop HDFS BookKeeper Journal …………. SUCCESS [ 4.995 s]

[INFO] Apache Hadoop HDFS-NFS ………………………. SUCCESS [ 2.647 s]

[INFO] Apache Hadoop HDFS Project …………………… SUCCESS [ 0.058 s]

[INFO] hadoop-yarn ………………………………… SUCCESS [ 0.138 s]

[INFO] hadoop-yarn-api …………………………….. SUCCESS [ 31.854 s]

[INFO] hadoop-yarn-common ………………………….. SUCCESS [ 20.121 s]

[INFO] hadoop-yarn-server ………………………….. SUCCESS [ 0.105 s]

[INFO] hadoop-yarn-server-common ……………………. SUCCESS [ 5.776 s]

[INFO] hadoop-yarn-server-nodemanager ……………….. SUCCESS [ 10.490 s]

[INFO] hadoop-yarn-server-web-proxy …………………. SUCCESS [ 3.321 s]

[INFO] hadoop-yarn-server-resourcemanager ……………. SUCCESS [ 8.311 s]

[INFO] hadoop-yarn-server-tests …………………….. SUCCESS [ 0.510 s]

[INFO] hadoop-yarn-client ………………………….. SUCCESS [ 3.929 s]

[INFO] hadoop-yarn-applications …………………….. SUCCESS [ 0.060 s]

[INFO] hadoop-yarn-applications-distributedshell ……… SUCCESS [ 1.720 s]

[INFO] hadoop-mapreduce-client ……………………… SUCCESS [ 0.062 s]

[INFO] hadoop-mapreduce-client-core …………………. SUCCESS [ 16.204 s]

[INFO] hadoop-yarn-applications-unmanaged-am-launcher …. SUCCESS [ 1.779 s]

[INFO] hadoop-yarn-site ……………………………. SUCCESS [ 0.111 s]

[INFO] hadoop-yarn-project …………………………. SUCCESS [ 2.287 s]

[INFO] hadoop-mapreduce-client-common ……………….. SUCCESS [ 11.855 s]

[INFO] hadoop-mapreduce-client-shuffle ………………. SUCCESS [ 2.560 s]

[INFO] hadoop-mapreduce-client-app ………………….. SUCCESS [ 6.985 s]

[INFO] hadoop-mapreduce-client-hs …………………… SUCCESS [ 3.319 s]

[INFO] hadoop-mapreduce-client-jobclient …………….. SUCCESS [ 4.021 s]

[INFO] hadoop-mapreduce-client-hs-plugins ……………. SUCCESS [ 1.508 s]

[INFO] Apache Hadoop MapReduce Examples ……………… SUCCESS [ 4.176 s]

[INFO] hadoop-mapreduce ……………………………. SUCCESS [ 2.367 s]

[INFO] Apache Hadoop MapReduce Streaming …………….. SUCCESS [ 2.902 s]

[INFO] Apache Hadoop Distributed Copy ……………….. SUCCESS [ 5.365 s]

[INFO] Apache Hadoop Archives ………………………. SUCCESS [ 1.673 s]

[INFO] Apache Hadoop Rumen …………………………. SUCCESS [ 4.095 s]

[INFO] Apache Hadoop Gridmix ……………………….. SUCCESS [ 2.962 s]

[INFO] Apache Hadoop Data Join ……………………… SUCCESS [ 2.089 s]

[INFO] Apache Hadoop Extras ………………………… SUCCESS [ 2.190 s]

[INFO] Apache Hadoop Pipes …………………………. SUCCESS [ 5.887 s]

[INFO] Apache Hadoop Tools Dist …………………….. SUCCESS [ 1.149 s]

[INFO] Apache Hadoop Tools …………………………. SUCCESS [ 0.028 s]

[INFO] Apache Hadoop Distribution …………………… SUCCESS [ 6.514 s]

[INFO] Apache Hadoop Client ………………………… SUCCESS [ 2.199 s]

[INFO] Apache Hadoop Mini-Cluster …………………… SUCCESS [ 0.121 s]

[INFO] ————————————————————————

[INFO] BUILD SUCCESS (####看到BUILD SUCCESS說明編譯成功####)

[INFO] ————————————————————————

[INFO] Total time: 05:39 min

[INFO] Finished at: 2014-05-23T13:37:06+08:00

[INFO] Final Memory: 135M/300M

[INFO] ————————————————————————

本文轉自SummerChill部落格園部落格，原文連結：http://www.cnblogs.com/DreamDrive/p/4546792.html，如需轉載請自行聯絡原作者