Spark修煉之道（進階篇）——Spark入門到精通：第一節 Spark 1.5.0叢集搭建
五柳-先生發表於2015-11-14
本節主要內容
- 作業系統環境準備
- Hadoop 2.4.1叢集搭建
- Spark 1.5.0 叢集部署
注：在利用CentOS 6.5作業系統安裝spark 1.5叢集過程中，本人發現Hadoop 2.4.1叢集可以順利搭建，但在Spark 1.5.0叢集啟動時出現了問題（可能原因是64位作業系統原因，原始碼需要重新編譯，但本人沒經過測試），經本人測試在ubuntu 10.04 作業系統上可以順利成功搭建。大家可以利用CentOS 6.5進行嘗試，如果有問題，再利用ubuntu 10.04搭建，所有步驟基本一致
1. 作業系統環境準備
（1）安裝VMWare
下載地址：http://pan.baidu.com/s/1bniBipD
密碼：pbdw
安裝過程略
（2）下載作業系統並安裝
Ubuntu 10.04作業系統下載地址：
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">連結：http://pan<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.baidu</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>/s/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>kTy9Umj 密碼：<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>w5b</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
CentOS 6.5下載地址：
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">下載地址：http://pan<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.baidu</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>/s/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>mgkuKdi 密碼：xtm5</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>
本實驗要求裝三臺：CentOS 6.5，可以分別安裝，也可以安裝完一臺後克隆兩臺，具體過程略。初學者，建議三臺分別安裝。安裝後如下圖所示：
（3）CentOS 6.5網路配置
安裝好的虛擬機器一般預設使用的是NAT（關於NAT、橋接等虛擬機器網路連線方式參見本人部落格：http://blog.csdn.net/lovehuangjiaju/article/details/48183485），由於三臺機器之間需要互通之外，還需要與本機連通，因此採用將網路連線方式設定為Bridged（三臺機器相同的設定），如下圖所法：
修改主機名
（1）修改centos_salve01虛擬機器主機名：
<code class="hljs has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">vim /etc/sysconfig/network</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
/etc/sysconfig/network修改後的內容如下：
（2）vim /etc/sysconfig/network命令修改centos_slave02虛擬機器主機名
/etc/sysconfig/network修改後的內容如下：
（3）vim /etc/sysconfig/network命令修改centos_slave03虛擬機器主機名
/etc/sysconfig/network修改後的內容如下：
修改主機IP地址
在大家在配置時，修改/etc/sysconfig/network-scripts/ifcfg-eth0檔案對應的BOOTPROT=static、IPADDR、NETMASK、GATEWAY及DNS1資訊即可
（1）修改centos_salve01虛擬機器主機IP地址：
<code class="hljs lasso has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">vim /etc/sysconfig/network<span class="hljs-attribute" style="box-sizing: border-box;">-scripts</span>/ifcfg<span class="hljs-attribute" style="box-sizing: border-box;">-eth0</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
修改後內容如下：
<code class="hljs ini has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-setting" style="box-sizing: border-box;">DEVICE=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"eth0"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">BOOTPROTO=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"static"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">HWADDR=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"00:0c:29:3f:69:4d"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">IPV6INIT=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"yes"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">NM_CONTROLLED=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"yes"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">ONBOOT=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"yes"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">TYPE=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Ethernet"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">UUID=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"5315276c-db0d-4061-9c76-9ea86ba9758e"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">IPADDR=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"192.168.1.111"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">NETMASK=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"255.255.255.0"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">GATEWAY=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"192.168.1.1"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">DNS1=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"8.8.8.8"</span></span></span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li></ul>
（2）修改centos_salve02虛擬機器主機IP地址：
<code class="hljs lasso has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">vim /etc/sysconfig/network<span class="hljs-attribute" style="box-sizing: border-box;">-scripts</span>/ifcfg<span class="hljs-attribute" style="box-sizing: border-box;">-eth0</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
修改後內容如下：
<code class="hljs ini has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-setting" style="box-sizing: border-box;">DEVICE=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"eth0"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">BOOTPROTO=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"static"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">HWADDR=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"00:0c:29:64:f9:80"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">IPV6INIT=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"yes"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">NM_CONTROLLED=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"yes"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">ONBOOT=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"yes"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">TYPE=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Ethernet"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">UUID=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"5315276c-db0d-4061-9c76-9ea86ba9758e"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">IPADDR=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"192.168.1.112"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">NETMASK=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"255.255.255.0"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">GATEWAY=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"192.168.1.1"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">DNS1=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"8.8.8.8"</span></span></span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li></ul>
（3）修改centos_salve03虛擬機器主機IP地址：
<code class="hljs lasso has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">vim /etc/sysconfig/network<span class="hljs-attribute" style="box-sizing: border-box;">-scripts</span>/ifcfg<span class="hljs-attribute" style="box-sizing: border-box;">-eth0</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
修改後內容如下：
<code class="hljs ini has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-setting" style="box-sizing: border-box;">DEVICE=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"eth0"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">BOOTPROTO=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"static"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">HWADDR=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"00:0c:29:1e:80:b1"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">IPV6INIT=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"yes"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">NM_CONTROLLED=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"yes"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">ONBOOT=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"yes"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">TYPE=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Ethernet"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">UUID=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"5315276c-db0d-4061-9c76-9ea86ba9758e"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">IPADDR=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"192.168.1.113"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">NETMASK=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"255.255.255.0"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">GATEWAY=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"192.168.1.1"</span></span></span> <span class="hljs-setting" style="box-sizing: border-box;">DNS1=<span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"8.8.8.8"</span></span></span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li></ul>
/etc/sysconfig/network-scripts/ifcfg-eth0檔案內容解析：
<code class="hljs delphi has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">DEVICE=eth0 <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//指出裝置名稱</span> BOOTPROT=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">static</span> <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//啟動型別 dhcp|static，使用橋接模式，必須是static</span> HWADDR=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">06</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>B:FE:DF:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7</span>C <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//硬體Mac地址</span> IPADDR=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.2</span> <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//IP地址</span> NETMASK=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">255.255</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">255.0</span> <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//子網掩碼</span> NETWORK=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.0</span> <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//網路地址</span> GATEWAY=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.1</span> <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//閘道器地址</span> ONBOOT=yes <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//是否啟動應用</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">TYPE</span>=Ethernet <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//網路型別</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li></ul>
設定完成後，使用
<code class="hljs has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">service network restart</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
命令重新啟動網路，配置即可生效。
設定主機名與IP地址對映
（1）修改centos_salve01主機名與IP地址對映
<code class="hljs has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">vim /etc/hosts</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
設定內容如下：
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">127.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span> slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> localhost localhost<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.localdomain</span> localhost4 localhost4<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.localdomain</span>4 ::<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.111</span> slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.112</span> slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.113</span> slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> 具體如下圖：</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>
（2）修改centos_salve02主機名與IP地址對映
<code class="hljs has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">vim /etc/hosts</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
設定內容如下：
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">127.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span> slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> localhost localhost<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.localdomain</span> localhost4 localhost4<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.localdomain</span>4 ::<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.111</span> slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.112</span> slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.113</span> slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li></ul>
具體如下圖：
（3）修改centos_salve03主機名與IP地址對映
<code class="hljs has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">vim /etc/hosts</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
設定內容如下：
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">127.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span> slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> localhost localhost<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.localdomain</span> localhost4 localhost4<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.localdomain</span>4 ::<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.111</span> slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.112</span> slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.113</span> slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>
修改主機DNS
採用下列命令設定各主機DNS（三臺機器進行相同的設定)
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">vim /etc/resolv<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.conf</span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
設定後的內容：
<code class="hljs vala has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># Generated by NetworkManager</span> search example.com nameserver <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8.8</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.8</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.8</span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>
8.8.8.8為Google提供的DNS伺服器
網路連通測試
前面所有的配置完成後，重啟centos_salve01、centos_salve02、centos_salve03使主機名設定生效，然後分別在三臺機器上作如下測試命令：
下面只給出在centos_salve01虛擬機器上的測試
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root@slave01 ~]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># ping slave02.example.com</span> <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">PING</span> slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.112</span>) <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">84</span>) bytes of data. <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> bytes from slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.112</span>): icmp_seq=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> ttl=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> time=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.417</span> ms <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> bytes from slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.112</span>): icmp_seq=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> ttl=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> time=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.355</span> ms <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> bytes from slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.112</span>): icmp_seq=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span> ttl=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> time=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.363</span> ms ^C --- slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">ping</span> statistics --- <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span> packets transmitted, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span> received, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>% packet loss, time <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2719</span>ms rtt min/avg/max/mdev = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.355</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.378</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.417</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.031</span> ms [root@slave01 ~]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># ping slave03.example.com</span> <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">PING</span> slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.113</span>) <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">84</span>) bytes of data. <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> bytes from slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.113</span>): icmp_seq=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> ttl=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> time=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.386</span> ms <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> bytes from slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.113</span>): icmp_seq=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> ttl=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> time=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.281</span> ms ^C --- slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">ping</span> statistics --- <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> packets transmitted, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> received, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>% packet loss, time <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1799</span>ms rtt min/avg/max/mdev = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.281</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.333</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.386</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.055</span> ms </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li></ul>
測試外網的連通性（我在裝的時候，8.8.8.8，已經被禁用….心中一萬頭cnm）：
<code class="hljs vbnet has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root@slave01 ~]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># ping www.baidu.com</span> ping: unknown host www.baidu.com [root@slave01 ~]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># ping 8.8.8.8</span> PING <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8.8</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.8</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.8</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8.8</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.8</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.8</span>) <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">56</span>(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">84</span>) bytes <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">of</span> data. <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">From</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.111</span> icmp_seq=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> Destination Host Unreachable <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">From</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.111</span> icmp_seq=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span> Destination Host Unreachable <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">From</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.111</span> icmp_seq=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span> Destination Host Unreachable <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">From</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.111</span> icmp_seq=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span> Destination Host Unreachable <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">From</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.111</span> icmp_seq=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7</span> Destination Host Unreachable <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">From</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.111</span> icmp_seq=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span> Destination Host Unreachable </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li></ul>
（4）SSH完密碼登入
#### (1) OpenSSH安裝
如果大家在配置時，ping 8.8.8.8能夠ping通，則主機能夠正常上網；如果不能上網，則將網路連線方式重新設定為NAT，並修改網路配置檔案為dhcp方式。在保證網路連通的情況下執行下列命令：
<code class="hljs axapta has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">yum install openssh-<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">server</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
#### (2) 無密碼登入實現
使用以下命令生成相應的金鑰（三臺機器進行相同的操作）
<code class="hljs lasso has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">ssh<span class="hljs-attribute" style="box-sizing: border-box;">-keygen</span> <span class="hljs-attribute" style="box-sizing: border-box;">-t</span> rsa </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
執行過程一直回車即可
<code class="hljs vbnet has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root@slave01 ~]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># ssh-keygen -t rsa</span> Generating <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span>/<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">private</span> rsa <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">key</span> pair. Enter file <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> which <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">to</span> save the <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">key</span> (/root/.ssh/id_rsa): Enter passphrase (empty <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> no passphrase): Enter same passphrase again: Your identification has been saved <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> /root/.ssh/id_rsa. Your <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">key</span> has been saved <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> /root/.ssh/id_rsa.pub. The <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">key</span> fingerprint <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">is</span>: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span>e:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>f:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">39</span>:ed:f4:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>e:a3:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">55</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">62</span>:f5:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span>a:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>d:c5:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>c:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span> root@slave01.example.com The <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">key</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">'s randomart image is:</span> +--[ RSA <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2048</span>]----+ | E | | + | | o = | | . + . | | S . . | | + X . | | B * | | .o=o. | | .. +oo. | +-----------------+ </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li></ul>
生成的檔案分別為/root/.ssh/id_rsa（私鑰）、/root/.ssh/id_rsa.pub（公鑰）
完成後將公鑰拷貝到要免登陸的機器上（三臺可進行相同操作）：
<code class="hljs lasso has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">ssh<span class="hljs-attribute" style="box-sizing: border-box;">-copy</span><span class="hljs-attribute" style="box-sizing: border-box;">-id</span> <span class="hljs-attribute" style="box-sizing: border-box;">-i</span> slave01<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">.</span>example<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">.</span>com ssh<span class="hljs-attribute" style="box-sizing: border-box;">-copy</span><span class="hljs-attribute" style="box-sizing: border-box;">-id</span> <span class="hljs-attribute" style="box-sizing: border-box;">-i</span> slave02<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">.</span>example<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">.</span>com ssh<span class="hljs-attribute" style="box-sizing: border-box;">-copy</span><span class="hljs-attribute" style="box-sizing: border-box;">-id</span> <span class="hljs-attribute" style="box-sizing: border-box;">-i</span> slave03<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">.</span>example<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">.</span>com</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>
2. Hadoop 2.4.1叢集搭建
叢集搭建相關軟體下載地址：
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">連結：http://pan<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.baidu</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>/s/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>sjIG3b3 密碼：<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">38</span>gh</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
下載後將所有軟體都放置在E盤的share目錄下:
設定share資料夾為虛擬機器的共享目錄，如下圖所示：
在linux系統中，採用
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> /]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cd /mnt/hgfs/share</span> [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> share]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ls</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>
命令可以切換到該目錄下，如下圖
Spark官方要求的JDK、Scala版本
<code class="hljs applescript has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">Spark runs <span class="hljs-function_start" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">on</span></span> Java <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7</span>+, Python <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.6</span>+ <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">and</span> R <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3.1</span>+. For <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">the</span> Scala API, Spark <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.5</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span> uses Scala <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.10</span>. You will need <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">to</span> use a compatible Scala <span class="hljs-property" style="box-sizing: border-box;">version</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.10</span>.x).</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
（1）JDK 1.8 安裝
在根目錄下建立sparkLearning目前，後續所有相關軟體都放置在該目錄下，程式碼如下：
<code class="hljs coffeescript has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-property" style="box-sizing: border-box;">@slave01</span> /]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># mkdir /sparkLearning</span> [root<span class="hljs-property" style="box-sizing: border-box;">@slave01</span> /]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ls</span> bin etc lib media proc selinux sys <span class="hljs-reserved" style="box-sizing: border-box;">var</span> boot hadoopLearning lib64 mnt root sparkLearning tmp dev home lost+found opt sbin srv usr </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>
將共享目錄中的jdk安裝包複製到/sparkLearning目錄
<code class="hljs perl has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> share]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cp /mnt/hgfs/share/jdk-8u40-linux-x64.gz /sparkLearning/</span> [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> share]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cd /sparkLearning/</span> //解壓 [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># tar -zxvf jdk-8u40-linux-x64.gz </span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul>
設定環境變數：
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># vim /etc/profile</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
在檔案最後新增：
<code class="hljs bash has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> JAVA_HOME=/sparkLearning/jdk1.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8.0</span>_40
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> PATH=<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${JAVA_HOME}</span>/bin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">$PATH</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>
如下圖：
測試配置是否成功：
<code class="hljs d has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//使修改後的配置生效</span> [root<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">@slave01</span> sparkLearning]# source /etc/profile <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//環境變數是否已經設定</span> [root<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">@slave01</span> sparkLearning]# $JAVA_HOME bash: /sparkLearning/jdk1.8.0_40: <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">is</span> a directory <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//測試java是否安裝配置成功</span> [root<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">@slave01</span> sparkLearning]# java -<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">version</span> java <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">version</span> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"1.8.0_40"</span> Java(TM) SE Runtime Environment (build <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.8</span>.0_40-b25) Java HotSpot(TM) <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span>-Bit Server VM (build <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">25.40</span>-b25, mixed mode) </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li></ul>
（2）Scala 2.10.4 安裝
<code class="hljs java has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//複製檔案到sparkLearning目錄下</span> [root<span class="hljs-annotation" style="color: rgb(155, 133, 157); box-sizing: border-box;">@slave</span>01 sparkLearning]# cp /mnt/hgfs/share/scala-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.10</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>.tgz . <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//解壓</span> [root<span class="hljs-annotation" style="color: rgb(155, 133, 157); box-sizing: border-box;">@slave</span>01 sparkLearning]# tar -zxvf scala-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.10</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>.tgz > /dev/<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">null</span> [root<span class="hljs-annotation" style="color: rgb(155, 133, 157); box-sizing: border-box;">@slave</span>01 sparkLearning]# vim /etc/profile</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li></ul>
將/etc/profile檔案末尾內容修改如下：
<code class="hljs bash has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> JAVA_HOME=/sparkLearning/jdk1.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8.0</span>_40
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> SCALA_HOME=/sparkLearning/scala-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.10</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> PATH=<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${JAVA_HOME}</span>/bin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${SCALA_HOME}</span>/bin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">$PATH</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>
測試Scala是否安裝成功
<code class="hljs coffeescript has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-property" style="box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># source /etc/profile</span> [root<span class="hljs-property" style="box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># $SCALA_HOME</span> <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">bash</span>: /sparkLearning/scala-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.10</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>: <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">is</span> a directory [root<span class="hljs-property" style="box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># scala -version</span> Scala code runner version <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.10</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span> -- Copyright <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2002</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2013</span>, LAMP/EPFL </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>
（3）Zookeeper-3.4.5 叢集搭建
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cp /mnt/hgfs/share/zookeeper-3.4.5.tar.gz .</span> [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># tar -zxvf zookeeper-3.4.5.tar.gz > /dev/null</span> [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cp zookeeper-3.4.5/conf/zoo_sample.cfg zoo.cfg</span> [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># vim zookeeper-3.4.5/conf/zoo_sample.cfg zoo.cfg</span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>
修改dataDir為：
<code class="hljs fix has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-attribute" style="box-sizing: border-box;">dataDir</span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">/sparkLearning/zookeeper-3.4.5/zookeeper_data </span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>
在檔案末尾新增如下內容：
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">server<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>=slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2888</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3888</span> server<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.2</span>=slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2888</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3888</span> server<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.3</span>=slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2888</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3888</span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>
如圖所示：
建立ZooKeeper叢集資料儲存目錄
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cd zookeeper-3.4.5/</span> [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> zookeeper-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3.4</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># mkdir zookeeper_data</span> [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> zookeeper-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3.4</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cd zookeeper_data/</span> [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> zookeeper_data]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># touch myid </span> [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> zookeeper_data]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># echo 1 > myid </span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>
將slave01.example.com（centos_slave01）上的sparkLearning目錄拷貝到另外兩臺伺服器上：
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> /]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># scp -r /sparkLearning slave02.example.com:/</span> [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> /]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># scp -r /sparkLearning slave03.example.com:/</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>
/etc/profile檔案也進行覆蓋
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> /]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># scp /etc/profile slave02.example.com:/etc/profile</span> [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> /]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># scp /etc/profile slave03.example.com:/etc/profile</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>
修改zookeeper_data中的myid資訊：
<code class="hljs perl has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">//配置slave02.example.com上的myid [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> /]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ssh salve02.example.com</span> [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave02</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># echo 2 > /sparkLearning/zookeeper-3.4.5/zookeeper_data/myid</span> [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave02</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># more /sparkLearning/zookeeper-3.4.5/zookeeper_data/myid</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> //配置slave03.example.com上的myid [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave02</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ssh slave03.example.com</span> Last login: Fri Sep <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">18</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">01</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">33</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">29</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2015</span> from slave01.example.com [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave03</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># echo 3 > /sparkLearning/zookeeper-3.4.5/zookeeper_data/myid</span> [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave03</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># more /sparkLearning/zookeeper-3.4.5/zookeeper_data/myid</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li></ul>
如此便完成配置，下面對叢集進行測試：
<code class="hljs coffeescript has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">//</span>在slave03.example.com主機上 [root<span class="hljs-property" style="box-sizing: border-box;">@slave03</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cd /sparkLearning/zookeeper-3.4.5/bin</span> [root<span class="hljs-property" style="box-sizing: border-box;">@slave03</span> bin]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ls</span> README.txt zkCli.cmd zkEnv.cmd zkServer.cmd zkCleanup.sh zkCli.sh zkEnv.sh zkServer.sh <span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">//</span>啟動slave03.example.com上的ZooKeeper [root<span class="hljs-property" style="box-sizing: border-box;">@slave03</span> bin]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ./zkServer.sh start</span> JMX enabled <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">by</span> <span class="hljs-reserved" style="box-sizing: border-box;">default</span> Using <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">config</span>: <span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">/sparkLearning/zookeeper-3.4.5/bin/</span>../conf/zoo.cfg Starting zookeeper ... STARTED [root<span class="hljs-property" style="box-sizing: border-box;">@slave03</span> bin]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ./zkServer.sh status</span> JMX enabled <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">by</span> <span class="hljs-reserved" style="box-sizing: border-box;">default</span> Using <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">config</span>: <span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">/sparkLearning/zookeeper-3.4.5/bin/</span>../conf/zoo.cfg Error contacting service. It <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">is</span> probably <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">not</span> running. <span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">//</span>在slave02.example.com主機上 [root<span class="hljs-property" style="box-sizing: border-box;">@slave02</span> bin]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ./zkServer.sh start</span> JMX enabled <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">by</span> <span class="hljs-reserved" style="box-sizing: border-box;">default</span> Using <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">config</span>: <span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">/sparkLearning/zookeeper-3.4.5/bin/</span>../conf/zoo.cfg Starting zookeeper ... STARTED <span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">//</span>檢視zookeeper叢集狀態，如果Mode顯示為follower或leader則表明配置成功 [root<span class="hljs-property" style="box-sizing: border-box;">@slave02</span> bin]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ./zkServer.sh status</span> JMX enabled <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">by</span> <span class="hljs-reserved" style="box-sizing: border-box;">default</span> Using <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">config</span>: <span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">/sparkLearning/zookeeper-3.4.5/bin/</span>../conf/zoo.cfg <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">Mode</span>: follower <span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">//</span>在slave01.example.com主機上 [root<span class="hljs-property" style="box-sizing: border-box;">@slave01</span> bin]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ./zkServer.sh start</span> JMX enabled <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">by</span> <span class="hljs-reserved" style="box-sizing: border-box;">default</span> Using <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">config</span>: <span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">/sparkLearning/zookeeper-3.4.5/bin/</span>../conf/zoo.cfg Starting zookeeper ... STARTED [root<span class="hljs-property" style="box-sizing: border-box;">@slave01</span> bin]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ./zkServer.sh status</span> JMX enabled <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">by</span> <span class="hljs-reserved" style="box-sizing: border-box;">default</span> Using <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">config</span>: <span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">/sparkLearning/zookeeper-3.4.5/bin/</span>../conf/zoo.cfg <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">Mode</span>: follower <span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">//</span>在slave03.example.com主機上zookeeper狀態 [root<span class="hljs-property" style="box-sizing: border-box;">@slave03</span> bin]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ./zkServer.sh status</span> JMX enabled <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">by</span> <span class="hljs-reserved" style="box-sizing: border-box;">default</span> Using <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">config</span>: <span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">/sparkLearning/zookeeper-3.4.5/bin/</span>../conf/zoo.cfg <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">Mode</span>: leader </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li></ul>
（4）Hadoop 2.4.1 叢集搭建
（1）Hadoop 2.4.1基本目錄瀏覽
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root@slave01 bin]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># cp /mnt/hgfs/share/hadoop-2.4.1.tar.gz /sparkLearning/</span> [root@slave01 bin]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># cd /sparkLearning/</span> [root@slave01 sparkLearning]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># tar -zxvf hadoop-2.4.1.tar.gz > /dev/null</span> [root@slave01 sparkLearning]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># cd hadoop-2.4.1</span> [root@slave01 hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># ls</span> bin include libexec NOTICE<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.txt</span> sbin etc lib LICENSE<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.txt</span> README<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.txt</span> share cd [root@slave01 hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># cd etc/hadoop/</span> [root@slave01 hadoop]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># ls</span> capacity-scheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xml</span> hdfs-site<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xml</span> mapred-site<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xml</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.template</span> configuration<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xsl</span> httpfs-env<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.sh</span> slaves container-executor<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.cfg</span> httpfs-log4j<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.properties</span> ssl-client<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xml</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span> core-site<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xml</span> httpfs-signature<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.secret</span> ssl-server<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xml</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span> hadoop-env<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.cmd</span> httpfs-site<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xml</span> yarn-env<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.cmd</span> hadoop-env<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.sh</span> log4j<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.properties</span> yarn-env<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.sh</span> hadoop-metrics2<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.properties</span> mapred-env<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.cmd</span> yarn-site<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xml</span> hadoop-metrics<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.properties</span> mapred-env<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.sh</span> hadoop-policy<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xml</span> mapred-queues<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xml</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.template</span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li></ul>
（2）將Hadoop 2.4.1新增到環境變數
使用命令：vim /etc/profile 將環境變數資訊修改如下：
<code class="hljs bash has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> JAVA_HOME=/sparkLearning/jdk1.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8.0</span>_40
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> SCALA_HOME=/sparkLearning/scala-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.10</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> HADOOP_HOME=/sparkLearning/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> PATH=<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${JAVA_HOME}</span>/bin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${SCALA_HOME}</span>/bin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${HADOOP_HOME}</span>/bin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${HADOOP_HOME}</span>/sbin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">$PATH</span>
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul>
（3）將Hadoop 2.4.1新增到環境變數
使用命令：vim hadoop-env.sh 將環境變數資訊修改如下，在export JAVA_HOME修改為：
<code class="hljs bash has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> JAVA_HOME=/sparkLearning/jdk1.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8.0</span>_40</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
（4）修改core-site.xml檔案
利用vim core-site.xml命令，檔案內容如下：
<code class="hljs xml has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"> <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 指定hdfs的nameservice為ns1 --></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>fs.defaultFS<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>hdfs://ns1<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 指定hadoop臨時目錄 --></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>hadoop.tmp.dir<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>/sparkLearning/hadoop-2.4.1/tmp<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 指定zookeeper地址 --></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>ha.zookeeper.quorum<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>slave01.example.com:2181,slave02.example.com:2181,slave03.example.com:2181<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li></ul>
（5）修改hdfs-site.xml檔案
vim hdfs-site.xml內容如下：
<code class="hljs xml has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!--指定hdfs的nameservice為ns1，需要和core-site.xml中的保持一致 --></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.nameservices<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>ns1<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- ns1下面有兩個NameNode，分別是nn1，nn2 --></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.ha.namenodes.ns1<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>nn1,nn2<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- nn1的RPC通訊地址 --></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.namenode.rpc-address.ns1.nn1<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>slave01.example.com:9000<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- nn1的http通訊地址 --></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.namenode.http-address.ns1.nn1<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>slave01.example.com:50070<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- nn2的RPC通訊地址 --></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.namenode.rpc-address.ns1.nn2<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>slave02.example.com:9000<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- nn2的http通訊地址 --></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.namenode.http-address.ns1.nn2<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>slave02.example.com:50070<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 指定NameNode的後設資料在JournalNode上的存放位置 --></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.namenode.shared.edits.dir<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>qjournal://slave01.example.com:8485;slave02.example.com:8485;slave03.example.com:8485/ns1<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 指定JournalNode在本地磁碟存放資料的位置 --></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.journalnode.edits.dir<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>/sparkLearning/hadoop-2.4.1/journal<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 開啟NameNode失敗自動切換 --></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.ha.automatic-failover.enabled<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>true<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 配置失敗自動切換實現方式 --></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.client.failover.proxy.provider.ns1<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 配置隔離機制方法，多個機制用換行分割，即每個機制暫用一行--></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.ha.fencing.methods<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
sshfence
shell(/bin/true)
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 使用sshfence隔離機制時需要ssh免登陸 --></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.ha.fencing.ssh.private-key-files<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>/home/hadoop/.ssh/id_rsa<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 配置sshfence隔離機制超時時間 --></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.ha.fencing.ssh.connect-timeout<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>30000<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li><li style="box-sizing: border-box; padding: 0px 5px;">61</li><li style="box-sizing: border-box; padding: 0px 5px;">62</li><li style="box-sizing: border-box; padding: 0px 5px;">63</li><li style="box-sizing: border-box; padding: 0px 5px;">64</li><li style="box-sizing: border-box; padding: 0px 5px;">65</li><li style="box-sizing: border-box; padding: 0px 5px;">66</li><li style="box-sizing: border-box; padding: 0px 5px;">67</li><li style="box-sizing: border-box; padding: 0px 5px;">68</li><li style="box-sizing: border-box; padding: 0px 5px;">69</li><li style="box-sizing: border-box; padding: 0px 5px;">70</li></ul>
（4）修改mapred-site.xml檔案
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> hadoop]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cp mapred-site.xml.template mapred-site.xml</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
vim mapred-site.xml修改檔案內容如下:
<code class="hljs xml has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"> <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 指定mr框架為yarn方式 --></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>mapreduce.framework.name<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>yarn<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li></ul>
（6）修改yarn-site.xml檔案
<code class="hljs xml has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-pi" style="color: rgb(0, 102, 102); box-sizing: border-box;"><?xml version="1.0"?></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
--></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 開啟RM高可靠 --></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>yarn.resourcemanager.ha.enabled<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>true<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 指定RM的cluster id --></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>yarn.resourcemanager.cluster-id<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>SparkCluster<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 指定RM的名字 --></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>yarn.resourcemanager.ha.rm-ids<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>rm1,rm2<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 分別指定RM的地址 --></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>yarn.resourcemanager.hostname.rm1<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>slave01.example.com<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>yarn.resourcemanager.hostname.rm2<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>slave02.example.com<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 指定zk叢集地址 --></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>yarn.resourcemanager.zk-address<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>yarn.nodemanager.aux-services<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>mapreduce_shuffle<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li></ul>
（7）修改slaves檔案
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>
（8）配置檔案拷貝到其它伺服器
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">//slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>上的配置檔案拷貝到slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> [root@slave01 hadoop]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># scp -r /etc/profile slave02.example.com:/etc/profile</span> profile <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2027</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.0</span>KB/s <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span> [root@slave01 hadoop]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># scp -r /sparkLearning/hadoop-2.4.1 slave02.example.com:/sparkLearning/</span> //slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>上的配置檔案拷貝到slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> [root@slave01 hadoop]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># scp -r /etc/profile slave03.example.com:/etc/profile</span> profile <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2027</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.0</span>KB/s <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span> [root@slave01 hadoop]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># scp -r /sparkLearning/hadoop-2.4.1 slave03.example.com:/sparkLearning/</span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li></ul>
（9）啟動journalnode
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">//使用下列命令啟動journalnode [root@slave01 hadoop]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># hadoop-daemons.sh start journalnode</span> <span class="hljs-label" style="box-sizing: border-box;">slave02.example.com:</span> starting journalnode, logging to /sparkLearning/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/logs/hadoop-root-journalnode-slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.out</span> <span class="hljs-label" style="box-sizing: border-box;">slave03.example.com:</span> starting journalnode, logging to /sparkLearning/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/logs/hadoop-root-journalnode-slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.out</span> <span class="hljs-label" style="box-sizing: border-box;">slave01.example.com:</span> starting journalnode, logging to /sparkLearning/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/logs/hadoop-root-journalnode-slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.out</span> //JournalNode程式存在，啟動成功 [root@slave01 hadoop]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># jps</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">11261</span> JournalNode <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">11295</span> Jps [root@slave01 hadoop]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># ssh slave02.example.com</span> Last login: Fri Sep <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">18</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">05</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">33</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">05</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2015</span> from slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> [root@slave02 ~]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># jps</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6598</span> JournalNode <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6795</span> Jps [root@slave02 ~]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># ssh slave03.example.com</span> Last login: Fri Sep <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">18</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">05</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">33</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">26</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2015</span> from slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> [root@slave03 ~]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># jps</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5876</span> JournalNode <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6047</span> Jps [root@slave03 ~]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># </span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li></ul>
（10）格式化HDFS
登入slave02.example.com伺服器，執行下列命令
<code class="hljs markdown has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root@slave02 ~]# hdfs namenode -format //下面是執行結果 15/09/18 06:05:26 INFO namenode.NameNode: STARTUP_MSG: /<span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span> STARTUP_MSG: Starting NameNode STARTUP_MSG: host = slave02.example.com/127.0.0.1 STARTUP_MSG: args = [-format] STARTUP_MSG: version = 2.4.1 STARTUP_MSG: classpath = /sparkLearning/hadoop-2.4.1/etc/hadoop:/sparkLearning/hadoop-........省略無關資訊............... STARTUP_MSG: build = http://svn.apache.org/repos/asf/hadoop/common -r 1604318; compiled by 'jenkins' on 2014-06-21T05:43Z STARTUP<span class="hljs-emphasis" style="box-sizing: border-box;">_MSG: java = 1.8.0_</span>40 .....................................................省略..... /sparkLearning/hadoop-2.4.1/tmp/dfs/name has been successfully formatted. 15/09/18 06:05:30 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0 15/09/18 06:05:30 INFO util.ExitUtil: Exiting with status 0 15/09/18 06:05:30 INFO namenode.NameNode: SHUTDOWN_MSG: /<span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span> SHUTDOWN_MSG: Shutting down NameNode at slave02.example.com/127.0.0.1 <span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span><span class="hljs-strong" style="box-sizing: border-box;">*****</span>/ </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li></ul>
（11）格式化HDFS資訊複製到slave03.example.com伺服器
<code class="hljs erlang-repl has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[<span class="hljs-function_or_atom" style="box-sizing: border-box;">root</span>@<span class="hljs-function_or_atom" style="box-sizing: border-box;">slave02</span> ~]# <span class="hljs-function_or_atom" style="box-sizing: border-box;">scp</span> -<span class="hljs-function_or_atom" style="box-sizing: border-box;">r</span> /<span class="hljs-function_or_atom" style="box-sizing: border-box;">sparkLearning</span>/<span class="hljs-function_or_atom" style="box-sizing: border-box;">hadoop</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>/<span class="hljs-function_or_atom" style="box-sizing: border-box;">tmp</span>/ <span class="hljs-function_or_atom" style="box-sizing: border-box;">slave01</span>.<span class="hljs-function_or_atom" style="box-sizing: border-box;">example</span>.<span class="hljs-function_or_atom" style="box-sizing: border-box;">com</span>:/<span class="hljs-function_or_atom" style="box-sizing: border-box;">sparkLearning</span>/<span class="hljs-function_or_atom" style="box-sizing: border-box;">hadoop</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>/ <span class="hljs-function_or_atom" style="box-sizing: border-box;">fsimage_0000000000000000000</span>.<span class="hljs-function_or_atom" style="box-sizing: border-box;">md5</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">% 62 0.1KB/s 00:00 </span> <span class="hljs-function_or_atom" style="box-sizing: border-box;">seen_txid</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">% 2 0.0KB/s 00:00 </span> <span class="hljs-function_or_atom" style="box-sizing: border-box;">fsimage_0000000000000000000</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">% 350 0.3KB/s 00:00 </span> <span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">VERSION</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">% 200 0.2KB/s 00:00 </span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul>
（12）格式化ZK(在slave02.example.com上執行即可)
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root@slave02 hadoop]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># hdfs zkfc -formatZK</span> Java HotSpot(TM) <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span>-Bit Server VM warning: You have loaded library /sparkLearning/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/lib/native/libhadoop<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.so</span> which might have disabled stack guard. The VM will try to fix the stack guard now. ......省略無關資訊............... //執行成功 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">09</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">18</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">06</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span> INFO ha<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ActiveStandbyElector</span>: Successfully created /hadoop-ha/ns1 <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> ZK. <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">09</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">18</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">06</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span> INFO zookeeper<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ZooKeeper</span>: Session: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0x34fe096c3ca0000</span> closed <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">09</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">18</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">06</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span> INFO zookeeper<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ClientCnxn</span>: EventThread shut down </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li></ul>
（13）啟動HDFS(在slave02.example.com上執行)
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave02</span> hadoop]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># start-dfs.sh</span> [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave02</span> hadoop]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># jps</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7714</span> <span class="hljs-constant" style="box-sizing: border-box;">QuorumPeerMain</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6598</span> <span class="hljs-constant" style="box-sizing: border-box;">JournalNode</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8295</span> <span class="hljs-constant" style="box-sizing: border-box;">DataNode</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8202</span> <span class="hljs-constant" style="box-sizing: border-box;">NameNode</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8716</span> <span class="hljs-constant" style="box-sizing: border-box;">Jps</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8574</span> <span class="hljs-constant" style="box-sizing: border-box;">DFSZKFailoverController</span> [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave02</span> hadoop]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ssh slave01.example.com</span> <span class="hljs-constant" style="box-sizing: border-box;">Last</span> <span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">login:</span> <span class="hljs-constant" style="box-sizing: border-box;">Thu</span> <span class="hljs-constant" style="box-sizing: border-box;">Aug</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">27</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">06</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">24</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2015</span> from slave01.example.com [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># jps</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13744</span> <span class="hljs-constant" style="box-sizing: border-box;">DataNode</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13681</span> <span class="hljs-constant" style="box-sizing: border-box;">NameNode</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">11862</span> <span class="hljs-constant" style="box-sizing: border-box;">QuorumPeerMain</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14007</span> <span class="hljs-constant" style="box-sizing: border-box;">Jps</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13943</span> <span class="hljs-constant" style="box-sizing: border-box;">DFSZKFailoverController</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13851</span> <span class="hljs-constant" style="box-sizing: border-box;">JournalNode</span> [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave03</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># jps</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5876</span> <span class="hljs-constant" style="box-sizing: border-box;">JournalNode</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7652</span> <span class="hljs-constant" style="box-sizing: border-box;">Jps</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7068</span> <span class="hljs-constant" style="box-sizing: border-box;">DataNode</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6764</span> <span class="hljs-constant" style="box-sizing: border-box;">QuorumPeerMain</span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li></ul>
（14）啟動YARN(在slave01.example.com上執行)
<code class="hljs r has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">//slave01.example.com [root@slave01 ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># start-yarn.sh</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">...</span>輸出省略..... [root@slave01 ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># jps</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14528</span> Jps <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13744</span> DataNode <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13681</span> NameNode <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14228</span> NodeManager <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">11862</span> QuorumPeerMain <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13943</span> DFSZKFailoverController <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14138</span> ResourceManager <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13851</span> JournalNode </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li></ul>
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">//slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> [root@slave02 ~]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># jps</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">11216</span> Jps <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10656</span> JournalNode <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7714</span> QuorumPeerMain <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">11010</span> NodeManager <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10427</span> DataNode <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10844</span> DFSZKFailoverController <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10334</span> NameNode </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li></ul>
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">//slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> [root@slave03 ~]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># jps</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8610</span> JournalNode <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8791</span> NodeManager <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8503</span> DataNode <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">9001</span> Jps <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6764</span> QuorumPeerMain </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li></ul>
（15）檢視hadoop執行管理介面
開啟瀏覽器，輸入http://slave01.example.com:8088/，可以得到hadoop叢集管理介面:
輸入http://slave01.example.com:50070 可以得到HDFS管理介面
至此Hadoop叢集配置成功
3. Spark 1.5.0 叢集部署
（1）將Spark新增到環境變數
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> hadoop]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cp /mnt/hgfs/share/spark-1.5.0-bin-hadoop2.4.tgz /sparkLearning/</span> [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># tar -zxvf spark-1.5.0-bin-hadoop2.4.tgz > /dev/null</span> [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># vim /etc/profile</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul>
將/etc/profile內容修改如下：
<code class="hljs bash has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> JAVA_HOME=/sparkLearning/jdk1.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8.0</span>_40
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> SCALA_HOME=/sparkLearning/scala-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.10</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> HADOOP_HOME=/sparkLearning/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> SPARK_HOME=/sparkLearning/spark-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.5</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>-bin-hadoop2.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> PATH=<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${JAVA_HOME}</span>/bin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${SCALA_HOME}</span>/bin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${HADOOP_HOME}</span>/bin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${HADOOP_HOME}</span>/sbin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${SPARK_HOME}</span>/bin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">${SPARK_HOME}</span>/sbin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">$PATH</span>
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>
（2）將Spark新增到環境變數
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root@slave01 sparkLearning]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># cd spark-1.5.0-bin-hadoop2.4/conf</span> [root@slave01 conf]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># ls</span> docker<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.properties</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.template</span> metrics<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.properties</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.template</span> spark-env<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.sh</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.template</span> fairscheduler<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.xml</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.template</span> slaves<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.template</span> log4j<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.properties</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.template</span> spark-defaults<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.conf</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.template</span> //複製模板檔案 [root@slave01 conf]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># cp spark-env.sh.template spark-env.sh</span> [root@slave01 conf]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># vim spark-env.sh</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li></ul>
在spark-env.sh檔案中新增如下內容：
<code class="hljs bash has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> JAVA_HOME=/sparkLearning/jdk1.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8.0</span>_40 <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> SCALA_HOME=/sparkLearning/scala-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.10</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> HADOOP_CONF_DIR=/sparkLearning/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>/etc/hadoop</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> conf]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cp slaves.template slaves</span> [root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave01</span> conf]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># vim slaves</span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>
slaves檔案內容如下：
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># A Spark Worker will be started on each of the machines listed below.</span> slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span> slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>
（3）將配置資訊複製到其它伺服器
<code class="hljs r has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root@slave01 sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># scp /etc/profile slave02.example.com:/etc/profile</span> profile <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2123</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.</span>1KB/s <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span> [root@slave01 sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># scp /etc/profile slave03.example.com:/etc/profile</span> profile <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2123</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.</span>1KB/s <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span> [root@slave01 sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># vim /etc/profile</span> [root@slave01 sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># scp -r spark-1.5.0-bin-hadoop2.4 slave02.example.com:/sparkLearning/</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">...</span>執行過程省略..... [root@slave01 sparkLearning]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># scp -r spark-1.5.0-bin-hadoop2.4 slave03.example.com:/sparkLearning/</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">...</span>執行過程省略..... </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li></ul>
（4）啟動Spark叢集
因為本人機器上裝了Ambari Server，佔用了8080埠，而Spark Master預設端是8080，因此將sbin/start-master.sh中的SPARK_MASTER_WEBUI_PORT修改為8888
<code class="hljs bash has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">if</span> [ <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">$SPARK_MASTER_WEBUI_PORT</span>"</span> = <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">""</span> ]; <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">then</span> SPARK_MASTER_WEBUI_PORT=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8888</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">fi</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root@slave01 sbin]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># ./start-all.sh </span> starting org<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.apache</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.spark</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.deploy</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.master</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Master</span>, logging to /sparkLearning/spark-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.5</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span>-bin-hadoop2<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>/sbin/../logs/spark-root-org<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.apache</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.spark</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.deploy</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.master</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Master</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>-slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.out</span> <span class="hljs-label" style="box-sizing: border-box;">slave03.example.com:</span> starting org<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.apache</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.spark</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.deploy</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.worker</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Worker</span>, logging to /sparkLearning/spark-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.5</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span>-bin-hadoop2<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>/sbin/../logs/spark-root-org<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.apache</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.spark</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.deploy</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.worker</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Worker</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>-slave03<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.out</span> <span class="hljs-label" style="box-sizing: border-box;">slave02.example.com:</span> starting org<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.apache</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.spark</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.deploy</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.worker</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Worker</span>, logging to /sparkLearning/spark-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.5</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span>-bin-hadoop2<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>/sbin/../logs/spark-root-org<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.apache</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.spark</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.deploy</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.worker</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Worker</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>-slave02<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.out</span> <span class="hljs-label" style="box-sizing: border-box;">slave01.example.com:</span> starting org<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.apache</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.spark</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.deploy</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.worker</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Worker</span>, logging to /sparkLearning/spark-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.5</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span>-bin-hadoop2<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>/sbin/../logs/spark-root-org<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.apache</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.spark</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.deploy</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.worker</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Worker</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>-slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.out</span> [root@slave01 sbin]<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># jps</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13744</span> DataNode <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13681</span> NameNode <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14228</span> NodeManager <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16949</span> Master <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">11862</span> QuorumPeerMain <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13943</span> DFSZKFailoverController <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14138</span> ResourceManager <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13851</span> JournalNode <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">17179</span> Jps <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">17087</span> Worker </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li></ul>
瀏覽器中輸入slave01.example.com:8888
但是在啟動過程中出現了錯誤，檢視日誌檔案
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@slave02</span> logs]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># more spark-root-org.apache.spark.deploy.worker.Worker-1-slave02.example.com.out</span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>
日誌內容中包括下列錯誤：
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-label" style="box-sizing: border-box;">akka.actor.ActorNotFound:</span> Actor not found for: ActorSelection[Anchor(akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.tcp</span>://
sparkMaster@slave01<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.example</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7077</span>/), Path(/user/Master)]
at akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.actor</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ActorSelection</span>$$anonfun$resolveOne$1<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.apply</span>(ActorSelection.
<span class="hljs-label" style="box-sizing: border-box;">scala:</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">65</span>)
at akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.actor</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ActorSelection</span>$$anonfun$resolveOne$1<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.apply</span>(ActorSelection.
<span class="hljs-label" style="box-sizing: border-box;">scala:</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">63</span>)
at scala<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.concurrent</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.impl</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.CallbackRunnable</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.run</span>(Promise<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.scala</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span>)
at akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.dispatch</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.BatchingExecutor</span>$AbstractBatch<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.processBatch</span>(BatchingExe
<span class="hljs-label" style="box-sizing: border-box;">cutor.scala:</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">55</span>)
at akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.dispatch</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.BatchingExecutor</span>$Batch<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.run</span>(BatchingExecutor<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.scala</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">73</span>)
at akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.dispatch</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ExecutionContexts</span>$sameThreadExecutionContext$<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.unbatched</span>
Execute(Future<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.scala</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">74</span>)
at akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.dispatch</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.BatchingExecutor</span>$class<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.execute</span>(BatchingExecutor<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.scala</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20</span>)
at akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.dispatch</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ExecutionContexts</span>$sameThreadExecutionContext$<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.execute</span>(F
<span class="hljs-label" style="box-sizing: border-box;">uture.scala:</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">73</span>)
at scala<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.concurrent</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.impl</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.CallbackRunnable</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.executeWithValue</span>(Promise<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.scala</span>
:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">40</span>)
at scala<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.concurrent</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.impl</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Promise</span>$DefaultPromise<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.tryComplete</span>(Promise<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.scal</span>
<span class="hljs-label" style="box-sizing: border-box;">a:</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">248</span>)
at akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.pattern</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.PromiseActorRef</span>.$bang(AskSupport<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.scala</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">266</span>)
at akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.actor</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.EmptyLocalActorRef</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.specialHandle</span>(ActorRef<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.scala</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">533</span>)
at akka<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.actor</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.DeadLetterActorRef</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.specialHandle</span>(ActorRef<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.scala</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">569</span>)
.....省略.....................</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li></ul>
沒找到具體原因，在ubuntu 10.04伺服器上進行相同的配置，叢集搭建卻成功了（心中一萬頭…..），執行介面如下:
（5）測試Spark叢集
採用下列命上傳spark-1.5.0-bin-hadoop2.4目錄下的README.md檔案到相應的根目錄。
<code class="hljs livecodeserver has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"> hadoop dfs -<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">put</span> README.md </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
如下圖：
進入/spark-1.5.0-bin-hadoop2.4/bin目錄，啟動./spark-shell，如下圖所示：
執行REDME.md檔案的wordcount操作：
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">scala> val textCount = sc<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.textFile</span>(“README<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.md</span>”)<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.filter</span>(line => line<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.contains</span>(“Spark”))<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.count</span>() </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>
如下圖：
執行結果如下圖：
至此，Spark 1.5叢集搭建成功。
轉載: http://blog.csdn.net/lovehuangjiaju/article/details/48494737
相關文章
- Spark修煉之道（進階篇）——Spark入門到精通：第七節 Spark執行原理2015-11-14Spark
- Spark修煉之道（進階篇）——Spark入門到精通：第八節 Spark SQL與DataFrame（一)2015-11-14SparkSQL
- Spark修煉之道（進階篇）——Spark入門到精通：第四節 Spark程式設計模型（一)2015-11-14Spark程式設計模型
- Spark修煉之道（進階篇）——Spark入門到精通：第五節 Spark程式設計模型（二)2015-11-14Spark程式設計模型
- Spark修煉之道（進階篇）——Spark入門到精通：第六節 Spark程式設計模型（三)2015-11-14Spark程式設計模型
- Spark修煉之道（進階篇）——Spark入門到精通：第二節 Hadoop、Spark生成圈簡介2015-11-14SparkHadoop
- Spark修煉之道（進階篇）——Spark入門到精通：第三節 Spark Intellij IDEA開發環境搭建2015-11-14SparkIntelliJIdea開發環境
- Spark修煉之道（高階篇）——Spark原始碼閱讀：第一節 Spark應用程式提交流程2015-11-15Spark原始碼
- Hello Spark! | Spark，從入門到精通2018-09-18Spark
- Spark SQL | Spark，從入門到精通2019-01-21SparkSQL
- 搭建spark on yarn 叢集2018-05-03SparkYarn
- 從0到1搭建spark叢集---企業叢集搭建2017-12-16Spark
- Spark3.0.0叢集搭建2023-01-28Spark
- Spark入門篇2020-11-04Spark
- 實時計算框架：Spark叢集搭建與入門案例2021-04-26框架Spark
- Spark從入門到放棄——初始Spark（一）2020-12-09Spark
- HADOOP SPARK 叢集環境搭建2016-08-24HadoopSpark
- Spark學習進度-Spark環境搭建&Spark shell2021-01-04Spark
- Spark修煉之道（基礎篇）——Linux大資料開發基礎：第九節：Shell程式設計入門(一)2015-11-14SparkLinux大資料程式設計
- Spark修煉之道（基礎篇）——Linux大資料開發基礎：第十節：Shell程式設計入門（二)2015-11-14SparkLinux大資料程式設計
- spark學習筆記--叢集執行Spark2018-07-12Spark筆記
- Spark修煉之道（基礎篇）——Linux大資料開發基礎：第十二節：Shell程式設計入門（四)2015-11-14SparkLinux大資料程式設計
- Spark修煉之道（基礎篇）——Linux大資料開發基礎：第十三節：Shell程式設計入門（五)2015-11-14SparkLinux大資料程式設計
- Spark修煉之道（基礎篇）——Linux大資料開發基礎：第十四節：Shell程式設計入門（六)2015-11-15SparkLinux大資料程式設計
- docker下，極速搭建spark叢集(含hdfs叢集)2022-08-17DockerSpark
- Spark修煉之道（基礎篇）——Linux大資料開發基礎：第七節：程式管理2015-11-14SparkLinux大資料
- 基於 ZooKeeper 搭建 Spark 高可用叢集2019-06-29Spark
- Spark3.0.1各種叢集模式搭建2020-11-21Spark模式
- 【Spark篇】---Spark初始2018-02-01Spark
- Docker中提交任務到Spark叢集2020-07-03DockerSpark
- Spark修煉之道（基礎篇）——Linux大資料開發基礎：第八節：網路管理2015-11-14SparkLinux大資料
- Spark 叢集搭建從零開始之3 Spark Standalone叢集安裝、配置與測試2017-09-19Spark
- Spark 1.5.0 遠端除錯2015-11-15Spark除錯
- Spark下載與入門（Spark自學二）2017-10-11Spark
- Spark 快速入門2019-04-24Spark
- Spark修煉之道（基礎篇）——Linux大資料開發基礎：第一節、Linux介紹、安裝及使用初步2015-11-14SparkLinux大資料
- Spark從入門到放棄---RDD2020-08-17Spark
- Spark入門（五）--Spark的reduce和reduceByKey2019-03-01Spark