小丸子學Hadoop系列之——部署Hbase叢集

wxjzqym發表於2016-07-19
Hadoop
0.叢集規劃
主機名 ip地址 安裝的軟體 執行的程式
AI-OPT-HBS01 10.46.52.30 hadoop,hbase namenode,zkfc,resourcemanager
AI-OPT-HBS02 10.46.52.31 hadoop namenode,zkfc,resourcemanager
AI-OPT-HBS03 10.46.52.32 hadoop,hbase datanode
AI-OPT-HBS04 10.46.52.33 hadoop,zookeeper,hbase datanode,nodemanager,journalnode
AI-OPT-HBS05 10.46.52.34 hadoop,zookeeper,hbase datanode,nodemanager,journalnode
AI-OPT-HBS06 10.46.52.35 hadoop,zookeeper,hbase datanode,nodemanager,journalnode


1.安裝1.建立使用者以及配置主機名
mkdir -p /aifs01/users
useradd -u 351   -g hadoop -G ibss -d /aifs01/users/hdpusr01 -m hdpusr01
passwd hdpusr01     --stdin <<< Linuhdp_0805
echo -e "\n. ~puwadm/wprofile\n" >> ~hdpusr01/.bash_profile 

useradd -u 352   -g hbase -G ibss -d /aifs01/users/hbsusr01 -m hbsusr01
passwd hbsusr01     --stdin <<< Linuhbs_0805
echo -e "\n. ~puwadm/wprofile\n" >> ~hbsusr01/.bash_profile 

vi /etc/hosts            
10.46.52.30 AI-OPT-HBS01
10.46.52.31 AI-OPT-HBS02
10.46.52.32 AI-OPT-HBS03
10.46.52.33 AI-OPT-HBS04
10.46.52.34 AI-OPT-HBS05
10.46.52.35 AI-OPT-HBS06

2.安裝Hadoop叢集
具體步驟請參考《小丸子學Hadoop系列之——部署Hadoop叢集》
http://blog.itpub.net/20801486/viewspace-1877048/

3.安裝HBase叢集
3.1 安裝HBase叢集（AI-OPT-HBS01 - AI-OPT-HBS06）
--選擇一臺主機操作（AI-OPT-HBS01）
[root@AI-OPT-HBS01 hbsusr01]# tar xvf /opt/software/hbase-1.1.2.tgz
[root@AI-OPT-HBS01 hbsusr01]# mv hbase-1.1.2 hbase
[root@AI-OPT-HBS01 hbsusr01]# chown -R hbsusr01:hbase hbase

--設定叢集節點檔案
hbsusr01@AI-OPT-HBS01:/aifs01/users/hbsusr01/hbase/conf> vi regionservers
AI-OPT-HBS03
AI-OPT-HBS04
AI-OPT-HBS05
AI-OPT-HBS06

--設定hbase配置檔案
hbsusr01@AI-OPT-HBS01:/aifs01/users/hbsusr01/hbase/conf> vi hbase-site.xml 
     1:
     2:
     3:         hbase.rootdir
     4:         hdfs://AI-OPT-HBS01:8920/hbase
     5:
     6:
     7:
     8:         hbase.master.port
     9:         60900
    10:
    11:
    12:
    13:         hbase.regionserver.port
    14:         60920
    15:
    16:
    17:
    18:         hbase.cluster.distributed
    19:         true
    20:
    21:
    22:
    23:         hbase.zookeeper.quorum
    24:         AI-OPT-HBS04,AI-OPT-HBS05,AI-OPT-HBS06
    25:
    26:
    27:
    28:         hbase.tmp.dir
    29:         /aifs01/users/hbsusr01/hbase/tmp
    30:
    31:
    32:
    33:         hbase.zookeeper.peerport
    34:         29888
    35:
    36:
    37:
    38:         hbase.zookeeper.leaderport
    39:         39888
    40:
    41:
    42:
    43:         hbase.zookeeper.property.clientPort
    44:         29181
    45:
    46:
    47:
    48:         hbase.rest.port
    49:         8980
    50:
    51:

--設定hbase環境
hbsusr01@AI-OPT-HBS01:/aifs01/users/hbsusr01/hbase/conf> vi hbase-env.sh 
export HBASE_CONF_DIR=/aifs01/users/hbsusr01/hbase/conf
export HBASE_PID_DIR=/aifs01/users/hbsusr01/hbase/pid
export HBASE_LOG_DIR=/aifs01/users/hbsusr01/hbase/logs

# export HBASE_MANAGES_ZK=true
export HBASE_MANAGES_ZK=false

--建立相關目錄
hbsusr01@AI-OPT-HBS01:/aifs01/users/hbsusr01> mkdir -p hbase/{conf,pid,logs,tmp}

--複製檔案到其他節點並解壓
hbsusr01@AI-OPT-HBS01:/aifs01/users/hbsusr01> tar cvf hbase.tar hbase
hbsusr01@AI-OPT-HBS01:/aifs01/users/hbsusr01> scp hbase.tar AI-OPT-HBS03:/aifs01/users/hbsusr01
hbsusr01@AI-OPT-HBS01:/aifs01/users/hbsusr01> scp hbase.tar AI-OPT-HBS04:/aifs01/users/hbsusr01
hbsusr01@AI-OPT-HBS01:/aifs01/users/hbsusr01> scp hbase.tar AI-OPT-HBS05:/aifs01/users/hbsusr01
hbsusr01@AI-OPT-HBS01:/aifs01/users/hbsusr01> scp hbase.tar AI-OPT-HBS06:/aifs01/users/hbsusr01
                    
--配置ssh互信
su - hbsusr01
ssh-keygen -t rsa
cat ~/.ssh/id_rsa.pub > ~/.ssh/authorized_keys
chmod 600 .ssh/authorized_keys

--選擇一臺master操作(AI-OPT-HBS01)
ssh AI-OPT-HBS02 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys 
ssh AI-OPT-HBS03 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys 
ssh AI-OPT-HBS04 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys 
ssh AI-OPT-HBS05 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys 
ssh AI-OPT-HBS06 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys 

scp ~/.ssh/authorized_keys AI-OPT-HBS02:/aifs01/users/hbsusr01/.ssh/authorized_keys
scp ~/.ssh/authorized_keys AI-OPT-HBS03:/aifs01/users/hbsusr01/.ssh/authorized_keys
scp ~/.ssh/authorized_keys AI-OPT-HBS04:/aifs01/users/hbsusr01/.ssh/authorized_keys
scp ~/.ssh/authorized_keys AI-OPT-HBS05:/aifs01/users/hbsusr01/.ssh/authorized_keys
scp ~/.ssh/authorized_keys AI-OPT-HBS06:/aifs01/users/hbsusr01/.ssh/authorized_keys

--驗證配置是否生效
ssh AI-OPT-HBS01 date
ssh AI-OPT-HBS02 date
ssh AI-OPT-HBS03 date
ssh AI-OPT-HBS04 date
ssh AI-OPT-HBS05 date
ssh AI-OPT-HBS06 date

--解壓hadoop包（AI-OPT-HBS03 - AI-OPT-HBS06）
tar xvf hbase.tar

--啟動hbase
start-hbase.sh
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=hbsusr01, access=WRITE, inode="/":hdpusr01:supergroup:drwxr-xr-x

解決方法：hadoop fs新增目錄和許可權
hdpusr01@AI-OPT-HBS01:/aifs01/users/hdpusr01/hadoop/bin> ./hadoop  fs -mkdir /hbase
hdpusr01@AI-OPT-HBS01:/aifs01/users/hdpusr01/hadoop/bin> ./hadoop fs -chown hbsusr01:hbase /hbase

至此,Hbase叢集搭建完畢。

來自 “ ITPUB部落格 ” ，連結：http://blog.itpub.net/20801486/viewspace-2122238/，如需轉載，請註明出處，否則將追究法律責任。

相關文章