大資料專案實踐（五）——Hue安裝

一、下載hue安裝包

http://archive-primary.cloude...

二、安裝依賴包

yum install gcc g++ libxml2-devel libxslt-devel cyrus-sasl-devel cyrus-sasl-gssapi mysql-devel python-devel python-setuptools sqlite-devel ant libsasl2-dev libsasl2-modules-gssapi-mit libkrb5-dev libtidy-0.99-0 mvn openldap-dev

三、編譯

如果之前是編譯過的，則刪除

$ rm app.reg
$ rm -r build

[root@bigdata-pro01 hue-3.9.0-cdh5.10.2]# su bigdata
[bigdata@bigdata-pro01 hue-3.9.0-cdh5.10.2]$ pwd
/opt/modules/hue-3.9.0-cdh5.10.2
[bigdata@bigdata-pro01 hue-3.9.0-cdh5.10.2]$ make apps 

四、編譯過程中遇到的問題

反覆編譯，直到成功，成功後，目錄中多出兩個檔案：一個是

• 問題1

/usr/include/features.h:164:1: 警告：這是先前定義的位置 
src/_fastmath.c:36:18: 錯誤：gmp.h：沒有那個檔案或目錄 
src/_fastmath.c:63: 錯誤：expected ‘)’ before ‘n’ 
src/_fastmath.c:66: 錯誤：expected ‘)’ before ‘m’ 
src/_fastmath.c:107: 錯誤：expected ‘)’ before ‘m’ 
src/_fastmath.c:148: 錯誤：expected specifier-qualifier-list before ‘mpz_t’ 
src/_fastmath.c:158: 錯誤：expected specifier-qualifier-list before ‘mpz_t’ 
src/_fastmath.c:196: 錯誤：expected declaration specifiers or ‘...’  before ‘mpz_t’ 
src/_fastmath.c:196: 錯誤：expected declaration specifiers or ‘...’  before ‘mpz_t’ 
......
src/_fastmath.c:1601: 警告：隱式宣告函式‘mpz_gcd_ui’ 
src/_fastmath.c:1608: 警告：隱式宣告函式‘mpz_divexact_ui’ 
error: command ‘gcc‘ failed with exit status 1 
make[2]: *** [/opt/hue/desktop/core/build/pycrypto-2.6.1/egg.stamp] 錯誤 1 
make[2]: Leaving directory `/opt/hue/desktop/core‘ 
make[1]: *** [.recursive-env-install/core] 錯誤 2 
make[1]: Leaving directory `/opt/hue/desktop‘ 
make: *** [desktop] 錯誤 2 

少個gmp
yum install gmp-devel

• 問題2

Modules/errors.h:8:18: error: lber.h: No such file or directory Modules/errors.h:9:18: error: ldap.h: No such file or directory

解決如下：

sudo apt-get install libsasl2-dev python-dev libldap2-dev libssl-dev
sudo apt-get install -y python-dev libldap2-dev libsasl2-dev libssl-dev
sudo yum install python-devel
sudo yum install openldap-devel

實際執行 sudo yum install openldap-devel 即可

五、配置

進入desktop目錄

[bigdata@bigdata-pro01 hue-3.9.0-cdh5.10.2]$ cd desktop/
[bigdata@bigdata-pro01 desktop]$ ll
total 284
drwxr-xr-x  2 bigdata bigdata   4096 Jun 27  2017 conf
drwxr-xr-x  5 bigdata bigdata   4096 Aug 27 21:40 core
-rw-r--r--  1 bigdata bigdata 264192 Aug 27 22:24 desktop.db
drwxr-xr-x 15 bigdata bigdata   4096 Jun 27  2017 libs
drwxrwxr-x  2 bigdata bigdata   4096 Aug 27 22:23 logs
-rw-r--r--  1 bigdata bigdata   3467 Jun 27  2017 Makefile
[bigdata@bigdata-pro01 desktop]$ cd conf/
[bigdata@bigdata-pro01 conf]$ ll
total 64
-rw-r--r-- 1 bigdata bigdata 54575 Jun 27  2017 hue.ini
-rw-r--r-- 1 bigdata bigdata  1843 Jun 27  2017 log4j.properties
-rw-r--r-- 1 bigdata bigdata  1809 Jun 27  2017 log.conf
[bigdata@bigdata-pro01 conf]$ 

編輯hue.ini 檔案

secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn<qW5o
http_host=bigdata-pro01.mars.com
http_port=8888
time_zone=Asia/Shanghai

修改desktop.db 許可權

[bigdata@bigdata-pro01 desktop]$ sudo chmod o+w desktop.db 

啟動hue 服務

[bigdata@bigdata-pro01 bin]$ pwd
/opt/modules/hue-3.9.0-cdh5.10.2/build/env/bin
[bigdata@bigdata-pro01 bin]$ ./supervisor 
[INFO] Not running as root, skipping privilege drop
starting server with options:
{'daemonize': False,
 'host': 'bigdata-pro01.mars.com',
 'pidfile': None,
 'port': 8888,
 'server_group': 'hue',
 'server_name': 'localhost',
 'server_user': 'hue',
 'ssl_certificate': None,
 'ssl_certificate_chain': None,
 'ssl_cipher_list': 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA',
 'ssl_private_key': None,
 'threads': 40,
 'workdir': None}

進入web介面，瀏覽器地址輸入：bigdata-pro01.mars.com:8888

建立超級使用者，牢記使用者名稱和密碼，如 admin/admin
然後就進入主介面

六、Hue與HDFS整合

1.修改core-site.xml
在core-site.xml中新增以下配置：

<property>
        <name>hadoop.proxyuser.hue.hosts</name>
        <value>*</value>
    </property>
    
    <property>
        <name>hadoop.proxyuser.hue.groups</name>
        <value>*</value>
    </property>

2.修改hue.ini

新增以下內容：

fs_defaultfs=hdfs://hadoop-ha
webhdfs_url=http://bigdata-pro01.mars.com:50070/webhdfs/v1
hadoop_conf_dir=/opt/modules/hadoop-2.6.0-cdh5.10.2/etc/hadoop
hadoop_bin=/opt/modules/hadoop-2.6.0-cdh5.10.2/bin
hadoop_hdfs_home=/opt/modules/hadoop-2.6.0-cdh5.10.2

3.將core-site.xml分發到叢集的其他每個機器上
重啟hdfs

$HADOOP_HOME/etc/hadoop/stop-dfs.sh
$HADOOP_HOME/etc/hadoop/start-dfs.sh

重啟hue服務

$HUE_HOME/build/env/bin/supervisor

瀏覽器檢視：

七、Hue 和 yarn 整合

八、Hue 和 hive 整合

八、Hue 和 mysql 整合(hive metastore)

九、Hue 和 Hbase 整合

同時，Hbase啟動thrift服務：
$HBASE_HOME/bin/hbase-daemon.sh start thrift