微影片連結: Apache Kylin初識



1.版本(當前實驗版本組合,版本一定要相容,否則各種問題)
Hadoop2.7.2+HBase1.1.5+Hive2.0.0
kylin-1.5.2.1   (apache-kylin-1.5.1-HBase1.1.3-bin.tar.gz)

2.Hadoop環境編譯以支援Snappy解壓縮庫
重新編譯hadoop-2.7.2-src的native以支援Snappy解壓壓縮庫

3.環境準備
Hadoop-2.7.2+zookeeper-3.4.6完全分散式環境搭建(HDFS、YARN HA)
Hadoop2.7實戰v1.0之HBase1.1.5 HA分散式搭建
配置Hadoop2.7.2和Hbase1.1.5支援Snappy解壓壓縮庫

Hadoop2.7實戰v1.0之Hive-2.0.0+MySQL本地模式安裝

4. 配置Hadoop2.7.2和Hbase1.1.5支援Snappy解壓壓縮庫

5. 主機規劃
IP                Host                    安裝軟體                 程式
172.16.101.55 sht-sgmhadoopnn-01   apache-kylin-1.5.1-HBase1.1.3-bin.tar.gz RunJar

6. 新增hive_dependency和KYLIN_HOME環境變數

點選(此處)摺疊或開啟

[root@sht-sgmhadoopnn-01 ~]# vi /etc/profile
export JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export HADOOP_HOME=/hadoop/hadoop
export HBASE_HOME=/hadoop/hbase
export ZOOKEEPER_HOME=/hadoop/zookeeper
export HIVE_HOME=/hadoop/hive
export hive_dependency=/hadoop/hive/conf:/hadoop/hive/lib/*:/hadoop/hive/hcatalog/share/hcatalog/hive-hcatalog-core-2.0.0.jar
export KYLIN_HOME=/hadoop/kylin
export PATH=.:$HADOOP_HOME/bin:$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$HBASE_HOME/bin:$HIVE_HOME/bin:$KYLIN_HOME/bin:$PATH
"/etc/profile" 99L, 2592C written
[root@sht-sgmhadoopnn-01 ~]# source /etc/profile

7.同步/etc/profile到叢集其他節點,生效

8.下載安裝配置kylin

點選(此處)摺疊或開啟

# 在sht-sgmhadoopnn-01上執行以下命令
[root@sht-sgmhadoopnn-01 ~]# cd /hadoop
[root@sht-sgmhadoopnn-01 hadoop]# wget https://dist.apache.org/repos/dist/release/kylin/apache-kylin-1.5.2.1/apache-kylin-1.5.2.1-HBase1.x-bin.tar.gz
[root@sht-sgmhadoopnn-01 hadoop]# tar -zxvf apache-kylin-1.5.1-HBase1.1.3-bin.tar.gz
[root@sht-sgmhadoopnn-01 hadoop]# ln -s /hadoop/apache-kylin-1.5.2.1-bin /hadoop/kylin
[root@sht-sgmhadoopnn-01 hadoop]# vi /hadoop/kylin/bin/kylin.sh
export KYLIN_HOME=/hadoop/kylin # 改成絕對路徑
export HBASE_CLASSPATH_PREFIX=${tomcat_root}/bin/bootstrap.jar:${tomcat_root}/bin/tomcat-juli.jar:${tomcat_root}/lib/*:$hive_dependency:$HBASE_CLASSPATH_PREFIX #在路徑中新增$hive_dependency

9.修改kylin.properties

點選(此處)摺疊或開啟

[root@sht-sgmhadoopnn-01 conf]# vi $KYLIN_HOME/conf/kylin.properties

######修改

# List of web servers in use, this enables one web server instance to sync up with other servers.

kylin.rest.servers=sht-sgmhadoopnn-01:7070

######新增

kylin.job.jar=$KYLIN_HOME/lib/kylin-job-1.5.2.1.jar

kylin.coprocessor.local.jar=$KYLIN_HOME /lib/kylin-coprocessor-1.5.2.1.jar

10.啟動kylin
[root@sht-sgmhadoopnn-01 kylin]# $KYLIN_HOME/bin/kylin.sh start
.........
.........
KYLIN_DEBUG_SETTINGS is not set, will not enable remote debuging
KYLIN_LD_LIBRARY_SETTINGS is not set, Usually it's okay unless you want to specify your own native path
A new Kylin instance is started by root, stop it using "kylin.sh stop"
Please visit <a href="

You can check the log at /hadoop/kylin/logs/kylin.log<a href="

You can check the log at

###   賬號: ADMIN    密碼:KYLIN

11.檢查kylin是否啟動成功

點選(此處)摺疊或開啟

[root@sht-sgmhadoopnn-01 kylin]# netstat -nlp |grep 7070

tcp 0 0 0.0.0.0:7070 0.0.0.0:* LISTEN 30939/java

  [root@sht-sgmhadoopnn-01 kylin]#

12.匯入官網測試案例

點選(此處)摺疊或開啟

[root@sht-sgmhadoopnn-01 kylin]# $KYLIN_HOME/bin/kylin.sh stop

[root@sht-sgmhadoopnn-01 kylin]# $KYLIN_HOME/bin/sample.sh

[root@sht-sgmhadoopnn-01 kylin]# $KYLIN_HOME/bin/kylin.sh start

1>.選中'kylin_sales_cube'示例立方體，點選'Actions'->'Build'，選擇一個截止日期，本試驗中選擇的是'2016-01-10'
(具體小時,分,秒隨便選,因為最終hive sql的語句類似     WHERE (KYLIN_SALES.PART_DT >= '2012-01-01' AND KYLIN_SALES.PART_DT < '2016-01-10');)

2>.在'Monitor'標籤中透過重新整理頁面檢查進度條，直到100%

3>.在'Insight'標籤中執行下面的SQL查詢：
select part_dt, sum(price) as total_selled, count(distinct seller_id) as sellers from kylin_sales group by part_dt order by part_dt;
###耗時2.87s
4>.在hive中執行同一個SQL查詢，驗證kylin的查詢結果(會開啟MapReduce Job計算)
###耗時65.205s
查詢執行和結果如圖所示

Build成功後，hive中建立了3+n個表，如圖所示(3個官網案例hive表,n個build的hive表)

Build成功後，hbase中建立了1+n個表，如圖所示(1個後設資料表,n個build的hbase表)

<a href="

You can check the log at

全網最詳細Apache Kylin1.5安裝(單節點)和測試案例

相關文章