擴充套件叢集blk數

1.擴充套件blk數

hadoop fs -setrep -w 3 /

2.hive載入資料並覆蓋

load data inpath '/user/date=2017-03-02/*' OVERWRITE into table compensator_p partition(date='2017-03-02')

3.hive通過zk配置

jdbc:hive2://192.168.6.61:2181,192.168.6.71:2181,192.168.6.81:2181/fjudm4;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2

4.spark提交任務

spark-submit --class com.epri.sgcc.DCloud.util.Oms_lzop --master spark://FJ-DCLOUD-RDB98:7077 --driver-memory 5g --executor-memory 10g --total-executor-cores 4 $kettle_jar_home/spark_load_data/sparkLoadHive.jar $db ems $td

內容均為作者獨立觀點，不代表八零IT人立場，如涉及侵權，請及時告知。