【Hadoop】pyhton連結hive

小亮520cl發表於2018-06-27

使用hive模組

  1. pip install sasl

  2. pip install thrift

  3. pip install thrift - sasl

  4. pip install PyHive

  1. [ root@ ip - 172 - 31 - 40 - 242 ~ ] # more testpyhive . py

  2. from pyhive import hive

  3. conn = hive . Connection ( host = 'xxxxxxx' , port = 10000 , database = 'collection' , username = '' )

  4. cursor = conn . cursor ( )

  5. cursor . execute ( 'select * from tb_partition limit 10' )

  6. for result in cursor . fetchall ( ) :

  7.     print result



[root@ip-172-31-40-242 ~]# python testpyhive.py 
(u'1', u'2', u'201707')
(u'1', u'2', u'201707')
(u'123', None, u'201709')
(u'123', u'456', u'201709')
(u'45678', u'456', u'201709')
(u'123', None, u'201709')
(u'123', u'456', u'201709')
(u'45678', u'456', u'201709')
(u'123', None, u'201709')
(u'123', u'456', u'201709')


官方API:




如何使用Python Impyla客戶端連線Hive和Impala

# -*- coding:utf-8 -*-
from impala.dbapi import connect
conn = connect(host='172.31.46.109',port=10000,database='collection',auth_mechanism='PLAIN')
print(conn)
cursor = conn.cursor()
#
param = '''SET hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
SET hive.support.concurrency=true;
'''
cursor.execute(param)
cursor.execute('SELECT uid FROM redefine_collection where uid=4028 limit 10')
print cursor.description  # prints the result set's schema
results = cursor.fetchall()
print results
#
# Python連線Impala(ImpalaTest.py)
#
# from impala.dbapi importconnect
#
# conn = connect(host='ip-172-31-26-80.ap-southeast-1.compute.internal',port=21050)
#
# print(conn)
#
# cursor = conn.cursor()
#
# cursor.execute('show databases')
#
# print cursor.description  # prints the result set's schema
#
# results = cursor.fetchall()
#
# print(results)
#
# cursor.execute('SELECT * FROM test limit 10')
#
# print cursor.description  # prints the result set's schema
#
# results = cursor.fetchall()
#
# print(results)
參考:https://cloud.tencent.com/developer/article/1078029



來自 “ ITPUB部落格 ” ,連結:http://blog.itpub.net/29096438/viewspace-2156928/,如需轉載,請註明出處,否則將追究法律責任。

相關文章