flink sql client讀取hive時卡住

問題復現如下：

 

檢視$FLINK_HOME/log/flink-appleyuchi-sql-client-Desktop.log

2020-12-23 11:48:56,811 INFO  org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Opening socket connection to server Laptop/192.168.0.103:2182. Will not attempt to authenticate using SASL (unknown error)
2020-12-23 11:48:56,814 INFO  org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Socket error occurred: Laptop/192.168.0.103:2182: Connection refused
2020-12-23 11:48:56,915 INFO  org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Opening socket connection to server Desktop/192.168.0.102:2181. Will not attempt to authenticate using SASL (unknown error)
2020-12-23 11:48:56,915 INFO  org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Socket error occurred: Desktop/192.168.0.102:2181: Connection refused
2020-12-23 11:48:57,016 INFO  org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Opening socket connection to server Laptop/192.168.0.103:2183. Will not attempt to authenticate using SASL (unknown error)
2020-12-23 11:48:57,019 INFO  org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Socket error occurred: Laptop/192.168.0.103:2183: Connection refused
 

解決方案：

啟動zookeeper叢集

########################################################################################################################################################################

然後重啟Flink SQL Client發現讀取hive時再次卡住,

再次檢視$FLINK_HOME/log/flink-appleyuchi-sql-client-Desktop.log

2020-12-23 11:01:07,786 WARN  org.apache.flink.table.client.cli.CliClient                  [] - Could not execute SQL statement.
org.apache.flink.table.client.gateway.SqlExecutionException: Error while submitting job.
	at org.apache.flink.table.client.gateway.local.LocalExecutor.lambda$executeQueryInternal$7(LocalExecutor.java:554) ~[flink-sql-client_2.12-1.12.0.jar:1.12.0]
	at org.apache.flink.table.client.gateway.local.ExecutionContext.wrapClassLoader(ExecutionContext.java:257) ~[flink-sql-client_2.12-1.12.0.jar:1.12.0]
	at org.apache.flink.table.client.gateway.local.LocalExecutor.executeQueryInternal(LocalExecutor.java:549) ~[flink-sql-client_2.12-1.12.0.jar:1.12.0]
	at org.apache.flink.table.client.gateway.local.LocalExecutor.executeQuery(LocalExecutor.java:365) ~[flink-sql-client_2.12-1.12.0.jar:1.12.0]
	at org.apache.flink.table.client.cli.CliClient.callSelect(CliClient.java:634) ~[flink-sql-client_2.12-1.12.0.jar:1.12.0]
	at org.apache.flink.table.client.cli.CliClient.callCommand(CliClient.java:324) ~[flink-sql-client_2.12-1.12.0.jar:1.12.0]
	at java.util.Optional.ifPresent(Optional.java:159) [?:1.8.0_131]
	at org.apache.flink.table.client.cli.CliClient.open(CliClient.java:216) [flink-sql-client_2.12-1.12.0.jar:1.12.0]
	at org.apache.flink.table.client.SqlClient.openCli(SqlClient.java:141) [flink-sql-client_2.12-1.12.0.jar:1.12.0]
	at org.apache.flink.table.client.SqlClient.start(SqlClient.java:114) [flink-sql-client_2.12-1.12.0.jar:1.12.0]
	at org.apache.flink.table.client.SqlClient.main(SqlClient.java:196) [flink-sql-client_2.12-1.12.0.jar:1.12.0]
Caused by: java.util.concurrent.ExecutionException: org.apache.flink.runtime.client.JobSubmissionException: Failed to submit JobGraph.
	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) ~[?:1.8.0_131]
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895) ~[?:1.8.0_131]
	at org.apache.flink.table.client.gateway.local.LocalExecutor.lambda$executeQueryInternal$7(LocalExecutor.java:552) ~[flink-sql-client_2.12-1.12.0.jar:1.12.0]
	... 10 more
Caused by: org.apache.flink.runtime.client.JobSubmissionException: Failed to submit JobGraph.
	at org.apache.flink.client.program.rest.RestClusterClient.lambda$submitJob$7(RestClusterClient.java:366) ~[flink-dist_2.12-1.12.0.jar:1.12.0]
	at java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:870) ~[?:1.8.0_131]
	at java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:852) ~[?:1.8.0_131]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[?:1.8.0_131]
	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977) ~[?:1.8.0_131]
	at org.apache.flink.runtime.concurrent.FutureUtils.lambda$retryOperationWithDelay$9(FutureUtils.java:361) ~[flink-dist_2.12-1.12.0.jar:1.12.0]
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) ~[?:1.8.0_131]
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) ~[?:1.8.0_131]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[?:1.8.0_131]
	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977) ~[?:1.8.0_131]
	at org.apache.flink.runtime.concurrent.FutureUtils$Timeout.run(FutureUtils.java:1168) ~[flink-dist_2.12-1.12.0.jar:1.12.0]
	at org.apache.flink.runtime.concurrent.DirectExecutorService.execute(DirectExecutorService.java:211) ~[flink-dist_2.12-1.12.0.jar:1.12.0]
	at org.apache.flink.runtime.concurrent.FutureUtils.lambda$orTimeout$15(FutureUtils.java:549) ~[flink-dist_2.12-1.12.0.jar:1.12.0]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_131]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_131]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_131]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.0_131]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_131]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_131]
	at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_131]
Caused by: java.util.concurrent.TimeoutException
	at org.apache.flink.runtime.concurrent.FutureUtils$Timeout.run(FutureUtils.java:1168) ~[flink-dist_2.12-1.12.0.jar:1.12.0]
	at org.apache.flink.runtime.concurrent.DirectExecutorService.execute(DirectExecutorService.java:211) ~[flink-dist_2.12-1.12.0.jar:1.12.0]
	at org.apache.flink.runtime.concurrent.FutureUtils.lambda$orTimeout$15(FutureUtils.java:549) ~[flink-dist_2.12-1.12.0.jar:1.12.0]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_131]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_131]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_131]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.0_131]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_131]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_131]
	at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_131]

解決方案:

看得我一臉蒙比，機緣巧合下手賤

啟動了Flink叢集(HA)

問題解決了．

##########################################################################################################################################################

綜上，Flink操作hive卡住問題有兩種情況

①Flink這邊的問題，

啟動Zookeeper叢集

啟動Flink叢集(HA)

②Hive本身操作表格和schema會卡住

 

Reference:

[1]Hive drop table時候卡死-各種情況的解決方案(持續更新中)