情況
很多時間遇到 "Caused by: java.lang.OutOfMemoryError: Java heap space"Caused by: java.lang.OutOfMemoryError: Java heap space
可能會有下面錯誤 "Reducer tasks of hive job fails with Out Of Memory error during shuffle fetcher stage"
解決辦法:
- 增加hive.tez.container.size .
- tez.runtime.shuffle.memory.limit.percent 改小一點
可選
- 減少tez.runtime.shuffle.fetch.buffer.percent 預設是0.25 , 儘管為0.15 if needed. (這個值要在區別 0.25 , 0.10)