hadoop之 node manager起不來, 執行mapreduce 程式hang住

張衝andy發表於2018-01-29

現象: node manager起不來, 執行mapreduce 程式hang住

namenode 程式狀態查詢
[root@hadp-master sbin]# jps
8608 ResourceManager
8257 NameNode
9268 Jps
8453 SecondaryNameNode

datanode 程式狀態查詢 , 發現 nodemanager 沒有起來
[root@hadp-node1 hadoop]# jps
4179 DataNode
4940 Jps

namenode 單獨起 nodemanager
[root@hadp-master sbin]# yarn nodemanager
18/01/26 03:53:54 INFO webapp.WebApps: Web app node started at 8042
18/01/26 03:53:54 INFO client.RMProxy: Connecting to ResourceManager at hadp-master/10.100.25.30:8031
18/01/26 03:53:54 INFO nodemanager.NodeStatusUpdaterImpl: Sending out 0 NM container statuses: []
18/01/26 03:53:54 INFO nodemanager.NodeStatusUpdaterImpl: Registering with RM using containers :[]
18/01/26 03:53:54 ERROR nodemanager.NodeStatusUpdaterImpl: Unexpected error starting NodeStatusUpdater
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN signal from Resourcemanager ,Registration of NodeManager failed, Message from ResourceManager: NodeManager from hadp-master doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the NodeManager. ## 提示 NodeManager 記憶體不足


解決過程:

1、原虛擬機器755m記憶體,調大虛擬機器記憶體至2G

2、修改 yarn-site.xml ,新增如下引數

<property>
<name>yarn.scheduler.minimum-allocation-mb</name>
<value>200</value>
</property>

<property>
<name>yarn.scheduler.maximum-allocation-mb</name>
<value>1600</value>
</property>

<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>1600</value>
</property>

<property>
<name>yarn.nodemanager.resource.cpu-vcores</name>
<value>1</value>
</property>


引數 : yarn.scheduler.minimum-allocation-mb 
預設 : 1024 m 
官方解釋: The minimum allocation for every container request at the RM, in MBs. Memory requests lower than this will throw a InvalidResourceRequestException.

引數 : yarn.scheduler.maximum-allocation-mb
預設 : 8192 m
官方解釋: The maximum allocation for every container request at the RM, in MBs. Memory requests higher than this will throw a InvalidResourceRequestException.

引數 : yarn.nodemanager.resource.memory-mb
預設 : 8192 m
官方解釋: Amount of physical memory, in MB, that can be allocated for containers.

引數 : yarn.nodemanager.resource.cpu-vcores
預設 : 8
官方解釋: Number of vcores that can be allocated for containers. This is used by the RM scheduler when allocating resources for containers. This is not used to limit the number of physical cores used by YARN containers.

來自 “ ITPUB部落格 ” ,連結:http://blog.itpub.net/31383567/viewspace-2150593/,如需轉載,請註明出處,否則將追究法律責任。

相關文章