CCAH-CCA-500-4題:Where are Hadoop task log files stored?

hackeruncle發表於2016-04-30
4.Where are Hadoop task log files stored?

For each YARN job, the Hadoop framework generates task log file. Where are Hadoop task log files stored?

A.
Cached by the NodeManager managing the job containers, then written to a log directory on the NameNode
B.Cached in the YARN container running the task, then copied into HDFS on job completion
C.In HDFS, in the directory of the user who generates the job
D.On the local disk of the slave mode running the task


問題:
   
對於每個yarn job,hadoop框架產生的task日誌檔案儲存在哪個位置上?

解析:
   
1.對於學習hadoop有段時間的和親手搭建過,就會知道log檔案一般 肯定是儲存在 local disk(本地磁碟)上,而不是儲存在hdfs檔案系統中。
    2.一般輸出位置由yarn.nodemanager.log-dirs引數設定(Default Path is ${yarn.log.dir}/userlogs)
    <property>
      <name>yarn.nodemanager.log-dirs</name>
      <value>/home/root/workspace-yarn/nm/log</value>
    </property>.
   如果不配置將使得NodeManager程式處於Unhealthy狀態,無法提供服務,現象是提交作業時,作業一直處於pending狀態無法往下執行,所以必須配置。

來自 “ ITPUB部落格 ” ,連結:http://blog.itpub.net/30089851/viewspace-2091248/,如需轉載,請註明出處,否則將追究法律責任。

相關文章