CCAH-CCA-500-4題:Where are Hadoop task log files stored?
4.Where are Hadoop task log files stored?
For each YARN job, the Hadoop framework generates task log file. Where are Hadoop task log files stored?
A.Cached by the NodeManager managing the job containers, then written to a log directory on the NameNode
B.Cached in the YARN container running the task, then copied into HDFS on job completion
C.In HDFS, in the directory of the user who generates the job
D.On the local disk of the slave mode running the task
問題:
對於每個yarn job,hadoop框架產生的task日誌檔案儲存在哪個位置上?
解析:
1.對於學習hadoop有段時間的和親手搭建過,就會知道log檔案一般 肯定是儲存在 local disk(本地磁碟)上,而不是儲存在hdfs檔案系統中。
2.一般輸出位置由yarn.nodemanager.log-dirs引數設定(Default Path is ${yarn.log.dir}/userlogs)
<property>
<name>yarn.nodemanager.log-dirs</name>
<value>/home/root/workspace-yarn/nm/log</value>
</property>.
如果不配置將使得NodeManager程式處於Unhealthy狀態,無法提供服務,現象是提交作業時,作業一直處於pending狀態無法往下執行,所以必須配置。
For each YARN job, the Hadoop framework generates task log file. Where are Hadoop task log files stored?
A.Cached by the NodeManager managing the job containers, then written to a log directory on the NameNode
B.Cached in the YARN container running the task, then copied into HDFS on job completion
C.In HDFS, in the directory of the user who generates the job
D.On the local disk of the slave mode running the task
問題:
對於每個yarn job,hadoop框架產生的task日誌檔案儲存在哪個位置上?
解析:
1.對於學習hadoop有段時間的和親手搭建過,就會知道log檔案一般 肯定是儲存在 local disk(本地磁碟)上,而不是儲存在hdfs檔案系統中。
2.一般輸出位置由yarn.nodemanager.log-dirs引數設定(Default Path is ${yarn.log.dir}/userlogs)
<property>
<name>yarn.nodemanager.log-dirs</name>
<value>/home/root/workspace-yarn/nm/log</value>
</property>.
如果不配置將使得NodeManager程式處於Unhealthy狀態,無法提供服務,現象是提交作業時,作業一直處於pending狀態無法往下執行,所以必須配置。
來自 “ ITPUB部落格 ” ,連結:http://blog.itpub.net/30089851/viewspace-2091248/,如需轉載,請註明出處,否則將追究法律責任。
相關文章
- Scripts to resize standby redolog files
- titan-hadoop “Too many open files”修正Hadoop
- Storm-原始碼分析-TopologySubmit-Task-TopologyContext(backtype.storm.task)ORM原始碼MITContext
- DROP AND RECREATE ONLINE REDOLOG FILES
- How to config (no)archivelog mode and manage archive log files by RMANHive
- SQL Where in list 問題SQL
- [LeetCode] 937. Reorder Log FilesLeetCode
- 10 Awesome Examples for Viewing Huge Log Files in UnixView
- Task 1 賽題理解
- Storm-原始碼分析-TopologySubmit-TaskORM原始碼MIT
- weblogic日誌報錯too many open filesWeb
- 解決WebLogic中Too many open files異常Web
- How Views are Stored (175)View
- Oracle OCP IZ0-053 Q311(Rman catalog stored scripts)Oracle
- Repair U Disk logo unvisiable in task bar on windows XPAIGoWindows
- MERGE新特性(UPDATE WHERE,DELETE WHERE,INSERT WHERE)delete
- How to rename an Oracle stored procedureOracle
- Oracle stored procedure to send emailOracleAI
- PostgreSQL DBA(151) - pgAdmin(log_XXX parameter:where)SQL
- Could not execute Write_rows event on table time_task.tt_log
- hadoop叢集,熱插拔硬碟後報錯:Task Tracker local directory is not writableHadoop硬碟
- Mybatis where 1=1 動態sql問題MyBatisSQL
- recover database using backup controlfile利用archivelog files.DatabaseHive
- C# Task若干問題淺析C#
- Camunda User Task:Task Listeners
- Ubuntu 解決 Too many open files 問題Ubuntu
- Initialization Parameter Files and Server Parameter Files (287)Server
- execute shell script from stored procedure
- How Indexes Are Stored (206)Index
- Archive Log Files Created With Permission Bit 440 Instead Of 640 [ID 458901.1]Hive
- Laravel 集合 where 返回的資料格式問題Laravel
- update沒帶where,尋找問題的思路
- Oracle FilesOracle
- 【PLAN STABILITY】 STORED-OUTLINE
- stored procedure 收集session wait 資訊SessionAI
- How to use sql result stored on shell variable?SQL
- Result Sets from Stored Procedures In Oracle (轉)Oracle
- 【REDO】重做日誌檔案(redo log files)管理(增,刪,改,查,切)