For each YARN job, the Hadoop framework generates task log file. Where are Hadoop task log
files stored?
A.
Cached by the NodeManager managing the job containers, then written to a log directory on
the NameNode
B.
Cached in the YARN container running the task, then copied into HDFS on job completion
C.
In HDFS, in the directory of the user who generates the job
D.
On the local disk of the slave mode running the task
Ans D
On the local disk of the slave mode running the task
0
0
D
0
0
D on localdisk
0
0
D
0
0
The question is different in the exam. It says:
Having log aggregation set to default, where are Hadoop log files stored?
There is a parameter to set log aggregation to true or false. So, by default is this is enabled the answer will be option B.
0
0
Hi Rakesh,
Do you have CCA-500 questions and answers. If yes can you please email me at : singhpramodk@yahoo.com
0
0
I disagree with you. The value is FALSE by default, therefore the answer is D.
Search for ” yarn.log-aggregation-enable ” in the defaults setting documentation for YARN here https://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
0
0
Now the question says — when yarn-log-aggreegation is enabled so the log is finally copied to /user/history/done once th job completes so option B is correct
0
0
I disagree with you. The value is FALSE by default, therefore the answer is D.
Search for ” yarn.log-aggregation-enable ” in the defaults setting documentation for YARN here https://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
0
0
I disagree with you. The value is FALSE by default, therefore the answer is D
Search for ” yarn.log-aggregation-enable ” in the defaults setting documentation for YARN here https://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
0
0
cloudera default ” yarn.log-aggregation-enable ” is true。
why ans C incorrect ？
https://www.cloudera.com/documentation/manager/5-1-x/Cloudera-Manager-Configuration-Properties/cm5config_cdh510_yarn_mr2included_.html
0
0
C
cloudera hadoop，yarn.log-aggregation-enable default is true
so，
yarn.nodemanager.remote-app-log-dir ，default is /tmp/logs
and
yarn.nodemanager.remote-app-log-dir-suffix , default is logs
The remote log dir will be created at {yarn.nodemanager.remote-app-log-dir}/${user}/{thisParam}
https://www.cloudera.com/documentation/manager/5-1-x/Cloudera-Manager-Configuration-Properties/cm5config_cdh510_yarn_mr2included_.html
0
0