PrepAway - Latest Free Exam Questions & Answers

Combiners Increase the efficiency of a MapReduce program because:

Combiners Increase the efficiency of a MapReduce program because:

PrepAway - Latest Free Exam Questions & Answers

A.
They provide a mechanism for different mappers to communicate with each Other, thereby
reducing synchronization overhead.

B.
They provide an optimization and reduce the total number of computations that are needed to
execute an algorithm by a factor of n, where is the number of reducer.

C.
They aggregate intermediate map output locally on each individual machine and therefore
reduce the amount of data that needs to be shuffled across the network to the reducers.

D.
They aggregate intermediate map output horn a small number of nearby (i.e., rack-local)
machines and therefore reduce the amount of data that needs to be shuffled across the network to
the reducers.

Explanation:
Combiners are used to increase the efficiency of a MapReduce program. They are used to
aggregate intermediate map output locally on individual mapper outputs. Combiners can help you
reduce the amount of data that needs to be transferred across to the reducers. You can use your
reducer code as a combiner if the operation performed is commutative and associative. The
execution of combiner is not guaranteed, Hadoop may or may not execute a combiner. Also, if
required it may execute it more then 1 times. Therefore your MapReduce jobs should not depend
on the combiners execution.

Reference:24 Interview Questions & Answers for Hadoop MapReduce developers,What are
combiners? When should I use a combiner in my MapReduce Job?
http://www.fromdev.com/2010/12/interview-questions-hadoop-mapreduce.html(question no. 12)

3 Comments on “Combiners Increase the efficiency of a MapReduce program because:

  1. Ramesh Hiremath says:

    C.
    They aggregate intermediate map output locally on each individual machine and therefore
    reduce the amount of data that needs to be shuffled across the network to the reducers.




    0



    0

Leave a Reply

Your email address will not be published. Required fields are marked *