PrepAway - Latest Free Exam Questions & Answers

why might using a combiner reduce the overall Job running time?

In the standard word count MapReduce algorithm, why might using a combiner reduce the overall
Job running time?

PrepAway - Latest Free Exam Questions & Answers

A.
Because combiners perform local aggregation of word counts, thereby allowing the mappers to
process input data faster.

B.
Because combiners perform local aggregation of word counts, thereby reducing the number of
mappers that need to run.

C.
Because combiners perform local aggregation of word counts, and then transfer that data to
reducers without writing the intermediate data to disk.

D.
Because combiners perform local aggregation of word counts, thereby reducing the number of
key-value pairs that need to be snuff let across the network to the reducers.

Explanation:
*Simply speaking a combiner can be considered as a“mini reducer”that will be applied potentially
several times still during the map phase before to send the new (hopefully reduced) set of
key/value pairs to the reducer(s). This is why a combiner must implement the Reducer interface
(or extend the Reducer class as of hadoop 0.20).
*Combiners are used to increase the efficiency of a MapReduce program. They are used to
aggregate intermediate map output locally on individual mapper outputs. Combiners can help you
reduce the amount of data that needs to be transferred across to the reducers. You can use your
reducer code as a combiner if the operation performed is commutative and associative. The
execution of combiner is not guaranteed, Hadoop may or may not execute a combiner. Also, if
required it may execute it more then 1 times. Therefore your MapReduce jobs should not depend
on the combiners execution.
Reference:24 Interview Questions & Answers for Hadoop MapReduce developers,What are
combiners? When should I use a combiner in my MapReduce Job?

4 Comments on “why might using a combiner reduce the overall Job running time?

  1. Ramesh Hiremath says:

    D.
    Because combiners perform local aggregation of word counts, thereby reducing the number of
    key-value pairs that need to be snuff let across the network to the reducers.




    0



    0

Leave a Reply

Your email address will not be published. Required fields are marked *