PrepAway - Latest Free Exam Questions & Answers

how many key-value pairs will there be in each file?

If you run the word count MapReduce program with m mappers and r reducers, how many output
files will you get at the end of the job? And how many key-value pairs will there be in each file?
Assume k is the number of unique words in the input files.

PrepAway - Latest Free Exam Questions & Answers

A.
There will be r files, each with exactly k/r key-value pairs.

B.
There will be r files, each with approximately k/m key-value pairs.

C.
There will be r files, each with approximately k/r key-value pairs.

D.
There will be m files, each with exactly k/m key value pairs.

E.
There will be m files, each with approximately k/m key-value pairs.

Explanation:
Note:
*A MapReduce job withm mappers and r reducers involves up to m*r distinct copy operations,

since eachmapper may have intermediate output going to every reducer.
*In the canonical example of word counting, a key-value pair is emitted for every word found. For
example, if we had 1,000 words, then 1,000 key-value pairs will be emitted from the mappers to
the reducer(s).

6 Comments on “how many key-value pairs will there be in each file?

  1. roja says:

    C. No guarantee that the no. of keys is always divisible by r, so the last file will have a bit less than the others.




    0



    0
  2. Ramesh Hiremath says:

    C.
    There will be r files, each with approximately k/r key-value pairs.

    Explanation: The word count job emits each unique word once with the count of the number of occurences of that word. There will therefore be k total words in the output. As the job is executing with r reduce tasks, there will be r output files, one for each mapper.
    The word keys are distributed more or less evenly among the reducers, so each output file will contian roughly k/r words. Note that the number of map tasks is irrelevant, as the intermediate output from all map tasks is combined together as part of the shuffle
    phase.




    0



    0

Leave a Reply to roja Cancel reply

Your email address will not be published. Required fields are marked *