If you run the word count MapReduce program with m mappers and r reducers, how many output
files will you get at the end of the job? And how many key-value pairs will there be in each file?
Assume k is the number of unique words in the input files.
A.
There will be r files, each with exactly k/r key-value pairs.
B.
There will be r files, each with approximately k/m key-value pairs.
C.
There will be r files, each with approximately k/r key-value pairs.
D.
There will be m files, each with exactly k/m key value pairs.
E.
There will be m files, each with approximately k/m key-value pairs.
Explanation:
Note:
*A MapReduce job withm mappers and r reducers involves up to m*r distinct copy operations,since eachmapper may have intermediate output going to every reducer.
*In the canonical example of word counting, a key-value pair is emitted for every word found. For
example, if we had 1,000 words, then 1,000 key-value pairs will be emitted from the mappers to
the reducer(s).
A
0
0
C
0
0
answer is C, check
http://www.quora.com/If-you-run-the-word-count-MapReduce-program-with-m-mappers-and-r-reducers-how-many-output-files-will-you-get-at-the-end-of-the-job-And-how-many-key-value-pairs-will-there-be-in-each-file-Assume-k-is-the-number-of-unique-words-in-the-input-files
0
0
C. No guarantee that the no. of keys is always divisible by r, so the last file will have a bit less than the others.
0
0
C.
There will be r files, each with approximately k/r key-value pairs.
Explanation: The word count job emits each unique word once with the count of the number of occurences of that word. There will therefore be k total words in the output. As the job is executing with r reduce tasks, there will be r output files, one for each mapper.
The word keys are distributed more or less evenly among the reducers, so each output file will contian roughly k/r words. Note that the number of map tasks is irrelevant, as the intermediate output from all map tasks is combined together as part of the shuffle
phase.
0
0
I agree with the answer. A
0
0