PrepAway - Latest Free Exam Questions & Answers

Assuming default settings, which best describes the order of data provided to a reducer’s reduce method:

Assuming default settings, which best describes the order of data provided to a reducer’s reduce
method:

PrepAway - Latest Free Exam Questions & Answers

A.
The keys given to a reducer aren’t in a predictable order, but the values associated with those
keys always are.

B.
Both the keys and values passed to a reducer always appear in sorted order.

C.
Neither keys nor values are in any predictable order.

D.
The keys given to a reducer are in sorted order but the values associated with each key are in
no predictable order

Explanation:
Reducer has 3 primary phases:

1. Shuffle
The Reducer copies the sorted output from each Mapper using HTTP across the network.
2. Sort
The framework merge sorts Reducer inputs by keys (since different Mappers may have output the
same key).
The shuffle and sort phases occur simultaneously i.e. while outputs are being fetched they are
merged.
SecondarySort
To achieve a secondary sort on the values returned by the value iterator, the application should
extend the key with the secondary key and define a grouping comparator. The keys will be sorted
using the entire key, but will be grouped using the grouping comparator to decide which keys and
values are sent in the same call to reduce.
3. Reduce
In this phase the reduce(Object, Iterable, Context) method is called for each <key, (collection of
values)> in the sorted inputs.
The output of the reduce task is typically written to a RecordWriter via
TaskInputOutputContext.write(Object, Object).
The output of the Reducer is not re-sorted.
Reference: org.apache.hadoop.mapreduce, Class
Reducer<KEYIN,VALUEIN,KEYOUT,VALUEOUT>

3 Comments on “Assuming default settings, which best describes the order of data provided to a reducer’s reduce method:


Leave a Reply

Your email address will not be published. Required fields are marked *