PrepAway - Latest Free Exam Questions & Answers

What metadata is stored on a DataNode when a block is written to it?

What metadata is stored on a DataNode when a block is written to it?

PrepAway - Latest Free Exam Questions & Answers

A.
None. Only the block itself is written.

B.
Checksums for the data in the block, as a separate file.

C.
Information on the file’s location in HDFS.

D.
Node location of each block belonging to the same namespace.

Explanation:
Each DataNode keeps a small amount of metadata allowing it to identify the cluster
it participates in. If this metadata is lost, then the DataNode cannot participate in an HDFS
instance and the data blocks it stores cannot be reached.
When an HDFS instance is formatted, the NameNode generates a unique namespace id for the
instance. When DataNodes first connect to the NameNode, they bind to this namespace id and
establish a unique “storage id” that identifies that particular DataNode in the HDFS instance. This
data as well as information about what version of Hadoop was used to create the block files, is
stored in a filed named VERSION in the ${dfs.data.dir}/current directory.
Note: Administrators of HDFS clusters understand that the HDFS metadata is some of the most
precious bits they have. While you might have hundreds of terabytes of information stored in
HDFS, the NameNode’s metadata is the key that allows this information, spread across several
million “blocks” to be reassembled into coherent, ordered files.
Protecting per-DataNode Metadata

2 Comments on “What metadata is stored on a DataNode when a block is written to it?


Leave a Reply

Your email address will not be published. Required fields are marked *