Home » Tables » Hbase Tablesnapshotinputformat

Hbase Tablesnapshotinputformat

TableSnapshotInputFormat allows a MapReduce job to run over a table snapshot.

The job bypasses HBase servers, and directly accesses the underlying files (hfile, recovered edits, wals, etc) directly to provide maximum performance.

The snapshot is not required to be restored to the live cluster or cloned.

, TableSnapshotInputFormat allows a MapReduce job to run over a table snapshot.

Further documentation available on TableSnapshotInputFormat.

See Also:.

, String snapshotName, org.

apache.

hadoop.

fs.

Path restoreDir, org.

apache.

hadoop.

hbase.

util.

RegionSplitter.

SplitAlgorithm splitAlgo, int numSplitsPerRegion) Configures the job to use.

, TableSnapshotInputFormat allows a MapReduce job to run over a table snapshot.

The job bypasses HBase servers, and directly accesses the underlying files (hfile, recovered edits, wals, etc) directly to provide maximum performance.

The snapshot is not required to be restored to the live cluster or cloned.

, TableSnapshotInputFormat allows a MapReduce job to run over a table snapshot.

Further documentation available on TableSnapshotInputFormat.

, On the other hand, in favored node assignment, 3 nodes will contain highly local blocks.

Here we are doing a simple heuristic, where we will pass all hosts which have at least 80% ( hbase.

tablesnapshotinputformat.

locality.

cutoff.

multiplier) as much block locality as #, TableSnapshotInputFormat runs one map task per region in the table snapshot.

This places unnecessary restriction that the region layout of the original table needs to take the processing resources available to MR job into consideration.

Allowing to run multiple mappers per region (assuming reasonably even key distribution) would be useful.

, TableSnapshotInputFormat static class TableSnapshotInputFormat.

TableSnapshotRecordReader extends Object implements org.

apache.

hadoop.

mapred.

RecordReader Field Summaryhbase tablesnapshotinputformat