|
|||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||
java.lang.Objectorg.apache.hadoop.conf.Configured
org.htuple.examples.SecondarySort
public final class SecondarySort
An example MapReduce job showing how the Tuple and ShuffleUtils can be used in conjunction with
each other to secondary sort people's names.
| Nested Class Summary | |
|---|---|
static class |
SecondarySort.Map
This map class simply tokenizes each input line, and emits a (tuple, line) pair, where the tuple contains the last and first name. |
static class |
SecondarySort.Reduce
The reducer just emits the map output values, allowing us to examine the resulting output and determine the results of the secondary sort. |
| Field Summary | |
|---|---|
static String[] |
EXAMPLE_NAMES
Sample input used by this example job. |
| Constructor Summary | |
|---|---|
SecondarySort()
|
|
| Method Summary | |
|---|---|
static void |
main(String[] args)
Main entry point for the example. |
int |
run(String[] args)
The MapReduce driver - setup and launch the job. |
static void |
setupSecondarySort(org.apache.hadoop.conf.Configuration conf)
Partition and group on just the last name; sort on both last and first name. |
static Tuple |
stringToTuple(String line)
Split the input line and return a Tuple representation of the last and first names. |
static void |
writeInput(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.Path inputDir)
Writes the contents of EXAMPLE_NAMES into a file in the job input directory in HDFS. |
| Methods inherited from class org.apache.hadoop.conf.Configured |
|---|
getConf, setConf |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Methods inherited from interface org.apache.hadoop.conf.Configurable |
|---|
getConf, setConf |
| Field Detail |
|---|
public static final String[] EXAMPLE_NAMES
| Constructor Detail |
|---|
public SecondarySort()
| Method Detail |
|---|
public static void main(String[] args)
throws Exception
args - arguments
Exception - when something goes wrong
public static void writeInput(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.Path inputDir)
throws IOException
EXAMPLE_NAMES into a file in the job input directory in HDFS.
conf - the Hadoop configinputDir - the HDFS input directory where we'll write a file
IOException - if something goes wrong
public int run(String[] args)
throws Exception
run in interface org.apache.hadoop.util.Toolargs - the command-line arguments
Exception - if something goes wrongpublic static void setupSecondarySort(org.apache.hadoop.conf.Configuration conf)
conf - the Hadoop configpublic static Tuple stringToTuple(String line)
Tuple representation of the last and first names.
line - a line containing a tab-delimited last and first name.
Tuple representation of the line
|
|||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||