Please Visit: http://lifelongprogrammer.blogspot.com
Sorting using Hadoop – TotalOrderPartitioner
http://pipiper.wordpress.com/2013/05/02/sorting-using-hadoop-totalorderpartitioner/
job.setPartitionerClass(TotalOrderPartitioner.class);
job.setSortComparatorClass(SortKeyComparator.class);
job.setPartitionerClass(TotalOrderPartitioner.class);
Path inputDir = new Path(partitionLocation);
Path partitionFile = new Path(inputDir, "partitioning");
TotalOrderPartitioner.setPartitionFile(job.getConfiguration(), partitionFile);
int numSamples = numReduceTasks;
int maxSplits = numReduceTasks - 1;
InputSampler.Sampler sampler = new InputSampler.RandomSampler(pcnt, numSamples, maxSplits);
InputSampler.writePartitionFile(job, sampler);
from Google Plus RSS Feed for 101157854606139706613 http://pipiper.wordpress.com/2013/05/02/sorting-using-hadoop-totalorderpartitioner
via LifeLong Community
Sorting using Hadoop – TotalOrderPartitioner
http://pipiper.wordpress.com/2013/05/02/sorting-using-hadoop-totalorderpartitioner/
job.setPartitionerClass(TotalOrderPartitioner.class);
job.setSortComparatorClass(SortKeyComparator.class);
job.setPartitionerClass(TotalOrderPartitioner.class);
Path inputDir = new Path(partitionLocation);
Path partitionFile = new Path(inputDir, "partitioning");
TotalOrderPartitioner.setPartitionFile(job.getConfiguration(), partitionFile);
int numSamples = numReduceTasks;
int maxSplits = numReduceTasks - 1;
InputSampler.Sampler sampler = new InputSampler.RandomSampler(pcnt, numSamples, maxSplits);
InputSampler.writePartitionFile(job, sampler);
from Google Plus RSS Feed for 101157854606139706613 http://pipiper.wordpress.com/2013/05/02/sorting-using-hadoop-totalorderpartitioner
via LifeLong Community