bsdfish / scalahadoop Goto Github PK

It is not quite easy to customize the Job right now. For instance, to specify a custom GroupingComparator for doing secondary sort. One possible workaround to this could be to implement a method getJob() that does pretty much the same that is done in execute() of MapReduceTaskChain. This way, the user of the API can add more parameters to the Job before launching it.

In abstract class MapReduceTask,
If reducer != null : Two lines of code need to be added :

    job setMapOutputKeyClass   mapper.kType;
    job setMapOutputValueClass mapper.vType

Otherwise when map output differs from final output Hadoop will throw an error.