modified HadoopMapper operator to work in two modes :
1) SelfReadMode: The mapper reads the input directly from HDFS instead of receiving it from another specific read operator
2) DependentMode : The mapper is not anymore a source operator, but requires input to be fed by some other operator ( eg a reducer in case of chained MR jobs )
For operators A & B that connect using a one-to-one connector, A & B can be fused together to form a single operator. The above change maked HadoopReadOperator redundant.
It is not being deleted here as it is a useful operator for reading from HDFS and could be used in other scenarios.
Modified AbstractHadoopReadOperator to take as argument in the constructor , the input arity. The input arity was earlier assumed to be 1 for Map and Reduce, but is
0 for Map in the SelfReadMode.
Modified Reducer to pass the inputArity to base class constructor
git-svn-id: https://hyracks.googlecode.com/svn/trunk/hyracks@176 123451ca-8445-de46-9d55-352943316053
3 files changed