Scalding: Pipe-specific split size for a multi-step flow

[ravi kiran holur vijay]

Hello,

Is it possible to set different split sizes for different stages of a multi-job flow in Scalding? If so, what's the best way to do this in Scalding?

In Cascading, this would look something like:

pipe.getStepConfigDef().setProperty("mapred.min.split.size", size);

pipe.getStepConfigDef().setProperty("mapred.max.split.size", size);

An example flow would be something like: Map(1) -> Reduce(1) -> Map(2) -> Reduce(2). We would want SplitSize(Map(2)) << SplitSize(Map(1))

Thanks!

Ravi