how best to implement variable number of steps?

[kiren]

Hi,

What is the recommended way of conditionally executing flows, based on
the results of previous flows? I have pipe assembly that should
execute a variable number of times, depending on the characteristic of
the input data. There are (fixed) assemblies that are to run before
and after as well. I am looking into using counters, though I'm
running into confusion because my pipe assembly that needs repeating
is made up of several map-reduce jobs.

Thanks in advance for your advice!
Kiren

[Chris K Wensel]

Hey Kiren

If you use Cascading wip 1.1, the FlowStats object will sum up
counters across all steps (mr jobs).

Couple this with simple loops and I think you are set.

There is no magic abstraction for this, like Cascade is for simply
chained Flows.

This is why Cascading is an API and not xml (how do you do loops in
Ant?) or a custom syntax.

cheers,
chris

--
Chris K Wensel
ch...@concurrentinc.com
http://www.concurrentinc.com

[kiren]

Thanks Chris, that was exactly the API I needed.