Temporary (inline/ephemeral) pipe

[jd]

Is it possible to declare a temporary pipe inline without binding it to a tap?

A use case would look like this:

        List<Object> tempData = //some list of temp data

        Pipe    fillData    = new Pipe("tempData");

        //create temp data set inline...

        fillData = new Each( fillData, new Insert(tempField, tempData ) );

        //Now join the temp data to an existing pipe bound to a tap.

        Pipe joinedPipe = new CoGroup(....)

[Chris K Wensel]

If the data fits in memory, as it seems it does, you are better off just creating your own hash join with a HashMap in a custom Function. vs hacking data into the pipeline and forcing a new MR job via a CoGroup.

ckw

--
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To view this discussion on the web visit https://groups.google.com/d/msg/cascading-user/-/YmcB2vVTwhsJ.
To post to this group, send email to cascadi...@googlegroups.com.
To unsubscribe from this group, send email to cascading-use...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cascading-user?hl=en.

--

Chris K Wensel

ch...@concurrentinc.com

http://concurrentinc.com