Unable to Merge outputs

[Vinay Shashank]

Hello,

I have 4 files, which I'm transforming and doing a merge in the end. The flow on a higher level is visualized below.

File1 -> FilterA -> GroupBy ->(File1Temp)                  

File2 ----------------------------> CoGroup(with File1Temp)->(CoGroup1) 

(CoGroup1) ->FilterB -> Output1

(CoGroup1) ->FilterC -> (CoGroup1Temp)

                                                              

File3 ->

File4 ->CoGroup(with File3) ->FilterD -> GroupBy ---->CoGroup(with CoGroup1Temp) -> FilterE -> (Output2)

I finally rename fields to match schemas for Output1 and Output2 and merge them together.

While doing the Merge I'm getting the below error:

6/09/13 14:38:09 Exception =

java.lang.IllegalArgumentException: may not merge a pipe with itself without intermediate operations after the split

        at cascading.pipe.Splice.<init>(Splice.java:283)

        at cascading.pipe.Splice.<init>(Splice.java:231)

        at cascading.pipe.Splice.<init>(Splice.java:192)

        at cascading.pipe.Splice.<init>(Splice.java:181)

        at cascading.pipe.Splice.<init>(Splice.java:436)

        at cascading.pipe.Merge.<init>(Merge.java:61)

Not sure what I'm missing here. Any help on this is highly appreciated.

Thanks,

Vinay

[Ken Krugler]

You might want to check your pipe names to ensure they’re appropriately unique.

— Ken

--
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cascading-use...@googlegroups.com.
To post to this group, send email to cascadi...@googlegroups.com.
Visit this group at https://groups.google.com/group/cascading-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/56b37ecf-8c04-48ff-a07e-f354cd415c99%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--------------------------

Ken Krugler

+1 530-210-6378

http://www.scaleunlimited.com

custom big data solutions & training

Hadoop, Cascading, Cassandra & Solr