Denormalizing cascading flow

[Ron Gonzalez]

Hi,

  If I have two avro files:

  Table X:

  Id, Name

  a, Value1

  b. Value2

  c, Value3

  Table Y:

  

  Id, Name, X_Id

  d, Value4, a

  e, Value5, a

  f, Value6, b

  g, Value7, b

  h, Value8, b

  Then I want to get

  a, Value1, [ { d, Value4 }, { e, Value5 }]

  b, Value2, [ { f, Value6 }, { g, Value7 }, { h, Value8 }]

 How could I do this with cascading? Schema of Y is fixed across all rows, so creating a target avro schema is acceptable.

Thanks,

Ron

[Prabodh Mhalgi]

How about using a GroupBy with X.Id as the key field. A custom aggregator will be required to implement a custom logic to derive the third field required in the output.

Thanks,

Prabodh

[Ron Gonzalez]

Thanks Prabodh.

I was able to solve it by doing a group by and a custom Buffer implementation.

For the cascading avro piece, if it's a list, then you have to make it a list of Tuple instances for cascading avro to support it...

--Ron