generate multiple records within a Command

13 views
Skip to first unread message

Norbert Burger

unread,
Mar 13, 2015, 2:56:54 PM3/13/15
to cdk...@cloudera.org
Hi folks - I have a JSON datasource which contains payloads with an embedded JSON list.  Within a Morphlines command, I'd like to "explode" these payloads so that one record becomes many records, each containing one element from the embedded list.  Any suggestions on how to do this easily?

Norbert

Norbert Burger

unread,
Mar 13, 2015, 3:18:43 PM3/13/15
to cdk...@cloudera.org
I should clarify that I'm currently using the dynamic Java code approach.  I suspect if I switch to a real Command, I could just use super.doProcess() within a loop to explode my records, like SeparateAttachments.  Any way to emulate this within a Morphlines config file?

Norbert


Wolfgang Hoschek

unread,
Mar 13, 2015, 3:53:12 PM3/13/15
to Norbert Burger, cdk...@cloudera.org
In your "java" command you can call child.process(Record) as often as you like. Typically, along these lines:

...
for (Object value : record.get("foo")) {
  Record childRecord = record.copy(); // or maybe childRecord = new Record();
  childRecord.put("bar", value);
  if (!child.process(childRecord)) {
    return false;
  }
}
return true;



--
You received this message because you are subscribed to the Google Groups "CDK Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cdk-dev+u...@cloudera.org.
For more options, visit https://groups.google.com/a/cloudera.org/d/optout.

Norbert Burger

unread,
Mar 13, 2015, 4:55:48 PM3/13/15
to Wolfgang Hoschek, cdk...@cloudera.org
Thanks Wolfgang,

Norbert
Reply all
Reply to author
Forward
0 new messages