Datafusion issue on nested structure

41 views
Skip to first unread message

Elisa Baeckmann

unread,
Jan 18, 2021, 3:08:15 AM1/18/21
to CDAP User

Good morning,

We are developing the population of a table with the creation of a NESTED structure in 2 levels, setting all the fields of type Array and Record NOT NULL and setting NULLABLE the nested fields. For the sake of completeness, an example follows: "Order Product" table nested to the "Order" table which in turn is nested to the "Order Header" table. We noticed that if the first level ("Order Header") does not have any match to the second ("Order"), the pipeline fails with a NullPointerException: null. The error log follows.  

error.png

Could the cause of the error be that, if the internal nested structure ("Order"-"OrderProduct" relationship) is not created, the nesting on the primary table ("Order Header") fails?

How is it possible to overcome this problem?

For any further information you might need, let us know.

Thanks in advance,

Elisa Baeckmann


Albert Shau

unread,
Jan 19, 2021, 1:58:48 PM1/19/21
to cdap...@googlegroups.com
Hi Elisa,

For the schema, do you have the corresponding record set as nullable? The javascript transform will assume the value is not null unless the checkbox next to the field name (see attached screenshot) is set.

Regards,
Albert

--
You received this message because you are subscribed to the Google Groups "CDAP User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cdap-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cdap-user/cab4908c-8e4d-4533-974e-fb7637a79439n%40googlegroups.com.
Screen Shot 2021-01-19 at 10.56.47 AM.png
Reply all
Reply to author
Forward
0 new messages