Manjunathan Ramasami
unread,Aug 16, 2021, 6:04:14 AM8/16/21Sign in to reply to author
Sign in to forward
You do not have permission to delete messages in this group
Sign in to report message as abuse
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to
Hi Eddie,
Thanks for the suggestions. Your sample code worked. Now I'm getting an another issue. CSV parser when dumping entries to output file it is missing some headers. Key in my input JSON is not constant as you see below.
INPUT JSON--->
{
"USERID":"AAAAAAA",
"DEPTNO":"5656565",
"ITDEPT":{
"SUBCATEGORY":["abc", "123"],
"ENDNO":"777777"
}
}
{
"USERID":"BBBBBBB",
"DEPTNO":"5656565",
"ITDEPT":{
"SUBCATEGORY":"2",
"ENDNO":"77777"
},
"DESC":{
"MESSAGE":"NA"
}
}
=====================================================
18:02:28,700 INFO - CTGDIS087I Iterating.
18:02:28,717 INFO - [FileConnector_1] CTGDIS351I Map Attribute OUTPUT [1].
18:02:28,717 INFO - [FileConnector_1] CTGDIS353I Script is: task.logmsg("FLATTENED ENTRY TO WRITE TO CSV:: " + flattenedEntry);
ret.value = flattenedEntry;
18:02:28,717 INFO - FLATTENED ENTRY TO WRITE TO CSV:: {
"DEPTNO": "5656565",
"USERID": "AAAAAAA",
"ITDEPT_SUBCATEGORY": "abc&123",
"ITDEPT_ENDNO": "777777"
}
18:02:28,717 INFO - [FileConnector_1] CTGDIS126I Return {
"DEPTNO": "5656565",
"USERID": "AAAAAAA",
"ITDEPT_SUBCATEGORY": "abc&123",
"ITDEPT_ENDNO": "777777"
}.
18:02:28,717 INFO - [FileConnector_1] CTGDIS123I Returned object class com.ibm.di.entry.Entry.
18:02:28,717 INFO - [FileConnector_1] CTGDIS504I *Result of attribute mapping*
18:02:28,717 INFO - [FileConnector_1] CTGDIS505I The 'conn' object
18:02:28,717 INFO - [FileConnector_1] CTGDIS003I *** Start dumping Entry
18:02:28,717 INFO - {
18:02:28,717 INFO - "DEPTNO": "5656565",
18:02:28,717 INFO - "USERID": "AAAAAAA",
18:02:28,717 INFO - "ITDEPT_SUBCATEGORY": "abc&123",
18:02:28,717 INFO - "ITDEPT_ENDNO": "777777"
18:02:28,717 INFO - }
18:02:28,717 INFO - [FileConnector_1] CTGDIS004I *** Finished dumping Entry
18:02:28,717 INFO - [FileConnector_1] CTGDIS506I The 'work' object
18:02:28,717 INFO - [FileConnector_1] CTGDIS003I *** Start dumping Entry
18:02:28,717 INFO - {
18:02:28,731 INFO - "DEPTNO": "5656565",
18:02:28,731 INFO - "USERID": "AAAAAAA",
18:02:28,731 INFO - "ITDEPT": {
18:02:28,731 INFO - "SUBCATEGORY": [
18:02:28,731 INFO - "abc",
18:02:28,731 INFO - "123"
18:02:28,731 INFO - ],
18:02:28,731 INFO - "ENDNO": "777777"
18:02:28,731 INFO - }
18:02:28,731 INFO - }
18:02:28,731 INFO - [FileConnector_1] CTGDIS004I *** Finished dumping Entry
18:02:28,731 INFO - [FileConnector_1] CTGDJW004I No column names provide. Will use first output entry for column names: '[ITDEPT_SUBCATEGORY, ITDEPT_ENDNO, USERID, DEPTNO]'.
18:02:28,731 INFO - [FileConnector_1] CTGDIS351I Map Attribute OUTPUT [2].
18:02:28,731 INFO - FLATTENED ENTRY TO WRITE TO CSV:: {
"DEPTNO": "5656565",
"USERID": "BBBBBBB",
"ITDEPT_SUBCATEGORY": "2",
"DESC_MESSAGE": "NA",
"ITDEPT_ENDNO": "77777"
}
18:02:28,731 INFO - [FileConnector_1] CTGDIS126I Return {
"DEPTNO": "5656565",
"USERID": "BBBBBBB",
"ITDEPT_SUBCATEGORY": "2",
"DESC_MESSAGE": "NA",
"ITDEPT_ENDNO": "77777"
}.
18:02:28,731 INFO - [FileConnector_1] CTGDIS123I Returned object class com.ibm.di.entry.Entry.
18:02:28,731 INFO - [FileConnector_1] CTGDIS504I *Result of attribute mapping*
18:02:28,731 INFO - [FileConnector_1] CTGDIS505I The 'conn' object
18:02:28,731 INFO - [FileConnector_1] CTGDIS003I *** Start dumping Entry
18:02:28,731 INFO - {
18:02:28,731 INFO - "DEPTNO": "5656565",
18:02:28,731 INFO - "USERID": "BBBBBBB",
18:02:28,731 INFO - "ITDEPT_SUBCATEGORY": "2",
18:02:28,731 INFO - "DESC_MESSAGE": "NA",
18:02:28,731 INFO - "ITDEPT_ENDNO": "77777"
18:02:28,731 INFO - }
18:02:28,731 INFO - [FileConnector_1] CTGDIS004I *** Finished dumping Entry
18:02:28,731 INFO - [FileConnector_1] CTGDIS506I The 'work' object
18:02:28,731 INFO - [FileConnector_1] CTGDIS003I *** Start dumping Entry
18:02:28,731 INFO - {
18:02:28,731 INFO - "DEPTNO": "5656565",
18:02:28,731 INFO - "USERID": "BBBBBBB",
18:02:28,731 INFO - "ITDEPT": {
18:02:28,731 INFO - "SUBCATEGORY": "2",
18:02:28,731 INFO - "ENDNO": "77777"
18:02:28,731 INFO - },
18:02:28,731 INFO - "DESC": {
18:02:28,731 INFO - "MESSAGE": "NA"
18:02:28,731 INFO - }
18:02:28,731 INFO - }
18:02:28,731 INFO - [FileConnector_1] CTGDIS004I *** Finished dumping Entry
18:02:28,731 INFO - CTGDIS088I Finished iterating.
18:02:28,731 INFO - [FileConnector_1] CTGDIS525I Closing Parser on output stream.
18:02:28,731 INFO - CTGDIS100I Printing the Connector statistics.
18:02:28,731 INFO - [FileConnector] Get:2
18:02:28,731 INFO - [EmptyScript] Calls: 2
18:02:28,731 INFO - [ParseJSONtoCSV_1] Calls: 2
18:02:28,731 INFO - [FileConnector_1] Add:2
18:02:28,731 INFO - [ClearValues] Calls: 2
18:02:28,731 INFO - CTGDIS104I Total: Get:2, Add:2.
18:02:28,731 INFO - CTGDIS101I Finished printing the Connector statistics.
18:02:28,731 INFO - CTGDIS080I Terminated successfully (0 errors).
And the final output in the CSV is missing header DESC_MESSAGE and its value as this key was not present for 1st entry. Is there a way to iterate through all JSON to get all the available keys which later can be used to define headers in CSV? This JSON is from some legacy system, hence standardizing keys is not possible.
ITDEPT_SUBCATEGORY|ITDEPT_ENDNO|USERID|DEPTNO
abc&123|777777|AAAAAAA|5656565
2|77777|BBBBBBB|5656565