Json to CSV Conversion

[Manas Kumar]

Hi All,

 

I am new to Streamsets, I have sample JSON file, I want to convert JSON file to CSV file, finding difficult to achieve, could you please help anyone here

 

Input:

 

{

    "Name": "Jane",

    "Country": "US",

    

    "children": [

        {

            "firstName": "Alice",

            "age": 6

        },

        {

            "firstName": "Bob",

            "age": 8

        }

    ]

}

 

 

Expected OutPut:

 

Name, Age, FatherName, Country

----------------------------

Alice, 6, Jane, US

Bob,6, Jane, US

[Mitch Barnett]

Hi Manas,

This question on ask.streamsets.com should help clarify how you implement this:

https://ask.streamsets.com/question/5391/can-streamsets-take-json-file-as-input-and-output-as-csv/

Hope this helps.

Thanks,

Mitch Barnett

Software Engineer - Support

mi...@streamsets.com

--
You received this message because you are subscribed to the Google Groups "sdc-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sdc-user+u...@streamsets.com.
To view this discussion on the web visit https://groups.google.com/a/streamsets.com/d/msgid/sdc-user/cda19a88-1c53-4fc3-9e9a-9163d7f23fd4o%40streamsets.com.

[Manas Kumar]

Hi Mitch Barnett

I tried with processors, as json have nested structure,unable to get the excepted output

 

Thanks,

Manas

To unsubscribe from this group and stop receiving emails from it, send an email to sdc-...@streamsets.com.

[Mitch Barnett]

Hi Manas,

That's covered in the link I sent over to you:

If the JSON is not ‘flat’, you will need to use Field Flattener

 

You'd need to get the JSON objects into a "flattened" state using a field flattener processor:

https://streamsets.com/documentation/datacollector/latest/help/datacollector/UserGuide/Processors/FieldFlattener.html#concept_njn_3kk_fx

Thank you,

Mitch Barnett

Software Engineer - Support

mi...@streamsets.com

To unsubscribe from this group and stop receiving emails from it, send an email to sdc-user+u...@streamsets.com.
To view this discussion on the web visit https://groups.google.com/a/streamsets.com/d/msgid/sdc-user/c0c28789-4de4-4794-8d15-72e43fa06ecco%40streamsets.com.

[Manas Kumar]

Hi Mitch Barnett

 

I tried with Field Flattener and other processors, I am getting single record only, I need to get 2 records, unable to get the excepted output.

 

 

Here I used bit of python code to get the result.

 

Output:

 

Is there any similar way to get the expect result in Streamsets, While using processor I am unable to get the expected result

 

Hi Pat ,

Could you please help here

Thanks,

Manas

To view this discussion on the web visit https://groups.google.com/a/streamsets.com/d/msgid/sdc-user/c0c28789-4de4-4794-8d15-72e43fa06ecco%40streamsets.com.

[Marty Norris]

Hi Manas.

Here is one way you could do it.

I did this using a 'Field Pivoter', 'Expression Evaluator' and 'Field Remover'.  The output would look like this:

The pipeline looks like this:

I have attached the pipeline export here that you can import into your environment.

To unsubscribe from this group and stop receiving emails from it, send an email to sdc-user+u...@streamsets.com.
To view this discussion on the web visit https://groups.google.com/a/streamsets.com/d/msgid/sdc-user/bb833740-90b4-4d6b-a875-a99c47909996o%40streamsets.com.

--

--

Marty Norris Solutions Engineer
OFFICE: 410.629.2934                            MOBILE: 443.992.6731 EMAIL: ma...@streamsets.com

[Manas Kumar]

Wow... Thank you so much Marty Norries