Transpose in Data Fusion

[Anthony Apollis]

Please assist?

I would like to transpose some data using the transpose add-on in Data Fusion. I just can not get pass the first stage, documentation is very unclear and not helpful.

[Dennis Li]

Hi Anthony, can you please clarify your use case?

[Anthony Apollis]

Hi Dennis  I would like to use Transpose and Pivot function in Data Fusion(CDAP) to get the desired results in tabs (i. Transposed and ii.Test Pivot Data).see: https://lnkd.in/dXrryK7m... There are no tutorials or documentation on these tools to get the results I want. The data is in the format as seen in the data tab(from Google Campaign Manager) in sheet.

The example shot I sent was from this tutorial(https://cdap.atlassian.net/wiki/spaces/DOCS/pages/512295317/Transpose+Transformation) first table. I could not even complete it so complicatedly it's written and not user friendly at all.

--
You received this message because you are subscribed to a topic in the Google Groups "CDAP User" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cdap-user/kuZ86gF78_c/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cdap-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cdap-user/3f02873b-69cb-42ae-ab84-260b213ec8a6n%40googlegroups.com.

--

Anthony Apollis Data Lead 021 447 3344 | tmicollective.com 3rd Floor, Central Building, Black River Park, 2 Fir Street, Observatory, Cape Town, 7925

The content of this email is confidential and intended for the recipient specified in message only. It is strictly forbidden to share any part of this message with any third party, without a written consent of the sender. If you received this message by mistake, please reply to this message and follow with its deletion, so that we can ensure such a mistake does not occur in the future.

[Dennis Li]

Hi Anthony,

For the transpose plugin, the "Fields to be Mapped" field performs a direct 1:1 mapping from provided input column (key) to provided output column (value). The "Fields to be Normalized" field takes the provided input column name (Normalize Field) and writes the name of the column to the output field (Field Name) and the value of the column in a particular row to another output column (Field Value).

Thus, for the screenshot you attached, you will need to update your output schema to include the Attribute_Type and Attribute_Value columns. Following the example documentation you linked should look like the following:

Regarding your use case, you may need to further split your column after you've mapped it to the Attribute_Type column, i.e. you'll need to split "readMore-j1-slide1: Counters" by some delimiter into separate fields and then eventually aggregate it; this may require multiple steps in your pipeline.

To view this discussion on the web visit https://groups.google.com/d/msgid/cdap-user/CAO8x8_z-Un8vzOL4Hprd3DYF3-Apb%2Bn5jeDUvnSBB3Kw5D1qkw%40mail.gmail.com.

--

Dennis Li lide...@google.com +1 (650) 789-0442 Software Engineer, Google Cloud Give me anonymous feedback: go/hello-dennis

[Anthony Apollis]

Thank you for your assistance. Does it mean I need Normalize all of the fields that made up my columns in my data set?

To view this discussion on the web visit https://groups.google.com/d/msgid/cdap-user/CADWXYT%2BB8kAE54J4kEa4JV8evsjn56uG6tMqHc8spVBtUrBCig%40mail.gmail.com.