How to properly handle nested objects when working between a database and JSON.

[Jamie Stackhouse]

I'm struggling with marshalling and unmarshalling JSON when dealing with composing structs. Specifically, in the Playlist struct below, when populating Videos, I receive this as a JSON array, and I'm struggling with the fact that I have duplicated field names.

For example...

type Video struct {

Id      string `json:"id"`

Network string `json:"-"`

Url     string `json:"url"` //derived field from configuration and id

Title   string `json:"title"`

Status  int    `json:"status"`

Times

PublishedAt NullTime    `json:"published_at"`

Tags        StringSlice `json:"tags"`

Author      Author      `json:"author"`

Updater     Updater     `json:"updater"`

}

When creating the struct, the Author and Updater both have fields that are annotated with the name "id" and populated from a Postgres database using row.Scan, so the duplicated field names are not an issue, as the position of each column is used rather than the column names.

However, I'm trying to create the following JSON structure which is the below structure and struggling with how to make it happen in Go while using structured data types because of the same dupped field name issue.

type Playlist struct {

Id      string `json:"id"`

Network string `json:"-"`

Url     string `json:"url"`

Title   string `json:"title"`

Times

PublishedAt NullTime `json:"published_at"`

Author      Author   `json:"author"`

Updater     Updater  `json:"updater"`

Videos      Videos   `json:"videos"`

}

The Videos type is defined as []*Video.

[

    {

        "id": "492b3478-acd6-40da-b357-a72d8655c018",

        "url": "http://localhost:8080/company/playlists/492b3478-acd6-40da-b357-a72d8655c018",

        "title": "sci-fi",

        "created_at": "2014-12-03T09:49:21.58386Z",

        "updated_at": "2014-12-03T09:49:21.58386Z",

        "deleted_at": null,

        "published_at": null,

        "author": {

            "id": "539ea7af-d366-4d13-9446-d427fb72a910",

            "url": "http://localhost:8080/company/users/539ea7af-d366-4d13-9446-d427fb72a910",

            "email": "sn...@gmail.com",

            "profile": {

                "title": "geek"

            }

        },

        "updater": {

            "id": "539ea7af-d366-4d13-9446-d427fb72a910",

            "url": "http://localhost:8080/company/users/539ea7af-d366-4d13-9446-d427fb72a910",

            "email": "sn...@gmail.com",

            "profile": {

                "title": "geek"

            }

        },

        "videos": [

            {

                "id": "df64a0a4-aa2b-4d96-8956-e9c324550b51",

                "url": "http://localhost:8080/company/videos/df64a0a4-aa2b-4d96-8956-e9c324550b51"

                "title": "Star Wars",

                "status": 0,

                "created_at": "2014-12-03T09:49:21.58386Z",

                "updated_at": "2014-12-03T09:49:21.58386Z",

                "deleted_at": null,

                "published_at": null,

                "tags": [

                    "dark"

                ],

                "author": {

                    "id": "539ea7af-d366-4d13-9446-d427fb72a910",

                    "url": "http://localhost:8080/company/users/539ea7af-d366-4d13-9446-d427fb72a910",

                    "email": "sn...@gmail.com",

                    "profile": {

                        "title": "geek"

                    }

                },

                "updater": {

                    "id": "539ea7af-d366-4d13-9446-d427fb72a910",

                    "url": "http://localhost:8080/company/users/539ea7af-d366-4d13-9446-d427fb72a910",

                    "email": "sn...@gmail.com",

                    "profile": {

                        "title": "geek"

                    }

                },

                

            }

        ]

    }

]

Thanks in advance to anyone who takes the time to help, I appreciate it!

[Edward Muller]

I'm not sure I understand the question.

Can you provide a simplified sample program via the go playground that demonstrates the marshaling problem and explain the results you expected?

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Edward Muller
@freeformz

[Jamie Stackhouse]

I'm not sure I understand it either, I just unmarshalled and then marshalled data in the playground from a json blob and it worked exactly as I expected.

I need to go back to the drawing board and see if the error is happening when the data is retrieved from Postgres using database/sql, or if I just have a bug in the original code.

Essentially, what is loaded into the Videos type is JSON that is rendered directly from postgres, and it wasn't appearing to be marshalled properly.

https://play.golang.org/p/U6p0walHw9

[Jamie Stackhouse]

Alright, so my issue is that my database is producing a row of values, one of which is a JSON representation, however it doesn't match my currently defined structure.

What is my best option for converting what starts out like a flatten json object into a organized object with sub-structs?

So far I can think of two methods:

• Creating an anonymous struct defined inside the custom types Scan method, unmarshal to that struct then build the actual representation I want by looping and creating individual Video objects into my Videos type above.
  
• Unmarshalling directly to a map[string]interface{} in the Scan method and doing the same operation.
  

In the playground link below I've implemented what would be the Scan function as the UnmarshalJSON function for the Videos type.

https://play.golang.org/p/8tqNIwQIRk

[roger peppe]

On 16 January 2015 at 14:23, Jamie Stackhouse

<jamie.st...@redspace.com> wrote:
> Alright, so my issue is that my database is producing a row of values, one
> of which is a JSON representation, however it doesn't match my currently
> defined structure.
>
> What is my best option for converting what starts out like a flatten json
> object into a organized object with sub-structs?
>
> So far I can think of two methods:
>
> Creating an anonymous struct defined inside the custom types Scan method,
> unmarshal to that struct then build the actual representation I want by
> looping and creating individual Video objects into my Videos type above.

This is probably the better of the two approaches. Avoid unmarshalling
into interface{}
if you possibly can. Also, don't ignore the possibility of using json.RawMessage
which can be very useful in some contexts.

You can't do an arbitrary transformation when transforming JSON into
Go structs -
if the representation you want eventually doesn't match the JSON sufficiently,
then an intermediate representation is the way to go.

[Jamie Stackhouse]

Alright, thank you.

Since I would prefer to avoid trashing memory for the intermediate representation since it is purely to format JSON, what are my options to limit garbage that is created?

Would using pointers in the intermediary and then referencing the value in the final object reduce the amount of garbage?

Ala,

var serializer struct {
  Id *string
}

video.Id = *serializer.Id

Thanks for any "pointers" in this area.

[Tamás Gulácsi]

Nope, string is already a pointer, an offset and a length. So another level of reference just slows.