Preserving extra properties using JSON unmarshal / marshal?

[Eric Johnson]

If I've got a structure like this:

type jsonData struct {
    FirstName string `json:"first_name"`
    LastName  string `json:"last_name"`
}

I can marshal / unmarshal JSON as:

{
  "first_name": "first",
  "last_name": "last"
}

What if my input JSON is this:

{
  "first_name": "first",
  "last_name": "last",
  "favorite_color": "orange",
  "age": 92
}

Is there an existing library to have a structure that's something like this:

type jsonData struct {
    FirstName string                 `json:"first_name"`
    LastName  string                 `json:"last_name"`
    Extras    map[string]interface{} `json:",extras"`
}

Such that a JSON unmarshal and marshal will preserve the data of the original JSON?

I think I can probably make something work with the standard library, except that it will require extra marshaling / unmarshaling steps that will be annoying to reproduce for each structure that needs to capture extra data. So I'm hoping there's a better solution?

Eric

[Chris Burkert]

Take a Look at 

https://blog.golang.org/json-and-go:

The json package uses map[string]interface{} and []interface{}values to store arbitrary JSON objects and arrays; it will happily unmarshal any valid JSON blob into a plain interface{} value. The default concrete Go types are:

• bool for JSON booleans,

• float64 for JSON numbers,

• string for JSON strings, and

• nil for JSON null.

Does this help?

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/630827e2-8277-4231-9ab1-31881198a386%40googlegroups.com.

[Eric Johnson]

Understand all that. It doesn't help.

Asking the question a different way, as concrete code always clarifies. For something like the following snippet of code:

const testJSON = `{

"first_name": "first",

"last_name": "last",

"favorite_color": "orange",

"age": 92

}`

func TestUnmarshalling(t *testing.T) {

var res struct {

FirstName string `json:"first_name"`

LastName  string `json:"last_name"`

}

err := json.Unmarshal([]byte(testJSON), &res)

assert.NoError(t, err)

out, err := json.MarshalIndent(&res, "", "\t")

assert.NoError(t, err)

assert.Equal(t, testJSON, string(out))

}

Is there anything I can do to the "res" struct" to preserve the "favorite_color" and "age" properties from the original JSON, so that the final assert.Equal() passes? Ideally, as I indicated in the original post, it would be something along the lines of an additional struct member that acts as the catch-all for any values not found for the struct. (hence the line "Extras map[string]interface{} `json:",extra" that I put in my example.)

Stepping through the standard library, it seems pretty clear that the standard library simply discards JSON properties that it doesn't map to fields. I think the crucial lines (decode.go, 753-769) indicate that if a field is not found, no target value ("subv") is set, and then the resulting data is discarded.

Since the standard library doesn't support it, I'm wondering if there are any other reliable JSON encode / decode libraries out there that might?

I have three options, that I can tell:

• Find an alternate library that supports something like this (hence the post)
• Use the standard library by implementing the Unmarshaler interface on the struct, use normal unmarshaling for the known fields, then also unmarshal as a map[string]interface{}, remove all the fields that I don't want, and store the results in the "Extra" field myself. This will result in double-parsing each struct and a lot of extra boilerplate code. Then also need to do appropriate somethings to implement Marshal as well.
• Fork the standard library JSON package, add the support myself, and submit it back as a possible enhancement(!).
  

Using the standard library as is seems like a bad option, given the extra boilerplate that would be required for each struct. I'm hoping someone has already made an implementation that does this!

Eric.

To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.

[Brian Candler]

Looks like your thoughts are the right ones:

https://stackoverflow.com/questions/33436730/unmarshal-json-with-some-known-and-some-unknown-field-names

[Eric Johnson]

Thanks for the pointers.

I was hoping for an easier answer. Maybe will look into patching stdlib.

Eric.

On Sat, Feb 22, 2020 at 1:15 AM Brian Candler <b.ca...@pobox.com> wrote:

Looks like your thoughts are the right ones:

https://stackoverflow.com/questions/33436730/unmarshal-json-with-some-known-and-some-unknown-field-names

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/UZri3aWypTI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/376071d9-6852-419b-923c-9d36f46bf158%40googlegroups.com.