Is json.Marshal deterministic?

[Juliusz Chroboczek]

Hi,

Suppose that I call json.Marshal on two structures that are deep equal,
or on the same structure at different times. Are the outputs guaranteed
to be bytewise identical?

(The reason I'm asking is that I'm sending JSON over HTTP, and I need to
know whether it is correct to send a strong ETag with my reply even
though I'm generating new JSON every time.)

Thanks,

-- Juliusz

[Nagaev Boris]

Hi Juliusz,

I don't know if JSON serialization is deterministic, but I know a couple of cases when it is not.

If the type or some type inside it has a custom JSON marshaller (method MarshalJSON), then if that function's output is not deterministic, the whole JSON for the type is not deterministic. Custom marshaller can do whatever it wants, e.g. traverse maps without ordering keys, resulting in non-determinism.

Another common pitfall with JSON: nil vs empty slice. E.g. []string{} is encoded as "[]", while []string(nil) is encoded as "null", while they both mean an empty slice in Go (though NOT considered deep equal according to reflect.DeepEqual). The workaround is to add `omitempty` to the JSON tag to remove such fields from the encoding. Otherwise you need to make sure that empty slices are always nil or always non-nil in the code producing structures that are serialized.

Boris

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/87h6gag1qk.fsf%40pirx.irif.fr.

--

Best regards,
Boris Nagaev

[Juliusz Chroboczek]

> I don't know if JSON serialization is deterministic, but I know a couple
> of cases when it is not.

> If the type or some type inside it has a custom JSON marshaller (method
> MarshalJSON), then if that function's output is not deterministic, the
> whole JSON for the type is not deterministic.

Obviously.

> Another common pitfall with JSON: nil vs empty slice. E.g. []string{} is
> encoded as "[]", while []string(nil) is encoded as "null", while they both
> mean an empty slice in Go

I'm not sure I follow. I was under the impression that the empty array
and nil are distinct values in Go, even though many functions treat them
the same.

-- Juliusz

[TheDiveO]

They are distinct indeed.

An "if sl == nil ..." will not match an empty slice. It is len(sl) that returns 0 for both nil and empty slices, and range working along the same idea.

playground example: https://goplay.tools/snippet/df0bG6YXfJZhttps://goplay.tools/snippet/df0bG6YXf

https://goplay.tools/snippet/df0bG6YXfJZ

[Matt Parker]

one of my favorite examples of this is google's protojson package, where the output of `json.Marshal` is [deliberately nondeterministic](https://protobuf.dev/reference/go/faq/#unstable-json) to prevent users from attempting to rely on that property