Gob encoding of maps is not consistent, should this be fixed?

[Tom Payne]

This example demonstrates that the Gob encoding of the same map value varies:

  https://play.golang.org/p/DZRV9i0uf_

This is because the encoder iterates over the map using a for ... range loop and the iteration order for maps is not defined:

  https://github.com/golang/go/blob/master/src/encoding/gob/encode.go#L379

Is this considered a bug and would the Go maintainers consider a patch to fix it? Such a patch would sort the keys first and emit the key-value pairs in order to provide a consistent encoding. The consistent encoding would likely be optional to avoid the performance and memory penalty of sorting the keys.

The reason this is a problem is that we generate a data file in Gob format as part of our build process. The encoded structure contains many maps and so the generated file varies on every run. This makes our build process non-repeatable: when using Gob the same inputs do not result in the same outputs.

Note that consistent encoding of the same value is *not* specified in the list of goals in https://blog.golang.org/gobs-of-data (in fact it is not mentioned at all). However, it could be considered a desirable feature :)

Regards,

Tom

[Jan Mercl]

On Mon, Dec 19, 2016 at 5:06 PM Tom Payne <twp...@gmail.com> wrote:

> Note that consistent encoding of the same value is *not* specified in the list of goals in https://blog.golang.org/gobs-of-data (in fact it is not mentioned at all). However, it could be considered a desirable feature :)

Map keys are comparable but not necessarily ordered, so I think it's a non-starter. If you need a reproducible map gob and the map has ordered keys, you can always use the GobEncoder interface implementation for that.

--

-j