encoding/json.Marshal() order of maps

[Jan Halfar]

Hi Everyone,

I am writing a RESTful JSON web service. When encoding maps I ran into a behaviour, which I thought to be a bug and I submitted an issue https://code.google.com/p/go/issues/detail?id=6244 .

Let me illustrate:

json.Marshal() sorts maps when it is encoding them, which does not seem right

    package main

    import (

    "fmt"

    "encoding/json"

    )

    func main() {

    fmt.Println("Go just broke my web form ...")

    countries := map[string]string {

    "USA" : "United States of America",

    "Afg" : "Afghanistan",

    }

    jsonBytes, _ := json.MarshalIndent(countries, "", "  ")

    fmt.Println(countries)

    fmt.Println(string(jsonBytes))

    }

Outputs

    Go just broke my web form ...

    map[USA:United States of America Afg:Afghanistan]

    {

      "Afg": "Afghanistan",

      "USA": "United States of America"

    }

Instead of

    ...

    {

      "USA": "United States of America",

      "Afg": "Afghanistan"

    }

Which basically would mean that I would have change my data into

    {

      "countries" : {

        "Afg": "Afghanistan",

        "USA": "United States of America"

      }

      "order" : [

        "USA",

        "Afg"

      ]

    }

http://play.golang.org/p/LL4euXjM7a

It got looked at right away by one of the the encoding/json authors and he closed it as "WorkingAsIntended".

I am kind of lost here with my team mates, because we can not make any sense of it and I wonder, if any of you guys could help us out / give us a hint.

[Péter Szilágyi]

Hi,

  Quoting from the specs: "A map is an unordered group of elements of one type". This means that you cannot rely on a map having any specific order, or even preserving one between multiple iterations of its elements. So I think your problem is deeper than json encoding.

Cheers,

  Peter

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Martin Angers]

It's impossible, since maps have undefined ordering. From the specs: "The iteration order over maps is not specified and is not guaranteed to be the same from one iteration to the next."

http://golang.org/ref/spec#For_statements

[Matthew Kane]

Depending on the order of an unordered structure is always wrong. Even
if the sort were removed, the order may not be preserved. In fact,
preserving the order is a vulnerability. If you need to preserve the
order of a 1-to-1 mapping, use a list of 2-element lists instead.

> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

--
matt kane
twitter: the_real_mkb / nynexrepublic
http://hydrogenproject.com

[Jan Halfar]

Thanks everyone for the fast answers!

I guess I need an object which preservers the order by keeping an index and implements UnmarshalJSON / MarshalJSON. Since this use case does not seem to be that exotic - is there a "standard" implementation of on ordered map, which maybe even comes with JSON marshalling support - I could not find anything obvious in that respect.

[Jan Halfar]

very cool, but my JSON service consumers will not understand that ;)

On Monday, August 26, 2013 6:26:08 PM UTC+2, island...@gmail.com wrote:

On Monday, August 26, 2013 9:20:16 AM UTC-7, Jan Halfar wrote:

I guess I need an object which preservers the order by keeping an index and implements UnmarshalJSON / MarshalJSON.

Use a slice. http://play.golang.org/p/tH0NFAdfNN

[Jan Halfar]

Thank you for your competent and constructive answer, it is much appreciated. Despite the specs the above way of using JSON objects is basically the standard, when consuming JSON in all other languages / libraries I have used in the last years. I also think that it is very pragmatic, since it is highly expressive. I also understand, that a maps implementation, that focusses on performance ignores order - being new to Go I just did not know it.

On Monday, August 26, 2013 6:16:05 PM UTC+2, island...@gmail.com wrote:

Use an array if order is important.

JSON: "An object is an unordered set of name/value pairs"

Go: "The iteration order over maps is not specified and is not guaranteed to be the same from one iteration to the next."

javascript: "The mechanics and order of enumerating the properties (step 6.a in the first algorithm, step 7.a in the second) 

is not specified."

Even if the sort is removed from the JSON encoder, your are SOL.

A simpler representation for your data with order is:

      "countries" : {

        ["USA", "United States of America"],

        ["Afg", "Afghanistan"]

      }

[John Asmuth]

What language do you use that has a JSON lib that unmarshals objects into an ordered map?

[Jan Halfar]

Any JavaScript implementation, that I have seen so far does so - just try this in your browsers js console: var foo = {"b":2, "a":1};JSON.stringify(foo);

{"b":2,"a":1}

[Jan Halfar]

I did not say, that it is everywhere like that and I am also not saying, that it is wrong or right ... all I am saying is, that this is the way it is being used in the javascript world and if I would present an object like {"data":{"b":2, "a":1}, "order" : ["b", "a"]} or [{"value":2, "id":"b"}{"value":1, "id" : "a"}] people would ask me why. 

On Monday, August 26, 2013 6:51:12 PM UTC+2, island...@gmail.com wrote:

On Monday, August 26, 2013 9:33:51 AM UTC-7, Jan Halfar wrote:

Despite the specs the above way of using JSON objects is basically the standard, when consuming JSON in all other languages / libraries I have used in the last years.

Insertion order is not persevered in many programming languages including Python, Java (HashMap, Hashtable), Clojure, Objective-C (NSDictionary) and Go.

 

[Matthew Kane]

Until you get bit: https://code.google.com/p/v8/issues/detail?id=164

[Jan Halfar]

One has to love javascript!