Looking for advice on the idiomatic way to handle schemaless maps/structures.

[Toby Lawrence]

Hey y'all.  Long time lurker, first time poster.  I've been trawling through the mud, clumsily finding my way around Go to get it to do what I want to do.  For the most part, I feel I have a good grasp on the language but something that constantly eludes me is schemaless maps/structures.  If you've ever used PHP, and I'm sure other languages would let you do the same, but you know that you can create a array and start assigning things to it, at arbitrary depths, to your heart's content.  Maybe not arbitrary, but it handles a lot of the grunge work for you by abstracting typing.  It makes building structures that, say, get rendered as JSON and spit out to a browser, much much easier.  I currently have a chunk of code where I'm building a big multi-dimensional map to represent a list of flat list of objects, and I'm in map[string]interface{} hell.  Does my code look OK?  Am I missing some key optimization here?  Is this code way off the mark as far as "idiomatic Go" is concerned?  Am I forever destined to use map[string]interface{} and append() to build schemaless maps/structures?

// Go through all the configured servers and organize them by category.
groups := make(map[string]map[string]interface{}, 0)
 
for _, s := range me.Configuration.Servers {
    // Create the group entry if we don't already have one.
    if _, ok := groups[s.Category]; !ok {
        group := make(map[string]interface{}, 0)
 
        servers := make([]map[string]string, 0)
        group["servers"] = servers
 
        groups[s.Category] = group
    }
 
    // Create a server entry and add it to the group.
    server := make(map[string]string, 0)
    server["internalName"] = s.InternalName
    server["displayName"] = s.DisplayName
 
    existingServers := groups[s.Category]["servers"].([]map[string]string)
    existingServers = append(existingServers, server)
    groups[s.Category]["servers"] = existingServers
}
 
// Wrap our result.
result := make(map[string]interface{}, 0)
result["groups"] = groups

Thanks in advance. :)

-toby

[Robert Melton]

On Sat, Jul 20, 2013 at 10:12 AM, Toby Lawrence <tobias....@gmail.com> wrote:

something that constantly eludes me is schemaless maps/structures.

When I started doing Go, I had this issue often, so I feel (felt?) your pain.  I have recently been working in Python a good bit, and got used to very flexible maps/structures.   

 

It makes building structures that, say, get rendered as JSON and spit out to a browser, much much easier.

Easier, absolutely.  But maybe not better?  "Show me your code and conceal your data structures, and I shall continue to be mystified.  Show me your data structures, and I won't usually need your code; it'll be obvious." -- Eric Raymond

 

Is this code way off the mark as far as "idiomatic Go" is concerned?  Am I forever destined to use map[string]interface{} and append() to build schemaless maps/structures?

I am not expert enough to speak to "idiomatic Go", but I can tell you personally -- giving up on schemaless structures in code I control, I am writing cleaner, and I would go so far as to say, better code.  It forces more work up front, it forces you to think through your data structures.  This upfront thought so far has yielded benefits over and over again in my handful of Go projects.  Is schemaless actually required for the problem at hand, or are you just doing it because it is what you are used to?  Also, turning a struct into json is also very easy.   

--
Robert Melton

[Rob Pike]

Eric Raymond? No, it was Fred Brooks who said that. It's one of the famous quotes from The Mythical Man Month.

Who attributes that to Raymond?

-rob

[Robert Melton]

http://www.dreamsongs.com/ObjectsHaveNotFailedNarr.html

Claims that the Brooks quote (from The Mythical Man-Month) was "Show me your flowchart and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won't usually need your flowchart; it'll be obvious." and it was paraphrased to "Show me your code and conceal your data structures, and I shall continue to be mystified. Show me your data structures, and I won't usually need your code; it'll be obvious." in The Cathedral and the Bazaar by Eric Raymond. 

--
Robert Melton

[Robert Melton]

I just did a quick double check -- The Mythical Man Month : Page 102 has the quote referencing flowcharts and tables, on page 12 of The Cathedral and the Bazaar - ESR paraphrases it (with citation) to what I quoted originally. 

--
Robert Melton

[David DENG]

For you original code, you can write it in this way:

groups := make(map[string]map[string]interface{}, 0)

for _, s := range me.Configuration.Servers {

group, ok := groups[s.Category]

if !ok {

group = make(map[string]interface{})

groups[s.Category] = group

}

group["servers"] = append(group["servers"], map[string]string{

"internalName": s.InternalName,

"displayName":  s.DisplayName,

})

}

// Wrap our result.

result := map[string]interface{}{

"groups": groups,

}

If you just want to generate the JSON, and save some memory, use struct literal for know data structures instead of map:

type Server struct {

InternalName string `json:"internalName"`

DisplayName  string `json:"displayName"`

}

type Group struct {

Servers []Server `json:"servers"`

}

groups := make(map[string]*Group, 0)

for _, s := range me.Configuration.Servers {

group, _ := groups[s.Category]

if !ok {

group = &Group{}

groups[s.Category] = group

}

group.Servers = append(group.Servers, Server{

s.InternalName, s.DisplayName,

})

}

// Wrap our result.

result := map[string]interface{}{

"groups": groups,

}

David

[Toby Lawrence]

Yeah.  It seems like adding the JSON naming hints to the existing structs and doing some minimal adjustments is the right way to make it cleaner / faster / better / etc.  I was just hopelessly trying to cling on to PHP's way of doing things. :P

Thanks all!