json encoding & decoding interface types

[Doğan Kurt]

Hi great Go community. Sorry it's a long question, but i would really appreciate some guidance.

--------------------

Let's say i have two different objects, browser and executable that implement ScanCleaner interface.

type ScanCleaner interface {

scan()

clean()

}

type browser struct {

Name string

}

type exe struct {

Size int

}

These two objects are different internally but they implement the same scan and clean methods. In general i don't need to know their internals. I have this instead;

var objects []ScanCleaner

I can iterate through the objects, scan and clean them. So far so good, but i want to decouple scan and clean processes. One can scan the system, get a report and later clean based on that report file. So that interface slice should be stored appropriately in a file and recovered later.

I encode the objects slice, and get a perfectly good json output. 

Here is the working code: https://play.golang.org/p/cYgnhgxkPL0

Output: [{"Name":"chrome"},{"Name":"firefox"},{"Size":1234},{"Size":4321}]

--------------------

The problem arises when i need to decode report file and recover the objects interface slice. Object types are completely lost.

My current solution is this; I encapsulate every object with a string that specifies it's type,

type detection struct {

Type   string

Object ScanCleaner

}

and encode the slice of detection instead. The Type string is main.exe or main.browser obtained by fmt.Sprintf("%T", obj).

Here is a working code: https://play.golang.org/p/KeJjl8IOqRP

Output: [{"Type":"main.browser","Object":{"Name":"chrome"}},{"Type":"main.exe","Object":{"Size":1234}}]

Finally, i use a dispatch routine that recognizes objects type by Type string, decodes the object with the correct type, and calls it's clean method.

func cleanAll(jsn []byte) {

detects := []struct {

Type   string

Object json.RawMessage

}{}

json.Unmarshal(jsn, &detects)

for _, d := range detects {

var sc ScanCleaner

switch d.Type {

case "main.exe":

var e exe

json.Unmarshal(d.Object, &e)

sc = e

case "main.browser":

var b browser

json.Unmarshal(d.Object, &b)

sc = b

}

sc.clean()

}

}

Here is the final working code: https://play.golang.org/p/HpHFiHrT2oz

Output: 

browser clean {chrome}
browser clean {firefox}
exe clean {1234}
exe clean {4321}

--------------------

This solution is obviously not good. If i change an objects name, i should change the Type string in switch. Also i need to rely on the string returned by fmt.Sprintf("%T").

Is there a better solution to accomplish this task?

Thanks.

[Burak Serdar]

You can use an intermediate structure to parse the JSON input, and
then process that to construct the slice:

var entry []struct {
Name string
Size int
..
}

json.Unmarshal(&entry)
for _,e:=range entry {
if len(e.Name)==0 {
... // This entry has size, create struct exe
} else {
... // This entry has name, create struct Browser
}
}

Of course, this assumes you can deduce the type of the element based
on its contents easily.

> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Doğan Kurt]

Hi Burak,

The example objects are very simplified. In real code, there are many more object types and some of them looks exactly same. For instance;

type chromeHomepage struct {

Url  string

File string

}

type firefoxHomepage struct {

Url  string

File string

}

They look exactly same in the json, but clean method for firefox and chrome objects are completely different.

[Krzysztof Kowalczyk]

The core issue seems to be that that round-tripping via json looses information i.e. type of serialized record.

You solved it by embedding type as field of the record, which works.

You could embed type implicitly.

One way to do it is to use a separate file for each type. The type is implicitly embedded in a file name i.e. you would have browsers.json with just []browser, exes.json with just []exe etc.

Another option is to serialize them as:

type AllRecords struct {

  Browsers []browser

  Exes []exe

  .... other types

}

This would serialize as { "Browsers": [...], "Exes": [...] }. Type is encoded implicitly in the top-level keys of the dictionary.

If JSON is not a strict requirement, you could consider using a more typed serialization format like Protobufs or msgpack or something else. There are many formats/libraries to choose from and some might offer a better API for this use case.

[Burak Serdar]

On Mon, Feb 19, 2018 at 5:14 PM, Doğan Kurt <kultig...@gmail.com> wrote:
> Hi Burak,
>
> The example objects are very simplified. In real code, there are many more
> object types and some of them looks exactly same. For instance;
>
> type chromeHomepage struct {
> Url string
> File string
> }
>
> type firefoxHomepage struct {
> Url string
> File string
> }
>
> They look exactly same in the json, but clean method for firefox and chrome
> objects are completely different.

In that case, I'd do what you did already. Maybe improve it a bit with
something like below, so you won't need a switch:

var dict map[string]func() interface{}

func init() {
dict=make(map[string]func() interface{})
dict["type1"]=func() interface{} {return type1{}}
dict["type2"]=func() interface{} {return type2{}}
...
}

func unmarshal(...) {
...
for _,d:=range detects {
if x, ok:=dict[d.Type]; ok {
u:=x()
json.Unmarshal( d.Object,&u)
// append u to slice