Decoding arbitrary "encoding/gob" data

707 views
Skip to first unread message

Dmitri Shuralyov

unread,
Mar 9, 2013, 6:28:23 PM3/9/13
to golan...@googlegroups.com
It's possible to decode arbitrary JSON data[1], without having a struct that represents the data.

Decoding arbitrary data

Consider this JSON data, stored in the variable b:

    b := []byte(`{"Name":"Wednesday","Age":6,"Parents":["Gomez","Morticia"]}`)

Without knowing this data's structure, we can decode it into an interface{} value with Unmarshal:

    var f interface{}
    err := json.Unmarshal(b, &f)
I want to do something similar with "encoding/gob", namely decode arbitrary data stored in there, in a situation where I don't have the concrete struct that was used while encoding.

According to goals of gobs[2], it sounds like it should be possible,

Gob streams must be self-describing. Each gob stream, read from the beginning, contains sufficient information that the entire stream can be parsed by an agent that knows nothing a priori about its contents. This property means that you will always be able to decode a gob stream stored in a file, even long after you've forgotten what data it represents.

However, there's a problem because Go is not a dynamic language and there currently is no way to create types in run-time.

According to this thread[3], 

> But it seems that to read a stream of gobs, you need to know the 
> type of each gob in advance. I cannot decode any gob into 
> interface{}. Why not? 

If you send a struct on a gob, the receiver needs to receive that into 
a struct type.  If you don't tell it the struct type to receive into, 
the receiving gob package would have to create the struct type on the 
fly.  But the reflect package, and the Go runtime, has no facility for 
that.

I accept that for now.

But it should still be possible to get a textual representation of the struct type that was used in encoding.

Say I do something like...
type P struct {
    X, Y, Z int
    Name    string
}

err := enc.Encode(P{3, 4, 5, "Pythagoras"})
To produce a slice of bytes.

Input: a slice of bytes of an encoded gob and nothing else (i.e. type P is absent)
Output: the following string or something that contains the following type information "type P = struct { X int; Y int; Z int; Name string; }"

How should I go about doing that? Thank you.

Rob Pike

unread,
Mar 9, 2013, 7:09:58 PM3/9/13
to Dmitri Shuralyov, golan...@googlegroups.com
Look at the code in encoding/gob/debug.go. It shows you how to parse
the stream without compiled-in types. It wouldn't be too hard to some
mappy thing based on that, not that I recommend doing so.

-rob

Dmitri Shuralyov

unread,
Mar 10, 2013, 4:53:23 PM3/10/13
to golan...@googlegroups.com
Thank you, I'll give that a shot!

What do you mean by "not that I recommend doing so"?
Reply all
Reply to author
Forward
0 new messages