RPC caused the output of gob encoder to change.

77 views
Skip to first unread message

Awaken

unread,
Dec 6, 2011, 2:01:38 AM12/6/11
to golang-nuts
Hi all,
I am experiencing a "bug" which drove me crazy for these two days...
I finally pinned down to the line code that caused the weird behavior
and tried to reproduce the problem with a simpler code. And it
successfully reproduced the same issue.
Here is the code I used to reproduce the problem:

package main
import (
"bytes"
"gob"
"rpc"
"http"
"net"
"fmt"
"os"
)
type Test int

func EncodeToBinary(value interface{}) ([]byte, os.Error) {
buf := bytes.NewBuffer(make([]byte,0))
encoder := gob.NewEncoder(buf)
err := encoder.Encode(value)
if err != nil {
return nil, err
}
return buf.Bytes(), nil
}

func (t *Test) RPC(args *int, reply *int) os.Error {
*reply = 0
return nil
}

func main() {
t := new(Test)
rpc.Register(t)
rpc.HandleHTTP()
l, _ := net.Listen("tcp", ":1234")
go http.Serve(l, nil)
client, _ := rpc.DialHTTP("tcp", "127.0.0.1:1234")
i := 0
a := make(map[int]int)
//fmt.Println(EncodeToBinary(a))
client.Call("Test.RPC", &i, &i)
fmt.Println(EncodeToBinary(a))
}

Pay attention to the commented line of fmt.Println.
If the first fmt.Println is commented out, the output is [14 255 133 4
1 2 255 134 0 1 4 1 4 0 0 4 255 134 0 0].
But if the second fmt.Println is commented out while the first is not,
the output is [14 255 129 4 1 2 255 130 0 1 4 1 4 0 0 4 255 130 0 0]

The reason why I think this is a bug is that I am trying to compute
hash on the binary encoding of objects.
So, it is really annoying if the same object encodes into different
binary representation from time to time.

Also, it is strange why a RPC can change the behavior of gob
encoder...

I am eager to know why this happens or you also believe it is a bug.

Awaken

unread,
Dec 6, 2011, 2:25:18 AM12/6/11
to golang-nuts
I think probably this is a better demonstration:

package main
import (
"bytes"
"gob"
"rpc"
"http"

"io/ioutil"


"net"
"fmt"
"os"
)
type Test int

func EncodeToBinary(value interface{}) []byte {


buf := bytes.NewBuffer(make([]byte,0))
encoder := gob.NewEncoder(buf)

encoder.Encode(value)
return buf.Bytes()
}

func DecodeFromBinary(buffer []byte, value interface{}) os.Error {
//fmt.Println("decodeFromBinary: ", buffer, value)
buf := bytes.NewBuffer(buffer)
decoder := gob.NewDecoder(buf)
return decoder.Decode(value)
}

func (t *Test) RPC(args *int, reply *int) os.Error {
*reply = 0
return nil
}

func main() {
t := new(Test)

b, err := ioutil.ReadFile("test.txt")
if err == nil {
var a map[int]int
DecodeFromBinary(b, &a)
fmt.Println(EncodeToBinary(a))
return


}
rpc.Register(t)
rpc.HandleHTTP()
l, _ := net.Listen("tcp", ":1234")
go http.Serve(l, nil)
client, _ := rpc.DialHTTP("tcp", "127.0.0.1:1234")
i := 0
a := make(map[int]int)

client.Call("Test.RPC", &i, &i)

b = EncodeToBinary(a)
fmt.Println(b)
ioutil.WriteFile("test.txt", b, 0666)
}

The first time you run the code, the output is


[14 255 133 4 1 2 255 134 0 1 4 1 4 0 0 4 255 134 0 0]

But the second time you run the code, it is supposed to read the
binary encoding of the object back, decode it and encode it again
which is expected to be the same as what's in the file.
However, you get:


[14 255 129 4 1 2 255 130 0 1 4 1 4 0 0 4 255 130 0 0]

Or put it this way, the behavior can be summarized as:
encode(decode(encode(x)) != encode(x)

Rob 'Commander' Pike

unread,
Dec 6, 2011, 2:41:05 AM12/6/11
to Awaken, golang-nuts
Gobs contain type information and unique ids. A stream is guaranteed to decode to the values put into it, but otherwise no guarantee is made about the contents of the stream itself.

-rob

Reply all
Reply to author
Forward
0 new messages