Using gobson to read bson with unknown structure, modify it, then write it

[Travis Reeder]

Hi,

I'm trying to read bson input from a stream without knowing the structure/types, etc, but can't see how to do it. Anyone have any tips on how I could do that?  I would like to be able read it, look for specific keys and values and perhaps modify them, before writing to output.

Regards,

Travis

[Gustavo Niemeyer]

Hi Travis,

You can use maps for that:

data, err := bson.Marshal(bson.M{"a": 1})
if err != nil {
panic(err)
}
var m bson.M
err = bson.Unmarshal(data, &m)
if err != nil {
panic(err)
}
println(m["a"].(int))

Note that bson.M is simply an alias for map[string]interface{}. You
can define the map type yourself if you wish to.

--
Gustavo Niemeyer
http://niemeyer.net
http://niemeyer.net/blog
http://niemeyer.net/twitter

[Travis Reeder]

Thanks Gustavo. So the Unmarshal returns an error "Unknown element kind ..." when running this on mongo messages, here's some output:

buf=<�_ ������admin.$cmd whatsmyuri

Unknown element kind (0xEC)

map[]

buf=N÷���_ � * you 127.0.0.1:43862 ok�?

Unknown element kind (0xC3)

map[]

buf=P�_ ������admin.$cmd ) replSetGetStatus forShell

Unknown element kind (0xED)

map[]

buf=\ķ���_ � 8 errmsg not running with --replSet ok

Unknown element kind (0xC4)

map[]

buf=*�_ ������test.foo

Unknown element kind (0xEE)

map[]

buf=Eŷ���_ � !_idM��4 ��I

                         �Y� a�?

Unknown element kind (0xC5)

map[]

The code snippet for the output above is below and buf is the bytes read from the tcp stream.

                       fmt.Printf("buf=%s\n", buf[0:nr])

var m bson.M

err = bson.Unmarshal(buf[0:nr], &m)

if err != nil {

                     fmt.Println(err)

// panic(err)

}

fmt.Println(m)

[Gustavo Niemeyer]

> Thanks Gustavo. So the Unmarshal returns an error "Unknown element kind ..."
> when running this on mongo messages, here's some output:

Your email contains unreadable information. Either way, I think the
issue is most likely that you're passing corrupted data to gobson.
Most likely a partial document.

[Travis Reeder]

Ok, maybe I can't directly unmarshal the bytes from the mongo protocol? where is the source for mgo so I can see how it's doing it (don't see a link on the mgo page)?

Posted the unreadable stuff to a gist here: https://gist.github.com/975847 . It shows what I'm getting from mgo and mongo's responses.

[Gustavo Niemeyer]

> Ok, maybe I can't directly unmarshal the bytes from the mongo
> protocol? where is the source for mgo so I can see how it's doing it (don't
> see a link on the mgo page)?

Ah, indeed.. you can't simply unmarshal things from the socket..
there's a protocol involved. You can find the source code for mgo
through the goinstall link itself: https://launchpad.net/mgo