encoding.BinaryMarshaler interface

[han...@gmail.com]

Suppose there's a struct, say Foo, that needs to be serialized and sent over the network very frequently (about every 100 microseconds to one millisecond, to a multicast group).  To serialize Foo to write it to an io.Writer (e.g., a net.UDPConn), it would think it would be nice to have Foo implement the encoding.BinaryMarshaler interface.  However, the method

func (f *Foo) MarshalBinary() ([]byte, error) { ... }

returns a byte slice rather than takes one in as an parameter (like an io.Reader does).  I assume the main reason for this is that the caller may not easily know how big the backing array should be.  But if Foo implements its MarshalBinary method to make a new backing array with every call, wouldn't that be very unkind to the garbage collector, considering how frequently it will be called?

I suppose Foo could contain a byte array that gets reused (and a slice of it returned) with every call to MarshalBinary, along with a comment warning the caller that the returned slice is volatile.  Would that be considered appropriate, or would it violate the spirit of the encoding.BinaryMarshaler interface?  (A drawback to this is that it could consume a lot of memory if many Foo structs need to also persist on the heap.)

Another approach is to use sync.Pool.  Would this be an appreciate use case?

Or, Foo can have an additional method that is similar to MarshalBinary but takes a byte slice as a parameter, something like

func (f *Foo) PutBinary(b []byte) ([]byte, error) { ... }

which returns a slice of b (or makes the backing array if b is nil) and a non nil error if b is too small (or does an implicit panic via an "early bounds check" a la binary.BigEndian.PutUint32 et. al.).

Or maybe Foo should also be an io.Reader by providing

func (f *Foo) Read(p []byte) (n int, err error)

which returns a non nil error if p is too small.

Has anyone done anything similar to this and has experience with some (or all) of the approaches, or a new approach?  I very much appreciate any thoughts and suggestions!

[Matt Harden]

I would start with the naive implementation first, and only make it fancier (and more complicated) if that turns out not to be fast enough. Go's GC is pretty good already and getting better.

You must not change the data in the returned slice after returning it, unless you somehow know that the caller is done with the slice by then and even then it would be brittle.

Don't implement Read with semantics other than those defined in the io module (returning an error if the buffer is too small); that's asking for trouble. It would be fine to implement io.WriterTo though, and the caller could pass you a bytes.Buffer to write into.

I think this seems like a good use case for sync.Pool, but you should benchmark and profile to be sure.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[han...@gmail.com]

Thanks so much for your reply!

Yes, making Foo an io.WriterTo is exactly what I was looking for!  I'll have Foo implement both WriteTo and MarshalBinary methods (with the latter making a new backing array with each call, and possibly using a sync.Pool later on, after benchmarking actual use cases).

The added benefit of implementing an io.WriteTo method is that the caller can pass in a net.UDPConn (or other network io.Writer) and write directly, or pass in a bytes.Buffer to send multiple messages with one write, if needed to reduce the number of network writes, which are relatively slow.

Thanks again!  Much appreciated.