Bin-prot: implicit changes in binary protocol

[Stanislav Artemkin]

Hi all,

Bin-prot looks great for data (de)serialization, especially family of auto-generated writers and readers.

The only issue is how to not miss a change in binary protocol? If I extend a data type used in binary protocol, all writers/readers will be implicitly regenerated and I won't get a compile error. But, a program will become broken as I won't be able to read old serialized data anymore.

How do you handle this?

Thanks

[Yaron Minsky]

The idea is that Bin-prot is stable as long as you use the same types.
So, to have stabie types over time when the underlying codebase
changes, you need some discipline around that.

Look at the data and time hanlding code for examples. We have
explicit versions, and unit tests to make sure that changes don't
break serialization and deserialization for either s-expression or
bin-io. You'll see very few cases of "V2" types, since we rarely
change the physical representation of a value in Core. But we want
the freedom to do so, so you should use Stable types wherever possible
for stable protocol or storage of OCaml values.

y

> --
> You received this message because you are subscribed to the Google Groups
> "ocaml-core" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ocaml-core+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Stanislav Artemkin]

It seems I've got it! In serialization code we just need to use Foo.Stable.V1.t explicitly, so it won't be compiled once Foo.t is switched to next version. At the same time, unit tests guarantee that changes in nested types do not affect Stable type serialization. Good approach.

type nonrec t = t = … is a nice trick.

Thanks

[Yaron Minsky]

Yup, that's the basic approach.  At some point, I want to get support for checking this with more than just unit tests using typerep, but that's not quite ready yet.