cross platform serialization is impossible to get right

[Raoul Duke]

but i still utterly despise all versions of the protobuf ecosystem. 

https://mip-cloud.gitlab.io/post/2020/11/protobuf-default-value/

[David Barbour]

Cross-platform serialization of plain-old-data doesn't seem too difficult. Naturally, serialization of shared references, forming a distributed graph, is a big challenge to make robust or secure. Similarly, serialization of *functions* - e.g. to represent lazy data (such as infinite streams) - would present significant challenges to make cross-platform or efficient. But if we limit to tree-structured data or complete subgraphs, it is nowhere near "impossible to get right".

There is a common ecosystem option to use schema to build PL-specific types and automatically generate the serializer and parser functions. This isn't specific to protocol buffers; it's also common for DDS, CORBA, and other systems. Some people like this, some people hate it. I tend to prefer writing my own serialization and parser functions, using JSON or MsgPack.

[Raoul Duke]

iiuc the things that kill are cross platform data semantics and long term forward/backward compatibility. 

[David Barbour]

Even within a single platform or product, compatibility is a concern. If you used MsgPack or JSON or an app-specific representation instead of Protocol Buffers, perhaps schema updates would be more ad-hoc, e.g. change from a single item to a list of items, or from a list to a choice of a list and a reference.

But long-term compatibility is mostly an issue for long-term storage of data. For ephemeral communications, I think it's much less of a concern.

On Wed, Jan 19, 2022 at 10:44 AM Raoul Duke <rao...@gmail.com> wrote: