Message schema updates

[Joseph Lisee]

Will there be a standard way to handle scheme updates in Blink?

In my application domain, robotics, it's pretty common to add new fields to a messages as development progresses.   We have found it useful to have the serialization system deal with the issue rather coding our applications to understand MessageV1, MessageV2, etc.

When serializing by hand with Boost.Serialization it's pretty trivial to handle new or removed fields. Other language agnostic serialization systems like Protobuf, Thrift, and Cap'n Proto all have a defined process for handling updates.

-Joe L.

[david.rosenborg]

This is actually the topic of an upcoming blog post, but in short Blink offers two ways of handling extensions to existing messages.

First, there is the built-in support for unsolicited message extensions. Any message can carry an extra sequence of Blink objects. This makes it possible to convey new content to and through an otherwise closed system without modification. However, this approach is probably best suited for ad-hoc updates and annotation-style content.

The other way, which deals with the more important subject of schema evolution, relies on what we refer to as a two-schema approach. One schema is the (compiletime) schema used when implementing the application/component and the other schema is the (runtime) schema for the messages actually transmitted. Knowing the two schemas, the encoders and decoders will then deal with the addition and removal of fields and potentially type conversions.

The two-schema approach is to some extent used in the jblink implementation where the compiletime schema is implicit in the structure of the objects being populated. If a new field appears in the runtime schema, a decoder will just read and throw away the result since there is no matching field in the target object. A more elaborated implementation could opt to store such extra fields in an appendix making them dynamically accessible.

/D

[rytis.d...@gmail.com]

Does your java implementation support message extensions, i. e., how do you encode/decode a message with extensions?

[david.rosenborg]

There is currently no support for accessing or writing extensions in the java implementation. The decoder will silently drop them.