Generating field numbers automatically

[Branislav Katreniak]

When prototyping capnproto structs, I feel a bit of burden to make the compiler happy to assign valid numbers to all fields. 

What about having and option to let the compiler generate the sequence numbers automatically if the field number is not assigned to any field?

Flatbuffers IDL works this way.

[Kenton Varda]

Hi Branislav,

Thanks for the feedback.

This is a frequent request for protobuf and Cap'n Proto. Presumably the compiler would automatically number the fields in the order they appear.

My concern is that in this mode, it is very easy to accidentally introduce backwards-incompatibility by inserting a field into the middle of the struct, or by re-ordering field declarations during refactoring. I think it would be non-obvious to many developers that the ordering of field declarations mattered -- many developers assume that fields are identified on the wire by name, by sending the schema, or by some other magic. Such accidents likely wouldn't be caught in automated testing since it's unusual to test compatibility with binaries built with different versions of the protocol, and they wouldn't be caught in review since reviewers may be similarly confused about the importance of the field ordering.

Meanwhile I think that if we made field numbers optional, most developers would in fact omit them, since developers tend to take the path of least resistance.

OTOH, with field numbers being required, the path of least resistance is actually pretty robust. Most developers will infer that the numbers are probably important for something, and than renumbering fields is likely to be a bad idea, even if they don't know exactly what they are for. In my experience, it is remarkably uncommon for people to accidentally make backwards-incompatible changes in Protobuf or Cap'n Proto, which is a great thing.

-Kenton

--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.

[Kenton Varda]

Ah, yes, if you're just converting to JSON then the field numbers are irrelevant.

Conceptually, I would be OK with a way to annotate a struct which is not allowed to be serialized in capnp format. When such a struct is constructed in a message, the message would be marked such that attempts to serialize it would fail. But, it could be used for JSON or similar operations.

Practically speaking, though, I think the invasiveness of such a change may not be worth the convenience it brings.

I'm open to other opinions on this.

-Kenton

On Tue, Jul 25, 2017 at 11:45 PM, Branislav Katreniak <katr...@gmail.com> wrote:

Hi Kenton

Thank you for prompt reply.

I agree that requiring field numbers greatly helps to ensure capnproto binary format backwards compatibility and I like current convenience / backwards compatibility trade off.

On the other side, when capnproto is used as IDL for json apis, the field numbers feel like a pure noise. It would be nice to have a less verbose option for this use case. What do you think about special struct annotation?

  Brano

Kind regards

 Brano