Coming soon: Protobuf Editions

[Mike Kruskal]

TL;DR: We are planning to release Protobuf Editions to the open source project in the second half of 2023. While there is no requirement to move from proto2/proto3 syntax to Editions syntax at initial release, we encourage you to plan a move in your software project’s future timeline.

Protobuf Editions will replace the proto2 and proto3 designations that we have used for Protocol Buffers. Instead of adding syntax = "proto2" or syntax = "proto3" at the top of proto definition files, you will use an edition number, such as edition = “2024”, to specify the behaviors your implementation will conform to. Editions enable the language to evolve incrementally over time.

Instead of the hard-coded behaviors that older versions have had, editions will represent a collection of “features” with a default value (behavior) per feature, which you can override. Features are options on a syntax entry—file, message, field, enum, and so on— that specify the behavior of protoc, the backends, and protobuf runtimes. You can explicitly set the desired behavior at those different levels (file, message, field, …) when your needs don’t match the default behavior for the edition you’ve selected.

The introduction of Editions requires the one-time conversion of your .proto definition files into a new format. We will be providing a tool, called Prototiller, for this conversion. The code generated by protoc will work with your existing code without modification.

For more information about Protobuf Editions, see the full announcement at https://protobuf.dev/news/ and the overview at https://protobuf.dev/editions/overview/.

[Jeff Sawatzky]

Looking at https://protobuf.dev/news/2023-06-29/#editions-syntax-1 it seems like with the "2023" edition you are planning on making the field presence explicit by default, whereas with "proto3" the default field presence was "no presence" (if we are using the terms defined here https://github.com/protocolbuffers/protobuf/blob/main/docs/field_presence.md)

This seems like a step back to "proto2" where explicit presence was the default.

Also, I believe that in "proto3" going from non-optional (the default in proto3) to optional (by specifying the optional flag) is a breaking change and will require a new version. So if we have a "proto3" definition that doesn't use optional fields, then when editions land we will need to update the field to have the features.field_presence = IMPLICIT in order to not be a breaking change (as shown in the example).

Therefore, is it safe to say that if we are creating a new API using "proto3" that we should mark all fields as optional in order to avoid an unnecessary breaking change in the future when editions become a thing and features.field_presence = IMPLICIT becomes deprecated.

Are there any other breaking changes from "proto3" that will need special features.field_presence options, and if so, what are they and how can we design our APIs now to avoid this and breaking changes in the future?

Thanks!