can protobuf3 be used with protobuf2?

[Bo Gao]

suppose server side is updating into protobuf3, but client side still use protobuf2, can then communicate will?

[Feng Xiao]

On Tue, Apr 26, 2016 at 7:04 PM, Bo Gao <gblo...@gmail.com> wrote:

suppose server side is updating into protobuf3, but client side still use protobuf2, can then communicate will?

Yes, as long as you only use proto3 features, they are wire compatible.

 

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.
To post to this group, send email to prot...@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[Kostiantyn Shchepanovskyi]

It would be nice to have a migration guide (checklist) somewhere, like:

1. All fields should be optional.

2. Do not use custom defailt values.

3. All enums should have first element with tag = 0.

4. Do not use extension for anything except custom options.

Something else?

[Artem Kazakov]

+1

Yes, a checklist would be extremely helpful. 

[osu...@gmail.com]

Protobuf schema evolution rules:
https://developers.google.com/protocol-buffers/docs/proto#updating

[Feng Xiao]

On Wed, May 18, 2016 at 9:27 AM, Artem Kazakov <kaz...@gmail.com> wrote:

+1

Yes, a checklist would be extremely helpful. 

On Friday, April 29, 2016 at 5:04:56 PM UTC-4, Kostiantyn Shchepanovskyi wrote:

It would be nice to have a migration guide (checklist) somewhere, like:

1. All fields should be optional.

2. Do not use custom defailt values.

3. All enums should have first element with tag = 0.

4. Do not use extension for anything except custom options.

Something else?

In 3.0.0-alpha-1 release note there is a list of main proto3 changes:

The following are the main new features in language version 3:

1. Removal of field presence logic for primitive value fields, removal of required fields, and removal of default values. This makes proto3 significantly easier to implement with open struct representations, as in languages like Android Java, Objective C, or Go.
2. Removal of unknown fields.
3. Removal of extensions, which are instead replaced by a new standard type called Any.
4. Fix semantics for unknown enum values.
5. Addition of maps.
6. Addition of a small set of standard types for representation of time, dynamic data, etc.
7. A well-defined encoding in JSON as an alternative to binary proto encoding.

[Jeremy Ong]

Big fan of 4, 5, 6, and 7. Huge un-fan of 2, and 3. I am mixed on 1 because I love the removal of required fields, hate the removal of field presence. All the changes I dislike are significant losses in functionality and break compatibility with existing users of proto2 and I'd be interested to understand why "ease of implementation" is good justification for this break in compatibility and what I perceive to be a loss in functionality.

Jeremy Ong

PlexChat CTO

[Tim Kientzle]

After studying proto3 pretty carefully, I’ve come around quite a bit on these changes:

I believe adding JSON requires dropping unknown fields. You simply cannot preserve unknown fields and properly support multiple encodings.

I’m less sure about replacing extension support with Any. Extensions have some ugly problems, but I feel the current spec for Any also has some real drawbacks.

Removing field presence is a subtle issue, but I’m starting to suspect it was actually a very good change. It reduces the generated code and the workaround of using a single-element oneof is cleaner than it might sound. In essence, a single-element oneof is just a way to explicitly declare that you want to track presence for that field. And oneof is supported by proto2 now, so you can use that technique there as well.

Finally, remember that proto2 is not going away: If proto2 assumptions are deeply baked into your systems, you can keep using it. protoc will continue to support it for a very long time.

Cheers,

Tim

[Jeremy Ong]

Why does adding JSON require dropping unknown fields? So long as fields are keyed to field number, I don't see why the JSON encoding requires special treatment with respect to the binary one.

I can understand how transitioning between major versions may require breaks in compatibility. However, proto3 makes breaks in compatibility with the underlying data (proto2 encoded), which is where I find myself in disagreement. Why not preserve data compatibility so that overtime, proto2 users can migrate? Unknown field handling (or lack thereof) is honestly the one I find most egregious.

Jeremy Ong

PlexChat CTO

650.400.6453

[Tim Kientzle]

> On May 18, 2016, at 10:01 PM, Jeremy Ong <jer...@plexchat.com> wrote:
>
> Why does adding JSON require dropping unknown fields? So long as fields are keyed to field number, I don't see why the JSON encoding requires special treatment with respect to the binary one.

JSON fields aren’t keyed to field number. They’re keyed to field name.

Even apart from field naming, JSON and protobuf wire formats don’t correspond 1:1, so you can’t even correctly translate the primitive values without the schema.

> However, proto3 makes breaks in compatibility with the underlying data (proto2 encoded), which is where I find myself in disagreement.

What do you think is different? Having decoded (by hand) a fair bit of proto2 and proto3 data, they look exactly the same to me.

As I mentioned before, if preserving unknown fields is essential for you, you should stick with proto2. It’s still around and will be for a long time.

Cheers,

Tim

[Jeremy Ong]

The handling of unknown fields is precisely where the difference lies and sticking to proto2 is indeed the plan. However, because you have many clients forced to stick with proto2, you inevitably bifurcate the users, and implementations are now forced to support two standards. Proto3 was designed supposedly to simplify the implementation of the protocol, but this is moot when the old implementation is necessary anyways. Honestly, it feels like a repeat of a python2 and python3 thingy (although admittedly perhaps not as serious).

As for JSON fields not being keyed to field number, that was a choice made with the encoding and has nothing to do with JSON itself. Do those JSON blobs also lose forward compatibility when the schema changes?

Thanks for the discussion,

J