Proto3 conversion growing pains: missing optional / has_foo() functionality.

[Dylan Justice]

I'm really missing the has_foo() functions on primitive elements.  Was the decision to drop it from proto3 based on a survey of code on github, or just on an anecdotal type sentiment that "nobody's using it"?

Here's my anecdote, for the record:  I've started switching from Proto2 to Proto3.  I'm converting an application with a Java server and several clients in Java and C#.  One is .NET CF 3.5, which has no support beyond proto2.  The application sends a lot of complex messages back and forth, several of which have currency values.  There is a big difference in the business logic between a message with a currency value of "not set" and "$0".

The only alternatives I'm seeing to support this protocol with proto3 are to either (a) change each currency amount into a submessage, to (b) pair each primitive with a boolean "isSet" flag, or (c) back out and stick with proto2.  I'm leaning towards (c), the others seem like more work to implement, and more annoying to maintain.

Anyhow, I understand that the optional / has_foo() functionality would add a lot of complexity to the tool for several platforms, but would really simplify the user code, and just generally strikes me as being a critical functionality for a large family of messaging protocols.

[Feng Xiao]

On Thu, Oct 15, 2015 at 10:49 AM, Dylan Justice <dylanj...@gmail.com> wrote:

I'm really missing the has_foo() functions on primitive elements.  Was the decision to drop it from proto3 based on a survey of code on github, or just on an anecdotal type sentiment that "nobody's using it"?

The decision was made based on years of working experience on Google's huge code base with billions lines of code and more than 100,000 .proto files. It's believed to be the right thing to do by some of the most senior engineers in Google. 

 

Here's my anecdote, for the record:  I've started switching from Proto2 to Proto3.  I'm converting an application with a Java server and several clients in Java and C#.  One is .NET CF 3.5, which has no support beyond proto2.  The application sends a lot of complex messages back and forth, several of which have currency values.  There is a big difference in the business logic between a message with a currency value of "not set" and "$0".

The only alternatives I'm seeing to support this protocol with proto3 are to either (a) change each currency amount into a submessage, to (b) pair each primitive with a boolean "isSet" flag, or (c) back out and stick with proto2.  I'm leaning towards (c), the others seem like more work to implement, and more annoying to maintain.

For (a), you can use the wrapper types we added in proto3:

https://github.com/google/protobuf/blob/master/src/google/protobuf/wrappers.proto#L50

That's the recommended way to implement has function in proto3. In some languages (C# is one example), these wrapper types will be translated to built-in nullable types. See (for C#):

https://developers.google.com/protocol-buffers/docs/reference/csharp-generated#wrapper_types

 

Anyhow, I understand that the optional / has_foo() functionality would add a lot of complexity to the tool for several platforms, but would really simplify the user code, and just generally strikes me as being a critical functionality for a large family of messaging protocols.

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.
To post to this group, send email to prot...@googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[Robert Edmonds]

Dylan Justice wrote:
> I'm really missing the has_foo() functions on primitive elements. Was the
> decision to drop it from proto3 based on a survey of code on github, or
> just on an anecdotal type sentiment that "nobody's using it"?

Hi, Dylan:

As far as I can tell, protobuf design decisions are made internally at
Google, Inc., with little to no consultation with users on the public
'protobuf' group on Google Groups, or the 'google/protobuf' project on
GitHub.

This extends even to code development -- if you look at the Git commit
logs, you'll see commits like "Down-integrate from internal code base"
or "down-integrate internal changes" from Google employees, sometimes
adding thousands of lines of code with no actual commit messages
describing the changes. "Down-integrate" appears to be jargon for
taking commit history from an internal Google source repository and
squashing it all into a single commit, and then applying that commit to
the public source repository on GitHub.

The lack of so-called "hazzers" in protobuf3 has come up repeatedly on
the protobuf Google Groups forum but it would appear that this design
decision is set in stone and the Google, Inc. folks have no desire to
revisit it:

"Yes, proto3 design is finalized."

(https://groups.google.com/d/msg/protobuf/ZRpcfmeGK6s/Ur6iSkPz1K8J)

Apparently you're supposed to rev your schema and change your
application logic in order to migrate from proto2 to proto3, if you were
using a feature in proto2 that was dropped in proto3. However, Google
doesn't actually recommend that proto2 users migrate to proto3:

We recommend that new Protocol Buffers users use proto3. However, we
do not generally recommend that existing users migrate from proto2
from proto3 due to API incompatibility, and we will continue to
support proto2 for a long time.

(From the protobuf 3 pre-release release notes.)

It's unclear what "long time" means (months? years?) or whether there
will be any notice before proto2 support is dropped from the official
open source Google 'protobuf' code base.

(I'm one of the maintainers of the independent 'protobuf-c' project,
which currently only supports the proto2 language. We've had a few
inquiries about proto3 support, but no one has actually volunteered to
write any code for that yet, and I suspect if we ever do implement
proto3 support we'll probably fork a separate 'protobuf3-c' project with
separate SONAMEs, command-line utilities, etc. and leave the folks happy
with proto2 in peace.)

--
Robert Edmonds