What are well-known types?

[Zellyn]

There are increasing numbers of references to "well-known" types in protos. For instance, I see changes in the Go implementation to support them.

There were passing references in release notes in this group.

However, the main protobuf site includes no narrative explanation that I can find.

The idea of a few well-known types to represent "Boxed" values since proto3 removes the ability to null out fields makes sense, but the only documentation I could find, in the reference section of the protobuf site at https://developers.google.com/protocol-buffers/docs/reference/google.protobuf, includes all sorts of things like Struct, Method, Mixin, etc. that are entirely unclear.

Is there a conversation happening somewhere that I'm missing, or is it Google-internal but not documented outside yet?

Thanks,

Zellyn

ps - the reintroduction of message types for primitives rather undermines my belief in the arguments for removing optional fields in proto3 in the first place. I'd like to give the benefit of the doubt to the folks designing proto3: is the thinking articulated clearly somewhere?

[Feng Xiao]

On Tue, Feb 23, 2016 at 12:55 PM, Zellyn <zel...@gmail.com> wrote:

There are increasing numbers of references to "well-known" types in protos. For instance, I see changes in the Go implementation to support them.

There were passing references in release notes in this group.

However, the main protobuf site includes no narrative explanation that I can find.

The idea of a few well-known types to represent "Boxed" values since proto3 removes the ability to null out fields makes sense, but the only documentation I could find, in the reference section of the protobuf site at https://developers.google.com/protocol-buffers/docs/reference/google.protobuf, includes all sorts of things like Struct, Method, Mixin, etc. that are entirely unclear.

Is there a conversation happening somewhere that I'm missing, or is it Google-internal but not documented outside yet?

Sorry, the documentation for proto3 is not complete yet. We are still working on improving it. There are a couple of internal design docs for well-known types, but they are not yet ready to be included in the public developer guide.

 

Thanks,

Zellyn

ps - the reintroduction of message types for primitives rather undermines my belief in the arguments for removing optional fields in proto3 in the first place. I'd like to give the benefit of the doubt to the folks designing proto3: is the thinking articulated clearly somewhere?

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.
To post to this group, send email to prot...@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[Josh Humphries]

What do you think about creating issues for them on the protobuf github project and then include links to the design docs?

Also, the addition of boxed types seems to be back-pedaling a bit on the decision to make zero and absent indistinguishable for primitive types. This seems to be a concession that the proto2 behavior was useful, but at the expense of on-the-wire bloat for present values.

----

Josh Humphries

Manager, Shared Systems  |  Platform Engineering

Atlanta, GA  |  678-400-4867

Square (www.squareup.com)

[Feng Xiao]

On Thu, Mar 3, 2016 at 8:16 PM, Josh Humphries <j...@squareup.com> wrote:

What do you think about creating issues for them on the protobuf github project and then include links to the design docs?

 These design docs were created for Google internal consumption only. Without some proper clean-ups, we can't publish them in the current form.

 

Also, the addition of boxed types seems to be back-pedaling a bit on the decision to make zero and absent indistinguishable for primitive types. This seems to be a concession that the proto2 behavior was useful, but at the expense of on-the-wire bloat for present values.

Yes, the proto2 behavior is useful in some scenarios, but it also posts a constraint (to support has-bits) on proto implementations and that makes the implementations in some languages more complicate and inefficient. The decision made for proto3 is a tradeoff that we hope can benefit the majority of use cases with an extra cost on a few use cases.

[Dzmitry Lazerka]

Hi guys,

Is there any documentation now on the purpose of well-known types? Maybe things have changed in 3 years. What problem they are intended to solve?

Basically, bumping this...

[Adam Cozzette]

The well-known types all have different purposes. The wrapper types allow for optional primitive fields, Any is a replacement for proto2 extensions, and Struct is a way of representing JSON in a protobuf form. I'm not sure what the plan is for the types like Api and Method defined in api.proto, though.