https://github.com/gogo/protobuf/issues/691It looks like the life of gogo/protobuf is endangered and we heavily depend on it, without knowing if the golang/protobuf alternative is good enough. Last time I micro-benchmarked the two, there was a massive gap in favor of gogo/protobuf.We need to be careful and run real-life benchmarks with golang/protobuf to see if this would cause performance degradation sooner rather than later. +Clayton Coleman do you know other reasons than performance why golang/protobuf wouldn't work? I remember that we have the encapsulation in "Unknown" that uses manually written serialization code, not sure if that'd work with golang/protobuf.
On May 20, 2020, at 8:51 PM, 'Joe Tsai' via K8s API Machinery SIG <kubernetes-sig...@googlegroups.com> wrote:
Hi Clayton,
--
You received this message because you are subscribed to the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/e485a8ad-327f-4065-bbed-c97f3b686512%40googlegroups.com.
On May 20, 2020, at 9:15 PM, 'Joe Tsai' via K8s API Machinery SIG <kubernetes-sig...@googlegroups.com> wrote:
An upcoming feature to proto3 is the ability to explicitly specify presence.
Today in proto3, if you were to specify "int32 field_foo = 1;" it would be generated as a Go struct field of type int32. For the rare cases where you need to preserve presence, you would be able to explicitly specify the "optional" keyword such that "optional int32 field_foo = 1;" would be generated as a Go struct field of type *int32 (just like proto2).
> This was pre proto3 support was finalizedProto3 support came in a some time ago (4 years ago). Has there been any substantial performance comparisons between gogo/protobuf and golang/protobuf since there? Since 2016, golang/protobuf has gained a number of performance optimizations.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/e9e0e7d3-c06d-4db1-83de-3b9d89f8abdc%40googlegroups.com.
course we’d have to deal with implications of that as a new serialization api version (proto2 to 3).
No - all optimization work in the last four years has been in gogo.
You received this message because you are subscribed to a topic in the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kubernetes-sig-api-machinery/tcwFubV9Boo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kubernetes-sig-api-m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/CAH16ShLuU2GercvJOgaqJT884Uf%3Dp1PpWSXyBii2DrjrLn6RWw%40mail.gmail.com.
course we’d have to deal with implications of that as a new serialization api version (proto2 to 3).Proto2 and proto3 have the same underlying wire format, so I'm not sure how this would matter. The distinction between proto2 and proto3 primarily applies to API-level semantics and have almost no impact on the wire format.
No - all optimization work in the last four years has been in gogo.Is that a "No" to my question regarding whether any substantial benchmarking has been performed? or a "No" regarding my statement that golang/protobuf has been optimized?
I'm not discounting anything gogo/protobuf may have done on their end to optimize, but there have been a number of optimization efforts to golang/protobuf. If a substantial comparison hasn't been done in recent times, then now seems like a good time to get those numbers.
> The switching cost is possibly so high that even a big perf improvement wouldn’t be worth it.Performance is certainly a consideration, but this thread was instigated by the fact that gogo/protobuf's non-maintenance has been observed for at least half a year, and has just formally announced that the original developers are not maintaining it anymore and need someone to step up to own it.Since Kubernetes heavily relies on gogo/protobuf, it seems that the project is forced to make a decision on what to do. Possible options are:
- Kubernetes does nothing and hopes that gogo/protobuf finds a good maintainer. A glance through the recent history of gogo/protobuf shows that even in past year, most changes have been submitted with little observable code review. Is this level of maintenance acceptable for a project that is a critical piece of cloud infrastructure?
- Kubernetes' maintainers volunteer to maintain gogo/protobuf, in which case they can provide their own level of support and maintenance.
- Kubernetes switches to golang/protobuf which has a high guarantee of long-term maintenance.
- Kubernetes avoids both gogo/protobuf and golang/protobuf and maintains its own serialization logic.
I don't feel strongly what Kubernetes plans to do, but if the 3rd option is chosen, I'm willing to extend support.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-machinery+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/e9e0e7d3-c06d-4db1-83de-3b9d89f8abdc%40googlegroups.com.
--
You received this message because you are subscribed to a topic in the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kubernetes-sig-api-machinery/tcwFubV9Boo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kubernetes-sig-api-machinery+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/e9e0e7d3-c06d-4db1-83de-3b9d89f8abdc%40googlegroups.com.
--
You received this message because you are subscribed to a topic in the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kubernetes-sig-api-machinery/tcwFubV9Boo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kubernetes-sig-api-m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/CAH16ShLuU2GercvJOgaqJT884Uf%3Dp1PpWSXyBii2DrjrLn6RWw%40mail.gmail.com.
--
You received this message because you are subscribed to the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/c490e285-0849-48ed-ac7e-504be056e376%40googlegroups.com.
I vaguely recall +Antoine Pelisse and/or +Jenny Buckley investigating this for server side apply late last year and finding that gogo proto was still better for k8s, but I could be misremembering.> users expect that protobuf messages provide protobuf reflectionHow is this possible, if you do not have the proto descriptor?We are extremely atypical proto users. Kubernetes users (programmatically) interact with our static types, not proto-generated wrappers. So, this is not meaningful for us in general. Proto is a means to an end for us, not a feature. Proto is actually rather awkward for us as a wire format because it's not self-describing. When humans interact with our data, they use yaml or json, not textpb or anything like that.(There's a chance we could find a very different use for this since we don't yet have a binary type for custom resources.)
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-machinery+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/e9e0e7d3-c06d-4db1-83de-3b9d89f8abdc%40googlegroups.com.
--
You received this message because you are subscribed to a topic in the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kubernetes-sig-api-machinery/tcwFubV9Boo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kubernetes-sig-api-machinery+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/CAH16ShLuU2GercvJOgaqJT884Uf%3Dp1PpWSXyBii2DrjrLn6RWw%40mail.gmail.com.
--
You received this message because you are subscribed to the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-machinery+unsub...@googlegroups.com.
-- Dims
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/e9e0e7d3-c06d-4db1-83de-3b9d89f8abdc%40googlegroups.com.
--
You received this message because you are subscribed to a topic in the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kubernetes-sig-api-machinery/tcwFubV9Boo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kubernetes-sig-api-m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/CAH16ShLuU2GercvJOgaqJT884Uf%3Dp1PpWSXyBii2DrjrLn6RWw%40mail.gmail.com.
--
You received this message because you are subscribed to the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/c490e285-0849-48ed-ac7e-504be056e376%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/0eaf1804-484f-498c-844a-189298deabe6%40googlegroups.com.