(very brief) Exploration of moving to non-LITE_RUNTIME for Chromium protobufs

[Dan Murphy]

Hey all,

I noticed the following need in PWA code (and other code, at least 'sync' and likely other places) for extra support for protobuf messages not included in the MessageLite implementation, like:

• Equality operator
• Debug value generation (to base::Value, or at least to string)
• in-memory size estimation
• (more?)

As well as, in general, the usage of proto message in chromium doesn't seem to really need performance (not like mojom does). But - there might be exceptions.

I did a brief exploration on:

• Before 'making a new thing', what would it take to change us from LITE_RUNTIME to SPEED or CODE_SIZE, which should provide these methods?
• If not possible, what other solutions can we roll for folks who need these things?

I didn't have enough time to make this too thorough or make the doc very good, but if others are interested in this, you can check out my findings at Equality, Debug Serialization, and Size Estimation in Protobufs, LITE_RUNTIME. In the end it was too complicated for me to build Chromium protobufs in a different configuration in a non-trivial amount of time, so I gave up trying to do that (but wrote down the problems).

My initial conclusion here is that for PWA we are going to roll something like this CL to generate "<ProtoMessage>Extras" compile-time types with this functionality. This at least will be a zero-cost thing for us as we can delete our current custom-rolled code. Perhaps we can also migrate the sync system to this to further possibly reduce code size.

If others want to pick this up and do the unknown-full-complexity-work of "determine the cost & benefits for migrating Chromium's protobuf library away from LITE_RUNTIME and possibly to CODE_SIZE", then here is some initial work to get you started.

Thanks @Peter Kasting for some help here.

Dan

[Adam Rice]

As an alternative, could we get the proto compiler to also generate Python bindings, and then use those Python bindings at compile time to generate C++ code for <ProtoMessage>Extras classes?

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CA%2B4qT32k_KbrEGgjN8oRz-3EUTGHDKf1sroy2JOWd5326tfqPA%40mail.gmail.com.

[Daniel Rubery]

I've suggested this on the doc, but I think there's an easier solution than parsing Python as a compile-time input. protoc has a notion of "plugin" that we could use to generate new code at compile time. Some GN bindings would need to be exposed by the //third_party/protobuf owners, but I don't think that adds more risk to rolls than needing to consistently parse generated Python. I haven't found any public examples of plugins, but there's plenty of prior art internal to Google.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev+unsubscribe@chromium.org.

[Adam Rice]

The idea is not to parse Python, but to generate C++ code using the metadata that the protoc compiler exposes to Python. ProtoMessage.DESCRIPTOR will be a google.protobuf.descriptor.Descriptor object which contains all the information that should be necessary to generate the C++ implementation of ProtoMessageExtras.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

[Daniel Rubery]

Oh sorry I misunderstood! That does sound like it might be easier than a protoc plugin.

[Steven Holte]

The Python approach is basically what this is doing: components/optimization_guide/tools/gen_on_device_proto_descriptors.py

I do think it would be good to look into more options here though, I don't think that will necessarily continue to scale, may not be making the right tradeoffs.

I think DynamicMessage is underling thing for cc_proto_descriptor_library?  https://engdoc.corp.google.com/eng/doc/devguide/proto/reference/cpp/api-docs/google.protobuf.dynamic_message.md?cl=head

I think maybe that requires linking in libprotobuf_full.so though?  go/mobile-cpp-protos has some related strategies

[Dan Murphy]

Hey all,

Following up to close some loops. This has mostly been 'done', with some follow-up additions like gtest matchers, and possibly rolling 'cheap' reflection to reduce binary size. I also plan on adding a bit more support for specifying, say, 'sensitive' proto members that shouldn't be included in serialization to support the sync use-case, as it has that functionality in proto_visitors.h and I want to replace that.

I attempted to see what it would look like for using protobuf_full, and using CODE_SIZE as the optimization, and it was really intersting! The binary size differences are interesting, and maybe in the future someone will show that it's actually better to fully do that for Chromium. But for now, I think we can get a lot out of this proto_extras library and have much fewer hurdles doing so.

Let me know if you have other questions! 

Dan