Protobuf is a Godzilla comparing to fluffy beautiful JSON

[Petras Vestartas]

Hi,

I would like to ask why people use Protobuf at all. Is it worth it for geometry kernel serialisation?

The reason for asking is this:

- I have used Protobuf for 2 years

- 3 languages: C++, Rust, Python

- Linking is is montstrous, and building from source is slow

- On dependes on various version packages on each language.

- Usage is far from easy compared to JSON

How much do we save in memory and execution time for serialising a session of geometry? Is it really worth the hassle for geometry kernels?

[Em Rauch]

I don't think we can comment about geometry kernels specifically, but I can comment in broad strokes.

Protobuf has the downsides you mentioned (complexity, build-time steps, etc),and in exchange you get higher performance serde, highly-matching-semantics in bindings consistent interpretation of data (JSON for all its simplicity is mostly just a grammar and not semantic spec, it has plenty of edge cases that different standard library parsers will silently interpret differently).

Using simple JSON is a perfectly sensible choice for a very large number of usecases: the context where Protobuf really shines is in larger distributed systems that evolve over long horizons. You write the schema once, you know your servers will be able to interpret it.

Comparing performance is actually not so straightforward, but more relevantly the serde costs are often not at all relevant cost in overall systems... until the day that it is. If one microservice starts getting too costly you know you can rewrite it to C++ in-place with confidence. And with predictable semantics when people are adding and removing fields without doing atomic updates to the readers and writers of the data.

Use in file formats is something of a nonprimary usecase for Protobuf: in that context it makes sense if you want a convenient binary format under super long term parsability, but a hand-targeted bespoke binary format will always be able to be more efficient, and so if maximum efficiency is truly of utmost importance probably then likely nether Protobuf or JSON are the best fits for file encoding.

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/protobuf/71215b35-7b94-431d-97ce-658dd34b6e92n%40googlegroups.com.

[David Raleigh]

protobuf is about message size. If you're pumping a lot of data across a network, maybe it's worth it. there is a serialization cost.

if you are looking for a smaller message size, then maybe flatbuf is better for you for geometries (for example https://flatgeobuf.org/)? for floats you don't get a compression benefit that you might see in integer values in protobuf. floats in protobuf are definitely better than floats in json. serialization in flatbuf is better than protobuf.

Another benefit of protobuf over json is message definition and evolution, but openapi probably provides some tools for doing the same with json now if you prefer json.