'Delta' Protocol Buffers

[Paddy W]

I am working on an application that compares two protocol buffers of
the same type for differences and generates a new 'delta' protocol
buffer with only the differences. This works fine for top level fields
but I can not see a way of dealing with repeating fields or nested
messages in a way that avoids including all of the repeating field
entries/nested message fields in the delta. Is it the case that
repeated fields are positional, in that they must all be present, as
opposed to being represented with their index? Is there any way of
approaching this issue or will the idea of 'delta' messages only work
with 'flat' buffer definitions. Any advice would be appreciated.

[Kenton Varda]

What do you mean by "represented with their index"?

I don't understand the problem.  Why do repeated fields pose a challenge for diffing?

[Jeremy Leader]

I think he's saying that he wants support for sparse repeated fields.

He wants to diff 2 messages (of the same type) producing an output protobuf
containing only the fields that differ between the 2 inputs. For a repeated
field, if the first 100 instances of the field are the same in both input
messages, and only the 101st instance of the field differs, he wants the output
message to contain a 101st instance of the repeated field, without having to
contain the preceding 100 instances.

I suppose you could define some meta-format that specified that a repeated field
in the inputs would result in a repeated field in the output containing a nested
message with an index field and a difference field.

--
Jeremy Leader
jle...@oversee.net

[Kenton Varda]

Ah, yeah.  A profobuf "diff" would have to use its own encoding.  You might define that encoding using protocol buffers, but you really can't use the original message type's own encoding to represent a diff between two messages of that type.