Why not use XDR?

[ala...@gmail.com]

This thing seems like a reimplementation of XDR. The variable length
integers strike me as the only substantive difference... why didn't
you folks just extend the existing Python/C XDR implementations and/or
write your own? There's BSD-licensed source code for XDR in, eg,
NetBSD!

[Alkis Evlogimenos ('Αλκης Ευλογημένος)]

What happens with XDR when you change a struct by adding a field in it? How do two clients work with each other if one is still using the previous XDR definition (without the added field)?

--

Alkis

[Jason...@gmail.com]

And Alkis Evlogimenos ('Αλκης Ευλογημένος) writes:
> What happens with XDR when you change a struct by adding a field in it? How
> do two clients work with each other if one is still using the previous XDR
> definition (without the added field)?

XDR is just an encoding. It can express key/value pairs just as
easily. The 7N1-style integer encoding
isn't part of XDR, but enough people have written it themselves that
adding it to a new XDR version would
have been reasonable. Basing on XDR would have rendered the protobuf
code as a thin wrapper over
platform libraries that often are quite well optimized and made it
much simpler to port to other languages.
Oh well.

Jason

[Alkis Evlogimenos ('Αλκης Ευλογημένος)]

I am not talking about key/value pairs.

I am talking about this in XDR:

Machine A talks:

struct Foo {
  int a;
}

And Machine B talks:

struct Foo {
  int a;
  int b;
}

If they try to exchange messages between each other, what happens?

--

Alkis

[Jason...@gmail.com]

And Alkis Evlogimenos ('Αλκης Ευλογημένος) writes:
> I am not talking about key/value pairs.

> I am talking about this in XDR: [two different structs]

The XDR routines in the receiver will signal an error that the stored
struct does not match the input struct. I assume that the Protocol
Buffers do something similar of the fields are not considered
optional, otherwise all hell would break loose.

That's why it's just an encoding. Both sides must always agree on the
basics. That's why Protocol Buffers are key/value pairs for optional
data; the rest becomes framing.

Jason

[Alkis Evlogimenos ('Αλκης Ευλογημένος)]

In protocol buffers if the fields are required you will get an error if you use ParseFrom*, but you can use ParsePartialFrom* which allows you to read a message partially even in the absence of required fields. In the case of optional fields the message will parse just fine.

Protobuf's are just an encoding too. But they are a different one than XDR. If you think that protobufs will be more useful if they are built on top of XDR, you should send some patches or create protobuf-xdr if the implementation is substantial enough. If they prove to be more useful and just as fast, google might even switch to those someday.

--

Alkis

[austirg]

You might check out a project out of Cambridge UK called Yield. Parts
of its library can auto generate parsing & serialization classes. It
uses a subset of IDL to describe messages and it supports multiple
protocols, XDR is among them.

http://code.google.com/p/yield/

The only draw back is the code generator isn't a separate project...
so there may be some work required before being usable in other
projects.