[protobuf] Can I add a new item into an existing list of enumerations safely?

[john mani]

Hi

This is my first attempt at defining a command pattern :
message Command {
required Type type = 1;
optional Value value = 2;
optional Push push = 3;
optional Pop pop = 4;
}

enum Type { POP = 0, PUSH = 1 }

I deploy code using this file.

Later, I want to add a new command, but not break deployed clients.
Can I do so safely:

enum Type { POP = 0, PUSH = 1, TOP = 2 }
message Command {
required Type type = 1;
optional Value value = 2;
optional Push push = 3;
optional Pop pop = 4;
optional Top top = 5;
}

If not, what's the suggested way for achieving something like this?
I read up on 'extend' - would that be a better way? My concern is that
the other language ports (The C one, for example) do not yet support
extend.

thanx
-john

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To post to this group, send email to prot...@googlegroups.com.
To unsubscribe from this group, send email to protobuf+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.

[Jason Hsueh]

Clients with the old binary will not be able to read messages containing the new enum value. This would get treated as an unknown field, meaning that the type field would not be sent in an older client. Since your field is also required, it would fail to parse. So, with this example, you would need to make sure that all clients are updated before sending messages containing the enum value.

The solution would be to use optional fields, and define an UNKNOWN enum value.

e.g.:

enum Type { POP = 0, PUSH = 1, UNKNOWN = 2 }  // set unknown = 2 for backward compatibility, assuming POP and PUSH are already defined and in the wild.

message Command {

  optional Type type = 1 [default = UNKNOWN];

  ...

}

When you add TOP, you can start sending messages with that value before all clients are updated. The message will now parse successfully on old clients, who will get UNKNOWN when they access type().

[john mani]

Thanks.

To make this generic and allow for multiple commands to be added in
future, would a good strategy be
to define:
enum Type { POP = 0, PUSH = 1, UNKNOWN = 100 }, and later add TOP,
TAIL, etc. etc. as items after PUSH (meaning with values 2, 3 ... ).

So, later we have: enum Type { POP = 0, PUSH = 1, TOP = 2, TAIL = 3,
UNKNOWN = 100 } and so on ...

Or it really does not matter?

Is this the recommended approach for this pattern? Or should I be
using extends?

-john

> > protobuf+u...@googlegroups.com<protobuf%2Bunsu...@googlegroups.com>
> > .

> > For more options, visit this group at
> >http://groups.google.com/group/protobuf?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
> To post to this group, send email to prot...@googlegroups.com.
> To unsubscribe from this group, send email to protobuf+u...@googlegroups.com.

> For more options, visit this group athttp://groups.google.com/group/protobuf?hl=en.

[Jason Hsueh]

It doesn't matter that much. I only set UNKNOWN = 2 on the assumption that pop and push were already assigned and you needed to preserve wire compatibility. If I were starting from scratch I would probably define UNKNOWN as the first value and set it to 0. Making it the first value defined makes it the implicit default value - even if its integer value is non-zero you can still do this. Mostly it depends on how you want your proto file to look.

Whether to use this approach or extensions is up to you. This approach works nicely with switch/case. You could do the same with extensions, but it's a little bit more verbose. Extensions are primarily useful so that you don't have to have a centralized definition - other users of this proto can define their own commands as extensions. Plus there's the lack of support in other implementations that you've mentioned.

[Jason Hsueh]

Oh, the other thing about extensions: it may result in smaller message sizes. For each optional message (command) you add, you're adding a pointer to the message. In-memory size is O(number of fields defined) whereas with extensions it's O(number of fields set). There's some overhead to using extensions; I think extensions become smaller when you have 8 fields or so.