Versioning: New features for existing RPCs

[Brett Lawson]

Hey everyone,

I'm trying to understand what the best practices are around versioning for GRPC.  There have been a number of strategies employed in the world of RESTful APIs (like versioning headers), and protobuf's are typically argued as being cross-compatible as long as you make changes in a reasonable set of ways.  I'm however struggling to understand how it would be possible to allow cross-version compatibility when it comes to new features being added to existing RPCs whilst still dealing with older servers.

For instance, if I have an RPC method that permits searching a data-set, and I later want to add a field which allows you to search with a higher degree of data consistency.  If a newer client were to send this field, a potentially older server might ignore this and not apply the new semantics, leading to the results being silently incorrect.

I see a couple of mechanism that could solve this, namely:
a. Have servers reject fields they do not understand.  This seems like a bad choice since it precludes other kinds of non-breaking changes.
b. Use some form versioning header such that using the new fields requires you to advertise being a 'new client' and old servers would reject those requests.  This seems like unneccessary overhead.
c. Add a new major version of the service (ie: adding my_service.v2 in addition to  my_service.v1).  This seems frustrating to deal with, since you have to duplicate the entire surface area of the API simply to iterate on a specific method.

d. Add versions to individual RPCs (ie: adding MyQueryV2() in addition to MyQuery()).  I don't really see any examples of this 'in the wild' though, and seems like a messy solution.

None of these solutions seem great or particularly often used, so I'm hoping there might be some beautiful and elegant solution I'm missing :)

Thanks, Brett

[Eric Anderson]

Other options in no particular order:

1. Have a repeated extensions field of type Any. An unknown message type should cause the server to fail the RPC. This is nicer than (b) because you can't forget to change the version when expecting a new feature, and it allows servers to implement subsets of features
2. You could do something specialized for certain cases. For the search case, maybe you want to add a new query filter. That sort of thing could be handled by having a map<string, Value> filters or Struct filters. The server can fail if it doesn't understand a key. Assuming new filters is the most common change to the RPC, that could reduce the number of times you need to use one of the more annoying versioning approaches. You might not even need anything extra; for example if you already use a FieldMask for specifying which response fields you want set, then the server can fail if it doesn't know one of the requested fields. To avoid similar limitations to (a), you could use two FieldMasks, one for optional and one for required
3. The server responds with its version in the response proto, or sets some field when it supports a feature. This approach would let the RPC complete, and then have the client verify the response. This could be like a revered (b), or the server response could include extra fields that need to be set based on the request fields. For example, let's say you add a "include_column_X" in the request, then that missing from the results would tell you the server is incompatible
1. If more than one RPC is involved, the server can set a field in the first RPC that then can influence the client in later RPCs. For example, let's say the server supports deletes, then in a "get" or "create" method it could set bool delete_allowed to true in the response. This is essentially the same as (4), but can feel cleaner in some circumstances
4. Use a separate versioning service or method. Basically, instead of negotiating every RPC, you could have the client talk to the server once and get a list of versions or features. The client would verify that the server supports what it wants and then uses the other services/methods without additional checking
   
5. Old-school bitfield/enum/feature-name set. This is basically the same as (b), but instead of a single version you have multiple dimensions that the server supports and the server checks it supports all the set bits/enum/feature-name values

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/bec19f82-553c-462b-b84d-e48b0f03e47bn%40googlegroups.com.