Evolving the protocol to remove stale fields

[Vitali Lovich]

Hi,

What's the equivalent process to removing a protobuf field in capnproto? In protobuf you could just delete it but for capnproto it looks like not only does it have to stick around so do it's type definition?

So something like:

Struct Foo {

   deprecatedField @0 :Bar

}

Means I have to keep the definition for Bar around. Is there a process through which I can eventually get rid of Bar? Any recommendations on how to demarcate the deprecated field as "deleted"?

The best I've come up with is to just delete the contents of Bar and change its name to indicate it's no longer used but wondering if there's a safer way (I'm guessing changing the field to Void won't work)

Thanks,

Vitali

[Ian Denhardt]

If it's a pointer type (structs, lists, interfaces), you can change the
field's type to AnyPointer without affecting layout, which will let you
delete the unused type definition.

Other than that there's not really support for this; you'll see
obsoleteFoo and deprecatedFoo in other codebases and afaik that's the
best you can do.

-Ian

Quoting Vitali Lovich (2019-11-20 14:46:53)

> --
> You received this message because you are subscribed to the Google
> Groups "Cap'n Proto" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to [1]capnproto+...@googlegroups.com.
> To view this discussion on the web visit
> [2]https://groups.google.com/d/msgid/capnproto/CAF8PYMhhqv6groL50Kbn_%3
> DRPrg4OJuKrxyxpO2qTiYz7onJi4w%40mail.gmail.com.
>
> Verweise
>
> 1. mailto:capnproto+...@googlegroups.com
> 2. https://groups.google.com/d/msgid/capnproto/CAF8PYMhhqv6groL50Kbn_%3DRPrg4OJuKrxyxpO2qTiYz7onJi4w%40mail.gmail.com?utm_medium=email&utm_source=footer

[Kenton Varda]

Hi Vitali,

In general, the way we "remove" fields is by renaming them to add the prefix "obsolete". As Ian points out, you can also change any pointer type to AnyPointer, and enum types to UInt16, to allow you to delete the referenced type definition.

This does leave unused space in your messages on the wire. That space will be zero'd, so it compresses well, at least. You can also potentially reuse the space for a new field *if* you are confident that all users of the old field are gone. This of course tends to be easy to guarantee for internal protocols and impossible for public ones.

-Kenton

--

You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.

To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/CAF8PYMhhqv6groL50Kbn_%3DRPrg4OJuKrxyxpO2qTiYz7onJi4w%40mail.gmail.com.

[Vitali Lovich]

Thanks Ian and Kenton. That's very helpful.

For the unused wire space, is that still true for AnyPointer fields that aren't set?

[Kenton Varda]

Yes, in Cap'n Proto, fields take space regardless of whether or not they've been "set". This is because fields are identified by their offset from the start of the struct, not by tags, which is what makes efficient zero-copy random access possible. Unset fields are always encoded as zeros on the wire (even if they have non-zero defaults).

-Kenton