Schema Refactoring and Unique IDs

23 views
Skip to first unread message

Matt Stern

unread,
Nov 25, 2020, 2:39:01 PM11/25/20
to Cap'n Proto
Hi all,

Suppose I have a simple schema file:

0xabbeabbeabbeabbe;

struct Foo {
  val @0 : UInt32;
};
struct Bar {
  val @1 : UInt32;
};

I would like to move Bar into a separate schema file. If I understand the docs correctly, then this will change Bar's unique ID.

I have two questions about that:
  1. Will changing Bar's unique ID cause backwards incompatibility with old messages that are serialized with the old ID?
  2. If so, what can I do to prevent this? I would like my change to have no side-effects (a pure no-op).
Thanks!

Erin Shepherd

unread,
Nov 25, 2020, 2:52:14 PM11/25/20
to 'Kenton Varda' via Cap'n Proto
1. This depends. The only places that the IDs get used "behind the scenes" are (a) those on interfaces are used in RPC calls to identify the interface and (b) when encoding schema annotations

On the other hand, someone might be explicitly reading the ID from the schema file or the constant from the generated code

2. capnpc can be asked to generate capnp format output. In that case, it'll emit a copy of the schema with comments stripped and all automatically generated IDs inserted. You can grab the ID (and syntax) from there

- Erin
--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.

Kenton Varda

unread,
Nov 25, 2020, 3:15:57 PM11/25/20
to Matt Stern, Erin Shepherd, Cap'n Proto
Erin is correct.

For #2, the command-line syntax to have the compiler echo back capnp format (with all type IDs defined explicitly) is:

    capnp compile -ocapnp foo.capnp

This of course requires that `capnp` and the generator plugin `capnpc-capnp` are in your $PATH, which they should be after installing Cap'n Proto globally. If you haven't installed it globally, you can do:

    path/to/capnp compile -opath/to/capnpc-capnp foo.capnp

In any case, the output will go to the terminal (no files are generated).

-Kenton

Matt Stern

unread,
Nov 25, 2020, 4:43:05 PM11/25/20
to Cap'n Proto
Thanks for the quick responses and clarifications!

My actual schema file is compiled in C++ and Java. As such, it has the following preamble:

using Cxx = import "/capnp/c++.capnp";
$Cxx.namespace("MyNamespace");

using Java = import "/capnp/java.capnp";
$Java.package("com.myorg");
$Java.outerClassname("MyOuterClass");

When I try to run capnp compile, I get the following:

error: Import failed: /capnp/java.capnp

I have tried some of the workarounds in this thread but had no luck:

If I comment out the Java bits and just compile in C++ (which works for me), will this have any effect on the unique IDs for the structs in my schema file?

Thanks!

Ian Denhardt

unread,
Nov 25, 2020, 5:26:52 PM11/25/20
to Cap'n Proto, Matt Stern
The ID doesn't affect the encoding itself, so the basic things will
still work.


You can avoid changing the id by specifying it explicitly, e.g.

struct Bar 0xfeefefffefeefefe {
val @0 :UInt32;
}

You can discover the current id by running:

capnp compile -ocapnp myschema.capnp

Which will output a version of the schema including the ids, as well as
some other information.

-Ian

Quoting Matt Stern (2020-11-25 14:39:01)
> Hi all,
>
> Suppose I have a simple schema file:
>
> 0xabbeabbeabbeabbe;
>
> struct Foo {
>
> val @0 : UInt32;
>
> };
>
> struct Bar {
> val @1 : UInt32;
> };
>
> I would like to move Bar into a separate schema file. If I understand
> the [1]docs correctly, then this will change Bar's unique ID.
>
> I have two questions about that:
>
> 1. Will changing Bar's unique ID cause backwards incompatibility with
> old messages that are serialized with the old ID?
> 2. If so, what can I do to prevent this? I would like my change to
> have no side-effects (a pure no-op).
>
> Thanks!
>
> --
> You received this message because you are subscribed to the Google
> Groups "Cap'n Proto" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to [2]capnproto+...@googlegroups.com.
> To view this discussion on the web visit
> [3]https://groups.google.com/d/msgid/capnproto/aa247efb-ca69-40fa-97a7-
> 415792fd0c1dn%40googlegroups.com.
>
> Verweise
>
> 1. https://capnproto.org/language.html#unique-ids
> 2. mailto:capnproto+...@googlegroups.com
> 3. https://groups.google.com/d/msgid/capnproto/aa247efb-ca69-40fa-97a7-415792fd0c1dn%40googlegroups.com?utm_medium=email&utm_source=footer

Matt Stern

unread,
Nov 25, 2020, 8:11:57 PM11/25/20
to Ian Denhardt, Cap'n Proto
Great, thanks everyone. I will get the current unique ID from the capnp tool and specify it explicitly for my struct before I do my refactoring.

Kenton Varda

unread,
Nov 27, 2020, 10:27:09 AM11/27/20
to Matt Stern, Cap'n Proto
On Wed, Nov 25, 2020 at 3:43 PM Matt Stern <mjst...@gmail.com> wrote:
When I try to run capnp compile, I get the following:

error: Import failed: /capnp/java.capnp

You will need to specify the same -I flags (import path) that you normally specify to `capnp compile` when running the Java code generator.

If I comment out the Java bits and just compile in C++ (which works for me), will this have any effect on the unique IDs for the structs in my schema file?

No, the auto-generated IDs do not in any way depend on the contents of other files. Auto-generated IDs are constructed by concatenating the parent scope ID and the type name, and then taking a hash of that.

-Kenton
Reply all
Reply to author
Forward
0 new messages